Removing the background of a video you've shot can be a real pain if you don't have the kind of tools and setup used by professionals — and even then it isn't as easy as it should be. Kaleido's one-step background removal tool for images, remove.bg, has graduated to full-motion video with the company's new product, Unscreen.
The service itself is simple enough. You drag a video onto the Unscreen webpage, and a few minutes later (depending on the size and resolution of the content) you get it back, with everything gone but the person or object in the foreground.
The company's first product was remove.bg, which was the same sort of thing but for still images; it was a big hit on Product Hunt. As someone who has to do a bit of work in that line myself now and then, it's nice to know there's a very simple, effective web service for snipping out the background quickly and accurately, even though I know I could do the same thing in Photoshop with a bit of work.
After nine months out there, remove.bg is serving millions of users monthly, 25,000 of which are actually paying customers. Clearly there is demand for this type of service, and going from stills to video is a natural move, though of course the amount of computing power required is many times more. Kaleido tested the waters with an experimental MVP back in March, and developed Unscreen on the back of that.
The idea is, understandably, to become the go-to tool for creators who have little time to spare or don't want to deal with heavy-duty options like Premiere. On YouTube and other platforms, speed is paramount if you want to have the first unboxing video or reaction to news, but maintaining production value is important, and people will be put off by janky live background removal that makes the creator look like an amateur.
Speaking of which, Kaleido chose to compete in the offline video processing space because there are entrenched and competitive offerings available from Zoom, Microsoft and others in the video chat space, where "good enough" is just fine. But there are comparatively fewer options for offline video editing, and fewer still that anyone can operate with no expertise at all.
In Hollywood (which is to say in cinema and high-end video production in general) the world of compositing is changing, with LED walls like those used on The Mandalorian an attractive, though expensive and complex, alternative to the standard green screen or frame-by-frame rotoscoping. A simple one-step process to easily remove backgrounds for quick-turnaround shoots, dailies and other situations could be a godsend for many a VFX tech or production studio.
At all events the market is evolving but clearly exists, as paying customers attest. Kaleido is totally self-funded so far, with no need or desire to take on investors, since its income scales with its expenses and exposure.
As with most media products these days, Unscreen comes in a freemium subscription model. You can try it out with clips up to 10 seconds long for free (but the watermarked, low-resolution files aren't really suitable for publication), then there are the usual subscription tiers, from $9 to $389 per month depending on how much footage you plan on uploading. Two-thirds of its income is from small businesses, but it also counts several major enterprises and media companies among its paying customers.
Of course, none of that matters if the product itself doesn't work. I tested it out on a five-minute, 720p video of a woman with long hair, and it finished in about 45 minutes. The end result was good, with the hair nicely preserved and only a handful of small glitches that would be easy to paint out if desired. A minute and a half of myself talking into the camera in 1080p took about 33 minutes, and a somewhat clearer 23-second video of a colleague turned out very crisp in about 10 minutes.
I bet at first you thought this still was from Fashion Week too, right?
You might say, why so long? Zoom does it in real time. Yes, but at a low resolution and quality. It's not the kind of thing you'd want to put online publicly and permanently, or use in a commercial shoot. From what I could tell, the quality of Unscreen's removal was considerably better, but not straight-to-final good. You'll want to watch it first to snag any issues.
Users have the option to render the video directly with a still, video or solid color background, or a two-channel (alpha and color files) version for feeding into an editor. Other options are limited, so if you want to upscale, resize, re-render with a different color, etc., you're out of luck. This isn't an online video editing platform — it's a web-hosted video effect and should be treated as such.
One thing Kaleido has been careful to demonstrate — and it's sad to think that this is a differentiator — is that its products work with people whose skin tone and hair confound other solutions. The bare fact that some background removal processes work better with light-skinned people than dark-skinned, or with straight hair than curly, is a sad indicator of a lack of diversity in the training set that produced those tools.
Curly hair is notoriously difficult for computer vision algorithms, but Unscreen's does a decent job of it.
Kaleido's Bernhard Holzer told me that this was top of mind from the beginning, and that the team has been careful to assemble training data from all over the world to make sure the product works equally well no matter which country or hemisphere the user is in. They keep an eye out for unexpected issues; for instance, it was found that a person with a commencement cap on moving the tassel from one side to the other wasn't handled well by the system — so they added a bunch of data to fix it. Users are encouraged to give feedback and the system is constantly evolving to take advantage of it.
The company itself is growing, and expects to double in size to about 30 employees this year — and as noted remains funded by its own income. The appetite for web tools is certainly considerable, and until now the idea of a single-click background removal one didn't exactly make sense. But by being the first, Unscreen hopes to become -- and remain -- the best.
