AI Tool Shows How Machine Learning Could Make Video Editing More Efficient

A new AI tool is attempting to harness some of the recent advancements made in machine learning's video production capacities behind a sleek user interface. "The idea here is that we're trying to get him to take advantage of all the ML ecosystem algorithms that we have in the bag from the previous parts of Runway and put them now in a very easy-to-use video interface," said Runway co-founder Cristóbal Valenzuela Barrera. "So the things that you can do here are very simple and very intuitive for people who edit video." Runway ML is one of a growing number of new startups that are attempting to turn breakthroughs in generative AI into easy-to-use tools for creatives who don't necessarily have a coding background. Advances in generative AI research made in the past few years, such as generative adversarial networks (GANs) that can can create wholly original images, have helped spawn a burgeoning machine learning-based art scene. But many of those tools remain out of reach for those without technical know-how. Runway has already racked up a client list that spans high-end production firms and VFX studios, agencies like R/GA, WPP and VMLY&R and brands like New Balance and Nike as well as independent creators, YouTube influencers and amateur enthusiasts. The Brooklyn-based company raised $8.5 million in venture capital in December 2020 in a funding round led by Amplify Partners with participation from Lux Capital and Compound Ventures. Runway allows users to easily select an object within a video and remove it or place it in another video through a tool called Greenscreen. The new video functionality will add to Runway's existing suite of tools, which range from the experimental—models that will generate an image based on a text prompt or turn doodles into photorealistic images—to more tried-and-true AI applications, like research group OpenAI's state-of-the-art text generator or programs that can cast an image in the style of, say, an anime or a classic painter's work. One of Runway's more experimental features turns text into abstract images. R/GA, which has been an early experimenter with creative AI technologies like deepfakes, has used Runway's tools for creating various prototypes and experiments, but many of the tools remain too early stage for a full-fledged campaign, according to John Tubert, svp of technology at R/GA. Tubert said he organized a workshop with Runway's Barrera to better acquaint the agency's employees with Runway's new video editing offerings. Tubert said while the pace of research in what machine learning is able to do for creative fields moves fast, it can be expensive and time-consuming for the agency to turn those into applications that are useful to its clients on its own. That's why Tubert keeps an eye on the market for tools and software that can make these capabilities more accessible. "Otherwise, the time that it will take for us to white paper, to understand and build it, to run an experiment—this is just a much, much faster way of working," he said. "Instead of having to have multiple data scientists and technologists combine all of this for ad creatives, you can have creatives go into the tool directly and try these things out." Pandemic-fueled innovation Gartner Research analyst Nicole Greene said the pandemic and the constraints it put on video production have fueled interest in AI-assisted editing techniques and content creation that could help augment existing footage or produce videos remotely through tools like deepfake-like synthetic media. "All of content creation was disrupted this year—you couldn't do photo shoots, you couldn't do video shoots—so companies tried to be more creative with how they utilize existing assets," she said. "As far as [the use of AI in] video, there's a lot of opportunity there. I think it's just starting." Greene said tools like Runway can help make high-end video editing capabilities more accessible and efficient to smaller creators and marketers who might not have the time or resources to use more expensive effects tools. "It's a really exciting change in the market, where these tools can really just shorten the time it takes [to] make things more productive and allow for that expansion of creativity," Greene said. AI-Powered Creativity Tools Are Now Easier Than Ever For Anyone to Use