This Startup Uses Deepfake Tech to Let Brands Create Video Sales Pitches at Scale

A new startup wants to make video production as easy as choosing a spokesmodel, a tone of voice and typing out a script.

Bangalore-based Rephrase.ai raised $1.5 million this week in a seed round led by Lightspeed Venture Partners and AV8 Ventures for a platform that uses the same technology behind deepfakes to allow brands to create personalized video sales pitches at scale. The tool uses text-to-speech artificial intelligence to narrate input text in a natural-sounding manner with a selected tone of voice, then syncs it to the lip movements and facial expressions of any of its preset human models.

The result is often passably realistic—if sometimes disconcertingly mismatched—enough that CEO Ashray Malhotra claims that the videos have doubled clickthrough rates for marketing campaigns using the videos as compared to the average personalized email. So far, the startup has worked with clients in the retail, b-to-b and software-as-a-service spaces, including Lowe’s Innovation Labs.

Malhotra says one of the main advantages to the tech is the ability to personalize video production at a large scale. In the same way that one might insert a spreadsheet tag for a name or individualized discount in a text email campaign, the platform allows brands to do so in a video script.

“It’s going to become a fundamental way of reaching out,” Malhotra said of the technology. “As compared to five years ago, people consume significantly more video content. But in terms of production and creation, it hasn’t moved that much.”

The platform allows businesses to customize every aspect of a video, including the background setting, a spokesmodel, soundtrack music and a voice for the model. This selection process can result in some unsettling combinations, but Malhotra said the businesses that have the most success take the time to perfect a formula.

The company is also cognizant of the nefarious potential that deepfake technology is more widely known for. On its website, Rephraise.ai has an ethics policy and some safeguards against such misuse, such as only allowing authorized businesses access to the platform and creating synthetic media from scratch, rather than superimposing footage, as is common in deepfakes.

Malhotra said he envisions the technology eventually being used for celebrity endorsement videos as well. The idea is that a famous spokesperson could record a template video for a brand and synthetic media could be used to personalize certain snippets, like the name of a customer or a specific promotion.

Rephrase.ai was founded in 2018 by Malhotra and two other co-founders, Shivam Mangla and Nisheeth Lahoti, and was among 10 startups chosen for the first class of the TechStars India accelerator program in 2019.

Synthetic media like the kind that Rephase.ai uses has been gradually gaining commercial traction, especially as the pandemic has made full-scale video production more difficult. Hulu, State Farm and Spotify have all used deepfake-like technology to produce advertisements in the past several months.