Video Creation Platform VidMob Is Using Amazon Image Recognition Software to Fine-Tune Ad Design

Video creation platform VidMob is tapping Amazon's object recognition and sentiment analysis software to give brands a better sense of which creative elements resonate with ad viewers. The company's Agile Creative Suite tool uses Amazon Rekognition to identify the people, facial expressions, objects, language, text density and other design aspects for each frame of a given video and displays them against engagement metrics like drop-off rates. VidMob claims the platform, which is still in beta form, has already enlisted around 100 clients and indexed more than 300,000 pieces of media. The program is one of several examples of how marketers are increasingly using machine learning functions like computer vision and sentiment analysis to inform a more quantitative approach to creative decisions. "What we're giving here is a visual tool and analysis product to enable people to basically have more access to creative data than they've ever had before," said VidMob CEO and founder Alex Collmer. "For us this whole thing is about really tying together the information with the ability to react to it so that we can tighten up this creative learning loop. [We really want to] break data away from being an academic pursuit and for the first time make it be something that advertisers can react to in more ways than just manipulating their media spend." Amazon Web Services' Rekognition software offers client companies machine learning tools to do everything from identifying faces and emotions to tracking object paths across a video frame and finding and reading on-screen text. The product has faced controversy over its use by law enforcement agencies to identify persons of interest, including a pitch Amazon made to the U.S. Immigration and Customs Enforcement (ICE). But the service also counts among its customers media companies like PopSugar and C-Span, people- and image-focused search engines like Spokeo and Klear and other businesses that rely on image or facial indexing. VidMob supplements the tool with two other AWS machine learning services, Amazon Transcribe and Amazon Comprehend, which allows it to incorporate speech audio. A team of human taggers then works to label more qualitative descriptors the machine learning systems might not be able to discern--aspects like humor or aesthetic beauty. In addition to real-time feedback, VidMob's analysts also use the tool to draw top-line conclusions about what works best for specific consumer categories and social channels--for instance, when a particular actor should appear in a movie trailer, at what point a logo should pop up in an Instagram video for a beverage company or what emotion should be on display in the first few frames of a financial services ad. "We think that this is really the interesting application right now for AI and computer vision because what we saw before this was this really manual process," said VidMob CMO Stephanie Bohn. "Why this is so important is that audiences have shorter and shorter attention spans and the amount of time you have to influence them has gotten smaller. What this allows you to do is to fine-tune every second you have their attention."