Demonstrations of Movie Gen's capabilities included videos featuring animals swimming and surfing, as well as clips that utilized real photographs of individuals to portray them engaged in activities like painting.
According to a blog post from Meta, Movie Gen is also capable of producing background music and sound effects that are synchronized with the video content, and it can be employed to edit pre-existing videos.
In one example, the tool was used to add pom-poms to a man's hands as he jogged alone in the desert, while another instance transformed a dry parking lot into one with a splashing puddle as a man skateboarded.
The videos generated by Movie Gen can last up to 16 seconds, while the audio clips can extend to 45 seconds. Meta provided data from blind tests indicating that the model performs well in comparison to offerings from startups like Runway, OpenAI, ElevenLabs, and Kling.
This announcement arrives at a time when Hollywood is grappling with the integration of generative AI video technology, particularly following the February demonstration by Microsoft-backed OpenAI of its product Sora, which can create feature film-quality videos from text prompts.
Technologists within the entertainment sector are keen to leverage these tools to improve and accelerate the filmmaking process. However, there are concerns regarding the adoption of systems that seem to have been trained on copyrighted materials without authorization.
Lawmakers have also expressed worries about the use of AI-generated fakes, or deepfakes, in elections globally, including in countries such as the U.S., Pakistan, India, and Indonesia.
Representatives from Meta indicated that the company is unlikely to make Movie Gen available for open use by developers, unlike its Llama series of large-language models, citing a careful evaluation of risks for each model. They refrained from providing specific insights regarding Meta's evaluation of Movie Gen.
Instead, they mentioned that Meta is collaborating directly with the entertainment industry and other content creators to explore applications for Movie Gen, with plans to integrate it into Meta's products in the coming year.
As outlined in a blog post and a research paper released by Meta, the company utilized a combination of licensed and publicly accessible datasets to develop Movie Gen.
This year, OpenAI has engaged with Hollywood executives and agents to explore potential partnerships related to Sora, although no agreements have been finalized as of yet. Concerns regarding the company's practices heightened in May when actress Scarlett Johansson accused the ChatGPT developer of replicating her voice without consent for its chatbot.
In September, Lions Gate Entertainment, known for franchises like "The Hunger Games" and "Twilight," announced that it would provide AI startup Runway with access to its film and television library for training an AI model. In exchange, the studio and its filmmakers will be able to utilize the model to enhance their creative processes.