Meta’s Movie Gen model puts out realistic video with sound, so we can finally have infinite Moo Deng
No one really knows what generative video models are useful for just yet, but that hasn’t stopped companies like Runway, OpenAI, and Meta from pouring millions into developing them. Meta’s latest is called Movie Gen, and true to its name turns text prompts into relatively realistic video with sound… but thankfully no voice just yet. And wisely they are not giving this one a public release.
Movie Gen is actually a collection (or “cast” as they put it) of foundation models, the largest of which is the text-to-video bit. Meta claims it outperforms the likes of Runway’s Gen3, LumaLabs’ latest, and Kling1.5, though as always this type of thing is more to show that they are playing the same game than that Movie Gen wins. The technical particulars can be found in the paper Meta put out describing all the components.
Audio is generated to match the contents of the video, adding for instance engine noises that correspond with car movements, or the rush of a waterfall in the background, or a crack of thunder halfway through the video when it’s called for. It’ll even add music if that seems relevant.
