VFX génératif avec Runway Gen-3 | Créer des effets visuels IA

Vidéo

Résumé

• Je découvre les possibilités incroyables des effets visuels génératifs (VFX) avec des exemples simples comme une plante et une substance visqueuse, montrant que c'est accessible même sans beaucoup de budget.
• Pour utiliser Runway et créer des VFX, je peux exporter la dernière image d'une vidéo depuis Premiere en cliquant sur "exporter image" ou en utilisant d'autres méthodes comme "vidéo à JPEG".
• L'utilisation de la dernière image comme première image pour générer de nouveaux clips permet d'obtenir des résultats satisfaisants avec des prompts simples. Par exemple, j'ai utilisé "un homme ouvre une porte révélant un portail vers l'espace".
• La durée de génération peut varier, mais des clips de 10 secondes sont efficaces pour les VFX.
• Je peux combiner plusieurs clips générés pour créer un effet continu, bien que je doive faire attention aux différences de format entre mes vidéos et celles de Runway.
• Lors de mes essais, j'ai réalisé que le son peut faire toute la différence dans la qualité finale de mes vidéos ; j'explore donc différents outils de son comme Storyblocks et 11 Labs.
• Les effets de fond peuvent être appliqués directement dans Runway, je peux choisir les zones à conserver et ajuster manuellement, ce qui fonctionne bien pour masquer les arrière-plans indésirables.
• J'apprends également à utiliser des effets de VFX génératifs que je peux superposer avec un fond vert pour obtenir des résultats professionnels.
• Les outils de vidéo ont énormément évolué récemment, rendant la création de contenus de haute qualité plus accessible que jamais.
• En explorant les techniques de création et les exemples d'autres utilisateurs, je découvre des inspirations variées qui m'aident à améliorer mes compétences en vidéo et à élargir ma créativité.

Citations de Futurepedia

"Les possibilités avec les effets visuels génératifs sont incroyables"

– Futurepedia

 

"Il y a plein de manières d'utiliser ça"

– Futurepedia

 

"La bonne lumière est le héros méconnu du monde vidéo"

– Futurepedia

 

"Ajouter des effets sonores aide vraiment à vendre la vidéo"

– Futurepedia

 

"Les outils vidéo ont fait un long chemin ces derniers mois"

– Futurepedia

Comment passer à l’action ?

Je vous suggérerais de commencer par explorer les effets visuels génératifs en utilisant Runway, même sans beaucoup d'argent. J'ai appris que je peux créer des vidéos étonnantes à partir d'une image unique. Ainsi, je peux exporter la dernière image d'une vidéo, puis l'utiliser comme première image dans Runway. Cela ne prend pas beaucoup de temps et offre de grandes possibilités créatives.

Une bonne façon de le faire est d'utiliser des prompts simples pour générer des clips. Par exemple, en demandant simplement : "un homme ouvre une porte révélant un portail vers l'espace", je peux obtenir des résultats fascinants. Je devrais commencer petit, avec des clips de 10 secondes, et ensuite les combiner pour créer une histoire animée.

Il est aussi important de porter attention à l'audio. Je peux utiliser des sites comme Storyblocks ou 11 Labs pour ajouter des effets sonores. Un bon son peut transformer mes vidéos, leur donnant une qualité professionnelle sans un budget élevé.

En plus, je peux utiliser Runway pour masquer des arrière-plans indésirables et superposer des effets avec un fond vert. Cela me permet de créer des vidéos attrayantes. Ensemble, ces stratégies m'aideront à afficher ma créativité et à produire des contenus de qualité, sans nécessiter de grands investissements en temps ou en argent.

Transcription

there’s some amazing possibilities with generative VFX this plant is a simple example same with this slime these can involve some masking but there’s a super easy way to do that right in Runway or the even easier method is expanding from an end frame like this you can even generate VFX assets there’s endless possibilities and it’s all pretty easy to get these shots that would take a lot of time and skill or a huge budget to get otherwise I’ll cover a lot more examples as we go through but let’s start simple the Slime and the doorway portal were done in the same way using the last frame from the clip I filmed as the first frame of the generation and Runway there’s a few ways to get that last frame in Premiere you just click export frame if you have a video open in QuickTime Player on a Mac when you’re on the last frame click command C then open up preview then file new from clipboard or something that can work no matter what computer you’re on is video to jpeg on easy.com once you have that open up runways gen 3 upload that frame then type what you want to happen this is the prompt I used man opens a door revealing a portal to space the camera then flies through the doorway into space with swirling Interstellar objects and galaxies fairly simple prompt then I want to make sure first is selected that will use this as the first frame for the generation in some scenarios you may want to use last where it will end on that frame and you choose either five or 10 seconds I’ll do the full 10 seconds for this one and generate it takes a couple minutes and here’s what I got this time this isn’t bad the arm got pretty weird at the beginning but this prompt did pretty well consistently only rolled it a couple times for the one I wanted earlier from there all you have to do is place the clips one after the other now in my case I film in a 16×9 aspect ratio but Runway generates a little differently at 1280x 768 that’s the pixels it’s 5×3 aspect ratio so using a video that’s already in that aspect ratio is easiest but it’s not a big deal either way I just have to resize a little bit and they’ll sync up perfectly and it’s really that easy pretty crazy how else would you have been able to get get that shot this easily it was the same process for the Slime so here’s the prompt I used a static shot of a man at a desk inside a modern studio a waterfall of multicolored slime falls from the ceiling and covers the man the Slime drenches his entire face and body I added the last sentence because in my first two generations the Slime didn’t land on me so I reiterated it now some of the prompts I’ll get to in a second took a lot more tries but these two were pretty easy this one might even look better than the one one I got earlier as a quick side note I did also try running these images with the same prompts through Luma lab’s dream machine and cling I think Runway looked better in every case but I’ll show the comparisons later I’ve got a lot more to cover with generative VFX but it’s also possible to create a video without recording or editing your own footage at all with today’s sponsor in video AI you can transform a simple prompt into a publish ready video you start with a detailed prompt this one’s about lighting tips something I need to work on then you can choose the audience the look and feel and the platform you plan to upload to and submit then after just a few minutes you have the full video good lighting is the unsung hero of the video World it can make or break your Masterpiece even if your Masterpiece is just you ranting about your favorite video game but fear not aspiring Spielberg you don’t need a Hollywood studio budget to make your videos look amazing if you don’t like that first version you can hit regenerate and get a whole new version from there to fine-tune your video further you can edit the video using natural language in the command box that includes changing the voice over swapping out scenes adding captions or updating the hook then the edit button lets you make manual edits to the media script and music editing the media lets you swap out the b-roll with footage from the Nvidia library or your own Clips the script editing option lets you make changes to your script then it will adjust the voice over and re-edit the b-roll to match then you can also swap out the song there’s some more advanced options as well like voice cloning I’ll switch this over to my voice since I’ve already cloned mine in here reflectors these bad boys bounce light back onto your subject I can also translate this into any language so with inid you go from idea to fully edited video incredibly fast you can generate four videos per week for free to remove the watermark access voice cloning and generate more upgrading to the paid plans starts at $20 a month you can try it out for free using the link down in the description and thank you to invido for sponsoring this video now moving on to the plant and monster shot for these there is an additional step involved to generate the mask so that can also be done right in Runway I’ll show that after I generate the videos it is easiest to use a separate shot of the scene without whatever it is that you’ll be masking out in this case it’s without me in it so runaway can see the full window I was originally going to do this with zombies but the zombies just weren’t looking great a couple were okay also got a few Generations where they were playing instruments like some sort of zombie folk band no idea why it was very strange so I ended up switching to the monster since it seemed like it was going to burn through too many credits trying to figure out how to prompt for the zombies this is the final prompt I used a static shot of a window a monster with glowing red eyes breaks through the glass window from the outside while the camera remains static glass shatters as the monster climbs through the window so you can see it didn’t ever follow that perfectly it might have been able to get it right with enough tries but again it is really easy to burn through credits I didn’t want to use them all up on that one shot I think I did a little over 10 Generations total between the zombies and the monster I also weirdly had issues with the plant growing I thought that would be the easiest shot of all but it kept saying it couldn’t use the prompt there’s no word I was using that should have been triggering any sensors but I would try like 5 to 10 times before I’d get one that would generate when that happens they do refund you your credits but it was happening even when I used the prompt that they used in their demo video word for word it would still fail I think it took over an hour for me to get a shot that worked and also looked good that was the only time I ran into that issue for this video but it has happened to me before on a different prompt so just FYI that’s an issue that you may run into sometimes and it can be pretty frustrating if you do anyways once I have that shot I need to put it underneath the shot of me speaking to mask out the background I can do that right in Runway they have a ton of other really useful video Tools in here this one is called remove background so I upload the clip put that on the timeline then select the area that I want to keep it will make a key frame with that selection you can select more or less if you need to then it will apply that mask over the entire clip tracking all the movement this one is a pretty easy shot for it to figure out you can scrub through to see if there’s any spots it missed if there is you can do the same process and it will add another key frame there and for problem areas you can make manual adjustments as well shots like this it does a really good job on the first try and when you click done masking you can watch the preview to double check and jump back into the mask if there’s any issues this is good so I’ll export Gen 3 is really fun but this background remover is actually the number one tool I use in Runway now in Premiere I layer that on top of the clip I generated then I need to remove the green this part may have a different name depending on what video editor you’re using but pretty much all of them will have this in Premiere it’s called Ultra key I apply that effect then use the ey dropper to select the green and it’s gone sometimes the masks are a little harsh around the edge I’ll bring up the choke a little which takes the edge in a few pixels then I’ll add some soften as as well that’ll make it blend much more naturally that’s all there is to it and that plant growing worked the exact same way now for all of these adding sound effects really helps sell it sound design can be one of the most important parts of video editing I’m definitely still learning I use a combination of story blocks and 11 Labs the story blocks is a paid site for stock image video and audio there’s plenty of free Alternatives I wouldn’t recommend paying for a site like that unless you’re doing video all the time pixabay is the free option I’ve used the most personally but there are lots of others and I also use 11 Labs where you can generate sound effects from a text prompt like for this one I could say monster growling then it will generate four options these were all really good I also used it for the plant growing on the timeline you can see I actually layered a bunch of different sound effects the other option I wanted to cover was VFX assets where you generate something on a green screen so you can use it however you want like what I did with the orb there’s tons of way to use this and most stock platforms will have this type of thing like Flames orbs explosions splashes and on and on but you can also generate them if you can’t find something you’re looking for on a different site or if you want something really specific just add into the prompt on a green screen background and this orb looks awesome and that one worked on the first try now that I have my orb I’ll bring that into Premier and remove the background just like before with the ultra key and I’ll adjust the settings a little bit for the example I have I need movement it is a short shot so I’ll just do it manually with key frames and I’ll use the transform effect since it has the option to add motion blur by adjusting the shutter angle and I’ll place it where I want and set a key frame I’ll move forward five frames and move it a little bit to track my hand that will automatically set the next key frame I’ll do that every five frames then switch to every two frames at the end when my hand was moving faster now I’ll make the movement smoother the time inter interpolation bezier and there it is that looks awesome again a lot of the time all you’ll need to do is just place this onto the scene and resize it like if I was adding an explosion or a lightning strike or something like that I did a comparison with some of the other leading video tools to make it fair I’ll show the first generation I got from each without any rerolls I used lum’s dream machine cling and pix firste cling and dream machine did all right here pixers not so much for the monster they were actually all pretty good I really like the one from pix firste now Runway is the only one that consistently followed the static shot part of the prompt the others tended to add at least a little camera movement which would ruin the effect when you overlay the subject unless you track the motion to that as well which is not worth the hassle for these shots clling was the only other one that listens to the plant growing prompt the Slime was a struggle for all of them cling tried clling and pix first both kept the static shot this time the orbs all looked good although cing was the only other one that worked for this use case since the others got cut off around the edges that’s not the generator’s fault though since I didn’t prompt for that overall Runway was better in every single shot but the others weren’t bad and with more rolls I may have been able to get there with each of them video tools have come a really long way in the past few months now now here’s some other people using this technique that should give you a lot of ideas I’ll start with the ones that Runway themselves showcased those really good at fluid simulation the physics of the water flooding out of this building is really accurate and we got a big plant growing in the back here I’ve seen quite a few using this colorful slime that’s why I tried it for mine it does a really good job at that this big yellow fabric blowing in the wind is amazing especially with that bit of transparency where you can see the background through it I like this one with this dog that I guess has magical PE and then this picture that prints out and comes alive there’s a lot of places you could go with that idea Runway posted another full thread of these I won’t go through them all but I thought this one was particularly good so it starts with just the shot of the sky and buildings then it turns into a monster walking down the street it’s got a really solid walk cycle there’s some weirdness in the buildings if you look at the details but for how much the shot transforms this is really solid here’s from christobal Valenzuela pouring a magical drink into a glass he did a really good job at making this seamless I thought this one was cool from Jared Lou adding some effects to photos he’d taken around New York City Dave Clark added a creepy clown in his house there’s a lot of ways you could use this for horror shots where it’s like darker scenery so there’s less details that Runway needs to get [Music] [Laughter] right this one Alex petrascu did using images someone uploaded unsplash is great it’s 2 and a half minutes long he goes through a ton of ideas in it I won’t play this whole thing but it has so many examples you can watch that to get more ideas of what types of shots work it also goes really well with that song that he has to it but I don’t think it’s something I can play on YouTube there’s another really good one Runway posted combining a bunch of different shots all around New York City into one piece the music adds a lot to it but it’s another that sounds like a song I wouldn’t be able to play on here they go through tons of ideas in this as well seems like Runway has some really great artists on their team if you like this video you’ll probably also like this video where I cover seven AI video tools and different ways to use them you’ll probably also like futurepedia where you can find the best AI tool for any use case see AI news and Innovations browse a curated list of AI tutorials and a lot more thank you so much for watching I’ll see you in the next one