Finally, Style Transfer For Smoke Simulations! 💨

No video

Finally, Style Transfer For Smoke Simulations! 💨

Рет қаралды 126,969

Күн бұрын

❤️ Check out Lambda here and sign up for their GPU Cloud: lambdalabs.com...
📝 The paper "Transport-Based Neural Style Transfer for Smoke Simulations" is available here:
www.byungsoo.me...
💨 My fluid control paper and its source code (and Blender implementation) is available here, pick it up if you're interested! If I remember correctly, you will have to be able to compile Blender. - users.cg.tuwie...
(If you improved this in some way, please let me know!)
Wavelet Turbulence - one of the best papers ever written:
www.tkim.graphi...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Brian Gilman, Bryan Learn, Christian Ahlin, Claudio Fernandes, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, John De Witt, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Matthias Jost,, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil.
/ twominutepapers
Splash screen/thumbnail design: Felícia Fehér - felicia.hu
Károly Zsolnai-Fehér's links:
Instagram: / twominutepapers
Twitter: / karoly_zsolnai
Web: cg.tuwien.ac.a...

Пікірлер: 200

@Raudaschl 4 жыл бұрын

You can hear the passion in Károly's voice here! Amazing research.

@TwoMinutePapers 4 жыл бұрын

😍

@Raudaschl 4 жыл бұрын

Also found this nice videos on generating real time clouds which may be of some interest kzfaq.info/get/bejne/ard_lqZ4rpzcf4k.html

@generalfishcake 4 жыл бұрын

This algorithm smokes the competition. 💨💨💨

@TwoMinutePapers 4 жыл бұрын

Excellent pun. Man, I shouldn't have missed that one.

@CGFUN829 4 жыл бұрын

London England you mean xforce will crack this and smoke it ?

@hydraslair4723 4 жыл бұрын

* uses smoke image style on smoke simulation * Hmm yes, *the smoke here is made of smoke.*

@jokinglimitreached1503 4 жыл бұрын

this joke made it's way to this corner of youtube... whhyy *screaming in pain and agony* /s

@bgtubber 4 жыл бұрын

It's actually really cool because it adds detail to a lower quality simulation which save quite a bit on simulation time.

@Jianju69 4 жыл бұрын

@@bgtubber Did you miss the part ( 4:40 ) about 10-15 minutes to render a single frame?

@bgtubber 4 жыл бұрын

@@Jianju69 I see. looks like I overlooked this part, thanks for pointing it out. Still, most likely this will take a lot less as they train and improve the algorithm with time.

@simpson6700 4 жыл бұрын

the smoke is smokier.

@Aldraz 4 жыл бұрын

Blender addon? Wow, beautiful. That's actually a lot more useful than most of these algorithms that only stays for the teoretical side with zero immidiate practical uses.

@MobyMotion 4 жыл бұрын

I hope more scholars follow your team's lead and start releasing Blender add-ons for their work 😍. That said, what do we have to do for a smoke sim, stylised as your chihuahua Lisa 😶

@TwoMinutePapers 4 жыл бұрын

We love Blender over here. 👌 If everything goes well, a chihuahua shape should be relatively easily doable with the provided code, although it needs a little programming expertise.

@Dixiklo-yl4tg 4 жыл бұрын

Agreed, it would be great if it were the norm to provide addons for industry standard products with your papers.

@MrMargaretScratcher 4 жыл бұрын

@@TwoMinutePapers I downloaded the blender_fluid_Control.zip, but I have no idea what to do with it to turn it into an addon....Any tips?

@TheBlownapart 4 жыл бұрын

Me: Wow, wouldn't it be cool to have something like that implemented in games! *15 minutes per frame*

@hey8174 4 жыл бұрын

Please someone invent a pair of virtual reality glasses that turn my weed plumes into starry night.

@jsbarretto 4 жыл бұрын

Awesome! Six Minute Papers made another video!

@mrisaacs3089 4 жыл бұрын

I also thought to think about it ^^ the videos are often longer than two minutes :D

@afailable 4 жыл бұрын

Watching these simulations makes me realise I still have so much to learn 12 years after starting my programming career. I love it

@iminni3459 4 жыл бұрын

2019: It also takes a while, the computation time is typically around 15 minutes per frame, but who cares! 2059: It also takes a while, the computation time is typically around 15 micro-seconds per frame, but who cares!

@randommm-light 4 жыл бұрын

3019: It also takes a while, the entanglement diffusion is typically up around 15 femto-seconds per Eigen-frameset, but who cares!

@nal8503 4 жыл бұрын

By that time frames will be spatial instead of flat as well.

@GuRuGeorge03 4 жыл бұрын

@@nal8503 wouldn't they be called cubes then? hehe

@ARVash 4 жыл бұрын

Bizzarely it seems like it will be 2023 instead of 2059. I'm excited but also slightly uncomfortable at the pace of things

@nal8503 4 жыл бұрын

@@GuRuGeorge03 Interesting point. What would you call whatever it is that the frames of human perception are made up of?

@infinityryvus 4 жыл бұрын

My god... It's like looking at smoke from a universe with different constants. Or maybe affected by other dimensions. Truly bizarre. Utterly amazing.

@RussDnB 4 жыл бұрын

Wow! I don’t REALLY understand everything here, but do find it fascinating.

@BalancedEarth 4 жыл бұрын

Oh wow! So that's what the people at the Google Stadia demo was talking about. They called it something else but Style Transfer will be BIG once it's perfected! I can imagine having a neuralink and adapting your favorite movie into a more stylized look and feel. With technology in the future I'm sure they'll make movies, video games, and art a whole personal experience. Having machine learning create style transfers is the first step to being able to customize and create anything in the digital field. The possibilities are endless! 🤩

@ginsan8198 4 жыл бұрын

I love your love and enthusiasm for the world of virtual reality.

@unintentionallydramatic 4 жыл бұрын

>casually throws in the fact he's got a frikken Blender addon Hot. Damn.

@arianullah6257 4 жыл бұрын

All CG artists that have been working with smoke simulations : careful he is a hero.

@AkshayKumarX 4 жыл бұрын

I was going to run off to Blender developer forums to ask for Wavelet Turbulence algorithm within Blender and then you drop the amazing surprise that there is already an addon available for it. Its just fantastic how Open Source is becoming the norm, combined with the news that even Nvidia has become a Patron level supporter on the Blender Development Fund, What a time to be alive! :)

@TwoMinutePapers 4 жыл бұрын

Indeed, it is still one of the best features in Blender in my opinion. Love it! 👌

@jimmysgameclips 4 жыл бұрын

Wow, style transfer in 3D! Very impressive

@tomasmpereira 4 жыл бұрын

were is that blender addon!!?? does it work with wavelet style transfer too?? great work

@mariohawat 4 жыл бұрын

Looking for the addon as well!

@lightning_4480 4 жыл бұрын

4:40 "15 minutes per frame, but who cares?" heuheueh what a pleasant phrase to listen

@RockSolitude 4 жыл бұрын

This is a great channel. Makes me excited for things which I barely understand, and are probably not relevant to me. But this channel makes me _want to_ understand and somehow utilize the tech.

@andyp123456 4 жыл бұрын

Although the style transfer results are amazing, your enthusiasm for this work comes across very strongly and is inspiring in itself. Nice video! :)

@dannes22 4 жыл бұрын

First time i've said outloud 'Holy shit' during one of your videos. This is some sci-fi stuff happening right now!

@Muonium1 4 жыл бұрын

Someone needs to do a channel like this for nuclear fusion research papers

@advocatusdiaboli9351 4 жыл бұрын

Not possible because there is not a library of animations to use for fusion, like there is for graphical effects. Each graphical effect paper comes with cool animations. He does not do them by himself. He just shares it.

@ryanekapanjisuhartanto7167 4 жыл бұрын

"...both the simulation and the controlling force field is computed in real time on the graphics card..." I blow my mind

@elmax5748 4 жыл бұрын

now we can make *fancy* fire

@420bongking 4 жыл бұрын

your excitement on the subject makes me excited too

@Dawoodkhan1988 4 жыл бұрын

Thats the only channel where I want to watch ads

@Billary 4 жыл бұрын

Wow I was literally wondering if anybody had tried this this morning

@mosog8829 4 жыл бұрын

Thanks very much for this paper. This is very good.

@animowany111 4 жыл бұрын

3:08 - Uh, personally, I believe that the Sato et al. 2018 paper has more realistic results. The biggest issue with this technique is that features appear and disappear from nowhere. I'd love to see a version of this approach that only applied (heavily constrained) forces on the simulated smoke instead of (and without) manipulating the densities. This is challenging, because you need to choose the forces such that there is no overall bias that makes the simulation give completely different results (But this is preferable still to smoke appearing from nowhere - and just looking at the videos, without reading the paper, it seems to stylize just a pre-baked simulation, instead of truly affecting the simulation to produce a style), and you need to be able to choose a set of forces that will create a stylized result _in the future_.

@at.3am 4 жыл бұрын

"Dear felaw schawlors did is to minute papers wit caroljonaifer here" love that phrase

@Potatinized 4 жыл бұрын

digging the flower smoke. I know what to give for anniversary now.

@Olav3D 4 жыл бұрын

Amazing.

@dissonanceparadiddle 4 жыл бұрын

Wow this is just an incredible system! You humans are gonna hit singularity very soon

@Jimmygarn 4 жыл бұрын

In about five years, give or take, I believe is the general consensus. Time moves differently these days, so who knows.

@DavidSaintloth 4 жыл бұрын

Jeesh. The last few years, despite my reading more papers than in the previous 20....I feel more and more the reality that the bleeding edge is rapidly moving away from my grasp. It is both terrifying and exciting as a lover of learning to realize that the horizon of new things to learn continues to rapidly accelerate away faster than I can discover and engage it.

@fxtdorg 4 жыл бұрын

It's like sprites but I think it will be perfect in the future, I'm thinking to implement it with Houdini, Thanks guys for the amazing research.

@verebellus 4 жыл бұрын

imagine a simplistic cartoony game that can now much more easily make the kinda of smoke they want

@CariagaXIII 4 жыл бұрын

dude A: bruh what if we combine starry night with smoke dude B : Smoke vapes. wicked.

@UsernameNULL755 4 жыл бұрын

Hardly anyone is a scholar in the comments, you flatter me by calling me so

@randommm-light 4 жыл бұрын

Well, you are gentleman..and a scholar!

@RobertF- 4 жыл бұрын

Indeed, I concur my good sir.

@StudioColossus 4 жыл бұрын

This is going to make movie effects look insane!

@sofast8149 4 жыл бұрын

"What a time to be alive"

@joshuascholar3220 4 жыл бұрын

I was once working on a coin-op video game where they wanted vampire characters who would turn into smoke - so they basically wanted this. I worked on this for a month or two after having done smoke for fires and smoke for rockets for the same game. I was totally failing to get this kind of effect when the project ran out of funders and stopped. In my defense this was a long time ago and my target machine had something like a 300 mhz Pentium II and a 3dfx card. I was working by specifying 3d arrays of vectors to move the smoke around with and watching the screen with colored glasses to see the effect in real 3d.

@activemotionpictures 4 жыл бұрын

at 2:59 - Yes I am interested to explore that in Blender. Is the .blend file available for study? Thanks.

@MsJeffreyF 4 жыл бұрын

That's very cool. I think the video (or rather the paper's videos) could use an example of where you'd use this, to really drive home WHY this is important. The volcano scene is along that lines, but I wasn't sure why it was really necessary. What problem does this solve?

@sanghyeonyu1931 4 жыл бұрын

This is a truly fundamental breakthrough, Byungsoo Kim!

@BoDodge 4 жыл бұрын

Amazing paper! Very exciting results.

@TheNewton 4 жыл бұрын

@Two minute papers 2:40 I believe this decades next killer feature in graphics will be: Use pictures of X to make better X in real or near real time. So YES more info on "fluid control" at least for me not sure about the general audience though everyones interest in style xfer(more on that below). Watching the channel it's been obvious there's a bent towards fluid simulations but never put together the background and that key phrase "fluid control" (your names kinda hard to search and fluid control is mostly physics hydraulics stuff). Being able to make realistic fluidic imagery is a slow burning hobby for me( probably would take me 50 years to make something consumable for others compared to the papers you've covered). Great visual examples of "flowing fluidic imagery" are the title sequences of: girl with a dragon tattoo's( ink heavy look), daredevil( "blood" ), marco polo(watercolor bleeding ink

@FuckOrFudge 4 жыл бұрын

Every video blows me away lol

@malfattio2894 4 жыл бұрын

This would look great in a Laika film

@Ludifant 4 жыл бұрын

Style transfer in 3D seems actually a very good idea for models too. You could take a render, style transfer it in 2D and then teach a network to create displacement maps and other textures as inputs for the principled BSDF to get the most logical (contigruous) 3D result. This would be a great help for artists and if you could bake this for modelers too. You could walk around Monet´s garden in an instant. It would be like photogrammetry on steroids.

@zbytpewny 4 жыл бұрын

Honestly not been this excited since using some of the early fractal viewers and photoshop particle & fluid dynamics simulations in the 90s. Your channel is an inspiration, thank you. Subscribed. Have been interested and inspired by A.I. research as long as can remember but was overwhelmed not knowing where to start and with other challanges like dyscalculia, ADD and more: recommend me a good current book, course and development platform for someone with a more hands on and visual learning style like mine? What areas of math were required for most of your A.I. and simulation work so far?

@josephbrandenburg4373 4 жыл бұрын

You should absolutely do a 2-minute paper on your research.

@TwoMinutePapers 4 жыл бұрын

Noted, thank you! :)

@josephbrandenburg4373 4 жыл бұрын

@@TwoMinutePapers Thank YOU. I'm excited to spend some time playing with your addon. I might also take a peek at the source code.

@mcinacio8323 4 жыл бұрын

Where is the blender add-on? Im kinda noob

@SapientPearwood 4 жыл бұрын

This is definitely really cool, but as a cfd professional I gotta say I kinda prefer the coarse versions? I think it has something to do with the fine resolution detail triggering a "spurious noise" panic response in my brain that just makes me want to go double check that my residuals converged. It's almost like the exact opposite of the numerical dissipation of highly damped commercial codes. Still, the applications for graphics and visualization are obviously super compelling

@thisisfyne 4 жыл бұрын

Amazing!! This one 4:12 is particularly pleasing :)

@Vagumcookbook 4 жыл бұрын

I hope someone develops software to let us interact with stylized physical environments in VR with smoke like this. Here’s to waiting for the PlayStation 11 VR 😂

@smugless191 4 жыл бұрын

Hold on to your vapors!

@vgfxworks 4 жыл бұрын

stunning !!

@torq21 4 жыл бұрын

I enjoy your enthusiasm.

@nal8503 4 жыл бұрын

Holy smokes

@PawelGrzelak 4 жыл бұрын

This is truly amazing. I can't wait for it to be compiled in a way so we could use it in vfx software like Houdini. Sharing this video!

@ShiroiAkumaSama 4 жыл бұрын

What if I tell you that a Houdini plugin is on their Github? github.com/byungsook/neural-flow-style

@DovahDiaz 4 жыл бұрын

Oh no, I m trying to be an FX TD, guess I will have to learn machine learning too, *cries in Houdini*

@davidmcsween 4 жыл бұрын

Hmm, smoke filling defined shape looks a lot like ink diffusion in water... where the turbulence and vector is constrained more quickly by the fluid volume (as opposed to air)

@swancrunch 4 жыл бұрын

4:12 this one is beautiful

@koko969w 4 жыл бұрын

Blender!! Woo!

@NourahHamed 4 жыл бұрын

Amazing!!! I really want to know more, Thaaank you very much

@umutcoskun4247 4 жыл бұрын

Holy smokes!

@borstenpinsel 4 жыл бұрын

Yes, finally! I waited my whole life for this. ;D

@hypersonicmonkeybrains3418 4 жыл бұрын

15 min per frame... I guess we're not going to see this in PC games any time soon then.

@somerandomdragon558 4 жыл бұрын

Why? You just need to be a very patient gamer.

@Blue_Torch 4 жыл бұрын

i love that you added the blender code i just wish i knew what to do with it, do i compile blender then put that zip as an addon, do i unzip and compile the code, very excited to try this but i have no idea where to start

@affectreflect 4 жыл бұрын

Damn...SideFx we want this in Houdini 18.5

@amyshaw893 4 жыл бұрын

Two Minute Papers: posts 5:47 long video Me: *Confused Screaming*

@DEV8795 4 жыл бұрын

THANK YOU

@azwraithlance5159 4 жыл бұрын

wow very cool video, btw do you know about the fire simulation?? is someone already created fire simulation??

@gossipopithecus 4 жыл бұрын

I only can say, that you can apply these techniques possibly for creating galaxies???

@pixel7038 4 жыл бұрын

So i can finally breath out fire 🔥

@simoncarlile5190 4 жыл бұрын

Deepfakes are just the beginning. With more advanced versions of style transfer, soon we'll have deeprealities.

@GG64du02 4 жыл бұрын

Do you think that would be any use case to predict/make stylish smoke effet? Because the holy use I can see is for shape production in particles. Maybe something for 3d printing, if you can spray a resin curable with uv light?

@WildAnimalChannel 4 жыл бұрын

This will be useful for when we really need to make some spikey smoke? Maybe for the next Avengers movie.

@matthewgenilo3158 4 жыл бұрын

In the future, art will be randomly generated! Art in the future: WEED EATER

@iuer4643 4 жыл бұрын

we are building the matrix slowly but surely ...

@marty34534 4 жыл бұрын

This is scary stuff. Photographic evidence won't mean squat in the future

@dustycarrier4413 4 жыл бұрын

"10-15 minutes per frame" Hol'up 10-15 minutes per frame means, at 60fps, this means 15 hours per second. That's really stupidly low for straight-up production worthy output.

@TwoMinutePapers 4 жыл бұрын

Indeed! 👌

@Mark73 4 жыл бұрын

Give them some time. They'll get it down to something reasonable.

@betalars 4 жыл бұрын

That's not even anywhere near to bring a long render time for modern CGI.

@electroflame6188 4 жыл бұрын

Y'all need better reading comprehension

@Vysair 4 жыл бұрын

Average time taken to render for this effect normally on SSD for 20 seconds 60fps clip is about 2-3 days depending on the complexity. Avengers took more than a year

@erikziak1249 4 жыл бұрын

Nice!

@TracksWithDax 4 жыл бұрын

This is just crazy!!

@jessebassett6684 4 жыл бұрын

i want to pick it up

@rubenssautter9242 4 жыл бұрын

Could we use in the opposite direction too? If so, we might be able to apply data assimilation by changing the initial condition at various spatial scales.

@MulleDK19 4 жыл бұрын

Me: "Cool, realistic smoke in video games." "Takes 15 minutes to render per frame." Me: "Oh. No, I can't go from 60+ fps to 0.001111 fps."

@ryan382 4 жыл бұрын

where is the blender link

@Jone952 4 жыл бұрын

How was this ann trained? If the desired result is not well defined, how do you create a loss function?

@ALVY_ 4 жыл бұрын

Wow, truly amazing

@Lugmillord 4 жыл бұрын

This is what black magic looks like.

@lordkekz4 4 жыл бұрын

Exciting 1000%

@CananaMan 4 жыл бұрын

I LOVE THIS

@fiveoneecho 4 жыл бұрын

I want that.

@Sparkyhobbies 4 жыл бұрын

OP just wants a plausible cover to hide his fascination with Death eaters and the Dark arts!!!!

@fargoththemoonsugarmaniac 4 жыл бұрын

Where can I download such programs to enhance photos or stylized them as shown in the video?

@soranuareane 4 жыл бұрын

EDIT: Disregard. I should have continued watching the goddamn video. Thank you!! old comment: Would you be willing to share source code for part of your thesis, or related projects you've worked on? I'm trying to do physics simulation with Bullet3 (everyone's favorite solid body simulation engine with soft body support) and want to add particle (or even fluid!) simulation to it. Love the videos. Keep them coming. Please.