3D to AI - THIS is the REAL Power of AI

Рет қаралды 69,300

Күн бұрын

Using 3D to control AI Scenes is very powerful. Watch me create a scene in Blender and then render it into a AI scene in Krita with Stable Diffusion XL Lightning
#### Links from the Video #####
My 32 min. Patreon Video: / 101132172
3D Model used: sketchfab.com/3d-models/aband...
Krita AI install Video: • LCM for Krita - OVERPO...
#### Join and Support me ####
Buy me a Coffee: www.buymeacoffee.com/oliviotu...
Join my Facebook Group: / theairevolution
Joint my Discord Group: / discord
AI Newsletter: oliviotutorials.podia.com/new...
Support me on Patreon: / sarikas

Пікірлер: 167

@novantha1 Ай бұрын

Using something like ReShade I could see an interesting situation where a person could get depth maps from games and re-skin them for really cool screenshots, or re-imaginings of old games.

@NorisSpecter Ай бұрын

After so many years working in Blender, my heart flutters, seeing new possibilities.

@johntnguyen1976 Ай бұрын

My thoughts exactly!

@talontoth4402 Ай бұрын

Just projection map it to the mesh then bake to your UV set and clean up the texture after. If a few different projections were made in the same style you could probably do a large chunk of texturing done very quickly. Sometimes I wish I worked in 3D still with all the new stuff coming out.

@f4ust85 Ай бұрын

The "possibility" of hacks mimicking great in-depth texturing and rendering in 5 seconds via generators to compete with your job? Yay, so exciting.

@joefawcett2191 Ай бұрын

@@f4ust85 luddites can scream at traffic

@orangehatmusic225 Ай бұрын

Pretty soon you won't even need to know how to use Blender.

@aysenkocakabak7703 Ай бұрын

This is a really nice technique, i wanna use it in my next set design project! Thank you❤

@I-Dophler Ай бұрын

Having spent years mastering Blender, the sense of excitement I feel now, as I look towards the horizon filled with new opportunities, is truly unparalleled. The journey with Blender has been a transformative one, marked by continuous learning and growth. Today, as I stand on this precipice, eager to dive into what lies ahead, the prospect of exploring these new avenues fills me with an exhilarating sense of anticipation.

@OmerAbdalla Ай бұрын

That last move you made, rotating the scene in Krita, sold the workflow to me. Now I need to go back and refresh my old Blender brain muscles. Thank you Olivio!

@Av-uv6xu Ай бұрын

last move was rotating in blender

@TheSnakecarver Ай бұрын

He switched back to Blender to rotate the scene.

@OmerAbdalla Ай бұрын

@@Av-uv6xu Thank you for pointing that out. You are right we are not importing a 3d model into Krita. That means a few additional steps to get the scene you like and I am OK with that.

@DeltaZ10000 Ай бұрын

Then you should check out a addon for Krita called "Blender Layer", that syncs in realtime your blender scene into the AI Diffusion plugin in Krita.

@jeffg4686 Ай бұрын

Not sure if it would give better results, but you can get a depth map by enabling "z" pass, then using the compositor. In the compositor, you can output depth socket into normalize node, then into Math (subtract from 1) node (to invert) then to output. Might yield better results than mist but can't really say for sure since SD just using "overall" feel of it.

@fernandolener1106 Ай бұрын

I think he tried to simplify and avoid blender specifics.

@Marcus_Ramour Ай бұрын

Great workflow & thanks for sharing 👍🏻

@user-kq2es6ip2g 29 күн бұрын

very cool, thanks Olivio, always love your videos for when I need to find out what awesome new stuff is going on in AI!

@renzocheesman6844 Ай бұрын

this is freaking amazing, thanks for sharing!

@dimamitlenko1830 Ай бұрын

Thank you, Olivio! That is amazing!

@MrSheduur Ай бұрын

Nice to see we are already at that point. This can be nice for future AI based lookdev, once we achieve a decent amount of consistency for style of objects. would love to see some examples of 3d meshes turned to depth maps and how consistent they look.

@shahidmahmood7252 Ай бұрын

Always something new to learn in Blender but this feels like a quantum leap using AI. Thank you.

@user-kt2kz5qg4z 26 күн бұрын

Wonderful. Thank you. Very instructive and it seems a much better approach. More flexible.

@AironExTv Ай бұрын

Wow. Thanks for sharing this.

@moviecartoonworld4459 Ай бұрын

I really get a lot out of these always new and magical lectures. thank you!

@waurbenyeger Ай бұрын

Pro tip: If you hold down the Windows key and tap the plus key to open the magnifier tool (or click the start button and type magnifier) and then hold ctrl + alt + i you can invert the colors of your screen when making your adjustment in Blender. Just make sure to set the magnifier to 100% so that it's not zoomed in and shows your full screen.

@sabrinoor Ай бұрын

@I-Dophler Ай бұрын

In a world where the boundaries of reality stretch and bend, we journey from the tangible, three-dimensional spaces we know so well into the vast, uncharted territories of Artificial Intelligence. This narrative unfolds, revealing the genuine, unparalleled potency that AI holds, not just as a tool, but as a transformative force reshaping our world, our perceptions, and perhaps, our very essence.

@Jackripster69 Ай бұрын

This is very cool, lets hope for some simplier workflow soon tho. Very handy to just model and allow AI to take care of imagine texturing, lighting etc.

@ceegeevibes1335 13 күн бұрын

As a blender-head _ I'm very happy you are directing attention to Blender. this is truly one of the the best workflows, depth map is so powerful,

@oneroom2660 16 күн бұрын

This technique may answer a lot about our own reality, the black and white misty place is referred as the Ethereal plane, it's somewhat of a 2.5D dimension, and you have a 4D Dimension which is a collective consciousness that projects a 3D reality we all agreed on like a shared dream.

@Mranshumansinghr 12 күн бұрын

I have been learning Blender since 1 year now. Now I have jumped on to the ConfiUI and Stable Diffusion bandwagon. This is exactly what I was looking for.

@juanjesusligero391 Ай бұрын

I'm happy to see Blender + Krita + ComfyUI! Open Source software rules! ^_^

@Patheticbutharmless Ай бұрын

By the way, in Blender, in the world settings of the mist pass just set the "Falloff" to "inverse quadratic" and you can get the right b/w gradient at the render.

@sb6934 Ай бұрын

Thanks!

@alxleiva Ай бұрын

I find that in order to preserve extra details is better to use depth plus canny maps. You also need yo adjust a but what's more important depending on your image, sometimes the prompt is more important, sometimes it's controlnet

@marcihuppi Ай бұрын

quicktip for cinema 4D users: you can create a material with a b/w gradient in the luminance channel and use it as material in the render settings with material override to convert your whole scene to a depth map ♥ extra tip: you can hook the start and end coords of the gradient to the position of 2 separate null objects via xpresso to control the gradient.

@alejmc 19 күн бұрын

This is something, you could even get some textures ideas to project back into the model later on. It is at least a beyond polished starting point. I wonder how it would look with simpler boxes and shapes, if it would be possible to give you a ‘detailed’ home out of boxes that then we can use back in Blender to actually model it away.

@chickenmadness1732 15 күн бұрын

Yeah, rotate the image until you're perpendicular to the wall. Then spam a few AI gens and you have a bunch of material you can splice together in photoshop. Really huge time saver.

@Thomas_Leo Ай бұрын

The future is now! Amazing stuff. 👍

@Max-Blaze Ай бұрын

Its funny that you stopped at the difficult part which is when you change the perspective and use img2img (depth) again it still tends not to keep the same building. Unless it is already pre colored till a extend.

@liialuuna Ай бұрын

of course it's not keeping the same building. just use blender for your project instead of ai.

@Max-Blaze Ай бұрын

@@liialuuna This is a combination of both, and soon it will probably possible.

@lefourbe5596 29 күн бұрын

@@liialuuna just use blender... Just do this... Still missing the point.

@nekoanomaly 21 сағат бұрын

Love your tutorials

@nathanbanks2354 Ай бұрын

Finally installed the kirta plugin. It's *so* powerful. The AI selection mechanism is great too--it's easy to touch up only the part I want to change. I can also mask out various layers if one AI generation is better in one part than another. Guess I should experiment with depth maps and poses. My favorite part is the upscale--it takes forever, but it seems to reuse the prompt to fill in detail keeping the image mostly the same. A few minutes later on an old P5000 graphics card, and I've got a great 4k image. It's more much powerful than RealESRGAN. It's also more powerful than Dall-E 3 simply because you can modify images, though it's tempting to start with a Dall-E 3 image and tweak it in krita ai diffusion.

@Av-uv6xu Ай бұрын

is krita plugin going to work fine on gtx 1070?

@Sylfa Ай бұрын

There's also a plugin for rendering directly with AI in Blender. Not sure how it compares though as I've not tried it. Would enable a swifter workflow though.

@OlivioSarikas Ай бұрын

yes, but krita has all the power of a full image editing program giving so many more options to quickly adapt and edit. This video only shows the 3D input part. But if you check out my recent art jam live streams, you can see me using krita for painting as well

@chickenmadness1732 15 күн бұрын

This is huge for game developers and artists. It's going to save so much time.

@ilanlee3025 4 күн бұрын

Great vid, subscribed

@erdbeerbus Ай бұрын

this is indeed a great workflow! thank you! is there a way to animate things like this? maybe there is the option to save a sequence of depthIMGs and to import it //as a seq ... into KRITA ... would be great to see a way ... have an egg-siting Easter!

@GS-ef5ht Ай бұрын

Amazing!

@cho7official55 17 күн бұрын

I'm looking forward to see the other way around, from 2d images, to 3D

@homayounkarimpourr3923 5 күн бұрын

Beautiful idea, thanks a lot. How could we do the same process with an animted scene?

@DigitalForest0 Ай бұрын

Olivio, how did you know i was looking videos like this! superb! please more videos about marrying AI generated photos and 3dscenes! Thank you!

@R0TH81LLY Ай бұрын

I miss the affinity photo tutorials, i hope you get well soon

@BabylonBaller Ай бұрын

Interesting 👍👍

@LockedPuppy 21 күн бұрын

Sooner than we think, this is how games will be rendered .. and not so long after, not even 3d models will be needed anymore. just some persistence for the AI. now all we need is a holo deck

@I-Dophler Ай бұрын

The video effectively demonstrates how to blend 3D modeling with AI for creative compositions, offering clear, step-by-step instructions suitable for a range of skill levels. It leverages free tools like Blender, making it accessible, and showcases innovative applications of AI in art, encouraging experimentation and creativity.

@seduttoridaincubo1722 7 сағат бұрын

that's EXACTLY what i was looking for...

@Suthriel 28 күн бұрын

Krita and AI is really super, especially since it just requires some clicks to install :) I just wonder, is there any AI, LoRA or checkpoint, that can cut out characters to separate them from the background? Like the depth map, just with also throwing out the background.

@RonaldH 8 күн бұрын

Pretty cool. But, as it's based on comfyUI in the background, is it possible with this AI plugin inside of Krita to save the workflow? Like comfyUI embeds the workflow that created the image inside the image. Can that be done here?

@spidermarcusPOP Ай бұрын

Danke Olivio! Schön dich zu sehen

@OlivioSarikas Ай бұрын

hey markus, dich gibt's auch noch. Schreib mir mal :)

@spidermarcusPOP Ай бұрын

@@OlivioSarikas Jaaa mich gibts auch noch ^^ Aber mit dieser neuen Version von Blender komm ich garnicht klar.... o.O

@pandelik3450 Ай бұрын

I don't think you mentioned about resolution when exporting from Blender. What should be take into consideration? Should I always export on a standard resolution for SD 1.5, respectively SDXL?

@tanveerahmad2865 Ай бұрын

woww.

@spearcy Ай бұрын

I don’t get too excited about all the cool single images people can generate. What’s fun to me is the video creation potential of things like this.

@CNCAddict Ай бұрын

This is the future of video game rendering, which will be incredible.

@chrsl3 Ай бұрын

wow

@siddharthpanchal718 5 күн бұрын

Gone are the days when they made expensive movie sets.

@shongchen Ай бұрын

Great work, but you don't need controlnet?

@serhiikalenik1155 Ай бұрын

I do the similar, but use the composing and rendering with Z depth (Blender 3D). Then when I get image and depth I send result to the automatic111 img2img with controlnet depth. Btw. I'm tried to render dragons, but even when I has a 3D model with textures and depth and canny I could't get fine result from AI.

@OlivioSarikas Ай бұрын

does Z Depth also have ranges?

@serhiikalenik1155 Ай бұрын

@@OlivioSarikasSorry but I'm not sure what exactly you mean "ranges". If you mean "darker-deeper" - yes, it's has. But no so contrast image like in your video.

@garrettbates2639 Ай бұрын

@@OlivioSarikas What exactly do you mean by ranges? Are you talking about the mist start / stop that you were changing? The z-depth doesn't, because it is a rendering of the actual z-buffer from the graphics pipeline. But I suspect that you should be able to accomplish the same sort of result with some additional nodes in the compositing editor.

@tstone9151 Ай бұрын

I've been using this method ever since controlNET supported depth maps

@johnanchor2415 27 күн бұрын

Why dont you just slap the depthmap into ComfyUI and use it as controlnet depthmap? Must be much easier?

@user-kt7uz9xc5m Ай бұрын

awesome. so you can download any picture, ai recognise it and make a 3d models. in different style. and you can tell it what you like or dont like and moderate this 3d world. 😮 and even download scenes with actors, like from any movie, choose avatars and create even movies, moderate scenes like a director of a movie 😮

@OlivioSarikas Ай бұрын

no, this goes the other direction: turning 3D models into 2D images

@user-kt7uz9xc5m Ай бұрын

@@OlivioSarikas a "prnt scrn" button? 😳👍

@supersupersocco Ай бұрын

Nice! Can you render a sequence of images?

@EvokAi Ай бұрын

Can you make a new 1111 installation tutorial for newbies? I really cant get those earlier tutorials

@lefourbe5596 29 күн бұрын

Type on youtube : Royal skies 1 min install stable diffusion. But instead look not at automatic1111 but ForgeUI instead. Same interface but faster

@Hudson1615 10 күн бұрын

Definitely useful for creating consistent concept arts. But that's it. Not useful for other 3D tasks.

@OlivioSarikas 9 күн бұрын

it's "3D to Ai" not "Ai to 3D". So this isn't intended for 3D tasks, it's intended to use 3D as a versatile input to create better 2D images

@Hudson1615 9 күн бұрын

@@OlivioSarikas Yup. As I said, useful for creating things like concept art.

@jasonjohnson4133 22 күн бұрын

Someone should do Magic Eye with this depth technique 😅

@pasindumadulupura8462 Ай бұрын

time to relearn Blender I guess. Let's make some lemon juice. cheers Olivio

@legacylee Ай бұрын

also, I couldn't resist.... "yaml be there..."

@I-Dophler Ай бұрын

Greetings, my most excellent companions! I'm eager to know, how are each of you navigating the intricate pathways of our vast and unpredictable temporal journey today?

@xevenau Ай бұрын

Man I wish I haven't sold my 3d printer.

@restwiththetablet 23 күн бұрын

You can reproject this in texture and make ai-texture artisit)

@HiThere.ItsTom Ай бұрын

Can generate an image sequence?

@OlivioSarikas Ай бұрын

should be possible, you could try to use IPA or Img2Img to keep the style more consistent. Or a Lora traint on the style you want

@cmeooo Ай бұрын

whaat is the music at 14.second?

@OlivioSarikas Ай бұрын

It's simplöy called "The Heavy Metal" by fatbunny on Envato Elements

@SosyalMedyaArge-so5bs Ай бұрын

That's great, now AI can actually work.

@KDawg5000 Ай бұрын

If Krita is going to fully embrace AI, it would be nice if they incorporated a tool that works like (and as well as) Magnific for upscaling & detail.

@DeltaZ10000 Ай бұрын

With the plugin he uses in Krita you can switch from Live to Generate and to Upscale.

@KDawg5000 Ай бұрын

@@DeltaZ10000 What is your opinion of the upscale results?

@DeltaZ10000 Ай бұрын

I´m using 4x NMKD Superscale that comes with when you install AI Diffusion for Krita. I like it, and that i can weight it adding details or not.@@KDawg5000

@MysteryFinery Ай бұрын

oh, I thought its an actual 3d model

@VisibleMRJ 24 күн бұрын

Can’t you just flip the depth map in blender?

@OlivioSarikas 23 күн бұрын

yes, probably. i'm new to belder

@Amelia_PC Ай бұрын

I've actually been using 3D with Stable Diffusion + ControlNet for a few months now. While 3D works alright with rendered line art and uses the image as a guide in ControlNet for anime line art, it still has limitations. AI-generated art can struggle with clean lines (eew enlarged images) and complex perspectives, especially in building compositions. But it works fine for... that boring perspective, mainly with the whole building on the screen. (AI still is dumb af, and I expected it'd be way better than what we have today. Apparently, they will never fix the same issues.)

@liialuuna Ай бұрын

yes, it's dumb af, but you can make some complex buildings too, a lot depends on your controlnet settings and the details in your depth map.

@Amelia_PC Ай бұрын

@@liialuuna Yup, I tried depth with MLSD using line art rendered by 3ds Max as a guide, along with all the other possible combinations. But here's the kicker: I was trying to use it for comics, which demands the most complex perspectives possible. AI just can't handle that yet. So, I gave up and now only use AI to add textures to my 3D renders and drawings. It's definitely unusable for serious gigs at this point in time.

@mostafamostafa-fi7kr Ай бұрын

as 3d artist i already down it just after controlnet invented

@ZergRadio Ай бұрын

@HansEgonMattek Ай бұрын

Don't just turn the view...that's not accurate. You should better rotate the model, just press R, X, 180> R, Z, 180 or press R, hold Ctrl and drag the mouse, to rotate aligned to the grid. And don't forget to apply rotation and scale. (Ctr+A) If you get to more advanced rendering e.g. with shading, Normal maps a.s.o it will caus problems. So better do it the right way from the beginning. Up and Down is very important in the 3D universe And you don't have to hide the Overlays, it will not be shown in render.

@riufq Ай бұрын

So it's 3D to 2D Ai right?

@CCoburn3 Ай бұрын

Great video.

@nolanzor Ай бұрын

The intro do be banging

@23dsin Ай бұрын

Great. I was doing this last 12 months. I thought it was a common trick. ;D

@rakly3473 Ай бұрын

Hmmm, I wonder where you got this idea?! hehehe - Spoiler, it was me in the live stream chat mentioning untextured low poly for depth maps ;-) You can check in your live replay of the chat on the channel. Seeing all these positive reactions; You're all welcome! :P

@audiogus2651 Ай бұрын

This is the most basic use of control net for well over a year now.

@rakly3473 Ай бұрын

@@audiogus2651Not of Blender

@jonrich9675 27 күн бұрын

just ask midjourney for this feature.

@NotThatOlivia Ай бұрын

this can be done way easier in other 3D packages (I mean the whole export/render to depthmap) but not all of them are free, of course ;)

@StephenShafferengineer Ай бұрын

Why do you rock soo hard!!??? 😂

@analia390 21 күн бұрын

The problem is that when rotating it doesn't matter if you use the same seed it will give you a completely different image.

@OlivioSarikas 21 күн бұрын

you could try to use IPadapter to get a similar style than the image that you created and liked. however the details are still going to be different. that's just how AI works at this point in time

@InTheCity3D Ай бұрын

Just use the C4D plugin for SD (and use Forge) its vastly superior and will render 4k

@FulguroGeek 11 күн бұрын

to be really fair, i think 3d models to Ai image is a weird non usefull process, sine ai image are already amazing... Me what im waiting its a real powerfull Image to 3d model with texture lol...

@OlivioSarikas 11 күн бұрын

The point of this is control and in that regard AI is anything but amazing. Try to create a AI image from a specific viewing angle, using a specific focal length without guidance from any image input. You can't.

@olivere5497 Ай бұрын

Blender is 4.1??? Anyone think they went from 2.x to 4.x too quickly?

@jacosteenkamp8748 Ай бұрын

As cool as this might be and look, be warned, if you want to use this in any agency setting, make sure you know how to edit your files exactly to client spec and that you know how to make this an open file that clients creative team can amend and alter. Have fired more than enough junior artists that have incredible portfolios and then they cant do the work because they relied on AI.

@ozzyosbourne6 Ай бұрын

To be honest just using a 3d software to create a single image or small animation is not logical thing to learn. I see lots of people put their time to learn blender and try to create environments etc. However, AI can easily create short videos or a single image. My advise to people is learning technicial parts. Learn a game engine, shader creation etc. There are so many free videos regarding blender. But most of the people can not make money at all. Don't waste of your time. If we consider this video, it can be helpful for architectures or level designers who create a blockout scene of a game.

@nicholaspostlethwaite9554 Ай бұрын

Interesting, but why not just render it in Blender? Then you have actual control not some ai random generation. Better still make the model yourself. Is it that modern generations all want no work and no input, just easy lazy results? End results are not really the point! Making is the pleasure. It is like the pretence flicking paint at a canvas is art.

@ibrahimsojiv88 Ай бұрын

No control in project create random stuff not art directable then need many changes 99.9 % specific

@EvokAi Ай бұрын

The only way to stop it is not to support it

@nicholaspostlethwaite9554 Ай бұрын

@@EvokAi Not really, it has been invented and will be used. individuals will soon not even know they are consuming ai content.

@liialuuna Ай бұрын

you are right! personally i have wasted a lot of time experimenting with ai while i could have learned to get better in blender. you can make some quick images with this technique described here but it's all just random stuff and you have no control. it's great to explore some ideas though.

@travissmith5994 Ай бұрын

There's an infinite number of things a person can create, but a finite amount of time and energy they can devote to creating. There is joy in the process of creating things, but I personally struggle with finishing projects when I start getting a lot of ideas for other projects piling up. Using AI to speed up the creation process - even if it means giving up some control - is worth it for those cases. Rendering in Blender requires you to nail down every single detail of the scene (detailed enough model and textures, lighting, grass blowing in the wind, sunsets, etc), which takes a lot of time. If you don't care about the minutiae of the scene but focus more on the big picture, then the time to create the minutiae doesn't bring you joy either. If you can control the big picture using 3D, then use AI to fill in the minutiae, you can complete the work to your liking and feel satisfied with it. On some basic level the art community understands this already. It's why there are marketplaces for 3D models and textures for other artists to use. It's not "lazy" to use those provided resources, it's just picking what your project needs.

@strong8705 Ай бұрын

No, no, no! 10 minutes of precise clicking in different apps, and calling it power of AI. Just wait a few months and it should really help you. A bit selfish, but last time i checked the goal was to be served, not to serve apps.