GPT-4o & Unreal Engine 5

Рет қаралды 4,838

Marvel Master

Ай бұрын

GPT4o can now see the world in Unreal Engine 5
made using some plugins... and lost some hair to get it to work :D

Пікірлер: 41

@DirkTeucher Ай бұрын

This is really cool . That high res screenshot is definitely slowing this down. If you ran a python screen grabber in the background that just grabbed an image at 2fps i bet it would double the speed of the responses. GPT 4o is supposed to be able to respond in something like 300ms which is nuts. Its virtually realtime. At this rate we will have real time AI next year.

@sunilnarine6313 Ай бұрын

This gpt takes so much time and nothing anything like falsy advertised gpt 4o presentation. The same boring ai with some improvements.

@Cusey 23 күн бұрын

@@sunilnarine6313 nothing anything like falsely advertised? Sick grammar. Do you think that because it's slow in this persons version of it running in UE5? You can't put together that maybe it's this particular implementation that's slow? It's literally describing in detail exactly what he's looking at. Calling it the "Same boring ai" is pretty fucked up

@ThiagoLionheart Ай бұрын

Incredible, very fantastic ❤

@unitednorthpole 7 күн бұрын

Good, but a tutorial or a marketplace solution (involving the end to buy the little plugin you used) would be welcome)

@wolfgangsam4652 Ай бұрын

Thats really pretty cool! Kudos!

@YahiyaJasem Ай бұрын

Great stuff!

@devon9374 2 күн бұрын

You really are the GOAT to me

@sheikhsadab Ай бұрын

This is the beginning of something cool. It still needs to be a lot faster to be actually used in shipped real time products.

@Nkrasart Ай бұрын

Awesome man! 🔥

@FUNBBTV Ай бұрын

wow, amazing

@imxtwinky Ай бұрын

That will change everything in the future.

@tbarsnessvfx Ай бұрын

in a couple months we wont need to send over high res screenshots. It will be able to take audio directly as well as video. No translation to text needed as input transform.

@aj08coder Ай бұрын

This is amazing! It would really help me if you could tell me how you get the 4o model in unreal? What plugin did you use? Thanks :) edit: Nvm i found your vid thanks!

@aj08coder Ай бұрын

@@GES1985 I found it’s using the api and he uses a json plugin to do it. He has another whole tutorial

@whome.773 Ай бұрын

cool plugin but i think it would be cooler if the camera would be added to her face, I think the problem is that there is not quite an illusion in this implementation. We have been using the photo analysis for a while now so its not super impressive.

@thronosstudios Ай бұрын

Hey, as long as this is turned into something practical where you can give unreal prompts for character creation, material making and animation, then I'm all for it.

@OnionFestival-si7tr Ай бұрын

The "scene" should be what is visible from her head camera.

@MarvelMaster Ай бұрын

i know but you can do that 😁

@MaxStudioCG2023 Ай бұрын

@@MarvelMaster so how can we make this ? implement this chat gpt ? is it works with chat gpt 3.5 ? (because i gues this 4 is payd version )

@MarvelMaster Ай бұрын

@@MaxStudioCG2023 open ai api is always paid... per tokens

@MarvelMaster Ай бұрын

used it 1 day about 50 cents

@MaxStudioCG2023 Ай бұрын

@@MarvelMaster yea well thar sucks :))

@LeonGustin Ай бұрын

Gosh. I really can't wait until npcs Gain the ability to observe its virtual surrounding and react to it. Naturally that's gonna revolutionize gaming, Please experiment more with this. Thank you for the video

@MarvelMaster Ай бұрын

in theory i can at least make walk along a path an ut explains what it sees and on certain triggerwords icould make it do stuff

@LeonGustin Ай бұрын

@MarvelMaster Is there a way to give the NPC a camera like the player sees through as we walk around in the virtual world? Cause, it seems like it's not a continuous feed that the AI uses. It always needs a screenshot. Am I correct about that?

@MarvelMaster Ай бұрын

@@LeonGustin you can place a camera on the npc head and take screenshots from there when needed

@emrehan328 Ай бұрын

JUST TAKE MY MONEY 💵💵💵💵💵

@yanhu Ай бұрын

You make the robot look around animation but still upload the screenshot to the GPT.😂 If the character camera works like a cellphone camera in GPT4o that will be really Awesome.

@rofethegeek5820 Ай бұрын

one tutorial please is awesome

@merljns Ай бұрын

Hi, how many time until AI game maker ?

@Nereal_Dvigatel Ай бұрын

Когда нибудь NPC перестанут быть тупыми и начнут анализировать окружение в реальном времени, тогда в офлайн игра выйдет на другой уровень

@nilox4037 Ай бұрын

this is not the fast version

@MarvelMaster Ай бұрын

thats at least the latest api I could get

@nilox4037 Ай бұрын

@@MarvelMaster In the video they are talking to GPT4-o and can interrupt it. This functionality isn't yet integrated into the API and when it will it will be many many time fasten than the current implementation.

@MarvelMaster Ай бұрын

@@nilox4037 for conversation and image recognition I used gpt-4o and for text to speech tts-1, I also expected faster speeds but not sure if its because UE5 uses JSON for comunication.

@artoriasdenostradamus3628 Ай бұрын

Tutorial ?

@MarvelMaster Ай бұрын

hm uses a little marketplace plugin that I needed to edit in c++ and also the api is pay per token

@artoriasdenostradamus3628 Ай бұрын

@@MarvelMaster what is the plugin name ?

@OriginRow Ай бұрын

@@MarvelMaster Plugin name please

@MarvelMaster Ай бұрын

@@OriginRow its just for uploading an image to google drive...I had to edit its c++ to get the link where it uploaded to, but also wrote a request for the creator www.unrealengine.com/marketplace/en-US/product/drive-messenger?sessionInvalidated=true