Unreal Engine 5 + ChatGPT + Elevenlabs Voice = Ultimate AI Assistant

Рет қаралды 9,911

Жыл бұрын

not sponsored AI generated
**New Tutorial available here: • Unreal Engine 5 - Ulti...
In this video, I'm thrilled to demonstrate a revolutionary integration of Unreal Engine 5 with OpenAI's ChatGPT. You'll see how voice commands interact with the gaming environment in unprecedented ways.
I've used the ChatGPT API and Elevenlabs text-to-speech technology to transform voice inputs into intelligent, human-like responses. The result? A gaming experience that's as interactive as it is immersive.
This project was made possible with multiple free plugins, each contributing to this fusion of voice recognition and AI conversation.
Don't forget to like, comment, and subscribe for more exciting showcases.
Tutorial might happen if there is excessive demand.
_______________________________________
Want to support me?
/ marvelmaster
or
www.paypal.me/marvelmasteryt
All donation money flows into better software, equipement, assest and cloud space.
_______________________________________

Пікірлер: 76

@JessicaMarinaRushie Жыл бұрын

I would love a tutorial of this!

@legato3067 Жыл бұрын

amazing as always

@AICineVerseStudios Жыл бұрын

We are already in demanding for tutorial :)

@KillerHacking1212 Жыл бұрын

You right! We all know Marvel Master don't miss!

@MarvelMaster Жыл бұрын

😁

@sebastiansuarezpuntonet2018 Жыл бұрын

you should ask chat gpt

@MarvelMaster Жыл бұрын

@@sebastiansuarezpuntonet2018 hehe i also did and it at least pointed me into the right direction

@nicoafandi4726 Жыл бұрын

wow, that's so cool!

@Vanderer11 Жыл бұрын

how cool it would be to have such ai integrated into the editor so you can always ask a technical question and get some quick answers without need to google it

@MarvelMaster Жыл бұрын

hm in theory you use editor utility to run it outside of a game

@SuperBorchert Жыл бұрын

Fantastic, Would love to have a tutorial

@quas67 Жыл бұрын

great waiting for tutorial

@tysonwilliam1722 Жыл бұрын

I'd love a tutorial, that'd be so amazing bro !

@Insectula Жыл бұрын

OMG, yes! Tutorial please!

@vasilgrancharov8025 Жыл бұрын

Bro, a tutorial would be incredible! I already asked it if you can use it in game as a unique narrative driven character. It's answer of course was yes, as long as you, I quote "you can design a set of prompts or inputs that provide context and guide the behavior of the AI assistant. These prompts can include information about the character's background, personality, goals, and any relevant narrative elements." ~GPT3.5 Just thought I can share this. In retrospect, YES, personally would love to see a tutorial on this! Cheers

@MarvelMaster Жыл бұрын

the first command is actually telling it who it is and such... otherwise it would not have known who I am. And you can probably put a whole character background story in.

@Breadbandit1 Жыл бұрын

Second a tutorial. Would be wicked mate!

@gangedup1 Жыл бұрын

Awesome

@BlueLineVibes Жыл бұрын

Looks great, just hope they can speed up the response time.

@MarvelMaster Жыл бұрын

they surely can sooner or later

@hawapxwa Жыл бұрын

Tutorial please , currently trying to figure out how to do this by myself

@RAVISINGH-hv7bl Жыл бұрын

Tutorial pleasssseeeee😍

@majorsmashbox5294 Жыл бұрын

Great work mate. Did you try OpenAI's whisper API as well? Curious how it rates for this kind of thing vs the other options

@MarvelMaster Жыл бұрын

I initially wanted to do that but i only used free plugins and blueprints... for whisper ai api I would have needed to create custom http requests to upload audio files to open ai, but i dont know how to do that. So I used a free plugin that is based on that

@zakaria20062 9 ай бұрын

How you trigger action ? Trough function calling ? Did you try Bark AI ?

@Lumhax 7 ай бұрын

Very nice system wow ! Amazing one Marvel happy 2024 year to you ! its been a while ^^

@GhanyTricks Жыл бұрын

you understand how much impact this can do for the gaming industry! specially NPCs!!!

@MarvelMaster Жыл бұрын

ofc

@nikolasiturregi3163 Жыл бұрын

I would pay for a step by step tutorial of this well done integration, congratulations, how can I contact you?

@Ready-Assets Жыл бұрын

Class

@prodev4012 Жыл бұрын

Why is there such a delay? Is it the plugins fault? Or the gpt servers being slow? Or a combination of everything? Which plugin are you using and are there any others?

@MarvelMaster Жыл бұрын

voice recognition delay send to gpt... get response from gpt... send response to elevenlabs... voice generation delay then receive audio and play

@justahumanbeing1578 Жыл бұрын

Making relationships with characters will be interesting in the future

@MarvelMaster Жыл бұрын

hm...

@manyarimusicandpoetry Жыл бұрын

Any possible way to give the AI autonomy with SuperAGI?

@MarvelMaster Жыл бұрын

no idea man

@ue4learn191 Жыл бұрын

when you get mp3 format from elevenlab how did you convert them to wave?

@MarvelMaster Жыл бұрын

runtime audio importer can import anything

@MrJodiH Жыл бұрын

Tutorial please!

@chairgraveyard4790 Жыл бұрын

How did you handle having her jump? Langchain tools? Some other token based method in the prompts and replies?

@MarvelMaster Жыл бұрын

there is surely a more elegant way but I just looked for the word jump in the reply string and called a jump function after voice was finished 😬

@chairgraveyard4790 Жыл бұрын

@@MarvelMaster I highly recommend checking out Langchain if you want to have more control over how actions get performed (the LLM decides when to do the action by itself, so specific keywords aren't required). You'd use the Agent with Tools to make that work btw

@MarvelMaster Жыл бұрын

@@chairgraveyard4790 youd probably do that when trying to integrate it as game npc.

@lawrencequick7218 Жыл бұрын

We need a tutorial!

@JoelLagunaYomama Жыл бұрын

I would love to add it to my game so that the gameplay story is never the same. AI rules.

@MarvelMaster Жыл бұрын

sooner or later there will be such a game

@benblaumentalism6245 Жыл бұрын

Anyone recognize the voice synthesis? It sounds pretty good!

@MarvelMaster Жыл бұрын

the ai introduces itself in the beginning... or just read the title :3

@benblaumentalism6245 Жыл бұрын

@@MarvelMaster I read the thumbnail. It really sounds impressive. I’m going to have to look into Elevenlabs!

@UnsupervisedUniverse Жыл бұрын

Hey I’m working on something similar. Let me know if you want to talk about some interesting capabilities that could be added when it comes to these kind of things!

@MarvelMaster Жыл бұрын

hm?

@lightningrod6925 10 ай бұрын

does it respond correctly if you ask it the current date and time. If not, it is limited in knowledge. Each time I set one up, that is the first thing I ask it. If it cannot answer this simple query, then I know it is not "live on the internet" and will have a limited database from which to respond. Ex. What is the max thrust on a Cessna Citation Longitude engine. An ai on the "net" would know, if it does not, is is a simple computer with limited response.

@MarvelMaster 10 ай бұрын

it is. cjatgpt so it answers like that I think

@s1p0 Жыл бұрын

She could do Avina from Mass Effect right away. Just before the first use, it would be enough to describe to her only one thing: Do not use the word "Marvel" when addressing players. 😸

@MarvelMaster Жыл бұрын

you can tell it anything to be 😬

@aurelianobuendia24 7 ай бұрын

Did You train the voice model?

@MarvelMaster 7 ай бұрын

@aurelianobuendia24 7 ай бұрын

@@MarvelMaster do You know if i could make My own avatar training My own model? Because i need to make one that speaks spanish but spanish voices are not that good

@medvega6447 Жыл бұрын

Tutorial!

@santhosh8395 6 ай бұрын

are u using paid chat gpt api? or any alternatives?

@MarvelMaster 6 ай бұрын

yes jts kind of paid... like 4cent for 100000 words or so

@chairgraveyard4790 Жыл бұрын

ChatGPT's constant need to ask if you need assistance is way more annoying when voiced haha

@MarvelMaster Жыл бұрын

you can probably tell it not to do that

@chairgraveyard4790 Жыл бұрын

@@MarvelMaster I actually prefer the uncensored local-hosted models for characters for this reason, they are easier to keep "in-character" due to the lack of political correctness training in their datasets. Wizard mega 13B seems like one of the top ones right now for that, though MPT is also good, and has the state-of-the-art 65k context length version as well, which I think would be a game-changer for this use case (imagine the AI remembering everything you've ever done with it)

@BOOMER_T_V 11 ай бұрын

Russian language supported?

@MarvelMaster 11 ай бұрын

I think not jet

@crusaderpr7683 Жыл бұрын

Is so concerning when he ask that if the AI is okay to be recorded like AI dont have actual concious it is a bunch of sources gather in one place that has a ram and cpu install to organize them so our technology hanst evolve yet to create an actual robot with feelings Please dont confuse a monotone robot to an actual concious being

@SirSpence99 Жыл бұрын

Your grammar and spelling is awful but you are right. Anthropomorphizing AIs is a far larger risk at this point than an AI uprising. That might change in 10 years (I don't think so) but for now, I'm more concerned with people taking advice from autocomplete than from autocomplete figuring out the nuclear launch codes.

@crusaderpr7683 Жыл бұрын

@@SirSpence99 is good that you know english i wonder if you can write spanish as well but yeah

@SirSpence99 Жыл бұрын

@@crusaderpr7683 I have no interest in learning Spanish. In this day and age and of all of the languages that are used today, Spanish is third to last on my list of languages I'd be interested in learning. French and Chinese (and its variants) are lower on my list... Not that I'm particularly interested in learning any new non-programming languages. That said, did you read the rest of what I wrote? I was, trying to expand upon what you said while not disagreeing with you.

@MarvelMaster Жыл бұрын

@@SirSpence99 we can also ask chatgpt about that 🙃