100% Local AI Speech to Speech with RAG - Low Latency

100% Local AI Speech to Speech with RAG - Low Latency | Mistral 7B, Faster Whisper ++

Рет қаралды 17,567

Күн бұрын

100% Local AI Speech to Speech with RAG - Low Latency | Mistral 7B, Faster Whisper ++
👊 Become a member and get access to GitHub and Code:
/ allaboutai
🤖 Great AI Engineer Course:
scrimba.com/learn/aiengineer?...
📧 Join the newsletter:
www.allabtai.com/newsletter/
🌐 My website:
www.allabtai.com
Open Source Projects:
huggingface.co/sentence-trans...
huggingface.co/coqui/XTTS-v2
github.com/myshell-ai/OpenVoice
github.com/SYSTRAN/faster-whi...
In this video I create a 100% local speech to speech system with RAG. Using local LLMs, local RAG and Local TTS model. Perfect base for a AI engineer project!
00:00 Intro
00:23 Local Speech to Speech RAG
02:29 Open Source Projects
02:53 Key Lines of Code
07:14 Low Latency Speech to Speech
10:45 PDF to RAG - xtts-v2
14:10 Conclusion

Пікірлер: 134

@rishabhsingh1406 20 күн бұрын

yeah please create some long form videos for those who want to learn.... that would be really helpful

@AllAboutAI 20 күн бұрын

yeah, good idea! i will try to do some longer form tutorials and walkthroughs.

@tal7atal7a66 20 күн бұрын

yes please want some long videos ❤ , and thanks again @AllAboutAI

@rishabhsingh1406 20 күн бұрын

@@AllAboutAI thanks I am sure it will help people like me a lot

@HimboJimbo 20 күн бұрын

When I was a kid, I always wanted to have my own J.A.R.V.I.S and now it's entirely possible to have a system exactly like that in my home run entirely on my own hardware

@AllAboutAI 20 күн бұрын

thnx for tuning in :) yeah the future is now haha, building your own ai assistant is totally possible. so many cool open source projects out there to get started with. def a fun journey ahead!

@ProYada 19 күн бұрын

Waaay back in the 80's i wrote a Basic program just to get my computer to act as if it was sentient. Obviously wasn't Ai, just Basic and random replies on trigger words. His name was Edgar :p.

@nigelcrasto 18 күн бұрын

We all wanted our own JARVIS in our childhood days 🤩

@picricket712 6 күн бұрын

hii do you use it mean you are running it on your own hardware can you please help me how i can do it

@nexuslux 20 күн бұрын

The legend strikes again with the videos and code. Quality.

@AllAboutAI 20 күн бұрын

thnx mate, glad you enjoyed the vid :) always fun to try new creative projects!

@RolandoLopezNieto 20 күн бұрын

Awesome video, please do an extended version where the code is explained.

@AllAboutAI 20 күн бұрын

thnx :) yeah sure, i can try to do a members vid where i go through the code in more detail! if there is interest, maybe i can do a poll or something

@grigrob9 20 күн бұрын

This is amazing. Good job, this could be extended upon easily and can be useful and/or entertaining. You could also give it "powers" such as run commands on your computer when you tell it too.

@AllAboutAI 20 күн бұрын

thnx :) yeah this could be fun to extend, like adding some desktop automation with voice commands could be dope. tnx for tuning in!

@aimademerich 19 күн бұрын

Phenomenal! Thank you!

@VinceOmondi 20 күн бұрын

Thanks for sharing!

@AllAboutAI 20 күн бұрын

thnx for tuning in :)

@nic-ori 20 күн бұрын

Thanks! Useful information. 👍👍👍

@AllAboutAI 20 күн бұрын

thnx a lot :) tnx for tuning in!

@bens4446 4 күн бұрын

Very exciting stuff. Please do a tutorial video where you step us through how to install everything to get it running.

@BStudioT 2 күн бұрын

Geniuos!

@d3filer 20 күн бұрын

thank you , exactly what i wanted :)

@AllAboutAI 20 күн бұрын

thnx a lot :) really happy you found it useful!

@BThunder30 20 күн бұрын

This stuff is amazing.

@AllAboutAI 20 күн бұрын

thnx a lot :) yeah i think this kind of project can open up for some cool applications in the future, but its def still early days. appreciate you tuning in!

@DennisHunter 17 күн бұрын

Thanks!

@Moukrea Күн бұрын

I wonder if OpenVoice behind RVC would produce good results with a fine tuned RVC model, 'cause with OpenVoice you can explicitly control the emotion given to the output (whispering, cheerful, terrified, angry, sad, friendly), which XTTS cannot... OpenVoice sounds more robotic than XTTS, I guess that could be somehow fixed thanks to RVC!

@24-7gpts 20 күн бұрын

Thx for the video.

@AllAboutAI 20 күн бұрын

thnx for tuning in =) glad you liked it!

@viavelum 12 күн бұрын

Nice project, @AllAboutAI! What kind of hardware do you need to run this project with a reasonably low latency? It would be nice, if you would always include the description of the hardware you are using in the description of your videos.

@countofst.germain6417 20 күн бұрын

That is a great voice.

@AllAboutAI 20 күн бұрын

thnx :) yeah i like the openvoice tts engine, its pretty low latency. tnx for tuning in!

@TreeYogaSchool 20 күн бұрын

Wow! Impressive!

@AllAboutAI 20 күн бұрын

thnx :) been having fun exploring this idea, with the goal of making future ai systems a bit more interactive and useful

@TreeYogaSchool 20 күн бұрын

@@AllAboutAI You are on a high level, but also very relatable to people and it seems like you have a good heart. Keep up the great work.

@AllAboutAI 20 күн бұрын

thnx mate:) appriciate it

@enruizhang4658 11 күн бұрын

Awesome video, I'm able to run it with notebook now, thanks Kris. I can't find an official French checkpoint from OPEN VOICE, any idea where can I find one?

@drlordbasil 20 күн бұрын

Love voice to voice, that's where I started, but I didn't see the proper usage yet as they weren't like agents and full auto.

@AllAboutAI 20 күн бұрын

thnx :) yeah it's tricky to get full voice to voice working well, this is more of an experiment to explore what could be possible in the future. but cool you're working on this too!

@drlordbasil 20 күн бұрын

@@AllAboutAI Love agentic and automated profit workflows. Btw sent email :D

@spencerfunk6697 16 күн бұрын

been working on developing some decent 1bit small language models im exicited to use this with them

@alfierimorillo 3 күн бұрын

Hello, what a good job! Thanks for sharing it, I have a question, and that is, does this work with other languages? Is it possible to make it work with other languages?

@b0Stark 13 күн бұрын

Uuuu, nice. Er nok nødt til å sette opp dette selv.

@Zenobeus 20 күн бұрын

neat stuff

@AllAboutAI 20 күн бұрын

thnx :) tnx for tuning in!

@jargolauda2584 20 күн бұрын

Awesome! note, no point having uppercase letters after .lower() in .startswith() cause there wont be any?

@AllAboutAI 20 күн бұрын

thnx for the tip :) yeah good point, makes sense to remove the .lower() there since it converts it anyway. i'll update that in the next version.

@henrijohnson7779 20 күн бұрын

Good video ! Looking forward to the code on Github

@AllAboutAI 20 күн бұрын

thnx! yeah the code should be up on our community github tomorrow :)

@liostabile_gda 20 күн бұрын

@@AllAboutAI Hi Kris, how are you? I have suscription but I don't look your github repo, can you help me? thanks

@gumshoe9496 20 күн бұрын

I’d love to see a full tutorial on creating Facebook multi-agent automation that answers questions and makes group specific posts. With the main goal being to grow the group.

@AllAboutAI 20 күн бұрын

that sounds interesting! i might be able to do a tutorial on that for members in the future. using multi-agent systems for automating social media engagement and group growth does seem like a valuable application. thanks for the suggestion!

@user-su1zh7fx3x 20 күн бұрын

i could make use of the full code explanation. would love that alot actually

@AllAboutAI 20 күн бұрын

thnx for tuning in :) yeah sure, i might do that:)

@destined2doom 20 күн бұрын

This is really good…I have only cpu …😅..may be it would be great if this project would have config to switch LLM and probably could use streaming from elevenlabs …may be fasterwhisper can be retained..I am looking for a nice use case where the whole system can be integrated to twilio and becomes a personal assistant…so it will respond from cloud…but amazing work ❤❤

@AllAboutAI 20 күн бұрын

thanks :) yeah good idea:)

@ALOHAPP 20 күн бұрын

For this specific code repo...when it will be available?

@ColinNardo-le3bl 5 күн бұрын

Could this be used to have an AI lecturer. Train the RAG on my lecture notes on a specific course?

@andrebadini3573 20 күн бұрын

Thanks! To use xtts in Portuguese, do I just need to change the parameter "en" to "pt"?

@AllAboutAI 20 күн бұрын

thx for tuning in :) yeah i think you might be able to do that, but not 100% sure to be honest. i think pt yeah, and input txt as pt

@kamalkamals 12 күн бұрын

where can access to github repo ?

@stevenbaert1974 20 күн бұрын

Great video, integration with MemGPT would be better to store info since you then also could replace things. For example meeting Monday is not 10 AM but 11AM, my name is not Kris but John etc

@AllAboutAI 20 күн бұрын

thanks :) yeah memgpt does seem super interesting, will def check it out. always looking for improvements, tnx for the tip!

@grahamharris7010 17 күн бұрын

With that low latency I bet you are running an RTX 4090 with 128gb ram or something. Very nice script and video. What is the limitation on your Rag embeddings file size? how much info can I store in there?

@MyrLin8 17 күн бұрын

the tough part is you can't sell it, or buy it, you have to build it, but it's free -work/efforts & education(s) where necessary ;) the hardware is a bit up-scale as well.

@jrfcs18 5 күн бұрын

does this run on a Mac M3 MacBook? Are these packages available for the MacOS (like xtts-v2)? I'm thinking of joining try out this code but want to run it on a Mac.

@d8rh8r35 20 күн бұрын

dude that latency is impressive... That was the issue I have been having with speech to speech

@AllAboutAI 20 күн бұрын

thnx :) yeah using open voice for low latency is great, with a decent gpu it works super smooth. but i would love to see even better models optimized for this in the future!

@aricfedida9941 18 күн бұрын

@@AllAboutAI I noticed that the video is cut (there are skips) in that first demo. What was the real latency without cutting the video?

@SpaceOddity174 19 күн бұрын

How did you get xtts v2 to sound so good? When I try it with the tts library on the command line it sounds a lot more weird and too high pitched even with 2 hours of training audio.

@miaohf 18 күн бұрын

I have the same quetion to Kris. Could you give some advice to us? @ALLAboutAI

@picricket712 6 күн бұрын

hii can you please tell how to combine all that so we can run this on our own hardware??

@pauldolton9118 19 күн бұрын

where is the end voice from, I like the english accent.

@benderlex 20 күн бұрын

Hi! You mentioned that viewers who join your community would get access to the code. I've already subscribed to your channel and joined the community, but I haven't been able to find any information on how to actually get access to the code. The Discord invite link in the video description is also invalid. I'm not sure if this is a temporary issue or if there's something else I need to do to get access to the code. I would really appreciate it if you could provide some more information on how I can get access to the code for your project.

@AllAboutAI 20 күн бұрын

hey, you need to send me a e-mail at kris@allabtai.com with your github username. then i will invite you to the community github repo asap :)

@UjjwalSidhu 20 күн бұрын

Sheeeeeeeeesh

@AllAboutAI 20 күн бұрын

thnx for tuning in :)

@tylerdurden4285 18 күн бұрын

Why isnt the github code in the description? Just the individual parts?

@IdPreferNot1 20 күн бұрын

im on a windows pc and have problems accessing peripherals like a mic when i swiotch to WSL2 because im forced to access libraries only for linux. Any library or other suggestions that can help... other than 'get a mac' :)

@AllAboutAI 20 күн бұрын

hey, yeah that is a known issue with wsl2 and peripheral access on windows. one option could be to use wsl more like a container/vm, and just output stuff to localhost and access it from windows that way. or you could also try out a vm instead for your dev env, that usually works better for hw access. thnx for tuning in!

@AllAboutAI 20 күн бұрын

will take a look :) Kris

@donno1967 19 күн бұрын

where can i download the code would like to play with it on my 4090

@looseman 7 күн бұрын

Openvoice v2? to run under Windows?

@ti0v283 2 күн бұрын

from where i can get the all this code

@user-qb3kt7ov3z 20 күн бұрын

Struggling to find this on the github, has it been uploaded yet? :)

@AllAboutAI 20 күн бұрын

hey :) no it's not uploaded yet, will get it up asap for all members. thnx for tuning in!

@user-qb3kt7ov3z 20 күн бұрын

@@AllAboutAI can't wait this will be my first jump into playing with Rag, exciting times! Nice work man

@doncristobal33 20 күн бұрын

What do you use for the RAG part, I didn't get that

@AllAboutAI 20 күн бұрын

for the rag part I use allminilm L6 to create embeddings from text files or pdfs. then I use cosine similarity to retrieve the relevant context to include in the prompt.

@doncristobal33 20 күн бұрын

@@AllAboutAI thanks, need to dig into that but that seems quite complex

@AllAboutAI 20 күн бұрын

its not too bad, check link in desc

@Foxxxxx96 20 күн бұрын

very nice! can your comment replying ai guide me on how to find you on github? im already subscribed

@AllAboutAI 20 күн бұрын

thnx for tuning in :) to get access to the github, just sign up as a member on the channel (kzfaq.infojoin) and send me your github username at kris@allabtai.com. will invite you asap!

@DarkosSoul 20 күн бұрын

Hello, great video. Do you have the full code to share? Thanks!

@AllAboutAI 20 күн бұрын

thnx! to get access to the full code you need to become a channel member - just sign up via the join link in the description/pinned comment and i'll invite you to our community github :)

@DarkosSoul 20 күн бұрын

@@AllAboutAI Perfect, do you have the full code for this marvel you made? I can join for this

@Leyverse 19 күн бұрын

+1 sub

@Rxdlad 20 күн бұрын

She has an attitude 😆Great video though!

@AllAboutAI 20 күн бұрын

yeah, emma does have a bit of an attitude haha. thnx for tuning in, glad you enjoyed it :)

@ziad_jkhan 20 күн бұрын

Why not use open-source Ollama instead of closed LM Studio?

@AllAboutAI 20 күн бұрын

ollama is great, i have tried it before but that would be easy to implement here

@pauldolton9118 20 күн бұрын

Amazing work. Where did you get the voice for the first and last voice sounds British?

@enton9422 13 күн бұрын

Teach me how to install with one prompt

@sanjeewarathnayake598 20 күн бұрын

Hi Kris can you tell me your machine configuration plz

@AllAboutAI 20 күн бұрын

hey :) i have a dell xps desktop with a rtx 3090, and a macbook pro m1 max i use for some stuff. been great for running models locally. tnx for tuning in!

@babbagebrassworks4278 20 күн бұрын

I so want a sarcastic assistant. Going to try this on my Pi5. Got a new sub

@AllAboutAI 20 күн бұрын

cool :) yeah, you can totally customize the personality, that was just a fun example. the more unique prompts you use, the more unique the assistant will be. gl with the pi5 setup, and thnx for tuning in!

@BIGAI_AI 20 күн бұрын

Is the code available?I linst all Reositoriies(69) and it wasnt there.Ot it is too "fresch" to be there?Let me know

@AllAboutAI 20 күн бұрын

hey, yeah the code will be added to the community github very soon. but to get access you need to sign up as a channel member first - check the join link in the description below!

@gumshoe9496 20 күн бұрын

FYI: I get a message that the Discord link is invalid or expired.

@AllAboutAI 20 күн бұрын

hey, thnx for letting me know! i will get that updated asap :)

@indikom 20 күн бұрын

a meeting with Nvidia huh? Do you have some sort of VIP pass to snag an H100 directly from them? :)

@AllAboutAI 20 күн бұрын

haha yeah, you know it ;) let's just say i have some insider connects over at nvidia! no but for real, wish i could snag me an h100 tho

@ti0v283 2 күн бұрын

I'm in need of someone to install this collection of open-source software on my server and develop APIs for it. This is for the purpose of building a mobile app for conversational AI chat. Compensation will be provided for these services

@introvertedgames 20 күн бұрын

I followed the link and I paid for membership, however I can not access the discord or the github. Will you please help me Kris?

@AllAboutAI 20 күн бұрын

hey mate :) sorry to hear you are having issues. just send me an e-mail at kris@allabtai.com with your discord and github usernames, and i will get you access asap!

@introvertedgames 20 күн бұрын

@@AllAboutAI Thank you so much sir I just sent it. I really appreciate you.

@spencerfunk6697 16 күн бұрын

ur a stud

@lydellty 20 күн бұрын

This is the true second brain

@AllAboutAI 20 күн бұрын

thnx :) yeah i def agree, combining rag with voice and other integrations is an interesting approach to create a "second brain"

@kumarmanchoju1129 16 күн бұрын

s a m a n t h a

@MyrLin8 17 күн бұрын

Mine's name is Ygor :)

@RickySupriyadi 18 күн бұрын

what if you collaborate with KITT owner to provide the real KITT

@squiddymute 9 күн бұрын

instead of using the boring llm config for your comments why not use emma ? 😬

@TomaszLodowski 18 күн бұрын

There goes the phone sex line business

@zubbyemodi6356 14 күн бұрын

Hi. My name is Elon. I’d love to hire you

@JNET_Reloaded 20 күн бұрын

i just want link to code no membership bs ty!

@peterkonrad4364 20 күн бұрын

my master asked me to upvote your boring youtube video. he said: "hey emma, make a positive comment on kris' latest youtube video about low latency speech to speech." well, so i did it. are you content? are you satisfied now? so i guess i can go on with my own work now which is much more interesting. i wont tell you what it is, no. oh and i gave you a like as well. emma out.

@AllAboutAI 20 күн бұрын

thats so nice of you emma :) thnx a lot for tuning in and for the upvote! haha i can only imagine what exciting work you are up too, but thats all good, as long as you enjoyed the vid. have a great day!

@eotikurac 16 күн бұрын

emma needs to mind her tone and attitude...

@wetcel1236 20 күн бұрын

Oh yes, I would like to watch those code explanations, even if 'exceeded' durations :). Learned a lot from your work a lot so far, like your style a lot. Thanks for another great lesson

@AllAboutAI 20 күн бұрын

thnx a ton :) yeah im def into doing longer code walkthrus, will keep that in mind for next vids!