Voice assistants are a privacy nightmare

The sales person that secured the funding for this dead on arrival product is a genius. The slightest amount of critical thinking would lead to the conclusion that this is an idiotic idea.

As a nurse I frequently have contaminated hands, and I find Siri can be helpful to do certain things without needing to touch my phone screen; "hey Siri, what's 235 + 473", "hey Siri, set a timer for 1 hour", "hey Siri, turn on torch" etc. But Siri could do so much more for me. She's too verbose for night shift use, where I don't want her to be silent, but one-word answers would be perfect, and I can see that the torch has come on, so I don't need her to confirm it. And I don't know why she can't control apps. I have the tp-link tapo app for the lights in my bedroom, and it would be great if Siri could access that and control my lights for me, just on and off would be fine.

My Mom recently had to get a wheelchair and is losing her eyesight, which made turning lights on/off harder than it already was, so I finally broke down and replaced all her light bulbs with smart bulbs, put smart plugs on her lamps and set up her smart TV with Google Home... It was a real game changer for her and now she's started using her Google Assistant to really augment and assist her day to day. AI Pin has it completely wrong, to think their product and voice is going to replace our handheld computers and screens (cat videos aren't going away folks), when the future of assistants is actually going to be more in the background of our day to day, **assisting** us with the small things that add up to bigger things, like easily shutting off all the lights with a phrase and saving on that power bill (and eventually automatically shutting off the lights as we leave rooms). With that said, they could all take a hell of a leap forward by making the activation phrase optional and allowing us to name our assistant and to better personalize it.

This voice computing trend reminds me of the metaverse hype. These products are mainly solutions in search of problems

Humane is like someone trying to make a new kind of pocket camera in a world where everyone already have one in their hands.

The issue with voice assistants is that 90% of the time, telling something to do something just isn't faster or convenient and sometimes even straight up annoying. I only ask the voice assistant to turn on the lights when I am not near my phone or lightswitch which is almost never.

There's an old saying that applies really well here: "A picture's worth a thousand words."

What is consistently overlooked is the question of language. Until now, all speech-to-text (STT) efforts are led from the US where the people working on this (well, most of the US to be more precise) somehow imagine a world where we all speak the same language (English). STT fails spectacularly outside the confines of Silicon Valley. The other day I was trying to tell my Tesla to call my mother-in-law. She has a French name. There was no way the STT could find her as the sentence "Can you call X?" switched from English to French. Similarly with navigation. I ask it to go to the "faubourg de l'hopital" and again, the assistant cannot understand this. Even when I try to pronounce it with an English accent.

The only time I found myself really wanting a voice assistant was when I realized how much I wanted to say, "Hey Google, find me a gas station" to my phone while I was already driving on the highway with Google Maps running. Of course it didn't just work, and when I looked it up I found out that it would be a kludge and a half-working hassle to get it working at all. That was the last time I bothered about voice assist.

It just never stopped feeling a bit weird interacting with a voice assistant in public

Yeah, that shopping thing was what always astonished me about Amazon with Alexa and Buttons. People don't just buy things sight unseen, especially on the Internet where they could be scammed, and *never* on a marketplace like Amazon is now where crap is the dominant product.

Also, I would be so tired of voice-only control, simply because I hate talking a lot. Like, not in a "terribly social akward" kind of way, but more of a "Yeah I can have conversations and read you a little article, but I'd appreciate not having to talk every few seconds just to do literally anything"

I think the voice assistant should be treated as a feature rather than a product. Google assistant is the main factor for me to not switch to the apple ecosystem because I use it a lot and it is far superior to siri.

The only voice assistant I use is google and it's only to tell it to shut up when it activates by accident.

One thing to keep in mind with science fiction, especially for movies and TV, is that a voice assistant shoots better than a character using a smartphone-style device. You can continue shooting the main character from any angle you want and simply have them speak, rather than needing to show their screen or provide contextual queues to make it clear what the text input and output are. They also typically don't show people killing time on their electronic devices, which is a core feature lacked by a voice interface.

Right, you don't just do the Friday thing... I forgot you used to upload videos that weren't just tech news. Good to have you back man.

the speech coherency limitation is actually not an insignificant issue. I’m at least bilingual and constantly switching languages when talking, I don’t trust the speech model enough to pick up context whenever I blurt out a non-english word in the middle of an English sentence.

I think speech input is more of an accessibility feature than a primary function

You just demolished that start up

I think the most general limitation to voice interaction, as you mentioned, is how slow it is. Provided that I have my hands available, I can do just about anything in less time than it would take for a voice assistant to be activated, hear and understand my prompt, then process it and take the corresponding action. Wonderful for when my hands are not free, or for things that I don’t care to get up and use my hands for (e.g. turning on the lights), but everything else I could just pull out the computer that’s always within arm’s reach anyway.

4:23 tech support questions from the elderly i don’t think gpt could answer ‘it just beeps now, everything is broken’, ‘how do i get back to the beginning’

I thought that Microsoft Copilot (Bing Chat) was the perfect candidate to be the new Cortana. i think Ai Chatbots with few tweaks could be the perfect digital assistant and Cortana is such a perfect name and lore that it's such a shame that microsoft has wasted it.

i honestly feel The Expanse does computer interaction in a way that would actually be the most convenient IRL provided we can get it to work: voice commands are used primarily when your hands are occupied, otherwise it's largely just vague gestures much like how we use touch screens and touchpads. What i want to see is things like being able to point at a lamp and turn an imaginary dimmer knob, THAT is the ideal user interface.

Humane's device comes off as something that could be neat as an accessory, but is basically dead on arrival as a dedicated device. Even on Star Trek, voice was consistently presented as being part of a larger interface. The characters still used screens, laptops & handheld devices for different tasks. The voice commands were just one of many ways the characters would interact with their devices, much like it should be in real life.

Digital voice assistants can be incredibly helpful for disabled people, who may have limited vision or can’t move much, so that someone who’s unable to physically control a phone can still use it. I’m doing my master’s degree in linguistics and there’s multiple projects at my university that are developing language technology applications for people with disabilities, and I love that. It’d be great to see more big companies invest in the space, but it’s probably just not profitable enough.

I have to turn Google Home off when I have meetings because I keep triggering talking about work, plus my colleagues keep trying to trigger my home automation, such as "hey google, lights off"

Voice is just an awful way to interact with computers. It is way too slow.

Also, imagine troubleshooting the a.i pin with just your voice, it's sound like a nightmare

voice to computer never works for me. When i speak my language (Icelandic) i have yet to meet a computer that understands me (google assistant comes up with nonsense, Siri does not get anything neither) ... and when i talk to the computer normally in English, my rather thick accent makes the AI fail most of the cases .. unless i ask for the most mundane things like "what time is it?". I can communicate with the AI when i talk very, very slowly and clearly in English .. like talking to a baby. But then, i really do not want to walk around talking to thin air as if i am talking to a baby. Also .. Siri is just rude. When i say "hey, siri" - it replies with a rather annoyed sounding "ehh?!". When i asked it if that can be changed it just replied "no, i can´t" (change the greeting) .. also the Apple phone is not mine, its my sisters .. so maybe she set it on being a di** to me when i use it .. i dont know.

your video is soooo good. how can it be so engaging, informative, fun, and reasonable. reasonable is the key here, like it's so clearly a personal pov and articulation of yours, still i find it objective and logical. soooo nice. i cant wait for this week Friday check out. but i would wait even more for a tech alta video on the openai saga 2 years from now!

I really hope that If voice assistant would become big thing, that it will be open source, so people can create custom integrations with their devices. Then if you want to control something new, using voice - you just download extension created by some other people. At the same time, I don't know if we should make everything "natural talking experience". Maybe we should make it more command like. Or both: if we don't know voice commands then we speak naturally, but when we want to be effective, we use short and precise commands.

I have basically never used a voice assistant. I think it is not really a thing in non-native english countries, but GPT can actually do different languages well, so I wounder if it will help on this.

Preached king, I'm so annoyed by all of these out of touch tech CEOs trying to make something they saw in a movie and didn't think twice about.

Phew... for a tech video this had some deep sociology insights about how much computing has changed human communication and general lanaguage needs. As a linguist and a language teacher, some of the points made hit pretty hard. I see why I have problems motivating my students to even attempt any routine language exercise.

As someone who has worked behind the counter at both a book store and a board game store, I can attest shoppers trying to force an online price match. That use case is more likely than you think, and while this would be a great tool to find the cheapest price, it would ironically be very annoying to any small brick and mortar store that doesn't operate at the volume of Amazon.

I think I would start using voice assistants a lot more if they would become as good as the experience you would get if you were to give your phone to a friend and ask them to do something on it (like when you're driving or whatever). This would include being able to interpret vague commands from some available context and being able to naturally interface with every app on your phone. Voice-based navigation would become pretty good as well, like being able to see outside and say things like 'yeah you have to take that turn right there, where the black car is coming from' and being able to ask it for some extra clarification. Or asking it to do something on your phone while your hands are full, like opening a guitar tuning app or reading out some specific information from some other app. With natural language processing on the level that ChatGPT does, I can see this stuff becoming reliable. Especially when you don't really have to thing about perfectly phrasing your commands.

Imagining voice controlled computers made sense in a time when controlling computers meant typing up machine code in a typewriter, having somebody else turn that into punch cards, having somebody else feed that into a computer, then getting back the result in the same way in reverse; it sounds really efficient compared to that, and I think that's the time the sci-fi trope comes from. But since then PCs, monitors, GUIs and touchscreens happened, people claiming they can beat that sound like snake oil salespeople.

Thank you so much Marton for your consistent quality! This articulates so much of how I feel about all of this. Unlike my feelings though, you actually do the research and back it all up. Keep it up! IMO, you occupy such a unique space in the tech journalism world. Covering how tech and business intersect + evidencing the insanity of bad ideas

When you were talking about voice coding, I will note that it does exist. I particularly have heard of Talon voice as a voice control for desktops, with meaningful amount of terminal type capability, but most people don't want to deal with terminals. If you are, voice control as input is potentially useful, especially if you don't want to use you hands. Voice output is less than useful as a sole method, however.

you know what would be cool, an easily programmable/tuneable voice assistant where you can program it the way you can program autohotkey cause that's really what i see voice assistant to be good at, do simple task while you're distracted. Problem is those "simple task" is very specific to the work you're doing and being able to pick and choose how and what it response with would be so cool

I think AI-integrated voice assistants (that are still on our phones) will be useful, at least, more useful than the current ones. I am looking forward to Google Assistant with Bard, which seems useful and most likely will be, given how much I already use Bard to do stuff. So, that just being integrated with Assistant (which I also sort of semi-regularly use) would be great.

This reminds me to be thankful for the sense of vision & how difficult it must be for those who don't have it.

Good video… and the example email gave me a good chuckle, so mission accomplished

Even for playing music these voice assistants are terrible. They often trip up over the language change Dutch to English (with a Dutch accent), let alone that they succeed if I want to play a song that has a non-English title in a language I don't speak at all.

many of the problems voice control is meant to solve seem to assume the phones visual interface is some how bad or broken, which as a UX designer feels pretty hurtful

Out of top 5 most populated countries in the world, only one (USA) is English speaking nation and all others are non-English regions. In these countries, there are further hundreds of languages & dialects. This is precisely why google & apple couldn’t penetrate voice assistant markets here. I speak fairly good English but i struggle to use voice assistant even for the simplest of tasks.

11:35 People so busy trying to invent the future they forget how to make the present work better. Half the webpage here is empty space and forces lots of scrolling; this contrasts massively with the spreadsheet app that follows.

I think the issue with voice assistants is that they lack conversational skill. They have a kind of "raisin bread" of ability, where the raisins are actual things they can do like turning on your lights or preheating your car, while the bread is the conversational rapport that gets you from raisin to raisin. The voice assistants of today have ignored the bread for the raisins, and, when you can't even ask Siri how her day was and expect a reasonable answer that segues into what you actually want, using a voice assistant is just a chore. I hope GenAI will help! I would love to be able to have small talk with siri one day

Part of it is that the assistant cannot understand what you're saying half the time. I speak with a "neutral" American English accent and the assistant just can't understand me, even when I try my best to speak not too fast and annunciate. I also speak another language natively, and a third with a thick accent; the assistant is basically unusable then. Another thing as you said is the non interactivity of the assistants. In real life, people correct each other all the time so that the conversation stays coherent. You can't do that with a voice assistant. Even for simple things this makes doing a task purely with voice so annoying.

Very nice analysis!

The problem with voice assistants is that you very quickly realise that they don’t actually understand you. Once the initial impressed feeling of this sci fi tech wears off you stop using it for anything complex because it’s not worth it. A true voice assistant won’t work until there is actually AGI.

Unfortunately, voice assistants are still pretty limited. I asked Alexa yesterday if Menards (local store) was open, she assured me that they were. It was Thanksgiving, so they were not. Fortunately it was a trick question on my part.

Memory and context is a very big requirement for audio assistant. If I had a techaltar audio assistant at my home then i would talk to him for days and have debate about tech industry. But that is only possible if he remembers past conversation and have a perfect context knowledge. We humans like to have talks and debates. Imagine having all these great personality at your home to debate anytime you want. Your own personal podcast hehe

I do think that the idea of a voice assistant, at least in my opinion is, that it compensates with generative AI for all its shortcomings. For example, instead of reading out long e-mails, finding lists of google maps, or browsing sites for shopping, it could simply condense the information, and notify possible responses. For example, If I wanted a new drill, instead of going to amazon, typing in drill, and looking, I tell it "Can you buy me a new drill" whereupon it will verify things like charging cable, and probably ask 1 or 2 questions about preferred price range, battery life and what drill bits I need, whereupon it can order it. This would even be preferred, as to me the main barrier to online shopping is finding the thing I am looking for in a sea of garbage.

The bugle podcast always joked they were "an audio newspaper for a visual world" . Voice assistants are the form factor equivalent

I don't use voice systems. Not for privacy reason(we give away our data every second that we are on the internet) but there is little benefit compared to just using an phone app. It looks like a gimmick to me.

I think *the interface of the future* is going to be direct thought control. That's probably still wayyyys off - but it will become reality one day. I don't think voice is a good choice, for many reasons. One being, that language is inherently ambiguous and imprecise - the other being, that you'll never be able to use voice in an office setting, out on the street, or for anything that requires any level of privacy. While you can have a room full of people all typing simultaneously - a room full of people all talking simultaneously can only end in chaos. Especially in terms of UI and feedback given to the user, voice does a lot worse than a screen - and once you have that screen in place, and all the UI representations you need on the screen anyway - using touch to interact with those representations works better than voice as well. Voice control only has strong benefits, in any hands-free & screen-free scenarios. And there just aren't really a lot of those.

That was the first time seeing you brutally KO a product 😂😂 very accurate and thoughtful analysis ❤

A very nice analysis. Thank you!

I disagree actually. ChatGPT's new voice-chat mode is already RIDICULOUSLY GOOD, to the point that I regularly end up just speaking with it for getting work done or for study purposes. Its extremely quick, gives long in-depth answers and sounds pretty much exactly like a real human. Its everything GoogleAssistant, Siri, Bixby etc should've been.

It's weird. I've used all Voice assistants. And Cortana was the best. Now with this new AI push, Copilot is basically having Cortana back. I honestly see the future for it being very bright. For the past month, I've been using my Computer, just like the computer from star trek. It's dope and very useful. I'm also currently integrating it into my business. With copilot in Office 365 and sharepoint, I'm building my business from the ground up to include AI in my work flow. I do see AI being the "next OS". That AI pin is just using the wrong input and output method. Voice should be an augmented input method not a replacement. What I suspect is an OS that can potentially create apps that you need on the fly. And fully adapt to you. You can then interchange the device's screen with what ever type of screen you need.

Perfect summary. Thank you for making this video!

Haha, reminds me of the "connected fridge" concept that appeared in the 80's (yes, not a typo). They want to decide what you need. Doesn't always work.

I have the compact one. Got it to replace my Mac mini hdd for an ssd. Good stuff this kit! I have it in my bag all the time

1:18 Apple doesn't seem as much into layoffs as much sort of redirection: don't fire good employees, put them to work elsewhere in the company - whoever worked in Siri is clearly working on Vision Pro now

Not just the privacy concern, but also just self-consciousness is a big reason why people wouldn't use that

The only possible success is fully local and open source voice assistants that you can own yourself.

I can imagine when tech advances and these gadgets become really useful and common, you will start seeing people talking to themselves on the streets. When you think they are talking to you and you ask "Sorry are you talking to me?", the person just shake his head and move on, communicating to the device.

One of the classic principles of interaction design / user experience design is an "affordance". Affordances show/signal what is possible, e. g. a red pylon on the street shows that you cannot go there. Affordances don't exist with voice, therefore, you cannot figure out the constraints. Voice can't work without an immediate feedback of what we can or cannot do.

Voice assistants are just a dumb concept in general. I remember many years ago someone saying that the test of usefulness of any voice-based interface would be to have a human as your voice interface. It would be nice for a few tasks but not for replacing a keyboard and mouse, and much less a screen.

I think a big issue with voice assistance is the balancing act between the abilities of the assistant and privacy. There could be much more functionality if the assistant had full control of the inputs for one's device, maybe even allowing it to perform actions with the user not present, but most people do not have the trust in letting the company that runs the AI potentially doing anything they want, or finding out literally all of their personal information.

Very good points!! The only real application of voice assistants I can see would be for GPT voice conversations in computer RPGs. Somme modders have already implemented that in Skyrim, processing voice queries to verbally answer question ls based on the NPC's piint of view.

The real shame is that when Amazon bought Ivona Software to do the fancy spy speaker, they destroyed the voice synthesis for the blind department. I am really happy that at least it meant that Jeff has slightly less money.

The inferiority of consuming informational content through audio over text/visuals is what I tried to explain to a friend who refused to pick up reading and believed watching videos (especially with speed controls) was just the same. We've always known most of our input is through our eyes, and we can very quickly scan an image/paragraph to get the gist of it, figure out what to click, or flip through a book compared to speech where you have to listen to a lot of unnecessary words and skipping through is completely a guessing game (you won't just jump between the beginnings of paragraphs). Having said that, it's not dair to judge an audio-only device with its lack of visual output lol. The main difference between the two paradigms IMO is that on screens you search for things with your eyes while with voice assistants you ask it questions. Reviewing a document? You might just ask about specific details that you know are important and check whatever else you need later. Social media? You might ask if some specific people posted anything today and what the posts were about (if it can scan the content somehow). Recorded some content? Tell it where to save/send diff moments. It's a way to keep updated without being too engaged. Admittedly, you can use your phone that way with just some ear buds, with the ease if pulling it out when needed.

I tried voice assistants to aid my work on a desktop. Somehow, after a few days, the lag in getting things done, the constant monotone voice, and the interruption to music that I listen too, irritated the hell out of me.

I think that what surprises me most about this device is that it is such an obvious miss - like it will never be a drop in replacement for a phone, so why position it that way? Also - these are presumably smart people, did they not come across these challenges in earlier phases of product development? How can a whole company be so far out of touch?

Even in the movie ‘her’, which is the best case scenario for an ai voice assistant, the role of the voice assistant is pretty minimal, doing mostly administrative tasks, on a device with a screen.

As an accessibility feature, voice I/O is great to have but I can't imagine the mainstream user willingly making it their default way to interact with computers any time soon

Honestly. GPT4 in voice mode is very helpful and smooth while being on a walk listening to podcasts, and answering any questions I have. The free back and forth dialogue works well without any need to press buttons.

Voice is good only when you are doing something else you can't look away from, like driving, painting, cooking

Back in 2008, my teacher said smartphones were going to replace PCs. 15 years later and everybody still uses their PCs everyday. So no, voice assistants will not replace smartphones either.

Even the vast majority of scifi shows this sort of thing as a supplementary interface, not something that is the main interface

Standalone voice assistants simply dont work. The best implementation seems to be paired to your phone and as part of some wearable that is accessed quicker than opening up an app on your phone yourself. They also might get more productive if they individualize more, as in storing recent inputs and context on you on device to suit the needs of the individual user better than a verbal google search alternative.

This kinda reminds me of a "Cicret Bracelet" this weird startup project about smartband, with projector that displays "phone screen" on your wrist

How I love this "rost" videos. Keep them coming.

Your crease seems to be cracking, Sir! But thanks for the video, I enjoyed it very much :)

i come from a chinese family living outside of china. in terms of wechat usage, i notice out-china chinese don't use voice message features much, while in-china chinese loves using audio to send messages.

11:30 We like visual information because we use more of our visual sense than our ears, which means "more control" and "more sense of speed" (it doesn't have to be objectively faster as long as you _feel_ fast)

That's literally all I use my Alexa/GA for - set a timer, set an alarm, turn X on, turn X off. Sometimes I may ask what time is it if I'm in bed and just woke up and don't want to move. That's literally it. I never used Cortana, never used Bixby (not really), GA is very annoying most of the time - there was a bug for ages where if you said - set a timer for 12 minutes - it will do one for 2, no matter how it was said or asked, always 2 minutes. Turn my lights off - turns the whole house off. Which is why I switched to Alexa for the home control, I use GA for timers and alarm as it's easier to say Stop and it will, rather than Alexa, wait, Stop.

People: "And that boomer keeps calling instead of just texting" Same people: "Talking to a computer is a great idea, it's the future!"

All of the challenges being pointed out about the difficulty of using apps non-visually are already being dealt with by blind users every single day. I get why visual users don’t want to use apps in that way but it’s definitely possible with the right training and properly designed apps.

That pin really just sounds like something that should be a smart watch.

Sell the thing with glasses equipped with thought reading sensors to help with the security issue . Plus cameras and augmented reality and hand gestures capabilities.

voice/sound is just part of the experience of how we perceive reality. Phones are the best rn because they engage multiple of the senses and include sounds along with touch and a visual response.

5:37 that is a really impressive screen! The play icon is darker than the hand it's projecting on 😮 It's projecting negative light! They should definitely get at least one nobel price for that insane feat in physics :P

Took me a few years to even use the thing after getting it for Christmas one year, but now I kinda like being able to ask my alarm clock tomorrow's forecast.

At this point I just use KZfaq notifications to tell me when a new video is out on Nebula. Thanks for another upload TA.

What you said is kinda true, if we assume that in the future AI won't just be doing every productivity task perfectly on its own when told to and asking for clarification when needed. If that is the case however, voice would be a viable way of working. Images are still important though, so for content consumption, it will never be enough.

The biggest reason even though I want Assistants to succeed but not using them right now is the lack of 'AI'. i asked my google assistant if i should carry an umbrella today ? It threw at me Quota results instead of actually answering me. Similar thing also happened when I asked google if i should wear a sweater or a shirt would be fine ( it was 15 C outside) and again it couldn't answer that. Improving these small things would really make me use it more often