I really enjoyed this conversation with Jitendra. Here's the outline: 0:00 - Introduction 3:17 - Computer vision is hard 10:05 - Tesla Autopilot 21:20 - Human brain vs computers 23:14 - The general problem of computer vision 29:09 - Images vs video in computer vision 37:47 - Benchmarks in computer vision 40:06 - Active learning 45:34 - From pixels to semantics 52:47 - Semantic segmentation 57:05 - The three R's of computer vision 1:02:52 - End-to-end learning in computer vision 1:04:24 - 6 lessons we can learn from children 1:08:36 - Vision and language 1:12:30 - Turing test 1:16:17 - Open problems in computer vision 1:24:49 - AGI 1:35:47 - Pick the right problem

Jitendra is a clear and organized mind in a world of confusion, at least in the academic world

One of the best things that can happen to you, is wondering what to watch/listen to, and then a Lexcast comes out at that very moment. I'm excited! Thank you, Lex!!!

I literally just finished reading one of his papers, Deep Isometric Learning for Visual Recognition, before seeing what's new on my subs hehe.

I always learn something with every podcast thanks Lex

You get that smile out of them at the end (and often throughout, of course) almost without fail. Thanks for creating so many enjoyable rides.

Awesome Podcast! Loved Prof.Jitendra's take on CV, perception & Cognosis. Absolutely amazing how he transitions from CV to Psychology and other domains. Really gave me a great perspective on child perception & long-form video understanding. Thanks Lex!

This was amazing, always love to hear Prof. Jitendra Malik. Could you please schedule a discussion with Jeff Dean as well?

thank you very much Lex for sharing such interesting and informative talks with us!

Enjoyed it. Keep up the good work lex

Excellent conversation Lex. I feel the podcast has started exploring a lot more territories outside of AI but I am glad you brought it back to computer vision and deep tech. The demarcation between images and video is excellent. This is what I come here for!

This was a really good one.

Awesome thanks Lex!

Thanks you Lex Fridman, helps me understanding Computer Vision

Pretty insightful video. Lex, you are doing a great job spreading out the knowledge for free. That's one of the ways on how you can make a change in the world; bringing us these foundations. Thank you very much! We all owe you a lot.

Can u do a podcast with Hofstadter, that would be great

great interview, thanks

Another great interview Lex. Thanks for sharing this

Thanks for the wonderful videos, Lex, lots of love! Can we get Naval Ravikant, Jeffrey Sachs on the Lexcast? That'd break the intellectual part of the net!

You two are honest working intelligent men. What great conversation very refreshing in knowledge I’m glad to still see people still get good things from dedication and studies. God bless you both Jitendra and Lex

Took ML class with him last fall, great professor!

Thanks Lex

Thanks Lex ❤

I really like talks about AGI. Specifically, how to build it.

Please interview Noam Brown! The guy from Libratus and Pluribus

Loved it... !!!

Love the intros too :)

"humans are not fully explainable". Interesting (counter)point.

"however the heck you spell that" haha

Great! Eventually you'll have to interview Linus Torvards and Richard Stallman

Could you invite Prof. David Mumford. An amazing researcher with deep math background and who later on did vision. He has done very interesting work, everything from Mumford Shah functional, his paper on statistics of images, theory of shapes and patterns.

Any infomation on the Google Quantum Computer??? :):):)

Lex, could you invite a Russian scientist Sergei Saveliev for a conversation about Human brain?

The way Dr. Malik gives space for the social issues around AI when it’s his field is so reassuring. I agree, we should be aware of the risks of AI as proactively as possible. We need to have more diversity in tech and user testing pools to develop safer and less biased algorithms and AI!

Way to schedule the greats! There are probably a few interesting, as yet undiscovered, minds out of the UC San Diego Cognitive Science dept that could be worth interviewing.

Off topic, but if you can, please bring Terence Tao to your podcast!

I would love to be able to listen to these podcasts in the background while working, but it's impossible, these conversations with world-class experts in their fields demand full attention and concentration.

Did Jitendra say " tabular ...learning in a supervised world". can someone repeat the term/phrase again as i could not understand the full term he used and neither can i find it on google. Thank you

Hi Lex, you should get Sam Harris on the show. Hes extremely well spoken on the topic of AI and even Elon says hes great and understands the future of AI really well. He would be a really popular guest.

Our son's taking a class from Maliik now at Cal and I wanted to hear Malik speaking for himself. On 3D, I wondered: how has he intersected with research done on the brain's grid cells? And googled. Interesting. Consulted but not necessarily done research directly on the subject. Jeff Hawkins (Palm, Numenta) considers that grid cells may be fundamental to human intellect. Wonder how Hawkins and Numenta are considered by academia? There was an explosion in the capabilities of the homo sapien brain around 70,000 mya and given how quickly the transformation occurred could that have been based on, literally, one physical change?

This conversation is probably your best.

I think we should use tiktok, KZfaq, Snapchat...data for visualizing A.I before getting it on the road 17:50

Next Podcast with Andrej Karpathy please.

Lex, go high! Try to interview Noam Chomsky on AI (again). Sad that Josef Weizenbaum (Eliza) is not around any more.

Is it possible to create a network that can understand general motion much like how a human understands on a coarse level, inertia? Consider a Tesla that had a computer-aided model of the world outside the car that simulated inertial moments for objects, this would be another step in the right direction towards a spatial awareness that can accurately map objects and their likely trajectory. Humans can learn what skateboarders do with relatively little data because we understand general motion

children do need labels. I spend ages with my children, pointing and naming. It is a normal and vital part of life, especially in the first 3 years. It is about 90% of conversation, labeling stuff.

Parents and child development specialists could help if you really want to go the child like learning route... Start with eyes that can move, throw away most of the pixels to emulate the fovea and peripheral vision... set up a learning network that just tries to predict the next few frames... then work backwards from that, and add layers... until you get to where you can emulate neck muscles and the ability to look around... then you can add a throat, detecting "food" (nerf balls with rfid chips in them) that reward eating... but not too much.... No simulation, all reality... it would take time, but when you're done... you've got a robust model that works in the real world.

The problem in computer vision is that the concept of neural networks as we know them represent an abstraction in software for representing how to capture and propagate signals that represent a logical state or "feature". Most of the work in AI is how to build these networks in based on data inputs and "machine learning" models. But in biology, the creation of these signals and propagation of information is hard coded into the retina and optical nerve. It is hard wired based on evolution and biological function and not really "learned". The visual cortex in the brain is not "building" these networks from scratch. It is making sense of the networks of signals provided to it from the "real world" by networks of signals provided to it by the optical nerve. That is where the learning takes place in terms of how to make associations between sets of signals propagated via networks of neurons firing in a certain pattern based on physical input. And then the other parts of the brain can capture and retain certain properties of those signals and networks of signals in memory and recombine and manipulate those as part of 'imagining', 'recall' and basic brain function. The issue for computer vision today is that most of the work is in Re-encoding buckets of RGB values that are stored on a computer disk somewhere based on statistical math. The RGB values represent the initial encoding layer from the camera sensor and that form of encoding is 100% lossy in terms of providing networks of meaningful signal data to process. So using statistical mathematical operations will then reproduce the network of signals that propagate "bottom up" through a network to produce an overall statistical result signal at the top level. And that "neural network" represents a statistical encoding function with error correction to filter out "fake" data during training. But that is not how the human brain understands or encodes visual signals. There is no monolithic network representing things in a brain, as opposed to collections of groups of smaller networks representing signals captured from the real world with no need to error correct or filter out "fakes". For example, a tree would be a collection of signals representing shapes of leaves, shapes of branches, shapes of the trunk, color of the branches, color of the leaves, plus textures and light and shadow information. From that, the brain can learn what a tree is based on the encoding layers of the optical nerve that provided those inputs to the brain. Each of those various layers of encoding are first class entities and distinct networks into them selves. And the brain can learn to associate them together in a loose coupling of individual lower level entities (networks of signals). Because of that, the brain can still see a tree, even if it only sees the leaves an branches and not a trunk because logically those features (shape/color/texture of a leaf or branches) are memorized as part of a larger collection of features called a "tree". TLDR; It is a signal encoding problem where neural networks are simply the first pass of encoding data, the "intelligence" happens on another layer where meaning and understanding is produced based on input. The job of the encoding layer is to consistently produce the same range of signal values (groups of shallow network signals) based on the provided input, much like a computer logic circuit. You cannot put all of that into one model and expect the same kind of results.

if he gives way more technical answers it will actually be better, although more people will not understand, but thats why we are here

@1:29:29 golden words! the problems of safety, biases, risks are today. The bigger problems NOBODY is paying enough attention to, is the governance of AI that will be in your brain Lex and Jitendra. And your children. I am still surprised that no one is giving it enough attention. The video with Ben he mentioned some of those problems and how we are building resolutions for those. but we need an army. . Point is, I am NOT being negative here, quiet the opposite, AI can be fabulous for humanity but until then the governance is not being taken seriously. Attack from within... Pick a single entity you are comfortable with in taking over your "free will"... Or what we are proposing, "Atonomous Decentralized Governance" so no one can repurpose your extended brain/ AI without your control. . Today your phone is your extended brain. Tomorrow will be a lot closer to your brain.

Where is Demis Hassabis?

wow

Please bring Peter Thiel as a guest Lex.

Humans are black boxes too. Good point.

wasn't it Donald Rumsfeld that said " there are known knowns, and unknown unknowns"

How important is prediction, if the computer can observe and react at super human speeds? Sure prediction might save you from a .0001 situation but I think FSD will be good enough and save more lives than our current unsafe driving conditions. Perfection should not kill progress.

Nice yo

Do I think we’ll be able to understand it ever? Sure. In the next 20 years or so? No 🤣🤣 I died at this message and it’s so true it’s like we don’t even know how much we don’t know lmaoo

Fix the lighting. Jitendra shows up in well-lit frame and Lex in the different colored frames. Otherwise content is good.

Lex, I am surprised to hear you expressing so much doubt regarding autonomous driving. Especially after watching your interview with Jim Keller. Also you must have seen the video out there of teslas driving themselves on and off the 280 freeway, passing cars and being passed changing lanes negotiating intersections on the Sandhill Road. There’s also this thing about autonomous driving having to be perfect. I get this impression from people; That they believe autonomous driving can never have a mistake. Must always has to be perfect. If someone gets injured, or killed you have to throw it out. I think the more realistic way to look at it, and the way that Elon looks at it - it’s like this, when autonomous driving is 100 times safer than having humans behind the wheel, that will be a great achievement. Why would you ever have a human behind the wheel after that?

Always ready for a funeral

Without a Soul a machine cant see a reality. Not because it has no camera but because it only does what its programed to do.

Was that am jokes?! hahahaha

1:16:00 the blind assistant could be like a virtual Amelie Poulain :D Could be exhausting over time ^^ kzfaq.info/get/bejne/g7V0ZJSY07qvqXU.html

Lex. If our consciousness stops when we die. How would we know we ever existed? I mean us personally not through the people around you. If we are just gone. There is nothing, how are we experiencing this now? When we die we can't remember anything because there is nothing. I am not smart and I don't think I can properly describe what I am asking. Sorry if this makes no sense.

I guess the only place left in the universe for it innovation is radiation hardened hardware

I have a dream to talk to you.

I posted this comment for Joe that I am certain no-one will ever read.... maybe you will Brother/doctor/strong like Russian LEX FRIDMAN -LOVE U TOO LEX (am I crazy or is this the answer AI will come to eventually?): "Hey Joe, I love you my brother! Think of "DARK ENERGY/MATTER" as simply particles like Photons (force carriers) that only exist at a speed faster than photons and that is why us humans can not "see" it yet. Think of gravity as the only force that can interact with particles at "BOTH SPEEDS" (i.e. photons only exists at about 3 x 10(to the eight) meters per second or less depending on the medium, but "dark" faster than light particles exists at some much faster quantized speed (tachyon)). Think of BLACK HOLES as as tornados acting as particle accelerators taking known force carriers like photons and swirling them up to some, as yet undiscovered, faster-than-light quantized speed where particles become invisible to us yet still interacts with gravity. I have the mathematics to prove it but just like this comment, nobody will ever read it. See Joe, all this "dark matter/energy" bullshit is really very simple. Love what you do bro and THANK YOU JOE ROGAN for the free education you provide my friend!"

We need to raise robot children.

My brain needs a neuralink bc it’s so shit and fucked from drugs lul

Auto pilot uses radar and sonar as well as its cameras. I would love to hear Jim Keller or Elon debating this guy about whether or not autonomous driving can be accomplished.

Awesome interview, Lex, as always! Connected to the evolutionary discussion, Jordan Peterson lectures give great insights on the current theories of human and animal minds. It was particularly surprising to me how much action is in the hypothalamus, to the point that an animal can be quite functional with the cortex removed, and just acting based on the hypothalamus. I believe a discussion between you and Jordan would be super entertaining and insightful for the audience. Reference: KZfaq video "2017 Maps of meaning 05: story and metastory (Part 1)", timestamp 1h 40m 23s. m.kzfaq.info/get/bejne/iNuUftCo36rUfo0.html&t=1h40m23s

I presume that Malek shared some high octane ganja with fridman because during the introduction fridman presented to be uncharacteristically less Russian, almost to appear happy.

Jitendra is a clear and organized mind in a world of confusion, at least in the academic world