As people have correctly noted: When I talk about the way we train at 9:50, I should say we maximise the similarity on the diagonal, not the distance :) Brain failed me!

Dr. Pound's videos are on another level! He explains things with such passion and such clarity rarely found on the web! Cheers

thank you for "if you want to unlock your face with a phone".. i needed that in my life

I'm a simple guy. I see a Mike Pound video, I click

This guy is one of the best teachers I have ever seen.

"There's a lot of stuff on the internet, not all of it good, I should add" - Dr Mike Pound, 2024

Thanks for taking us to Pound town. Great explanation!

These guys are so watchable, and somehow they make an inherently inaccessible subject interesting and easy to follow.

9:45 - wasn't it supposed to be "minimize the distance on diagonal, maximize elsewhere"?

A very important bit that was skipped over is how you get an LLM to talk about an image (multimodal LLM)! After you got your embedding from the vision encoder you train a simple projection layer that aligns the image embedding with the semantic space of the LLM. You train the projection layer so that the embedding of the vision encoder produces the desired text output describing the image (and or executing the instructions in the image+prompt). You basically project the "thoughts" of the part that sees (the vision encoder) into the part that speaks (the massive LLM).

The biggest problem with unlocking a face with your phone is that you'll laugh too hard to hear the video for a minute or so.

The man, the myth, the legend, Dr. Pound. The best lecturer on Computerphile.

Excellent. Could listen to him all day and even understand stuff.

I love these encoder models. And I have seen these methods implemented in practice, usually as part of a recommender system handling unstructured freetext queries. Embeddings are so cool.

Love seeing AI problems explained on fanfold paper. Classy!

Clip is fantastic. It can be used as a 'zero-shot' classifier. It's both effective and easy to use.

One day, we'll give these models some high resolution images and comprehensive explanations and their minds will be blown! It's astonishing how good even a basic perceptron can be given 28x28 pixel images!

6:09 "There isn't red cats" Mike is hilarious and a great teacher lol

The legend has it that the series will only end when the last sheet of continuous printing paper has been written on.

Computerphile and Dr. Pound ♥️✨ I've been learning AI myself these past few months so this is just wonderful. Thanks a ton! :)

Computerphile, This is great! I liked it and subscribed!

Awesome explanation

Just writing this to crystallize my understanding: (and for others to check me for accuracy) So by circumventing the idea of trying to instill “true” understanding (which is a hard incompatibility in this context, due to our semantics); On a high level it’s substituting case specific discrepancies (like how a digital image is made of pixels, so only pixel related properties are important: like color and position) and filtering against them, because it happens to be easier to tell what something isn’t than what it is in this case (like there are WAAAY more cases where a random group of pixels isn’t an image of a cat, so your sample size for correction is also WAAY bigger.) And if you control for the specific property that disqualifies the entity (in this case, of the medium: discrete discrepancies), as he stated with the “ ‘predisposed noise’ subtraction to recreate a clean image’“ training, you can be even more efficient and effective by starting with already relevant cases. Once again because a smattering of colors is not a cat so it’s easier to go ahead and assume your images will already be in some assortment of colors similar to a cat to train on versus the near infinite combinations of random color pixel images. And then in terms of the issue of accuracy through specificity versus scalability, it was just easier to use the huge sample size as a tool to approximate accuracy between the embedded images and texts because as a sample size increases, precision also roughly increases given a rule, (in crude terms). And that it’s also a way to circumvent “ mass hard coding” associations to approximate “meaning” because the system doesn’t even have to deal directly with the user inputs in the first place, just their association value within the embedded bank. I think that’s a clever use of the properties of a system as limitations to solve for our human “black box” results. Because the two methods, organic and mathematical, converge due to a common factor: The fact that digital images in terms of relevance to people are also useful approximations, because we literally can only care about how close an “image” is to something we know, not if it actually is or not, which is why we don’t get tripped up over individual pixels in determining the shape of a cat in the average Google search. So in the same way by relying on pixel resolution and accuracy as variables you can quantify the properties so a computer can calculate a useable result. That’s so cool!

i just feel 10 times smarter after watching any computerphile video

When they say "high dimensional" in the vector context, I like to imagine it like an RPG character stat sheet, as each independent stat on that sheet can be considered its own dimension.

The way he explains is very interesting.

i'm an it-guy & programmer, but this is kinda hard to understand. thanks for the video, gives a little idea about the concepts!

Personally I woulda just did the colour comparison by putting the 24bit RGB integer colour into a double (the 64bit fpn type) and divided one by the other. If the result is greater than 0.01 or less than -0.01 then they're not close enough to deem the same overall colour and thus not part of the same facing of a shape. **Edit:** When searching for images it might be better to use simple line path (both a 2d and 3d one) matching the given text of what to search for and compare the shapes identified in the images to those 2 paths. If at least 20% of the line paths matches a shape in the image set then it likely contains that what was searching for. Similarly when generating images the line paths should then traced for producing each image then layered on to one image. Finally for identifying shapes in a given image you just iterate through all stored line paths. I believe this is how our brains conceptualise shapes in the 1st place given how our brains have nowhere to draw shapes to compare to. Instead they just have connections between...cells? neurons? Someone will correct me. Anyways they just have connections between what are effectively physical functions that equate to something like this in C: int neuron( float connections[CHAR_BIT * sizeof(uint)] ); Which tells me the same subshapes share neurons for comparisons which means a bigger shape will likely be just something initial nueron to visit, how many neurons to vist, and what angle to direct the path at to identify the next neuron to visit. In other words every subshape would be able to revisit a previous subshapes neruon/function. There might be an extra value or 2 but I'm no neural expert so a rough guess should be accurate enough to get the ball rolling.

Brilliant mike !

4:35 "There is a lot of stuff on the internet, not all of it good." Today I learned 😀 6:05 I enjoyed that you mentioned the issues of red/black cats and the problem of cat-egorization Video was helpful, explained well, thanks

Thanks Mike, nice clear explanation. You can still get that printer paper!? Haven't seen that since my Dad worked as a mainframe engineer for ICL in the 80s!

Great video!

6:30 if for LLM you have nodes of meaning then you could look fof "nodes of meaning" in description and make classes based on those "nodes", if you are able to represent every language based on same "nodes of meaning" that is even better to translate text from one language to another then average translator that is not LLM, then you should be able to use it also for clasification.

Mike is the best

At 9:30, doesn't a distance of zero mean the image and caption have the same "meaning", therefore, shouldn't we want to minimize the diagonal, and maximize the rest?

When will wee see the multimodal LLMs be able to answer a question with a generated image. Could be how do you wire an electric socket, and it would generate either a diagram or illustration of the wire colors and position. Should be able to do this but it can't yet. Next would be a functional use of SORA rendering a video how you install a starter motor in a Honda.

approx how many samples do i need when i just want to do image categorisation (but with multiple categories per image)

i was curious, when you are showing the paper from above, are you transforming the original footage?

A good edging session (for my brain)

14:36 Why can't you just train a network that would decode the vector in the embedded space back into text (being either fixed sized or using a recurrent neural network)? Wouldn't it be as simple as training a decoder and encoder in parallel and using the text input of the encoder as the expected output in the decoder?

For using CLIP as a classifier: couldn't you train a decoder network at the same time as you train CLIP, such that you now have a network that can take image embeddings and produce semantically similar text, i.e. captions? That way you don't have to guess-and-check every class one-by-one? Anyway, I can't believe CLIP has only existed for 3 years... despite the accelerating pace of AI progress, we really are still in the nascent stages of generalized generative AI, aren't we?

Can AI say "I don't know what I'm looking at"? Is there a limit to how much it can recognize parts of an image?

Stable Diffusion 3 = potential topic Optimum workflow strategies using Control Nets, LORAS, VEAs etc....?

@dr Pound: sorry if this is off topic here but, i wonder if the problem of hallucinations in AI comes from us not treating the 'i don't know what I'm looking at ' answer of a model, as a very negative outcome? If it was treated by us as a valid neutral answer, could it reduce the rate if hallucinations?

So the Visio-Text encoder is dependent on the quality of the captioned photo dataset? If so, where do you find quality datasets ?

there is stray audio in the background that you can faintly hear at 0:05

How do you go from embedings to text of something never been see. before?

Awesome! I always wondered how the hell does the AI “gets” that an image is made with a certain type of lens or film stock. Or how the hell AI generates objects that were never filmed in a way, say, The Matrix filmed on fisheye and Panavision in the 1950s.

Amazing. Imagine the day when AI is able to detect digital image steganography. Not by vision primarily, but by bit inspection.... iterating over the bytes and spitting out the hidden data. I think we're still years away from that though.

Where did they get those images from? Any copyright infringement?

I didn't read the cited paper, but if I understood correctly, the 5 billion images need to be labeled for the training step?

There is a paper on The Gradient about reverting embeddings to text ("Do text embeddings perfectly encode text?")

Great explainer. Appreciated. I hope someone will cover AI music next

3:55 As someone born in the former GDR, I find it cute to label a Trabi as "a car"...

Quick question from someone outside computer science: Does the model actually instill "meaning" into the embedded space? What I mean is: Is the Angel between "black car" and "Red car" smaller than "black car" and "bus" and that is smaller than "black car" and "tree"?

This is as simple as it’s ingenious and it wouldn’t be possible without the internet being what it is.

Oh look, a Mike!

Difficult to see how convergence can be ensured. Lots of very different functions can be closely mapped over certain controlled ranges, but then are wildly different outside those ranges. What I have missed in many AI discussions is these concepts of validity matching and range identities to ensure that there's some degree of controlled convergence. Maybe this is just a human fear of the unknown.

How can AI determine the "importance" of parts of an image? why would it output "people in front of boat" instead of "boat behind people" or "boat surrounded by people"? Or maybe the image is a grid of square white cells. One cell then get its color progressively darken to black. Would the AI describe these transitioning images differently?

13:54 you actually probably can, at least to an extent there’s been some recent research on the idea of going backwards from embeddings to text, maybe look at the paper “Text Embeddings Reveal (Almost) As Much As Text” (Morris et al) the same thing has been done with images from a CNN, see “Inverting Visual Representations with Convolutional Networks” (Dosovitsky et al) neither of these are with CLIP models so maybe future research? (not that it’d produce better images than a diffusion model)

9:48 Been a while since I did machine learning class... Anyone care to tell me why I'm thinking of PCA? What's the connection?

Once you have a clip, can't you train a diffusion on pure images, just by putting an image into clip, and training the diffusion to output the same image?

Poundtown.. sweet...

The answer given for the "we feed the embedding into the denoise process" still felt a little hand-wavey to me as someone who would like to understand better, but overall good video.

But confusing to say that you want to maximise the distances on the diagonal. Of course you can define things however you want but usually you'd say you want to maximise the cosine similarity and thus minimise the cosine distance on the diagonal.

I wish I could hear Professor Brailsford's thoughts on AI these days man

9:48 I didn't quite get how do similar embeddings end up close to each other if we maximize the distances to all other embeddings in the batch? Wouldn't two images of dogs in the same batch will be pulled further away just like an image of a dog and a cat would? Explain like Dr. Pound please.

Heyo, a Pound dogette here!

I thought diffusion models had VAE and not ViT Correct me if I m wrong

Can't you do something analogous to stable diffusion for text classification? Get the image embedding, and then start with random noisy text, and iteratively refine it in the direction of the image's embedding to get a progressively more accurate description of the image.

I'm curios about a thing. If you have a bunch of millions of photos of cats and dogs and they are also correctly labeled (with descriptions) but all these photos have the cats and dogs in the bottom half of the image, will the transformer be able to correctly classify them after training if they are put in the upper half of the image? (or images are rotated, color changed, filtered, etc..).

5.02 Never seen a "boat wearing a red jumper" before lol

Question 🤚 Up until recently all images of a cat on internet were photos of real cars and the system could use them in training. But now more and more cat images are AI generated. If future systems use generated images in training it will be like a blind leading a blind. More and more distortion will be added. Or? Can that be avoided?

Do any of you ever get this weird feeling that you need to buy Mike a beer? Or perhaps, a substantial yet unknown factor of beers?

Each time I watch one of these videos, I could might as well watch Apocalypto without subtitles. My brain is not made for this 🤓

10:37 "If you want to unlock a face with your phone". Ha ha ha!

I wonder, if every cat image has specific "cat" types of numerical curves, textures, eyes and so on. So a completely numerical calculation would conclude that the image is of a cat. There's only so much variety of pixel arrangements at some resolution, it seems like images could be reduced to pure math. Im probably so wrong. Just curious.

"a mike"

'and we want an image of foggonstilz' me: wat 'we want to pass the text of farngunstills' me: u wot m8

Crypto Bull run is making waves everywhere and I have no idea on how it works. What is the best step to get started please,,

How does he stay on top of all of this technology and still have time to teach? …and sleep?

Stable Diffusion needs a CEO BTW ....just saying ... 😅

Me: added to resume ai expert

a yellow cat is called red cat

Truly a marvel of human applications of mathematics and engineering, but boy do I think these tools have significantly more cons than pros in practical use.

AI art is great.

At 16:20 the 'cat' looks more like a shower head.

Why is there always so much mess and clutter in the background of these videos? Do you film them in abandoned buildings?

the gorilla camera moving around all the time is making me dizzy

Isn't there an AI that fixed shaky video by now?

The exchange at 6:50 made me laugh out loud and I had to show that part of the video to the people around me😆😆

I swear that P took longer to come up than the rest.

Normally this guy is great for explaining things and super clear, but in this case it feels like he's kind of assuming some prior knowledge or understanding, and not really giving us the 'explain it like I'm five years old' version. And for a subject this complicated, you kind of need that.

This must have been the entire purpose of googles Picasa.

se han ddo cuenta que estos son un grupito de federicos a los que les patila el cloche.. ellos hacen videos qué no dicen nada

Oh, no… All cats are missing a tail 😿

Cat picture needs a tail!

Your cat needs a tail!

10:36 When you unlock a face with your phone... Umm what?

AI now understand what I cannot understand