Really Cool! I never thought of PCA in this way. Thanks for showing!
@user-eg8mt4im1iАй бұрын
Another fun video thanks ! 🎉 I believe this compression/decompression may be used in fraud detection isnt it ? The visualization is great !
@alexloftus8892Ай бұрын
Cool! The way I'm thinking about this is that \hat{X_w} is at most rank 2 (in this case), since it appears as the result of a linear transformation of a rank 2 matrix. Therefore since it is the matrix that minimizes MSE loss from X_w, the span of \hat{X_w} defines the best-fitting (by MSE) 2D plane for the 64-dimensional datapoints. So an 'outlier', in this case, is 'any point which is far away from the plane'. Because we're doing PCA, the axes of that plane are the two directions of highest variance within those 64 dimensions. One thing to bear in mind here is that as you increase dimensionality, distances in the space increase. So you're going to have to change whatever threshold you use if you go up to, for example, 64x64 images instead of 8x8, and things might get weird.
@ilearnthings1235 күн бұрын
genius
@Melle-sq4dfАй бұрын
I never thought of PCA this way but it's clever :) Would you say that you basically re-invented autoencoder ? or do you consider that it'll have subtle behavior difference with this method?
@probabl_aiАй бұрын
PCA existed well before autoencoders did, so it feels strange to call it a reinvention. But there is a relationship for sure. If you remove the activation functions and just have a single hidden layer then it feels pretty much identical to me.
@zb9458Ай бұрын
Amazing video! You really explained this in such a nice and easy step by step process, I feel my mental model of PCA is better now! I wonder, could you do a video on discrete cosine transform for encoding MNIST digits? I know patch-wise encoding is how JPG works, but I always got kind of lost in the linear algebra there...
@coopernikАй бұрын
That’s how encoders and decoders are used to spot anomalies no?
@probabl_aiАй бұрын
There's certainly a similar thing happening in neural network land. You could (somewhat handwavingly) look at PCA as an autoencoder with a single linear layer with a smaller dimension and no activation function that needs to output the input.
@denniswatson43269 күн бұрын
Great video by the way. Back in the day before generative AI for images I thought about trying this on something like the oliveti faces dataset. Reduce the dimensions to some latent space and then draw from there randomly to make "PCA faces"? I wonder if there is a distribution of the dimensions on the latent space? That is to say each individual dimension has some kind of non-flat distribution that would have to be drawn from in order to make a good face. I suppose if the chosen values where outliers then the face would be distorted?
@probabl_ai8 күн бұрын
There will for sure be 'a distribution' in the latent space but the question remains if this distribution can translate back to reality. PCA really is loosing a bunch of information and there is no guarantee that you might recover it. If you really want to go and reconstruct, why not just use a neural network? You may appreciate this old blogpost of mine: koaning.io/posts/gaussian-auto-embeddings/
@denniswatson43269 күн бұрын
Is it my imagination or do a lot of datasets exhibit that rotated square or diamond shape you see around 3:22 when reduced to two dimensions?
@carlomotta2742Ай бұрын
hey cool video, you show a simplified and linearized version of an autoencoder! Btw, I was checking the notebook you provide, when you import `from wigglystuff impor Slider2D` what is wigglystuff actually? I could not find any reference on the web. Thanks!
@probabl_aiАй бұрын
(Vincent here) It is a super new library that's mostly personal that I use for these demos. It's a collection of UI elements made with Anywidget to help explain stuff in Jupyter. Hence the name. It's widgets that wiggle to explain stuff. Wigglystuff!
@carlomotta2742Ай бұрын
@@probabl_ai that's really awesome! any plan to release it to make the nbs reproducible, or can you recommend an alternative to get the interactive pca graph? Thanks!
@probabl_aiАй бұрын
@@carlomotta2742 The shownotes of the video show a link to the notebook. Wigglystuff is on PyPi so that's just a pip install away. pypi.org/project/wigglystuff/
@carlomotta2742Ай бұрын
@@probabl_ai couldn't find it before - thank you!
@kanewilliams1653Ай бұрын
One part I don't understand is 5:38 - why is it x_w hat instead of just x_w? There is no epsilon/error term in a PCA. And assuming T is invertible, wouldn't it just return exactly the same original matrix x_w?
@probabl_aiАй бұрын
There is information loss when you go from the original input to the low dimensional representation. When you then go back you still have this information loss that makes it impossible to fully reconstruct the original array.