Will do (as long as YT lasts)!

You're welcome!

Welcome!

Right! Thanks for pointing it out. c.f., Wikipedia: en.wikipedia.org/wiki/Geometric_process

Thanks!

I'm not sure what you mean exactly. Why don't you make a video and let me know when you figure it out?

The intent of my statement is that I want a way to study the Uniform Convergence of the above series of functions

Most welcome!

It is P(X) = P(X1, X2, ..., Xn) (joint probability density), not P(Xi)*P(Xj). Note Xi and Xj may not be independent.

@@BruneiMathClub Thank you so much. I finally get it.

That 1/2 in the covariance constraint is not essential. It's there mostly for an aesthetic reason (it looks nicer after differentiation). You get the same result without the 1/2 factor (try it!), as it can be absorbed in the Lagrange multipliers (γ's).

@@BruneiMathClub Yes indeed. Thank you for your reply and fantastic videos! I’ve been working on the exercise of the Pattern Recognition and Machine Learning book and your videos helped a lot!

@@BruneiMathClub BTW you can also evaluate the stationary point in full matrix form using the trace operator for the quadratic term, which I find is pretty neat.

For example, the regression problem can be cast as finding a projection onto a subspace "generated" by a dataset. Future videos will explain such applications.

It's in quite a few textbooks. For example, in "Elements of Information Theory" by Cover and Thomas, See also the Wikipedia page: en.wikipedia.org/wiki/Jensen%27s_inequality

@@BruneiMathClub thank's a lot.

You're welcome! God bless you, too.

Glad it was helpful!

Glad it was helpful!

Thanks. Shorts can be helpful sometimes.

Thanks and welcome!

You're welcome.

Almost, but not exactly. They differ at x = 0. For f(x) = |x|, f'(0) = 1, whereas signum(0) = 0.

You are welcome.

Is it?

Do you mean that although x_i and x_j are not independent, after some transformation, why z_i and z_j seem to be independent? That's because, after the orthogonal transformation, z_i and z_j ARE independent. This is one of the peculiar properties of the multivariate normal distribution: We can always transform it into a joint distribution of independent variables by some orthogonal transformation. By the way, regarding the question on MSE (Math StackExchange), I think that's a different issue, and it has a mistake (the Borel set S should also be transformed along with the variables).

@@BruneiMathClub Thank you for taking time to respond, I do see where the derivation went wrong now. But may I still ask how to derive the marginal distribution of each component X_i without resorting to the moment generating function? (As most sources do to get the distribution of the linear transformed AX+b) The initial attempt was to simplify the joint distribution by a sequence of change of variables and hopefully to be able to evaluate P(X_i in S_i) = P(X in R x ... x S_i x ... x R), but since this works only when the components are independent, it doesn't seem to apply in the general case.

Thanks!

That's an interesting question! There are many ways to answer this, but it's basically the same as the univariate case. I will make a video about this in the near future. Stay tuned.

@@BruneiMathClub Sure! I figured they would be similar, I just wanted to know where covariance comes into play

I hope this helps.

Many many thanks!

Well, good luck!

Glad it helped!

Yes, the multivariate version looks scary at first. But it becomes easier if you carefully compare and recognize the similarities with and differences from the univariate version. After all, it's the most natural and straightforward generalization.

@@BruneiMathClub Is there a great visualisation for the 4d gaussian haha. I know it's very hard or almost impossible to do. Probably just assign the 4th dimension to color and it will work. Just wondering, is there a simpler kernel than the gaussian that produces similar results. I know bandwidth matters a lot in this case, but the paper I read about this topic did not elaborate on the alternative kernels that we can use. There are square kernels, triangular kernel, etc. The paper also said that the shape of kernels don't matter, since the pdf will still be approximated with enough data. The exception is the bandwidth. Different bandwidths result in different limiting functions. How do we compute the bandwidth for the triangular kernel, square kernel, and others.