Max-pooling is a switch. Convolutions and weighted sums are dot products. ReLU is a switch, f(x)=x is connect, f(x)=0 is disconnect. Then conventional neural neural networks are switched compositions of dot products. If all the switch states are known then there exists a linear mapping between the input and output vectors. You can use metrics like the variance equation for linear combinations of random variables to examine that. Fast transforms are systems of dot products you can also include. In the extreme with adjustable (parametric) dot products to form fixed filter bank neural networks.

at 41:46 you showed an interesting graph between prob of error on y-axis and and number of samples on x-axis , do you have any idea if it is implemented within standard ML libraries or not whether in R or Python ? your input is highly appreciated

do they work equally well for data streams which have heavy tails where sample kurtosis is way higher than 3 ? best regards