Very interesting. Thank you!

Does it make sense to speak of using regularization like L1 in the DARTS equations to make sure that some of the connections to some of the considered components of the model architecture (the "architecture coefficients" you mentioned) are reduced to zero to end up with a discrete architecture instead of a soft one?

Thank you so much for this lecture. Just wanted to clarify one question. So if am understanding correctly, Each operation in DARTS has 2 parameters: alpha (operation strength/ operation coefficient) and actual weights (w) of those operations (which are matrices probably for CNN). Initially all these operation weights (w) and alpha are random and fixed at the start of the process. We are NOT manually intervening on them. However the question would be, in contrast to usual CNN architectures, does DARTS assign a new parameter - alpha on top of the actual weights of these operations and are these alphas kept same at the start of the process ? The set of all alphas are vectors I understand (as per the DARTS paper) and each alpha associated to each operations are scalers. nodes are states or latent representations and edges have operation choices. Then, each intermediate nodes (j) receive mixed operations (i.e. summation of {product of [Softmax scores based of each alpha ]multiplied with the feature map obtained by putting that operation on 'x' where 'x' represents feature map from predecessor node (i)}) . The operations we are talking about are the choices that lies in the edge connecting that node (j) to its predecessor (i). Now, DARTS minimizes training loss by updating the weights of the operations. i.e. it tries to optimize the actual weights (matrices) of the operations by gradient-descent. Using those actual weights , it then tries to minimize the validation-loss too by gradient descent and in this case it optimizes each respective alphas of each associated operation choices (ie scalers which represents strength and being used in the Softmax operations .So alpha vector (set of all individual alphas) is are getting changed here, I am hoping ?). The alphas we get at the end of the process are compared and choose the one that has highest value. Thus DARTS jointly optimizes WEIGHTS (by training loss ) and alpha or operation coefficient or operation strengths (by validation loss) . Questions : 0) Is my above understanding correct ? 1) What is the relation between alpha (operation strengths) and its actual weights (weights of those operations, example weight of a conv filter which is a matrix and that is getting optimized during training process) 2) how are alphas initialized at the beginning ? Are these scalers randomly assigned or are all same at the beginning ? We are not manually assigning weights to these operations right ? 3) Also the authors assumes that each cell that 2 input nodes and one output. While output nodes can be one, in my implementation I used more than 2 input nodes or 1 input node even. As per equation (1) of DARTS paper which just sums the operations from each predecessor nodes, I don't think this losses generalizability. Please let me know your thoughts. I would be really very grateful to you if you can kindly clarify this question. Lectures on NAS are very rare so there are little chances to ask questions like this. Thanks a lot for uploading your videos. It helps!

Unfortunately it is in audible