Thanks for the comment. I’m glad you’ve liked it. I ran into the same problem while I was trying to understand NEAT. It took me a long time, so I decided to share what I’ve learned.

I’m glad :-) That makes sense. Thanks for the suggestion. I’ll give it a go. I just didn’t understand one part: “without running each snake until death”. When would you stop the training? What would the fitness function look like?

​​​​​@@TechWithNikolafor the first que. When you stop the training? You should just take no. for score like 130 when you hit stop the trainning and then you increase your bar like hit to 150 score and stop the training (end of the game like level 1 , level 2 cleared) For the Fitness function Here you start your snake size with 3 and score is 0 then you start with snake size like 10 or 20 and score zero and you can adjust the score when you stop training by reducing it Your this approach is good for train short snake again and again(overfitting with short snake) so ai has knowledge about how to play with short snake but when it become longer AI has very less knowledge about how to play with long snake compare to short snake so first you train with short snake then you train with long snake (you should try like level 1 when snake hit the size 20 level 1 cleared then increase snake size and go for level 2 and hit that mark and so on so like I mentioned above)

I’m so glad you’ve liked it! Thanks a lot for recommending my channel, it will help a lot with the growth.

Thanks Peter. I have actually tried many variations with different kinds of inputs before publishing the video, one of them being the distance to the food (encoded as you suggested). Unfortunately, I didn't get much better results when training the network - some comments suggest that this is due to overfitting, and I should be training the network by generating random states that include long snakes, and reward it for each eaten food. I'm keen to try this at some point.

@@TechWithNikola I took it upon myself to try this using my own crude python implementation and ended up finding some genomes that were able to seek out food very quickly by adding the proximity of the head to the food into the fitness function, so that the snake not only fits for a high score, but also sums the average proximity of head to food to the score. This gave me a lot more aggressive behavior for food seeking behavior, but I weighted it too much in relation to the score, because it will move towards food regardless of whether or not it needs to move through its own body to get to the food. So I was able to find some genomes that achieved scores of 65 on a 50x50 grid but it would only be because the food did not have the snake's body in the way, and 37 on a 10x10 grid was my highest score with a max step of 200 per game. I noticed that if i gave the snake more steps to train before getting cut off, even at 1000 steps, it would run into itself before the step limit and end the game. I think moving out of its own way to get to food will require an even more complex input embedding, complete with free spaces to move towards, so that it knows to leave room for itself to escape from a dead end. I'm not sure how to get much better results than that without increasing the input space by a lot, and thereby increasing the size of the neural net, requiring much longer training and much tighter mutation rates. I don't have the compute power or time to give this a try, but it's a fun experiment to run. Thanks for introducing this idea to me, it was a nice recommendation from the youtube algorithm after diving into a few machine learning videos recently.

I'm so glad that you've enjoyed it Sara, and thank you for the kind words :) Indeed, it's amazing how much of real-life ideas are applied to computer science.

Thanks a lot! I'm so happy to hear you've enjoyed it.

Thank you! I will consider more neat videos in the future if there’s enough interest.

I used an animation library called manim

@@TechWithNikola oh right libraries exist. i forgot

You're welcome! I'm so glad. :)

Thank you! I’ll try to make more videos like this going forward.

That’s very interesting. Thanks a lot for the suggestion. I will give this a go at some point.

Thank you Danya. I’m very happy to hear that you’ve loved it.

Thank you, I'm glad you've enjoyed it :) Also, thanks for taking the time to suggest improvements. I'll porobably play a bit more with the configs at some point. What are the operators with infinite states?

​@@TechWithNikola you can look in this document page 154 it is a bit old but should to the job, there might be more modern implemejntation now document:dspace.lib.cranfield.ac.uk/bitstream/handle/1826/93/J-M.Roger2003.pdf;sequence=2

Thanks, I'm glad you've liked the video! Thanks for the suggestions too. That makes sense. I will try the proposed updates at some point. I'm curious to see how that performs :)

Indeed. It was a lot of work to research the topic, a lot to implement it, and also a lot to make a video :-) i’m glad when people appreciate it, so thank you taking the time to comment!

Thank you :-) I’d like to get back go this project at some point. There are a few ideas that I’d like to try. Maybe even experiment with other ML techniques. I think that one of the downsides of my current model is that it overfits was short snakes, so I’d like to try different fitness function. Maybe generate longer snakes and instead of scoring snake length, try to score hoe many times it ate food.

Yep, agreed. However, I don’t know if that matters much in practice? My feeling is that it doesn’t because that’s just a way to interpret the genotype, and if it was relevant I’d hope that other nodes would appear over time. I may be wrong though.

Thank you!

Glad you've liked it. Yeah, a few people have suggested ways to train the snake, starting with levels, or more generally, generating random-length snake and rewarding it based on how many times it ate food, rather than length. I might try this at some point.

Thanks. I will watch these at some point. I do want to get back to this project again, and give it another go. FWIW, I have tried all of those suggestions without much progress. I think the problem is in overfitting the solution for short snakes.

Thank you! I code most of them yeah, then edit it later. I use manim, powerpoint and adobe premiere to do that.

Thanks man!

Thanks. I'm using Keynote on MacOS for code animation.

That's a great question. I have spent a lot of time trying to understand how original neat paper handles cycles, and as you've said it simply updates the network N times. I think the choice of using RNN (Recurrent Neural Networks) or a multilayer percepton networks shouldn't be coupled with how the network is trained. I see NEAT as a way to train the network, similar to backpropagation. My current implementation can work with RNNs as well, but I haven't implemented it - it wouldn't be too hard to add this functionality. As for why I decided to go with layered network for this project, I just thought it was easier to implement. I also have a better understanding of such networks (i know very little about RNNs), so I didn't want to diverge from that. Given that this is the first time I'm implementing NEAT, layered NNs made debugging easier.

@@TechWithNikola thanks for answer 😃 Also congrats on a great video 👍

Hello, yes this can be done in python as well. In fact, python already has python-neat library that you can use out-of-box.

Hi, thank you! AI is fun. The official NEAT whitepaper is here: nn.cs.utexas.edu/downloads/papers/stanley.cec02.pdf Just a heads up, it's not a very detailed paper and I've had to figure out a lot of details on my own and by reading lots of source code.

Thank you. It's called "Matrika - Lazer Beam"

Hi, apologies for the late response. It's been a very long time since I've looked at or implemented backpropagation. If you can do it by hand but you struggle to convert this to code, maybe I can try and help. Can you provide more information on what is troubling you? Have you tried representing the network as a matrix or vectors? If so, then it shouldn't be too hard to do this if you define operators such as dot product on the vector (multiplying each field with the corresponding field in another vector).

Ohh, that sounds like a better idea. I tend to OOP everything. What I'd do is try represent everything as an object, from the nodes all the way to the layers. Thanks for the reply@@TechWithNikola

Haha thanks. Not an AI (but, would an AI say that?) 😀

I assume you're referring to what I called neurons. Correct, they are known as perceptrons in mathematical models.

Hi, source code is currently available for Pateron supporters: www.patreon.com/TechWithNikola I will probably make the neat library public in a couple of months.

Hey, sure. I use a combination of Manim, powerpoint, keynote for code animations, adobe premiere for editing. I make formulas in powerpoint or Manim depending on the complexity.

@@TechWithNikola thank you very much for the tips

The input would just be a 4-bit bitmap: food?, empty?, head? , body?. Along with an extra 4-bit value to store the current direction. (as the bitmap will lose this)

Yeah, I have used function smoothness for better rendering which sometimes adds noise. The more likely result here was 0.