finally, my print("hello world") can run at the speed of light!

Amazing, amazing video. As a PhD student, I'm sure you've saved me much more than 8 hrs of compute.

Seriously this is brilliant. By applying this to the 3 main functions of a 1500 lines project I went from a 24 minutes runtime to about 20 seconds. that's 70X faster for a day's work (it'd be faster now that I learned the more complicated details). The C++ equivalent of the code does it in about 15 seconds.

One thing to note is that Numba really optimizes nested loops. If you say have to call a function a 100 or million times, better to have an outerloop instead that iterates within the function. With parallel turned on, you can see a 1000x increase.

Numba really is great. Here is my first experience with it: I have some code that I estimated would take up to half a year to run. With some minor changes to the code for compatibility with Numba, the run time for the same conditions was up to 1.5 hours. This was a game-changer for me as I needed to run this script many times. One of my colleagues also implemented it into his machine learning code and managed to obtain some significant speed bumps.

I implemented it in my code, and it took around an hour worth of computation and put it down to 30 seconds. Amazing stuff.

Similar to other commenters, I have to say this is one of the best KZfaq tutorials on writing fast Python code. Great buildup to the mic drop moment, amazing progression. Jack, I hope you make it big in the CS-tutorials KZfaq space and have a great day.

I cannot thank you enough for this video. I made a simulation of Gray Scott's Reaction diffusion in python, but it was painstakingly slow. After using numba, it now runs quite fast. Earlier it took nearly about 1 hour to render just 100 frames at a mere 400x400 resolution, now it renders 100 frames in 45 seconds, in 1280x720 resolution. Thank you very much for this video.

I am in tears to see my code run so fast, thank you so much for this video

Finally, this is what Python has needed

This is one of the best programming-related video tutorials I've ever seen. No-nonsense style, very direct and still computationally rigorous. Awesome stuff!

One of the most awesome Python tutorials I have seen. The narratives, the examples and the way you explain it is just awesome, 5 starts to you man! keep on doing it!

Grad students from around the world are thankfull for this video.

I had to submit a project in 2 days (that I had months to do but waited until the last minute) and in one piece of code, I needed to clean data of 6 million tweets. Regular python code showed runtime of 400+ hours (I used tqdm library for it). I got cold feet thinking I’m going to miss the deadline, but finally got the idea to search for ways to make the code faster. Found this library out on the internet and using it, I was able to run that piece of code in JUST 10 MINUTES. This library saved my project that day 👌.

11:45 - try this: np.zeros_like(input_list, dtype = np.int64, order='C') My experience with numba - it works great, but types have to be very precisely defined for every single object. Otherwise nopython will fail, and where numba sucks is generating traceback messages - in the case of complex functions it will take forever to debug. So type whatever you can - it is not a 'Pythonic' way of thinking and writing, but in this particular case it is necessary. Anyway, great introduction!

What an amazing video. I'm a student in Computer Science and we just did a week ago an introduction to Numba. And you sir not only explained to me the concepts in a positive astonishing way, but also gave me the desire and the excitement to apply it on all my old and future codes. Thanks you ! PS : Is there more videos where you're explaining the others topics ( convolution, neural networks .. )

Excellent video, in one day I implemented 80% of the code and strategies you explain. Very good ! Greetings from Rosario - Argentina.

After being skeptical about this library for a while, I've gotten used to it. I've watched many tutorials and read many articles and documentation. After weeks of determination to learn this technique properly,... I have become, comfortably Numba. 🤟

I came back to say a huge thank you. currently doing my Phd. my code was to run 300k iterations it took like 3days+. your short video changed everything, i took few hours to rewrite the code to suit numba as you explained. boom 2hrs i have results on the screen instead of 3days+

This was wonderfully clear and helpful! Thank you! My PhD involved running a lot of these forward problems under uncertainty, Lost much time trying to parallelize physics simulations and fighting with the GIL. Sometimes it is just worth waiting.

I nod my head with approving noises as if I understand what is going on.

I really liked that you stayed real and just said that you aren't qualified to comment on why the python loop with the numpy array took so much longer to execute than just saying something with the risk of talking bullshit and spreading misinformation in order to sound competent, a behaviour I have seen way to often online and which bothers me quite much. Very great, clear tutorial, truly one of the best python tutorials I have seen so far on KZfaq. Keep it up! :)

These are some seriously impressive results. I wonder if this could give Python a niche in indie game development.

I don't know why I need this. I'm still a High School student doing python for fun. But imma keep this tucked away in my mind. I have a feeling I'll need it soon enough.

I already had a background in Numba but I thoroughly enjoyed this video and learned some useful details as well. I'd be glad to see some more python related videos from you

numba is really sometimes absolutely stunning in it's acceleration per extra work.

That's an awesome video. Thank you! I've been running orekit scripts which took me ~4h, and with numba there are taking ridiculously 2min!

Jack!!! Congrats for this fast, clear and insightful video. This is a game changer for me (I'm a Fortran 77...20** programmer now starting with python). Live long and prosper \\//

Amazing tutorial and Numba is a great software. I hope Python officially will one day ship with a compiler because the community got so huge that people often write performance-intense code in Python that goes against its nature. Moreover, it'd be amazing to ship a Python product statically compiled without gigabytes of libraries!

Thanks for your introduction! This really helps me. I have been working with Numpy but it is too time-consuming. Working with Numba is really fantastic, my Python module runs more than 1000x faster than the original one.

I really thought the title was clickbait. It wasn't. This is honestly amazing, thanks for showing us this :D

Ah yay. Thanks for this. I must say Numba is very impressive. What I am most interested in though is applying it to a research setting which you helped a lot with as well! I am just happy that it was designed for scientific applications as well!

YES, I LOVE PYTHON AND THIS IS PERFECT THANK YOU MATE

Nice tutorial, I'm gonna try numma in my plasma physics simulations

I love how "incredibly fast" python code is pretty much about going back to c/c++ syntax.

I have struggle for Numba for 3 days. This Vedic is really clear

Both Numba and this video are amazing. The video was really really clear and concise. I'm definitely gonna remember Numba for future projects.

Why wasn't this video in my KZfaq recommendations a year ago? Nonetheless, this is an amazing tool for python. I wouldn't have found it without you!

Holy shit, I'm sold. Examples on point, mic indeed touched the ground. Thanks for the tip

Simple, effective, to-the-point! Encountering a topic modeling problem that requires a lot of steps of training, I was about to try cython which would have taken a huge amount of time. Numba turned out to be ENORMOUSLY SIMPLER and equally effective. Many thanks to numba developers and Jack!!

this is absolutely perfect because i just found this while running a really oddly intensive random number process (something like one million terms per second? i need billions) and numba sped it up by a factor of 100 halfway through the video, and with njit it didn't actually speed up any further, it just got a more consistent time (randomly jit is 10% faster or slower depending i guess on the random values

This is one of the best intro/tutor videos I’ve ever watched on any subject

As someone who tried numba and was disappointed I appreciate you showing possible causes and solutions for that

This is revolutionary. I always wanted a library to allow these but without the hassle of installing something like Cython, etc. Just a good ol' pip lib that can allow my code to speed up orders of magnitude. A nice bonus is the ability to release the GIL.

Thank you for your video. I work with satellite image processing and we are used to spend hours waiting for a particular processing task to finish... I for sure will be trying to use Numba

Why didn't I find this sooner. I am still so glad I found it will 100% use this in the future. Literally had a situation identical to ur example a month ago needed to simulate over 160,000 situations. Was taking 11+ hours to run. I assumed there were better ways but I haven't had a lot of experience. Thanks for this very clear explanation

You deserve my like. Nice work sir. Now I can show to my friends how a optimizated and we'll written code looks like. Numba.

This is the best Numba video, short , good examples etc.

Liked and subscribed. The length and pace of the video were spot on, especially as you’ve shared the notebook.

Damnit, why did I only discover this 6 days before my PhD submission deadline?!

This needs much more views.. This is a revolution..

Presentation level is ingenious!

Awesome video. Simply forwarded this to two of my junior devs and they completely got it now. THANK YOU!

Initially, it took Python 25min to run, but now it took 2min just by adding one line of code! You have definitely saved more than 8h for me! Edit: it actually took 2.5 sec.

couldn’t understand what this video was saying 4 months ago. i’m back and i finally understand.

Man. I must have been living under a rock to have missed numba for so long

Great video. The fact that you are sharing the jup notebook is amazing. Thanks a lot. Please consider making a follow up video with more complex applications of numba. For example, integrations!

This was the voice of god. Just today I was running a slow heavy code

Damn this is really cool, just came across it and it should definitely help in speeding up machine learning or ai projects

I didn't understand anything as a beginner to Python (after learning C,C++ and Java) but it looks really cool man. I will use this for python, especially in competitive programming

My new fav channel. Have you done a video talking about your Linux setup? If not I would love to see it/could you share some info?

Your way of explanation is very nice. Excellent video :)

thank you i had a code which took like a whole 5 minutes to start running and now it is much faster

Thank you so much! I was trying and failing to turn to C++ for number crunching, but this has just saved me!

Damn, I love your Python videos ! Very interresting, you talk about stuff other don't. Thx man !

The fact i got this in my recomended when i made a screencapture to table that was at 1 - 4 fps at 480p This is gonna help a lot!

This video was very interesting for me. I just understood why Python is so slow.

mate you're a jupyter magician

The first delay thing is not just the compile overhead for numba. It also takes a long long time apparently for "warm up" of the SIMD or AVX instructions. Even if the code is precompiled, if you change the matrix & recalculate it's gonna take a whole long time again for the first run. In applications when you need only a few iterations this is really not worth it. If you are doing several iterations on each matrix like more than 10 times then yes it's worth it.

Me: Watching videos about optimizing Python code to procrastinate studying for a systems and controls exam Video: Talks about systems and controls applications of Numba to mock me :)

Very well explained and saved me from some mistakes. You deserve way more subscribers!

10:10 Why are numpy arrays slow to dynamically allocate? -> The basic answer is that it is a C++ vector underneath and that vector which can be given extra capacity to append elements but at a certain point it will have to ask the operating system for more contiguous memory and has to be reallocated (this is very slow). The solution is to assign the numpy array a certain length beforehand with the np.empty() function. Python is designed to be fast to allocate an object of any amount of memory and a uses a linked list for its List type. This does have a cost in terms of defragmentation and garbage collection and it is slower to iterate through a linked list as well. Thank you for this informative video.

This is amazing. This completely changes python for me.

I bet my friend who called Python a "fake language" is soiling himself right now.

that's fantastic, I used to resort to using my uni's HPC cluster to get some more speed but now it's so much faster even on my old thinkpad, thank you so much!

Heyy man! This is a very nice tutorial for numba! There are only few numba tutorials out there and I hope you could make a bit more. For example, I became interested in using numba to speed up my Runge-Kutta Algorithms that I coded myself, but I couldn't make it work. I would try to apply it on the for-loop part but it would throw an error that numba doesn't now the class function, something like that. Hopefully, an extensive numba tutorial would be made in the future.

Fist visit on this channel, but you are the best ;-)

I'm a Java dev but I'm saving this for later, don't mind me.

this is golden. You explain stuff very clearly and easy to understand. Please do more vids :) ps. do you still playing around with Reinforcement Learning? Does your job relate to it in any way?

Amazing video! I'm starting to think to give numba a try.

Awesome tutorial! Thank you so much. I shared the video with my old Intro to Parallel computing professor. Hopefully he sees the email coming from non UW email id lmao. Video would be pretty helpful to any new(&old) students of the course.

Dude, I'm an aerospace engineer student currently taking my masters studies, and I have to program with python and some programs take some minutes to calculate, because I have to solve complex nonlinear equation systems, and that takes some time that makes me desperate, I'll definitely try this, thanks a lot

This was awesome !! Sooo interesting, straight up going into this

Your video was that amazing and well executed that I was hypnotized to hit subscribe and like. Keep it going.

Perfect video! Are you really writing that fast or do you talk over the fast-forwarded video? I would go for the second one but then I saw you going like "this one" or "that one" with the cursor and I am not sure anymore :)

Great work man! Can you kindly make a video showing where the numba takes more time than conventional python and where use of cython is more efficient than numba. Thank you.

Dude your great. Is that suckless window manager? You know I find videos all the time where they just aren't at the right level for me but this was just perfect pace, perfect insight. No fat just perfect. Reading other comments it seems im not alone as well. Thanks.

oh nice, you just gave me lots of work to do xDDDD time to change the whole project code... thanks :'D , no really i mean it THANK YOU it was a great vid will def look into that 10% for sure.

Great video mate. Really enjoyed it

Thank very much for your insights! Clear explanations and with examples (I run from documentations :p )

Wow thats hot, why am I just learning this now

Great video, thank you very much ! I hope that you will make numba tutorial for class too.

Thank you for this excellent presentation.

Man You saved my calculations

Bruh, you deserve all the bitcoins out there, this is so cool

Nice, easy to understand.

Okay, I'm hooked. Now i just need this for DDEs.

Love the video, I'll definitely have to try this on some future projects.

Thanks for the tip, let's try it!

Numba might just have fixed my biggest bottleneck on the computation for my master's thesis :)