Why I Switched from Python to Rust for AI Deployment

Рет қаралды 18,320

4 ай бұрын

In this video, I share my journey of deploying the same AI project with both Rust and Python, highlighting the challenges, successes, and performance differences. I also compare The performance difference between Polars and Pandas.
🤔 Key Points:
* Polars VS Pandas
* Speed comparison between Rust VS Python
* Scalability comparison between Rust VS Python

Пікірлер: 58

@zendr0 4 ай бұрын

So what's the conclusion? Is it ready for ML or datascience production systems

@codeinajiffy 4 ай бұрын

I wouldn't say so. Python still has more libraries for ML and is easier to develop. I would use Rust for only deployment and only in the condition that Python is too slow for your use case.

@zendr0 4 ай бұрын

@@codeinajiffy gotcha. Have you tried the candle crate by huggingface. They say it is Rust alternative to Pytorch. You should give it a shot. May be make a video on that if you like it.

@codeinajiffy 4 ай бұрын

@@zendr0 That sounds super interesting. I will consider it in my next videos if I find it to be a useful.

@berkes 2 ай бұрын

It is. But very much "it depends". Rust isn't just "faster to run" (less CPU and less memory, which the video dies not go into). It's above all stricter and thus protects you against many mistakes and bugs. Not all, but many. While that may even slow down a developer in the initial days or hours, it starts paying back within weeks. And seeing that we typically do some POC or exploration at first, Python (or even bash) is probably a much better fit there. But the moment we put something into production, make it a team project and/or start maintenance and continued development, the strictness of rust pays off. Often withind days. I typically cludge together a PoC in python. Once/if I'm happy with the direction and setup, I'll throw out that hodgepodge of python, bash/sed/awk, JavaScript and whatnot. And write it in Rust.

@julyort6762 2 ай бұрын

@@berkes Yeah so design in python and deploy in Rust seems really good

@Mn16Cr45 4 ай бұрын

Now you can switch from Rust to Mojo and make new video about why I switched

@jorge1869 2 ай бұрын

Agree❤

@houstonbova3136 Ай бұрын

I’m curious if in the Rust implementation of polars, did you use lazy execution or eager?

@codeinajiffy Ай бұрын

I used Lazy mostly to optimize the execution as much as possible. But yes you will need to understand lazy and eager to make this work fast.

@francismatsika3727 4 ай бұрын

I have been trying rust and the learning curve it steep . Unfortunately python has a huge ecosystem and u cant just jump it. Thats why i am looking carefully at Mojo which has similar memory management features to rust , the borrower checker and how referencing is done in rust. This can be a game changer. Mojo can bring in C perforance, Rust security and python versatility in one language

@kevinmcfarlane2752 13 күн бұрын

From what I can see, Mojo is going to clean up in this area in the long run. Rust isn’t going anywhere but it will clean up outside of AI and scientific computing tasks. Right now, F# is more mature for AI and can play nicely in the existing ecosystem.

@julyort6762 2 ай бұрын

Anyway polar can run also in 1 thread, maybe you were using only the default which is multi-thread and that's bad for non intensive request

2 ай бұрын

👏👍👍

@vectoralphaAI 4 ай бұрын

What about Mojo? The language that was literally designed for AI from the ground up? Plus Mojo's speed could be tested too.

@codeinajiffy 3 ай бұрын

Yup, I will look into that very soon. It's just a new language that might not yet be ready for production, but we can give it a try.

@mahakasem810 3 ай бұрын

@@codeinajiffy Their benchmark was recently revealed a lie after someone uncompleted the binary. Turns out something shady is happening in their compiler that doesn't match their claims. Big disappointment.

@houstonbova3136 Ай бұрын

Mojo is low-key trash.

@vectoralphaAI Ай бұрын

@@houstonbova3136 Its better than Rust in every way for AI though.

@houstonbova3136 Ай бұрын

@@vectoralphaAI Having done Data Science in a mix of professional and hobby for the last 5 years I’m going to have to hard disagree with that. I strongly prefer just using Python for exploration and then subbing out to raw Rust when I need gains. But, I also understand Rust. Mojo rarely matches its performance in practice and reduces my ability to create effective FFIs or directly manage the data when needed.

@Mbd3Bal7dod 4 ай бұрын

running multi workers for the api will solve the concrent request issue ,

@codeinajiffy 4 ай бұрын

Nice suggestion, but multi-workers here also increased resource consumption by quite a bit. It was quite close to increasing k8s pods.

@Singlton Ай бұрын

when trying to learn rust polars functionality , each time I run the code the compilation time takes around 10 seconds! any body knows how to reduce the time spent in compilation to make it ideal to learn and try polars functionality as a newbie ?

@1____-____1 Ай бұрын

Are you doing debug builds or release builds? Debug builds will be faster you can also set your debug builds to be incremental. You can also use sccache to cache compiler output. You could change the default linker or send -Z threads=8 to the rustic compiler. That is an unstable option but it will break up the front-end compilation of each package into 8 slices. But a 10 sec compile time isn't that bad considering the performance gains you get. Oh there is also cargo-watch, it will recompile your code when the file is saved. It also depends on how you are running/testing this code. Are you reading in your data from a file or stdin or are you hardcoding data in the source code?

@djthedadiofficial 3 ай бұрын

look at virtually any published benchmarking of pandas vs polars and polars is many times faster. Without seeing your code it's impossible to prove, but this was almost certainly a skill issue in terms of writing the rust. I am also extremely skeptical about your Golang conclusions, were you using goroutines? as for your last slide, you can spawn threads in Python too but, for all the newbies out there, alas you're unlikely to see almost any of these options. Much more likely to see something like Java, Kafka, Flink, or even Postgres with plugins. For applications that need super quick inference speed, very likely to see C or CPP

@martincontreras1521 3 ай бұрын

is there any git code link to check your recommendation project?

@codeinajiffy 3 ай бұрын

no github link because this was company code. I am thinking of doing another project that I can share.

@its_code 4 ай бұрын

Why not mojo language

@codeinajiffy 4 ай бұрын

I am considering trying it out next actually. It seems very interesting.

@its_code 4 ай бұрын

@@codeinajiffy 🥰🥰😝

@nurmohammed9642 3 ай бұрын

Biggest advantage of using rust is Zero-Cost-Abstraction, For example `sumation()` function could be write as: (1..=n).sum() and it would still out performed.

@johanngambolputty5351 4 ай бұрын

The only thing that was stopping me from moving from python to something like c/c++ before was ergonomics, not that c++ can't be concise, but a lot of libraries seem to require verbose interfacing. Rust, like python, seems to be very concise, so it offers a best of both, in brevity and performance, that I always thought were mutually exclusive. I was never under the illusion that python was efficient, it just had great syntax, which made prototyping a joy. It feels like I can prototype in rust same as I used to in python, and then get the proper implementation for free at the end. All that said, wasn't the GIL getting removed and couldn't you use multiprocessing.pool?

@codeinajiffy 4 ай бұрын

I didn't try multiprocessing.pool but that is a good suggestion. It might not be consistent between requests, but I will try it out.

@cloudguru3018 2 ай бұрын

Now you can switch to Mojo

@anon_y_mousse 3 ай бұрын

Well, I didn't learn anything from the video, but it did add yet another data point confirming what I already knew. Python makes it immensely difficult to write efficient code and requires external libraries written in better languages to achieve any degree of speed and general usefulness. Rust is basically a more obtuse C++ that nearly everyone in the future will use because the propaganda machine surrounding it is exceedingly effective. I know that a lot of people will claim that Rust is easier to learn, but since you can write your code in C++ exactly the same way as in Rust to achieve the same goal the same way and it will be equally as fast, the ease of use of C++ for simple projects shouldn't be understated. If you write idiomatic modern C++, it's as safe as Rust.

@mahakasem810 3 ай бұрын

Sad but true reality.

@Codotaku 2 ай бұрын

"it's as safe as Rust", sorry but that's an absurd statement if you knew what you're talking about

@anon_y_mousse 2 ай бұрын

@@Codotaku There were some qualifiers on that, but if you missed them and all the rest of what I said, then you must be a Rust user.

@1____-____1 Ай бұрын

@@anon_y_mousse Idiomatic C++ won't guarantee memory safety because it's unreasonable to expect devs not to make mistakes. Not to mention the safety you get out of the box with Rust, thits a fact. 100% idiomatic C++ 100% of the time?

@anon_y_mousse Ай бұрын

@@1____-____1 If you write idiomatic C++, then the compiler will catch most errors. The more insidious errors that aren't logic errors can be caught by external tools like the open source and often used `valgrind`, which catches everything the compiler misses and more than `rustc` will catch. If you had learned to program 20 years ago when they were still teaching people how to actually program instead of merely use a tool, then you would know these things. So I feel sorry for anyone learning how to use tools now instead of how to actually solve problems like an actual programmer, but we get what we allow to exist instead of what should because we've slacked off and not done our job for future generations. Sorry, my generation's intransigence in doing our duty for your generation has royally screwed you.

@enchantedshit 2 ай бұрын

please provide free resources for rust

@codeinajiffy 2 ай бұрын

I learned Rust from this video: kzfaq.info/get/bejne/r818ktuUxtzegWg.html . Then just jump on project and figure things out along the way. This should enough if you have some background in programming.