Why I Switched from Python to Go Lang for AI Deployment

Рет қаралды 50,357

5 ай бұрын

In this video, I share my journey of deploying the same AI project with both Go and Python, highlighting the challenges, successes, and performance differences.
🤔 Takeaways:
* Go excels in response times, scalability, and resource management.
* Some data science libraries might be less efficient in Go compared to Python.
* Consider switching to Go for performance gains when Python hits traffic limits.
🚨 Important Note:
While Go may not cover all data science libraries, its strengths shine in specific scenarios. Explore the performance gains and make a strategic decision based on your project's requirements.

Пікірлер: 137

@andrewzielinski399 4 ай бұрын

This was a fantastic video. Thanks for posting

@newmoodclown 4 ай бұрын

Damn that's one great video, thanks a ton.

@codeinajiffy 4 ай бұрын

Thanks a lot 🤩. Those words mean a lot for me 😁

@electricz3045 4 ай бұрын

Another reason for golang is that it's really picky for code design. For example if you have an if else loop, you need to put the else behind the closing curly barsckt from the if stement or it's like you have an else statement without an if statement. You can't have an empty line at the bottom of a list, can't have variables which are not used etc. Another nice thing is that golang's error handler is way better and give more details about the error before you can even run it.

@salehmir9205 4 ай бұрын

this was a very practical and a real use case you made some really good points about when to use Python and when to use Golang thank you man

@codeinajiffy 4 ай бұрын

Thanks a lot for the support 😁

@frazuppi4897 4 ай бұрын

amazing video man!

@SomebodyYouUsedToSee 5 ай бұрын

NIce Explanation . Thanks for the vide

@ankitsanghi 5 ай бұрын

“I wanted to dump my spaghetti code aside” lmao so relatable

@md.jannatultasnim2661 5 ай бұрын

Most of the developers do not know how to use python efficiently

@ankit.chaurasia 5 ай бұрын

The title was misleading but the content was on point. Python is slow bydefault. Numpy and pandas are fast cuz of C language used as wrapper. If you write native code your performance reduces so much

@fieldmojo5304 5 ай бұрын

That sounds like a Python problem. Shouldn't most developers know how to use it correctly?

@sebas11tian 5 ай бұрын

@@fieldmojo5304 My thoughts exactly. The languages itself should have a small gap between possible and efficient code; otherwise, the default is to start thinking about optimizing code from rhe start, which creates overhead to the effort of writing even a simple implementation.

@envynoir 4 ай бұрын

Most of the developers dont know how to code properly

@marcotroster8247 4 ай бұрын

@@envynoirFinally someone said it 😂😂😂

@Incertophile 5 ай бұрын

Very cool video, nice job and thank you for sharing.

@DifferencialX 5 ай бұрын

Cool video, did you tried Mojo language? I'm just currios if it is faster than python.

@codeinajiffy 5 ай бұрын

No not yet, but it's on my bucket list to try it out 😍

@thegrumpydeveloper 5 ай бұрын

Great speed increase.Did the ai inference use gpu? Would be interested to hear about the multi armed bandit recommendation engine.

@codeinajiffy 5 ай бұрын

It was only using CPU. I can make a video explaining multi-armed bandits if it will help you out 😃

@fojico1234 5 ай бұрын

⁠would be really nice to explain implementation concepts, thank you.

@rembautimes8808 5 ай бұрын

Excellent video , as data volumes increase so will the demand for more performant solutions

@TheHolyRolle 5 ай бұрын

Currently learning python thinking about learning go next!

@dalton4035 5 ай бұрын

Cool video, and I'm also a big fan of go. Did you consider rewriting the problem code as a C extension for Python, or was that not viable for some reason?

@codeinajiffy 5 ай бұрын

That's a cool option. At the time I didn't think of that, but am intrested to try that out.

@greyshopleskin2315 5 ай бұрын

@@codeinajiffyalso, the slow python version couldn’t be implemented using numpy, pandas or something like that? I’m not a DS guy, and I don’t know what that function was doing. But things like deduplication should be doable with pandas (?). Also, maybe the problem was a poorly optimized implementation. Python is slow, but these numbers surprises me.

@codeinajiffy 5 ай бұрын

@@greyshopleskin2315 yes general removal of duplicate rows is doable in pandas, but this use case was conditional over the previous rows and I had to loop over the rows here. Also the loop did a couple more things that were not easily done with pandas operations. I spent some time trying to optimize the code, because I didn't want to rewrite the whole code base again, but got no major speed increases and thus I went with go

@fallout__boy1130 5 ай бұрын

Most likely the bottelneck would be in the python code itself not in pandas or numpy (which are supposed to be well optimized).

@fabianwachmann9153 5 ай бұрын

Nice Comparison - thanks 🙂 did you consider julia (programming language) for prototyping? It could be fast enough for inference and can access the ML Libraries...

@codeinajiffy 5 ай бұрын

not yet, but It's on my agenda of things to try out.

@InfiniteQuest86 5 ай бұрын

Yeah definitely is by far the superior option. I tried importing a file in Pandas, and it took 6 minutes. Julia's DataFrames did it in under 10 seconds. Julia for ML is the only real option. If you want a language like Go, you should use Odin. It's like Go but better in just about every way.

@pranaydas7382 4 ай бұрын

I have a question, like how did you make predictions in real time without being the lib's available for go ? as there are for python such as scikit etc ?

@codeinajiffy 4 ай бұрын

Great question, It's a multi-armed bandit approach so all I needed to do here was to sample a distribution using the numpy go alternative. But there is sklearn in go language if you want to use it, but note that it doesn't have as many models as the python implementation. There is Tensofrlow and Pytorch in go lang as well.

@RootsterAnon 4 ай бұрын

This was very informational. Thank you for sharing your observations and conclusions. Is there a public github repo that we can check?

@codeinajiffy 4 ай бұрын

This is company code so I am not allowed to share it, but I am creating a new video with python vs Rust with the same use case and I am thinking of putting some code there.

@ANONAAAAAAAAA 5 ай бұрын

It's maybe a good idea to start from instrumenting APMs or running benchmarks to identify the performance bottlenecks, as Rob Pike famously said in his 5 Rules of Programming. In general, Go excels when your application is mainly IO bound. In the context of machine learning, Python is actually a wrapper language to call scientific calculation libraries written by C and Fortran. These libraries are developed and tuned-up for decades and Go (lol) has literally zero chance to match up with them.

@kamilziemian995 5 ай бұрын

Very well said.

@bebobauomy1265 5 ай бұрын

Can not he just use these C libraries in Go instead of Python?

@dragonova69 5 ай бұрын

These libraries are built for python only(you can find vice-versa in other languages but it's limited)! The main advantage of python is it's so fking popular that the libraries support is very vast. Also you can't swap support libraries b/w different languages...@@bebobauomy1265

@grimonce 5 ай бұрын

@@bebobauomy1265 would need to write and maintain a wrapper himself. They're written in cython not python or C... (at least many parts of them). But yea, just not by himself in this "tight" schedule.

@eptic-c 4 ай бұрын

If they were written in C then you would just write a CGO wrapper and you're done with the job, but, they are not.

@seasong7655 4 ай бұрын

I had a similar experience, when I tried to switch python to nim. I've found out nim does not have the same amount of libraries as python and not everything is available in nim.

@Queracus 5 ай бұрын

did you consider polars in python. seems like it would do the trick replacing pandas and numpy

@codeinajiffy 5 ай бұрын

I will give that a go 👍. Thanks 🙏

@AlexanderBorshak 5 ай бұрын

So what the actual _AI_ code has been written on the Go (or Python) in your project? Rearrange a list, sum it and handle endpoint looks liike no-AI part.

@codeinajiffy 5 ай бұрын

I only moved the deployment part to go lang, but preparing the multi-armed bandit recommender per user was in python. I then read the multi-armed bandits in go lang and predicted the scores in go lang with the numpy alternative.

@JulianHarris 2 ай бұрын

I find this very fascinating indeed. Given how much C/C++ optimisation has been done in the popular libraries like numpy, pandas etc I'm very curious what was the key bottleneck that caused the massive (30x?) increase in number of pods required? Sounds like there were some inner loops that were running solely in python.

@Summersault666 5 ай бұрын

Does your Go program runs in the GPU like python does?

@codeinajiffy 5 ай бұрын

I was only using CPUs on this project, but it would be interesting to see a GPU comparison.

@srivatsa1193 5 ай бұрын

Awesome! More proof that model inference requires more efficient implementations. What do you think about rust in this context ? Given that there are libraries such as polars and arrow, would implementation effort in rust be worth the performance gains if any at all ?

@codeinajiffy 5 ай бұрын

Rust is on my bucket list of things to try out. I saw a couple articles about Polars being faster than pandas. If that's true that would be very exciting.

@hendrikmehrtens3138 5 ай бұрын

Maybe you should also give Julia a try. It's written as a dynamic, high level language like python, but can run with the speed of a a low-level language due to JIT compilation during runtime. It was specifically build for scientific computing and data sciene and has a really fun programming pattern based on multiple dispatch, which you should definitly check out. You have access to all data science tools, including tensor/deep learning libraries and trivial GPU computing. The core idea of julia was to bridge this gap, between fast and easy to code languages like python and low-level languages, so that you don't need to rewrite your finished project again for deployment. Have fun :)

@andredasilva9458 5 ай бұрын

I do use Julia for scientific programming, but I'm not sure about using it for creating servers and APIs (based on me never even trying to use it for these purposes).

@derzasekolahdidd 5 ай бұрын

So, what was the cause? Was it because of the python, the fast api, or the pandas and numpy?

@codeinajiffy 5 ай бұрын

The main cause of slowness was the only function in the code base that did not use pandas and numpy and used a normal for loop. This was slowest part of the code. Also Python's concurrency was bad and it affected scalability a lot, requiring a big number of resources to scale to all the user base. On the other hand pandas and numpy were actually efficient and fast.

@bastabey2652 5 ай бұрын

nice video.. Python ecosystem is very rich.. I hope Mojo delivers

@mymoviemania1 5 ай бұрын

Nice job

@hiddenworld1445 4 ай бұрын

What should we do to optimize things, there are so many things to consider 😢

@KrishnanshAgarwal 4 ай бұрын

how can panda(c-written) be slow than your custom go implement ?

@JonitoFischer 4 ай бұрын

How can he be sure that his custom code will be more reliable than the already tested and debugged Pandas library? How are we sure that his Python code could not be improved any more?

@homieinthesky8919 4 ай бұрын

Have you tried implementing numpy and pandas with numba b4 finishing the tests. You should see a much higher speedup especially with parallel computation, math, loops etc just by using vectorization or jit with nogil, caching and static types. Jax is also a good alternative to get a faster speed up in python but that mainly replaces numba. I personally have used jax ports of some ai libraries and seen huge training improvements.

@codeinajiffy 4 ай бұрын

Yeah I actually did. There were two modes one is called object mode which improved speed by less 15%. And another one called nopython mode. In this mode Numba aims to generate machine code that relies solely on Numba-supported operations and data types, avoiding the use of Python objects and the Python C API. This one was way faster than normal python indeed, but the main problem is that it required all data types to be supported in numba. That meant no pandas data frames and other useful tools which made writing code restricting and more time consuming than writing with go lang.

@tacorevenge87 4 ай бұрын

Numbs nor pypy would never match go’s speed

@SzaboB33 4 ай бұрын

That was a good story :D

@flaviovitoriano2429 4 ай бұрын

Hey nice video! Usually python IA libs are not written in python, but in rust/c/c++. The people use python just for the "easy" part. Try the mojo language, will be nice to see the diff between mojo x python x go

@codeinajiffy 4 ай бұрын

I'll give it a go in the upcoming days.

@darkkuja22 5 ай бұрын

why show sklearn, torch, and tf if only numpy and pandas were used? what are the torch, sk learn alternative in go?

@codeinajiffy 5 ай бұрын

I was just making a statement that maybe not all data science libraries will be found go lang. There is tensorflow in go lang with the same package name. and sklearn can also be found in go lang but note that it will have a very limited number of models and functions. It's not as comprehensive as python's library.

@darkkuja22 5 ай бұрын

@@codeinajiffy that makes sense thanks! Great video btw keep it up

@viki_sky007 5 ай бұрын

Brother can u show us simple example on How u used Structs Maps and various ways to make an alternative for pandas. Loved your perspective and insights keep doing.

@codeinajiffy 5 ай бұрын

Sure here is some sudo code for joining two data frames. df1 = { 'item_id_1': struct({ 'item_name': ..., item_tags:....score:}), 'item_id_2': struct({ 'item_name': ..., item_tags:....score:}), } df2 = { 'item_id_1': struct({ 'prediction_score': 0.35}), 'item_id_2': struct({ 'prediction_score': 0.45}) } for item_id, _ in df1.items(): score = df2[item_id]['prediction_score'] df2[item_id]['score'] = score and now you have the scores of df2 in df1. effectively merging the two data sources. And because those are maps the lookup time in df2 is bigO(1).

@kryffon 5 ай бұрын

Did you try numba? It is supposed to make normal python code faster

@codeinajiffy 5 ай бұрын

@pepperparkffm 3 ай бұрын

Why I Switched from Go Lang to Mojo (Python) for AI Deployment... ^^

@LuisFernandoGaido 5 ай бұрын

What about train in python, export the model to file and deploy for Go read it?

@codeinajiffy 5 ай бұрын

Yup that's what I did. Only the deployment was in Go but preparing the multi--armed bandits was in in python

@LuisFernandoGaido 5 ай бұрын

@@codeinajiffy Nice!

@maxkosh4839 2 ай бұрын

so how you could switch to GO for AI development if it has no tensorflow or scikit???

@ennwy 2 ай бұрын

but it has ahahah

@kingthame 2 ай бұрын

This is what I've been looking for. I love using Golang but I also love ML

@climentea 5 ай бұрын

Maybe Polaris instead of Pandas could've help. But, Golang rocks too.

@breakablec 5 ай бұрын

Have you tried profile the python code to find what was the bottleneck?

@codeinajiffy 5 ай бұрын

yup it was the piece of code that didn't have pandas and numpy. This part was building the discovery screen row by row in a for loop and this was the slowest part here.

@breakablec 5 ай бұрын

@@codeinajiffy it sounds to me that you could find a way to optimise this rather than rewriting

@runaway09 5 ай бұрын

@@breakablecwell loops in python are known to be slow. cannot optimize the runtime of a dynamically typed language that much

@skillfulactor09 5 ай бұрын

Did you try rust ? The performance is great.

@codeinajiffy 5 ай бұрын

not yet, but It's on my agenda of things to try out.

@LukeDickerson1993 4 ай бұрын

lol why are go langs data packages slower than structs, maps and loops? I'm a python dev mostly. Strongly considering Go after this video!

@kamilziemian995 5 ай бұрын

Go is much funnier to use that Python, even when Python is very good on its own.

@Luix 5 ай бұрын

the problem of Go is that the ML libraries are not as good as python (because python is all C++ behind the scenes)

@Cordic45 5 ай бұрын

Isn't C?

@Luix 5 ай бұрын

@@Cordic45 the code behind the ML libraries on python?, I think is CPP.

@Cordic45 5 ай бұрын

@@Luix if u mean tensorflow and pytorch, yes

@seven1_studio 5 ай бұрын

Make video how to use python efficiently

@pixel1145 5 ай бұрын

What about Mojo? It's not finished yet, but its very promising.

@codeinajiffy 5 ай бұрын

yeah it looks promising. It's on my bucket list of things to try out 👍

@user-oq2vq9co7e 5 ай бұрын

Sir, In future, golang has the potential to replace python.

@user-oq2vq9co7e 5 ай бұрын

Can you share your point of view?

@BosonCollider 4 ай бұрын

I'd use DuckDB instead of dataframes imo. Faster than pandas when using it from python, but is also available in Go and makes it easy to port code between the two.

@codeinajiffy 4 ай бұрын

Sounds interesting, I'll give it a go. Thanks for the tip

@kunaldhuria3935 5 ай бұрын

what about node express server with ngnix

@codeinajiffy 5 ай бұрын

I did not try that, but I might try it out in the near future.

@hilmarhaukur5736 5 ай бұрын

Why not just write that slow Python bit in C++ and then call it from your Python code using pybind?

@codeinajiffy 5 ай бұрын

I didn't think of that, but It sounds like a good idea to try out.

@davidagyakwa288 5 ай бұрын

I would wish you submit request to the developers of that package

@codeinajiffy 5 ай бұрын

I am planning to do so

@naeembilal 5 ай бұрын

Yes, Go is a wonderful work in progress.

@skobanemusic5752 25 күн бұрын

lol regardless of the result of the video. i felt that Austin power joke lol inlaughed and they laughed

@grimonce 5 ай бұрын

This is false, golang just automagically schedules threads and coroutines with a cool scheduler. So this guy probably blocked the eventloop and then wondered why the response time didn't scale with the users. Not saying it was wrong to rewrite to go, because in the end it was easier to get the performance you need. Python has a lot of gotchas....

@nikolamar 2 ай бұрын

mojo maybe

@user-tl4sc9wz4r 5 ай бұрын

Why not to rust?

@codeinajiffy 5 ай бұрын

I wanted to use a language already being used in the company, but I am considering to try out soon.

@abdessalemmohellebi 4 ай бұрын

Goatlang🗿

@vectoralphaAI 4 ай бұрын

You need to do it in Mojo. Its aim is to become the default language for all things AI and ML.

@Little-bird-told-me 5 ай бұрын

Only learn GO if you have to other wise python is plenty

@BosonCollider 4 ай бұрын

Learning Go is not difficult though, an experienced programmer can become productive with it in a weekend so the investment needed is not that big. But the ops team will generally like you a lot more if you give them a go binary