Have to tried different sizes of datasets, to see whether there is some underlying system cause... 230 million words is a lot, way more than I know... and it´s all in one file... and not very parallel...

what about Go ?

So rust essentially takes half the time than python....nice, but I thought rust would be a lot faster.

So at the end is Julia vs. C

This is the second time I have viewed this video. Thank you for performing the benchmark-testing work I would have had to do- saved me quite a bit of time. Now a question: Do you believe Mojo will progress to point where its dictionary performance will equal or exceed Python's?

That's a significant speed improvement, 3x folds faster in the newer version. However, it still doesn't explain why mojo code is still slower than actual identical python code, given that mojo was going for machine code compilation but with python syntax and ease, and not bytecode... For the little documentation overview i have read, mojo team explained that mojo is not python, but python will be mojo, in the sense that python will instead be an interpreted subset of the compiled mojo, and that features in python not yet implemented in mojo will dynamically switch to run in an included actual python runtime, but in the future, mojo will be self-contained to run all python code on mojo runtime.

How come you know that dunder methods provide high-level sugar for Python, but you call the methods directly in Mojo? You don't to call object.__len()__. object.__setitem()__ etc directly in Mojo, they work pretty much the same way they do in Python.

And ome could also try out the Dictionaries.jl package in Julia that is much more performant and efficient than he base Julia Dict type.

Please try splitting the data in half and assigning each halves to a dictionary. Then measure the time taken to copy or interchange elements from one dictionary to another. Maybe the problem is in the file management and not in dictionary!!

I wonder if PYPY version of python is even faster, since it does JIT compiler…. Would love to see that result

the real benefit of Mojo is it can easily target other hardware without having to write C code. Google's work with cuda acceleration and pandas is a good example.

hashmaps are made for O(1)ish retrieval, insertion is far less important. So yeah, insertion gets slower, especially if you haven't chosen a hashing method specific to your data (this can be really important), but the true test of a hashmap/dicitonary is how the size of the corpus affects retrieval. It would be interesting to run the experment on the retrieval end, now that you know the performance of insertion, ansd see how they compare. (You may also want to see how pickling the dictionary and restoring it performs, since this is done a lot more than generatign the initial map in research.) Also, for initial trials, maybe speed things up and get a rougher idea by increasing the quantity by orders of magnitude - 10, 100, 1000, etc. A sparser plot some times helps things jump out of the data more clearly. Thank you for the post, it was good fun!

I think that "small" waa about 10-50 keys as most objects in everyday python are actually these tiny dicts.

Mojo official benchmark with python performance needs a comprehensive "mojo feature to python feature" comparison benchmark... We would like then to see where the 35,000X speed up claim came from...

my test with the List also shows mojo is slower than python, not sure if I did anything wrong.

The curve really doesn’t have a name, but it is an increasing function tending towards being linear. It is not exponential or parabolic, they refer to specific kinds of curves. The reason behind this is due to the fact python dictionaries are implemented as hashmaps in c, which when empty take roughly a constant, very small amount of time to access, but when fuller take longer and longer proportional to how many items are already inside it.

Hey there! I think the stuff you make is helpful and you should keep doing it :) That being said though, you'll always have this problem with dictionaries (and most other data structures). Dealing with LOTS of memory is going to become a performance problem since, most likely, some of this memory is going to have to be stored in disk and disk reads are very. Also resizes of big data structures are particularly slow. This btw could explain some of the "outliers" that you have. I just wanna point this out because maybe you were under the impression that this performance decrease as the dict gets bigger is because of the optimizations for small-sized dictionaries. It might play a part (haven't looked at the implementation) but it should be something minor.

Nice video. I think that "exponential" was the right word to use to describe the increasing rate.

can R do a quadratic "regression" to bound the data set from below?

Hey, do you have the code for this test somewhere publicly visible, like Github? I would like to see (in a bit more detail) what was benchmarked here.

Python dictionaries are essentially a hash map and naturally have a limited index range for the hashes. More items means that are more chances of collision and having to do a linear search on the index.

loved it

Great video man 👍👍

This will be an issue for us. Thanks for research!

Maybe in Mojo the Strings are copied and reallocated as new objects, maybe because of the different StringKey type, or maybe (I am not as familiar with Mojo) because Dict stores and owns an own instance of the string, similar to as in C++. There the string and the memory associated with the string is "owned" by e.g. the unordered_map. Maybe you should retry measuring the time, now inserting floating point numbers for examples, that might not be allocated in Mojo.

Sir,can you share a quick tutorial on how to create a model for source seperation using speechbrain in google colab from scratch?Please sir!!!

Sir,I do have a doubt regarding this project and it is kinda urgent .There is this command-cd recipes/WSJ0Mix/separation python train.py hparams/sepformer.yaml --data_folder=your_data_folder...what is data_folder here..what exactly should i put in data_folder?Sir,I need your help.

Parabolic is not exponential. I would not use the exponential term so nonchalantly

You should not do scripts in Julia. (or anything else). You pay the compilation time at each run.

Thank you so much. Really helpful explanation of the statistical measures! Would you consider a video on the key word measures!

nice video :)

You're not supposed to call those dunder methods directly like that lol. __contains__ is called by the language is you do: if wd[] in freqs. __len__ is called by the language if you do len(wds). __getitem__ is called by the language if you do: freqs[wd[]+1] They're used to design library APIs which fit orthogonally into the core of the language

nice observation.

thank u for sharing this, i wish if the audio quality was better after the vocals are seperated tho

I tried everything seems like julia don't know the type of the values in eachmatch and that's causing recompilation (around 10% of the time is taken from there), I think it would be great to change it to use vector of char preallocated to avoid allocations and try to avoid regex match even if it means making loop over char as long as it compiles to machine code it will be fine

Thank you! It is helpful~

Hi @EKB PhD, thanks for sharing! I am trying to use whisper_timestamped to extract word level timestamp from singing, so I can create karaoke from the result. However, the transcribe quality went from almost perfect to completely useless when I switched from using stable_whisper to whisper_timestamped. Same audio, but using the "large" model. Any suggestions or insights why this would happened? Also, given lyrics is always available, I wonder if there are other models that would create timestamp from a audio, given the full lyrics (which I assumed the timestamp would be much more accurate, since there is no guesswork on the content of the audio, but only when a word start/end in the audio. THANKS!!!

nice thanks

lmao up until minute 9 i was expecting that you were going to explain it ^^

If you do this little change in your julia code, the runtime would be even faster: wds = String.(split(txt, " ")). This will eliminate the allocations happening inside the loop, cause SubStrings were converted to Strings inside the loop. Now we do this outside of the loop. I also recommend benchmarking with the @time macro.

The plot you showed at the end of the video was cut off by your camera. We couldn't see what the results were for the 1.11.0 run

They recently announced opensource their product except LLVM (Apache License), so it should get much better despite my bad impressions on some of the open source authors in the past.

please could you make a video teaching me how to use this program, I'm confused but I wanted to use it, where do I send my voice to clone?

So, Julia is fast (optimized) for numerical analysis, but not so much for other problems.

Julia being slower is a surprise- given all the hype.

my guess would be that the mojo implementation is a quick lazy thing just to get maps in, while python uses a C implementation.

Python actually has a realy optimized dictionary for its constraints (taking untyped keys and values). Mojo will catch up (and should surpass due to static types), but the current dict is new and mostly unoptimized. Also, this should be memory bound instead of compute bound so even in the long term, I would not expect much of a difference.

Now using mojo just a Python skin language, everything you want to do must import Python package…

Excellent! What about Rust?

In the discord channel I read the current dictionary implementation is not optimized. It's been included to allow developers to experiment more with the language, but it will be optimized in future releases.