How much faster is Rust than Python when finding neighboring words?
Рет қаралды 904
Күн бұрынRust is a badboy systems language, so I assume it's faster than Python when calculating the word association metric log dice. But, how much faster is it with this particular linguistic task?
Here's my Rust script (with generous help from my son):
github.com/ekbrown/scripting_...
Here's my Python script:
github.com/ekbrown/scripting_...
#corpuslinguistics #rustlang #python
@NoX-512 9 күн бұрын
If you convert the text into an array of integers, where each integer is an index into an array (or tree) of unique words from the text, you could possibly speed up things by a lot, depending on how long it takes to set up the arrays/tree.
@ekbphd3200 6 күн бұрын
Very good idea! I'll have to try this.
@patates1165 2 ай бұрын
nice video :)
@ekbphd3200 2 ай бұрын
Thanks!
@abanoubha 24 күн бұрын
what about Go ?
@ekbphd3200 20 күн бұрын
I haven't yet ventured into Go for text processing.
@techinsider3611 18 күн бұрын
Also try mojo.
@ekbphd3200 17 күн бұрын
Yeah, I need to try Mojo too. I'm finding that Mojo isn't yet good at text processing. I hope and assume that it will be get better as it is developed more and more.
@kilianklaiber6367 24 күн бұрын
So rust essentially takes half the time than python....nice, but I thought rust would be a lot faster.
@ekbphd3200 20 күн бұрын
Yeah. Nearly twice as fast.
@0xedb 16 күн бұрын
could be a lot faster. it all depends on what's being done and how efficient the code is. not always tho
@JavierHarford 2 күн бұрын
I can imagine 2x is just a function of the complexity x the sample size, which makes me wonder about the curve at scale. There would also be some unrelated but important measures, such as speed of development and the effect of higher level abstractions vs lower level optimisation too