1 billion row challenge in Rust using Apache Arrow

Рет қаралды 4,518

Күн бұрын

I walk through how I did the One Billion Row challenge using Rust and the Apache Arrow DataFusion query engine.
Code: github.com/JosiahParry/youtub...
DataFusion: docs.rs/datafusion
1brc: github.com/gunnarmorling/1brc
Follow me on twitter: / josiahparry

Пікірлер: 11

@porlando12 Ай бұрын

R dev here trying to broaden my horizons and wow I'm blown away with the performance! Keep making these!

@richardbennett4365 Ай бұрын

Nice video, and this man has an accent similar to Will of SpaceRex. I like the presentation. The only thing i caught at the end was again a young man in the field of computer science, the third one (and one is from the UK so its not just North American English), who doesn't use the past participle form with have. He said "I've ran" instead of "I've run." It seems to be a phenomenon among younger folks, and I wonder if this grammatical form is just not taught in schools anymore, or is it how people, like his parents or friends are speaking in this region of the world. But, i have noticed Elliott Minns of another KZfaq channel (@Dreams Of Code) also using the past tense form of the verb with an auxiliary and not the past participle form of the verb. But, it's the message and content that matters. Good job.

@blaisepascal3905 Ай бұрын

Thank you for this nice video! What plugin do you use in VS Code to have the option of running/debugging just above the main function?

@josiahparry Ай бұрын

It's just the rust-analyzer! I have it mapped to shift+cmd+enter my keybindings.json file though { "key": "shift+cmd+enter", "command": "rust-analyzer.run" }

@blaisepascal3905 Ай бұрын

@@josiahparry Thanks!

@joshstat8114 Ай бұрын

I was wondering if there is a linear algebra library in extendr (Rust), like RcppArmadillo

@josiahparry Ай бұрын

check out faer! faer-rs.github.io/

@richardbennett4365 Ай бұрын

Nice brief code, but on my machine I'm getting an error with the pretty: implementation Display statement.

@richardbennett4365 Ай бұрын

I got the code debugged. I just have a Code issue where the buffer for the lines of the Terminal needs increasing, and I think I would also like to `tee` the output to file as well as STDOUT.

@josiahparry Ай бұрын

ah so this was an editor issue?