Parse JSON with Rust & Serde
2:51
Valve: R belongs in production
8:26
S3 Objects and Functions in R
16:28
RAW STRINGS IN R?!
1:19
Жыл бұрын
Making R 300x times faster!
21:28
Жыл бұрын
SLICED! Live Machine Learning
2:06:02
3 жыл бұрын
Creating a t test with R
7:10
3 жыл бұрын
ANOVA in R
7:29
3 жыл бұрын
Making your R Markdown Pretty
8:34
3 жыл бұрын
Пікірлер
@vacon04
@vacon04 6 күн бұрын
Hey thank you for the video. I saw that you also already have the docker instructions in the package website. I'm hoping to run a simple app with plumber in production in the next few weeks. I was going to just use caddy as a load balancer for multiple plumber instances but I think using Valve should provide better functionality out of the box. Thanks again!
@josiahparry
@josiahparry 6 күн бұрын
Just make sure that your Docker container has enough vCPU to handle multiple threads and ensure that you don't over-allocate them as well! And you can also use a load balancer in front of multiple Valve apps
@vacon04
@vacon04 6 күн бұрын
@@josiahparry thanks! I have a cloud instance with 8 vcpus so it should be good. I'm not expecting massive traffic either and the computations that R has to do are very quick. I just want to make the UX as good as possible. I think 4 workers max should be good enough for my purposes. In case more is needed well that's the beauty of your package, I can just reload with more workers and if should be good to go =)
@rafabws
@rafabws 9 күн бұрын
That was one of the best video tutorials I've seen lately, super informative and easy to follow! I cannot wait to test this out with some different datasets. Thanks for the amazing work!
@jiojio97
@jiojio97 10 күн бұрын
Hi, i'm trying to implementing the algorithm in java and following the same steps i got the same results, so i guess i'm reimplementing it correctly. The only problem is, if i try to use any lat/long coordinates for example, my home adddress lat/long, got a geohash that is totally far away from the addres used. In fact if i use some online tool the hash generated is totally different and correct.
@josiahparry
@josiahparry 9 күн бұрын
You got the order wrong! Its long lat! Classic mistake. :) longitude is the x axis and latitude is the y axis
@mukulbhave9376
@mukulbhave9376 21 күн бұрын
super clear ,easy and concise walkthrough. Amazing
@mitchbet
@mitchbet 25 күн бұрын
I'm curious how this works. Is it loading the whole file into memory before doing the aggregation? Or the query planner smart enough that it knows it only needs to load 1 line at a time (and then handles keeps track of the required aggregations accordingly)
@kamertonaudiophileplayer847
@kamertonaudiophileplayer847 Ай бұрын
It isn't fair.
@manzyzuzajnr3508
@manzyzuzajnr3508 Ай бұрын
There a chance to show us how to do species distribution modelling using Biomod2 or SDM package
@NicholasTierney
@NicholasTierney Ай бұрын
Great video, Josiah! For benchmarking, just a fun FYI in case you didn't know, you can do `summary(benchmark_obj, relative = TRUE)` and it will compare the two in relative terms, which is neat!
@brentonbills5946
@brentonbills5946 Ай бұрын
Wish i knew more R 😅. Looks useful for helping out the R community with rust 🙂
@brentonbills5946
@brentonbills5946 Ай бұрын
Nice library 🎉.
@BrianPondiGeoGeek
@BrianPondiGeoGeek Ай бұрын
Clear explanation.
@superslash7254
@superslash7254 Ай бұрын
What it boils down to is that R is at its core a fully featured programming language. To some degree it will always be inherently unsafe, as will any other programming language, simply because by nature they have to be in order to function. Much like a good quality knife is also able to cut you as well as your dinner. There are some shenanigans at play here, but it still pales in comparison to the likes of Java's Fracturiser hidden in minecraft mods or malicious executables.
@davidbosak7503
@davidbosak7503 Ай бұрын
This whole thing seems suspicious to me. A contract threat research company sets out to find a way to exploit R, and then, (Wa-La!) finds it! Then advertises it like crazy! And in the write-up they specifically reference the usage of R in the Pharmaceutical industry, and even links to a talk by R/Pharma. This whole thing feels like a set up. Was HiddenLayer contracted to do this? By who? Very suspicious that a *potential* exploit got as much coverage as way more damaging *actual* exploits.
@superslash7254
@superslash7254 Ай бұрын
Especially when the exploit basically boils down to blindly running completely unknown code. This is the coding version of downloading random executables from the internet and running them as administrator. Or to use a physical metaphor it's like saying household appliances are unsafe because someone told you to stick a fork in the socket and you did it. P.s. "Voila". Because the french were so rich they had to use extra letters just to show off.
@joshstat8114
@joshstat8114 Ай бұрын
I emailed Hadley and he said, you don't have to update R to 4.4.0. Someone in reddit said, it is no use to update R if you use RDS/RDA files from unknown source. Also, for a follow up question, is it possible to combine C, FORTRAN, C++ and Rust code together in one package? I tried to combine C++ and Rust but failed to debugged my R package but I know there's a certain package that uses C and C++ together (EBImage as an example).
@josiahparry
@josiahparry Ай бұрын
You can use all of them together it just requires extra work on your behalf to make sure they dont conflict with eachother. These tools aren't designed to work out of the box with eachother.
@joshstat8114
@joshstat8114 Ай бұрын
@@josiahparry yeah you're right. I tried both C and C++ and in order to be working, you need to call the C code from C++. And yes, it is such a pain.
@skeleton_craftGaming
@skeleton_craftGaming Ай бұрын
What is R used for? Like with syntax like that I understand why it is used [That might be the most beautiful code I've ever seen] But for what like what is its practical applications?
@josiahparry
@josiahparry Ай бұрын
Biomedical research, econometrics, data engineering, machine learning, big geospatial analysis, interactive dashboards, nlp, web scraping, idk whatever you want to do you can do
@superslash7254
@superslash7254 Ай бұрын
R's biggest strength is data engineering. Cleaning, analyzing, and reporting tabular data can be done with a fraction of the code and a fraction of the runtime in R compared to pretty much any other language. Data.table is several times faster than Pandas, and Collapse is even faster still. The nature of the interactive console also makes it incredibly easy to really get hands-on with the data step by step before cementing an optimized pipeline.
@skeleton_craftGaming
@skeleton_craftGaming Ай бұрын
@@superslash7254 hmm.. like I said that is some beautiful syntax
@JasonMitchellofcompsci
@JasonMitchellofcompsci Ай бұрын
I don't get how it is a huge vulnerability. When you download code from the internet it can run code. cmd() or similar is a common resource in nearly every programming language. There is nearly no scripted programming language where a downloaded module couldn't do that. I guess it might better let them obfuscate because it is calling it indirectly? In general you should assume all downloaded code can run anything within userland within the same user on the machine. Launching a calculator is just something code can do. Any module in any programming language could probably delete all your files or send them off to some server.
@josiahparry
@josiahparry Ай бұрын
I think the idea is that these are data formats and you might think you're loading data but actually you're loading a promise to unexecuted code. Then that code is executed instead of being a piece of data already. It feels like a feature not a bug but alas, you shouldn't need to have a binary file to store unexecuted code.
Ай бұрын
1:26 I can confirm that note LOL
@josiahparry
@josiahparry Ай бұрын
Can confirm. You smell good.
@vlemvlemvlem3659
@vlemvlemvlem3659 Ай бұрын
I'll be sure to never share this with my legal/sec department. They'd freak out at your blatant use of the R-word (whispers 'risk'). On the other hand, I've been able to take away their dread by filling out copious forms in the past so should they be exposed to your scary talk I know what to do. After all, everyone knows killer-bureaucracy is what truly keeps our corporate networks safe
@josiahparry
@josiahparry Ай бұрын
LOL!! I love this so much. Yeah....I'm sure they're allowing far scarier things than **manually** inspected R packages from CRAN. And, as always, its not the language that's dangerous...it's the developers. You can allow SQL injections using just about any language / tool out there. It is up to the developer to prevent them.
@rising1underground
@rising1underground Ай бұрын
Thanks for the content. As an R user i don't usually think about this but very informative context!
@josiahparry
@josiahparry Ай бұрын
<3
@pizzaprosciuttofunghi
@pizzaprosciuttofunghi Ай бұрын
Very interesting!
@pizzaprosciuttofunghi
@pizzaprosciuttofunghi Ай бұрын
Very interesting!
@spacelem
@spacelem Ай бұрын
I do write plenty of for loops in R, but they're typically in places where either the speed really doesn't matter (e.g. `for (i in vars) ggplot(...)`) or where there's no way to vectorise the body (e.g. in stochastic simulations where there's a load of stuff going on and the loop depends on what happened in the previous loop, and I'm not that good at using `reduce()`). Typically I do what's easiest to read, and usually when something can be vectorised, it's easier to read that way anyway. If I need it to be fast, I have Julia, which says for loops are good actually since all the code is JITted, although Julia makes vectorising code trivial with its dot syntax (e.g. `log.(x)`).
@pizzaprosciuttofunghi
@pizzaprosciuttofunghi Ай бұрын
This is so cool!
@user-hl1fk6lw8k
@user-hl1fk6lw8k Ай бұрын
Thanks for the intuitive tutorial. But I would like to point out that str::len() method counts bytes instead of characters. Non-alphanumerical characters will return inappropriate results, i.e. nchar2("你好世界") (Hello World in Chinese) returns 12 instead of 4. Interestingly, NA values are 2 bytes. One should use str::chars().count() to count at a higher computation cost than str::len().
@vlemvlemvlem3659
@vlemvlemvlem3659 Ай бұрын
I'm currently in an R youtube rabbit hole and this one has been an absolute joy. I tip my fedora to your excellent content, good sir
@zjardynliera-hood5609
@zjardynliera-hood5609 Ай бұрын
same this guy is nuts
@jorgegomezabrante8780
@jorgegomezabrante8780 Ай бұрын
sudo dnf install helix
@gaborbakos9012
@gaborbakos9012 Ай бұрын
A very cool video! I find loops easier to understand and use than the apply functions. However I always had a bad feeling about using them as they are supposed to be inefficient. This video made me feel better about my terrible loopy loops :).
@priyanshnegi03
@priyanshnegi03 Ай бұрын
Really great explanation! Saw both part 1 and 2 and now I understand geohashing clearly. Thank you.
@orgs804
@orgs804 Ай бұрын
which theme is that?
@josiahparry
@josiahparry Ай бұрын
I think it’s from material and it’s the material pale night high contrast
@user-hl1fk6lw8k
@user-hl1fk6lw8k Ай бұрын
How does one document functions exported from rextendr::document() so the document is readable and can be a reference used in Rstudio? Seeing the arguments are referred to as C objects?
@josiahparry
@josiahparry Ай бұрын
I'm not sure I follow with the last part of "arguments are referred to as C objects" There's a couple ways to do this. Personally, I like to write wrappers around my Rust functions so I can do better argument validation with {rlang}. So I use roxygen comments like normal. But you can also use roxygen comments in rust too!! You just add `///` which is a doc comment in rust. For example above the function write /// This is my function title /// /// this is my function description /// @param x a list /// @export
@JordiRosell
@JordiRosell Ай бұрын
2:02, 16:19, etc What are you typing here to autocomplete typing? Is it some VS code Rust extension?
@josiahparry
@josiahparry Ай бұрын
Are you referring to the type hints? That’s the inlay hints from the rust analyzer!
@JordiRosell
@JordiRosell Ай бұрын
​@@josiahparry yeah. I'm just starting / willing to start in rust, so this is why I ask newie questions.
@josiahparry
@josiahparry Ай бұрын
@@JordiRosell would a video on setting up VS Code for Rust help? TL;DR install the following extensions - rust-analyzer - Better Toml - todo tree (personal opinion on this one)
@joshstat8114
@joshstat8114 2 ай бұрын
I was wondering if there is a linear algebra library in extendr (Rust), like RcppArmadillo
@josiahparry
@josiahparry 2 ай бұрын
check out faer! faer-rs.github.io/
@richardbennett4365
@richardbennett4365 2 ай бұрын
Nice brief code, but on my machine I'm getting an error with the pretty: implementation Display statement.
@richardbennett4365
@richardbennett4365 2 ай бұрын
I got the code debugged. I just have a Code issue where the buffer for the lines of the Terminal needs increasing, and I think I would also like to `tee` the output to file as well as STDOUT.
@josiahparry
@josiahparry 2 ай бұрын
ah so this was an editor issue?
@richardbennett4365
@richardbennett4365 2 ай бұрын
Nice video, and this man has an accent similar to Will of SpaceRex. I like the presentation. The only thing i caught at the end was again a young man in the field of computer science, the third one (and one is from the UK so its not just North American English), who doesn't use the past participle form with have. He said "I've ran" instead of "I've run." It seems to be a phenomenon among younger folks, and I wonder if this grammatical form is just not taught in schools anymore, or is it how people, like his parents or friends are speaking in this region of the world. But, i have noticed Elliott Minns of another KZfaq channel (@Dreams Of Code) also using the past tense form of the verb with an auxiliary and not the past participle form of the verb. But, it's the message and content that matters. Good job.
@shobhitarya1637
@shobhitarya1637 2 ай бұрын
Very well explained.Thank you
@porlando12
@porlando12 2 ай бұрын
R dev here trying to broaden my horizons and wow I'm blown away with the performance! Keep making these!
@blaisepascal3905
@blaisepascal3905 2 ай бұрын
Thank you for this nice video! What plugin do you use in VS Code to have the option of running/debugging just above the main function?
@josiahparry
@josiahparry 2 ай бұрын
It's just the rust-analyzer! I have it mapped to shift+cmd+enter my keybindings.json file though { "key": "shift+cmd+enter", "command": "rust-analyzer.run" }
@blaisepascal3905
@blaisepascal3905 2 ай бұрын
@@josiahparry Thanks!
@user-sl5ds8kn5x
@user-sl5ds8kn5x 2 ай бұрын
Super clear and interesting to follow, Thank you!
@lanceferrer9023
@lanceferrer9023 2 ай бұрын
what happens if your function returns Vec<T>, where T is, say, struct Person { name: String , age: i32 }?
@josiahparry
@josiahparry 2 ай бұрын
You'll need to make sure that the struct is compatible with R using the #[extendr] attribute macro on the structs impl. Then, if you plan on returning the result to R, you should collect them as a list instead of Vec<T>. We need to think with two hats on at the same time: Rust and R. R doesn't have arbitrary types. It does, though, let you store a list of external pointers and thats what #[extendr] does. See: extendr.github.io/user-guide/extendr-macro.html#exporting-impl-blocks You might have some code like x.iter().map(|xi| { make Person }).collect::<List>() I do this extensively in github.com/JosiahParry/rsgeo/
@difflocktwo
@difflocktwo 2 ай бұрын
noob here. could you make your own unclass method for matrices to return the values the way you did it my assigning $dims to NULL?
@josiahparry
@josiahparry 2 ай бұрын
no you can't make a method because it's not an s3 generic. But you could make an `unclass_matrix()` function though!
@ElonTusk..
@ElonTusk.. 2 ай бұрын
subbed, keep up dwag
@howardbaek5413
@howardbaek5413 3 ай бұрын
Solid video Josiah. Would be interested to see your take on the R6 OOP system next!
@YuheiKuratomi
@YuheiKuratomi 3 ай бұрын
Thank you. Great video. I think I can use dots much better now.
@elbarzwalid7344
@elbarzwalid7344 3 ай бұрын
amazing content
@HorrorInsides-dm8fc
@HorrorInsides-dm8fc 3 ай бұрын
Hi Josiah - do we know why interleaving is being done?
@josiahparry
@josiahparry 3 ай бұрын
The interleaving is done to create a z-coordinate in a z-order curve (en.wikipedia.org/wiki/Z-order_curve#Coordinate_values) which helps identify a 2 dimensional position with only one value
@HorrorInsides-dm8fc
@HorrorInsides-dm8fc 3 ай бұрын
Keep up the good work Josiah. This is fantastic. Hope to see you hit the 100K subscriber soon.
@joshstat8114
@joshstat8114 3 ай бұрын
Is there any cookbook for `extendr` and `rextendr`? Also, how to set up `extendr` or `extendrsrc` in R markdown? When I run a chunk in extendr, it works and saved in environment however, when I knit the rmd file, it failed
@wyattstevens8574
@wyattstevens8574 4 ай бұрын
2:17 You forgot Plus codes!
@josiahparry
@josiahparry 4 ай бұрын
Never heard of them! They sound like an S2 derivative of placekeys! Thanks for sharing <3
@wyattstevens8574
@wyattstevens8574 4 ай бұрын
​@@josiahparryI don't think that's what it's from, but Google Maps uses it. It could be an S2 variant, but I don't know.