Probabl Livestream: Exploring Ibis for DataFrames

Рет қаралды 579

Ай бұрын

We're experimenting with live streams!
In this live-stream Vincent will dive into Ibis, which is a DataFrame library that allows one to be flexible with their backend. You can use one API to support pandas, polars or ... postgres or bigquery. There's some interesting things to explore here, so let's do that in a live stream.
So tune in if you'd like to see a demonstration and feel free to ask questions

Пікірлер: 10

@rkenne1391 16 күн бұрын

This is the most practical session, I have seen on Ibis. Thank you Sir. The ability to write composable, testable and portable code for so many backend is insane.

@JordiRosell Ай бұрын

23:00 using other backend for testing is what I do in R with dplyr, I'm happy to see a similar tool in Python.

@JordiRosell Ай бұрын

41:07 I guess you can use ibis create_table method to create a new table with one column and then use a projection to merge the original columns that you want with the new column of the new table.

@probabl_ai Ай бұрын

(Vincent here) Yeah, this was later recommended to me by an ibis contributor. It feels "fine" but the fact that you'd need a join here (which is an expensive operation) is a bummer. The join seems needed to guarantee the ordering.

@argh44z Ай бұрын

thanks for the stream, YT just rec'd it to me. is there any place for links for the original notebook?

@cpcloud Ай бұрын

Ibis is coming out a bit fast on some of these. It's because %%time doesn't include the time to __repr__, which is where execution happens!

@probabl_ai Ай бұрын

(Vincent) Ah, yeah that makes sense. It deserves to be stressed that all of this was mostly exploratory stuff and not really a proper benchmark. Still, I'd say it's fair to assume that, at least usually, the DuckDB backend is usually a fair bit faster than Pandas.

@wexwexexort Ай бұрын

ibiş (ibish) means dumb, stupid in Turkish slang. The word is originated from an open theater culture in late ottoman era, a classic role in those theaters was a stupid, clumsy servant named ibiş. not related to ibis, just wanted to share what this name reminds me.