Working with time series data in MySQL

  Рет қаралды 20,581

PlanetScale

6 ай бұрын

To learn more about PlanetScale, head to planetscale.com/youtube!
💬 Follow PlanetScale on social media
• Twitter: planetscale
• Discord: discord.gg/pDUGAAFEJx
• TikTok: www.tiktok.com/@planetscale
• Twitch: www.twitch.tv/planetscale
• LinkedIn: www.linkedin.com/company/planetscale

Пікірлер: 80
@ahmad-murery
@ahmad-murery 6 ай бұрын
Wow Steve's graphic design skill is getting better, compared to the previous video Aaron in this video looks more realistic and is barely distinguishable from the real one 👍😁 anyway, even when the query is fast enough, we also need to consider the number of times we may execute it (multiple users requesting the data), so using a summary table is definitely helpful here. Thanks Aaron,
@DevanSabaratnam
@DevanSabaratnam 6 ай бұрын
I've been working in MySQL for over 20 years now, but learned a TON from this video - Thank you for putting out such great learning content Aaron!
@hans-joergwahmkow2621
@hans-joergwahmkow2621 6 ай бұрын
If I understand this correctly, one important feature of a window function is that everything inside is on a rolling basis. That is for sure not something you could ever do with a simple GROUP BY. I do have an application where this may actually come in very handy. Thanks Aaron, great content as always!
@edgars-strods
@edgars-strods 6 ай бұрын
Maybe I did not catch the specific point why window function was used here instead of simple GROUP BY "bucket"?
@mentalmaniac76
@mentalmaniac76 6 ай бұрын
my guess is so that you are able to have multiple, separate, "groups" and operate on them all within the same query
@xelhuajosephcoronaperez3531
@xelhuajosephcoronaperez3531 6 ай бұрын
@planetscale???
@PlanetScale
@PlanetScale 6 ай бұрын
Window functions are a great way to compute running totals, averages, sums, etc. Or do something specific with the first or last row. Or peek at the next row or the previous row within a group. Or group by different things within a single query. They're very powerful!
@CodeErrorOfficial
@CodeErrorOfficial 6 ай бұрын
@@PlanetScale yes but this specific task would be better with group by. still a great thing to know
@victorcosenza254
@victorcosenza254 6 ай бұрын
in this case it could be done with groupby right? Or im missing something?
@thewayis_meh987
@thewayis_meh987 6 ай бұрын
Classic Aaron, using MySQL as a calculator
@PlanetScale
@PlanetScale 6 ай бұрын
Haha didn't even think about it
@anderskozuch7838
@anderskozuch7838 6 ай бұрын
Thank you for this video!! Wonderful to dive into SQL like this and absolutely fantastic to see your enthusiasm Aaron. It's very inspiring and I am really excited to learn about the summery table when you make that video. Such a nice idea to make a query faster by just sort of "computing it before its needed" !
@warutt5356
@warutt5356 6 ай бұрын
Great job! So far, the best tutorial clips on common windows functions.
@jricardoprog
@jricardoprog 6 ай бұрын
Thank you!!! Good to see this type of content. 🍻
@OnlinePseudonym
@OnlinePseudonym 6 ай бұрын
Amazing! Been trying to do this for ages! The explanation is great.
@jfdirienzo
@jfdirienzo 6 ай бұрын
This channel just keeps on getting better 😍
@referralhelper
@referralhelper 6 ай бұрын
One of the best KZfaq channels when it comes to sql!
@alexrix4063
@alexrix4063 6 ай бұрын
Video request: Geospacial indexing
@d47im5e
@d47im5e 6 ай бұрын
Cool, detailed, niche case handling explanation video, keep it up!
@chao-chienchen4393
@chao-chienchen4393 6 ай бұрын
Perfect content right there!
@jackjsy
@jackjsy 6 ай бұрын
Perfect timing!
@massaynus
@massaynus 5 ай бұрын
GROUP BY looking at you menacingly from the corner 😂
@ZacZajdel
@ZacZajdel 6 ай бұрын
The fact this content is free is outstanding. Beyond blessed to be a developer in 2024!
@GringoDotDev
@GringoDotDev 6 ай бұрын
this is great!
@zeratax
@zeratax 6 ай бұрын
slick animations!
@JacoBoogie
@JacoBoogie 6 ай бұрын
Thanks Steve!
@mogadanez
@mogadanez 5 ай бұрын
with rollup table potential problem is with presenting data in different timezones, especially when doing stats per day
@shohruxshomaxmudov1225
@shohruxshomaxmudov1225 6 ай бұрын
This is like learning chess openings: at first everything is clear and easy, but in the end, your mind will explode.
@hannielgutierrez9139
@hannielgutierrez9139 6 ай бұрын
Hi. I wanna thank you for beeing a great teacher. I have learnt a lot of things from you. I have a question, traditionally, when i have the case where i need to do some calculations, and the reuse them, I usually create a temporary table, so that i can consume the data that has been calculate and not repeat it. What would it be your recommendation , use subqueries, use temporary table or the one that I just leaned, the with utility.
@dhawaljoshi
@dhawaljoshi 6 ай бұрын
MySQL lacks mviews, anyway to get similar functionalities using just DB options. I've experimented with triggers on insert,update,delete that reruns the query to fill the table but incase of multiple inserts or updates it's not quite what was expected.
@noamanahmed1
@noamanahmed1 6 ай бұрын
Video Suggestion How to handle case where you have to insert n numbers of rows and then you need the primary key(id) for those to be inserted in a child row WITHOUT making n insertion queries
@cagdasucar3932
@cagdasucar3932 6 ай бұрын
Idea for your summary table video: in order to maximize performance and reduce unnecessary storage, I like to do summary view + index instead of an actual table.
@barmalini
@barmalini 6 ай бұрын
Thanks for a great video. SQL is clearly not amazingly fast when it gets to bigger datasets and more complex analysis. Wouldn't it be beneficial to use another tool for a task like this, something like Spark perhaps?
@BloodyClash
@BloodyClash 6 ай бұрын
😁only understand about 1% but i'm sooo impressed what you are able to and also what mysql is capable of. i only use it as a middleman in my java and c# programs
@joelkuijper
@joelkuijper 6 ай бұрын
Have you ever heard of an espresso martini?
@mogadanez
@mogadanez 5 ай бұрын
I get same output with groupBy but times faster, only one thing missed is first_value(dt), but it can be replaced by MIN(dt)
@GringoDotDev
@GringoDotDev 6 ай бұрын
another video I'd be very interested to see is a deep dive on MySQL replication
@conceptrat
@conceptrat 6 ай бұрын
Multi master is so much fun 🤔😬
@abdelilahou2822
@abdelilahou2822 6 ай бұрын
I dont use mysql i dont even use planetscale (no offense) but i learn alot of things from you, and I like how you explain stuff
@developertpl
@developertpl 6 ай бұрын
17 minutes in and still asking myself why are we not grouping by key and bucket instead of trying to get the last row of each group.
@Szarlus
@Szarlus 6 ай бұрын
My thoughts exactly as well :P Probably could even do `GROUP BY bucket ORDER BY bucket`
@goaserer
@goaserer 6 ай бұрын
For the aggregations a simple group by should work as well. It would complicate the query in case you actually need the fields for first- and last timestamp (although one could be calculated back from the bucket). In the end using window functions was most likely the more interesting and educational topic for a video, even though the simple example could be solved in other ways
@developertpl
@developertpl 6 ай бұрын
@@goaserer I think it’s more like a leading video for the summary tables topic. I recently worked on a topic like this so I keep seeing group by solutions everywhere I look.
@Szarlus
@Szarlus 6 ай бұрын
@@developertpl I agree that showing window functions approach here is a nice educational approach. However, since it was mentioned that it is a no go for PlanetScale, it would be nice to also show the group by approach as an alternative here. Especially after we filtered only last record of each bucket which is literally what would group by achieve. From my point of view window functions are best used when you're reviewing per-record data in context of its group/bucket, which wasn't the case here. Tl;dr I enjoyed the video and always appreciate videos on this channel as they are of high quality and very informative. Just pointing out showing group by approach as well would make it "whole" :)
@developertpl
@developertpl 6 ай бұрын
@@Szarlus Great content for sure! Always a pleasure.
@xesau
@xesau 6 ай бұрын
Can't you just SELECT MIN(dt), AVG(v), MIN(v), MAX(v) FROM timeseries GROUP BY FLOOR(UNIX_TIMESTAMP(dt) / bucket_length_in_secs)?
@michaelmo5
@michaelmo5 6 ай бұрын
Bump! I was thinking the same.
@batvetone
@batvetone 6 ай бұрын
If we have a bucket as calculated column, would group by do the job?
@fredyy
@fredyy 6 ай бұрын
Why don't just group by bucket and use mysql's AVG function to get the average value in the bucket?
@userasd360
@userasd360 6 ай бұрын
can you provide the data file for the data that you have used?
@milad8589
@milad8589 6 ай бұрын
🤯👏
@LenWoodward
@LenWoodward 6 ай бұрын
If you keep making videos like this, my `favourites` saved list is just going to look like your channels `videos` page.
@user-lx9tx8ie8o
@user-lx9tx8ie8o 6 ай бұрын
SAD there is no more hobby plan in PlanetScale, :/
@RajveerSingh-vf7pr
@RajveerSingh-vf7pr 6 ай бұрын
Hi PlanetScale, I have a Table, with one column, which are the timestamps on which server was observed alive. I want to write a select query, which returns me the data for every minute, if there exists a timestamp in that minute, 1, else 0.
@YandiBanyu
@YandiBanyu 6 ай бұрын
Is there an advantage counting these on the database level rather than the application level? Strictly speaking, is there any performance advantage when doing this calculation on the database?
@_MB_93
@_MB_93 6 ай бұрын
It is faster First, you don't pull tens of thousands of rows over the network Second, even if MySQL is not optimized for timeseries data, it is still dozen of times faster than say, Javascript or Java. After you got the grouped result which has reduced size, then you pull that data to application level and display them as charts
@YandiBanyu
@YandiBanyu 6 ай бұрын
@@_MB_93 But, will it create another bottleneck elsewhere? I mean, database connection is a resource. Will performing calculation hold those connections? Because if it is, then wouldn't it be better to just dump data and then perform calculation on the application side since the application server is typically easier to scale than db server? But of course, this also needs to be performed after profiling.
@_MB_93
@_MB_93 6 ай бұрын
@@YandiBanyu It is a bottleneck no matter where you put it, its just that database is a specialized software for aggregating and transforming data, I personally think it is "re-inventing the wheels" if things such as "group by" is performed on application side. As for scaling, well, as Aaron said in the video, MySQL can take you a long long way before you actually need it.
@YandiBanyu
@YandiBanyu 6 ай бұрын
@@_MB_93 yeah, there will always be bottleneck somewhere. But some of it is easier to solve than others. I agree that not many applications will scale beyond what RDBMS such as MySQL offer, but it is still good to keep in mind too when doing costly operations on the database level. After all, it is (IMHO) harder to reduce the bottleneck of a single database server (before going to a more complex clustering strategy) than say, spin up another server instance behind a load balancer. This can matter when making an application that needs on premise data for some security/legal reasons.
@JakobStrasser
@JakobStrasser 6 ай бұрын
Always forget the end on the case. Every time.
@nikhilpsathyanathan
@nikhilpsathyanathan 6 ай бұрын
Totally new for me😅
@Vali615
@Vali615 6 ай бұрын
What about data gaps?
@davidharting3119
@davidharting3119 6 ай бұрын
brb gonna go build an APM tool
@Wizatek
@Wizatek 6 ай бұрын
Can't you just group by the bucket calculation?
@TheEkkas
@TheEkkas 6 ай бұрын
MySql Gandalf
@mityukov
@mityukov 6 ай бұрын
GROUP BY on steroids 😅
@sililoconnect8846
@sililoconnect8846 6 ай бұрын
😅😂😂😂😂
@SashNone
@SashNone 6 ай бұрын
I was lost around 10 minutes in.....
@testdummy6792
@testdummy6792 6 ай бұрын
With all the talk on indexing, what does it really do? Is there such a thing as too many indexes? I'd imagine there's eventually a limit, either through practicality or performance, stopping you from adding an index for each and every column.
@userasd360
@userasd360 6 ай бұрын
Isn't summary table similar temp table?
EVOLUTION OF ICE CREAM 😱 #shorts
00:11
Savage Vlogs
Рет қаралды 8 МЛН
КАК ДУМАЕТЕ КТО ВЫЙГРАЕТ😂
00:29
МЯТНАЯ ФАНТА
Рет қаралды 10 МЛН
low battery 🪫
0:10
dednahype
Рет қаралды 1,2 МЛН
Как бесплатно замутить iphone 15 pro max
0:59
ЖЕЛЕЗНЫЙ КОРОЛЬ
Рет қаралды 8 МЛН
Look, this is the 97th generation of the phone?
0:13
Edcers
Рет қаралды 7 МЛН
Опасность фирменной зарядки Apple
0:57
SuperCrastan
Рет қаралды 11 МЛН
Сколько реально стоит ПК Величайшего?
0:37