Time Series Forecasting with XGBoost - Use python and machine learning to predict energy consumption

Рет қаралды 376,519

Күн бұрын

In this video tutorial we walk through a time series forecasting example in python using a machine learning model XGBoost to predict energy consumption with python. We walk through this project in a kaggle notebook (linke below) that you can copy and explore while watching.
Notebook used in this video: www.kaggle.com/code/robikscub...
Timeline:
00:00 Intro
03:15 Data prep
08:24 Feature creation
12:05 Model
15:35 Feature Importance
17:33 Forecast
Follow me on twitch for live coding streams: / medallionstallion_
My other videos:
Speed Up Your Pandas Code: • Make Your Pandas Code ...
Speed up Pandas Code: • Make Your Pandas Code ...
Intro to Pandas video: • A Gentle Introduction ...
Exploratory Data Analysis Video: • Exploratory Data Analy...
Working with Audio data in Python: • Audio Data Processing ...
Efficient Pandas Dataframes: • Speed Up Your Pandas D...
* KZfaq: youtube.com/@robmulla?sub_con...
* Discord: / discord
* Twitch: / medallionstallion_
* Twitter: / rob_mulla
* Kaggle: www.kaggle.com/robikscube
#xgboost #python #machinelearning

Пікірлер: 405

@casperj4784 Жыл бұрын

A comprehensive yet succinct tutorial. And, having only just finished my Data Science degree, I found it very reassuring to see that you do get faster and more proficient with time.

@robmulla Жыл бұрын

I absolutely love messages like this. Glad to hear you found this helpful and it gave you the reassurment that things get faster. I can tell you that they do! The goal of my channel is to "spark curiosity in data science" I hope this video did that for you.

@RaviKumar-uf3eo Жыл бұрын

Yes. It is very reassuring, but most probably he would have kept all the things ready.

@amirghorbani7922 6 ай бұрын

It is better to use icdst Ai predict lstm model.

@karishmakapoor4285 Жыл бұрын

Amazing flow, comprehensive yet smooth. Detailed yet generic. I love the way you think and your float across the entire process. I did this project myself and thoroughly enjoyed it. Cant wait to apply this to other datasets. A Big thumps up👍

@naderbazyari2 2 ай бұрын

Second time watching this and doing every step on my notebook as Rob goes through the task. I am still blown away by the intricacy of his approach and how he investigates the case. fascinating how he makes it look effortless. Many thanks

@flel2514 Жыл бұрын

Hi Rob, I am a fresh data science graduate, and I find this tutorial very well done and very helpful for those that approach TS for the first time as well as for those that want to refresh the topic

@beckynevin1 3 ай бұрын

Wow! I'm trying to get up to speed on XGBoost, so I clicked on this video. There are a lot of meh data science tutorials out there, so it was such a treat to come across this one after slogging through youtube. I immediately subscribed and am headed to your channel to watch more videos on time series prediction!

@rodolfoviegas8504 11 ай бұрын

Amazing. We've learnt time series prediction only by statistical methods and/or making ML models to act like ARIMA - making lags for feed them. This approuch very interesting and intuitive. Thanks, Rob

@sevenaac4783 Ай бұрын

Thank you for teaching me. It allows me to understand the time series XGBoost in the shortest time.

@musicplace9205 5 ай бұрын

Thanks! one of the best video I've ever seen. Simple, clear and overall why each concept is used for.

@MilChamp1 Жыл бұрын

This was a very nice introduction to this topic. You might consider turning this into a miniseries, since it's such a large topic; the next video might be on how to create the best cross-validation splits for timeseries

@robmulla Жыл бұрын

Thanks so much. There is so much to cover with time series. I may consider a miniseries that’s a great idea. I’d like to make one on prophet which is a great package for time series forecasting too.

@jelc Жыл бұрын

Really well focused and clearly explained. Love your work!

@robmulla Жыл бұрын

I appreciate the feedback Julian

@luismisanmartin98 20 күн бұрын

As someone just getting introduced to time series analysis, this video was gold, thank you for making it!

@hussamcheema Жыл бұрын

I love your content. Liked the video before watching it because I know this is gonna be a great tutorial. Thanks for making these tutorials. 😊

@robmulla Жыл бұрын

Thanks! Glad you find it helpful.

@a.h.s.3006 Жыл бұрын

I worked with time series before, and this tutorial is very thorough and well made. Additional features you could think about are lag/window features, where you basically try to let the model cheat from the previous consumption, by giving it a statistical grouping of previous values, let's say the mean of consumption within a window of 8 hours, or by outright giving the previous value (lag), let's say the actual consumption 24 hours ago. This will greatly improve performance, because it helps the model to go follow the expected trend.

@robmulla Жыл бұрын

Thanks for the comment! Glad you enjoyed the video even though you already have experience with time series. You are 100% correct about the lag features. Check out part 2 where I go over this and a few other topics in detail.

@fudgenuggets405 Жыл бұрын

I like this dude's videos. They are informative and to the point.

@JacksonWelch Жыл бұрын

Love these videos. As a data engineer I love seeing other peoples workflows. Thanks so much for posting.

@robmulla Жыл бұрын

Glad you liked it. Thanks for watching Jackson.

@22niloc 10 ай бұрын

I'm getting to know Time Series and your vid has loads of great starter points.

@nirbhay_raghav Жыл бұрын

Hands down, the bestest (if that is a word) video on the entire internet about implementation. No fancy stuff. Not too beginner and toy examples. Hust the right thing what a budding data scientist needs to see. And it is definitely reassuring to see that one can really get better and faster at doing these after a while. It takes me a lot of time reach what you have done in under 30min. Debugging things take a lot of time.

@robmulla Жыл бұрын

I really apprecaite your positive feedback! Glad to hear you find it encouraging that eventually things will get faster.

@Singularitarian Жыл бұрын

Very illuminating! Learned a whole lot in just 23 minutes.

@adityaraikwar6069 9 ай бұрын

Being a sort of early intermediate data scientist myself, it's very cool watching him do all these things and the most amazing thing is how everybody's mind works differently and how proficient you become in not only coding but also in approach towards a problem. keep that up man

@paultvshow 6 ай бұрын

Hey, have you landed a job in data science field?

@digitalnomad2196 4 ай бұрын

also curious to know, recent data science graduate here@@paultvshow

@ADaBaker95 8 ай бұрын

Best video on the subject I've found so far!

@H99x2 Жыл бұрын

Incredible content and explanation. You definitely have a knack for this. I subscribed for more videos like this! Thanks :)

@robmulla Жыл бұрын

Thanks for watching and the feedback!

@egermani Жыл бұрын

Great content! Thanks a lot for the explanations, they are a great incentive to dive deeper into the subject.

@robmulla Жыл бұрын

Glad you think so! My hope is that by making short videos that explain a topic at a high level like this will spark curiosity in people so they will dive deeper into the topic, just like you said.

@troy_neilson Жыл бұрын

Informative and well-structured. Thanks!

@NotesandPens-ro9wx 5 ай бұрын

Man I am seeing this after an year and your teaching style is just hell .. now sub done and will follow you on other things :) for sure

@TrueTalenta 10 ай бұрын

I am new to time series and this by far is very informative and quit succinct!

@69nukeee 7 ай бұрын

Such an amazing video, thank you Rob and keep 'em coming! ;)

@zhuoningli Жыл бұрын

Hi Rob! Your tutorials help me get a job offer! When I was searching for a job, I received a take-home technical exercise about time series forecasting. I watched this video and finished my exercise. Finally, I got my dream job! Thank you so much!!! I really appreciate your tutorials! 🥰

@robmulla Жыл бұрын

Whoa, I really love hearing stories like this. That's amazing and I wish you the best in the rest of your career.

@sandyattcl Жыл бұрын

what an amazing tutorial! I just had to give a thumbs up even before finishing the video.

@robmulla Жыл бұрын

Really appreciate that Sandeep. Please share the link with anyone else you think might also like it.

@leo.y.comprendo Жыл бұрын

This is incredible! Instantly subscribed!! thanks for your knowldege

@robmulla Жыл бұрын

Thanks for watching!

@tomshaw7179 Жыл бұрын

Thanks for this video Rob. I am quite new to data science and this was really clear. Have you done a video on optimization maybe using light GBM?

@akshaymbhat9144 Жыл бұрын

Thanks for the wonderful video. It's very insightful ❤️ from India . Keep inspiring and aspiring always!!

@robmulla Жыл бұрын

My pleasure! So happy you liked it!

@romanrodin5669 Жыл бұрын

Great video! Very clear and easy for understanding! Thanks a lot for clear explanation! I've got a few questions though regarding lagging data for better prediction) will jump into next video, it seems I get an answer there) thanks again!

@robmulla Жыл бұрын

Glad you liked it. Yes, the next video covers it in more detail!

@lovettolaedo223 8 ай бұрын

I enjoyed watching this as it has given me more insight into prediction. Kindly do a video on GDP growth forecasting using machine learning. Thank you.

@lolmatt9 4 ай бұрын

Very well explained and useful. Thank you!

@user-xr3bc4vn5t 8 ай бұрын

You have helped me so much with this video, you don't even know!!! Thanks so much :)

@peralser Жыл бұрын

Great Video ROB, Thanks for sharing with us!!

@robmulla Жыл бұрын

Thanks for watching!

@Arieleyo Жыл бұрын

Love your videos Rob!! cheers from Argentina ♥

@robmulla Жыл бұрын

Sending my ❤ back to Argentina. Thanks for watching!

@azizbekurmonov6278 11 ай бұрын

Thanks! Love your explanations.

@gabrielmoreno2554 Жыл бұрын

Wow, this is exactly what I needed to learn to improve my COVID death predictor. Great job!

@robmulla Жыл бұрын

So glad you found this helpful. Thanks for watching!

@inovosystemssoftwarecompan6724 5 ай бұрын

short and potent, great fluid presentation !!

@evandrogaio7003 Жыл бұрын

Such an excellent video. Thanks for sharing!

@robmulla Жыл бұрын

Glad you liked it!

@PRATEEK30111989 4 ай бұрын

I have never seen a better data science video. You are a savant at this

@michaelmebratu2921 Жыл бұрын

What a quality tutorial! Thank you so much

@robmulla Жыл бұрын

Glad you learned something new!

@kvafsu225 Жыл бұрын

Great lesson on machine learning. Thank you.

@robmulla Жыл бұрын

Thank you for watching. Share with a friend!

@tatulialphaidze90 Жыл бұрын

Thank you for this tutorial, definitely helped me out

@robmulla Жыл бұрын

Glad it helped!

@yosafatrogika3129 Жыл бұрын

so clear explanation, thanks for sharing!

@robmulla Жыл бұрын

Glad it was helpful!

@Tonitonichoppa_o Жыл бұрын

This is the best!! Thank you so much :D 감사합니다!!

@chrispumping 11 ай бұрын

Very informative and easy to understand tutorial....Thanks you

@robmulla 11 ай бұрын

You are welcome! Thanks for watching.

@liliyalopez8998 Жыл бұрын

I just started studying ML and this tutorial is super helpful. I would like to see how you would use the model for forecasting future energy consumption though

@robmulla Жыл бұрын

Welcome to the wonderful world of ML Liliya! Yes, I did forget to cover that in detail but I may in a future video. It's just a simple extra step to create the future dates dataframe and run the predict and feature creation on it.

@yourscutely Жыл бұрын

Perfectly explained, thanks a lot

@robmulla Жыл бұрын

You are welcome! Glad you found it helpful. Check out parts 2 and 3 and share with a friend!

@MeghaKorade Жыл бұрын

Hello Rob, Great tutorial! I have a question - In eval_set you're using [(x_train, y_train), (x_test, y_test)] whereas in most data split practices I've seen validation set separated from training data (which not part of either training or testing set)? Can you please check at timestamp 14:02 ? I'm trying to implement something similar on an interesting dataset and this is a great tutorial!!

@lamborghiniveneno8423 Жыл бұрын

Simply awesome tutorial😀

@robmulla Жыл бұрын

Thanks so much!

@revathyb1663 10 ай бұрын

Great video. How are you taking into account the sequence in information while training the xgb model? Also, what method do you suggest while I deal with multiple time series, meaning say for example I have energy consumption from multiple regions and would like to have predict for each region.

@super-eth8478 Жыл бұрын

Dude your channel is a gold mine ..

@robmulla Жыл бұрын

Thanks so much for that feedback. Now share it with anyone you think might appreciate it too!

@super-eth8478 Жыл бұрын

@@robmulla Actually I have shared it to my friends . Cheers !

@demaischta1129 Жыл бұрын

This is so helpful. Thank You!!

@magicdimension6073 Жыл бұрын

Have you tried SARIMA Models for time series forecasting? I'm curious which perform better. Excelent content Rob!

@ademhilmibozkurt7085 Жыл бұрын

I love this video. Please make more. Thanks

@robmulla Жыл бұрын

Thanks! I apprecaite the comment. Have you seen the part 2 that I have on this topic?

@prasadjayanti Ай бұрын

Very good explanation.

@ramizajicek Жыл бұрын

Thank you for the great presentation

@robmulla Жыл бұрын

I appreciate you watching and commenting. Share with a friend!

@blueradium4260 Жыл бұрын

Brilliant video, thank you :)

@robmulla Жыл бұрын

Thanks for taking the time to watch.

@anatoliyzavdoveev4252 7 ай бұрын

Fantastic video tutorial 👏👏🙏

@nguyenduyta7136 Жыл бұрын

Best one I ever seen ❤thank so much.

@robmulla Жыл бұрын

So glad you like it. Thanks for the comment.

@datalyfe5386 Жыл бұрын

Just came across your channel, awesome content!

@robmulla Жыл бұрын

Welcome aboard! Glad you like it.

@gustavojuantorena Жыл бұрын

"And depending who you ask" 🤣Great video!

@robmulla Жыл бұрын

I’m glad you got the reference. I was hoping he would see and appreciate that part of the video.

@Mvobrito Жыл бұрын

Great video! If the goal was prediction only, and not inference (meaning you don't care about what's driving the energy consumption), you can the energy consumption of the previous days as feature for the model. When predicting consumption at T, you can use T-1, T-2, .. T-x. And even a moving average as feature as well.

@robmulla Жыл бұрын

I totally agree! It all depends on how far in the future (forecasting horizon) you are attempting to predict.

@wazzadec16 Жыл бұрын

FYI for anybody who is doing this recently. The part where combing training set and test set graphic and using a dotted line has to be modified. Before: '01-01-2015' After ax.axvline(x=dt.datetime(2015,1,1) Since matplotlib now needs it in a datetime series. I guess because of changing the index to a t0_datetime format?

@shrunkhalawankhede2611 8 ай бұрын

from datetime import datetime ax.axvline(x=datetime(2015,1,1), color='black', ls='--')

@ChrisHalden007 Жыл бұрын

Great video. Thanks

@robmulla Жыл бұрын

Appreciate that 🙏

@Burnitall220 3 ай бұрын

This is incredible!!

@jianyugong8277 Жыл бұрын

This video is the best I've seen! You are so good at explaining this. Instead of RMSE, could you please advise how I can show the confidence interval of the predictions? I tried to google an answer but nothing worked.

@robmulla Жыл бұрын

Thanks so much for the kind words. Unfortunately with a model like xgboost there are no confidence intervals. However something like an ARIMA or fbprohet model (which i'm planning on making another video about) can give you confidence intervals. Check out part 2 if you haven't already!

@raasheedpakwashi2961 Жыл бұрын

LEGEND...no other words needed

@robmulla Жыл бұрын

Thank you 🙏

@vlplbl85 Жыл бұрын

Great video. Don't you think adding time lags would increase performance the most? In my time series forecasts I find them very helpful, especially with seasonality.

@robmulla Жыл бұрын

Hey Vladimir - thanks for the feedback. You are correct, lag variables can be very helpful. You need to remember, however that your lag variables can not be shorter than your forecasting horizon. So if you add a 1 week lag variable, then your model would not be able to predict further than 1 week. 1 year lags can be very helpful though.

@abinsharafm.s5168 Жыл бұрын

I follow you on twitch.. you should definitely do a video on how you setup your system for data science ( I mean you had Linux working with your ide and you were pulling data from the websites (api).. i found that very cool ! )

@robmulla Жыл бұрын

Oh. Great idea! I’ve thought about doing this but need to think more about how to best explain my setup.

@Dongnanjie 4 ай бұрын

Thank you, Rob!

@Lnd2345 Жыл бұрын

Great video, thanks.

@robmulla Жыл бұрын

Glad you liked it! Thanks for the feedback.

@ErikaGPF Жыл бұрын

Hi, thanks for the video! Pretty good! I have a question, wouldn't improve your model to use the actual 'PJME_MW' as input? It's a honest question, it is because I saw in other examples for timeseries forecasting that uses the metric you wanna predict as input as well. Thank you!

@robmulla Жыл бұрын

Great question! If you use the actual value for a future time step you would be leaking information. Check out my part 2 video where I talk about the forecasting horizon. Hope that helps!

@adityagavali3158 11 ай бұрын

Thank for this!

@user-cf5pf7on7k 8 ай бұрын

Great video - you briefly mentioned stationarity in the beginning, but you didn't actually test for it. This data looks stationary to me, but if it wasn't would that cause a problem? Or is that only an issue with ARIMA models? Thanks!

@mohamednedal 8 ай бұрын

Hi Rob, Great tutorial! Could you please make a tutorial on how to use Shapley values to interpret LSTM models for timeseries forecasting?

@lucasfescina Жыл бұрын

I love your videos

@AQ-jh5fr Жыл бұрын

Nice tutorial and when you said quick tutorial you sure meant it xD, I had to pause like a 100 times. but still thanks for the video

@robmulla Жыл бұрын

Glad you liked the video. I'd rather it be too fast than too slow :D - you can always slow down the playback speed if that helps.

@haleemahabulaimon8081 11 ай бұрын

I really appreciate it

@neclis7777 Жыл бұрын

Excellent video ! For weather, I suggest you look into HDD and CDD (heating degree days and cooling degree days) which focus on the amount of heating and cooling rather than the mean temperature.

@robmulla Жыл бұрын

Thanks for the tips! I'm not familiar with those but I will look into it. The one main issue I see when people are training forecasting models like this is using the ground truth weather for future dates- which are not available at the time of prediction. That's why I think it's best to use forecast values from the historic dates.

@massoudkadivar8758 Жыл бұрын

Perfect job👌

@indranil9874273176 Жыл бұрын

How rational is it to use a tree-based model for time series forecasting? Not sure about XGBoost, but in general tree-based models can not extrapolate, meaning the predictions would be bounded by the minimum and maximum of the training target variable. If we have a time series with an increasing trend, is that a good option? Btw, just subscribed :D

@robmulla Жыл бұрын

Thanks for subscribing. You are correct that this type of model will no do well predicting unseen values. However for this type of dataset it can work well. I mention this earlier in the video when talking about the different types of time series. Hope that helps.

@datasciencesolutions2361 Жыл бұрын

Great job sincerely!

@robmulla Жыл бұрын

Thanks for the feedback!

@selenkokten1708 Жыл бұрын

Don’t use features like year which will not have the same value in the future. It is a bad idea for prediction purposes. Instead use the difference from the minimum date to see if there is an increasing trend year by year.

@paultvshow 6 ай бұрын

Please elaborate

@irshadyasseen146 6 ай бұрын

Can you provide an example?

@solisoma1012 5 ай бұрын

Can I have ur social media handle so I can ask you some questions

@John5ive Ай бұрын

I get it. The year increments and provides no value to the model.

@warmpianist Ай бұрын

The difference from minimum date also won't have the same value in the future. I don't know what you mean.

@darenhunter3725 4 ай бұрын

Super helpful. Do you have any videos on how we might be able to correlate historical weather or historical forecasts in an example like this? I'm struggling to wrap my head around how this could be done and every search I do tries to teach me about how to create weather models - not use time-series weather or forecasts to predict a value. Thanks for your videos

@a.a.elghawas Жыл бұрын

Cool video Rob!

@robmulla Жыл бұрын

Thanks for watching!

@christianmannke2335 4 ай бұрын

First of all, thank you for this comprehensive video. It helped me a lot to understand this kind of prediction better. However, what I still don't understand is how can I make predictions on new data that the model hasn't seen before? Let's say I want to make predictions from 2018-08-03 for the next 30 days.

@anwarsaidan3959 Ай бұрын

Hi Rob, Thank you very much for this tutorial. When using XGBoost , we don't do these kinds of data prep : scaling, checking for seasonality, filtering outliers ?

@digitalnomad2196 5 ай бұрын

Great content, I am curious where you got the data set. I know you mentioned you uploaded it previously but what was the original source of this data. Cheers !

@rohitraghunathan Жыл бұрын

Very good introduction to time series forecasting in xgboost. One thing to note is xgboost has a function for plotting the feature importance. xbg.plot_importance() would have done the trick for you

@sunaxes 3 ай бұрын

Would be useful to look at feature importance at the inflexion point of the test set performance and at the end of training and compare. Features highly ranked in both are the ones useful to understand pattern in data and also also satisfy labelling requirements.

@exampyklx7953 Жыл бұрын

WOW! This is AMAZING content. May I know what book you studied. Thank you.

@robmulla Жыл бұрын

Glad you liked it. Most of what I shared in this video I learned through kaggle and working in industry, there was no one book I could point to.

@THE8SFN Жыл бұрын

great tutorial

@robmulla Жыл бұрын

Thx!

@adityaghai220 4 ай бұрын

amazing video

@mirror1023 Жыл бұрын

Amazing video

@robmulla Жыл бұрын

Thanks!

@sathvikmalgikar2842 Жыл бұрын

sir you are legend. thank you i was banging my head with lstm model in pytrch previosly but this is way better

@robmulla Жыл бұрын

Glad you got something working.

@kaaz4044 10 ай бұрын

A question. I see the prediction was done on test data which are already available. This is good to see how accurate the model is but I am wondering how we can use this model (and xgboost in general) to forecast the upcoming years for which we do not have any data.

@santinonanini6107 8 ай бұрын

Should you not split the training data into train and validation sets, such that you can use validation set instead of test set during training ? (when you use "eval_set" parameter ?)