I absolutely love messages like this. Glad to hear you found this helpful and it gave you the reassurment that things get faster. I can tell you that they do! The goal of my channel is to "spark curiosity in data science" I hope this video did that for you.

Yes. It is very reassuring, but most probably he would have kept all the things ready.

It is better to use icdst Ai predict lstm model.

Whoa, I really love hearing stories like this. That's amazing and I wish you the best in the rest of your career.

Thanks for the comment! Glad you enjoyed the video even though you already have experience with time series. You are 100% correct about the lag features. Check out part 2 where I go over this and a few other topics in detail.

I really apprecaite your positive feedback! Glad to hear you find it encouraging that eventually things will get faster.

Hey, have you landed a job in data science field?

also curious to know, recent data science graduate here@@paultvshow

Thanks so much. There is so much to cover with time series. I may consider a miniseries that’s a great idea. I’d like to make one on prophet which is a great package for time series forecasting too.

Glad you liked it. Thanks for watching Jackson.

from datetime import datetime ax.axvline(x=datetime(2015,1,1), color='black', ls='--')

Really appreciate that Sandeep. Please share the link with anyone else you think might also like it.

Please elaborate

Can you provide an example?

Can I have ur social media handle so I can ask you some questions

I get it. The year increments and provides no value to the model.

The difference from minimum date also won't have the same value in the future. I don't know what you mean.

Thanks! Glad you find it helpful.

Thanks so much for that feedback. Now share it with anyone you think might appreciate it too!

@@robmulla Actually I have shared it to my friends . Cheers !

Glad you got something working.

Glad you learned something new!

Totally agree. Its data snooping. Nevertheless there are some cases where you can use all data to validate if you then receive the test set where you can check how the model generalizes.

So glad you found this helpful. Thanks for watching!

Sending my ❤ back to Argentina. Thanks for watching!

I appreciate the feedback Julian

Thanks for watching!

My pleasure! So happy you liked it!

Glad you liked it!

I’m glad you got the reference. I was hoping he would see and appreciate that part of the video.

Thank you for watching. Share with a friend!

You are welcome! Glad you found it helpful. Check out parts 2 and 3 and share with a friend!

Thanks so much!

Welcome to the wonderful world of ML Liliya! Yes, I did forget to cover that in detail but I may in a future video. It's just a simple extra step to create the future dates dataframe and run the predict and feature creation on it.

I totally agree! It all depends on how far in the future (forecasting horizon) you are attempting to predict.

Glad it was helpful!

Thank you 🙏

So glad you like it. Thanks for the comment.

Glad you think so! My hope is that by making short videos that explain a topic at a high level like this will spark curiosity in people so they will dive deeper into the topic, just like you said.

Thanks for watching!

You are welcome! Thanks for watching.

Glad it helped!

I appreciate you watching and commenting. Share with a friend!

Thanks for watching and the feedback!

Glad you liked it. Yes, the next video covers it in more detail!

Thanks! I apprecaite the comment. Have you seen the part 2 that I have on this topic?

Thanks for taking the time to watch.

Thanks for the tips! I'm not familiar with those but I will look into it. The one main issue I see when people are training forecasting models like this is using the ground truth weather for future dates- which are not available at the time of prediction. That's why I think it's best to use forecast values from the historic dates.

Welcome aboard! Glad you like it.

I appreciate the feedback.

Thanks!

I could make a video about LSTMs however those types of complicated models tend to be outperformed by models like XGBoost on these types of datasets. There are a few papers on the topic and forecasting is both an art and science.

@@robmulla yes, but LSTM should pick up better the long lag influence, while XGB is random based and can be tuned in some directions, but not as much / fine for this aspect as the former. And it seems that SARIMAX is out of favor, shame. Also a try out with VAR was pretty ok for +- known time lags. Any thoughts? ps: add more max depth, to get more feat coverage...

Thanks for watching!

Glad you liked the video. I'd rather it be too fast than too slow :D - you can always slow down the playback speed if that helps.

Thanks!

Yes! I works well for time series data that is stationary. It wouldn't work well for time series that will have values in the future way outside of what has occurred in the past.

​​@@robmullao be fair, there is no model that would work if the process is non-stationary - SARIMAX, Random Forest, Linear Regression etc. How about addressing autocorrelation in the underlying process when using xgboost? I think you should've plot pacf and acf, and add some lagged power consumption to the features accordingly.

I also thought of that. I suppose it was out of the scope of the tutorial to keep it simple.

Thanks for the feedback!

Appreciate that 🙏

Thanks for subscribing. You are correct that this type of model will no do well predicting unseen values. However for this type of dataset it can work well. I mention this earlier in the video when talking about the different types of time series. Hope that helps.

Check out part 2 where I discuss models that can pick up on those types of trends.

@@robmulla I did everything as per part 2. This works fine when I ran with test and train data it shows all correct but when I provide future dates it starts showing a seasonal kind of output. Can you post some examples or share a link where data is exponentially growing based on time...And our modal should be able to predict the future value. My data looks like 2023-01-01 21:00:00.643 178352674 2023-01-01 15:00:00.640 178322674 2023-01-01 09:00:00.443 178307674 2023-01-01 03:00:00.907 178302674 2022-01-01 18:00:00.793 132197674 2022-01-01 12:00:00.920 132192674 2022-01-01 06:00:00.227 132187674 2022-01-01 00:00:00.333 132182674 2021-01-01 18:00:01.197 72887423 2021-01-01 12:00:00.830 72812423 2021-01-01 06:00:00.703 72812423 2021-01-01 00:00:00.783 72812423 2020-01-01 21:00:00.917 22532423 2020-01-01 15:00:00.903 22532423 2020-01-01 09:00:00.613 21962423 2020-01-01 03:00:01.050 21937423

Thanks for liking

I don’t think so. What do you mean?

Glad you liked it! Thanks for the feedback.

Ooops. Did I? Can’t you point me to the timestamp?

​@@robmullaYes you did. 00:14:00 minute

Indeed the eval_set should not include test data. A clear case of data leakage

And also, for the eval_set, should you not pass validation data instead of all the X_train? As I see it, you are evaluating the performance on the very same data it learned from. Should it not be a subset of the training data that is evaluated?@@vzinko

Oh. Great idea! I’ve thought about doing this but need to think more about how to best explain my setup.

Thx!