Kishan Manani - Feature Engineering for Time Series Forecasting | PyData London 2022

  Рет қаралды 73,471

PyData

PyData

Күн бұрын

Kishan Manani present:
Feature Engineering for Time Series Forecasting
To use our favourite supervised learning models for time series forecasting we first have to convert time series data into a tabular dataset of features and a target variable. In this talk we’ll discuss all the tips, tricks, and pitfalls in transforming time series data into tabular data for forecasting.
Github/Slides: github.com/KishManani/PyDataL...
www.pydata.org
PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.
PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.

Пікірлер: 47
@ninjaturtle205
@ninjaturtle205 Жыл бұрын
thank you thank you. This information is skipped out in most machine learning courses, and no one will teach you this. In practice, a lot of data has temporal nature, while all along you only learned how to classify cats and dogs, and regress house pricess.
@julien957
@julien957 Жыл бұрын
Genius. He Makes python and time series almost easy to understand.
@lashlarue7924
@lashlarue7924 7 ай бұрын
Thank you. This was 43 minutes very well spent.
@youknowmyname12345
@youknowmyname12345 Жыл бұрын
Very good talk. The presenter is a great teacher!
@user-lo1fb6to8z
@user-lo1fb6to8z Жыл бұрын
Excellent presentation. Great work Kishan
@anirudhsharma3879
@anirudhsharma3879 6 ай бұрын
Amazing dump of knowledge, I have multiple times came back to this video
@duscio
@duscio Жыл бұрын
Great Presentation ! Interesting and clear
@Xaphanius
@Xaphanius 2 жыл бұрын
Great presentation!
@shivamgoel0897
@shivamgoel0897 10 ай бұрын
Amazing! So easy to understand.
@14loosecannon
@14loosecannon Жыл бұрын
Really informative talk!
@olegkazanskyi9752
@olegkazanskyi9752 4 ай бұрын
This is a truly useful session. Thank you for sharing the knowledge!
@bitzelcortez4011
@bitzelcortez4011 Жыл бұрын
Excellent talk!
@5112vivek
@5112vivek Жыл бұрын
I will checkout these libraries. Very informative, thanks
@georgiosmitrentsis6171
@georgiosmitrentsis6171 Жыл бұрын
Great talk!
@DiscomongoEGE
@DiscomongoEGE Жыл бұрын
Thank you very much. Great talk
@wexwexexort
@wexwexexort Жыл бұрын
Fantastic!
@zakkyang6476
@zakkyang6476 7 ай бұрын
finally, someone can articulate this topic well...
@GandelfTheGrey
@GandelfTheGrey Жыл бұрын
Great talk thanks
@laizerLL572
@laizerLL572 Жыл бұрын
Hi Am so grateful for this tutorial
@onuragmaji
@onuragmaji 8 ай бұрын
Great talk hope will get more contents like that on Practical TS
@user-cy6ck1vy5t
@user-cy6ck1vy5t 7 ай бұрын
dude is a PhD for a reason, awesome stuff god damn
@HEYTHERE-ko6we
@HEYTHERE-ko6we Жыл бұрын
This is by far one of the best wholesome videos on time series forecasting!!! loved it
@hp5072
@hp5072 Жыл бұрын
The word wholesome doesn't mean what you think it means :) Did you mean comprehensive or extensive?
@wexwexexort
@wexwexexort 8 ай бұрын
​@@hp5072what does it mean
@solvem_probler
@solvem_probler 3 ай бұрын
Nice talk
@yuh850321
@yuh850321 Жыл бұрын
Great talk
@user-cy6ck1vy5t
@user-cy6ck1vy5t 7 ай бұрын
this is some sysly good stuff!
@aakashnandrajog7035
@aakashnandrajog7035 Жыл бұрын
Amazing
@satyakiray8588
@satyakiray8588 Жыл бұрын
excellent and very informative presentation. Will definitely checkout darts and sktime
@kaidendubois
@kaidendubois 2 жыл бұрын
Super helpful presentation, thank you, will definitely be checking out your course!
@TraininData
@TraininData Жыл бұрын
Here is the link, just in case ;) www.trainindata.com/p/feature-engineering-for-forecasting
@neo_otaku_gamer
@neo_otaku_gamer Жыл бұрын
thoughts on using TFT model for multi time series forecasting
@wolpumba4099
@wolpumba4099 3 ай бұрын
*Abstract* This talk explores how to adapt machine learning models for time series forecasting by transforming time series data into tabular datasets with features and target variables. Kishan Manani discusses the advantages of using machine learning for forecasting, including its ability to handle complex data structures and incorporate exogenous variables. He then dives into the specifics of feature engineering for time series, covering topics like lag features, window features, and static features. The talk emphasizes the importance of avoiding data leakage and highlights the differences between machine learning workflows for classification/regression and forecasting tasks. Finally, Manani introduces useful libraries like Darts and sktime that facilitate time series forecasting with tabular data and provides practical examples. *Summary* *Why use machine learning for forecasting? (**1:25**)* - Machine learning models can learn across many related time series. - They can effectively incorporate exogenous variables. - They offer access to techniques like sample weights and custom loss functions. *Don't neglect simple baselines though! (**3:45**)* - Simple statistical models can be surprisingly effective. - Ensure the uplift from machine learning justifies the added complexity. *Forecasting with machine learning (**4:15**)* - Convert time series data into a table with features and a target variable. - Use past values of the target variable as features, ensuring no data leakage from the future. - Include features with known past and future values (e.g., marketing spend). - Handle features with only past values (e.g., weather) by using alternative forecasts or lagged versions. - Consider static features (metadata) to capture differences between groups of time series. *Multi-step forecasting (**8:07**)* - Direct forecasting: Train separate models for each forecast step. - Recursive forecasting: Train a one-step ahead model and use it repeatedly, plugging forecasts back into the target series. *Cross-validation: Tabular vs Time series (**11:32**)* - Randomly splitting data is inappropriate for time series due to temporal dependence. - Split data by time, replicating the forecasting process for accurate performance evaluation. *Machine learning workflow (**13:00**)* - Time series forecasting workflow differs significantly from classification/regression tasks. - Feature engineering and handling vary at predict time depending on the multi-step forecasting approach. *Feature engineering for time series forecasting (**14:47**)* - Lag features: Use past values of target and features, including seasonal lags. - Window features: Compute summary statistics (e.g., mean, standard deviation) over past windows. - Nested window features: Capture differences in various time scales. - Static features: Encode categorical metadata using target encoding, being mindful of potential target leakage. *Overview of some useful libraries (**27:01**)* - tsfresh: Creates numerous time series features from a data frame. - Darts and sktime: Facilitate forecasting with tabular data and offer functionalities like recursive forecasting and time series cross-validation. *Forecasting with tabular data using Darts (**28:04**)* - Example demonstrates forecasting with lag features and future known features on single and multiple time series. disclaimer: i used gemini 1.5 pro to summarize the youtube transcript.
@Neilstube356
@Neilstube356 Жыл бұрын
Great talk! How would account for availability in your model? For example let’s say a SKU was out of stock for a portion of the training period. This could result in the sale lag feature being low for the out of stock SKU and high for substitute SKUs that were in stock.
@hurfable
@hurfable Жыл бұрын
you can create a dummy boolean variable feature.
@aliwaheed906
@aliwaheed906 Жыл бұрын
Very informative and intriguing talk. I've been using SARMIAX and things like fbprophet for time series forecast. I have a question about the value of the ML approach. Considering there is a host of things you need to account for while modeling a time-series problem as an ML problem, is it actually that significantly better than traditional algorithms? Is this production-grade stuff or is this in early experimental stages? I must admit the ML approach sounds way more interesting than what I've been doing for the past few years.
@umitkaanusta
@umitkaanusta Жыл бұрын
*by ML models, I mean the tree based ML models here
@mingilin1317
@mingilin1317 Жыл бұрын
I have a question. If I have a time series data for a market, and the data is from 2012 to 2022. now I need to forcast the number of customer that visit the store. But from 2020 to 2022 ,because of COVID19, the number of customer has drop a lot. for this case, If I use last 30% data(from 2019 to 2022) to testing. Model can't get any data that influences by COVID19 when model training (all of them use to test) Isn't that make forcast mape very high? how should I do for this case? (sorry for my poor english)
@yogiekusumah1148
@yogiekusumah1148 Жыл бұрын
Is anybody ever compared model result using same dataset and same parameters from sktime and Darts? for example ARIMA model from both packages. I've try it, and both models gave a different MAPE result. I hope i have made a mistake in my code.
@pranavkhatri9564
@pranavkhatri9564 10 ай бұрын
can we perform this with stock data with models such as Linear Regression ?
@5112vivek
@5112vivek Жыл бұрын
how is y_train_all defined in the last example?
@onlineschoolofmath37
@onlineschoolofmath37 6 ай бұрын
Awesome lecture! I just have one question @32:38, Kishan mentions that we may have different time indexes for different groups can be different which is fine. But the original consolidated data (all groups included) has continuous time stamps whereas when we consider different groups, there may be gaps in the time stamps. Would you still consider them as time series? Will the rest of the process work normally under these circumstances?
@AhmedThahir2002
@AhmedThahir2002 9 ай бұрын
Hi, does anyone know how to implement the recursive forecasting that he did in Darts using sktime. I couldn't really find an intuitive explanation online.
@py.master
@py.master 9 ай бұрын
if you are imputing mean from your training set in place of a missing datapoint, does that mean that the imputed datapoint does not change your model estimation anyway as predicted model passes through mean of variables anyway? I dont think it is information leakage in this way, it is just saying ignore this datapoint
@d.p.1980
@d.p.1980 3 ай бұрын
Enyone tried to apply this DART model on real world data? My MAPE score show me 26% ;-(
@marciamarquene5753
@marciamarquene5753 8 ай бұрын
1:41 gente vê a gente vê a gente
@masaeed44
@masaeed44 9 ай бұрын
Have you used Darts ever? From Darts I got "ValueError: `lags` must be strictly positive. Given: -1."
Feature Engineering for Time Series Forecasting - Kishan Manani
1:02:14
DataTalksClub ⬛
Рет қаралды 23 М.
Khóa ly biệt
01:00
Đào Nguyễn Ánh - Hữu Hưng
Рет қаралды 21 МЛН
Always be more smart #shorts
00:32
Jin and Hattie
Рет қаралды 40 МЛН
Я нашел кто меня пранкует!
00:51
Аришнев
Рет қаралды 1,4 МЛН
The Bayesians are Coming to Time Series
53:17
AICamp
Рет қаралды 22 М.
Feature Engineering Secret From A Kaggle Grandmaster
22:23
Forecastegy
Рет қаралды 34 М.
Nixtla: Deep Learning for Time Series Forecasting
35:12
Databricks
Рет қаралды 21 М.
Challenges in Time Series Forecasting
44:25
Databricks
Рет қаралды 7 М.
What is Time Series Analysis?
7:29
IBM Technology
Рет қаралды 168 М.
Feature Engineering Techniques For Machine Learning in Python
47:58
Time Series Talk : Autocorrelation and Partial Autocorrelation
13:16
Урна с айфонами!
0:30
По ту сторону Гугла
Рет қаралды 8 МЛН
В России ускорили интернет в 1000 раз
0:18
Короче, новости
Рет қаралды 905 М.
Will the battery emit smoke if it rotates rapidly?
0:11
Meaningful Cartoons 183
Рет қаралды 37 МЛН