Predict Bitcoin Prices With Machine Learning And Python [W/Full Code]

Рет қаралды 38,632

Күн бұрын

We'll predict the future price of Bitcoin using historical price and sentiment data. We'll use data on the USD/BTC price from Yahoo Finance, along with data from Wikipedia about edits to the Bitcoin page.
We'll merge and combine this data, then use it to train a random forest model that will tell us if Bitcoin prices will increase or decrease tomorrow. We'll then switch to an XGBoost model and better predictors to improve accuracy.
We'll develop a backtesting system and use a robust error metric so we can tell if the algorithm is performing well. This system will also be able to make next-day predictions.
This project can be extended to other cryptocurrencies, or be used to predict multiple days ahead.
You can find the full code and README here - github.com/dataquestio/projec... .
Chapters
0:00 Project overview
02:33 Downloading edits from Wikipedia
04:57 Finding the sentiment of page edits
14:17 Converting sentiment data into a DataFrame
19:32 Downloading Bitcoin price data
23:49 Preparing data for machine learning
27:38 Training our baseline ML model
32:07 Evaluating error with backtesting
37:05 Using an XGBoost model
39:12 Improving precision with trends
46:03 Generating future predictions
46:56 Next steps to improve the model
---------------------------------
Join 1M+ Dataquest learners today!
Master data skills and change your life.
Sign up for free: bit.ly/3O8MDef

Пікірлер: 61

@user-gu4dt4tn8q Жыл бұрын

Vik, thanks for creating these and sharing your knowledge.

@mahmoodhossainfarsim6292 Жыл бұрын

Great one Vik. please keep them coming

@DanielGarcia-uq8yz Жыл бұрын

For anyone stuck trying to merge the wiki and btc history data frames, if you get a time-zone error (Cannon merge tz-naive with tz-aware), you will need to edit the datetime index of the btc history data frame after converting it to a datetime object to make it time-zone naive. Here was my fix: btc.index = pd.to_datetime(btc.index) [Original Code from Vik] btc.index = btx.index.tz_localize(None) [Fix to make index time-zone naive]

@adityagavali3158 11 ай бұрын

eres una leyenda, Muchas Gracias tio

@brianmcclain1916 4 ай бұрын

is "btx" a typo..you must mean "btc"

@it_s_just_me1030 Жыл бұрын

Awesome video, I learned a lot. Thanks!

@cacus_ Жыл бұрын

awesome videos! keep it up! have you tried using transformers for timeseries? also, in your experience, what's your opinion on training using technical indicators (let's say RSI, CCI, stochastic, etc) and VIX, rates (could be us 10 years yield for example)?. an idea I'm exploring is trying to figure out how to use also the rolling correlation with a set of assets (let's say you are fiting for QQQ comparing or correlating with dxy, gold or any other asset to check if the rolling correlation impacts in any way)... take care!

@ricebarbie 5 ай бұрын

Thank you for the video. It helped me alot better than my boot camp course.

@ahmetselmanayberkbozkurt6898 5 ай бұрын

if you had other variables such as indices, commodity prices would you need to take differences of the variables in order to make the dataset stationary?

@pranitchand Жыл бұрын

Amazing video. Thank you so much for sharing, I was able to modify it and develop it further to predict the price of Ethereum! Thanks for the inspiration.

@Why_interested 3 ай бұрын

Can u provide me help regarding final year project on bitcoin price prediction in btech

@nguyenduyta7136 Жыл бұрын

Great sir! Thank so much

@atenea3930 Жыл бұрын

For those stuck in cell 7, be aware that one data entry has no comment (20:44, 10 September 2022‎ Staxringold) You have to delete that entry with "del revs[x]" being x the position. It depends on when you obtain the data.

@Dataquestio Жыл бұрын

Thanks for finding the issue! Replacing the line `comment = rev["comment"]` with `comment = rev.get("comment", "")` should also work. This will just fill in the comment with an empty string if it can't be found.

@ch-qk8dk Жыл бұрын

can you give me the exact code?

@atenea3930 Жыл бұрын

@@ch-qk8dk the Dataquest's solution is better, it's in my comment

@ch-qk8dk Жыл бұрын

@@atenea3930 okok thanks

@ashanbandaranayeke9344 29 күн бұрын

hi! enjoying making this great video, it's the first time I incorporated sentiment analysis at all (in my ML journey). *I'm having a problem with the history attribute/method of yf!* Anyone else?

@sunday-thequant8477 10 ай бұрын

are you putting look ahead bias using trend variable, because you sum the target and in the current bar you dont have the bar closed so in real time wont have the trend predictor

@atenea3930 Жыл бұрын

I also found that rolling by 15 instead of 30 increases the accuracy in 0.5 points

@Theskiescreator 9 ай бұрын

@@b43f4bd3dz did you manage to get it to work?

@abhinavab5673 Жыл бұрын

line 56:Cannot join tz-naive with tz-aware DatetimeIndex.Any solution?

@mariajanechan5906 Жыл бұрын

Awesome!

@santoshpatil1029 10 ай бұрын

Are you using Jupyter notebook??

@chriscoleman1818 Жыл бұрын

Would/Can you build me one for a certain crypto coin?

@clusterbyblocks 5 ай бұрын

Aren't you leaking data by using the Close Value as a predictor? I mean the algorithm just have to look at this to predict next day. Can you reason about it?

@fsDragonDKlion3000 Жыл бұрын

hello, im trying to run cell 7 and it takes forever to load, i tried the different edits below but it still won't work, is there something else you can advise me to replace it with?

@fsDragonDKlion3000 Жыл бұрын

just realized that its because it's trying to pass through +17000 comments, other than cutting the data (revs) into less than 1000 records, is there a way to process this any faster?

@wmgallo1 Жыл бұрын

the mwclient somehow it gives an error

@kwokcheongyeung665 Жыл бұрын

I can't proceed further in appending (find_sentiment(comment)) to an empty dictionary edit = {}. It keeps loading instead. Can someone help to solve? Thanks!

@Dataquestio Жыл бұрын

Hi Kwok - does the find_sentiment function itself work? It may be that the transformers package isn't installed or configured correctly.

@munchy5667 Жыл бұрын

is it possible to make a multi step forcast with classification models such as this?

@Dataquestio Жыл бұрын

What is a multi step forecast? Do you mean a forecast multiple days ahead? If so, then yes, just change the .shift(-1) to the number of days ahead you want to predict.

@munchy5667 Жыл бұрын

@@Dataquestio yes forcasting for multiple days ahead. In your video your making a prediction for tommorow. What I wanted was to know if I can make predictions for the next 3 days rather than changing the model with multiple targets

@ASH8376 8 ай бұрын

@@Dataquestio I changed the shift-1 to shift-5 what does it show me , prediction for next 5 days ?

@ASH8376 8 ай бұрын

I saw the precision score went up to .70 , what does that mean ?

@nadendlapranaychowdary7703 2 ай бұрын

Can you send the document of this

@Sake679 Жыл бұрын

Hey, mine is not predicting next day

@dongjunpark7006 Жыл бұрын

For the line "if len(edits[key]["sentiments"]) > 0:", I got an error saying that "object of type 'float' has no len()" May I kindly ask how to solve this problem?

@Dataquestio Жыл бұрын

Hi Dong - the "sentiments" key should be a list, but it looks like it is a single number. This would happen if you assigned the sentiment value directly to the key instead of appending to the list.

@yussufolarinde8329 Жыл бұрын

Can I try out these codes on pycharm?

@Dataquestio Жыл бұрын

Yes, you can use any IDE to complete this project.

@ajaychhetri482 Ай бұрын

How much for this project sir ?

@shivanshyadav1041 5 ай бұрын

you are taking yesterday's closing price as tomorrow's closing price that does not make sense...

@dbarbedillo Жыл бұрын

Hi Vik! Hope everything is well! On code cell 7 I keep getting KeyError: 'comment' (The find_sentiment function is working) Would really appreciate some help to solve this error :)

@Dataquestio Жыл бұрын

It looks like calling `page.revisions()` isn't downloading the comment for you for some reason. Try running `page.revisions(prop='ids|timestamp|flags|comment|user')` instead to see if that helps.

@dbarbedillo Жыл бұрын

@@Dataquestio Thank you very much! I greatly appreciate your help!

@dbarbedillo Жыл бұрын

Solved the issue the following way: (code cell 7) edits = {} for rev in revs: date = time.strftime("%Y-%m-%d", rev["timestamp"]) if date not in edits: edits[date] = dict(sentiments=list(), edit_count=0) edits[date]["edit_count"] += 1 for k, v in rev.items(): if k == 'comment': comment = v edits[date]["sentiments"].append(find_sentiment(comment))

@Dataquestio Жыл бұрын

That fix looks great. Someone else investigated and found that there is one revision missing the comment key for some reason. This revision is from after I created the original project, so I didn't see it. You could also replace the line `comment = rev["comment"]` with `comment = rev.get("comment", "")`. This will just fill in the comment with an empty string if it can't be found.

@bryanbuiser2953 Жыл бұрын

@@dbarbedillo How long did it take to run that part of the code?

@aarondelarosa3146 10 ай бұрын

Where's the plot?

@Theskiescreator 9 ай бұрын

did you manage to get it to work?

@slava9298 9 ай бұрын

also got stuck on cell 7 :( the fixes from the comments don't help

@Theskiescreator 9 ай бұрын

did you manage to get it work?

@mytube1000javed 7 ай бұрын

😂ha ha ha. Very funny. Kuch bhi. So u r going to predict the price of Bitcoin. 😅. Very funny. It moves according to whim and fancy of Elon Musk and other big players. Your ML will shamefully fail. So don't even give a try. Anyways great conlmexy😂.

@JoaoRodrigues-yf7xc 2 ай бұрын

Mate, you just arrived on a spaceship from Mars and are a little bit lost... The point of this is to learn how to use ML and to code with jupyterlab. It is in no way focused on actually predicting btc moves.