Correlations and Covariance in R with Example | R Tutorial 4.12

Correlations and Covariance in R with Example | R Tutorial 4.12 | MarinStatsLectures

Рет қаралды 207,968

MarinStatsLectures-R Programming & Statistics

11 жыл бұрын

Correlations and Covariance in R with Example: Learn how to calculate Pearson's correlation, Spearman's rank correlation, Kendall's rank correlation, and covariance in R (with example). 📝 Find R practice dataset (LungCapData) here: (statslectures.com/r-scripts-d... )
👍🏼Best Statistics & R Programming Language Tutorials: ( goo.gl/4vDQzT )
►► Want to support us? You can Donate (bit.ly/2CWxnP2), Share Our Videos, Leave Comments or Give us a Like!
!
In this R video we will learn how to calculate Pearson's correlation, Spearman's rank correlation, and Kendall's rank correlation, create confidence intervals and hypothesis tests and calculate Covariance in R.
Here we will be using "cor", "cor.test","cov" ,"pairs", method”, functions and arguments in R and more.
Pearson’s correlation is a parametric measure of the linear association between two numeric variables, Spearman’s rank correlation is a non-parametric measure of the monotonic association between two numeric variables, Kendall’s rank correlation is another non-parametric measure of the association based on concordance or discordance of x-y pairs. Covariance is a measure of the joint variability of two random variables.
Table of Content:
0:00:08 When should we use Pearson's correlation in statistics and in research?
0:00:16 When should we use Spearman's rank correlation in statistics and in research?
0:00:24 When should we use kendall's rank correlation in statistics and in research?
0:00:54 how to access the help menu in R statistical software for correlation functions
0:01:05 how to produce a scatterplot in the R programming language to explore the relationship between variables using "plot" function
0:01:39 how to calculate the correlation between variables using the "cor" function in R
0:01:46 how to calculate Pearson's correlation using "method" argument in R
0:02:17 how to calculate Spearman's rank correlation using "method" argument in R
0:02:24 how to calculate kendall's rank correlation in R using "method" argument
0:02:34 how to produce a confidence interval and test the hypothesis for the correlation in R using the "cor.test" function
0:03:21 how to calculate the p-value when there are exact values in dataset using "exact" argument in R
0:03:42 how to change the alternative hypothesis using the "alt" argument in R
0:04:03 how to change the confidence level using the "conf.level" function in R
0:04:13 how to calculate the covariance in R using the "cov" function
0:04:27 how to produce all possible pair-wise plots using the "pairs" function in R
0:04:50 how to produce a "pairs" plot only for some of the variables in the dataset by sub-setting data using square brackets in R
0:05:26 how to produce a correlation matrix in R using the "cor" function and "method" argument
0:05:37 how to deal with categorical variables in the dataset when creating a correlation matrix by subsetting data using square brackets in R
0:06:18 how to produce the covariance matrix using the "cov" function in R
These video tutorials are useful for anyone interested in learning data science and statistics with R programming language using RStudio.
► ► Watch More:
► Intro to Statistics Course: bit.ly/2SQOxDH
►Data Science with R Course bit.ly/1A1Pixc
►Getting Started with R using RStudio (Series 1): bit.ly/2PkTneg
►Graphs and Descriptive Statistics in R using RStudio (Series 2): bit.ly/2PkTneg
►Probability distributions in R using RStudio (Series 3): bit.ly/2AT3wpI
►Bivariate analysis in R using RStudio (Series 4): bit.ly/2SXvcRi
►Linear Regression in R using RStudio (Series 5): bit.ly/1iytAtm
►ANOVA Concept and with R using RStudio bit.ly/2zBwjgL
►Hypothesis Testing: bit.ly/2Ff3J9e
►Linear Regression Concept and with R Lectures bit.ly/2z8fXg1
Follow MarinStatsLectures
Subscribe: goo.gl/4vDQzT
website: statslectures.com
Facebook: goo.gl/qYQavS
Twitter: goo.gl/393AQG
Instagram: goo.gl/fdPiDn
Our Team:
Content Creator: Mike Marin (B.Sc., MSc.) Senior Instructor at UBC.
Producer and Creative Manager: Ladan Hamadani (B.Sc., BA., MPH)
These videos are created by #marinstatslectures to support some Statistics and R Programming courses at The University of British Columbia (UBC) (#IntroductoryStatistics and #RVideoTutorials for Health Science Research), although we make all videos available to the everyone everywhere for free.
Thanks for watching! Have fun and remember that statistics is almost as beautiful as a unicorn!
#rprogramming #statistics #Rstats

Пікірлер: 91

@marinstatlectures 5 жыл бұрын

👋🏼In this #R video we learn working through examples to calculate Pearson's correlation, Spearman's rank correlation, and Kendall's rank correlation, create confidence intervals and hypothesis tests and calculate Covariance in R using various functions and arguments. Make sure to download the R practice dataset (LungCapData) (statslectures.com/r-scripts-datasets ) to practice your new skills! #Pearson’s correlation is a parametric measure of the linear association between two numeric variables, #Spearman’s rank correlation is a non-parametric measure of the monotonic association between two numeric variables, #Kendall’s rank correlation is another non-parametric measure of the association based on concordance or discordance of x-y pairs. #Covariance is a measure of the joint variability of two random variables. May all your learnings be #statistically and #scientifically significant 😎🦄

@adamrupnik697 7 жыл бұрын

Awesome videos! I have relied on them throughout my masters, and recommend them to undergraduate students who are starting to learn R. Thank you very much!!!

@hristinastankovic2283 7 жыл бұрын

Hi Mike, vielen, vielen Dank für die so tollen Tutorials. Ich bin ein Fan aus Deutschland.

@tilia-zj7vj 10 жыл бұрын

Thank you for the tutorials, they are extremely helpful! Great job!

@aurorasart9458 3 жыл бұрын

Thank you so much! Your video is really clear and easy to follow. It helped me a lot to solve a university´s task in 2 minutes :D

@talkohavy 7 жыл бұрын

Great tutorial. That helped so much :) Thank you!

@marinstatlectures 10 жыл бұрын

Thanks Gustavo! we take a lot of pride in them, and are happy with the way they're turning out!

@boukailsameh3107 6 жыл бұрын

Thank you Mike, super useful :)

@GustavoMontanha 10 жыл бұрын

Great videos Mike, thanks a lot for what you're doing!

@gustavomattos7669 6 жыл бұрын

Really helpful! Cheers

@missghani8646 2 жыл бұрын

short and simple andddd HELPFUL. I just hope that these codes run my computer now

@hawamohdanuar2753 8 жыл бұрын

Hi Mike, I was really helpful since im the 1st batch learning R studio in my U. watching ur vid made me want to be like u. help other with knowledge

@marinstatlectures 8 жыл бұрын

thanks +Hawa Anuar !

@midnightsantaful 5 жыл бұрын

Thank you so much for this series! It is super helpful!

@marinstatlectures 5 жыл бұрын

nina alison you are very welcome!

@alirezamoslemzade 4 жыл бұрын

I went through a lot of videos on R, yours is the most helpful. Thanks so very much.

@marinstatlectures 4 жыл бұрын

we like to think so ;) thanks!

@dogandogan3384 5 жыл бұрын

Thanks for the help

@gedimarbarbosa6403 8 жыл бұрын

Hi Mike! Thank you very much for your videos, they are very helpful!

@marinstatlectures 8 жыл бұрын

you're welcome +Gedimar Barbosa !

@amit4help 10 жыл бұрын

Extremely helpful to me. Thanks Marin a lot for posting these videos.

@marinstatlectures 10 жыл бұрын

you are welcome amit kumar subudhi :)

@tarunkeswani2078 10 жыл бұрын

. .

@bimalrajshrestha9417 4 жыл бұрын

Thank you very much. Its highly appreciated tutorial to the beginners like me. Thank you once again.

@marinstatlectures 4 жыл бұрын

you're welcome

@manjunathamanju8390 7 жыл бұрын

awesome explanation sir

@eileenboswell1507 5 жыл бұрын

thank you so much for saving my life!! xoxo

@marinstatlectures 5 жыл бұрын

you're welcome :)

@SilvanaBuilesG 7 жыл бұрын

Hi Mike! Hope you're fine :) . Mike , which command do you use in R to check for the correlation between categorical variables?

@elghark 7 жыл бұрын

How did u manage to plot "pairs(LungCapData)" for those not numeric columns? I tried to convert them with "as.numeric" but it returns "NAs" values for yes-no. In the video You didn't do any changes....so how did u do that?

@bridgettsmith7206 Жыл бұрын

Thanks

@syedasidratulmuntaha898 9 жыл бұрын

do i need to convert integer to number inorder to calculate the correlation?

@MrMotocross23 6 жыл бұрын

Thanks for the help! Also your voice sounds exactly like John Mulaneys haha

@marinstatlectures 6 жыл бұрын

thanks, i guess :)

@marcoventura9451 3 жыл бұрын

Very interesting. Is there a stat lesson on correlation and covariance?

@MariMorenaM 10 жыл бұрын

what if i want to make a correlation between the columns of X and the columns of Y?

@marinstatlectures 10 жыл бұрын

HI Mariana Morena, im assuming in your question here that both X and Y are matrices, and that they have the same number of rows (but may have a different number of columns) and you want the correlation between the columns of X and the columns of Y. to do so, you can use "cor(x,y)" and this will return a correlation matrix...the element in row i, column j of that matrix will refer to the correlation between the elements in column i of X and column j of Y. hope that helps, and thanks for watching!

@createwhatyouwant2 7 жыл бұрын

I'm trying to use the correlation function using two variables (or columns) and it gives back the error: Error in is.data.frame(y) : object 'Uturn" not found Uturn is one of the columns and the U is capitalized. Not sure what to do.

@CHURCHISAWESUM 5 жыл бұрын

Make sure you attach the dataset.

@yanishappadoo9067 4 жыл бұрын

given that i want to see the correlation between Number of wins and Age..how to add two columns for number of wins(ie fighter A and fighter B) to scatterplot it to the age of fighters.

@TiagoSilva-gb4kv 5 жыл бұрын

Hi great video. Thank you for help. But i have a question. I have like 40 variables. I want to check the spearman correlation between multiple independent variables and the depedent variable. Is there a way to make it automatic? Or i have to do it 1 by 1 like you did it in the video between age and lungcapacity?

@marinstatlectures 5 жыл бұрын

Hi, if you use * cor(data, method="spearman")* , R will return the Spearman correlation for all pairs of variables (a correlation matrix). you can just focus on the ones related to Y. also, if there are categorical variables (factors), it won't calculate the correlation...you will most likely have to remove those first

@TiagoSilva-gb4kv 5 жыл бұрын

@@marinstatlectures ok thank you :)

@IllusiveMan343 8 жыл бұрын

Hi - Mike. Could you explain how you would do this with a matrix of data such as an OTU table? I have linked a image of the table drive5.com/usearch/manual/qiime_classic.gif. Basically it is a tab delimited file containing samples as columns, and the taxa of interest as horizontal rows. Each cell is a measure of abundance of each taxa for a particular sample. I am interested in doing a correlation between two OTU tables. Thanks.

@jamilahmedchw 8 жыл бұрын

Hey Marin, these videos are really helpful. Thanks. Is there any book about R?

@marinstatlectures 8 жыл бұрын

Hi +Jamil Ahmed Chowdhury (Probal) , thanks! if you're asking if I have created any book for R, i have not. if you're just asking about books for R in general, there are many out there...a good one would depend on your level and what you are looking to learn, etc.

@muhammadsulaiman6 8 жыл бұрын

what is the range of covariance?

@tuanandiilhamtuannordin7073 7 жыл бұрын

hi mike...thanks for tutorial... however i would like to ask spearman's rank test.. do we need sort the variables??? cuz usually we need to rank the variable first b4 run analysis...

@marinstatlectures 7 жыл бұрын

Hi +andi ilham , no you don't need to sort anything (R does that in the background when calculating spearman's rank correlation)

@gosiama480 4 жыл бұрын

How can i convert categorical data into numeric so I could calculate the correlation?

@andreea-alexandrat3424 7 жыл бұрын

Hey Mike, Can you please advise me what type of statistics would be best suited for analysing the effect of 22 essential oils (in 3 concentrations) on the growth of a fungus (in mm) for a period of 40 days and also correlating it with the growth of the untreated fungus? Hope it doesn’t sound too hectic, I am in a huge need of help...

@talkohavy 7 жыл бұрын

Customarily, one should ask a simple question, and not: "Hey mike, I have a project to make, can you do it for me?"

@andreea-alexandrat3424 7 жыл бұрын

I was just asking which method is best for my situation since Mike really knows stuff and i know nothing, yet. Don't give my words another meaning, please :).

@kanikapachori3504 8 жыл бұрын

Hi Mike, Great videos! I have a doubt in converting the data type of my variables.For eg. I am trying to convert a factor variable into numeric using as.numeric() command but when I am applying any data manipulating operation like sum or mean it is showing null.Please help. Thanks

@marinstatlectures 8 жыл бұрын

Hi +kanika Pachori , the first thing to ask is how are the categories of the factor coded? if they are categories of 1,2,3,... then you can convert the to numeric. if it is something like category A, B, C,... then R wont be able to convert those to numeric. you should try using the command *x

@georgem3197 4 жыл бұрын

If the P value is the significance, what does the rho value mean?

@kanikapachori3504 8 жыл бұрын

Hi Mike, yes it is coded as 1,2,3 as I am converting those to numeric and checking the class it is showing that it is numeric but if I am doing any manipulation on the variable the answer is coming out to be zero.

@marinstatlectures 8 жыл бұрын

Hi +kanika Pachori , I'm not really sure what the issue is...these things can be hard to troubleshoot from a distance. are there any missing values? also, you've mentioned both that it is coming out as zero as well as that it is coming out as Null. these are different, and the error would be different d4pending on if you're getting a 0 or a Null. if you give me more info i may be able to figure it out...but as i mentioned, sometimes things like this can be difficult to figure out from a distance.

@Johnny20110914 8 жыл бұрын

hi Mike, May I ask how to compute covariance matrix, for example ,I have a series of data, and I would like to have a covariance matrix, what should i do

@marinstatlectures 8 жыл бұрын

Hi +Johnny Ni , you can calculate covariance using the *cov* command. if it is for just 2 variable, then you can use *cov(x,y)*, and if you data a dataset "data" with numeric variables, you can calculate the covariance matrix for the data using *cov(data)*...this will return the covariance matrix, for all pair-wise covariances.

@mubarak198226 8 жыл бұрын

Hi Mike Thank you very much really great vidieos. Do you have videos on how to perform Meta-analysis using R?

@marinstatlectures 8 жыл бұрын

Hi +Mubarak Mashrah , we don't have any videos for that, but id recommend checking out the "meta" package for R.

@mubarak198226 8 жыл бұрын

Thanks alot

@AC-si7dz 10 жыл бұрын

Hi, these are my variables, mpg ,cylinders, displacement, horsepower ,weight, acceleration ,model_year, origin the for is giving me the error 'x' must be numeric please suggest what should i do. I tried cor(mpg[,3:6]) still i am getting the error

@marinstatlectures 10 жыл бұрын

Hi, ive answerd you in your other post about this...check that one out.

@jeroenvankasteren9338 8 жыл бұрын

Hello Mike! Thanks you for the video's. I was hoping you can explain to me how I could know if something is parametric or non-parametric. Google won't help me any futher and I thought maybe you could explain. Gr Jeroen

@marinstatlectures 8 жыл бұрын

Hi +Jeroen van Kasteren , sure, here's the main difference. parametric methods use "parameters"...a few numbers, like the mean/SD while non-parametric methods do not use parameters, and tend to rely on things like "ranks" of values, rather than their actual values. parametric methods often will use a mean/sd (or things like that) in their calculation, and often rely on assuming some probability distribution, while non-parametric methods are usually distribution free. with correlation, the parametric method (Pearson's) uses the mean/sd of X and Y in its calculation, while the non-parametric one (Spearman's) ranks the X and Y values, and uses the ranks (not the actual values) in its calculation. as another example, linear regression is parametric. the line is defined by 2 parameters, the slope and the intercept. something like a smoother (a spline, or loess, or things like that) are non-parametric, and are not easily summarized by a few simple numbers. parametric methods tend to use all of the data, are more powerful, but require more assumptions,...and tend to rely on large samples. non-parametric methods ten to use ranks rather than the actual numeric values, are less powerful, but can be used with smaller sample sizes, and make far fewer assumptions. as a general guideline, if the assumptions for a parametric method are met they are generally a better choice (but those assumptions are often not met). for example, if the relationship between a numeric X and numeric Y is roughly linear, then linear regression is a good choice...but if the relationship is non-linear (and not a smooth curve, like something log-linear) then a non-parametric method, such as a spline, may be a better choice. i hope that helps clarify the difference a bit. i think the best way to really separate the two is to look at example of both (the parametric method, and a non-parametric alternative to it) and compare and contrast them to see the main differences. i really find that it is always easier to understand things like these with concrete examples, rather than abstract ideas like i have tried to describe above.

@drallarreivax419 8 жыл бұрын

really cool and instructive! I learned a lot of R during my baccalaureate but I haven't had the chance to practice very often. I know now that cor ( ) is for the pearson correlation but can you tell me the command for the coefficient of determination? I can't find it.

@marinstatlectures 8 жыл бұрын

Hi +ledzep for life , if you fit a linear regression model (we have a whole series of videos talking about linear regression) the R output will return that for you. in the case of simple linear regression (just one X variable) the coefficient of determination will just be pearson's correlation squared, (e.g.) cor(x,y)^2

@drallarreivax419 8 жыл бұрын

+MarinStatsLectures Thank you for the answer! Now my problem is, i'm trying to type it on my plot using the text command but I can't put the exponent on top of the r. Can you tell me how to put an exponent while typing in the consol?

@marinstatlectures 8 жыл бұрын

Hi ledzep for life , i don't believe you can do that (or if you can, i don't know how). i would just write it as "R2" or write out the word "R-square"

@AC-si7dz 10 жыл бұрын

Hi, How do I plot the data for say column (1,3,4,5,6) that is i don't want column 2

@marinstatlectures 10 жыл бұрын

Hi Abhijit Choudhury , im guessing you want to make the pairs plot? it will depend on what you have named your data set in R...i will suppose that you have named it *"data"*. if you want the pairs plot for columns 1,3,4,5,6, you can use: *pairs(data[ , c(1,3,4,5,6)])* . This will make the pairs plot of the data, for only variables in columns 1,3,4,5,6. You could also make the pairs plot for the data, removing only column 2 using: *pairs(data[ , -2])* . This will remove column 2, and keep all other columns of the data set. hope that helps...

@pavanchunduri4222 4 жыл бұрын

Hello, Sir, I have a doubt when I import the dataset and checking the class type of Age, its showing "OBJECT AGE IS NOT FOUND" whereas the object is clearly mentioned in a dataset. Pls, help me how to resolve this.

@marinstatlectures 4 жыл бұрын

you either need to type *class(data.name$Age)*, or you have to *attach(data.name)* and then you can call on the variables by name. the first is the better way to go...

@Nwilliams1996 5 жыл бұрын

What does attaching the data do?

@marinstatlectures 5 жыл бұрын

If you attach them you can call in variables directly, if you don’t then you need to use the $. You can either use things like *mean(data$variable)* or if you attach then you can use *mean(variable)* It is usually better to work without attaching, but when first learning R, it may be simpler to attach, especially if one doesn’t have much experience with coding

@brettknoss486 5 жыл бұрын

Is there a command to calculate lambda for ordinal and nominal variables?

@marinstatlectures 5 жыл бұрын

sure, it is explained pretty clearly here: www.rdocumentation.org/packages/DescTools/versions/0.99.19/topics/Lambda

@brettknoss486 5 жыл бұрын

@@marinstatlectures thanks. Only thing is I can't get DescTools to wor in 3.3.3 and 3.5.2 can't be used in SPSS 25 🤔. I'll still take a look.

@marinstatlectures 5 жыл бұрын

oh, the joys of trying to get different software and different versions of software to work with each other! good luck sorting that out ;)

@yanishappadoo9067 4 жыл бұрын

> cor(R_age,R_wins,method="spearman") [1] NA

@kppphd 5 жыл бұрын

Hello - how can we please get the pairwise correlations in a table with significance levels? (i.e., 0.001*** 0.01** 0.05* 0.10+). Thanks for all your videos, they are really amazing.

@marinstatlectures 5 жыл бұрын

Hi, there likely is a package out there that can do this, but i dont know of it. i would just build it manually using *cor(data)* to get the table of all pairwise corrections. then, id use *cor.test(x1,x2)* , *cor.test(x1,x3)* for all pairs to get the significance levels. if you have a LOT of variables to do this for you can always use 2 loops to do it: for (i in 1:dim(data)[2]){ for (j in (i+1):dim(data)[2]{ cor.test( data[,i] , data[,j] ) } } this would run through producing the test of significance for each pair of variables. you can always automate this more, and have it extract just the p-values using something like: cor( data[,i] , data[,j] )$pvalue

@kppphd 5 жыл бұрын

@@marinstatlectures Thank you Mike. You seem to suggest making 'functions' and I am not familiar with this yet but I will check it immediately. Thx, again.

@marinstatlectures 5 жыл бұрын

you're welcome. and just a note that writing a function isn't completely necessary. you can get all needed info by using *cor.test(x1,x2)* , *cor.test(x1,x3)* , *cor.test(x1,x4)*, ..., *cor.test(x2,x3)*,... writing a function (or just using a few loops) would only really be worth it if you have to do this for a LOT of variables, or you will be doing it many times. if it is just for this one instance, id just do it the "long" way as i just describe in bold

@kppphd 5 жыл бұрын

@@marinstatlectures Thank you again. The truth is I usually have multiple papers (3-4) I work on and each of them usually has 15 variables, so there are a lot of paired correlations I need to run. Above all, I want to be able to export them as a table to MS Word afterwards. Let me dig into your suggestion a little bit more. BTW, I subscribed to your channel and I appreciate your help!

@tariqbadshah8731 4 жыл бұрын

Hi Mike i a master student i havent study about R in my bachelor and the same we have no course about R in our master program i was scared from R when my supervisor told me to process your data on R, your videos is safe my degree thanks for your knowledgeable videos and hope to learn much about R from your videos

@marinstatlectures 4 жыл бұрын

great to hear, glad you found them helpful!