AWS Glue Tutorial for Beginners [FULL COURSE in 45 mins]

Рет қаралды 240,146

Күн бұрын

ℹ️ johnnychivers.co.uk
☕ www.buymeacoffee.com/johnnych...
📁 github.com/johnny-chivers/glu...
☕ www.buymeacoffee.com/johnnych...
00:00 - Intro
00:36 - Note About Job Tab Locations
01:01 - Course House Keeping
02:11 - Course Overview
03:07 - What is AWS Glue?
04:03 - Why use AWS Glue?
04:44 - Set Up
09:32 - AWS Glue Data Catalog
12:05 - AWS Glue Databases
14:40 - AWS Glue Tables
15:57 - AWS Partitions
18:00 - AWS Glue Crawler
18:33 - Create Table In The Glue Data Catalog
25:37 - AWS Glue Connections
27:19 - AWS Glue ETL Job
34:41 - AWS Glue Triggers
39:23 - AWS Glue Dev Endpoints
40:44 - Outro
In this course we take a look at AWS Glue in full using the console. In under 45mins we go from zero to hero. The course covers the AWS Glue Data Catalog, AWS Glue ETL, AWS Glue Triggers and AWS Glue Dev Endpoints. We use the customer CSV data, available on my GitHub, to create S3 backed tables before ETLing this data into Parquet format.
😎 About me
I have spent the last decade being immersed in the world of big data working as a consultant for some the globe's biggest companies.My journey into the world of data was not the most conventional. I started my career working as performance analyst in professional sport at the top level's of both rugby and football. I then transitioned into a career in data and computing. This journey culminated in the study of a Masters degree in Software development. Alongside many a professional certification in AWS and MS SQL Server.

Пікірлер: 281

@sayakutube Жыл бұрын

you were preparing the video when the entire west world was yelling "Happy New Year"! Great commitment and awesome result!

@TerminatorAyan Жыл бұрын

in 41 minutes, I never knew I can gain this much knowledge and at least be ready with AWS glue as we are in a transformation project. It really helped, thank you so much!

@farhodotaboev3324 4 ай бұрын

If you had watched with 1.25 playback speed, it would take you 33 minutes

@martinvuong6652 Жыл бұрын

Such an amazing tutorial! Thanks for getting this together. Looking forward to more content.

@adeyemitunji343 Жыл бұрын

This the most detailed AWS glue video i have ever seen. Keep up the great work Johnny.

@hankbirkdale2154 Жыл бұрын

One of the best tutorials I've ever seen - let alone on a topic that is not easy.

@Josh-og9eo 2 жыл бұрын

So excited to find this channel. Looking forward to watching all your videos!

@gabesusman4592 Жыл бұрын

this channels a great resource for learning data engineering on aws, it's been a big help. Keep up the good work Johnny!

@roarmarketingconceptsllc5074 Жыл бұрын

This was a great tutorial! I really learned a lot about AWS Glue and I plan to leverage this knowledge to be more effective in my job tasks. Thanks so much, Johnny!!

@asifalimir3043 3 ай бұрын

Such an amazing video. Gives end to end idea about Glue in just 45 mins. Keep it up.

@waeldimassi3355 2 жыл бұрын

Amazing content by far !! Please continue with Glue. Never seen such high quality tutorial :D

@JohnnyChivers 2 жыл бұрын

Thanks for watching

@ThanhTran-wf2jf 2 жыл бұрын

This is one of the best tutorials i've seen in youtube videos. Very well-explained, very useful. Thank you for uploading this!

@JohnnyChivers Жыл бұрын

Thanks for the comment and supporting the channel.

@JulioSerratos Жыл бұрын

This tutorial is pure gold. Thanks

@Pavan-kn5pg Жыл бұрын

One of THE BEST videos on AWS Glue. Thank you Johnny :)

@vedprakash9413 Жыл бұрын

41 minutes was sufficient for me to get hands-on AWS glue. Thanks, Johhny for this awesome tutorial.

@Rk23able 8 ай бұрын

Wonderful and neatly described in such a short video

@ryoutausami6817 Жыл бұрын

Thank you. This is an awesome introduction to AWS Glue for beginners.

@lyndsaymizen1881 2 жыл бұрын

Great intro to AWS Glue for someone who's never seen it before. Needed to know these basics as we're moving to Glue for our data transformation/warehousing and this video does just that! Thanks!

@JohnnyChivers 2 жыл бұрын

Thanks for watching Lyndsay!

@GauravSakhuja 6 ай бұрын

Great tutorial in 40 mins you covered everything essential in Glue 👏

@chamila.fernando.us2fernan663 Ай бұрын

You are Awesome. watching in 2024... ETL steps needs minor updating but I was still able to follow ! Keep up the great work !

@danielboza5747 5 ай бұрын

For me the most amazing thing is that he was working in the morning of January 1st. My respect! 😅

@jairogarcia432 10 ай бұрын

I am using this in 2023, the interface has changed, but this still working, thank you so much for the video.

@bitashamsss 11 ай бұрын

fantastic video, very helpful and useful. Looking forward to seeing more. keep up the great work and thank you so much.

@augustogoldner4267 Жыл бұрын

That was fantastic, mate! Thank you very much for sharing your knowledge!

@venkatrao7868 26 күн бұрын

You are amazing and a natural teacher !!

@diegogiardini Жыл бұрын

It was very usefull for someone who is starting with AWS from scratch! Thank you :)

@EEdgerocks Жыл бұрын

excellent Glue tutorial .. One of the best I have come across which teaches in a composed pace and easy to pickup for anybody

@JohnnyChivers Жыл бұрын

Thanks for watching!!!

@akashdasgupta5088 Жыл бұрын

This is one of the best AWS Glue tutorial that I've come across on KZfaq. Totally worth a like and sub❤️

@JohnnyChivers Жыл бұрын

Thanks for the sub!

@hemantmattoo 8 ай бұрын

Very nice and easy to learn video especially for begineers. Thanks for posting this.

@junaidraza6774 7 ай бұрын

I am not sure, does anyone said this before or not, but i wanted to say, You are a rock start.

@JeffLentz Жыл бұрын

Fantastic! Thank you for putting this together. It helped me a lot.

@sriravi815 Жыл бұрын

Thanks for nice video, it helped me understand glue and setting up the pipeline in aws.

@obedrajugantala3489 Жыл бұрын

Good tutorial. I was able to follow and execute. Thanks Johnny!

@lucasmatborges 7 ай бұрын

Excelent tutorial. I made three others courses before that and only with this that I understead complete this tool

@debarshiacharya3729 8 ай бұрын

This was a really nice video. Lot of learning within a very short time. Thanks

@dbeckett 6 ай бұрын

Finally an aws tutorial that uses "wee" and talks in a NI accent, made me laugh that I finally found a fellow NI person :) followed every word :D

@1209youandme Жыл бұрын

Superb course, covering all features with examples

@JohnnyChivers Жыл бұрын

Thanks for comment.

@Ajay-nc9yn 9 ай бұрын

I have spent many hours on Udemy for Glue Job and Glue data Catalog, but after watching your video. I must say Damn Good Stuff, Sir!

@shadabbigdel5017 5 ай бұрын

Thank you very much for you great explanation and hands-on! It was very useful for me.

@javiermadriz7834 3 ай бұрын

Great video for beginners I hope to build some projects for keep learning

@PURAMSAIPRIYA Жыл бұрын

Thank you for this video, it is very helpful gives a clear glimpse of AWS glue. This is a very important video for me for the interviews :)

@guilhermejf8642 2 жыл бұрын

I just can't believe it's free here on KZfaq! Great content, very well explained. Thanks!

@JohnnyChivers 2 жыл бұрын

Thanks for watching the video and a supporting the channel!

@SungSam-wz2vk 2 жыл бұрын

Awesome. I've never met people like you that you're so young with solid data science & aws skills.

@JohnnyChivers 2 жыл бұрын

Thanks Sung!

@vjsnapp8178 8 ай бұрын

Amazing Content and style of teaching, Thanks a lot for making these videos.

@augustocarrillo5927 Жыл бұрын

I don't like subscribing to people but lit I wouldn't be able to learn aws without you. Best and unique content

@gonzalea35 5 ай бұрын

This was awesome explanation of glue, thanks.

@vrlchebolu Жыл бұрын

Great intro on AWS Glue! Thanks!!

@ryanshuell 5 ай бұрын

Very well done! Thanks for sharing this!!

@sagnikmukherjee5108 Жыл бұрын

Thoroughly enjoyed, Thanks.

@pnk Жыл бұрын

As a skilled Ops Engineer / Developer i have search for infomartion about whats Glue and how its working in the basic and thanks so much for your video, its late me understanding the basic about what Glue are and whats it can be used for, :)

@adarsharora6097 6 ай бұрын

Thankyou so much. Very informative Johnny!

@jeevangangavarapu8683 2 жыл бұрын

Excellent work Johnny. Keep continuing the work .

@JohnnyChivers 2 жыл бұрын

Thanks, will do!

@robdood2 9 ай бұрын

Thanks for this! So much better explained than AWS's own tutorials!

@patriciacafundo1626 7 ай бұрын

Very good course, gave me a good view of glue AWS service

@davidlean8674 2 жыл бұрын

Great & comprehensive coverage of AWS Glue. I was hoping it did much more. Makes you appreciate how much easier, faster & cheaper it is to develop in Azure DataFactory. Where a single pipeline can consume multiple different CSV files, parse the filename, figure out the metadata on load, & redirect the output based on lookup parameters. All graphical or generated via script.

@DataEngUncomplicated 2 жыл бұрын

Hi David, This video did not cover AWS Glue Studio which provides more of a compressive graphical interface that I think would be more comparable to Azure Data Factory.

@StephenGillie Жыл бұрын

Troubleshooting tip for especially complex environments: If the user has access to ALL S3 locations in the table, then Glue will assemble the table for Athena to query. If even one S3 location can't be accessed by the user, then the table won't show up in Athena. Hope this helps complex schema lovers.

@JohnnyChivers Жыл бұрын

Good tip.

@anarossetto2742 Жыл бұрын

wonderful tutorial, great explanation, thank you! Unfortunatly aws glue job console has changed a lot and I could not finished the tuto :(

@maxpayne6625 2 ай бұрын

I feel like now I am zero to a noob. It will take sometime to be a hero :)

@femaledeer 10 ай бұрын

Glad the tutorial took me through the console instead of running code.

@ishabisht3685 2 жыл бұрын

Thank you very much, it was very helpful video to learn end to end AWS Glue. 🙂

@victors9585 8 ай бұрын

Johnny, you ROCK, man!!! 🔥👏🔥👏🔥👏🔥 Thank you so much and please don't stop spreading the wisdom of the GURU!!!! 🙏🙏🙏👍👍👏👏🚀🌟

@vamsikrishna5950 2 жыл бұрын

Awesome content!! Appreciate the effort. Thank you

@ibinabobob-manuel323 5 ай бұрын

Towards the end of the course, I lost my way after we created the parquet file because my version is different from yours and I couldn't just get the visuals on my dashboard. Thank you so much and I will give this a try again some other time.

@nishaddheeraj2 Жыл бұрын

really amazing and fun learning with Johnny :)

@mohamedyasser5285 Жыл бұрын

Fantastic video, going to binge-watch your videos for the next few months!

@JohnnyChivers Жыл бұрын

Thanks for watching.

@ammar9700 2 жыл бұрын

Nicely done and explained. I have done my first AWS Glue job, without any issue... thumbs up bro 👍

@JohnnyChivers 2 жыл бұрын

Fantastic news Ammar!!

@ToddCunningham 2 жыл бұрын

right when i needed this, always amazing content

@JohnnyChivers 2 жыл бұрын

Thanks for watching Todd

@ambhat3953 Жыл бұрын

Very helpful for my studies

@KoEDeath 2 жыл бұрын

Impressive free content. Thank you!

@JohnnyChivers 2 жыл бұрын

Thanks for watching

@user-nt7xg2tm3i 11 ай бұрын

I am enjoying the video, thanks for this resource! I wanted to note that it seems you blurred your S3 buckets early in the video, but at 19:22 and also 21:33 when configuring glue, you do not blur your other S3 buckets. Also 24:49.

@kjewelson Ай бұрын

happy new year

@qadiralidanish7529 7 ай бұрын

Awesome tutorial.

@guilhermmontealto2160 Жыл бұрын

huge fan from brasil, thank you for your great content, it helped me a loooot

@ateshhastam Жыл бұрын

Nice session. Thanks!

@LoveisHell85 9 ай бұрын

Amazing content. Thank you

@muadddib4734 2 жыл бұрын

Excellent content! Hope you keep it coming. I just subbed

@JohnnyChivers 2 жыл бұрын

Thanks for watching.

@AvinnashP Жыл бұрын

excellent video on glue

@PatrickMcDonoughVanWash Жыл бұрын

Highly recommend!

@maryo1134 2 жыл бұрын

Amazing teacher..Many thanks for this

@JohnnyChivers 2 жыл бұрын

Thanks for watching.

@adalke2 2 жыл бұрын

As someone transitioning from a data scientist to DE role, I found this extraordinarily helpful! Subscribed! Thank you!

@arunsar7893 2 жыл бұрын

I am interested in understanding why did you decide to make the transition.

@JohnnyChivers 2 жыл бұрын

Awesome! Thank you!

@kicknotes 10 ай бұрын

Good video. Thanks for this!

@TheSimpGatsby Жыл бұрын

thanks mate, u made my day.

@gabrielegbenya7479 Жыл бұрын

Great content! very educative and informative.

@JohnnyChivers Жыл бұрын

Thanks for watching the video.

@sephiroth7818 11 ай бұрын

They've changed a lot in the last 16 months. Could only follow this til like 60% of the way

@whitewalker7750 Жыл бұрын

great content keep up the good work and I understood every concept thanks

@JohnnyChivers Жыл бұрын

Great to hear!

@ahmeddadjio2003 5 ай бұрын

Amazing video👏

@picassoofai4061 7 ай бұрын

Great Job, Thanks

@i_am_out_of_office_ 6 ай бұрын

Very helpful, thanks :)

@deetechbee 8 ай бұрын

awesome video!

@AdityaSingh0 8 ай бұрын

best video and very informative

@benitinmagnate4937 2 ай бұрын

@25:00 "I'll talk about connections quickly", LOL! That's what AWS Glue, Azure Data Factory, SSIS, Informatica, are all about: CONNECTIONS! You are moving data from a source to a target, and to do that, you need to be connected to both, the source and the target. Basically, you are an S3 guy, LOL!

@balledachandrahas8326 Жыл бұрын

Super helpful.... Thank you so much.

@JohnnyChivers Жыл бұрын

Thanks for watching!

@mgd.kn111 Жыл бұрын

Love this tutorial, thanks!

@JohnnyChivers Жыл бұрын

Thanks for watching!

@DeepakkumarSLatentView 4 күн бұрын

just amazing 🥳

@tullez01 Жыл бұрын

Amigo, muito obrigado pelo vídeo... Ficou ótimo! Abraços! Dear friend, thanks for this video. It's really great, helped a lot... Hugs from Brazil :)

@sany2k8 2 жыл бұрын

Awesome content 👍, please continue with AWS data services.

@JohnnyChivers 2 жыл бұрын

Thanks for watching Sany.

@machinimaaquinix3178 11 ай бұрын

Thank you and well done. Was able to do the course still in June of 2023, though the ETL chapter was a bit of a challenge as AWS has completely redone those screens.

@JohnnyChivers 11 ай бұрын

Great to hear!

@michealdmouse 11 ай бұрын

How did you get past the ETL job chapter? The interface is totally different.

@ravulasandeep2512 Жыл бұрын

Really Awesome!

@bmcseal01 Жыл бұрын

Excellent work.

@JohnnyChivers Жыл бұрын

Thank you! Cheers!

@AkashBhosale-mr7kk 2 ай бұрын

Great Video great efforts. Thanks a lot for detail explanation. When I ran the crawler for 1st time as per video, it did not create partition column, I again created a new crawler with same details using same s3 folders and now it created a partition. What might the possible reason that it failed to detect at 1st time? Any key point to remember during building or some miss that lands us into such situation?