ETL | AWS Glue | AWS S3 | Data Cleansing | Transforming data with AWS Glue in ETL workflows

Рет қаралды 4,487

4 ай бұрын

===================================================================
1. SUBSCRIBE FOR MORE LEARNING :
/ @cloudquicklabs
===================================================================
2. CLOUD QUICK LABS - CHANNEL MEMBERSHIP FOR MORE BENEFITS :
/ @cloudquicklabs
===================================================================
3. BUY ME A COFFEE AS A TOKEN OF APPRECIATION :
www.buymeacoffee.com/cloudqui...
===================================================================
In this lab you will learn about AWS Glue, which is a serverless data integration service that makes it easier to discover, prepare, move, and integrate data from multiple sources for analytics, machine learning (ML), and application development. You can use a crawler to populate the AWS Glue Data Catalog with tables. This is the primary method used by most AWS Glue users. A crawler can crawl multiple data stores in a single run. Upon completion, the crawler creates or updates one or more tables in your Data Catalog. Extract, transform, and load (ETL) jobs that you define in AWS Glue use these Data Catalog tables as sources and targets. The ETL job reads from and writes to the data stores that are specified in the source and target Data Catalog tables.
example data : github.com/RekhuGopal/PythonH...
#awsglue #etl #datatransformation #s3 #cloudquicklabs #crawler #etljob
#dataengineering #data #datacleansing #datamodificada

Пікірлер: 16

@yadi4diamond 7 күн бұрын

excellent

@cloudquicklabs 6 күн бұрын

Thank you for watching my videos. Glad that it helped you.

@user-mg8ol5gp1j 4 ай бұрын

Excellent video, please make more videos like that on ETL, admire your hard work, very smooth explanation…… Thanks

@cloudquicklabs 4 ай бұрын

Thank you for watching my videos. Indeed , I shall make videos on data engineering. Glad that my videos helped you.

@sujithadas9844 4 ай бұрын

Loved your series on AWS Glue.. nicely explained.. 😊 Please upload more scenarios with other transformations such as join and filter ! Subscribed already.. thank you !!

@cloudquicklabs 4 ай бұрын

Thank you for watching my videos. Glad that it helped you. Indeed I shall make more videos around this topic ahead.

@rashmishah5682 4 ай бұрын

Thanks a lot....please make more videos with different transformation and also the use of cloudwatch and lambda with it....

@cloudquicklabs 4 ай бұрын

Thank you for watching my videos. Glad that it helper you. I shall make more videos around this topic and Thank you for giving inputs.

@WolfmaninKannada 2 ай бұрын

Excellent brother❤

@cloudquicklabs 2 ай бұрын

Thank you for watching my videos. Glad that it helped you.

@bellamkondapushpashree4923 2 ай бұрын

It is showing failed at the end after running the job ...can you make a video on how to debug this issue

@cloudquicklabs 2 ай бұрын

Thank you for watching my videos. You can find ETL pipeline logs if you go pipeline runs or you can also find the same at Cloudwatch log groups.

@faisalmali3809 2 ай бұрын

How can I run multiple glue jobs.

@cloudquicklabs 2 ай бұрын

Thank you for watching my videos. Do you mean creating multiple job or multi threading within a job ?

@NandhuVlogs. 3 ай бұрын

Bro You create Glue job by using python script...... In real time..... Visual ETL is Never preferred......

@cloudquicklabs 3 ай бұрын

Thank you for watching my videos. Indeed , I would creating videos on creating job via python script. But this video is to focus on ETL topic.