AWS Tutorials - Build Enterprise Scale Python ETL Jobs using AWS Glue on Ray
Рет қаралды 2,160
Жыл бұрынAWS Glue comes with a new engine option called Ray. Ray allows to process large amount of data using python script and python libraries. Ray is based on open-source compute framework and it helps build enterprise level scalable jobs as it leverages distributed processing of the data.
@unw1718 11 ай бұрын
is there a way to add a vpc?
@jvmunhoz98 Жыл бұрын
Great video. I'm currently refactoring some python shell jobs into Ray cause most of them were getting a bit too big for just 1 DPU. But I'm having problems importing job parameters into the script. I usually import the getResolvedOptions function from the awsglue.utils library, but ray it doesn't support awsglue, which is odd. Should I add it manually or just use another method for importing job paramenters?
@AWSTutorialsOnline Жыл бұрын
Approach to retrieve the parameter is different. Check this link - docs.aws.amazon.com/glue/latest/dg/author-job-ray-job-parameters.html
@KannanPoompatham Жыл бұрын
Hi, I watched some of your videos and liked it, but I didn't see any videos that covers AWS Amplify. Do you want to cover that as well?
@AWSTutorialsOnline Жыл бұрын
Hi, unfortunately, I don't have expertise in AWS Amplify. Sorry about that.
@KannanPoompatham Жыл бұрын
@@AWSTutorialsOnline No Problem. Thanks
@cheluvesha Жыл бұрын
Brother Thanks for the video, When to use Glue and when to use EMR?
@AWSTutorialsOnline Жыл бұрын
Glue can be used only with Apache Spark and Python. While, EMR support additional frameworks such as Hadoop, Hive, Presto etc. So use EMR when you want to work on these framework other than Apache Spark and Python. Hope it helps.
@cheluvesha Жыл бұрын
@@AWSTutorialsOnline thanks it makes sense now.
AWS Glue Tutorial for Beginners| Learn everything about Glue in 30 mins| Glue Data Catalog| Glue ETL
AWS Made Easy
Рет қаралды 8 М.
AWS Events
Рет қаралды 4,2 М.
AWS Tutorials
Рет қаралды 1,2 М.
i-shoppers обзоры
Рет қаралды 1,4 МЛН
One More German
Рет қаралды 1,2 МЛН
Rj Mobile 01
Рет қаралды 11 МЛН