No video
Superstore Data Analysis | End to End AWS Data Engineering Project for Beginners
Рет қаралды 26,718
Күн бұрынIn this video we will build a end to end data engineering project using AWS. This project will use following AWS services:
1- IAM
2- S3
3- AWS Glue
4- AWS Athena
5- AWS Quicksight
Here is the Kaggle dataset: www.kaggle.com...
Steps to create AWS account: repost.aws/kno...
Start your Data Analytics Journey : www.namastesql...
Zero to hero(Advance) SQL Aggregation:
• All About SQL Aggregat...
Most Asked Join Based Interview Question:
• Most Asked SQL JOIN ba...
Solving 4 Trick SQL problems:
• Solving 4 Tricky SQL P...
Data Analyst Spotify Case Study:
• Data Analyst Spotify C...
Top 10 SQL interview Questions:
• Top 10 SQL interview Q...
Interview Question based on FULL OUTER JOIN:
• SQL Interview Question...
Playlist to master SQL :
• Complex SQL Questions ...
Rank, Dense_Rank and Row_Number:
• RANK, DENSE_RANK, ROW_...
#aws #dataengineer #project
@ankitbansal6 6 ай бұрын
I will be doing more complex projects in the future. Please support this initiative by liking the video and sharing it with your friends.
@rahularora3832 6 ай бұрын
Will you create only projects or as a pre-requisite you may dig deep into particular topics related to aws data engineering first and then create projects in which those topics would be used? BTW, thank you for sharing this. I liked it.
@AkshayaReddyKoduri Ай бұрын
It will be helpful if you do a project on AWS using Glue S3 Lambda and Redshift
@tejasnaikwade9193 Ай бұрын
more complex projects will br very helpful sir.
@prashantsingh6260 6 ай бұрын
Thank you, Ankit, for the informative and approachable contents to kickstart into AWS for beginners... There's much more to delve into for real-world challenges, but your dedication to taking time from your busy schedule to assist peoples is truly commendable... Your support for needy peoples is invaluable. Thanks and keep up the great work!
@ankitbansal6 6 ай бұрын
More to come 🫴
@user-zb9hz7dm2f Ай бұрын
Great guidance I ever scene in field of AWS. I completed that project successfully. You gave best guidance while teaching. I also viewed some youtubers but their guidance worst as when you are at mid of the project some error happens and they skip that process and you stuck. Made more videos on Data Engineering. We want your guidance sir.......Love from Maharashtra
@dillikaruru254 4 ай бұрын
It's a great initiative Ankit. You explained the project in a very detailed way and I am able to understand it easily. Crisp and clear. Please do more projects based on Glue jobs, SES, SNS, lambda, cloud formation, CI/CD for a project with github actions etc.
@ankitbansal6 4 ай бұрын
Sure
@ArvindPatel-ht3cp Ай бұрын
This is so amazing for someone to excel in data engineering!!
@MsMohanj 6 ай бұрын
When I worked at Anz, I saw you and became a member of your Python and SQL batch. I am your biggest admirer and follower. For all data engineering careers, your channel is the place to go. I'm grateful for your efforts 🎉🎉🎉🎉
@ankitbansal6 6 ай бұрын
Awesome, thank you! 🙂
@kaushaldubey140 6 ай бұрын
Good video for starting with AWS glue. I am looking forward for end to end ETL project using AWS glue and pyspark where we will be extracting data from a database (eg. MySQL), storing it into S3, doing some transformation on it and finally loading it into Redshift. Request you to please create one such project video.
@ankitbansal6 5 ай бұрын
Great suggestion!
@vivek29925 6 ай бұрын
Great video Ankit. Hope to see more real world project based videos from you. Thanks for your efforts.❤
@rishavsen5581 6 ай бұрын
This helped pave a way for one of my existing projects, where I was facing a blocker. Thank you Ankit
@ankitbansal6 6 ай бұрын
Great 👍
@vishwassharma3312 6 ай бұрын
Sir please upload complete play list on AWS and BIG Data. 🙏🙏
@sakshibakare9716 3 ай бұрын
Thank you , Ankit this very useful for AWS beginners. your videos are always helpful for interview
@ankitbansal6 3 ай бұрын
Glad to hear that😊
@vivekchandan3441 4 ай бұрын
This is really a great video. Please make more by taking bigger data & show us the lifecycle like end to end project on data analysis.
@user-st4yy6rh6b 6 ай бұрын
Hi Ankit, Thanks for creating AWS informative content that makes complex topics easy to understand. Much appreciated! Expecting more such videos..
@kolisettysasiram290 6 ай бұрын
Nice initiative bro I felt in the coming videos you can add some uses of Aws service we are using in the videos.. also some more usecases..
@ankitbansal6 6 ай бұрын
Sure 👍
@gururajhdesai4854 3 ай бұрын
Thank you. Create more basic projects.
@abhijeetjain8228 3 ай бұрын
Sir, please continue more on this series thanks.
@baskarandurai6702 6 ай бұрын
Thanks for your efforts Ankit... Please continue this alonn with python problem solving video like SQL.. Waiting for python Ankit..
@shivanandkumar6002 6 ай бұрын
Thnkss alot for this Ankit bro 😎
@hritikapal683 6 ай бұрын
That's a great initiative sir, keep them coming!🎉
@ankitbansal6 6 ай бұрын
More to come!
@prashantmhatre1328 5 ай бұрын
Really like this project Ankit Bhai, Thank you very much
@ankitbansal6 5 ай бұрын
Thanks for liking😊
@electricalsir 6 ай бұрын
Thanks man we are waiting for this video ❤😂
@zainaltaf4935 5 ай бұрын
great learning source. Thanks for it
@AnkitMinj-wc1pm 5 ай бұрын
Hello Ankit, Your content is very helpful. I was seeking clarification to understand the data source needed for connecting to Quicksight for creating visualizations, as explained in this particular video. Is the process used in AWS specific to a data engineer's skill set, or is it a must-know skill for data analyst roles as well? If so, please assist us in identifying the AWS topics we need to know in order to connect a data source to Quicksight.
@anshumanmahapatra9064 6 ай бұрын
Hi Ankit Thank you for the insight on different AWS tools. I just wanna know how Glue crawler will work in a database with multiple tables where they are related with Primary foreign keys etc, I mean does it crawl the entire schema? Also if possible make videos on how Glue is used in ETL process and can we load to AWS Redshift using Glue?
@ketanjain93 5 ай бұрын
Hello Ankit, I was facing Problem in Amazon Quiksight , when i create charts in it , it is giving error "the data source or data set was deleted or become unavailable while Amazon Quicksight for ingesting the data into Spice. you can import a data to try again". what wrong i have done?
@entertainmenttv1672 6 ай бұрын
Please make more videos like this, also try to include pyspark as well.
@abhishekn786 6 ай бұрын
Great initiative, thankyou ❤
@ravitejatavva7396 6 ай бұрын
Great Initiative, Keep going @ankit..😍
@ankitbansal6 6 ай бұрын
I will try my best
@suresh2105 6 ай бұрын
Can you make a playlist on how to start with aws account for data analysis. Like stwp 1 account creation Redshift, lamda, s3 etc. Each service and uses foe data science. Please?
@naveengvkvs1915 6 ай бұрын
Good video for starters👍
@sriharinair227 2 ай бұрын
Hey Ankit, any way to automate - 1) Creating different csv files for each date. 2) Moving each file into a separate folder. Any help will be appreciated!
@rakeshkumarsharma2250 6 ай бұрын
Bro I am following this channel since 1 years , you are doing awesome work and want to learn aws cloud for mlops suggest me good sources thanks
@adharshsunny5154 6 ай бұрын
Wow Amazing content as usual
@ankitbansal6 6 ай бұрын
Glad you think so!
@JamiewithData 6 ай бұрын
Appreciate your effort.
@Mehtre108 6 ай бұрын
Hello sir, its like a incremental load instead full load .
@ankitbansal6 6 ай бұрын
Yes, correct
@Mehtre108 6 ай бұрын
@@ankitbansal6 can you make project on data source would be MySQL, SFTP. Like realtime.
@sagarvajsharwa 6 ай бұрын
Great work 👍 thank you for sharing this
@ankitbansal6 6 ай бұрын
Thank you! Cheers!
@bihaskumar5857 6 ай бұрын
Thank you for this video.. We can upload the csv file in a s3 location and using aws glue we can create the snapshot folder. Just a thought
@swapnam1528 2 ай бұрын
for me partitions are not getting created. What could be the reason.
@Ashu23200 6 ай бұрын
In need of snowflake end to end project 😊
@mohitjoshi8984 6 ай бұрын
Thank you bro appreciate your efforts 🙂
@ankitbansal6 6 ай бұрын
It's my pleasure
@baskaranpbs8906 6 ай бұрын
Awesome video and really helpful
@ankitbansal6 6 ай бұрын
Glad it was helpful!
@heetshah5923 6 ай бұрын
On upcoming days can you make video that cover that redshift also. And if possible can you make fetch the data from redshift to powerBI process only
@ibizilong8907 6 ай бұрын
please do one video with ETL as well
@BALAJI-je1ty 5 ай бұрын
Thank you very much
@vikramjitsingh6769 6 ай бұрын
Good One!!
@aur505nj 6 ай бұрын
Thanks Ankit
@abhishekkumar-qi3is 6 ай бұрын
Sir please bring aws data engineering course also in live mode.thanks
@sushilkumarsss 6 ай бұрын
Sir, please suggest, are you planning any courses on AWS? If yes, when will you start it?
@mohansai9109 6 ай бұрын
Please create a project on Azure data engineering project. Pyspark, Databricks and Azure data factory. By using this please creata a data engineering project
@saikathazra6732 6 ай бұрын
Sir are you going to start any AWS course?
@abiramiarivazhagan1467 Ай бұрын
Can we upload this project on github?
@user-dg2jg3ow8z 6 ай бұрын
Thanks, Ankit for sharing this AWS project. Please could you tell me how we can schedule the dashboards on quicksight like when the data will be updated in S3 bucket dashboard automatically should be updated? Or is it possible
@ankitbansal6 5 ай бұрын
You can schedule quick sight dataset refresh in quick sight only
@ramanaswamy9845 6 ай бұрын
Do you have Any plan to launch AWS data engineering course?
@manjumohan7731 6 ай бұрын
Can you also create E2E Data Engg Projects using Azure
@shivam87480 3 ай бұрын
how to showcase this project, how to upload on github
@griffingalatta3619 6 ай бұрын
I'm a Talend developer.. I'm shifting my Career as AWS data engineering....this project looks simple..will the real time company data engineering pipeline will be like this onlyyy
@ankitbansal6 6 ай бұрын
It started with simple. Will do more complex projects 😊
@griffingalatta3619 6 ай бұрын
thanks looking forward to it @@ankitbansal6
@heetshah5923 6 ай бұрын
This will include redshift?
@amanahmed6057 6 ай бұрын
Sir do we need account of AWS , can you share any virtual playground who is giving aws platform
@DineshGudla 6 ай бұрын
@ankitbansal6 This was great video covering multiple technical stack of AWS. Simple yet insightful project for beginners
@ankitbansal6 6 ай бұрын
Glad you liked it 😄
@hiteshjha8850 3 ай бұрын
I am unable to create an AWS GLUE .its saying access denied. I searched on google and tried every possible solution mentioned , but none of them are working
@ankitbansal6 3 ай бұрын
If you have logged in as an iam user then make sure you have glue access.
@hiteshjha8850 3 ай бұрын
@@ankitbansal6 yes I tried both from iam user with glue access as well as from the root user
@partharora6023 6 ай бұрын
sir how to get job after 3 years of gap
@hasnainakbar8621 6 ай бұрын
❤
@mubashiraf2524 6 ай бұрын
Is these simple data engineer project?
@ankitbansal6 6 ай бұрын
Yes
@yashsaxena7139 6 ай бұрын
How much will this cost to run on AWS
@ankitbansal6 6 ай бұрын
less than 1 dollar.
@arshS27 6 ай бұрын
When is sql live starting end to end ?
@ankitbansal6 6 ай бұрын
March 2nd
@arshS27 6 ай бұрын
@@ankitbansal6 thanks Ankit ! And do u also cover tableau training and python as well as I want to learn this trio ?
@AbhishekJain-wl3st 4 ай бұрын
@ankitbansal6 I am getting access denied error while creating crawler can you please help me in resolving this issue
@nipunshetty9640 6 ай бұрын
Can u make FROM BEGINNING TO END PLAYLIST OF CLASSES OF MICROSOFT AZURE AND AMAZON AWS which we can utilise d same during work, please ankit bhansali my request for you @ankit bhansal
AWS Glue Tutorial for Beginners| Learn everything about Glue in 30 mins| Glue Data Catalog| Glue ETL
AWS Made Easy
Рет қаралды 16 М.
Fabiosa Best Lifehacks
Рет қаралды 21 МЛН
Ankit Bansal
Рет қаралды 19 М.
Prasoon Parashar
Рет қаралды 6 М.
IT k Funde
Рет қаралды 36 М.
Mr. K Talks Tech
Рет қаралды 267 М.
Ankit Bansal
Рет қаралды 38 М.
The Big Data Show
Рет қаралды 36 М.
Seattle Data Guy
Рет қаралды 79 М.
Chandoo
Рет қаралды 503 М.