How I'd Become A Data Engineer (If I had to start over as a data analyst in 2023)

Рет қаралды 58,780

Жыл бұрын

How would I become a data engineer if I were a data analyst?
Here are the skills I would learn.
1. Learn Python And SQL
• Building a BMI Calcula...
www.khanacademy.org/computing...
2. Learn Data Modeling
aatinegar.com/wp-content/uplo...
3. Learn Data Pipelines
www.estuary.dev/what-is-a-dat...
• What are some common d...
• What Is A Data Pipelin...
4. Learn About The Cloud
GCP - bit.ly/3NQVn7V
AWS - bit.ly/3HLdV9M
5. Everything Else
Docker And Kubernetes
bit.ly/3jbKBPj
Etc
Looking to start you're own data engineering/analytics consulting company, then you should check out my new course here
courses.technicalfreelancerac... - and use the coupon code "deconsult" to get 50% off
If you enjoyed this video, check out some of my other top videos.
Top Courses To Become A Data Engineer In 2022
• Top Courses To Become ...
What Is The Modern Data Stack - Intro To Data Infrastructure Part 1
• What Is The Modern Dat...
If you would like to learn more about data engineering, then check out Googles GCP certificate
bit.ly/3NQVn7V
If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.
seattledataguy.substack.com/
Or check out my blog
www.theseattledataguy.com/
And if you want to support the channel, then you can become a paid member of my newsletter
seattledataguy.substack.com/s...
Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio
_____________________________________________________________
Subscribe: / @seattledataguy
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.
*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.

Пікірлер: 110

@SeattleDataGuy 10 ай бұрын

If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k

@BLK_O 11 ай бұрын

I've just done the same thing. I've spent years being a solution designer / data analyst and I've just switched into data engineering as a senior technical specialist with no experience in actual hands on data engineering but I've got experience in designing data warehouses and ETL solutions.

@SeattleDataGuy 10 ай бұрын

Nice! how is it going!

@devdawg22 11 ай бұрын

If this isnt proof that my phone is always listening. Last night my husband was telling me how he (manager of data analytics) is being moved under the programming department and being made data engineer. This popped up in my suggested today lol

@sotumodo4317 8 ай бұрын

Happens all the time. I could talk about something with a friend over a phone call and the next day an ad about it pops up. Lol

@user-gq1ij 8 ай бұрын

Let's normalize this now. Let's show no surprise to these things. It has been long time now 😂

@RichardRietdijk 9 күн бұрын

It isn't necessarily proof. It's likely just the Baader-Meinhof phenomenon/frequency illusion.

@alexmelchior1731 11 ай бұрын

amazing video man!!!

@mcattaneo2195 8 ай бұрын

What an awesome explanation! Very usefull, thanks!!

@SeattleDataGuy 7 ай бұрын

You're very welcome!

@andreinitescu1768 11 ай бұрын

I transition from a cook to a Integration Developer 2 years ago and now I am taking a Master in Computer Science while working, hoping that 3 years from now to get into Data Engineering,

@vivekjoshi3769 11 ай бұрын

How did you become integration developer from cook/chef? 🤔 Please elaborate your path. I come from hospitality background & looking for inspiration.

@andreinitescu1768 11 ай бұрын

@vivekjoshi3769 I have studied for about 6 months to get AWS - Solutions architect certification and SQL. That was enough to get my first IT job

@Albidon 11 ай бұрын

@@andreinitescu1768what is the title of the job you applied for ?

@ekaterinakorolkoviene5824 11 ай бұрын

on my journey from DA to DE. Taking any opportunity to do ETL's, models, automatizations that I can find. Unfortunately, in my department our tech stack is so poor, that I realized that I'm never going to transit to DE internally and the DE work I'm doing not valued as much as of good DA or BA.

@sitrakaforler8696 8 ай бұрын

Great video man !

@SeattleDataGuy 7 ай бұрын

Appreciate it!

@Moonbat112 10 ай бұрын

Have you checked out the data engineering zoomcamp and if so, is it worth the time? Seems pretty popular and results in a personal project with decent resources to pull from. Currently in a hybrid ETL and Support role where I am trying to work in the DW/DE and hoping I can get some more hands on experience.

@aenima1_ Жыл бұрын

Already onboarding to DE work at Amazon :) thanks for the inspiration!

@SeattleDataGuy Жыл бұрын

Congrats!

@nguyenthianhphuong 8 ай бұрын

Amazing thank you Ben! How much time would it require to complete the switch in your opinion?

@gavinkalaher7314 Жыл бұрын

I've pivoted to a full-stack developer route which can also easily translate into a data engineer role at the 3-5 year career point.

@SeattleDataGuy Жыл бұрын

Yeah 100% any easy switch

@abhisekgupta5381 3 ай бұрын

Can you please help me in understanding how full stack and data engineering is related?

@joaodionisio2341 Жыл бұрын

I just got hired as a Cx (Customer Experience) Quality Analyst as my first job. With this video, you helped me a lot on how to start my path toward a Data Engineer role, thanks.

@SeattleDataGuy Жыл бұрын

Good luck!!!

@Ram-ku7me Жыл бұрын

Hey, I have gained some data engineering skills and have done end-to-end ELT. I am looking for some open-source projects to contribute to but to my shock I can't find any, could you suggest some maybe even well funded startups.

@Immortalus98 9 ай бұрын

So lucky to have found you. Doing my masters in data science after being a data analyst for about a year. Didn’t even start the vid I can tell this guy has real wisdom

@SeattleDataGuy 7 ай бұрын

you're too kind!

@abiygirma1925 2 ай бұрын

where are you doing the masters? i.e. which program? I'm thinking about it too

@Immortalus98 2 ай бұрын

@@abiygirma1925 Rutgers University! New Jerseys very own!

@Abayflo 10 ай бұрын

So great 👍🏿

@SeattleDataGuy 10 ай бұрын

Glad you liked it!

@user-qz6gv1jl5k 10 ай бұрын

Hi Ben, You mentioned in the video that you linked the interview test prep? I don't think I see it in the drop down. Would be very interested in taking a look at it - if available? thanks!

@lordgrim3415 9 ай бұрын

found it in the description of his other vid - title: "The Last Data Science/Data Engineer Interview Prep Video You Will Ever Need To Watch"

@JonathanBiemond 7 ай бұрын

Thanks for the very helpful video! Do you have an online community for your channel on Discord or something, or any other Data Engineering community you would recommend? Also the Data Modelling link in the description is broken for me, any other data modelling resources?

@mehdio Жыл бұрын

Camera upgrade ? :D Finally ;-)

@SeattleDataGuy Жыл бұрын

I bought a new phone hahahah

@malcorub Жыл бұрын

3:10 Hi Ben, where can I find the diagram you displayed here? I really like it! I tried the links in the description but couldn't find.

@SeattleDataGuy Жыл бұрын

The image is here res.cloudinary.com/practicaldev/image/fetch/s--cnXIakxd--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/github.com/datastacktv/data-engineer-roadmap/raw/master/img/roadmap.png

@malcorub Жыл бұрын

@@SeattleDataGuy Thx sir! You da man!

@christopherlucas1475 Жыл бұрын

Love this video! Could you make one for SE -> DE?

@SeattleDataGuy Жыл бұрын

Oh! thats a great idea!

@nzgokiwi Жыл бұрын

Cool idea. I am a SE and currently working toward DE. I would like to see a video that you suggested, too.

@jemiranhunter 3 ай бұрын

Great video. I'm currently in the process of becoming a data analyst but my end goal is to become a data engineer. Do you recommend any certifications to have that will help gain knowledge and experience I was looking at some of the Microsoft certifications for data engineers.

@theinquisitor18 Жыл бұрын

I'm taking the IBM Data Engineering certificate. I'm blowing through the intro courses because I already have those skills down. It's the warehousing, Spark, and so on that I'm taking that course for.

@panic_seller Жыл бұрын

Me too, I took it specifically for Kafka and Spark

@theinquisitor18 Жыл бұрын

@@panic_seller, I couldn't find any Kafka or Spark tutorials on KZfaq that worked for me, which is why I decided to enroll in the course. 😊

@boredsithlord9235 Жыл бұрын

@@theinquisitor18 im interested now !

@qahoushh 11 ай бұрын

Hey! I think to take a course in Data engineering . Do you recommend it IBM? What do you think about it ?

@AJGthree 11 ай бұрын

@Hamza Qahoush I'm also taking this certification since it's free through my university. So far I have only taken the first few classes on Python, SQL, and relational databases. The material is not very thorough or in-depth. It introduces a topic, guides you through a few simple exercises, and moves on to another topic. Unless you can get it for free, I would recommend skipping it and looking for free resources, then building your own projects.

@haidshah3516 7 ай бұрын

Hey Ben. Just pass my az900 dp900 and dp203(data engineer). There are many job's however no luck for me so far. What advice you would give to get my first job? Thanks

@adolfohc78 9 ай бұрын

Some of the links don't work. Where can I find the Data Modeling resource? Thanks!

@paulnyagah181 Жыл бұрын

Trying to head there at the moment

@SeattleDataGuy Жыл бұрын

Good luck!!!

@OmkarGurme 10 ай бұрын

I have learned about SQL, Python, Spark and Azure Databricks , should I start applying for Data Engineering jobs right away ?

@hasanreza3437 3 ай бұрын

Hi. when you say baseline skills what do you mean by it? Specifically for data scientist.

@Cadet1249 Жыл бұрын

Amazing video. I’m still in my undergrad majoring in CS but I on following this exact path. It’s my dream job to be a freelance data engineer/analyst for small businesses.

@andrewhting 9 ай бұрын

No dude, go away. You have a CS degree. Go be a developer so us lesser MIS-er’s can have a chance 😂

@ashraf7172 Жыл бұрын

I am having more skills & confidence in being a data engineer While as a data analyst i am not confident enough What should i do then?

@nicholascarlile6934 11 ай бұрын

Would you recommend taking a similar path, starting in Data analytics then transition to engineering?

@SeattleDataGuy 10 ай бұрын

A lot of people have done it! so it works great

@rohithnallam-fg4vz 8 ай бұрын

@seattledataguy Thanks for videos. Trying to find where I can get Sr Data Engineer prep courses which Includes system design and other important things

@ak6877 10 ай бұрын

Can you please share link to interview prep?

@MrDubzein 10 ай бұрын

You should considered seriously to launch an online course related to the Modern Data Stack: Fivetran, DBT and Snowflake.

@SeattleDataGuy 3 ай бұрын

maybe! for now i have launched a course on consulting because it felt like thats what people were asking for on linkedin

@MrDubzein 3 ай бұрын

@@SeattleDataGuy really? could you share the link pls?

@SeattleDataGuy 3 ай бұрын

@@MrDubzein sure!, you can find it here - courses.technicalfreelanceracademy.com/courses/starting-6-7-figure-consulting

@claudiu9110 4 ай бұрын

Can you share please the learning path diagram from minute 3:05 ?

@lostahmado Жыл бұрын

does his mean one has to start in another role before pivoting into data egineering?

@SeattleDataGuy 10 ай бұрын

You don't have to, but it does make it easier

@user-fw6ek9ur3v 6 ай бұрын

Thank you for taking such a positive, imposter-syndrome-fighting stance on becoming a data engineer. Sometimes the bar to land a tech role feels so high!!

@urielgilespinoza7376 11 ай бұрын

My English is not good because im learning bbut im gonna explain my situation 😅 I was hesitant to choose Data analytics or a front-end developer because before coming to the USA I worked as an accounting assistant and when I got here I dedicated working in a restaurant cooking. Three months ago I was just learning html and css on my own, it's giving a bit of difficulty js. I heard other people that i would have chosen data analytics because it would be a good combination with my accounting degree that I did in my native country.. So now I am lost.. 😅 Any advice please 🙏

@chikabenie2593 8 ай бұрын

You couldn't switch to data analyst or financial analysts instead of front end dev. It entirely far from account packages compared with data Analyst

@castangordon8023 9 ай бұрын

What would you recommend for someone with no experience in Data Analytics

@SeattleDataGuy 7 ай бұрын

In terms of breaking into DE? You can do it, it can be harder. But you should attempt to get some projects under your belt and perhaps if you already have a corporate job, see if you can move into a DA or DE position at your current role

@HarryManback0 5 ай бұрын

Step 1 - Get a B.S. in computer science.

@zorouzh 11 ай бұрын

Hey can you link the interview prep that contains SQL problems and videos

@griffinolson-allen8570 11 ай бұрын

@nchou646 11 ай бұрын

Hi where can I find the interview prep link?

@zorouzh 11 ай бұрын

Yeah wanted to try out those SQL problems

@calvindsouza1151 6 ай бұрын

where can i find the interview guide broski

@SeattleDataGuy 6 ай бұрын

Here is the Data Engineering Study Guide! docs.google.com/spreadsheets/d/1GOO4s1NcxCR8a44F0XnsErz5rYDxNbHAHznu4pJMRkw/edit#gid=0

@griffinolson-allen8570 11 ай бұрын

Can you link the interview prep sheet?

@lordgrim3415 9 ай бұрын

found it in the description of his other vid - title: "The Last Data Science/Data Engineer Interview Prep Video You Will Ever Need To Watch"

@Not_Clark_Kent 11 ай бұрын

How about mathematics? how much math should you know?

@SeattleDataGuy 10 ай бұрын

not too much, more solutions thinking and logic

@ailton.duarte 2 ай бұрын

Data engineer vs Data Scientist??

@SeattleDataGuy 2 ай бұрын

Hello! Thanks for your comment. What are you aiming for salary? work life balance? problems?

@adriancritchley2485 6 ай бұрын

Nice to see that it's not just my cat that sees my work chair as its arch nemesis.

@SeattleDataGuy 6 ай бұрын

Yeah, that chair is gone now hahah...

@durkamagik1589 8 ай бұрын

I’m 16 but I want to learn how to use python and other programming, I want yo be trained if anyone is free I need someone to explain everything

@azizmamatov6723 9 ай бұрын

All good, but I am pretty bummed about how everyone exaggerates their pays or skills, I guess it is a part of marketing culture here.

@SeattleDataGuy 7 ай бұрын

Who is exaggerating pay? Skills, I always try to be honest where I am and always trying to improve myself! The DE world never lets you rest in terms of learning