That's fair enough, it's great that you're so self-aware about your own preferences! I hope that you'll enjoy data engineering, now is definitely a good moment to pivot (given the rising demand for DEs)!

@@karolinasowinska Thanks! It definitely feels like the right move for me, and your videos have made me feel more confident in that decision, qnd identify the skills I need to grow. Thanks so much for this content! ♥️

> likes well-defined tasks/problems that have a real solution > no interest in business at all is'n it a contradiction?

Haha I know right! ;)

yeeees!!!!

@@sanchesrfl it's a great industry!

Exactly what I thought...😁

Same here. Let me code and build pipeline and I am happy

It's so good to be aware of you like and what you don't like! And it sounds like data engineering will be the perfect career for you - you can consider yourself lucky, given how in-demand those skills are now! ;)

@@karolinasowinska any advice on progressing other than reading and doing tutorials (just picked up designing data intensive applications last night). No much opportunity to get mentored i. My current role

Precisely like I think.

exactlyyyyy, I am currently an analyst and I am here because I realized I much more enjoy building tools and pipelines than analysing and searching for insights :D

Its so nice to see that i'm not the only who thinks like this.

Definitely, like you said, there are always two sides of a coin! Being in the spotlight all the time basically means worse work life balance, being in the shadows means awesome work life balance :)

Hi Remi! I work as an analyst at one of the decacorn at Southeast Asia, and I'm pretty sure that if you work at a company who has "move fast and break things" culture, there's no work-life balance in there :) Don't get me wrong, this is also the condition that suitable for you if you want to learn more. I agree with your point on "more long-term" and "satisfying", that's also why I envy data engineers and programmers because they can focus on the deep work. Just my 2cent.

Haha you might as well! If you enjoy what you're doing ;)

And I'm so glad that you've found me! Welcome aboard! ;)

I thought so too!

Oh, YT put this comment in "held for review" for some reason! I completely agree, how crucial and "central" you are as a data engineer depends on what industry you're in. And I also hope that Denodo/Databricks will only make data engineers' lives easier! :)

Thank you so much! :) 😊

Haha nice one! Definitely, if you're building data pipelines then you're a genuine data engineer! ;) I'm happy that you enjoy it!

Thanks, I do try to!

That's awesome!

Exactly!!

That was my impression as well!

Thanks! :)

My pleasure! ;)

I'm sorry to hear that. If you enjoy DE, don't give up! :)

All the best, you can do it!

I'm glad that this is relatable! :)

Great idea, I'll think about it! ;)

Like I mentioned in the video, no job guarantees the security that it'll be in-demand for decades. I wouldn't think about it too much. Go for what you enjoy doing most, and new opportunities will emerge for your skillset in the future. :)

Haha ciepłe bułeczki, dokładnie one chodziły mi po głowie! ;D

Pleasure:)

True! (Unless the firm only advertises itself as data driven)

Hey, I have a data engineering course for beginners on my channel :)

Actually a great idea, I can definitely think about a short ML course! :)

My theory on why DS is more popular is that most people want to make discoveries / draw conclusions rather than simply building the infrastructure that lets other people do that; they wanna analyze and build models and find answers - it just sounds more exciting! The thing is you're still solving problems and using logic when doing DE, its just not as "on the surface" if that makes sense.

I might order it. Although I haven't finished his first book yet, I couldn't get through very descriptive analogies and stories. But I appreciate the guy's mind.

I'm moving into a DE role from a Data Analyst role. I have a degree en economics, so I get what you're saying. You are not the only one struggling with this.

Of course not! Go for what you love, never put limitations on yourself! :)

You can do it straight out of college! And if not, I suggest general software enegineering :)

True. Although often MLs and DEs collaborate in teams, so that knowledge shouldn't be necessary. Definitely useful though, especially at start ups.

Haha I discovered that English say "hotcakes" instead, I was close! :D

Ouch :( I feel that :(

You're welcome:)

I'm sure there is a demand for those kinds of roles. If there's a will, there's a way, so good luck with getting your place there!

@@karolinasowinska about 20 years ago I heard about this technology called "sub vocal communication". They used it to help divers communicate underwater. They placed a transducer over the vocal cords to pick up the nerve impulses from normal speech. The system had to be trained so that they could connect the electrical pattern with known words. Very cool stuff BEFORE the term machine learning was widely used. I wanted to use it to help someone like Dr. Hawking recover speech and have some mobility. I feel this should be my life's work!

Ohh, how very flattering :>

Yes. But being a general software engineer is probably an even more direct route.

Ha, thanks! ;)

Data in general isn't going anywhere, its growing. If you know SQL even at least and have good problem solving skills, you're sorted you can easily get another position within the field. Careers are fluid sometimes, it can be exciting.

Good point! Although I'd say that there are new data engineering technologies coming out as well, or systems ar migrated to different environments like AWS. Obviously depends on the company you work for! If the company doesn't innovate, then you're right, writing ETL processes can get monotonous

@@karolinasowinska - Agreed. But what I am saying if you see the sheer variety of work in ML(let's say an anomaly detection, ranking algorithm, customer retention etc) is also more(which makes it more interesting and challenging) when compared to data engineering work. Great video by the way!!!!! I hope you reach 100K subscribers soon :) :)

@@samparkpradhan8849 I see what you mean, you're definitely right! :) And thank you! :)

If you are doing the same transformations across multiple projects, automate the process(es) and let the computer(s) do the work.

I feel the Data Engineering role can also be quite diverse, depending on the company strategy and/or needs. So it's actually very possible to find a DE, that gets to do ML maybe in a startup. So it might not be a monotonous role, it depends. There are also alot of Big Data tools out there aswell. For a Data Engineer who wants to self-improve by learning all these tools will have alot to learn

I know plenty of people from STEM (not just computer science) background who are data engineers :) You'll be fine!

​ @Karolina Sowinska Thanks for the inspiration! I hope we will be colleagues soon.

They can be found in almost any sector!

I'd say, it really shouldn't matter for you what your spouse is into!

I recommend the courses I mentioned in the description:)

I'd go for an engineering role, so data engineering or software engineering in general ;)

Hah well, true, I guess!

Thanks!

Do you have a portfolio of personal projects which demonstrates your technical and communication skills?

Good suggestion above!

The good news is that in order to do Machine Learning, you obviously need to work with tech;)

I Love it

Yep that's very correct!

Okay my mmmmmmmnmmmmm m mm m m m m m

Thanks ;) I enjoy editing music in ;)

I'll see, thanks for the suggestion :)

If you actually enjoy data science side of things, then don't do it! ;)

Thanks Karolina! Really appreciate to know your idea.

Haha I realised that English say "hotcakes" :D

@@karolinasowinska But how it will be in Polish? По русски это "разбирают, как горячие пирожки". Сiepłe bułeczki? Great talk!

@@SwapperTheFirst haha ciepłe bułeczki ;)

Did many DE interviews and was part of interviewing Jr DE candidates too. Here are 5 tips: 1. When you're asked to write a code/query to answer a question or to complete a task, you first want to explain what your approach would be: What functions will you use? What lines of code are you considering to write? Why? What do you think the output will be? Think out loud before typing anything. That way, even if you fail to complete the task or to completely get the desired answer, you're still showing that you understood what you were asked to do and that you knew how to get the desired output. +Most of the time, the interviewer will guide you as you are brainstorming too which can be very helpful. 2. Don't be afraid to Google stuff when asked to write a query or a code. It's better than spending a long time guessing. You're not expected to memorize coding lines or functions. For example, if you know you need to use a window function then say, "I will need to write a window function here", then google how to properly write it. What matters in this example, is that you know what we had to use to get the right answer. 3. Don't guess different methods to write your code until the interviewer confirms which one is right to use. Instead, explain the method you would use to get the answer or to complete the task then try it. If it's right, great. If the interviewer says it's wrong or your code shows it's wrong, don't be embarrassed. Instead think of another approach you can try, say it out loud, and try again. We all learn and perfect codes through trial and error. So, yes of course, the interview would love the right answer on the first try. But if you're not getting the right answer the first time, it's your chance to show your problem-solving skills and your tenacity. Show that you're not afraid to make mistakes, to troubleshoot, and to keep trying until you figure it out. 4. If you really don't know how to write the code/query the interviewer is asking for, then be honest about it. You can say, "I've never done this before", or, "I'm not familiar with this". But don't stop there. Always follow with, "But I can try to figure it out". The interviewer may move on to another question or they may give you a chance to try. If you can try, I would suggest to Google what you're asked to do. You can try to complete the task with instructions you find online or if it's too hard for you, at least share what you understand from the instructions. This shows you're willing to learn and try unfamiliar tasks. 5. Be friendly and approachable during the interview. Even when you nailed all the questions, if another candidate also nailed all or most of the questions and on top of that, they were friendlier, the other candidate will be picked. Even during technical interviews, we look at the personality. No one wants to work with cold people or hire someone the team may not be comfortable with. Most importantly, every interviewer is asked to find a candidate that's a good fit for the company culture. Good luck!

From my research a lot of employers interchange a data science title for data engineering tasks. To get familiar with tech, go through the CompTia A+ concepts to get a foundation on general hardware/software/networking etc … then look for Data Engineering courses. Coursera has one. Hope this helps you out, good luck😄

Hey, for the roadmap visit www.dataappreciationsociety.com

Go for it! :)

Sounds like you could have any job in data, just pick one and learn the skills they typically ask for! You'll get there.

@@karolinasowinska thanks for your support. You're the best! x

Even if companies dont use ML, they still at least need someone to take care of the data infrastructure,.ie. data engineers.

Pick what you're more interested in

you could easily switch one hundred percent!

why do u want to switch? -you have a job and earn a lot money

of

@@kathib2571 Hi Kathi , Actually since college days i was always enrolled in various development projects .as per my current profile which is mostly support A . data engineering is high demanding skill and i have little hands on over redshift and hadoop clusters .also in distant future i have a plan to move in ML field .so i think it could be great if i switch to BIG DATA / Data engineering because it also help me understand flow of data from scratch .

Definitely possible. I'd advise you to look at relevant job openings and see what they require of you

I see data engineering as a type of software engineering, with a focus on data. :)

Ahan, ok thanks

What did you choose in the end and why? I would really appreciate your insight if you don't mind

@@adamo1262 Well I somehow found the way to incorporate both. I went to a DevOps focused consultancy but with my data background they allowed me to take on cases in both areas when they arise. The majority of my work since I began is DevOps though (ELK stack, Kubernetes, Helm charts, Azure DevOps Pipelines, connecting to Sonarqube, and other CI/CD tools). I see a benefit in both because no matter what direction you take you are becoming more experienced in software development and its lifecycle which wasn't something I was really exposed to as a DBA. I think DE is definitely more focused than DevOps because there are so many moving pieces in DevOps, and it will take longer to understand how everything fits together. Sometimes I miss working with Data though, I've only had a few DE cases and when I have them I realise how much I enjoy that type of work. If I was doing it again, I would probably take a dedicated DE role. I say it because for me I find some DevOps work to be quite trivial (at least how I interpret the value of the work), with DE I see immediately the output of my efforts and I find that quite rewarding.

It seems like a very fancy job title for someone who is in fact a software engineer, perhaps doing some data analysis ;)