(Simple?) Real World Azure Pipelines YAML Walkthrough

Рет қаралды 28,080

3 жыл бұрын

Real world pipelines for real world applications are more than just unit tests and deployments. They must also handle multiple environments, safeguard production, secure pull requests, version releases, promote Docker images and more. Let's walkthrough a relatively simple example that only has 2 environments and deploys to a managed PaaS. Although I use Azure Pipelines in this video, most of the concepts are vendor-agnostic.
Note: this is a very opinionated workflow, not a golden solution for everyone.
#DevOps
Image background by Jeremy Thomas on Unsplash
unsplash.com/@jeremythomasphoto?CopyText
====== References ======
Containerized demo Node.js app in video
github.com/julie-ng/azure-nodejs-demo
YAML Pipelines and Documentation for the project
github.com/julie-ng/azure-nodejs-demo/tree/main/azure-pipelines
====== Contents ======
00:00 Intro
01:50 Hello World Pipeline
02:57 Release Workflow, Automated Change logs
07:30 Variables
10:37 Dev Environment
14:00 Docker Image Promotion
19:00 Closing

Пікірлер: 75

@aceHuertas 2 жыл бұрын

This women is a national treasure, we must protect her at all cost !!! 😮 Thank you for this video Julie ! This is exactly the type of content and take I wanted to see on how to make azure pipelines !

@JulieNgTech Жыл бұрын

That is soooo kind of you - thank you! ❤️

@riadhgharbi7985 Жыл бұрын

Came in to learn some azure pipelines best practices, and ended up learning about 50 new things in the first 10 mins. Every now and then you just come across a channel that makes you question whether you're even an IT professional, and I love that. keeps one going and eager to learn more. Great job Julie

@JulieNgTech Жыл бұрын

Thanks Riadh! Keep learning. Never question if you're a professional if you're a constant learner :)

@cesarmagalhaes8078 21 күн бұрын

Started digging into azure devops for 24 hours, and scrolling through the pipeline at the beginning of the video made me aware a stage condition can be stored on a variable. LAND !

@zlovcevic 3 жыл бұрын

Your videos are hidden gems. Thank you for sharing all this knowledge 🙇‍♂️

@JulieNgTech 3 жыл бұрын

Thank. you zlovcevic for the lovely comments! It means a lot to mean to hear that it's helping people and it motivates me to make more. Thank you!

@ferney0gaviria 11 ай бұрын

I can't explain how much I've learned with this video. Thank you very much for sharing all this knowledge

@Maphew69 Жыл бұрын

wow! what a find you are Julie. Great content and very well presented. Thank you

@JulieNgTech Жыл бұрын

Thank you for watching 🤗 let me know if you think I should make more pipeline YAML videos

@fusesvk 3 ай бұрын

Your channel should have more than 100k+ subscriber already. Great work!

@JulieNgTech 2 ай бұрын

Thank you ♥️♥️♥️♥️

@mkonji8522 2 жыл бұрын

Love the video format.and cadence. Really helped

@JulieNgTech Жыл бұрын

Glad it helped - thank you!

@jcarlosbellido 3 жыл бұрын

Great video. I'm learning a lot from your material.

@JulieNgTech 3 жыл бұрын

Thanks Carlos! Let me know if you have any particular questions or topics you'd like to see :)

@user-iv6zj3br3r 10 ай бұрын

Wow, absolutely wonderful. Im new to pipelines and i found your channel. Just amazing. I would have loved to have a pipleline tutorial video for beginners by you. What to think of, how to structure it and so on... Kepp up your good work and have a fantastic day =)

@krisnarusdiono1304 Жыл бұрын

Love ur videos, Julie

@JulieNgTech Жыл бұрын

Thanks Krisna!

@TymexComputing 9 ай бұрын

Truthfull laughter :) 4:13 - and laughtfull truth: 7:25(one week later....) - current media are missing with truth its good to have You here!

@rafaelchavesquaresma8652 4 ай бұрын

great class! tks!

@JulieNgTech Ай бұрын

Thank you Rafael. Glad you found it useful. ❤️

@jacksorjacksor Жыл бұрын

Shoutout's to the email notification at 2:26 that made me check through all my tabs

@JulieNgTech Жыл бұрын

😂 I've since learned to keep my computer muted when recording. But it also means I might miss a meeting notification 😅

@jacksorjacksor Жыл бұрын

@@JulieNgTech small price to pay imo XD

@mattkruczek2665 3 жыл бұрын

Hey Julie, love your videos! I am learning a lot of nice tips as to best practices. If you're looking for specific topics and such I do have a suggestion. Could you do a "start to finish" video of building a pipeline? Most of your videos are shown from the point of "I have this thing already built and I'm tweaking it." But it would be great to see a breakdown of the whole thing start to finish and an explanation of each part. I know that's a little heavy in terms of a topic but I would find it incredibly useful. Great stuff!

@JulieNgTech 3 жыл бұрын

Hi Matt would just explaining an existing pipeline but much slower help? Would including how to add it the CI UI (Azure Pipelines or GitHub Actions) help? Is that missing? What else may be missing for you? I am concerned that building a pipeline from scratch on camera will take some time.

@mattkruczek2665 3 жыл бұрын

@@JulieNgTech I think explaining an existing pipeline would definitely help, no need to actually type the whole thing out on camera. I lot of times I look for videos like these in order to make "best practice templates" for my organization. When doing so it's super important to understand "why" it's best practice and how it all fits together. The one off tips are great don't get me wrong but the overall picture is extremely useful. The git repo you have that has the example yaml file has been invaluable. Part of me wishes I could peruse your devops instance to see if there is any configuration in there that you have done that I should think about replicating. That sort of stuff I can investigate but putting it all together in a video would really help.

@JulieNgTech 3 жыл бұрын

@@mattkruczek2665 ok now I understand, the end to end perspective including UI is missing. It's on my list since it's something I regularly do with customers. I just need to record it for KZfaq and let you know :)

@mattkruczek2665 3 жыл бұрын

@@JulieNgTech Awesome thank you SO much Juile, your videos are really helping me create our "standard" for dev ops.

@jaggyjut 3 жыл бұрын

@@JulieNgTech That would be helpful to start from scratch please. Probably good idea to break down the tutorial into smaller videos.

@soundwalkie Жыл бұрын

Thank you Julie, 吴小姐! I find your videos really helping to make real-world yaml pipelines!! I've been coming back and forth for different topics and trying to adopt your recommended best practices. Thank you so much. Please make more yaml pipeline videos!! Do you provide training or consulting sessions to companies? One last question, where do you get other opinionated approaches for organizing pipelines? 😃

@JulieNgTech Жыл бұрын

I will definitely make more YAML pipeline videos. It's crazy cold and snowy this weekend so I may do a video walkthrough of some "real world" GitHub Actions pipelines. Yes, training and consult is part of my job. I don't offer it outside of work. But if your company is deploying to Azure, then you may qualify for the FastTrack for Azure program: azure.microsoft.com/en-us/programs/azure-fasttrack/ Thank you so much for the feedback and I'm so happy that it's helpful. So much so that I asked my brother what you wrote in Chinese :) My last name is five btw, so "Wu" in Mandarin. Because my parents emigrated to the U.S. before names were standardized, we're "Ng"s

@austin.valentine 10 ай бұрын

I’d love if you go through real world use of yaml release retention policies. Would love to see something that works with other considerations like managing source tagging (which could change based on whether it completes the release cycle or fails at some stage), and also managing retention in light of multiple in-process releases for an organization that may not complete a full release cycle deploying to production before producing more builds to promote to various testing environments.

@adeyemi9860 5 ай бұрын

Nice one Julie! So many fish (es?)

@pythongabi Жыл бұрын

We need more pipeline videos Julie please do not stop filming 😢

@JulieNgTech Жыл бұрын

hey, yeah, sorry. Been going through a rough few months and haven't found time or motivation to film. But thank you 🤗 for asking. That helps with the motivation bit.

@hamzahaftab3920 2 жыл бұрын

Hey Julie, love your videos! Thank you for sharing all this knowledge. Can you teach me how to create my own pipeline for my project? I'm learning DevOps.

@JulieNgTech Жыл бұрын

Like a video on how to create a pipeline from scratch? Because there are so many beginning and overview videos out there already - what's missing in them that you think I can do that others have missed?

@cc-hc1dr 2 жыл бұрын

Great video. I am just learning Azure Pipelines and trying to set this up for a monorepo. How would this translate to a monorepo? Say I have 15 "packages", Does each of them have a DEV/TEST/PROD pipeline? Thanks

@JulieNgTech 2 жыл бұрын

If you're starting out on a new project, consider avoiding monorepos especially if you have 15 packages. That's going to be a nightmare. Even with 2 packages with potential dev/test/prod, you have at least 6 triggers to manage. Imagine the combination with 15 🤯 That being said there are uncommon use cases for a mono-repo usually because of legacy code that I've seen. And in those cases the teams are working towards splitting the monorepo so it's not a goal but an intermediary state. A long time ago I built an example here that may be helpful to you github.com/julie-ng/azure-pipelines-monorepo

@cc-hc1dr 2 жыл бұрын

@@JulieNgTech yes I was working from your monorepo example. I have both an existing front end (react/js) and backend (.net) monorepos for separate services and common libs. If I break up these into separate repos won't I still have to deal with these triggers in the individual repos?

@WilsonMar1 3 жыл бұрын

How do I get the console prompt like yours with branch name and how long the last command took?

@JulieNgTech 3 жыл бұрын

I use the Spaceship Prompt but have customized it to be more minimal. You can copy it if you want github.com/julie-ng/dotfiles

@HenrikBinggl 3 жыл бұрын

Very nice, lots of info! Q: any advice, tips/tricks how to improve pipeline testing. I always create trillions of commits until it does what it should.

@JulieNgTech 3 жыл бұрын

I sounds like you're already doing like real life engineers - lots of commits. I also need many "oops" and "try…" commits before I know it's properly working. That's why I miss having a single tenant Jenkins when you could just write groovy in the UI and run it and get result immediately without waiting… If you're using GitHub Actions, you could try this github.com/nektos/act to run the pipelines locally for faster development.

@HenrikBinggl 3 жыл бұрын

@@JulieNgTech Thx! I am using AzureDevops Pipelines, so still have to look for something similar.

@CassioNoronha Жыл бұрын

Do you use this same folder structure to build infrastructure(using terraform)?

@JulieNgTech Жыл бұрын

No, infra is different. IaC is risky. There is an example here though you can check out github.com/Azure/devops-governance be sure to browse the backend/ and environments/ folders, not just the azure-pipelines one. They should be documented relatively well. If not, open a GH issue and let me know and I can try to improve it :)

@CassioNoronha Жыл бұрын

@@JulieNgTech thanks a lot. I am checking the repo.

@daka6887 Жыл бұрын

You have my shield

@JulieNgTech Жыл бұрын

🛡️

@julienmelard8775 3 жыл бұрын

I really like your videos, nice tone, lot of usefull information, thanks ! Keep going, please, even after the pandemic ;-)

@JulieNgTech 3 жыл бұрын

Thanks Julien! Knowing it helps people, I will try to continue even as the sun starts to shine… thank you for the feedback, I think it will really help me try to include this in daily 9x5 work - which is the only way I can do this long term.

@kegmysta 9 ай бұрын

how much is it costing you to run all theses resources?

@ER-zj3jv 2 жыл бұрын

Thank you

@JulieNgTech 2 жыл бұрын

You're welcome, glad this was helpful for you :)

@saitejareddygurujala2534 3 ай бұрын

Could you please show us Corporate workflow for AKS deployments using Azure DevOps :) and thanks for this vid

@JulieNgTech Ай бұрын

For workloads to AKS? I don't have a distinct video teaching. But the latest video is me trying to deploy to AKS using Copilot to write the GitHub workflow pipeline and k8s YAMLs. That might be interesting - or at least entertaining :) For deploying the cluster? Does it have to be Azure DevOps? If it's GitHub Workflows, maybe :) Did you watch the GitHub workflow video? kzfaq.info/get/bejne/i61kopedppjJkZ8.htmlsi=V0AFugZT281ORKC9 Asking because I think the pipeline concepts are generally the same. The syntax will be different and GitHub has some extra constructs like the "environment" and federated identities (watch my most recent video about deploying with AI to see).

@WilsonMar1 3 жыл бұрын

[4:31] That keyboard sounds like a repetitive injury building up. Be a keyboard ninja (make no sound) and you'll type faster too.

@JulieNgTech 3 жыл бұрын

😂it's an ergonomic keyboard - not mechanical, but it still makes a lot of sound. I don't think sound is an indicator of RSI. Look at all the people who swear by mechanical keyboards ;-) I have suffered from RSI in the past, but that was when I used the awfully flat Apple Keyboard. I now use a Surface Ergonomic keyboard and love it :)

@s3-shouldersurfersecurity209 3 жыл бұрын

Can you do a complete DevOps course?

@JulieNgTech 3 жыл бұрын

Maybe… probably only if my employer sponsored it, i.e. let me do it in working hours - in which case it would/should be published on one of their channels. Have you searched KZfaq for other DevOps courses? There must be many out there. What's missing that you're looking for?

@logicstv 3 жыл бұрын

@@JulieNgTech Securing the pipeline from SCM to Post Deployment.

@JulieNgTech 3 жыл бұрын

@@logicstv I wrote a few articles in the MSFT Documentation. They should be generic enough that they're helpful regardless of whether you use Azure. Let me know if you they answer your questions docs.microsoft.com/en-us/azure/architecture/example-scenario/governance/end-to-end-governance-in-azure#considerations docs.microsoft.com/en-us/azure/cloud-adoption-framework/secure/best-practices/end-to-end-governance I actually recorded versions of this video but the audio/video was so mangled, I need to re-record it.

@WilsonMar1 3 жыл бұрын

PROTIP: [13:31] production.yml is very slim. It doesn't run any test or build docker images. It grabs from container registry and assumes it's ok.

@JulieNgTech 3 жыл бұрын

Yep it only grabs the image and deploys because it's production. If the git workflow and permissions are configured properly then when this pipeline is running everything before it went fine.

@kpsher367 5 ай бұрын

Julie's 'simple pipelines' are complicated...

@kpsher367 5 ай бұрын

For perhaps it was cool to see how you generated the change log and kept a variables file embedded with various conditions, overall good job.

@JulieNgTech Ай бұрын

Real world pipelines are rarely simple 🫠

@medusaskull9625 2 жыл бұрын

Even Microsoft engineers are despite of windows. She is using linux shell in mac os. 😂

@JulieNgTech 2 жыл бұрын

To be clear, I don't despise Windows. It's just not for me as someone who's used a Mac for almost 20 years and mostly works with non-Microsoft specific open source technologies 🙃

@lattakia3812 Жыл бұрын

I am a noob. This sounds like goobledygook to me

@JulieNgTech Жыл бұрын

Sorry, that's why "simple" is in parentheses. Real world, esp. production code is always complex. If you do a KZfaq search, you should find plenty of beginner videos.