Event Driven Architectures vs Workflows (with AWS Services!)

Рет қаралды 87,447

Жыл бұрын

Event based applications and workflows are two ways to coordinate actions across a distributed system. Both have pros and cons and are appropriate in different circumstances. In this video, learn about Event Based architecture vs Workflows and how they could be built using AWS services.
Looking to get hands on experience building on AWS with a REAL project? Check out my course - The AWS Learning Accelerator! courses.beabetterdev.com/cour...
📚 My Courses 📚
AWS Lambda - A Practical Guide - www.udemy.com/course/aws-lamb...
🎉SUPPORT BE A BETTER DEV🎉
Become a Patron: / beabetterdev
📚 MY RECOMMENDED READING LIST FOR SOFTWARE DEVELOPERS📚
Clean Code - amzn.to/37T7xdP
Clean Architecture - amzn.to/3sCEGCe
Head First Design Patterns - amzn.to/37WXAMy
Domain Driven Design - amzn.to/3aWSW2W
Code Complete - amzn.to/3ksQDrB
The Pragmatic Programmer - amzn.to/3uH4kaQ
Algorithms - amzn.to/3syvyP5
Working Effectively with Legacy Code - amzn.to/3kvMza7
Refactoring - amzn.to/3r6FQ8U
🎙 MY RECORDING EQUIPMENT 🎙
Shure SM58 Microphone - amzn.to/3r5Hrf9
Behringer UM2 Audio Interface - amzn.to/2MuEllM
XLR Cable - amzn.to/3uGyZFx
Acoustic Sound Absorbing Foam Panels - amzn.to/3ktIrY6
Desk Microphone Mount - amzn.to/3qXMVIO
Logitech C920s Webcam - amzn.to/303zGu9
Fujilm XS10 Camera - amzn.to/3uGa30E
Fujifilm XF 35mm F2 Lens - amzn.to/3rentPe
Neewer 2 Piece Studio Lights - amzn.to/3uyoa8p
💻 MY DESKTOP EQUIPMENT 💻
Dell 34 inch Ultrawide Monitor - amzn.to/2NJwph6
Autonomous ErgoChair 2 - bit.ly/2YzomEm
Autonomous SmartDesk 2 Standing Desk - bit.ly/2YzomEm
MX Master 3 Productivity Mouse - amzn.to/3aYwKVZ
Das Keyboard Prime 13 MX Brown Mechanical- amzn.to/3uH6VBF
Veikk A15 Drawing Tablet - amzn.to/3uBRWsN
🌎 Find me here:
Twitter - / beabetterdevv
Instagram - / beabetterdevv
Patreon - Donations help fund additional content - / beabetterdev
#aws
#SoftwareDeveloper

Пікірлер: 71

@omrigoldberger9395 Жыл бұрын

There's a very good article about it called "Choreography vs Orchestration in the land of serverless", TL;DR: Use both in the same project, using orchestrators inside a bounded context and choreography between bounded contexts.

@BeABetterDev Жыл бұрын

Good article!

@tracy_gao Жыл бұрын

Did the author said how could define the boundary? AKA when to use orchestrator and when to use choreography

@aliveagain Жыл бұрын

I was wondering if he would end on that note. Seems like together they would counter each other's cons.

@getrasa1 Жыл бұрын

Love the way you explain things. Very clear and concise. Great work!

@europana7 4 ай бұрын

Be nice if channel owner just mentioned a quick appreciation note for your generosity …

@julianfranco7689 Жыл бұрын

Fantastic video! Extremely clear and thorough on both patterns

@mahdyyk Жыл бұрын

As always, this video covers everything in a comprehensive manner and makes it easy to apply practically

@JohnS-er7jh 10 ай бұрын

so glad I found this channel, much appreciate your effort. I am just learning AWS and this mthod of teaching find is so much easier to understand. Whereas others will just mention Services in silos and then start drilling down into all the granual info on each Service (without even explaining how it relates/connects to other services (making it difficult to get your head around it all).

@BeABetterDev 9 ай бұрын

You're very welcome and welcome!

@flashflexpro Ай бұрын

This is a great video, and can be greater when adding the monolith client retrying approach because tons of people are still stuck in there.

@deedoi Жыл бұрын

Thank you very much for your explanation, which is nice and easy to understand. 👍

@LawZist Жыл бұрын

Great explanation!

@et4493 Жыл бұрын

Great job man. Thanks

@viewerone 5 ай бұрын

With event driven you could use traces for observability to visualize what step you’re at.

@chetan3283 Жыл бұрын

Thanks for sharing this information, useful 👍

@Guzguz28 Жыл бұрын

Thank you for a great presentation

@BeABetterDev Жыл бұрын

Glad you enjoyed it!

@kipsangjacob270 Жыл бұрын

Awesome video

@explorer26 Жыл бұрын

Thanks for the video and example, learned new kinds of stuff on EDA and workflow. I was thinking, in the case of the EDA approach again for monitoring, maybe we should log or publish the state and the success/failure of every operation. Triggering a lambda on failed operation/step could also help us in unwinding initial steps and marking all of them as invalid. Does it make sense?

@shubhamdhingra6089 Жыл бұрын

Very helpful!

@pengdu7751 Жыл бұрын

great video. it reminds me of the difference between OOP (mapped to EDA) vs non-OOP (mapped to workflow). Solving the same problem but modeling the problem differently which leads to various tradeoffs

@TotallyNotAuroras2ndChannel Жыл бұрын

Wow. This is what I just said.

@josephattabenninjr7317 Жыл бұрын

another great video as always!

@BeABetterDev Жыл бұрын

Glad you enjoyed!

@dinushkam2444 Жыл бұрын

very helpful video

@turboboostcash3876 Жыл бұрын

Thank you so much for the explanation.. pleas can you do a video more on Aws IOT service and also the more on database

@deepsharma9056 Жыл бұрын

Informative! What s/w have you used to show the workflow?

@benisrood 3 ай бұрын

There are so many more sub-events in the logic and execution in handling the transaction steps in a distributed system just for placing the order based on the user's shopping cart that you gloss over. That would be enough for a single video alone. The credit card service would be inside the order placement, just for starters! Before doing any subsequent fulfillment steps let alone packaging, a hold would need to be placed for the required funds on the user's account with the card issuer 😂 You need something like Kafka/Kinesis as an event log to distribute the events to appropriate consumers, etc etc. I do agree that the Workflow approach has significant benefits, and you are right about the costs and tradeoffs of orchestration with this approach. I just think that you need to go into a bit more detail with the event-based architecture side, because your diagram is extremely oversimplified. And I don't think IoT is an essential aspect of the system at all, these sorts of systems functioned before such devices even existed.

@arkster00 Жыл бұрын

We heavily use Step Functions at work. We don't require a rollback in the event a problem happens but we do need to know that a failure happened. For that we use Cloudwatch integration with slack that monitors each lambda in the workflow. Note that in each step function task that executes a lambda, if mandated you could essentially add the retries, exponential backoff and trigger details as well to reprocess the lambda and perhaps move it into a DLQ for redriving it at a later time. BTW, thank you for all the AWS content and easy explanation that you provide. I've benefited a lot from it.

@cristiannechita2032 Жыл бұрын

There is another option if you are using Standard step functions. You could create an Event Bus rule that reacts to Step Function transition to Failed and as an action it can publish the message to SNS and from SNS you can send it everywhere. But this is an option only if you use Standard. For Express I usually use the Async invocation and let the caller deal with the potential failure of the Step function.

@arkster00 Жыл бұрын

@@cristiannechita2032 Is there a benefit to doing that using event bus instead of having the step function routing the error message and input payload to say another step for SNS in the step function itself? I've tested this and it works well when a lambda fails and the retries are exhausted since step functions are async operations.

@cristiannechita2032 Жыл бұрын

@@arkster00 The benefit that I see is that you keep your step function cleaner (without putting those transition from various nodes towards a common error handling task). Also, there are Tasks without Lambdas (e.g. Pass) that can apply transformation and those can fail as well. This should not replace the retry operations of your lambda step (it makes sense to keep it to deal with any transient errors).

@arkster00 Жыл бұрын

@@cristiannechita2032 Thanks. Something to think about.

@pythonbrothersandfamily 7 ай бұрын

you are a super hero. thanks a looot

@priyankaojha1 Жыл бұрын

Is Prefect a good tool for production ready , asynchronous microservices based workflow orchestrion?

@ViktorSchmidt 6 ай бұрын

Looks like absolute overkill architecture for 99,9999 % of all apps and businesses 🤔. But, good explanation and overview of AWS madness. 😊

@edikzlotnik1 Жыл бұрын

Great lecture!! Which tool do you use to draw a visual representation of architecture ?

@BeABetterDev Жыл бұрын

Thanks Eduard! I used Google slides for the first part, and the Step Functions Studio Editor for the second half. Cheers.

@isohumulone Жыл бұрын

I am not sure what is happening between the two lambda functions that can determine the success/timeout/default of the first Choice node. An iot device? Like a sensor? Is it forwarding data? What reads that data?

@fooflateka Жыл бұрын

@Be A Better Dev can't you use Distributed Tracing to see status of orders in SOA?

@hectorharris4051 Ай бұрын

Hi what do you think about using SQS + Dead letter queue in event-driven architecture to help monitor

@pollypravir5378 Ай бұрын

Thanks

@dEXtER2ks Жыл бұрын

wouldn't you have to revert changes made in a step in case of failure when using step function approach, or you would simply correct the failing part i.e ask customer for another credit/debit card and retry? In my experience only durable functions on azure have the functionality to re-submit the request, i am sure there would be something similar on AWS end.

@easycloudcomputing6901 Жыл бұрын

Well it depends on where you redirect the customer. If you want your customers to pay with another channel then you can build your logic in triggering another lambda and at the same time route the user on the frontend of payments channels list, so that user can select the payment channel of his choice. Auto resubmission will make user stuck in the loop.

@mayukhee1 Жыл бұрын

what will be the best template in lucid chart to replicate the step function workflow part?

@sankara-sabapathy Жыл бұрын

Hi, Thanks for this video. I would like to know why we cannot maintain microservice architecture that are handled by individual small team while using step functions? Still the step functions can be created or maintained by the architect and we can have different teams to maintain the services right? Thanks.

@cristiannechita2032 Жыл бұрын

Look at the step-function as being part of a micro-service. That team owns everything regarding that workflow (from it's definition in IaC till deployment).

@BeABetterDev Жыл бұрын

Christian's point is spot-on.

@mr_possible6197 Жыл бұрын

So basically if my application has 3 services working independently, i can assume that those services can be monitored separately using step functions right ? Its to say, micro monitoring sortof a way, eh ?

@sankara-sabapathy Жыл бұрын

Got it. Thanks Cristian.

@jasonsimmons3959 Жыл бұрын

Step functions are great, but I’ve found monitoring to be a weak spot. The state display on the diagrams is very rudimentary, and you’ve got to use other tools and work with the logs to actually get a real monitoring dashboard.

@outofbody4788 Жыл бұрын

can you do workflow withut step functions?

@xpkareem Жыл бұрын

What are you doing step-function?

@thealys Жыл бұрын

Nice

@TotallyNotAuroras2ndChannel Жыл бұрын

This really brings back memories of when I used to use OOP in C++ to create an event driven application. Thanks! Now, let me implement this in OOP Python first.

@mihai6564 Жыл бұрын

good video

@BeABetterDev Жыл бұрын

Glad you enjoyed!

@austinlowther4156 Жыл бұрын

Great content, I've been watching you for a few months. I wanted to give feedback on the timing at the end of your videos with suggestions. Within a few seconds of suggesting other videos and overlaying them to be clicked, the video ends and your suggestions are no longer available. It would be nice to have a bit more time to react: read the video titles, determine the relevancy to what I am trying to learn, and then clicking.

@akritworanithiphong Жыл бұрын

What do you use to draw the illustration? 9:20

@egomezr Жыл бұрын

I'm almost sure this is the editor for StateMachine (a.k.a StepFunctions) in AWS.

@rahulvarma2100 Жыл бұрын

Good video. Reality is fancy bla bla architects see the first diagram on event driven choreography as a short running stateful work flow just because there are arrows in diagram & then bar of argumentation starts with integration SMEs 😂😂

@Euquila Жыл бұрын

Workflow is Event-driven with global state.

@rishiraj2548 Жыл бұрын

👍💯

@MindMusicKushMittal Жыл бұрын

all workеd

@yansongguo8354 9 ай бұрын

Not totally true😊 In the Eda design you are using SNS and it will generate chatty conversation between teams And if we focus on microservice only following the essence of DDD Each step in Workflow could be a service so just handling service with well defined bounded context will fit team responsibilities very well

@honjiaful Жыл бұрын

well, good video but seems this is not event-driven.

@sarc007 Жыл бұрын

Customer Charging happens first and rest other events follow

@BeABetterDev Жыл бұрын

Not always. Cards are usually pre-authorized for a charge and charged only upon shipping.

@philtherealdeal2435 9 ай бұрын

While I love this vid, I think @sarc007 has a point. This is almost never the case in reality. Companies want your money ASAP and will confirm pre-auths in a nightly batch, or up to 3 days later. If your shipping flow gets stalled, you don't want your pre-auths timing out. It feels like you used artistic license to switch them around so you could make your point about observability.@@BeABetterDev