This is why you need a centralized logger on your software systems

Рет қаралды 14,741

Күн бұрын

00:00 - Introduction to Centralized Logging
00:05 - Logging Levels and Filtering
00:12 - Centralized Logger Overview
00:36 - Benefits of a Centralized Logger
01:05 - ELK Stack
01:11 - Elasticsearch
01:17 - Logstash
01:24 - Kibana
03:37 - Power of Centralized Loggers
04:52 - Using Centralized Loggers in Production
06:21 - Conclusion
📘 T3 Stack Tutorial: 1017897100294.gumroad.com/l/j...
🤖 SaaS I'm Building: www.icongeneratorai.com/
✂️ Background Cutter: www.backgroundcutter.com/
💬 Discord: / discord
🔔 Newsletter: newsletter.webdevcody.com/
📁 GitHub: github.com/webdevcody
📺 Twitch: / webdevcody
🤖 Website: webdevcody.com
🐦 Twitter: / webdevcody

Пікірлер: 58

@LucasFariaDev 10 ай бұрын

Cody, thank you so much for your content. I feel like you're one of the few content creators that is actually providing content for mid-level to senior+ engineers, which I deeply appreciate. Enough is enough with all this "create your Netflix/Twitter/Instagram clone" and similars. You're actually helping me getting my senior promotion (hopefully next year). Please keep it up.

@WebDevCody 10 ай бұрын

Glad to hear it!

@_timestamp 10 ай бұрын

I agree with you. Cody is doing something different here. Nice job. Really

@eshw23 10 ай бұрын

I agree, even though im a junior this stuff really sets a high bar for me on what I should know and learn, etc

@nova2wl 10 ай бұрын

Logging is huge, especially in larger orgs where uptime is mission critical. I am not a dev but work with the operations/ reliability team and logging allows our 24/7 team to identify issues faster and able to engage the correct team more accurately. We work with the devs and SREs to make the logging more accurate, create alerts and dashboards to make it easy for less tech operators to react to alerts that matter. Logging is boring but very useful lol

@oSpam 10 ай бұрын

Thanks for the last few videos Cody, I’ve recently added logs on my prod apps and they’ve picked up so many things I missed! Really appreciate it

@maddriven07 10 ай бұрын

Thank you for this kind of content that gives newcomers an idea what the industry is using outside of common stuff like React, NodeJS, etc

@katanaut 10 ай бұрын

Love your videos and learning a lot. thank you ❤️

@nimmneun 10 ай бұрын

Thanks for the video 😊 Looks similar to Graylog which we use at my current job for application side logging.

@thantzinoo938 10 ай бұрын

If possible please make a deep dive video on ELK stack, really love how simplify things...

@williamschaefermeyer7007 10 ай бұрын

I like the overview videos a lot. It'd be really nice to have a video on how to set this stuff up also

@sohansingh2022 10 ай бұрын

Nice!! Thanks 🙏

@hichamlotfi3668 10 ай бұрын

great video thanks

@KerimWillem 10 ай бұрын

You make me a better developer. Thank you :)

@keshavakumar9828 10 ай бұрын

Even though I don't understand much of these terms i come here just to expose myself to these new vocabulary. And later on learn them.

@keshavakumar9828 10 ай бұрын

Can we get an tutorial on elk stack please

@NoobaLV 10 ай бұрын

Could you use ELK stack with a service like Sentry? How do they compare? It feels like Sentry is used for catching errors in a detailed manner (stacktraces) where as the ELK stack would catch errors and general system information. I would like to hear your input because this seems like a tool my team could use

@cas818028 10 ай бұрын

We use glitchtips self hosted version and since it’s compatible with all of Sentry’s SDKs there is a lot you can do with it. Also quite a bit easier to setup then then solution mentioned above

@rotivanov 10 ай бұрын

Dude you rock! Also mate, would you mind sharing the brand/model of your main monitor, please? I am looking for a good monitor for my Mac mini. Thanks a lot for all the shared knowledge!

@WebDevCody 10 ай бұрын

I use my laptop monitor

@rotivanov 10 ай бұрын

@@WebDevCody thanks a lot Cody!

@SeibertSwirl 10 ай бұрын

Oooo a centralized logger huh lol you’re doing great babe! ❤u

@WebDevCody 10 ай бұрын

Love you! 😘

@pyrocentury 8 ай бұрын

How would you implement client-side logging? Winston is only used for Node.js logging. Seem like I'll have to implement an API backend that I'll make a post request to when errors are caught.

@WebDevCody 8 ай бұрын

yeah, you'll client will need to add logic to catch all exceptions and forward them to a authenticated backend endpoint so you know what errors happen for which users.

@Dom-zy1qy 10 ай бұрын

So this type of logger; do they typically send the longs to the centralized logger as they happen, or do they store the logs locally then send them in batches, or is this dependent on the framework / project / config.

@ccccjjjjeeee 10 ай бұрын

I work at Axiom (6:14 in the video). Our SDKs batch by default, but also have the option not to (as well as a lot of other config options to override the default behaviour. I don't know all the details about every other service, but in general I would expect them to not be too different in this regard.

@yarmgl1613 10 ай бұрын

we batch our logs, usually writes 100 at a time or if 10 seconds have passed

@WebDevCody 10 ай бұрын

I’m assuming it’s based on configuration, but batching would be the most performant for large amounts of logs I’d think

@Dom-zy1qy 10 ай бұрын

Thanks for all the replies guys, I guess batching would usually make the most sense, cause you could just adjust the batch intervals if you need really up to date logs anyway. I can imagine there might be some systems where concurrent logging is required.

@WahabSurfali 10 ай бұрын

Good 👍🏻

@GDEVELOPER0 10 ай бұрын

Hey Dev Cody, I wanna setup my custom back-end. I saw your videos and you're astonishing. Can you please create a video on how to setup a scalable application with laravel and docker. Or you can create a course for that! It'll be an ace of course.

@WebDevCody 10 ай бұрын

I don’t really use laravel, but I’d just find an existing docker compose file, I’m sure there are a lot

@GDEVELOPER0 10 ай бұрын

@@WebDevCody Oh Thank you so much

@congminhluu5068 10 ай бұрын

I've heard of this so many times but never actually implement it (I suppose as a web dev knowing that it exists is enough right?) There is one point that confuses me: Why ElasticSearch when Kibana is already a search + dashboard ? Isn't that redundant?

@WebDevCody 10 ай бұрын

Kibana just communicates directly with elasticsearch. Elasticsearch is where all the data is stored

@congminhluu5068 10 ай бұрын

@@WebDevCody ah I see I see ty for the reply

@benocxx7906 10 ай бұрын

I would love a video explaining how to secure user data from devs having access to the server. For example, let's you're encrypting phone numbers, nothing stops a dev to access the server, take the encryption key and decrypt the fields. What are the different approach to solve this issue?

@WebDevCody 10 ай бұрын

Idk if you can other than encrypt on the ui and never let any dev know the private keys used to decrypt. You don’t trust your devs?

@benocxx7906 10 ай бұрын

@@WebDevCody I trust my dev, but I've always been wondering how big companies with a lot at stakes do

@G0NZA11 10 ай бұрын

Have you used sst with kafka or msk? How is the support for sst to work offline? (Not only live lambda but db and others). Thanks!

@WebDevCody 10 ай бұрын

I only use sst for deploying, not for running anything locally

@G0NZA11 10 ай бұрын

@@WebDevCody gonna have to give a little try to see if I can replace serverless offline. Thanks for your time and your content, have a good one!

@WebDevCody 10 ай бұрын

@@G0NZA11 what issues are you having with serverless offline? I've used the serverless framework as well, it seems to work ok

@G0NZA11 10 ай бұрын

@@WebDevCody it's getting too big and too slow to deploy, and iac seems better. Also the plugin ecosystem is kind of dead, we already have some patches on the serverless-offline family of plugins (dynamo, sqs...)

@WebDevCody 10 ай бұрын

@@G0NZA11 I’d consider serverless framework as an IaC tool. our project started with serverless framework, but we found aws cdk is too slow and was brittle 4 years ago. Now we deploy everything using terraform, including all our lambda function (which kind of sucks to be honest), but I’ve found terraform to be a much better IaC tool. SST uses aws cdk under the hood, so I’d also be a bit cautious migrating to it without testing stuff out

@LawZist 10 ай бұрын

Great video! What log service do you use in your saas application?

@WebDevCody 10 ай бұрын

ELK and aws cloudwatch

@LawZist 10 ай бұрын

@@WebDevCody from your videos i understood that you don't use vps, how do you deploy your elk stack? thanks again!

@WebDevCody 10 ай бұрын

@@LawZist we use aws opensearch which has kibana built in

@damonmedekmusic2015 10 ай бұрын

For clarity, this would NOT be a solution for browser client side logging. This sounds more for server side Node systems or even lower level software/hardware configurations. For client side debugging, having tests run is also a great proactive solution.

@WebDevCody 10 ай бұрын

There are ways to ship any front end exceptions to your api so that are also indexed into you logging system