Brief Outline 00:00:59 Notification Service 00:02:12 Problem Requirements/Capacity Estimates 00:03:56 Fan Out Design Pattern 00:06:38 Idempotence 00:08:17 Idempotence Continued 00:11:17 Idempotence Continued 00:12:16 Bloom Filters 00:14:15 Client Connections to Notification Server 00:16:56 Final Diagram - Notification Service Thanks, Jordan~

My third favourite system design KZfaqr just uploaded, sweet!

Really appreciate recording and sharing these videos. They are super helpful. For idempotency part, I think we can give each message (per user dimension) one increasing number and keep maximum number on server side. By comparing service and client side's number, we can know if any message is missing or not.

you seem to actually know what youre talking about, its kind of ridiculous how out of thousands of these types of videos every single other one i've seen is either from ex-swe's that seem like they were only interns with how they've "designed" some systems. sincere thank you this was extremely helpful for creating a notification system for a social app im working on for context my needs are pretty basic/simple, notifications for actions, follows, etc. say a user likes a post, my current approach is 1. create the notification row in the notification table, referencing the action 2. in a sorted set with key "user_notifications": use a batch key as the score and the the id of the notification in the database as the value. the batch key looks like "like_postid". 3. on pull request, get all the values in the sorted set and first using the batch key, batch all things with the same key together and increment a count. using the most recent notification id, get the relevant data and attach it to the notification. 4. store the batched notification data in the database and return the batched notification to the user. the final 2 steps hits redis to get all the ids, then it hits the database and redis again to store the batched notifications, and finally returns it to the user. im not sure if this is a good approach at all

Ive been waiting for this video. This is one of the basic services

Storing the idempotency keys on client is a wild idea, and can be easily treated as a red flag. I wouldn't mention this as an option in a real interview.

The design assumes we have full control on the clients and we can control the logic of the clients to have bidirectional communication, but in reality we often don't have control over the clients. Notification to the clients often requires existing infra (e.g. Apple APNs) which we need to integrate with but cannot modify its protocol.

Love your videos Jordan! What are your thoughts on creating videos where you build simple projects/programs that implement certain concepts from your systems design playlist? I personally can't find other channels that teach these concepts as well as you, and I feel like doing so would be pretty complementary to your already quality content.

How does Flink handle (topicID userID) update? What if there is a new topic? What if a new user subscribes to an existing topic? What if a topic is deleted etc?

Jordan, another great video, thank you! Would you be interested in doing the system design of a Google Sheets like application(not the collaboration functionality that you have already covered in Google Docs) but with features more closely resembling Airtable/Causal, focusing on real-time formula application?

Hey Jordan, watched all your videos...blah blah blah... great content love it blah blah blah... funny descriptions blah blah blah... now that all the formalities have been taken care of (lol I promise I'm a fan), could you make the OneNotes for all the Sys Design Interview problems accessible as well please? I love going back to the playlist 1.0 powerpoints for concepts, but it would also be really nice to have these oneNote accessible too - ideally in a way I can clone and scribble lotion and tissue boxes on and oh annotate stuff. Willing to send feet pics if you get the job done before my interviews :P

we have two kafka queus, one is a CDC from topic subscriptions table which is partitioned by userid, and the other is partition by topicid. Both ingest data to Flink, so it needs to shuffle one or the other, doens´t it?

Thanks for the video. How feasible would that be to build reverse mapping at scale? Sounds like expensive operation + at what cadence, we will either spent a lot of compute rebuilding revery sharding or there will be a signiy lag beyey users subscribing and starting to recrcey notifications for that topic.

Hi Jordan I have a very basic question. Are we building the notification service for all notifications (from all applications) for all users ? In the final diagram, you have mentioned that flink node receives data from 2 sources. 1. Topic subscriptions of user. Basically topic_id --> userId's. 2. Messages from Notification Queue. topic_id --> message. Using above 2 sources, how does the flink sends the real time messages ? Some messages are user specific right for example one to one pings. Sorry if this sounds silly.

Jordan, Can you use the bigger mouse pointer, that way we can track, which box you are talking about? Thanks

You forgot about 2nd requirement - Offline clients must be able to receive notifications later

Could you explain a little bit more about what flink operator it use to handle this flink mapping?

Why is the subscription change events topic sharded on user id? I expected it to be on topic id as this stream needs to be joined with the events in notification queue which are also partitioned on topic id

Thank you.

only fans i m 🤣🤣

Video Editing is too low 👎👎👎👎 Edit the video in a good style. Add nice b-roll animation when you are speaking or explaining something. This will increase visualization. Put proper subtitles, effects, transitions, and colors in the video. Insert the B-Roll (footage and animation) very well because it is absolutely low in your video. Do all this and then see how the video doesn't turn out good. 🍏