Correction: The label at 4:46 should be inter-zone networking, not intra-zone. It is the network round trip between availability zones. The closed caption and verbal explanation are correct.

Another interesting intuition is to put those times on a human scale by setting 1ns CPU time = 1s human time. At this conversion ratio, a clock cycle is ~0.1-0.2s, which is more or less instantaneous for humans. This could be, for example, the time to add 2 small numbers that you already have in memory. If you need to read them off of a page in front of you (L1/L2 cache), that could take a few seconds. Going further, a system call takes a few minutes, context switching is an hour or two, HTTP request takes several hours, you need to wait a day to read something from SSD, and when writing it's gonna be in the mail for around a week. Seek time of HDD is like 2 months, if you need data over the network from another continent that's a few years, and a TLS handshake is like a decade. And God forbid you need to transfer 1GB over network in same cloud region - that's like 3 centuries.

Your content is awesome. The amount of knowledge per time is fantastic. Only facts no bs. And the visualizations are great. Thank you

in the UI if something takes longer than 100ms it becomes noticeable by the user. like if you click a button and it takes more than 100ms to react to the click the action doesn't feel instantaneous anymore

Have you consider to record videotutorial (or maybe whole course) about how to create such amazing visual effects for presentation? Your team very good at it, and those knowledges would be priceless

This channel deserves to be a lot more popular. It is head and shoulders above almost all other IT channels I have encountered on KZfaq.

This is a sheer jewel. Although I knew most of this by experience and interest, having a summary like this is incredibly eye opening. This should be standard curriculum in the first year of computer science education.

I literally love "working with you". Your videos and newsletters (and books) make my data science programming so much more effective and significantly easier to deploy to production. Thank You - you've taken my career to an entire new level.

Another great video. Have you created content that shows the impact of this latency on system designs? Examples: * Accessing memory by reference instead of cloning the content. * Single threaded, vs multi-threaded applications * Async-await vs multi-threaded applications * Impact of microservices and containerization * Impact of serverless I often feel that latency and the cost of communication is forgotten in the scale-out discussion. I have personally seen examples where a containerized microservice architecture was 40-200 times slower than a non-containerized approach. Have you seen the same?

> Latency Numbers Programmer Should Know Average Python guy who use openCV for croping images: Thanks, it's so helpful!

The many round trips of the TLS handshake is something that is actively being improved upon. TLS 1.3 currently supports 1-RTT handshakes and initial support for 0-RTT handshakes.. which would basically remove the delay of the TLS handshake all together as it can be performed simultaneously to an actual content request call.

The knowledge density of content on this channel is through the roof. The books are great too. Impatiently waiting for more videos! Keep it up!

This is one of the best channels ever.

I've been lucky enough to enjoy two simultaneous careers in the 35 years I've been working professionally, one in enterprise database design & troubleshooting, and the other I'm an RF mixed signal electronics engineer: I go all the way down to pico & sometimes 100s of femto seconds for the electronics side. One of the most useful facets of having experience in two apparently largely mutually exclusive skillsets is when it comes to performance & troubleshooting, whether it's knowing what looks about right, or knowing what's reasonably achievable. In my database work, I usually talk in terms of orders of magnitude improvements, but in my electronics work it's more often just ~10% or 20% improvements, as we're already hitting the state of the art. The same dependencies on both experience and underlying understanding are required. In terms of analytical skills, it's pretty much identical: you split a given challenge into manageable chunks, identify the area(s) where a problem lies, re-engineer accordingly., and put it back together. The two are similar in so many ways, even down to Heisenberg uncertainty, such as sticking in instrumentation into code or probing a high speed interface: both can substantially interfere with the result.

1 ns = 10^-9 s 1 us = 10^-6 s 1 ms = 10^-3s Register < L1/L2/L3 caches < memory < SSD < Harddisk < intranet < internet < TLS

main memory 10-100us - Network proxy HTTP request routing - Read 1 MB sequentially from main memory - Read 8 KB page from SSD 100-1000us - SSD write - Intra-zone networking round-trip 1-10ms - Memcache/Redis get round trip (~1 ms) - Inter-zone networking round-trip (~5 ms) - HDD seek time (~5 ms) 10-100ms - US east to west coast or Europe network round-trip - Reading 1 GB from main memory 100-1000ms - bcrypt a password (~300 ms) - TLS handshake (~250-500 ms) - US west coast to Singapore network round-trip - 1 GB sequential read from SSD >1s - 1 GB network transfer in same cloud region (~10 s)

I love the way each of the videos is short and rich with useful info that helps to expand your software engineering knowhow 👌

trap -> kernel space -> trap -> user space, not accounting for processing time of the system call itself) - MD5 hash of 64-bit number 1-10us - context switch between Linux threads (does not account for bringing large # of pages for the new thread, which would extend time further) - copy 64 KB memory to new location 10-100us - typical amount of time for network proxy to process a http request (nginx) - read 1 MB of sequential main memory data - read 8K page of SSD 100-1000us - SSD write latency for 1 page - intra-zone networking latency [100us] - memcache/redis get operation as measured by client [1ms] 1-10ms - inter-availability zone of modern cloud provider networking latency [5ms] - hard disk seek time [5ms] 10-100ms - us-west coast to us-east coast latency, us-east coast to europe latency - read 1 GB of sequential main memory 100-1000ms - bcrypt a password [300ms] - TLS handshake [250 - 500ms, depends on distance between machines] - us-west coast to singapore latency - read 1 GB of sequential SSD data 1s - transfer 1 GB over cloud network within same region [10s]

I am still astonished that content of this quality can be *free. It gives me a lot of hope for the world. Thanks you much and keep it the great work

Also important 10-100ms the time between two image in animated image. For a game you should typically aim at getting everything done in 16ms to hit 60fps. Or finish all calculations in the 1-10ms range to be able to scale to higher frequency displays. ~100ms is also the maximum amount audio and video can be out of sync before people start noticing it. Already at ~200-300ms it becomes difficult to hold an audio conversion, as people start missing queues about when it is their turn to talk.

Useful information. A bit of connected data: 10 milliseconds of difference between two notes sounds fine, and imparts a certain "feel", but 30ms starts to sound "off", like one is early or late.

These videos are cool! I miss the videos where you explain more basic concepts such as TLS or other protocols. I found those the most interesting

I love the last one, takes 10 sec to transfer 1GB over network within same region .. then some webpages take several seconds to load a little bit of text and a few pictures.. I mean there are of course many things at play here, it's not only the transfer, which is considered here, but still we have a lot to improve on in general when it comes to web. And I don't only mean the programming I mean choice of server OS and so on. Some of the extra latency can be due to hundreds and thousands of processes being run concurrently on the same servers . So if the kernel interrupts every n mikroseconds, it takes the amount of nano/microseconds for every process to go from userland to kernel land which adds up until the scheduler gets back to the process again.. And storing and loading the process state in it self also requires cyckles aka time to do every time. Then there are more stuff going on as well.. (I didn't remember the numbers presented in the video but there all there in the video :) ) We need less bloaty things even though hardware gets better. Sometimes today surfing the web is almost the same experience as the early 2000-s..

Human visual reaction time is about 200ms. An interface response faster than this has diminishing returns.

100 to 1000ms: User perception of a "fast response" on the internet is around this range. (Google says 200ms.) That means if you want your page to feel responsive, a response better be delivered and rendered to the user before 200ms have passed after a mouse click or screen tap. 1s to 10s: Range of spindown time of spinning hard drive. 10s to 100s: TikTok and other short format videos are in this range. User attention becomes this 15s limit, meaning users get bored super quickly. (Especially young users.) Also, interestingly, if a system does not respond in this amount of time, users might think your program has locked up and try force closing/rebooting/etc. Along the same lines, most UNIX and Linux systems send SIGTERM to all process, followed by 5 seconds wait, followed by SIGKILL. (Your program has 5 seconds to clean things up before it's forcefully killed.) 100s to 1000s: TCP maximum retransmission time is 120 seconds by default. If DRAM is cooled, the contents can be retrieved in this amount of time after power is removed.

Awesome video. This was important to get the time intuition of different processes.

This channel is incredibly handy

You're doing good work. I think this video will be helpful for anyone in computer science whether a student, professional or someone who is just curious. Also the visualisations are great. Thanks.

It always amazes me how few design based on these factors that can affect performance at such huge levels.

I learned these things in the 1990s, so nice to have an update. Wish you had mentioned cost of an ordinary subroutine call.

This video really helped me. Thanks a lot!

Your content is highest quality! And very insightful!

Thank you. Very important and helpfull context

Excellent video. I definitely have intuitively known some of these numbers for a long time being just a super nerdy programmer with a gamedev hobby. However, I came up a while ago not knowing a lot of the cloud numbers, and L3 cache didn't exist in CPUs and maybe not even L2 in the earlier days. No surprises really exist other than SSDs being a lot faster than HDDs. I've known intuitively that network transfer rates (bandwidth, not latency), but now even latency of network access is faster than an HDD. In otherwise, a locally connected HDD is _slower_ than an intrazone networking transfer if the server is reading from main memory or significantly faster IO storage.

Coming from a game dev background, I'm used to thinking of latency in clock cycles, and anything more than a few hundred is "too long".

That redis cache read number is pretty surprising. I suppose that's better than having to get a DB to get a result, but network latency for intra-zone is only 300 micros, so if you can just have a fast database, you won't lose a bunch of performance there. Honestly, if you're not using a cache system within your app, you're leaving a ton of gains on the table.

Fascinating to put this into perspective! Specially access speed differences between registers, cache and memory. Also context switching for threads

I would have loved to see gpu read and write latencies on here but I really loved this video

It is worth mentioning 100ms border of a human perception. Any operations faster then this we treat as "instant". Anything longer - as having some delays.

I'd be interested to see thread switching and kernel call latency comparisons between Linux and Windows.

simple yet elegant.

This is awesome content !!! Thanks.

I share your videos with all of my developer friends. Great for a refresher, summary, or introduction to these concepts. Keep up the excellent work!

Sound like a full course for me!

Decades of knowledge in 5 minutes ❤

Great summary of meaningful latencies. I think it would have been nice to include human reaction to in 100-200 ms range. This is the minimum time humans need to actually react any new information, not matter how little semantics that information contains. An example would be ability to press a mouse button after something appears on the screen. Another good latency to include would have been 10-20 ms for the minimum time to get something on the screen on most computers.

Another good important point PCI-E devices roundtrip is about 500ns

Minor nitpick: bcrypt has a variable workload parameter, which allows one to freely configure how long it will take to calculate a hash for a given amount of data on a given hardware. Thus, saying a slow hash-function takes between 100 and 1000 ms isn't really correct. It can take however long it is configured to take.

Fascinating, thanks. It had not occurred to me before that a (relatively) slow hash algorithm has the advantage regarding resistance to brute force trials. I'd love to see a comparison of server response latencies; I'm guessing a liteSpeed server using ssd storage located a continent away would respond faster than a local apache server with a moving disk.

Amazing video! Loved the visuals and the explanation!

Glad you had enough time available to make this video.

I am very impressed with your script writing. It is more clear and efficient than any other I've seen on this entire website. Extremely effective.

I would add that at 10-100ms is the range which many consider to be "real time", you should have latency under 15-30ms if you want to attain 60-30fps due to some complex command like computer vision.

Nice videos ...thank you.

Thank you! I have bought both your books!

Awsome video, both explanations and animation are fabulous! Thanks a lot!

100ms -- the time at which a human being regards a response from a computer as "instantaneous" -- for example, if the user presses down on a key "X", and then an "X" appears on the screen within 100 milliseconds, then the human being judges that the computer responded "instantaneously" -- VERY IMPORTANT for programmers to know!

You have to put into perspective the distance light travels in these times. In 1 ns light only travels ~30cm. With a speed of electricity of ~2/3 c that means it is physically impossible to access information further than 10cm from the CPU in 1ns. That means anything the CPU has to do in 1 clock cycle of let's say a 5 GHz CPU has to be physically within 2cm (a bit less than an inch for the americans). More realistically, it has to be within 1cm as any logic also takes time and electrical connections are rarely point to point. And this might soon or already now need a 100ps-1000ps range as clock speeds are getting faster again.

Simply thank you very much for your videos!

I would find it cool to add perceived latency on the ui. there are papers on this, like when does a ui feel "laggy". For example a button has some highlight effekt on mouse over and if the effekt takes too long to start it feels laggy. I have in the back of my head the mark of 50ms for those things, but could be wrong.

35-40 years ago, the latency hierarchy on a PC was: 1) Registers 2) RAM 3) HDD 4) Floppy drive 5) Monitor 6) Printer

Thank you for this very interesting video!

What blew my mind was that accessing main memory on an M1 Mac is in the same order of magnitude as accessing the L3 cache 😳

this is why for low latency software you do not use any databases... your system is a database except its better, because unlike a general purpose database your system is focused to do specific things

Would have loved to see a similar video with the comparison to how tech has evolved , anyway great video 🎉

Great video!

Wow greate explanations, thank you

Shouldn't reading 1 GB sequentially take about 50 milliseconds if reading 1 MB sequentially takes about 50 microseconds? Also, does an intra-zone networking round trip take 100 microseconds (according to 4:16) or 5 milliseconds (according to 4:46). Or is 4:46 referring to intra-zone networking latency instead of an intra-zone networking round trip?

Very nice video! Thanks!

A good sister video would be to review these numbers for an embedded system, maybe an ARM microcontroller.

You really picked up publishing pace ☺️👍🏻

Your videos are amazing. Thanks!

amazing videos It must took you a lot of effort to combine the resources and put it in a simple way , for viewers they need to take notes and then understand each things mentioned in the video :)

Greate video, thanks for making it! I think this knowledge is useful when reasoning about software, for instance if you have a simple task that takes a second, you might have made a mistake somewhere, like a n+1 query. If you don't have this intuition you might think: Oh, it's just a second. But a second is literally an eternity for a computer. If your operation takes a second, you better know why!

Awesome one - Thanks a lot

With the advent of NVMe SSD storage, the difference in latency accessing classical SSDs and NVMe SSDs would have been interesting to see included here. Especially since NVMe technology allows for sub millisecond latencies, breaking the order of magnitude barrier.

I liked the video. See you in 10 years for the new version !

Great content as usual, and amazing presentation to explain the same. Loved it 👍

Hey, nice plaque!

This is great - going straight to my bookmarks, thankyou :)

Very nice 👍 Helpful to have this kind of intuition while debugging performance issues in complex distributed systems

Great video! I'm curious about how you got the estimations for these operations.

Excellent, can be use for some software development. You have PDF version of this video, like cheat sheet ? Btw, if you don't mind, request video for "Acceptance latency number for user/client" so any software development including game, desktop, and of course web can have general idea how long they can ask user to wait. Thank you.

I don't know where you got this data from but on my laptop syscall(39) takes ~60ns, md5 of 16 bytes takes ~80ms

Some of the ratios between some obf the numbers have been surprisingly constant over many years such as memory access latency expressed in CPU clock cycles.. Sure, the exact values are changing but these exact values rarely matter.

Great content and presentation. Keep up the good work!

What a very fascinating thing to talk about.

Amazing video once again!

That’s why asyncio and coroutine are way better than multithreading NIO, and why I think Python + asyncio is a decent replacement for nodejs or even Go at API layer for folks enjoy coding in Python.

Thank you! It is really helpful)

thanks for the info good representing and good editing

The suggestion at 0:37 that there’s something in my system that doesn’t need to obey the laws of physics has me intrigued

Interesting!!!

Good content. How about some programming operations like add/subtraction/multiplication/division/assignment in different language on a regular PC by today's standard?

We should expect SSD latency to reach that of ramdisk accesses that don't fit into CPU cache during the 2020s as F-RAM and FeFET memory become large and cheap.

Great video!, how do you create those animations for your videos?

Thank you for sharing!

The day we will invent communication faster than light, will be great for space, but also for computers / internet

From 1s down to 1nano second would have made much more sense. Starting from something 'relateable' down to something almost "abstract" instead of something we have no direct "feeling" for and working back to something we know.

The beauty of computer science in one video