A Computerphile video on NTP? It's about time.

More protocols with people who were there to see them written please! I love the anecdotes

The reason why you back off the polling interval is to allow the effect of the frequency error between the two NTP peers to accumulate enough phase error that can be detectable. There's also some cool algorithms at work here, where as you get closer to the synchronizing the correct local clock frequency, you '"tighten up" the phased-locked loop that's corrected by measured clock offsets (after going through a filter). The clock offset computation also needs to have half of the measured delay time accounted for (which was missing in the video.) Of course, this assumes that the network latency across the network path is symmetric... which isn't always the case. Along with the NTP peer stratum number, another metric, "dispersion" is measured between NTP peers which is roughly how noisy the measurement is. An NTP peer will include the "root dispersion" which is the sum of the dispersions of that peer to the root of that synchronization hierarchy and a measurement of the quality. NTP is great fun. Along with Mike Petry, he and I built the first UNIX NTP implementation of the protocol, doing so only from the draft protocol document and peeking at the PDP-11 assembly code that Dave's implementation was built in. Mike and I were students of Dave's at the University of Maryland and enjoyed many time synchronization experiences ourselves. I built the NTPv1 UNIX implementation on a NeXT workstation in my office. One of the things the NTP program does is compute the intrinsic frequency error of the clock in your computer. It saves this in a file, so that in the event you lose connectivity to your synchronization peer, you can continue to apply the frequency correction, so at least your clock advances at a more correct rate than it would otherwise. From this, I could indirectly measure the temperature of the room as the cheap crystal on the computer's board had a relatively poor temperature coefficient, and it became clear that that the air conditioning in the office got turned off over the weekends and the ambient temperature would rise. And then there were the exciting New Year's Eve experiences! Will the code that handles leap-second insertion work correctly? A leap second is most frequently inserted at 23:59:59 UTC on 31 December, so about 7pm local time. I would disappear into my home office and watch NTP to see if the right thing would happen.. And then struggle to explain to family and friends why this had to happen right now.. Nice video! How novel to see NTP make an appearance on KZfaq!

As a network engineer I use NTP every day, but never actually knew how it worked. - Thanks for the explanation

Beyond NTP, for high precision time for science experiments and such (e.g. the LHC, or radio telescopes doing VLBI), there's White Rabbit developed at CERN. It can synchronize clocks to under a nanosecond over local distances (tens of km). I used to work in a place that had a dedicated optical fibre running 50 km from the national lab with atomic clocks to sync some radio telescope clocks.

I studied under Dr Mills. People would occasionally look at a file system / database protocol and say "But what if the client isn't sync'ed to the server?" He'd say "We solved that problem. Let's move on."

Brilliant! One of the things I love about computer science is that a lot of the people who worked on all these methods and algorithms are still around. Getting information straight from the source is always amazing, especially when its produced and presented as well as you do!

Rest in peace David L. Mills. We stand on the shoulders of giants.

A few years ago I moved from pure networking to networking/system administration in a very small company. Server care and maintenance hadn't happened for a while when I arrived and everything was a bit tottery and unstable. I wasn't quite sure what the best approach was but got NTP working across all servers, whether they talked to the outside world or not. This was the crucial first step to stabilising servers and their (inter-server) services. Thanks NTP, you are a winner!

It's really great that at 0:32 and 6:55 you can see a drawing of an 8 bit BBC Micro computer from the 80's. I have build a WiFi board for this computer and a friend of mine, Timo, wrote a program that fetches the time from a NTP server 🙂 Thanks for this explanation Mr Onions.

Back in the early 90s, I'd just started getting into Linux. I was running an Yggdrasil distribution on a 80486 (kernel versión 0.99PL14R). And I started hanging out with a local user's group - Unix Users of Minnesota. NTP clients had just started showing up for Linux, and all us geeks were busy getting things configured. So I'm waiting for the meeting to start, chatting with folks, and the moderator gets up behind the podium to kick things off. And then it turned 7:00 o'clock, and everybody's watch beeped. This was, of course, back in the day when people still thought digital watches were a pretty neat idea. There must have been thirty of us, perhaps twenty had watches with hour-beeps turned on, and we all of us beeped at exactly the same time. We'd all synced to our computers, and our computers were all synced to Internet time.

I remember people arguing about whose watch had the correct time. The only way to truly settle the argument was to fire up the shortwave radio and tune it to WWV in Fort Collins. Thanks to NTP and smartphones those tedious arguments are a thing of the past

The underlying architecture for the global NTP Pool is a really interesting as well. It was one of the first instances of GeoDNS to silently get clients connected to the closest regional servers

I’d love to see a video expanding on this to PTP!

This is awesome. Can we get one of these for Precision Time Protocol?

A student in my uni class did his dissertation that included a case of using NTP in a method of DDOS attack around 2014 (the "derp trolling" group, if that rings a bell for anyone). At the time it broke records on both total bandwidth used up in an attack and the return they got for every bit they sent (I think it was in the 400 bits per bit sent amounts?)

There's just something wonderful about seeing multiple clocks perfectly synced. I love NTP

I've been involved with the NTP Hackers team for 25+ years, wonderful to see such a nice explanation of what/why/how it works! A few small details: In ntp infinity equals 128 ms! I.e. if a local clock has a time offset larger than this limit, it will reset (typically by jumping the clock) and start the protocol engine from scratch. I.e. the combination of multiple seconds of offset and skewing to correct this doesn't actually occur, unless you have configured ntpd with non-default settings. The protocol will always assume the round trip delays to be symmetrical, but with the huff-puff algorithm, it can also handle temporary one-way overloading, i.e. when the round trip increases so much that the measured clock offset can be explained by such a one-way delay.

NTP came out in 1985. Prime computers didn't have crystal clocks. It was quite a chore to synchronize the clocks. The clock could only be set at the console. When time was set, it started at the indicated minute; there were no seconds. I wrote a program to query the other computers on the ringnet and set the clock. Some computers had more accurate clocks than others. The program had a list of computers from best to worst. If the best computer was unavailable, it would drop to the next. There was also code to do a sanity check if times were too divergent from each other. The time synchronization was added to the backup script because it was run daily from the console.

this might be the best science channel on this platform. It is in the top for sure.

In years past I really enjoyed looking at the delay-offset scatter plots from ntp's peerstats data. It reveals interesting network details like the asymmetry in DSL links, or big changes in global routing. A "network weather monitor" of sorts. Search for the great write up on a focus on NTP in Slovenia to see some of these scatter plots.

This channel is a real treasure!🐧💝

If you're able to ask the guy who spent half his life watching clocks about an interview, it would definitely be worth a watch too

Iirc, in some datacenters where synchronized clocks are critical, they'll have servers with their own little atomic clocks as PCIe cards to maintain high precision and reduce drift.

One of the best KZfaq channels

NTP is so awesome. I love that Canada has an official NTP time server. It's run by the NRC and is extremely accurate! Been syncing all my computers with NTP for many many years.

Too bad, I think the important part of the synchronization algorithm was left out. How should the client update its clock to be the same as the server's? Aiming for t1 - t0 = 0 is clearly not the answer since propagation delay is never exactly 0. What actually happens in NTP is that it aims to get the "midtimes" equal. i.e. make the difference (t2 + t1)/2 - (t3 + t0)/2 equal to 0. Under the assumption that propagation delays are equal in each direction, this quantity will be 0 for a well-synchronized server-client pair.

Wonderful, thank you! I used to assume that my computer sent a "what time is it?" question and the server replied. I wondered how they accounted for the transit time (although I was never so curious that I looked it up!). Now that I've seen this video the solution seems obvious.

What a coincidence! I was just starting to take a closer interest in NTP last week and hey presto, your presentation has appeared at the same time! Thank you.

I spent the start of this video trying to brush a hair off my phone screen. It turned out to be a drawing on the whiteboard!

NTP! One of my fav's and most used...and also one of the things even super nerds don't generally know about. I have microcontrollers running everywhere that need their time synced to my server, and it's amazing how simple a thing can be so useful....and so often forgotten...

This will be useful for a weather station I am building - it has multiple agents, and these need to be synchronised in time; I am still researching ways to do this - this is a good starting point; thanks for this video. I had not thought of this; I am not a computer scientist but more of a mathematical interest. Thanks again!

What a coincidence. I was just Reading about NTP today and now Computerphile releases a video about it. 👍🏻

I'm surprised no mention the NTP protocol dates all the way back to 1980!

Definitely need to follow up and investigate the predictive/detection possibilities with NTP and the drift patterns.

The NTP client on Windows NT6 has mind of its own. If I need to synchronize a computer without a CMOS battery, it's hard to get it to coax into running on startup, keep working, and not reject the time. And when I need it again, I have forgotten what I needed to do.

Nicely explained. Thanks for sharing 👍🏼

Way back I worked at a antarctic station and setup a server (debian linux) as a level 1 NTP server using data from a gps receiver. I would not be surprised if it is still running 20 years later.

Have been waiting for that topic!

Please do a video on alpha tensor! Would love to learn more about it and the algorithms it discovered

Thanks for sharing Julian 👏🏻

Love this channel...

At uni we had a 4pm network problem. It was destroying our student radio station. And as manager I was spending half my time getting the uni to deal with it. At 4pm the entire network ground to a halt for an hour. It was eventually found it was caused by the entire network doing a full backup of everything. This was 2009 and the techs were still getting used to the rapid use of everyone being constantly online.

Since there is latency in the packets getting to the server, why would you want t1 and t0 to be equal? Wouldn't you want t2-t1 to equal t1-t0? Wouldn't that mean that (assuming the packets have the same latency each way), the clocks are in sync? Otherwise if you targeted t1=t0, then your clock would be ahead since you are assuming the latency is 0?

In my experience comparing NTP time to a much more precise GPS time source NTP has usually been good only to within 6-10 ms. That's pretty good for most things, but I'm working with scientific equipment that we need to synchronize to within a few nanoseconds ideally, hence the GPS.

At 4:31 Dr Onions says that "the difference in clocks is t1-t0". But isn't that ignoring the time it takes for the round trip? If we assume the trip time is the same both ways then I'd think the difference in clocks would be t_1-t_0 - ((t_3-t_0) - (t_2-t_1))/2 . Or to simplify (t_1 + t_2 - t_0 - t_3) / 2 .

This is actually more basic than I thought. It only works because they assume that each round-trip contains two legs of similar duration, and this is usually true, but not always. With only two nodes, it's impossible to determine if the return trip takes more or less time than the outgoing trip. If one leg takes much longer than the other, you can't actually figure out when the other computer recorded its time. You need to coordinate connections between more nodes to figure out how long each leg takes, which is the only way to really get a guarantee about the time.

saw a guy uneasily sitting over the table in the thumbnail...... that told me instantly that this dude is obscenely intelligent and will be able to explain why exactly this gobbeldygoop is necessary for everyday life

I'm sure people will disagree but this is one of the most important protocols in network engineering. Certificates, RADIUS, logs, IPSEC etc.. On and on, you lose alot of important things should this fail in a network and things get out of synch.

Great explanation! Thanks!

Like most normal people i look at random conversations on networks via wireshark ( what its normal! ) but i have never looked at NTP, i will have to now.

I worked in a meteorological office which ran an S0 clock, and honestly, I never expected how much fiddling with synchronization I'd have to do! Everything always ran flawlessly, until suddenly, you'd find that some weather station was somehow now the S1 or 2 source for an entire network, and you'd only notice after some random windows authentication stopped working properly!

This is the best video ever

I used to design and procure national networks, both fixed and mobile. One of the most fun parts was figuring out the clock source - have your own Césium atomic clock, or Rubidium disciplined by GPS… How many to have, hierarchies of clocks…. I think I did 14 networks in all!

Thanks that's more complex than I would have thought 8⁠-⁠)

Awesome! Can you guys also do a video on PTP?

Such a deceptively complex issue that's basically a necessity for modern communications. The whole power grid thing blows my mind a bit

In Windows, you can still specify/override what time servers to use, if you find that NTP isn’t reliable enough using the defaults. Just as in Linux, you have system commands to force time recalibration and other fun things.

Electronic devices in electrical power grids are synched in time, often using a GPS clock to be able to exactly determine where a fault occurs in the power grid. Since AC electricity changes direction every 20 ms it's necessary to have all devices synched up to ms precision

That's another question answered, I use Net Time manually every so often, I was thinking about how they compensated for such time/distance variances, it syncs "perfect" with the "RF regulated" clock I have.

Nice sir explain. Really amazing. I like it.

During the BBC Teletext broadcast trials the on screen clock was always out by several hours because the Unix system clock was not synchronized to anything.

As a sysadmin I allways enjoyed NTP, it just works and is so useful For so many applications.

A few years ago there was one unloved server where I worked whose time was out by about 6 seconds. It caused all manner of weird problems with my builds until I figured it out. The admins fixed it when I reported it but never explained how it came to be so far out in the first place.

I thought I knew about NTP. Clearly I didn’t ! But it was the stratums that I need to know about. Thanks!

What a great passionate guy. Wish I had a lecturer like that in uni

Are you guys going to do a video on PTP (Precision time Protocol)?

You know this is going to be brilliant when he pulls out the old dot-matrix paper.

So, Dave Mills was a clock watcher! 🙂 How about a video on PTP (Precision Time Protocol)?

Early adopter of NTP before it was quietly cool. Recently did some "certification" of NTP on a RTL system. Was more challenging that I thought to do right (and did find a minor bug in the RTL NTP system (or rather a deeper RTL issue that manifested through those tests)). IAC, armed with a pretty cheap GPS and its 1 PPS signal one can check the NTP algo to about 1000x the accuracy of NTP. [hate to say it, but if you need better accuracy, get a GPS and use the 1 PPS to know the top-of-second quite accurately (less than 1 µs)].

As a nerd for precision time keeping, I like this video. (I have a Raspberry Pi 3B+ dedicated as a GNSS NTP Server with chronyd + gpsd)

Shouldn't (t1 - t0) (or (t2-t3)) be equal to half of the travel time? I'm confused about how they can be equal to 0 when the clocks are synchronized.

He's so gentle I forget I'm hearing super serious technical chat

I wonder how NTP would cope with asymmetric channel latency such as on RFC1149

The last job I had before giving up working for a living spent a fair bit of time working with PTP to synchronise small cells.

Happy Friday, nerds!

Was this created after mtc midi time code that is used to sync music on beat via midi across multiple recording equipment? Same issue very similar solution.

I’ve made a big NTP wall clock with ESP it’s amazing

wow this is way simpler than i was imagining. how do we account for huge outbound latency and tiny inbound latency? or is that rare enough that we just ignore it?

Could you do a video on clocks changing or timezone and how that affects things. Esp as it’s now automated and no one notices like they used to changing clock an hour by hand.

1. There are systems that monitor the power demand by looking at the change in frequency 2. The same thing happened with Railways in the 19th Century, where synchronising clocks between different cities was introduced 3. Hence time zones. 4. Einstein and relativity was all based around think about how time works, when you have to send signals between different observers.

I wish this explanation got into the conf setups

there's a strange series of articles if not books on the technical wonkiness surrounding tea time across so many industries

In America our computers also have to contend with the 2 time changes a year as well. In the spring, they jump forward 1 hour at 2:00am and then back again 1 hour in the fall again at 2:00am.

God damn fascinating, the way to see an earthquake in the clock drift.

6:10 defining jitter without using the word “jitter”

im not sure if this has any relevance to anything but if you have ever disconnected from internet and manually changed the computer date settings to as far back as it will let you (mine was 1980) and create a restore point if the date is set at 1980 it is visible on system restore but once you connect to the internet and resync it is no longer visible on the system restore point list. im not sure if this can be used for security considering it to use the internet the time must be in sync. from memory i backdated and created a restore point i then opened that restore point and dragged and dropped other files into it, and as we know data recovery software is quite good at restoring hdd data , i dont know just one of thing that might be overlooked by a hacker or someone might gain physical access if your trying to hide bitcoin wallets etc

Wait, why do I want t0 to be as close as possible to t1 - doesn't that ignore the packet travel time? I'd expect that I want to get the difference to be as close as possible to half the roundtrip time (assuming that the packet travel time is roughly the same in both directions - this can be improved once I synchronize to multiple servers).

What happens if there is a systematic bias in the direction of traffic? e.g. what if there is a computer behind a router where everyone watches Netflix and the incoming route has a 100ms queue on average, or if there's a firewall that inspects all outgoing packets, or something? If you're just using the round trip time and dividing by 2 you'll be systematically off in one direction.

really cool video, I actually implemented NTP as part of my home network, since we're in a house share with lots of computers and a file server (NAS) it's nice to have everything syncronised, so i turned it on as a DHCP option and have the NAS serve as the network time server for local clients and it gets it's time from one of the internet NTP pools.

I've got a few arduino projects that use a gps to get the time... does that make them a stratum 1 device? Or is there more to it?

Okay so you estimate the round trip time, but what time does the client actually set? t_3 minus half the round trip time?

I wonder what badge was he wearing that caused it to be blurred out :) initially I thought that computerphile did it but then I googled the name and lo and behold, the picture is literally blurred on the university website :)

Having spent months(!) debugging a pesky clock sync problem on a small IOT product, I've developed a distain for NTP. It's great when it works, but when problems emerge on remote customer devices in the field, default NTP provides very few clues. There's a reason why people are developing alternatives like chrony.

Was that half-circle on the whiteboard intentional? Been wiping my screen more times than I care to admit.

It's crazy that someone watching NTP traffic could see things happening in the real world, like kettles being started and earthquakes. It makes sense though. anything that impacts the electricity that computers are using could cause fluctuations.

Now, you should do one on IEEE1588 PTP.

Could a coding error in ntpd be called a jitter bug?

Next video Precision Time Protocol? NTP its more obscure cool cousing.

So are we assuming that there's no delay in data travel? (basically the theoretical light speed with no relay servers/hops being slow)?