CppCon 2017: Carl Cook “When a Microsecond Is an Eternity: High Performance Trading Systems in C++”
Рет қаралды 262,553
Күн бұрынCppCon.org
-
Presentation Slides, PDFs, Source Code and other presenter materials are available at: github.com/CppCon/CppCon2017
-
Automated trading involves submitting electronic orders rapidly when opportunities arise. But it’s harder than it seems: either your system is the fastest and you make the trade, or you get nothing.
This is a considerable challenge for any C++ developer - the critical path is only a fraction of the total codebase, it is invoked infrequently and unpredictably, yet must execute quickly and without delay. Unfortunately we can’t rely on the help of compilers, operating systems and standard hardware, as they typically aim for maximum throughput and fairness across all processes.
This talk describes how successful low latency trading systems can be developed in C++, demonstrating common coding techniques used to reduce execution times. While automated trading is used as the motivation for this talk, the topics discussed are equally valid to other domains such as game development and soft real-time processing.
-
Carl Cook: Optiver, Software Engineer
Carl has a Ph.D. from the University of Canterbury, New Zealand, graduating in 2006. He currently works for Optiver, a global electronic market maker, where he is tasked with adding new trading features into the execution stack while continually reducing latencies. Carl is also an active member of SG14, making sure that requirements from the automated trading industry are represented. He is currently assisting with several proposals, including non-allocating standard functions, fast containers, and CPU affinity/cache control.
-
Videos Filmed & Edited by Bash Films: www.BashFilms.com
*-----*
Register Now For CppCon 2022: cppcon.org/registration/
*-----*
@narnbrez 4 жыл бұрын
One of those jobs where finding a trick like this makes the company your salary back in earnings
@tissuepaper9962 2 жыл бұрын
"I did a runthrough this morning, threw away about half the slides." I felt that one. There's always too much to say and not enough time to say it.
@AdityaKashi 6 жыл бұрын
Eye-opening talk. That trick with replacing if's with templates was simple and neat.
@MrSam1ization 4 жыл бұрын
we have if constexpr now which is basically the same thing
@justinkendall8341 4 жыл бұрын
I've learned an immense amount from CppCon's uploads, this video included. Great talk, Mr. Cook.
@gnir6518 Жыл бұрын
Says in the description he holds a PhD, so Dr. Cook 😄
@grostig 4 жыл бұрын
Great talk, easy enough to understand. Thank you for taking the time.
@betajester5729 6 жыл бұрын
This was fascinating, gave me a lot to think about.
@GustavoPantuza 6 жыл бұрын
Very good talk. Thanks for sharing, Carl.
@Moriadin 2 жыл бұрын
Been looking for a talk like this for 10 years. Thanks!
@CppCon 2 жыл бұрын
Glad it was helpful!
@memoriasIT 5 жыл бұрын
This is a great talk, good content and well presented
@eto895 6 жыл бұрын
This is great .. love C++
@TheValiantFox 6 жыл бұрын
Awesome talk!
@Conenion 6 жыл бұрын
Very good talk. Thank you.
@butterfury 6 жыл бұрын
Very cool talk
@kumaranshuman5138 5 жыл бұрын
Hi Carl, Was just trying to see how one can measure the number of collisions happening while accessing STL unordered map?
@spinthma 3 жыл бұрын
Really great understandable insights!
@xealit 6 ай бұрын
Excellent practical talk
@HashanGayasri 5 жыл бұрын
As a person from the other end (Exchange), I find the differences quite interesting. We target to have an end to end latency of around 30us with a very low amount of jitter since we also care a little about throughput and use a fault tolerable distributed system. We often make performance sacrifices to keep the code base easy to understand for most people.
@LuizCarlosSilveira 5 жыл бұрын
Do you use Linux/C++ there, or what else? Also, if you don't mind telling what is the exchange in question I'd appreciate :)
@HashanGayasri 5 жыл бұрын
@@LuizCarlosSilveira Yes, that's generally the combination used. It's Millennium Exchange - used by the London Stock Exchange and others.
@VeiusIuvenis Жыл бұрын
An Exchange does not need to be too fast. When it slows, it slows equally.
@cavingada Жыл бұрын
currently working as an intern at an exchange and I'd agree, the differences are really interesting. Especially when looking at it from the high performance point of view
@kenjichanhkg 3 жыл бұрын
really want to know more about the measurement setup, how do you actually tap the line, how to actually do the configurations?
@ciCCapROSTi Жыл бұрын
This is a great talk. Very different from the others, but still interesting, easily approachable, valuable info.
@CppCon Жыл бұрын
Glad you enjoyed it!
@edubmf Жыл бұрын
I wonder if the original presentation is available? it's not as good as having the presenter talk through the "missing" slides however it's better than nothing.
@anthonyvays5786 5 жыл бұрын
great talk
@Hector-bj3ls 27 күн бұрын
Not sure you're supposed to mention the star gate. Mr SG14. Not SG1, I get it. You just want some recognition for your hard work.
@sirantoweif5372 5 жыл бұрын
The entire presentation is about why you need to learn assembly, machine language and your hardware inside out when going for the fastest execution possible. Mid and high-level languages don't have any of the tools you need. You need to create them specifically for each platform if you want to exploit every single cycle to your advantage. If you change something / upgrade the hardware, then you're 100% going to need to rewrite part of the code or all of it to make it optimized again. Mid-level languages are still insanely fast though, no question about that. The question is rather whether you can afford to place 2nd when you aimed to be 1st.
@donha475 4 жыл бұрын
great talk!
@crystalgames 6 жыл бұрын
great talk thanks
@AbhishekTiwari-hc5ph 2 жыл бұрын
Could anyone reproduce the problem shown at 44:47 ? I couldn’t with older GCC compilers..
@bearwolffish Жыл бұрын
Top tier talk
@zhengchunliu5839 4 жыл бұрын
thanks for the eye-opening talk. If you turn off all other CPU cores, where does your OS kernel run on? would there be interference from OS if your thread and OS share the single core?
@sn3k 4 жыл бұрын
That's why it's important not to generate kernel calls - in that case your kernel doesn't have to do anything.
@tonvandenheuvel 6 жыл бұрын
Hey Carl, great talk. During the talk you mention having to delete a whole lot of slides; I wonder whether it is possible for you to share those slides as well? I think folks will be interested in those as well.
@carlcook8194 6 жыл бұрын
Hi Ton, thanks, excellent point. I'm working on another (very similar talk now to be presented at pacificplusplus.com), once those slides are ready I'll post a link here. It's mainly just material on branch prediction, but will still be useful.
@superblitz2274 2 жыл бұрын
@@carlcook8194 link please? It's been 4 years haha
@oblivion_2852 3 жыл бұрын
How does the flat unordered map not have cache misses? I thought it'd be a cache miss whenever the data has to be dereferenced from a pointer? Is the idea to allocate the data such that the data follows the map in memory so that the l2 and l3 caches hold it?
@serg_joker 2 жыл бұрын
According to my understanding, it saves cache misses on traversing nodes in case of collisions. Still, it will have cache miss on the accessing `value` after we found correct node. With list-based node store, every jump to the next node is a cache miss. But, for example, if we have 3 items in a the bucket and you look up for the third, with linked list-based approach, you going to have two cache misses (because of traversing nodes in a bucket) and with flat map you'll have just one (when dereferencing data).
@lapetiteanessesoap7559 Жыл бұрын
good job
@colinmaharaj 2 жыл бұрын
22:45 I want to do my calculations for an entry point in separate threads man. I'm coding pattern detection, if I'm monitoring 20 stocks I need my threads. Thoughts?
@michieljaspers9889 6 жыл бұрын
Interesting talk. Do you actually measure ( aka. use performance counters ) cache hits / TLB hits when optimizing code, or do you optimize code based on instinct and experience? Modern cpu's often have HW prefetchers for instructions. Strive for long instruction streams (aka no jumping in Program Counter) to fully utilize this feature. There are many other optimizations to be done based on the microarchitecture of the processor. Do you guys have experts from Intel helping out?
@carlcook8194 6 жыл бұрын
Hi, indeed I do take a look at cache and TLB metrics, but really it's all about measuring end-to-end time, first in the lab, and then in production. So if I see that the results in the lab are slow (or even worse, production), one of the first things I'll look at is the output of linux perf. In terms of coding/optimizing based on experience, I have found that for both my code and other people's code, making educated guesses doesn't usually work. It's all about measurement, refinement, and then measuring again. For your next question, I don't talk to Intel engineers directly, although I have done a few training courses which gave me some useful insight (including learning that the micro-opcode inside the Intel chips is propitiatory, not documented, and not guaranteed to stay the same over different releases of CPUs). So it's a bit hard to specialize in a single Intel CPU. What I will say is that for when we really need ultra low latency, then FPGAs and other custom hardware is often most suitable. What helps is that for when you are competing against other software-based trading systems, then knowing to a reasonable level about how to make optimizations is often good enough. You can spend a lot of time with one chipset, bios, CPU, network card, operating system, and get a software setup very, very fast, but that's a lot of effort. Which in some cases that's fine and justifiable, but in other cases isn't worth the effort, i.e. it's sometimes better to be the first to find new market opportunities and get something relatively fast up and running in software, and be the first to have a successful trading strategy from this. This is why I think being good at writing fast C++ is always a useful skill to have.
@ajaardvark 6 жыл бұрын
good question; but.. a trading system is not just calc heavy; it's also about decisions, and that means branching.
@AdrienMahieux 6 жыл бұрын
Hi, I was wondering about the added insights of using vtune instead of perf. Seems there's a lot more pcm to get there, but didn't have time so far. Skylake-SP made such a great arch change, gotta check everything again ! Very nice talk !
@vedantsonu1983 6 жыл бұрын
Hey Carl. Great Talk ! How do you lock L3 cache for one core ( or a set of cores ) without actually turning off the CPUs ? I tried searching but in vain. Can you provide some pointers for that ? Thanks!
@carlcook8194 6 жыл бұрын
Hi, here is a link to a basic paper about cache access technology: www.intel.com/content/www/us/en/communications/cache-allocation-technology-white-paper.html. Hope that helps!
@christopherjohnston3772 6 жыл бұрын
Can be done with CAT, pmqos has a decent CLI to get that going.
@OlivierLi 6 жыл бұрын
Good job! Could you please provide a link about the warming up of SolarFlare adapters?
@carlcook8194 6 жыл бұрын
Sure thing. Check out the open onload API documentation for SolarFlare cards (support.solarflare.com/index.php/component/cognidox/?file=SF-104474-CD-23_Onload_User_Guide.pdf&task=download&format=raw&id=361), in particular this section: 8.15 ONLOAD_MSG_WARM
@YSoreil 6 жыл бұрын
Hey, great talk. I'm curious if there is any viability to using non intel parts for this type of single core workload. In particular I was curious if it is viable to use for instance POWER architecture CPUs here. The very fast 5HGz+ cores which are available there and the very large caches might be interesting. This seems like one of the few still existant markets where these parts might make sense for non legacy work. I originally had a more interesting reply but the KZfaq autoplay function deleted it, the bane of good videos one actually wants to watch to the end I guess.
@carlcook8194 6 жыл бұрын
Interesting comment! I've never looked at other architectures/CPUs, but I am sure some will probably give a decent advantage, once you get familiar enough with them. For example, ARM gives less guarantees about order of reads/writes from multiple threads, which probably means it is faster in some multi-threaded cases (as just one example).
@ludwig7686 6 жыл бұрын
how do you log the trading systems actions? that should be overhead
@carlcook8194 6 жыл бұрын
Indeed there is logging in there. But I will say a couple of things: 1.) We keep logging to an absolute minimum, and 2.) A lot of work goes into keeping the logging code as low-touch as possible, i.e. making sure that the logging is done from a separate thread, not evaluating format strings in the hotpath, and a few other tricks. I actually spoke about logging in this talk a year ago if it helps: kzfaq.info/get/bejne/q9J_f6qJmK6yc30.htmlm14s
@sami-pl 5 жыл бұрын
@@carlcook8194 that's seem more like tracing pre-determined things and for it to be fast it better fit in a cache line with some timestamp even at cost of precision or maybe even relative. This store can actually be even marked as non temporal for it not to eat cache (excluding WC). Then once in a while it may be picked by another thread or well memory mapped FIFO. Anyways it's nice hearing you things from you Carl, you do have that ability to go through hard stuff without unnecessary details which if one is interested, will have to go anyways. Good job
@gracelee7929 Жыл бұрын
How can you create an exception without throwing it?
@LordNezghul 6 жыл бұрын
Why fight with your operating system when your program can be operating system itself thanks to solutions like includeOS?
@carlcook8194 6 жыл бұрын
Fighting the operating system isn't too hard to be honest. It's the hardware that causes me the most problems. IncludeOS is cool, but it has a long way to go before the tooling for it is good enough to run commercial HFT software (think about debuggers, profilers/instrumentation, monitoring support, etc). I do see potential in includeOS, but won't be an early adopter of it!
@msnzuigt 4 жыл бұрын
Isn't includeOS DOS with extra steps?
@LuluTheCorgi 5 жыл бұрын
Is there even any infrastructure left to produce single core CPUs?
@mworld 2 жыл бұрын
Yes map is slower then unordered_map since map it trying to sort as well. I found this out when dealing with millions of records.
@VeiusIuvenis Жыл бұрын
It depends.
@dosomething3 3 жыл бұрын
14:38 "HandleError(errorFlags);" should only tell another thread to handle the error. The current thread should only perform the hot-path and non of the actual error handling. This way, the current thread's cache is hot-path-optimized. So we now have a hot-path-thread. Your thoughts?
@ThePizzaEater1000 3 жыл бұрын
i don't even do anything in C++ idk how I got here
@hashishishin 6 жыл бұрын
That should be an RT OS or a specialized FPGA sitting right there on that network card!
@carlcook8194 6 жыл бұрын
Indeed! But often you need to move faster than FPGA development, and/or hit limitations about how many FPGAs you can have per exchange. So far I've been fine not needing real time operating systems, but it's certainly worth thinking about.
@Pacificplusplus 6 жыл бұрын
Here is another version of this talk: kzfaq.info/get/bejne/eN6Wh5yXzrvXimg.html
@samuelutomo 6 жыл бұрын
35:08 👍
@scottsanett 6 жыл бұрын
He's speaking a rhotic New Zealand accent... Interesting
@carlcook8194 6 жыл бұрын
Ha ha, my wife is a linguist and she was impressed by your observation! Yeah, I lived in the southern part of NZ for a three years where there is a Scottish population, and then I also lived in The Netherlands which influenced my strange accent.
@rajesh09ful 5 жыл бұрын
@@carlcook8194 John oliver has done his part quite well to edify the global population about New zealand accent :)
@JiveDadson 6 жыл бұрын
Subtitle: How to trick the wrong operating system running on the wrong hardware into usually being good enough.
@carlcook8194 6 жыл бұрын
Indeed. And being able to move very quickly to new markets and strategies because you are running fairly standard hardware and operating systems which are relatively easy to program against and install. However, every trading company will no doubt also be running their ultra low latency systems on custom hardware and no real operating system to speak of.
@anthonyvays5786 5 жыл бұрын
might as well use Fortran instead of fighting with the compiler so much
@username4441 6 жыл бұрын
ok but which random bot do i play russian roulette with on github and download to use as my first bot?
@carlcook8194 6 жыл бұрын
This one: github.com/askmike/gekko
@allopeth 5 жыл бұрын
22:10 completely hilariuos slide xDDDDD
@blazkowicz666 2 жыл бұрын
Why not just use C? Also, I'm curious to know how QUIC and gRPC would fit into your plans to reduce latency
@bvskpatnaik9115 2 жыл бұрын
QUIC and gRPC are network protocols used for communication. This means stock exchange must support it and honestly I don't think any exchange would support gRPC since that's not ideal for good latency.
@UCCi6g4buFK8dVtAxbFYEKzA Жыл бұрын
Does this form of trading generate any value (for the world) or is it just parasitic?
@haha71687 Жыл бұрын
Market makers provide liquidity.
@petrupetrica9366 3 жыл бұрын
41:24 Why would you say something so controversial, yet so brave
@DanielMonteiroNit 6 жыл бұрын
Hope the use of float is for brevity :-P
@carlcook8194 6 жыл бұрын
It's funny that no one commented on this at the time. Indeed, float would never be used in practice... most likely fixed point integer.
@TeeDawl 6 жыл бұрын
Hey Carl, I really do enjoy the fact that you're here in the comments. I also really enjoyed the talk! Greetings!
@DanielMonteiroNit 6 жыл бұрын
Been playing with sg14::fixed_point lately and it's very good! But hey, which size of fixed points is usual in fintech? What "distribution of bits"? Is it fixed or varied distributions for different uses? Also, at 13:10, are you initialising the error flag somewhere else? Would it be faster with, say, int64_t errorFlags{NO_ERROR}; ? Thanks again for the talk.
@trejohnson7677 2 жыл бұрын
Zero cost abstractions == perpetual motion machine.
@vladosique528 6 жыл бұрын
Dude's ripped.
@JiveDadson 6 жыл бұрын
Given the job he admits to having - sneaking a profit by cutting in line (queue) - the rippitude may come in handy.
@KX36 Жыл бұрын
He says intel wouldn't be interested in making CPUs for this market, but this is literally the market that pays for all the international undersea fiber optic cables just to reduce latency slightly. You pay intel enough, they;ll be interested.
@VeiusIuvenis Жыл бұрын
Probably too much for them...
@ShakthiMonkey 8 ай бұрын
let him cook
@VincentDBlair 2 жыл бұрын
Now if this isn’t an expert way of saying “ Functional Programming in C++”
@ntrandafil 4 жыл бұрын
Thank you for the talk! 33:28 😂
@GeorgeTsiros Жыл бұрын
people who write "benchmarks" sometimes baffle me it's like they don't really care about measuring anything just to have the cpu pegged at "100%" or something and then spit out numbers with lots of significant digits
@magwo 6 жыл бұрын
Any thoughts on Rust? It seems to me an ideal language for the requirements - predictable, consistent high performance, and high productivity due to modern, high level features that still don't introduce high latency behaviour (by avoiding memory allocations and memory churn). It's like the "low latency techniques" happen to be the "best practices" of Rust development, whereas the "best practices" of C++ is not the same as the low latency techniques of C++ dev.
@carlcook8194 6 жыл бұрын
Yes, this is exactly the point. Rust does look cool, but C++ has so many 1000s of man-hours built into the language, the tools, etc, it's going to take one heck of a language to convince people to give up on C++. The same is true for includeOS. Another great idea, fresh thinking, and should in theory make a lot of the tricks that I need to pull obsolete. But it's still too early to start using this product commercially.
@nishanth6403 Жыл бұрын
Except that best C++ practices are largely variant across different domains, and usually extreme low latency design is considered "best practices" at places where Cpp is still used heavily today.
@scramjet4610 2 жыл бұрын
Why not use C instead of C++ when speed is so important?
@alexandrosliarokapis227 2 жыл бұрын
Because C is not faster than C++ and lacks a lot of abstractions that C++ can get without costs. Think templates vs Macros.
@nishanth6403 Жыл бұрын
C need not be faster than C++. And since comp time programming hit Cpp I've seen it beat C quite a no. of times.
@xl000 4 жыл бұрын
4:09 faster by a picosecond ? ie faster by 1E-12 second the granularity of ethernet packets is larger than that
@VeiusIuvenis Жыл бұрын
Good talk, but a lttle too brief... leaving me a lot of questions...
@jonastoth7975 6 жыл бұрын
I guess the trading CPUs are all 7700k? Why bother with Xeon there, which is there to have many cores?
@windfind3r 6 жыл бұрын
ECC memory access.
6 жыл бұрын
ark.intel.com/products/120499/Intel-Xeon-Platinum-8156-Processor-16_5M-Cache-3_60-GHz
@butterfury 6 жыл бұрын
Yuge L3 cache
@kyonsmith9046 5 жыл бұрын
16:39 He still needs one virtual function call to Mainloop.
@VeiusIuvenis Жыл бұрын
But that only run once, which is not in hotpath.
@tohopes 6 жыл бұрын
32:20 dragons lie here.. lol
@tohopes 6 жыл бұрын
tfw store didnt have any white shirts so you had to give the talk wearing a pink one
@tohopes 6 жыл бұрын
52:14 tfw the guy who bought the last white shirt you wanted used your profiling server as a build machine
@carlcook8194 6 жыл бұрын
Ha ha yes, I asked them to cut this calendar screen-grab from the video, but oh well...
@tohopes 6 жыл бұрын
I would be pissed if some tab did that to me during a presentation. You handled it well. Great talk btw, very informative.
@riachomolhado999 6 жыл бұрын
C ++ is only for males-alphas system. Do this in Java and you will lose millions of dollars.='D
@xExekut3x 2 жыл бұрын
"high frequency trading systems" .. should be illegal.
@llothar68 6 жыл бұрын
Total unethical industry he is working in. This should be forbidden, thats even more important then gun control.
@abblabblabalaba 6 жыл бұрын
As a leftist I don't disagree with you, but he's obviously not the bourgeois capitalist, just a developer, and this talk has some valuable information.
@tagged5life 6 жыл бұрын
it's the other way around, trying to provide liquidity causes low latency trading.
@richardhickling1905 6 жыл бұрын
Well that's the idea - but statistically it doesn't work that way: HFT is just too fast. The orders they provide - though they dominate numerically - don't provide better prices for the counterparty. Also they don't trade illiquid contracts. Paid market-makers have to be pulled in for that. Not blaming them BTW: they're just using technology effectively to make money. I can't see it as unethical either. But there should be more regulation - contact your democratic representative.
@budabudimir1 5 жыл бұрын
@@richardhickling1905 Isn't it entire idea that curtain HFT-s can make money by just squeezing in into the inefficient market? I.e. someone wants to sell curtain instrument for $100, but someone else wants to buy that instrument for $90. HFT is going to see this inefficiency and offer to buy for $92 and sell for $98 for example, thus saving $2 for a buyer, earning $2 for the seller and earning $6 itself.
@jonwise3419 5 жыл бұрын
High-frequency trading is in the high end of the spectrum of the soul-sucking experience (Speculation -> Day Trading -> High Frequency Trading). It probably feels like Satan himself is both sucking your soul and your dick at the same time, or, to use a good old trick of obfuscation through abstract parlance and legalese, it probably feels like adding liquidity to the market.
@kostyatrushnikov2067 6 жыл бұрын
At slide 19, how can we call manager->MainLoop()? Does it mean, that we have a "virutal void MainLoop() = 0; " at IOrderManager? Then we still have virtual funcs+templates instead of just virtual funcs :)
@BJovke 6 жыл бұрын
Exactly. And there also might be a memory leak there. Factory() function is implicitly casting OrderManager to it's base class. IOrderManager. If there are some member objects of OrderManager which allocated RAM from heap their destructor won't be called. Also I'm not sure that unique_ptr can be assigned a value from unique_ptr with different template.
@rationalcoder 6 жыл бұрын
There isn't a memory leak as long as the base defines a virtual destructor and OrderManager's destructor works to begin with. Also, that assignment of unique_ptr's is a valid conversion: en.cppreference.com/w/cpp/memory/unique_ptr/unique_ptr
@BJovke 6 жыл бұрын
"There isn't a memory leak as long as the base defines a virtual destructor". I don't see that anywhere in the example. Also, the derived class' destructor needs to call destructor of base class. Yes, you're right, unique_ptr assignment works, but there's an implicit cast from OrderManager to it's base class. There are too many "hidden" things here. I know that this is just an example but, in my opinion, an example for a conference like this one must be much better. I also don't like seeing some other stuff, like function definitions with empty brackets - "void SendOrder ()". Is it so hard to put "void" in it? A lot of work has been done on C++ standard to increase strict type enforcement. After all, C and C++ are supposed to be strongly typed languages. C programmers and even standard broke this a lot with huge amount of software exchanging "void *" pointers all around the code. And now C++ programmers are breaking this with a lot of implicit conversions and class to base casts. In this example they could have just used a pointer to sender function and switch it to different one when needed.
@rationalcoder 6 жыл бұрын
The source for IOrderManager isn't given in the example, it is simply implied that it has a virtual destructor (if not, you will get a leak like you mention). Also, dtor's of derived classes call the dtor's of their parents implicitly, reversing the order of construction, so this example works as intended assuming he didn't mess up IOrderManager. See this: stackoverflow.com/questions/677620/do-i-need-to-explicitly-call-the-base-virtual-destructor
@BJovke 6 жыл бұрын
Yes, you're right, my mistake.
Meeting Cpp
Рет қаралды 67 М.
Oscar's Funny World
Рет қаралды 229 МЛН
Core C++ 2019 :: Nimrod Sapir :: High Frequency Trading and Ultra Low Latency development techniques
CoreCppIL
Рет қаралды 46 М.