EuroPython 2022 - Why is it slow? Strategies for solving performance problems - presented by Caleb Hattingh
[Liffey A on 2022-07-15]
You have a performance problem, and you don't know what to do. All you know is that one of your endpoints or applications is too slow; and perhaps it only affects a certain user or customer. How do you figure out why it's slow, and what can you do to catch performance problems before they hurt users in production?
We'll go through a wide range of strategies for detecting and diagnosing performance problems in typical production workloads. We'll cover both web-based domains as well as backend domains and other analytical applications involving number-crunching and big-data applications.
We'll step through the following high-level strategies:
- Tracing: through instrumentation of your code, you will get detailed traces of where the time is spent in generating your web server responses.
- Profiling: we'll look at profiling strategies using both the Python built-in cProfile tool, as well as awesome 3rd party libraries like pyspy, including how to use these with pytest
- Isolation: how to figure out if performance is affected by CPU, or memory, disk, or network IO limitations.
- Reasoning: we'll look at common scenarios that result in performance regressions such as the needless execution of sub-queries in rendering web views, or algorithmic analysis and ""big-O"" notation, or concurrency problems resulting from exhaustion of threads in a pool and asyncio concurrency limitations resulting from overloaded subscription.
- Prophylaxis: we'll look at how to include benchmarks within your CI pipeline, including with pytest and other technologies to catch performance regressions ahead of time.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License creativecommons.org/licenses/b...