Today we are pleased to introduce Polar Signals Continuous Profiler, an open source based SaaS for Continuous Profiling. Profiling has long been a tool in the developer's toolbox. It allows CPU, memory, I/O usage and more to be understood down to a line of code. It is an immensely valuable tool for debugging. Continuous profiling is the act of taking profiles of programs in a systematic way. Polar Signals Continuous Profiler, based on the open source Conprof project, collects, stores and makes pprof profiles available to be queried over time. This is useful in post-mortem analysis, when it is too late to take measurements. In short, Continuous Profiling allows for profiling to be systematic, instead of a search for the needle in the haystack.
Traditional Workflow
To understand how Polar Signals Continuous Profiler works, and why we built it, let's go through the traditional workflow of what a developer goes through when profiling. A typical scenario starts out with either a production incident or the intent to optimize a piece of code. Optimization is often driven by the intention to make a system faster, or for example to reduce resource usage in order to save cost.
Incidents caused by running out of memory (OOMKill) can be a very stressful situation, the process has already been killed by the operating system, and so it is not possible to obtain profiles about the situation anymore. Obtaining profiles often requires SSHing onto production machines and capturing profiles manually. This is problematic for security and auditability, besides that, it is also easy to make mistakes when SSHing in production environments. Above all, profiling manually in this fashion is a search for the needle in the haystack. It is impossible to know whether a situation is reproducible in a way that allows capturing the right data.
Polar Signals Continuous Profiler Workflow
The goal of Polar Signals Continuous Profiler is to turn the search for the right data into a systematic one. With Polar Signals Continuous Profiler, profiles are captured periodically, ensuring that profiles are available for any moment in time, that way the right data is always only a search away. This way, when a process ran out of memory, a latency spike happens or when we want to start optimize code, the data is already available! This saves time obtaining the data manually, but more importantly the right data is already captured - no need to wait or try and reproduce a past situation.
Once the data is available over time, new useful workflows are unlocked. Take the scenario of optimizing a piece of code for reducing resource usage for example. A profile of a particular moment of time is useful, but it is only representative of that exact moment in time. When the data is collected over time, it can for example be summarized into a single report, making it not only representative of that moment of time, but the entire time range. We have found this to be particularly useful to summarize performance characteristics of an entire version of code.
And last but not least, not only is the data available over time, but also across deployment-rollouts. It allows answering questions that thus far have been notoriously difficult to answer, which boils down to "What changed in my code, that caused a change in performance?" This typically originates from one of two questions. Why did our resource usage increase? And did my optimization changes work?
To sum up, using Polar Signals Continuous Profiler an organization can:
- Save money on their cloud bill, by optimizing resource usage.
- Boost conversion rate, by improving application latency.
- Save time and stress, by make systems more reliable.
How Polar Signals Continuous Profiler Works
Polar Signals Continuous Profiler periodically scrapes pprof compatible profiles from HTTP endpoints. We are launching with support for Go, which supports profiling in pprof format via its standard library. Many programs written in Go already have these endpoints, so organizations that already use Go are likely to not need any changes to their code to start using Polar Signals Continuous Profiler! Rust, Python, NodeJS and JVM are already on the roadmap, but let us know what languages and runtimes you would like to see!
The collector is configured to discover targets using service discovery, this happens to be identical to Prometheus' service discovery, so organizations already using Prometheus can re-use configuration from it! And if not, we have automated deployment and configuration strategies prepared for you, for example to get started on Kubernetes seamlessly.
The collector regularly scrapes the HTTP endpoints and sends the results to Polar Signals. Once received by Polar Signals the exploration for improvements can start!
Next steps
Polar Signals Continuous Profiler is in private beta starting today! We are working hard to bring it to General Availability. We are in the process of onboarding users to our private beta, and if you want to participate, request access: