Core Dump Epidemiology: Fixing an 18-Year-Old Bug

At the scale OpenAI operates, even the most improbable bugs become daily occurrences. Recently, our engineering team was faced with a series of baffling crashes in the Rockset service—a critical piece of our ChatGPT data infrastructure used for real-time analytics and searching over conversations. These crashes weren't just inconvenient; they seemed impossible by the standard rules of C++ execution.

By shifting our perspective from "treating the patient" to "studying the population," we eventually uncovered two distinct culprits: a single failing hardware host and an 18-year-old race condition in a widely used open-source library. This is the story of how we used data infrastructure to debug data infrastructure.

The Mystery of the Impossible Crashes

Rockset’s execution layer is written in C++ to maximize performance and minimize memory usage. However, the lack of memory safety in C++ means that bugs often manifest as segfaults. Our initial investigation into these crashes revealed two strange symptoms:

1. Return-to-NULL: A function would finish execution and attempt to return to a memory address that was NULL.
2. Misaligned Stack Pointers: The CPU's stack pointer register (%rsp) would suddenly be off by 8 bytes, causing the program to crash upon returning from a function.

In standard compiled code, these failure modes are extremely rare. Compiled code only adjusts the stack pointer in specific, predictable ways. Every hypothesis we formed was met with contradictory evidence. We were stuck in "Doctor Mode"—focusing on individual core dumps one at a time.

Shifting to Epidemiology

The turning point came when we decided to stop acting like doctors and start acting like epidemiologists. Instead of deep-diving into one crash, we built a pipeline to analyze the entire population of crashes across our fleet.

Using a script (written with the help of ChatGPT), we automated the analysis of thousands of core dumps. We extracted registers and labeled crashes by type. Once we had a clean data set, the patterns became undeniable. What we thought was one weird bug was actually two unrelated issues occurring simultaneously.

Bug #1: The Silent Hardware Killer

The "misaligned stack" crashes were highly localized. They all originated from a single geographic region and were traced back to a specific physical host in the cloud. It was a case of silent hardware corruption—the CPU was simply performing math incorrectly. By denylisting that host, the misaligned stack crashes vanished instantly.

Bug #2: The 18-Year-Old Race Condition

With the hardware noise removed, the "return-to-NULL" crashes became much clearer. These were happening exclusively during exception unwinding.

When a C++ program throws an exception, it uses a library (like GNU libunwind) to walk back up the stack to find a catch block. We discovered a race condition in libunwind that had existed for nearly two decades.

The bug existed in a tiny, one-instruction window. During a context restore, the library would update the stack pointer (%rsp). For a few hundred picoseconds, the structure containing the next instruction to execute was no longer protected. If a signal (like our frequent SIGUSR2 for CPU accounting) arrived at that exact moment, the kernel would overwrite that memory, turning the return address into NULL.

Why Now?

If the libunwind bug was 18 years old, why did it only start crashing now? The answer lies in the "perfect storm" of our specific workload:

• High Exception Frequency: Rockset uses exceptions for backpressure.
• Frequent Signals: We deliver signals every few milliseconds for fine-grained CPU tracking.
• Heavier Signal Handlers: A recent update made our signal handlers use more stack space, making it more likely they would clobber the vulnerable memory.

Lessons Learned

We resolved the issue by switching to the libgcc unwinder and upstreaming a fix to the libunwind project. However, the true takeaway wasn't the specific fix, but the methodology.

• Data over Intuition: High-quality population data is better than brilliant guesswork.
• Automate Investigation: When manual inspection doesn't scale, build a pipeline to do it for you.
• Deep Instrumentation: Reliability is a byproduct of being able to see what your systems are doing at the lowest levels.

By treating infrastructure bugs as an epidemiological study, we turned a set of "impossible" crashes into a diagnosable, solvable problem. — and solvable — engineering challenge.