← All posts
FirmwareDebuggingESP32

A reboot is a symptom, not a root cause

Paul WaltonFounder, Corescope·

A device in the field hits a fault, the watchdog fires, and it reboots. Service resumes. By the time anyone notices, the one thing that could explain the failure - the call stack at the moment it broke - is already gone.

The black box problem

Unlike a web server, a microcontroller in production has no one watching the logs. When it crashes you typically lose:

  • the stack trace - which function actually faulted, and how it got there;

  • the fleet picture - is this one unit or ten thousand;

  • the lead-up - the breadcrumbs of what the device was doing before it died.

What good looks like

Instead of a reset reason, you want the fault resolved to a function, file and line - grouped with every other device that hit the same bug. Frame 0 is where it died (the leaf), with its callers below:

LoadProhibited  (fatal)  build a3f9c7e1
  0  app_tick           src/work.c:58   # null deref -> cfg->retries
  1  do_work            src/work.c:131
  2  app_main           src/main.c:84

That is the difference between knowing that a device rebooted and knowing why. The reboot was never the problem - it was the symptom that hid it.

Want this for your fleet?

Corescope turns field crashes into symbolicated Issues. Join the founding fleet for early access.

Request early access