Crashes
Windows and application crashes with decoded error details and local dump analysis when available.
Topic id: crashes.
Conditions (1)
Select a condition for background, what to check and an example.
Criticalkernel_crash_recurring The agent counts blue screens from the System log, so the count is right even when crash dumps are turned off. It names the stop code most of the crashes share, and the driver the newest of those crashes' dump analysis blamed. The same stop code and driver each time point at that driver, though a blamed driver can be the victim of another fault. Different stop codes with no common driver point more often at hardware: memory, storage, power or heat. Each crash also arrives as its own Windows crashed (blue screen) event, with its full details.
- The newest Windows crashed (blue screen) event for the shared stop code and the blamed driver.
- Drivers, BIOS and Windows updates installed shortly before the first crash.
- When the stop codes vary: Windows Memory Diagnostic, the disk's health, hardware error events in the System log, temperatures and the power supply.
- Started 0mSevereCrossed its bar and held long enough to open.started; 5 kernel crashes in the last 5d (threshold 5); mostly bugcheck 0x0000009F, 2 of 2 analysed in 10d (3 still being analysed)kernel_crash_count_30d 5kernel_crash_count_10d 5kernel_crash_count_5d 5kernel_crash_count_1d 2kernel_crash_days_since_last 0.08
Event as stored
crashes: kernel_crash_recurring: NOTABLE: started; 5 kernel crashes in the last 5d (threshold 5); mostly bugcheck 0x0000009F, 2 of 2 analysed in 10d (3 still being analysed) - Started +15mDisplayCrossed its bar and held long enough to open.started; 2 kernel crashes in the last 30d (threshold 2); mostly bugcheck 0x000000EA in "example.sys", 1 of 1 analysed in 30d (1 still being analysed)kernel_crash_count_30d 2kernel_crash_count_10d 1kernel_crash_count_5d 1kernel_crash_count_1d 1kernel_crash_days_since_last 0kernel_crash_quiet_awake_days 0
Event as stored
crashes: kernel_crash_recurring: NOTABLE: started; 2 kernel crashes in the last 30d (threshold 2); mostly bugcheck 0x000000EA in "example.sys", 1 of 1 analysed in 30d (1 still being analysed) - Subsiding +2h00mDisplayReadings are past the clear point; waiting to confirm.subsiding, not yet cleared; 1 kernel crash in the last 30d (below 2), last 702h00m ago (clears at 720h00m), quiet awake 1h55m (clears at 120h00m); mostly bugcheck 0x0000001E, 1 of 1 analysed in 30d, observed for 2h00mkernel_crash_count_30d 1kernel_crash_count_10d 0kernel_crash_count_5d 0kernel_crash_count_1d 0kernel_crash_days_since_last 29.25kernel_crash_quiet_awake_days 0.08
Event as stored
crashes: kernel_crash_recurring: ELEVATED: subsiding, not yet cleared; 1 kernel crash in the last 30d (below 2), last 702h00m ago (clears at 720h00m), quiet awake 1h55m (clears at 120h00m); mostly bugcheck 0x0000001E, 1 of 1 analysed in 30d, observed for 2h00m - Relapsed +2h30mDisplayBack over the bar before it cleared: same episode.relapsed; 2 kernel crashes in the last 30d (threshold 2); mostly bugcheck 0x0000001E, 1 of 1 analysed in 30d (1 still being analysed), observed for 2h30m, relapses 1kernel_crash_count_30d 2kernel_crash_count_10d 1kernel_crash_count_5d 1kernel_crash_count_1d 1kernel_crash_days_since_last 0kernel_crash_quiet_awake_days 0
Event as stored
crashes: kernel_crash_recurring: ELEVATED: relapsed; 2 kernel crashes in the last 30d (threshold 2); mostly bugcheck 0x0000001E, 1 of 1 analysed in 30d (1 still being analysed), observed for 2h30m, relapses 1 - Relapsed +2h30mDisplayBack over the bar before it cleared: same episode.relapsed; 2 kernel crashes in the last 30d (threshold 2); mostly bugcheck 0x0000001E in "example.sys", 1 of 1 analysed in 30d (1 still being analysed), observed for 2h30m, relapses 1kernel_crash_count_30d 2kernel_crash_count_10d 1kernel_crash_count_5d 1kernel_crash_count_1d 1kernel_crash_days_since_last 0kernel_crash_quiet_awake_days 0
Event as stored
crashes: kernel_crash_recurring: ELEVATED: relapsed; 2 kernel crashes in the last 30d (threshold 2); mostly bugcheck 0x0000001E in "example.sys", 1 of 1 analysed in 30d (1 still being analysed), observed for 2h30m, relapses 1
- Cleared 0mInfoStayed clear long enough; the episode closed.cleared after 519h52m, peaked Minor, relapses 0; last kernel crash 720h00m ago (clears at 720h00m), quiet awake 519h50m (clears at 120h00m)kernel_crash_count_30d 1kernel_crash_count_10d 0kernel_crash_count_5d 0kernel_crash_count_1d 0kernel_crash_days_since_last 30kernel_crash_quiet_awake_days 21.66
Event as stored
crashes: kernel_crash_recurring: RECOVERED: cleared after 519h52m, peaked Minor, relapses 0; last kernel crash 720h00m ago (clears at 720h00m), quiet awake 519h50m (clears at 120h00m)
Occurrences (2)
kernel_crash At the next start Windows records the stop in the System log (Kernel-Power event 41 with a bug check code, and Windows Error Reporting event 1001). Each event is one blue screen, with its stop code and four parameters. When Windows wrote a memory dump, the agent analyses it before reporting and names the first driver on the crashing stack that is not part of Windows. That driver is where a debugger looks first, not a proven cause. With crash dumps turned off, or a dump Windows could not write, the event carries the stop code and parameters only. The stop code narrows the cause: a driver fault, failing memory or storage, overheating, or power. A power loss or a held power button leaves no stop code; SparkLogs reports that as Unexpected host shutdown.
- The stop code in Microsoft's bug check code reference, and what its four parameters mean for that code.
- The blamed driver's vendor and version, and whether it, another driver or the BIOS changed shortly before the crash (Driver updated and BIOS version changed events).
- For memory, storage or hardware stop codes (0x1A, 0x7A, 0x124 and similar): Windows Memory Diagnostic, the disk's health, and hardware error events in the System log.
- Whether this machine has crashed before: Repeated Windows crashes counts them.
kernel crash 0x0000001E KMODE_EXCEPTION_NOT_HANDLED | analysis=partial_timeout param1=0xC0000005 param2=0xFFFFF8016A2B3C4D param3=0x0 param4=0x0 sparklogs.instance: crash_signature:0x0000001e|example.sys
kernel_crash, Error, crashes: kernel_crash: NOTABLE: kernel crash 0x0000001E KMODE_EXCEPTION_NOT_HANDLED | analysis=partial_timeout param1=0xC0000005 param2=0xFFFFF8016A2B3C4D param3=0x0 param4=0x0kernel crash 0x0000009F DRIVER_POWER_STATE_FAILURE | analysis=budget_skipped budget=per_key param1=0x3 param2=0xFFFFE0012A4B8060 param3=0xFFFFD000217A6960 param4=0xFFFFE0012C1E4010 time=estimated sparklogs.instance: crash_signature:0x0000009f
kernel_crash, Error, crashes: kernel_crash: NOTABLE: kernel crash 0x0000009F DRIVER_POWER_STATE_FAILURE | analysis=budget_skipped budget=per_key param1=0x3 param2=0xFFFFE0012A4B8060 param3=0xFFFFD000217A6960 param4=0xFFFFE0012C1E4010 time=estimatedkernel crash 0x0000018B SECURE_KERNEL_ERROR | analysis=complete modern_standby=true param1=0x1 param2=0x0 param3=0x0 param4=0x0 sparklogs.instance: crash_signature:0x0000018b|example.sys
kernel_crash, Error, crashes: kernel_crash: NOTABLE: kernel crash 0x0000018B SECURE_KERNEL_ERROR | analysis=complete modern_standby=true param1=0x1 param2=0x0 param3=0x0 param4=0x0os_bsod: The System log's own record of the same blue screen; count crashes from the Windows crashed (blue screen) events, one per crash.
process_crash Windows logs Application Error event 1000 when a program stops on an error it did not handle; each event is one crash. It names the exception code, the module the fault was in, and the program's version; a .NET program also reports its exception type and stack. The faulting module often matters more than the program: a printer driver, an Office add-in or a security product's library loaded into the program makes the program look broken. When Windows kept a local dump of the crash, the agent analyses it and names the first module on the crashing stack outside the Windows folder. A program that crashes again and again also raises Application crashing repeatedly.
- The faulting module: its vendor, its version, and whether it changed recently.
- Whether the same module appears in crashes on other machines.
- Update or remove the add-in or plugin the faulting module belongs to, or update the program.
process crash in "Contoso.Worker.exe" | exception=0xE0434352 exception_name=EXCEPTION_COMPLUS managed_exception="System.NullReferenceException" fault_module="KERNELBASE.dll" fault_module_version=10.0.20348.2849 fault_offset=0x3E2D2 app_version=3.1.0.0 analysis=complete managed_exception_message="Object reference not set to an instance of an object." sparklogs.instance: executable:contoso.worker.exe
process_crash, Warning, crashes: process_crash: NOTABLE: process crash in "Contoso.Worker.exe" | exception=0xE0434352 exception_name=EXCEPTION_COMPLUS managed_exception="System.NullReferenceException" fault_module="KERNELBASE.dll" fault_module_version=10.0.20348.2849 fault_offset=0x3E2D2 app_version=3.1.0.0 analysis=complete managed_exception_message="Object reference not set to an instance of an object."process crash in "sqlservr.exe" | exception=0xC0000409 exception_name=STATUS_STACK_BUFFER_OVERRUN fault_module="ucrtbase.dll" fault_offset=0x7F4DE app_version=2022.160.4135.4 analysis=dump_unavailable sparklogs.instance: executable:sqlservr.exe
process_crash, Warning, crashes: process_crash: NOTABLE: process crash in "sqlservr.exe" | exception=0xC0000409 exception_name=STATUS_STACK_BUFFER_OVERRUN fault_module="ucrtbase.dll" fault_offset=0x7F4DE app_version=2022.160.4135.4 analysis=dump_unavailableprocess crash in "example-app.exe" | exception=0xC0000005 exception_name=STATUS_ACCESS_VIOLATION fault_module="example.dll" fault_module_version=2.4.1.0 fault_offset=0x1234 app_version=5.0.2.17 analysis=complete sparklogs.instance: executable:example-app.exe
process_crash, Warning, crashes: process_crash: NOTABLE: process crash in "example-app.exe" | exception=0xC0000005 exception_name=STATUS_ACCESS_VIOLATION fault_module="example.dll" fault_module_version=2.4.1.0 fault_offset=0x1234 app_version=5.0.2.17 analysis=completeprocess crash in "legacy-tool.exe" | exception=0xC0000005 exception_name=STATUS_ACCESS_VIOLATION fault_module="legacy.dll" fault_module_version=1.8.0.0 fault_offset=0x4A0C app_version=1.8.0.0 analysis=complete sparklogs.instance: executable:legacy-tool.exe
process_crash, Warning, crashes: process_crash: NOTABLE: process crash in "legacy-tool.exe" | exception=0xC0000005 exception_name=STATUS_ACCESS_VIOLATION fault_module="legacy.dll" fault_module_version=1.8.0.0 fault_offset=0x4A0C app_version=1.8.0.0 analysis=completeFields (144)
One occurrence per crash, from the Windows event records that report it, with decoded codes and local dump analysis when a dump was available. The host row counts recent kernel crashes.
Full inventory every 8 hours. Supported changes are reported when the agent observes them.
Kernel crash counts (14)
sparklogs.data.crashes. kernel_crash_count_1d kernel_crash records; the latest is kernel_crash_last_ts.sparklogs.data.crashes. kernel_crash_count_5d kernel_crash records; the latest is kernel_crash_last_ts.sparklogs.data.crashes. kernel_crash_count_10d kernel_crash records; the latest is kernel_crash_last_ts.sparklogs.data.crashes. kernel_crash_count_30d kernel_crash records; the latest is kernel_crash_last_ts.sparklogs.data.crashes. kernel_crash_days_since_last sparklogs.data.crashes. kernel_crash_last_ts Application crash counts (7)
sparklogs.data.crashes. app_crash_count_1d sparklogs.data.crashes. app_crash_count_5d sparklogs.data.crashes. app_crash_count_10d sparklogs.data.crashes. app_crash_days_since_last sparklogs.data.crashes. app_crash_quiet_awake_days sparklogs.data.crashes. apps_crashing_10d Crash record (13)
sparklogs.data.crashes. crash_id sparklogs.data.crashes. time_basis dump_header (the time in the analyzed kernel dump), report_event (the crash's event log record), or estimated_bounds (no crash time is known; the time is time_upper_ts).sparklogs.data.crashes. time_lower_ts sparklogs.data.crashes. time_upper_ts sparklogs.data.crashes. detected_ts sparklogs.data.crashes. wer_report_ids Failure and blame (5)
sparklogs.data.crashes. blame_module System32, SysWOW64 and WinSxS count as inside). Absent when no dump was analyzed or no frame qualifies.sparklogs.data.crashes. blame_offset sparklogs.data.crashes. failure_name STATUS_ACCESS_VIOLATION. For a kernel crash, the stop error's name, such as IRQL_NOT_LESS_OR_EQUAL. Absent when no name is known.sparklogs.data.crashes. failure_space ntstatus for a process crash, bugcheck for a kernel crash. A .NET exception type in failure_name is not from that table. Present whenever the code is.sparklogs.data.crashes. failure_code 0x and 8 hex digits. Absent only when no record or dump carried one.Dump analysis (13)
sparklogs.data.crashes. analysiscomplete, partial_timeout, partial_error, resource_limited, dump_unavailable, unsupported_format or budget_skipped. complete means the analysis finished, not that the cause is known.sparklogs.data.crashes. budget_reason budget_skipped: per_key (the limit for crashes of this kind in a day was reached), overall (the daily limit for all crashes was reached) or queue_full. With resource_limited: commit_pressure (the machine's committed memory stayed too high to start the analysis).sparklogs.data.crashes. analysis_elapsed_ms sparklogs.data.crashes. analysis_peak_commit_bytes sparklogs.data.crashes. matching_dump_count sparklogs.data.crashes. dump_path Stack and modules (50)
sparklogs.data.crashes. context_ip rip or eip) at the crash, in hex.sparklogs.data.crashes. context_sp rsp or esp) at the crash, in hex.sparklogs.data.crashes. context_fp rbp or ebp) at the crash, in hex.sparklogs.data.crashes. modulesbasename and, where read, machine (hex), size_of_image, time_date_stamp (8 hex digits), file_version, and the symbol identity a debugger needs: codeview_kind (rsds or nb10), pdb_name, pdb_guid (32 hex digits), pdb_signature (8 hex digits), pdb_age, portable_pdb, pdb_checksum_algorithm and pdb_checksum. The *_source members say where each part came from: module_stream, triage_entry, dump_image or local_image.sparklogs.data.crashes. modules[].basename modules entry.sparklogs.data.crashes. modules[].machine modules entry.Kernel crash details (11)
sparklogs.data.crashes. identity_basis boot_start (the boot it was reported in), report_id (its error report) or power_41 (the Kernel-Power 41 unexpected-restart record).sparklogs.data.crashes. boot_start_missing sparklogs.data.crashes. bugcheck_code 0x and 8 hex digits. Absent only when no record or dump carried one.sparklogs.data.crashes. bugcheck_name IRQL_NOT_LESS_OR_EQUAL. Absent when the code has no known name.sparklogs.data.crashes. bugcheck_space bugcheck. Present whenever the code is.sparklogs.data.crashes. bugcheck_param1 Process crash details (31)
sparklogs.data.crashes. app_name sparklogs.data.crashes. app_version sparklogs.data.crashes. app_timestamp sparklogs.data.crashes. app_path sparklogs.data.crashes. pidsparklogs.data.crashes. process_start_ts Example
Inventory (every 8 hours)
crash history; kernel crashes in 10d 0, app crashes in 10d 0.
sparklogs.data.crashes.kernel_crash_count_1d: 0
sparklogs.data.crashes.kernel_crash_count_5d: 0
sparklogs.data.crashes.kernel_crash_count_10d: 0
sparklogs.data.crashes.kernel_crash_count_30d: 0
SparkLogs: CONTEXT, Info, crashes: INVENTORY: crash history; kernel crashes in 10d 0, app crashes in 10d 0.