Host performance
Whole-host CPU and memory posture over a measured window.
Topic id: performance.
This topic is still rolling out. Endpoints on the general release do not send it yet.
Conditions
A condition is a state that holds for a while. The agent opens it when the host enters it, keeps it open while it lasts, and closes it when the host comes back out, so one episode answers for the whole stretch instead of one alert per sample.
| Condition | Severity | How an episode ends |
|---|---|---|
cpu busy (cpu_busy) | Display to Notice | It closes when the measurement falls back past its recovery point. |
cpu interrupt storm (cpu_interrupt_storm) | Warning to Error | It closes when the measurement falls back past its recovery point. |
cpu kernel dominated (cpu_kernel_dominated) | Notice | It closes when any one of the several recovery conditions is met. |
cpu throttled under load (cpu_throttled_under_load) | Warning to Error | It closes on a recovery rule written for this condition, which reads more than one measurement together. |
committed memory high (ram_commit_high) | Display to Critical | It closes when the measurement falls back past its recovery point. |
hard fault storm (ram_hard_fault_storm) | Serious | It closes when any one of the several recovery conditions is met. |
Example
Inventory (every 5 minutes)
cpu 12% avg, commit 48%, run queue 0.3/core, interrupts 1.2%, hard faults 6/s.
sparklogs.data.performance.cpu_pct_time_over_90: 0.0
sparklogs.data.performance.run_queue_per_core_p90_10s: 0.3
sparklogs.data.performance.cpu_interrupt_dpc_pct_avg: 1.2
sparklogs.data.performance.cpu_busy_pct_avg: 12.0
sparklogs.data.performance.cpu_kernel_excl_drivers_pct_of_busy_avg: 18.0
sparklogs.data.performance.commit_pct: 48.0
sparklogs.data.performance.ram_pct_time_in_hard_fault_storm: 0.0
SparkLogs: CONTEXT, Info, performance: INVENTORY: cpu 12% avg, commit 48%, run queue 0.3/core, interrupts 1.2%, hard faults 6/s.
Selected conditions
cpu_busy
CPU is busy.
Example
started; time over 90% CPU 100% (threshold 80%)
sparklogs.data.performance.cpu_pct_time_over_90: 100.0
sparklogs.data.performance.run_queue_per_core_p90_10s: 5.0
sparklogs.data.performance.cpu_busy_age_h: 0.0
SparkLogs: cpu_busy, Notice, performance: cpu_busy: NOTABLE: started; time over 90% CPU 100% (threshold 80%)
cpu_interrupt_storm
CPU time is dominated by interrupt and DPC handling.
Impact: Device or driver interrupt load can starve ordinary work on the host.
Example
started; interrupt+DPC 34% (threshold 30%)
sparklogs.data.performance.cpu_interrupt_dpc_pct_avg: 34.0
sparklogs.data.performance.cpu_interrupt_storm_age_h: 0.0
SparkLogs: cpu_interrupt_storm, Warning, performance: cpu_interrupt_storm: NOTABLE: started; interrupt+DPC 34% (threshold 30%)
| Case | Severity | Ticket class |
|---|---|---|
onset | Trace to Fatal | performance |
held | Trace to Fatal | performance |
recovered | Trace to Fatal | performance |
cpu_kernel_dominated
Busy CPU time is mostly kernel time rather than application work.
Impact: Application throughput can be lower than the busy figure alone suggests.
Example
started; CPU busy 84% (threshold 70%)
sparklogs.data.performance.cpu_busy_pct_avg: 84.0
sparklogs.data.performance.cpu_kernel_excl_drivers_pct_of_busy_avg: 85.26
sparklogs.data.performance.cpu_kernel_dominated_age_h: 0.0
SparkLogs: cpu_kernel_dominated, Notice, performance: cpu_kernel_dominated: NOTABLE: started; CPU busy 84% (threshold 70%)
| Case | Severity | Ticket class |
|---|---|---|
onset | Trace to Fatal | performance |
held | Trace to Fatal | performance |
recovered | Trace to Fatal | performance |
cpu_throttled_under_load
The CPU is running below its rated frequency while under load.
Impact: Work takes longer than the hardware would otherwise allow; thermal, power or firmware limits are the usual cause.
Example
started; CPU busy 82
sparklogs.data.performance.cpu_throttled_under_load_age_h: 0.0
SparkLogs: cpu_throttled_under_load, Warning, performance: cpu_throttled_under_load: NOTABLE: started; CPU busy 82
| Case | Severity | Ticket class |
|---|---|---|
onset | Trace to Fatal | performance |
held | Trace to Fatal | performance |
recovered | Trace to Fatal | performance |
ram_commit_high
Committed memory is high.
Example
started; commit 92% (threshold 90%)
sparklogs.data.performance.commit_pct: 92.0
sparklogs.data.performance.ram_commit_high_age_h: 0.0
SparkLogs: ram_commit_high, Notice, performance: ram_commit_high: NOTABLE: started; commit 92% (threshold 90%)
ram_hard_fault_storm
The host is thrashing memory.
Impact: Paging pressure can degrade every workload on the host.
Example
started; commit 97% (threshold 95%)
sparklogs.data.performance.commit_pct: 97.0
sparklogs.data.performance.ram_pct_time_in_hard_fault_storm: 100.0
sparklogs.data.performance.ram_hard_fault_storm_age_h: 0.0
SparkLogs: ram_hard_fault_storm, Serious, performance: ram_hard_fault_storm: NOTABLE: started; commit 97% (threshold 95%)
| Case | Severity | Ticket class |
|---|---|---|
onset | Trace to Fatal | performance |
held | Trace to Fatal | performance |
recovered | Trace to Fatal | performance |