Low FPS / stuttering

Hypercolor’s render loop targets your configured daemon.target_fps ceiling using an adaptive five-tier controller (Minimal 10 → Low 20 → Medium 30 → High 45 → Full 60) that shifts between tiers based on measured frame budget. Healthy means holding the tier that matches your configured target. The default target is 30 FPS, so on a stock config the loop sits at Medium when everything is fine; that is the ceiling, not a downshift. Full (60) only comes into play if you raise daemon.target_fps. The first sign of a performance problem is the tier sitting below your configured target, not a hard crash.

This page walks you through reading the diagnose output, identifying which part of the pipeline is the bottleneck, and fixing the root cause.

#Step 1: run the diagnostics

hypercolor diagnose

This calls POST /api/v1/diagnose against the running daemon. By default it runs six check groups: daemon, render, devices, config, input, and memory. Use --check to focus on one:

hypercolor diagnose --check render
hypercolor diagnose --check devices

Add --system to include verbose system information (GPU, kernel, audio version, and daemon uptime).

To save a full report for a bug report:

hypercolor diagnose --system --report ~/hypercolor-bug-report.json

Output is a styled table by default. Use --format plain for a one-line text summary or --format json (shorthand: -j) for machine-readable JSON.

#Step 2: read the render checks

The render category has four checks for FPS problems.

#render_loop

render_loop   state=running, tier=medium

The tier field shows where the FPS controller has landed. Compare it against your configured daemon.target_fps: on the default config, medium (30 FPS) is the healthy steady state. A tier sitting below your configured target means the loop has downshifted under budget pressure. If state is anything other than running, the render loop has stopped; see common issues.

Tier reference:

The controller downshifts after 2 consecutive budget misses and only upshifts after sustained headroom, so a tier stuck below your target is a real signal.

#frame_liveness

frame_liveness   frame_token=18432, frame_age_ms=47.30

frame_age_ms is how stale the last completed frame is. A value of 2000 ms or more triggers a warning; 10 000 ms or more is a failure, meaning the loop is stalled or not producing output. Check daemon logs for Servo errors or device write failures.

#led_freshness

led_freshness   output_source=current_frame, gpu_sample_stale=false, devices_written=4, total_leds=512, sample_us=1200, push_us=3400

Key flags to watch:

• gpu_sample_stale=true: the GPU-side canvas sample was not ready in time and the compositor reused the previous frame’s sample. Sustained stale GPU samples accumulate in the recent_window counters (step 4).
• gpu_sample_wait_blocked=true: the render loop blocked on the GPU sample queue, meaning the sampling path is a bottleneck.
• gpu_sample_queue_saturated=true: the GPU import slot pool is exhausted; multiple frames are queued behind a single GPU fence.
• output_source: current_frame is healthy. published_frame or routed_reuse means the loop is serving a cached frame because the current render is not keeping up.
• sample_us / push_us: time in microseconds for spatial sampling (canvas to LED positions) and writing to devices. Elevated push_us across many devices points to USB saturation (step 3).

#recent_output_sources

recent_output_sources   frames=18432, current_frame=18201, published_frame=140, routed_reuse=91, reused_published_frame=12, gpu_sample_stale=3

This check always reports pass; it is a counter readout, not a threshold. It totals which frame each output push actually served over the session, so it tells you how often the loop fell back on a cached frame rather than the frame it had just rendered. A current_frame count close to frames is healthy. A large published_frame or routed_reuse share is the same signal led_freshness reports per frame, accumulated over time.

#Step 3: check device output queues

The devices category includes two checks relevant to USB-bus and output latency.

#output_queues

output_queues   queues=6, usb_queues=4, lagging=1, worker_finished=0, dropped_total=0, errors_total=0

• lagging: queues where completed delivered_fps is significantly behind the queued cadence. Use the coalescing split below to distinguish an expected device cap from transport pressure.
• worker_finished: a worker thread exited unexpectedly. Any non-zero value is a hard failure and appears as fail status.
• dropped_total: legacy sum of coalesced frames across all device queues.

For the detailed per-device breakdown, use -j and inspect snapshot.device_output.items. Compare delivered_fps with accepted_fps and fps_target. Rising coalesced_target_cadence is expected pacing; rising coalesced_backend_overrun, transport latency, or failures identifies real pressure.

#usb_actor_display_lane

usb_actor_display_lane   display_frames=1240, delayed_for_led=3, wait_avg_ms=0.12, wait_max_ms=1.80

The USB actor shares a single bus arbitration lane between LED output and display-face rendering. If wait_max_ms reaches 2 ms or higher, the check flips to warning, meaning LED writes are consistently blocking the display lane. This typically means too many USB devices competing for the same controller.

Fix for USB saturation: spread devices across multiple USB controllers (different root hubs, not just different ports on the same hub). Check your topology with lsusb -t. Adding a PCIe USB card is the reliable fix when you have more than four or five high-frequency RGB devices on a single system.

#Step 4: identify Servo overhead

If you are running HTML effects (TypeScript SDK effects or any effect with an .html source), the render pipeline passes through Servo, a full browser engine embedded in the daemon. Servo renders on a dedicated OS thread with its own event loop, paint, and readback phases, and that overhead adds directly to your frame budget.

Run the JSON report and examine snapshot.render.recent_window:

hypercolor diagnose -j | jq '.data.snapshot.render.recent_window'

Fields to watch:

• gpu_sample_stale / gpu_sample_deferred: count of frames in the recent window where Servo’s frame was not ready. A high ratio to frames means Servo is your bottleneck.
• gpu_sample_cpu_fallback: Servo fell back from GPU import to a CPU readback path. This adds a full-frame pixel readback cost and is significantly slower.
• push_p95_ms / publish_p95_ms: 95th-percentile device write and event-bus publish latency. Healthy values sit well under the frame budget for your current tier.

For Servo-specific timings, the system response (GET /api/v1/system) carries status.effect_health, with fields including servo_render_frame_max_ms, servo_render_evaluate_scripts_max_ms, servo_render_paint_max_ms, and servo_render_readback_max_ms. These are milliseconds, not microseconds. A servo_render_frame_max_ms value near or above the tier budget is a clear Servo bottleneck.

#Servo circuit breaker

Servo uses a circuit breaker to protect the daemon when the effect renderer fails repeatedly. After 3 consecutive failures it opens, blocking new render attempts for a 30-second cooldown with exponential backoff up to 5 minutes. The breaker state is visible in the Servo telemetry fields:

• servo_breaker_opens_total non-zero: Servo has tripped the breaker at least once this session.
• servo_soft_stalls_total: frames that stalled without a fatal failure.

When the breaker is open, effects fall back to a black/idle canvas, meaning your LEDs stop updating or hold the last frame. Switching to a native (non-HTML) effect clears the dependency on Servo immediately.

Fixes for Servo overhead:

• Switch to one of the native built-in effects if frame timing is critical; they run in compiled Rust with no browser overhead. Browse what is available in the effects section.
• If you want to keep an HTML effect, reduce the effect’s JavaScript complexity. Heavy requestAnimationFrame work, large canvas operations, and expensive audio FFT processing are common culprits. See debugging and diagnostics for Servo profiling techniques.
• If servo_gpu_import_fallback_reason appears in the JSON output, the GPU zero-copy import path is unavailable and Servo is doing full CPU readbacks every frame. Verify that your GPU drivers support the required Vulkan or OpenGL extensions for your platform.

#Step 5: check the render budget breakdown

The snapshot.render.latest_frame object in the JSON report gives a per-stage breakdown of the most recent completed frame in microseconds:

{
  "input_us": 120,
  "render_us": 8400,
  "producer_us": 9100,
  "composition_us": 310,
  "sample_us": 1200,
  "push_us": 3400,
  "publish_us": 90,
  "overhead_us": 200,
  "total_us": 22820
}

At Full tier the budget is 16 666 µs. If total_us consistently exceeds the budget for a tier, the loop cannot hold that tier regardless of what it is targeting.

#Common scenarios

#Tier stuck below your configured target, no device errors

The render loop is consistently overrunning its budget. Inspect total_us in latest_frame. If render_us or producer_us dominates, the bottleneck is the effect renderer, most likely Servo. If push_us is high, it is device output. Address each at its root; do not lower the tier ceiling.

#gpu_sample_stale climbing in recent_window

Servo is not delivering frames fast enough to keep up with the render loop. Check render_frame_max_us and render_evaluate_scripts_max_us in the Servo telemetry to isolate whether JavaScript evaluation or paint is the slow phase.

#Devices showing lagging=N in output queues

N device queues are receiving frames faster than they can send them. This is USB bandwidth exhaustion. Use -j and check avg_write_ms per device: values significantly above 1000 / fps_target indicate the device itself is slow to acknowledge writes.

#worker_finished=N in output queues

N device worker threads have exited. This is a hard error: those devices are no longer receiving output. Check last_error in the JSON. Common causes are USB disconnect, permission errors, or device firmware lockup. Try replugging the device. If errors persist, see USB devices for udev and hot-plug guidance.

#Servo breaker open

Check daemon logs for Servo-related errors, and check servo_session_create_failures_total in the system response. If the breaker is cycling (opens, cools down, opens again), the effect itself is broken, likely from a JavaScript runtime error. Switch effects to confirm, then file a bug with the --report output attached.

#Getting more detail

For a deeper look at the render architecture that produced these metrics, see render pipeline and renderer internals.

If device output looks healthy but you still see stuttering at the hardware level, check USB devices for controller topology and permission guidance.

To attach a full diagnostic report to a bug report:

hypercolor diagnose --system --report ~/hypercolor-bug-report.json