heap: add heap debugging tools by kv2019i · Pull Request #11264 · thesofproject/sof

kv2019i · 2026-10-02T15:37:48Z

A series to help debugging heap usage issues. We already have infra to print logs for each alloc, but these were not documented for developers and didn't cover all the heaps (especially not newer "virtual heap" and "vregion").

This series add similar debugging for all major heap implementations (print out usage stats at each alloc), and adds an overlay to enable these all with a single overlay file.

Add vmh_log_alloc_stats() which prints only a single-line summary of total allocated/free bytes; call it on every successful allocation in virtual_heap_alloc() instead of the verbose vmh_log_stats(). Both functions are compiled only when CONFIG_SYS_MEM_BLOCKS_RUNTIME_STATS is set. This matches the debug capability in place for plain heap allocs. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>

kv2019i · 2026-10-02T15:39:29Z

@wjablon1 If you can check. This also adds on top of the commit 02c9391 you did.

Copilot

Copilot review overview

🟡 Changes recommended

Invalid vregion states can query uninitialized heap metadata, and the overlay does not reliably enable its INFO-level logs.

Review effort: Balanced
Findings: 3 · 1

Open (4)

Enable INFO-level logging for allocation diagnostics · New
Snapshot heap type and counters before releasing the lock · New
Guard invalid interim heap state before querying runtime stats · New
Add Doxygen documentation for the new public helper · New

What changed in this PR

Adds allocation-time diagnostics across SOF heap implementations and a unified debugging overlay.

Changes:

Logs vregion allocation statistics.
Adds aggregate virtual-heap usage logging after allocations.
Provides an overlay enabling runtime heap statistics.

File	Description
`zephyr/lib/vregion.c`	Logs lifetime and interim heap usage.
`zephyr/lib/regions_mm.c`	Aggregates virtual-heap allocator statistics.
`zephyr/lib/alloc.c`	Logs statistics after virtual-heap allocations.
`zephyr/include/sof/lib/regions_mm.h`	Declares the statistics helper.
`app/debug_heap_allocs.conf`	Enables runtime statistics options.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

kv2019i · 2026-10-08T11:15:33Z

+CONFIG_SYS_HEAP_RUNTIME_STATS=y
+CONFIG_SYS_MEM_BLOCKS_RUNTIME_STATS=y


Adjusted the comment to indicate what level of logs are emitted. We do not want to enable logs in this overlay.

kv2019i · 2026-10-08T11:16:00Z

+#if CONFIG_SYS_HEAP_RUNTIME_STATS
+	if (vr->type == VREGION_MEM_TYPE_LIFETIME) {
+		LOG_INF("lifetime alloc of %zu, used %zu, free %zu",
+			size, vr->lifetime.used,
+			vr->lifetime.size - vr->lifetime.used);


This is ok for a debug feature like this.

kv2019i · 2026-10-08T11:16:30Z

+	} else {
+		struct sys_memory_stats stats;
+
+		sys_heap_runtime_stats_get(&vr->interim.heap.heap, &stats);


Ack, fixed in V2.

kv2019i · 2026-10-08T11:17:00Z

 	heap->logged = true;
 }
+
+void vmh_log_alloc_stats(struct vmh_heap *heap)


Skipping this for internal helper.

intel-sofci · 2026-10-05T10:26:12Z

PR 11264: test results

Run date: 2026-10-08 12:27 UTC

Tested commit: 3dec250263b83c3350bbb6ff39277e1da0d25764

tmleman · 2026-10-05T17:20:04Z

+	} else {
+		struct sys_memory_stats stats;
+
+		sys_heap_runtime_stats_get(&vr->interim.heap.heap, &stats);
+
+		LOG_INF("interim alloc of %zu, used %u, free %u, max %u",
+			size, stats.allocated_bytes, stats.free_bytes, stats.max_allocated_bytes);
+
+	}


Is it possible to get here with vr->type == VREGION_MEM_TYPE_INVALID? switch in line 503 suggests so. In that case these structures are probably not correctly populated.

Ack, fixed this in V2.

tmleman · 2026-10-05T17:38:10Z

+		struct sys_memory_stats stats;
+
+		sys_heap_runtime_stats_get(&vr->interim.heap.heap, &stats);
+
+		LOG_INF("interim alloc of %zu, used %u, free %u, max %u",
+			size, stats.allocated_bytes, stats.free_bytes, stats.max_allocated_bytes);


sys_heap_runtime_stats_get can return -EINVAL. You can initialize stats to avoid printing garbage in the log.

Likewise, fixed in V2, thanks.

wjablon1 · 2026-10-06T13:00:27Z

One complaint I got regarding my solution was that the logs are never displayed... because my intention was to provide a function that could be used ad hoc during debugging (e.g. called while deleting a pipeline). On the other hand, your solution displays the stats unconditionally. But if you confirmed it doesn't overwhelm the logging interface, I guess that's fine...

EDIT:
Also for VMH, the total used/free memory isn't that interesting. I would print only stats related to a current VMH allocator instead of looping though all if them... and then maybe find a good place to call my function (pipeline delete, domain delete or something like that) that would print the summary for all the allocators. That would make both solutions complementary... but since we are moving away from VMH, I am not sure if you want to do so.

Add a single-line LOG_INF to z_impl_vregion_alloc_align() reporting the lifetime allocator's used and free bytes after each successful allocation. Gated on CONFIG_SYS_HEAP_RUNTIME_STATS. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>

Add a debug overlay file to enable alloc heap debugging, with prints of memory usage via logging subsystem. This is added as a separate file as this creates a notable increase in logging traffic and is not something one wants enabled in all builds (e.g. depends on the logging backend bandwidth). Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>

kv2019i · 2026-10-08T11:28:46Z

@wjablon1 wrote:

One complaint I got regarding my solution was that the logs are never displayed... because my intention was to provide a function that could be used ad hoc during debugging (e.g. called while deleting a pipeline). On the other hand, your solution displays the stats unconditionally. But if you confirmed it doesn't overwhelm the logging interface, I guess that's fine...

Ack. I have the same problem here. I chose not to put this into debug overlay (which is used in some validation configurations), but instead a separate overlay (= used by developers for specific tasks). The amount of logs generated is enough to flood the mtrace window (on Intel DSPs) for sure, so unless you switch the logging backend (e.g. use a mipi syst dictionary logging backend) you can miss important logs. So definitely not something that can be left enabled by default.

EDIT: Also for VMH, the total used/free memory isn't that interesting. I would print only stats related to a current VMH allocator instead of looping though all if them... and then maybe find a good place to call my function (pipeline delete, domain delete or something like that) that would print the summary for all the allocators. That would make both solutions complementary... but since we are moving away from VMH, I am not sure if you want to do so.

Ack. I think for the per allocator stats, your existing print-on-error already covers this.

I found the total interesting when sizing up a new configuration (adding/removing modules, moving stuff in/out of DRAM) and running tests to see how much runtime heap different use-cases need (and then iterate on the config until you find a working configuration with enough features built in to the FW, and still enough heap space).

Copilot AI balanced review requested due to automatic review settings October 2, 2026 15:37

kv2019i requested review from abonislawski, dabekjakub, dbaluta, iuliana-prodan, lbetlej, lgirdwood, lyakh, mmaka1 and plbossart as code owners October 2, 2026 15:37

kv2019i requested review from tmleman and wjablon1 October 2, 2026 15:38

Copilot started reviewing on behalf of kv2019i October 2, 2026 15:39 View session

Copilot AI reviewed Oct 2, 2026

View reviewed changes

tmleman reviewed Oct 5, 2026

View reviewed changes

kv2019i added 2 commits October 8, 2026 14:11

kv2019i force-pushed the 202610-heap-debugging branch from b5363b4 to 3dec250 Compare October 8, 2026 11:13

Navigation Menu

Repository navigation

heap: add heap debugging tools - #11264

heap: add heap debugging tools#11264
kv2019i wants to merge 3 commits into
thesofproject:mainfrom
kv2019i:202610-heap-debugging

kv2019i commented Oct 2, 2026

Uh oh!

kv2019i commented Oct 2, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

kv2019i Oct 8, 2026

Uh oh!

kv2019i Oct 8, 2026

Uh oh!

kv2019i Oct 8, 2026

Uh oh!

kv2019i Oct 8, 2026

Uh oh!

intel-sofci commented Oct 5, 2026 •

edited

Loading

Uh oh!

tmleman Oct 5, 2026

Uh oh!

kv2019i Oct 8, 2026

Uh oh!

tmleman Oct 5, 2026

Uh oh!

kv2019i Oct 8, 2026

Uh oh!

wjablon1 commented Oct 6, 2026 •

edited

Loading

Uh oh!

kv2019i commented Oct 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		CONFIG_SYS_HEAP_RUNTIME_STATS=y
		CONFIG_SYS_MEM_BLOCKS_RUNTIME_STATS=y

Conversation

kv2019i commented Oct 2, 2026

Uh oh!

kv2019i commented Oct 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

intel-sofci commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR 11264: test results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wjablon1 commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kv2019i commented Oct 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

intel-sofci commented Oct 5, 2026 •

edited

Loading

wjablon1 commented Oct 6, 2026 •

edited

Loading