Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,23 @@ This document outlines the rules AI agents must follow when checking out branche
### Codestyle and Linting
* **Standard:** Use `clangd` instead of `checkpatch` for codestyle verification.
* **Rationale:** `checkpatch` is prone to confusion with assembly and non-standard C; `clangd` provides better integration with IDEs and AI tools and is easier to maintain.

### User/kernel boundary
On PTL and other userspace configurations, audio application logic runs in Zephyr user-space
while the OS/HAL stays in the kernel. The authoritative model, directory taxonomy, syscall map,
memory-partition markers and Kconfig hierarchy are documented in
[`src/include/sof/userspace/README.md`](src/include/sof/userspace/README.md). When adding or
moving code, follow it:

* **Place code in the right tier.** OS/HAL/boot/inter-core → a kernel-only directory
(`arch/`, `drivers/`, `idc/`, `init/`, `platform/`, `probe/`, `logging/`, `trace/`); audio
application logic → `audio/`; reusable pure code → a shared-library directory (`lib/`,
`math/`, `module/`) kept free of privileged operations. Each top-level directory's
`README.md` states its tier in a "Runs in:" banner — keep it accurate.
* **Cross the boundary only via a syscall.** Add a `__syscall` declaration, put the `z_vrfy_`
validator in the boundary tier (preferably `zephyr/syscall/<subsystem>.c`), validate every
user-supplied argument, and add the call to
[`src/include/sof/userspace/syscalls.h`](src/include/sof/userspace/syscalls.h).
* **Mark shared globals.** Any global a user thread reads or writes must carry a partition
marker (`APP_TASK_*`, `APP_SYSUSER_*`, or a `K_APP_*` partition) at its declaration site, and
kernel code must re-validate such data because it is user-writable.
8 changes: 8 additions & 0 deletions src/arch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Architecture Support (`src/arch`)

> **Runs in:** Kernel-only — CPU-architecture primitives (Xtensa) execute in supervisor context
> and are never linked into a user thread. See
> [User/kernel split](../include/sof/userspace/README.md).
This directory contains architecture-specific code. It sits below the RTOS and audio
application and has no user-space entry points.
18 changes: 18 additions & 0 deletions src/audio/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Audio Components (`src/audio`)

> **Runs in:** Application — under userspace configs (e.g. PTL with `CONFIG_SOF_USERSPACE_LL`),
> the pipeline, buffers, copiers and processing components in this tree run in MMU-isolated
> **user** threads. They reach kernel resources only through the SOF syscalls. See
> [User/kernel split](../include/sof/userspace/README.md).

This directory holds the SOF audio application: the pipeline graph and scheduling glue, host/DAI
copiers, and the ~36 processing components/modules (volume, EQ, SRC, mixers, codecs, etc.). This
is the code the user/kernel split is designed to protect the rest of the firmware from.

Notable subdirectories carry their own README with a matching "Runs in:" banner, e.g.
[`module_adapter/`](module_adapter/README.md), which is the boundary container that launches
modules in user context.

When adding a component here, keep it inside the application tier: allocate through the module
allocation syscalls (`mod_*`), mark any globals shared with the kernel using the partition
markers, and never call kernel-only subsystems (`drivers/`, `platform/`, `idc/`) directly.
6 changes: 6 additions & 0 deletions src/audio/module_adapter/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Module Adapter Architecture

> **Runs in:** Boundary / bridge — the module adapter is the container that runs audio modules
> in **user** threads (DP threads, or the userspace proxy in `library/userspace_proxy.c`) while
> being driven from the **kernel** pipeline. It hosts the `mod_*` allocation syscalls and sets
> up per-module memory domains and object grants. See
> [User/kernel split](../../include/sof/userspace/README.md).

This directory contains the Module Adapter.

## Overview
Expand Down
11 changes: 11 additions & 0 deletions src/drivers/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Device Drivers (`src/drivers`)

> **Runs in:** Kernel-only — hardware/HAL access runs in supervisor context and is never
> executed from a user thread. User-space audio code reaches driver functionality only through
> SOF syscalls (e.g. the `sof_dma_*` and `dai_get_device` calls). See
> [User/kernel split](../include/sof/userspace/README.md).

This directory contains low-level device drivers and hardware abstraction. Because it touches
MMIO, interrupts and other privileged resources, nothing here is mapped into a user memory
domain. If user-space code needs a driver operation, expose it through a validated syscall in
the boundary tier rather than linking this code into a user thread.
9 changes: 9 additions & 0 deletions src/idc/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Inter-Domain / Inter-Core Communication (`src/idc`)

> **Runs in:** Kernel-only — cross-core messaging (`idc.c`, `zephyr_idc.c`) runs in supervisor
> context. User-space audio code triggers cross-core work indirectly through higher-level
> subsystems, not by calling IDC directly. See
> [User/kernel split](../include/sof/userspace/README.md).

This directory implements the mechanism cores use to signal and hand work to each other. It is
part of the OS-integration layer, not the audio application.
236 changes: 236 additions & 0 deletions src/include/sof/userspace/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
# SOF User/Kernel Split — Architecture and Conventions

This is the canonical reference for how Sound Open Firmware (SOF) is divided between
**kernel (supervisor) context** and **user (unprivileged) context** on top of Zephyr
userspace. It defines the conventions that make the split evident in the source layout and
easy to maintain.

> **Status:** convention proposal. The execution model and the syscall/marker/Kconfig facts
> below reflect the code as it is today. The *directory taxonomy* and the per-directory
> "Runs in:" banners are the proposed mechanism for making the split visible; they are being
> rolled out incrementally.

## Why this exists

SOF supports several privilege configurations. On PTL (`intel_adsp_ace30_ptl`) the board
config enables `CONFIG_USERSPACE` and `CONFIG_SOF_USERSPACE_PROXY`, running individual audio
modules in isolated user threads via the proxy container. For Linux builds
(`os_linux_overlay.conf`), `app/CMakeLists.txt` automatically includes
`app/overlays/ptl/ll_userspace_overlay.conf`, which additionally enables
`CONFIG_SOF_USERSPACE_LL` so the entire LL pipeline and the module-facing IPC run in user
threads. This is the standard PTL Linux build configuration. The kernel keeps the OS/HAL,
boot, drivers, and the syscall validators in both cases.

The privilege boundary is real, but nothing in the source *layout* signals which side a given
file belongs to. This document makes that explicit.

## Execution model

```mermaid
graph TD
subgraph Kernel [Kernel / supervisor context - ring 0]
Boot[Boot & init]
Drv[Drivers / HAL]
Dispatch[IPC dispatch]
LLk[LL scheduler core + domain setup]
Vrfy["Syscall validators (z_vrfy_*)"]
end

subgraph User [User context - ring 3, MMU-isolated]
LLu[LL pipeline thread]
DP[DP module threads]
Mods[Audio modules / copiers]
IPCu[Module-facing IPC handler]
end

Dispatch -->|forwards cmd| IPCu
LLk -->|starts| LLu
LLu -->|"syscalls"| Vrfy
DP -->|"syscalls"| Vrfy
IPCu -->|"syscalls"| Vrfy
Vrfy -->|"z_impl_* (validated)"| Drv
```

- **Kernel context** owns hardware, boot, interrupt handling, memory-domain construction, and
the *validating* half of every syscall (`z_vrfy_*`).
- **User context** runs the audio application: LL pipeline processing, DP modules, copiers,
and the module-facing IPC path.
- The two contexts cross the boundary in two ways:
1. **Syscalls** — user code calls a kernel service (see
[The boundary is the syscall set](#the-boundary-is-the-syscall-set)).
2. **Kernel-to-user shared-memory handoff** — the kernel writes IPC message words to
shared `ipc_user` state, signals the user thread via a granted `k_event`, and waits on
a semaphore for the result (`ipc_user_forward_cmd()` in `ipc/ipc-common.c`). The user
thread wakes, processes the message, and posts the semaphore back.

## Directory taxonomy — where code runs

Every `src/` subdirectory falls into one of four tiers. This is the primary mechanism for
making the split evident: the tier tells you where a directory's code runs before you read a
line of it. Top-level directories carry a **"Runs in:"** banner in their `README.md` to
declare their tier. The banner rollout is in progress; directories in the table that still
lack a banner or a README are noted. The `src/include/` tree is not classified here because
its subdirectories span all tiers.

| Tier | Meaning | Directories |
|------|---------|-------------|
| **Kernel-only** | OS integration, HAL, boot, inter-core. Never executes in a user thread. Must not be reached directly from user code — only via a syscall. | `arch/`, `drivers/`, `idc/`, `init/`, `platform/`, `probe/` *, `logging/` *, `trace/` * |
| **Boundary / bridge** | Owns the syscalls and the thread / memory-domain setup. Split internally: some translation units run kernel-side, some user-side (e.g. `handler-kernel.c` vs `handler-user.c`, `zephyr_ll.c` vs `zephyr_ll_user.c`). | `schedule/`, `ipc/`, `library_manager/` *, `debug/debug_stream/` *, and (in the Zephyr integration tree) `zephyr/syscall/` |
| **Application** | Audio pipeline / component logic that runs in user threads under userspace configs. | `audio/` |
| **Shared library** | Context-agnostic code linked into whichever context calls it. Must stay "user-safe": no privileged operations (no direct MMIO, no spinlocks held across syscalls, no raw kernel object access) so it is safe when pulled into a user build. | `math/`, `module/` |
| **Mixed** | A directory whose files span more than one tier; see its own README for per-file classification. | `lib/` |

Directories marked * do not yet have a "Runs in:" README banner.

Classification is grep-verifiable — see [Verifying the taxonomy](#verifying-the-taxonomy).

> **`zephyr/syscall/` is the model to grow toward.** The Zephyr integration tree already
> collects the validating half of several syscalls in one place — `zephyr/syscall/alloc.c`,
> `cpu.c`, `vregion.c`, `dai.c`, `sof_dma.c` each hold the `z_vrfy_*` wrappers for their
> subsystem, physically separated from the `z_impl_*` logic (in `zephyr/lib/*.c` and
> `src/lib/*.c`). This is exactly the boundary-tier separation this document promotes; the
> migration opportunity is to move the *still-scattered* validators (IPC, module management,
> library manager, scheduling, debug) toward the same convention.

> Borderline cases (e.g. parts of `lib/` that host syscall *implementations* such as `dma.c`
> and `dai.c`) behave as boundary code even though the directory is mostly shared-library. When
> a single directory genuinely mixes tiers, its README states the exception explicitly.

## The boundary is the syscall set

Zephyr crosses the user/kernel boundary with system calls declared `__syscall` in a header and
implemented as two functions:

- `z_vrfy_<name>()` — **kernel-side validator**. Runs in supervisor mode, validates every
argument coming from user space (`K_SYSCALL_MEMORY_READ/WRITE`, `K_SYSCALL_OBJ`,
`k_usermode_from/to_copy`), then calls the implementation. This is the trust boundary.
- `z_impl_<name>()` — the actual logic. Callable directly from kernel code (fast path) or from
the validator when the call originates in user space.

**These pairs are the entire user/kernel API surface.** The table below summarises every SOF
syscall by subsystem:

| Subsystem | Syscalls | Declared in | Validator (`z_vrfy_`) / logic (`z_impl_`) |
|-----------|----------|-------------|-------------------------------------------|
| IPC | `ipc_msg_send`, `ipc_msg_list_remove`, `ipc_msg_reply`, `ipc_compound_pre_start`, `ipc_compound_post_start`, `ipc_wait_for_compound_msg`, `send_resource_notif` | `sof/ipc/msg.h`, `sof/ipc/ipc_reply.h`, `ipc4/handler.h`, `ipc4/notification.h` | scattered: `ipc/ipc-common.c`, `ipc/ipc3/helper.c`, `ipc/ipc4/handler-kernel.c`, `ipc/ipc4/notification.c` |
| Heap alloc | `sof_heap_alloc`, `sof_heap_free` | `rtos/alloc.h` | `zephyr/syscall/alloc.c` / `zephyr/lib/alloc.c` |
| Virtual regions | `vregion_create_map`, `vregion_set_interim`, `vregion_get`, `vregion_put`, `vregion_alloc_align`, `vregion_alloc_coherent_align`, `vregion_free` | `sof/lib/vregion.h` | `zephyr/syscall/vregion.c` / `zephyr/lib/vregion.c` |
| DMA | `sof_dma_get`, `sof_dma_put`, `sof_dma_get_attribute`, `sof_dma_request_channel`, `sof_dma_release_channel`, `sof_dma_config`, `sof_dma_start`, `sof_dma_stop`, `sof_dma_get_status`, `sof_dma_reload`, `sof_dma_suspend`, `sof_dma_resume` | `sof/lib/sof_dma.h` | `zephyr/syscall/sof_dma.c` / `lib/dma.c` (get/put); remaining 10 are `static inline z_impl_*` in `sof/lib/sof_dma.h` |
| DAI | `dai_get_device` | `sof/lib/dai-zephyr.h` | `zephyr/syscall/dai.c` / `lib/dai.c` |
| CPU | `cpu_get_id` | `sof/lib/cpu.h` | `zephyr/syscall/cpu.c` / `zephyr/lib/cpu.c` |
| Module management | `mod_balloc_align`, `mod_alloc_ext`, `mod_free`, `mod_free_all`, `mod_data_blob_handler_new`, `mod_fast_get` | `sof/audio/module_adapter/module/generic.h` | scattered: `audio/module_adapter/module/generic.c` |
| Library manager | `lib_manager_free_module` | `sof/lib_manager.h` | scattered: `library_manager/lib_manager.c` |
| Scheduling | `zephyr_ll_task_sem_alloc`, `zephyr_ll_task_sem_free`, `scheduler_dp_internal_free`, `scheduler_dp_ll_tick` | `sof/schedule/ll_schedule_domain.h`, `sof/schedule/dp_schedule.h` | scattered: `schedule/zephyr_ll.c`, `schedule/zephyr_dp_schedule*.c` |
| Debug | `debug_stream_slot_send_record` | `user/debug_stream_slot.h` | scattered: `debug/debug_stream/debug_stream_slot.c` |

The `zephyr/syscall/*.c` files hold the validators in one place per subsystem; the rows marked
**scattered** keep their `z_vrfy_` next to the kernel logic and are the migration targets.

Most syscall declarations are gated by `#if defined(__ZEPHYR__) && defined(CONFIG_SOF_FULL_ZEPHYR_APPLICATION)`; outside that guard (non-Zephyr host-test and ztest builds) the same name is a plain macro or inline calling `z_impl_*` directly. Non-userspace Zephyr builds (no `CONFIG_USERSPACE`) still use the `__syscall` declaration path, which Zephyr compiles to a direct `z_impl_*` call without the privilege-elevation trap.

Comment thread
kv2019i marked this conversation as resolved.
## Memory-partition markers

User threads can only touch memory that is mapped into their memory domain. SOF exposes shared
globals to user threads through Zephyr application memory partitions. The markers live in
[`rtos/userspace_helper.h`](../../../../zephyr/include/rtos/userspace_helper.h):

| Marker | Partition | Availability | Use for |
|--------|-----------|--------------|---------|
| `APP_TASK_DATA` / `APP_TASK_BSS` | `common_partition` | `CONFIG_USERSPACE` | Data shared with the userspace proxy worker thread and similar kernel-managed user tasks; not mapped into DP module domains in `SOF_USERSPACE_APPLICATION` mode |
| `APP_SYSUSER_DATA` / `APP_SYSUSER_BSS` | `sysuser_partition` | `CONFIG_SOF_USERSPACE_LL` | Data shared with the LL user thread and the LL userspace IPC thread |
| `K_APP_DMEM(part)` / `K_APP_BMEM(part)` | kernel-defined partition | `CONFIG_USERSPACE` | Per-subsystem partitions defined and mapped by kernel code (e.g. `ipc_context_part` in `ipc-common.c`); not intended for use by user components directly |

When userspace is disabled, all four SOF markers expand to nothing, so annotated globals are
ordinary variables in non-userspace builds.

**Rule:** *any global that a kernel-managed user thread reads or writes must carry a partition
marker at its declaration site.* Conversely, marked data is **user-writable**, so kernel code
must never trust it — re-validate on every use. DP modules running under
`SOF_USERSPACE_APPLICATION` access data through their per-module memory domain set up by the
proxy or DP thread infrastructure, not through `APP_TASK_*` / `APP_SYSUSER_*` markers. See the warning in
[`schedule/zephyr_ll_user.c`](../../../schedule/zephyr_ll_user.c) around `zephyr_ll_heap`, and
`zephyr_ll_user_heap_verify()` which exists precisely because the user-visible heap pointer
cannot be trusted.

## Kconfig hierarchy

The userspace behaviour is assembled from a small option tree (source of truth:
[`zephyr/Kconfig`](../../../../zephyr/Kconfig)).

```mermaid
graph TD
US["USERSPACE (Zephyr MMU)"]
US --> ALLOC[SOF_USERSPACE_INTERFACE_ALLOC]
US --> DMA[SOF_USERSPACE_INTERFACE_DMA]
US --> VREG["SOF_USERSPACE_INTERFACE_VREGION (needs SOF_VREGIONS)"]
US --> LL[SOF_USERSPACE_LL]
LL -.selects.-> ALLOC
LL -.selects.-> DMA
LL -.selects if SOF_VREGIONS.-> VREG
US --> SHEAP[SOF_USERSPACE_USE_SHARED_HEAP]
US --> DHEAP[SOF_USERSPACE_USE_DRIVER_HEAP]
PROXY[SOF_USERSPACE_PROXY]
PROXY -.selects.-> SHEAP
PROXY -.selects.-> DHEAP
PROXY --> MODIPC[SOF_USERSPACE_MOD_IPC_BY_DP_THREAD]
APP["SOF_USERSPACE_APPLICATION (auto = USERSPACE && !PROXY, needs SOF_VREGIONS)"]
```

Key options:

- **`SOF_USERSPACE_LL`** — run LL pipelines in user threads (`depends on USERSPACE`; selects
the ALLOC/DMA/VREGION interfaces). This is what the PTL overlay turns on.
- **`SOF_USERSPACE_PROXY`** — run modules inside a proxy container that forwards kernel-side
calls to user-privileged module code (selects the driver + shared heaps).
- **`SOF_USERSPACE_APPLICATION`** — not user-settable; a shortcut for
`USERSPACE && !SOF_USERSPACE_PROXY` (`depends on SOF_VREGIONS`), used to switch the DP
scheduler between the `_application` and `_thread` implementations in
[`schedule/CMakeLists.txt`](../../../schedule/CMakeLists.txt).
- **`SOF_USERSPACE`** — a separate WIP flag for userspace *modules*; do not confuse with
`USERSPACE` (the Zephyr option) or `SOF_USERSPACE_LL`.

## Rules for placing new code

1. **Pick the tier first.** OS/HAL/boot → a kernel-only directory. Audio application logic →
`audio/`. Reusable pure code → a shared-library directory (keep it user-safe).
2. **Cross the boundary only via a syscall.** Add a `__syscall` declaration, put the
`z_vrfy_` validator in the boundary tier — preferably in `zephyr/syscall/<subsystem>.c`
alongside the existing ones — and validate every argument.
3. **Mark shared globals.** Any global a kernel-managed user thread (LL user thread, userspace
IPC thread, proxy worker) reads/writes gets `APP_TASK_*`, `APP_SYSUSER_*`, or a `K_APP_*`
partition marker — and the kernel re-validates it. DP modules in application mode use their
per-module domain instead; see the DP scheduler and userspace proxy for how that is wired.
4. **Grant kernel objects explicitly.** User threads need `k_object_access_grant()` /
`k_thread_access_grant()` for each kernel object they use.
5. **Split by suffix inside boundary directories.** Where a subsystem has both sides, keep the
existing `-kernel` / `-user` file split so the two contexts stay physically separate.

## Verifying the taxonomy

The tier of any directory can be confirmed from its source signals:

```sh
# Boundary code: declares or implements syscalls
grep -rl 'z_vrfy_\|__syscall' src/<dir>

# User-visible data: partition markers
grep -rl 'APP_SYSUSER\|APP_TASK\|K_APP_BMEM\|K_APP_DMEM\|K_APPMEM_PARTITION' src/<dir>

# User threads / object grants / domains
grep -rl 'K_USER\|_access_grant\|mem_domain' src/<dir>
```
Comment thread
kv2019i marked this conversation as resolved.

A kernel-only directory returns nothing for all three. A boundary directory hits the first (and
usually the third). The application/shared tiers hit markers but should not host `z_vrfy_`
validators.

## Related files

- [`zephyr/include/rtos/userspace_helper.h`](../../../../zephyr/include/rtos/userspace_helper.h)
— partition markers and user-space helper API.
- [`schedule/README.md`](../../../schedule/README.md),
[`ipc/README.md`](../../../ipc/README.md),
[`audio/module_adapter/README.md`](../../../audio/module_adapter/README.md) — boundary
subsystems.
- [`AGENTS.md`](../../../../AGENTS.md) — contributor and AI-agent workflow rules that
reference this document.
Loading
Loading