Spatial audio: a movable listener, sounds placed in world pixels

[obiot]

Description

Positional audio could only be reached by doing the work the engine should be doing.

The listener was welded to the origin with no public call to move it, so a game had to subtract the player's position from every source, every frame, and re-express it in the listener's axes by hand. The panner defaults were WebAudio's, which are metres, so a sound given pixel coordinates went near silent a few tiles out. And the two ways to place a sound quietly excluded each other.

Added

• follow, at and stopWithTarget on audio.play(). A sound tracks a renderable's absolute position after every world update, or pins to a fixed world point, and can end with the entity that owns it. audio.unfollow(id) detaches one and leaves it playing where it is.
• A movable listener: audio.setListener(target), audio.listener(x, y, z) and audio.listenerOrientation(fx, fy, fz, ux, uy, uz), all of which read back when called with no arguments. A Camera3d target contributes its basis as well as its position. Source positions become absolute world coordinates rather than offsets from the player.
• Panner defaults shaped for pixels: refDistance: 240, maxDistance: 10000, inverse, equalpower. audio.setSpatialDefaults() / getSpatialDefaults() change what new voices inherit.

Everything takes melonJS world coordinates with y measured down, the same numbers already in pos. The flip into Web Audio's Y-up space happens in exactly one place.

Fixed

All three were found while building the above.

• stereo() and position() drive one panner node per voice, and the first of the two to be called decided what kind of node it was, which made the other a silent no-op for the life of that voice. The node follows whichever was called last now, so the order no longer matters.
• audio.panner(name, attributes) with no playback id wrote only the voices already playing and never the clip's own defaults, so a call before the first play() changed nothing at all, the getter read back construction-time values forever, and every later voice inherited those. It writes the group now, and merges rather than replaces.
• Sound effects kept playing while the window did not have focus, because pausing the game only ever reached the current music track. A looping effect such as an engine hum played on over whatever the player had switched to. The mix is muted on blur and restored on focus, which covers tone() / noise() and anything hung off getMasterGain() as well as ordinary clips. Gated by the existing pauseOnBlur and stopOnBlur settings rather than a fourth one, so a game that deliberately keeps running in the background keeps its audio too.

Backward compatibility

Opt-in end to end. Until a game calls setListener / listener or passes follow / at, no frame handler is installed, the defaults are untouched, and every existing position / stereo / panner call behaves exactly as it did. The three fixes only change behaviour that was broken: a call that previously did nothing now does what it says.

Type of change

• Bug fix
• New feature
• Documentation update
• Performance improvement
• Refactoring (no functional changes)

Checklist

• I have read the Contributing Guide
• My code follows the existing code style (pnpm lint passes)
• I have tested my changes locally (pnpm test passes)
• I have added tests that cover my changes (if applicable)
• The build succeeds (pnpm build)

Verification

• tests/audio-spatial.spec.js, 26 new tests covering the listener, placement, the pixel defaults, both call orders, the group write and the blur mute. Each one was mutation tested: the Y flip, the panner-type swap, the group write, the merge, the blur mute, the per-frame follow, the defaults and stopWithTarget in both directions were each reintroduced as a defect and every one made a test fail.
• One of those is a regression test for a bug this PR's own first draft had: a listener read in world space compared against a source read in audio space passes even with no conversion at all, so it compares the two as the panner sees them instead.
• Full suite: 314 files, 7674 passed, 10 skipped.
• typedoc warnings 166 to 163; PlayOptions, PannerAttributes and SoundEvents are exported as public types, which is what documents the new play options.
• Driven in a real game in a browser: the listener tracks the camera exactly frame to frame, a panner() call made before the first play() now reaches the group, blur mutes and focus restores, no page errors.

The melonjs-audio skill is rewritten to match, and the CHANGELOG entry that described the old behaviour as a documented limitation is corrected rather than left to contradict the code.

Related issues

🤖 Generated with Claude Code

https://claude.ai/code/session_01NGvtaUNATVCVxD2qcbiY4t

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.