docs: add design spec for #692 sandbox extended permissions

.agents/skills/ak-dev-architecture/SKILL.md

@@ -807,13 +807,31 @@ inert when disabled. To add a provider, use the `ak-dev-new-sandbox-provider` sk
 - **`SandboxProvider` / `Sandbox`** (`base.py`): the public ABCs a backend implements. `Sandbox`
   has two abstract methods, `execute_code` (`language="python"`) and `close`;
   `execute_command`/`upload_file`/`download_file`/`install_packages` are optional and raise
-  `SandboxCapabilityError` unless the provider declares them. `SandboxProvider` implements
+  `SandboxCapabilityError` unless the provider declares them.
+  `install_packages(packages, ecosystem="python", timeout=None)` raises
+  `SandboxCapabilityError(<handle>, "package_ecosystem:<e>")` for an ecosystem the instance does
+  not declare. The built-in installing providers delegate validation and argv building to
+  `PackageInstaller` (`packages.py`). `SandboxProvider` implements
   `create`/`destroy` (and `attach` when `capabilities.attach` is declared) and declares a
   `capabilities` class attribute.
 - **`SandboxCapabilities`** (`model.py`): the honest per-provider declaration (isolation tier,
-  shell, languages, files, package_install, stateful, attach, principal_user, policy_*). The
-  manager/worker consult it before routing; the worker enforces principal and policy **fail-closed**
-  against it (unenforceable + `strict` → `SandboxPolicyError`).
+  shell, languages, files, package_ecosystems, stateful, attach, principal_user, policy_*).
+  `package_ecosystems` is the source of truth for installs; a validator derives `package_install`
+  from it. Instance overrides go through `capabilities.derive(**update)`, never `model_copy`
+  (which skips that validator). On `docker` and `kubernetes`, `languages` is instance-declared
+  too: the keys of the provider block's `language_runtimes` (`_SandboxProviderRuntimeConfig`, default
+  `{python: [python, -c]}`, `python` required), set in the same single `derive()`; their
+  `execute_code` execs `[*language_runtimes[language], code]`. `e2b`/`daytona` stay `["python"]`.
+  The manager/worker consult it before routing; the worker enforces
+  principal and policy **fail-closed** against it (unenforceable + `strict` → `SandboxPolicyError`).
+- **`SandboxPermissionGuard`** (`permission.py`): decides what a profile **permits**, from the
+  request's own `SandboxPolicy` (`languages` for `execute_code`, `package_install` for
+  `install_packages`), never from `AKConfig` or capabilities. A denial raises
+  `SandboxPermissionDeniedError` after exactly one audit `WARNING` on `ak.sandbox.broker`. It runs
+  authoritatively in `BrokerWorkerCore` and, as defence in depth, in `ExecutionManager` before
+  session resolution (with package-spec validation), so a denied request has no side effects.
+  Not permitted (`SandboxPermissionDeniedError`) and not supported (`SandboxCapabilityError`)
+  stay separate.
 - **`ExecutionManager`** (`manager.py`): process-wide singleton (like `ConversationThreadManager`).
   Resolves the workload **profile** → provider, owns sandbox **sessions** (nv_cache registry,
   namespace-isolated per AK session; `per_call`/`per_session`/`per_runtime` scopes), and records
@@ -840,8 +858,9 @@ inert when disabled. To add a provider, use the `ak-dev-new-sandbox-provider` sk
   **Coupling amendment**: these three modules may import
   `pipeline.envelope`/`transport`/`consumer`/`thread_runner`/`response_store` (all core-only
   imports); the rest of `sandbox/` keeps the #494 core-only rule.
-- **Agent surface**: `tools.py` provides eight system tools (attached via `SystemToolFactory` when
-  enabled, optionally scoped to `sandbox.agents`); `hooks.py`'s `SandboxPreHook` ingests
+- **Agent surface**: `tools.py` provides eight system tools, plus `install_packages` when any
+  profile's policy sets `package_install` (attached via `SystemToolFactory` when enabled, optionally
+  scoped to `sandbox.agents`); `hooks.py`'s `SandboxPreHook` ingests
   task-completion events (the third system pre-hook). `principal.py` maps session/agent to a
   `SandboxPrincipal` (`principal_resolver` config for user identity). `testing.py` ships
   `FakeSandboxProvider` and the reusable `SandboxProviderContract`.
@@ -1076,11 +1095,14 @@ ak-py/src/agentkernel/
 │   ├── manager.py           # ExecutionManager (profile routing, sessions, fail-closed enforcement)
 │   ├── factory.py           # SandboxProviderFactory + ExecutionBrokerFactory
 │   ├── principal.py         # PrincipalResolver, AgentPrincipalResolver
-│   ├── tools.py             # The eight agent-facing system tools
+│   ├── tools.py             # Eight agent-facing system tools (+ conditional install_packages)
+│   ├── permission.py        # SandboxPermissionGuard, PermissionAuditContext (languages / package installs)
+│   ├── packages.py          # PackageSpecRule ABC, python/node rules, PackageSpecRuleFactory, PackageInstaller
 │   ├── hooks.py             # SandboxPreHook (task-completion ingestion)
 │   ├── testing.py           # FakeSandboxProvider + SandboxProviderContract (BYO test suite)
-│   ├── providers/           # local_subprocess, docker, e2b, daytona, ec2_ssm (+ planned: kubernetes, ...)
-│   └── broker/              # embedded, thread flavors + BrokerWorkerCore (+ planned: sqs)
+│   ├── providers/           # local_subprocess, docker, kubernetes, e2b, daytona, ec2_ssm;
+│   │                        #   process.py: ProcessKiller (env-marker kill for docker/kubernetes)
+│   └── broker/              # embedded, thread, queue flavors + BrokerWorkerCore + QueueBrokerWorker
 ├── schedule/                # Scheduling capability (deferred + recurring chat execution)
 │   ├── model.py             # ScheduledTask, ScheduledTaskPage, ScheduleStatus, occurrence tokens
 │   ├── errors.py            # ScheduleError

.agents/skills/ak-dev-new-sandbox-provider/SKILL.md

@@ -61,12 +61,17 @@ The sandbox capability (`ak-py/src/agentkernel/sandbox/`) is config-driven and p
   (the base default raises `SandboxCapabilityError`), and declares a `capabilities` class attribute.
 - **`Sandbox`** (`base.py`) — a handle to one live sandbox. Implements the two abstract methods
   `execute_code()` (`language="python"`) and `close()`, and optionally `execute_command()`,
-  `upload_file()`, `download_file()`, `install_packages()` (each raises `SandboxCapabilityError`
-  in the base until overridden).
+  `upload_file()`, `download_file()`, `install_packages(packages, ecosystem="python", timeout=None)`
+  (each raises `SandboxCapabilityError` in the base until overridden). An `install_packages` override
+  raises `SandboxCapabilityError(<handle>, "package_ecosystem:<e>")` for an ecosystem the instance
+  does not declare, and honors `timeout` (the install's effective timeout).
 - **`SandboxCapabilities`** (`model.py`) — the honest declaration of what the provider supports
-  (isolation tier, shell, languages, files, package_install, stateful, attach, principal_user,
-  policy_network/filesystem/resources). The manager/worker consult it before routing an operation;
-  an operation a provider didn't declare raises `SandboxCapabilityError`.
+  (isolation tier, shell, languages, files, package_ecosystems, stateful, attach, principal_user,
+  policy_network/filesystem/resources). `package_ecosystems` is the source of truth for installs;
+  `package_install` is derived from it by a validator. The manager/worker consult it before routing
+  an operation; an operation a provider didn't declare raises `SandboxCapabilityError`. What a
+  profile *permits* (`policy.languages`, `policy.package_install`) is decided separately by
+  `SandboxPermissionGuard`; providers never read the policy's permission fields.
 - **`SandboxProviderFactory`** (`factory.py`) — resolves a profile's `type` to a provider. Built-in
   short names are `if/elif` branches with real lazy imports; anything with a dot is treated as a
   dotted path to a `SandboxProvider` subclass (bring-your-own, no code change needed).
@@ -81,10 +86,16 @@ over-declaring is a security bug.
 **Instance-level capability override** (#503): when a capability's enforcement depends on
 operator-asserted infrastructure the provider cannot detect, keep the class declaration honest
 (`False`) and flip it per instance in `__init__` via
-`self.capabilities = type(self).capabilities.model_copy(update=...)` behind an explicit config
-flag. The reference is the `kubernetes` provider's `network_policy: true`, which asserts the
-cluster CNI actually enforces NetworkPolicy; consumers read `provider.capabilities` on the
-instance, so attribute shadowing is sufficient.
+`self.capabilities = type(self).capabilities.derive(...)` behind an explicit config flag.
+Use `derive()`, never `model_copy(update=...)`: `model_copy` skips the validator that keeps
+`package_install` consistent with `package_ecosystems`. Compose every override into **one**
+`derive()` call. The reference is the `kubernetes` provider: its `network_policy: true` (the
+operator asserts the cluster CNI enforces NetworkPolicy), its `language_runtimes` keys and its
+`package_installers` keys go into the same
+`derive(languages=list(runtimes), package_ecosystems=eco, policy_network=True)`. `derive()` keeps the two
+package fields consistent: updating `package_ecosystems` re-derives `package_install` (so `[]` means no installs),
+and `package_install=False` clears the ecosystems.
+Consumers read `provider.capabilities` on the instance, so attribute shadowing is sufficient.
 
 ## Step-by-Step
 
@@ -108,15 +119,24 @@ class <Provider>Sandbox(Sandbox):
     async def execute_code(self, code: str, language: str = "python", timeout: float | None = None) -> SandboxResult:
         # language="python" is MANDATORY. A failing program (non-zero exit) is a RESULT
         # (SandboxResult with exit_code != 0), NOT an exception. Raise only on machinery failure.
-        if language not in <declared languages>:
+        if language not in self._capabilities.languages:   # the instance's, not the class attribute
             raise SandboxCapabilityError(self.__class__.__name__, f"language:{language}")
         ...
+        # An argv-exec provider (docker, kubernetes) runs the configured interpreter prefix:
+        #   return await self._exec([*self._runtimes[language], code], timeout)
         return SandboxResult(stdout=..., stderr=..., exit_code=...)
 
     # Override ONLY the optional operations your capabilities declare. The base ABC raises
     # SandboxCapabilityError for execute_command / upload_file / download_file / install_packages
     # when not overridden — that is capability honesty in action.
 
+    # A provider that installs packages: check the instance's declared ecosystems, then let the
+    # injected PackageInstaller validate the specs and build the argv (or command string).
+    async def install_packages(self, packages: list[str], ecosystem: str = "python", timeout: float | None = None) -> SandboxResult:
+        if ecosystem not in self._capabilities.package_ecosystems:
+            raise SandboxCapabilityError(self.__class__.__name__, f"package_ecosystem:{ecosystem}")
+        return await self._exec(self._installer.argv(packages, ecosystem), timeout)  # or installer.command(...)
+
     async def close(self) -> None:
         # Idempotent. For per_session scope this must NOT destroy state needed for a later
         # attach() — only destroy() permanently disposes backend state.
@@ -126,6 +146,14 @@ class <Provider>Sandbox(Sandbox):
 Sync SDKs must be wrapped in `asyncio.to_thread` (see `docker.py`), since the sandbox runs on an
 event loop. Enforce `timeout` with `asyncio.wait_for` when the backend has no native timeout.
 
+Handles receive the provider **instance's** `capabilities` (and its `PackageInstaller`) as
+constructor keywords and check those, never the class attribute, so instance overrides count.
+A handle that execs processes inside a container should kill a timed-out or cancelled exec:
+`ProcessKiller` (`providers/process.py`) tags each exec with an environment marker and builds a
+procps-free `/proc` scan that kills exactly that exec and its children (see `docker.py`,
+`kubernetes.py`). Never kill in an environment the framework does not own (`attach_to`).
+The worker's own timer is only a backstop at `timeout + BrokerWorkerCore.BACKSTOP_GRACE_SECONDS`.
+
 ### 3. Implement the Provider
 
 ```python
@@ -140,7 +168,7 @@ class <Provider>SandboxProvider(SandboxProvider):
         shell=True,
         languages=["python"],
         files=True,
-        package_install=False,
+        package_ecosystems=[],               # e.g. ["python"]; empty = no installs (package_install is derived)
         stateful=False,
         attach=True,
         provisions=True,                     # False for attach-only backends (never create)
@@ -154,6 +182,15 @@ class <Provider>SandboxProvider(SandboxProvider):
     def __init__(self, config) -> None:
         super().__init__(config)             # config is the provider's Pydantic config block
         # Create SDK clients lazily (first use), not here — keep import/constructor cheap.
+        # A provider that installs packages builds its installer from its own config block and
+        # declares the configured ecosystems on the instance:
+        #   self._installer = PackageInstaller.from_config(config)
+        #   self.capabilities = type(self).capabilities.derive(package_ecosystems=self._installer.ecosystems)
+        # and passes capabilities=self.capabilities, installer=self._installer to every handle.
+        # A provider that execs an interpreter argv also declares its language_runtimes keys, in
+        # the same derive() call, and passes runtimes=self._runtimes to every handle:
+        #   self._runtimes = dict(config.language_runtimes)
+        #   self.capabilities = type(self).capabilities.derive(languages=list(self._runtimes), package_ecosystems=..., ...)
 
     async def create(self, *, principal: SandboxPrincipal, policy: SandboxPolicy) -> Sandbox:
         # Provision a new sandbox. Map `policy` onto the backend's real controls here; if a
@@ -203,7 +240,7 @@ wire it as an `Optional` field on **both** `_SandboxProfileConfig` and `_Sandbox
 latter for single-backend sugar), mirroring `_SandboxDockerConfig`:
 
 ```python
-class _Sandbox<Provider>Config(BaseModel):
+class _Sandbox<Provider>Config(BaseModel):   # or _SandboxProviderPackageConfig, see below
     # provider-specific fields, e.g. image, region, api_key_env, attach_to
     ...
 
@@ -214,6 +251,22 @@ class _Sandbox<Provider>Config(BaseModel):
 The factory's `_require_block` raises `SandboxConfigError` when a built-in's config block is
 missing, so the block must exist even if all fields have defaults (`<provider>: {}`).
 
+A provider that installs packages subclasses `_SandboxProviderPackageConfig` instead of
+`BaseModel`, as `docker`, `kubernetes`, `e2b` and `daytona` do. That mixin adds
+`package_installers` (ecosystem → installer argv prefix, default `{python: [pip, install]}`;
+declaring it replaces the default) and `package_rules` (ecosystem → dotted path of a
+`PackageSpecRule`), and `PackageSpecRuleFactory.validate_config` resolves its rules at agent and
+worker startup. So on the `queue` flavor, a custom rule module must be importable in the agent
+image as well as the worker image. Don't add your own installer fields.
+
+A provider that runs code by exec'ing an interpreter argv (as `docker` and `kubernetes` do) also
+subclasses `_SandboxProviderRuntimeConfig`. That mixin adds `language_runtimes` (language →
+interpreter argv prefix, default `{python: [python, -c]}`; declaring it replaces the default), with
+validators requiring a `python` key and non-empty argv prefixes. Its keys are the instance's
+`languages`, and `execute_code` runs `[*runtimes[language], code]`. A provider whose SDK picks the
+interpreter (`e2b`'s template kernel, `daytona`'s per-sandbox language) does not use it and keeps
+declaring `languages=["python"]`. Don't add your own runtime fields.
+
 ### 6. Add the Optional Dependency Extra
 
 If the provider needs an SDK, add an extra to `ak-py/pyproject.toml` (or reuse `aws` for boto3):
@@ -230,7 +283,10 @@ Add to `ak-py/tests/test_sandbox_providers.py`:
 - **Contract suite** — subclass the public `SandboxProviderContract` (from
   `agentkernel.sandbox.testing`) with a `provider` fixture returning your provider against a
   mocked SDK. It asserts the ABC semantics (mandatory `execute_code`, capability honesty,
-  idempotent close/destroy, result-vs-exception discipline, attach honesty).
+  idempotent close/destroy, result-vs-exception discipline, attach honesty). When the provider
+  declares `package_install`, the contract also checks that an undeclared ecosystem raises
+  `SandboxCapabilityError`, so the handle needs the `install_packages(packages, ecosystem, timeout)`
+  signature.
 - **Provider specifics** — create/attach/execute/destroy call shapes, policy/principal mapping
   arguments, `to_thread` usage for sync SDKs, timeout behavior.
 
@@ -264,7 +320,9 @@ including a profile that demonstrates enforced policy and sentinel-based determi
 ## Checklist
 
 - [ ] `ak-py/src/agentkernel/sandbox/providers/<provider>.py` with `Sandbox` + `SandboxProvider` subclasses
-- [ ] `capabilities` declared honestly (only what the backend truly enforces)
+- [ ] `capabilities` declared honestly (only what the backend truly enforces); instance overrides via `derive()`
+- [ ] If it installs packages: config subclasses `_SandboxProviderPackageConfig`, handle uses `PackageInstaller` and raises for undeclared ecosystems
+- [ ] If it execs an interpreter argv: config subclasses `_SandboxProviderRuntimeConfig`; instance `languages` = `language_runtimes` keys
 - [ ] Factory `if/elif` real-import branch + name in `_BUILTIN_PROVIDER_NAMES` (`factory.py`)
 - [ ] `_Sandbox<Provider>Config` block on `_SandboxProfileConfig` and `_SandboxConfig` (`config.py`)
 - [ ] Optional dependency extra in `pyproject.toml` (if the SDK isn't stdlib/boto3)