From 70d378e1a810dc3d601f96b46287733dc5020b13 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 28 Sep 2026 10:30:24 +0000 Subject: [PATCH 1/4] docs(manual): chapters for the other seven actions; delete QUICK-REFERENCE.md (#190) PR 3 of the user manual (#178). build-lectures, build-jupyter-cache, restore-jupyter-cache, preview-netlify, preview-cloudflare, publish-gh-pages and deploy-cloudflare each get a chapter in docs/user/actions/, from the chapter template, and their READMEs become generated signposts. - Each action.yml's input and output descriptions are rewritten for readers, since the manual copies them verbatim. They are corrected where they or the READMEs were wrong: restore-jupyter-cache's path is part of a cache's identity, so any path but _build finds nothing build-jupyter-cache saved; restore uses two build-cache fallbacks, not three; the preview actions fetch the base and head commits themselves. - README-only content moves into the chapters: the Netlify and Cloudflare Pages setup guides, their comparison (kept to what the actions do), the publish-gh-pages migration notes, and the deploy-cloudflare Access checklist, with no account details. - publish-gh-pages documents its release assets (archive, SHA-256 checksum, manifest) as a contract (#27). - docs/QUICK-REFERENCE.md is deleted, and every link to it repointed. - scripts/generate-docs.py makes a missing chapter an error, so the gate fails for an action without one. - deploy-cloudflare's refusal message points at the chapter's setup checklist instead of the README. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01FeKTxWtkzVMyPbqeSgM2Tb --- .github/copilot-instructions.md | 6 +- .github/workflows/test-actions.yml | 11 +- CHANGELOG.md | 24 ++ CONTRIBUTING.md | 17 +- README.md | 1 - build-jupyter-cache/README.md | 233 +----------- build-jupyter-cache/action.yml | 101 ++++-- build-lectures/README.md | 310 +--------------- build-lectures/action.yml | 64 +++- deploy-cloudflare/README.md | 203 +---------- deploy-cloudflare/action.yml | 67 ++-- docs/QUICK-REFERENCE.md | 395 --------------------- docs/dev/ARCHITECTURE.md | 4 +- docs/dev/CHAPTER-TEMPLATE.md | 2 +- docs/dev/PLAN.md | 4 +- docs/dev/TESTING.md | 2 +- docs/user/README.md | 14 +- docs/user/actions/build-jupyter-cache.md | 152 ++++++++ docs/user/actions/build-lectures.md | 187 ++++++++++ docs/user/actions/deploy-cloudflare.md | 221 ++++++++++++ docs/user/actions/preview-cloudflare.md | 157 ++++++++ docs/user/actions/preview-netlify.md | 176 +++++++++ docs/user/actions/publish-gh-pages.md | 236 ++++++++++++ docs/user/actions/restore-jupyter-cache.md | 171 +++++++++ docs/user/actions/setup-environment.md | 4 +- preview-cloudflare/README.md | 215 +---------- preview-cloudflare/action.yml | 47 ++- preview-netlify/README.md | 188 +--------- preview-netlify/action.yml | 35 +- publish-gh-pages/README.md | 283 +-------------- publish-gh-pages/action.yml | 37 +- restore-jupyter-cache/README.md | 224 +----------- restore-jupyter-cache/action.yml | 59 ++- scripts/generate-docs.py | 35 +- 34 files changed, 1715 insertions(+), 2170 deletions(-) delete mode 100644 docs/QUICK-REFERENCE.md create mode 100644 docs/user/actions/build-jupyter-cache.md create mode 100644 docs/user/actions/build-lectures.md create mode 100644 docs/user/actions/deploy-cloudflare.md create mode 100644 docs/user/actions/preview-cloudflare.md create mode 100644 docs/user/actions/preview-netlify.md create mode 100644 docs/user/actions/publish-gh-pages.md create mode 100644 docs/user/actions/restore-jupyter-cache.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index c39430d..67dcdfa 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -19,7 +19,7 @@ This repository provides **reusable GitHub Actions** for building QuantEcon lect | **Container usage** | [docs/CONTAINER-GUIDE.md](../docs/CONTAINER-GUIDE.md) | | **GPU AMI setup** | [docs/dev/GPU-AMI-SETUP.md](../docs/dev/GPU-AMI-SETUP.md) | | **Testing validation** | [docs/dev/TESTING.md](../docs/dev/TESTING.md) | -| **Quick reference** | [docs/QUICK-REFERENCE.md](../docs/QUICK-REFERENCE.md) | +| **Using the actions, one chapter each** | [docs/user/README.md](../docs/user/README.md) | | **Release process** | [CONTRIBUTING.md](../CONTRIBUTING.md) | | **Version history** | [CHANGELOG.md](../CHANGELOG.md) | @@ -60,8 +60,8 @@ Before merging action changes: ### Documentation Updates When changing actions, update: -- Action's `README.md` (inputs/outputs) -- `docs/QUICK-REFERENCE.md` (if inputs added) +- The action's `action.yml` descriptions, then run `python3 scripts/generate-docs.py` (inputs/outputs) +- The action's chapter, `docs/user/actions/.md` (what it does) - `CHANGELOG.md` (user-facing changes) - `docs/dev/PLAN.md` (if affects migration status) diff --git a/.github/workflows/test-actions.yml b/.github/workflows/test-actions.yml index a32ffde..7ebca34 100644 --- a/.github/workflows/test-actions.yml +++ b/.github/workflows/test-actions.yml @@ -259,9 +259,10 @@ jobs: [ "$drift" -eq 0 ] || exit 1 echo "✅ templates/ third-party pins match .github/workflows/ ($checked checked)" - # #178, decision 5: the user manual's Inputs and Outputs tables, its - # action table and the signpost READMEs are generated from each - # action.yml, and its examples are checked against them. In the gate for + # #178, decision 5: every action has a chapter in the user manual; its + # Inputs and Outputs tables, the action table and the signpost READMEs + # are generated from each action.yml, and the manual's examples are + # checked against them. In the gate for # the reason the pin check above is: a docs-only PR matches IGNORED and # skips every other job. PyYAML is the pin in scripts/requirements.txt, # installed into a venv because the runner's Python is externally managed. @@ -286,7 +287,7 @@ jobs: tar -xzf "$RUNNER_TEMP/$tarball" -C "$RUNNER_TEMP/actionlint" actionlint "$RUNNER_TEMP/actionlint/actionlint" -version - - name: Check the manual's generated tables match each action.yml + - name: Check every action has a chapter, and the manual's generated tables match each action.yml run: | "$RUNNER_TEMP/docs-venv/bin/python" scripts/generate-docs.py --check @@ -1706,7 +1707,7 @@ jobs: steps: - uses: actions/checkout@v7 - # The Node the READMEs tell hosted-runner callers to set up, and the major + # The Node the manual tells hosted-runner callers to set up, and the major # the QuantEcon containers ship. - uses: actions/setup-node@v7 with: diff --git a/CHANGELOG.md b/CHANGELOG.md index b0cc299..73e7bee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,6 +33,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 under `containers/` still triggers a build: the smoke-test fixture's pages are Markdown. (#178) ### Documentation +- **User manual** (#178): every action has its chapter. `build-lectures`, `build-jupyter-cache`, + `restore-jupyter-cache`, `preview-netlify`, `preview-cloudflare`, `publish-gh-pages` and + `deploy-cloudflare` get theirs in `docs/user/actions/`, and their READMEs become generated + signposts. What only the READMEs held moves into the chapters: the Netlify and Cloudflare Pages + setup guides; the comparison of the two, now kept to what the actions do, since provider pricing + is not something this repository can keep current; the `publish-gh-pages` migration notes; and + the `deploy-cloudflare` Access checklist, with no account details. The `publish-gh-pages` chapter + documents the release assets, the archive, its SHA-256 checksum and the manifest, as a contract + that a release changes only with a CHANGELOG entry (#27). `docs/QUICK-REFERENCE.md` is removed: + its action table is the manual's index, and the rest is in the chapters. The harness gate now + fails when an action has no chapter. (#190) +- **The seven actions' `action.yml` descriptions** are rewritten for readers, since the manual + copies them verbatim, and corrected where they, or the READMEs, were wrong: + - `restore-jupyter-cache`'s `path` said a different path restores the cache into a directory + the build does not read. The path is part of what identifies a cache, so any path but + `_build` finds none of the caches `build-jupyter-cache` saves. + - The `build-jupyter-cache` README listed a third, bare `build-` fallback key, which + `restore-jupyter-cache` no longer uses. The chapters give the two it does. + - The preview READMEs said change detection needs `fetch-depth: 0`. The actions fetch the pull + request's base and head commits themselves. + - The `preview-cloudflare` setup guide now asks for a token with Cloudflare Pages Edit alone, + not the broader "Edit Cloudflare Workers" template. + - `deploy-cloudflare`'s `account-subdomain` example no longer names a real account, and its + refusal message points at the chapter's setup checklist instead of the README. (#190) - **User manual** (#178): `docs/user/` gets its index, `docs/user/README.md`, and its first chapter, `docs/user/actions/setup-environment.md`, which absorbs `setup-environment/README.md`; that README is now a signpost to it. The chapter's Inputs and Outputs tables, the index's action diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index eb98682..e7c20a9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -122,12 +122,13 @@ skipped job reports success to a required check). If the harness should ignore a extend `IGNORED` in that job; it is an ignore list, so anything unrecognised runs the harness rather than silently passing. -The `gate` job also checks the user manual, so a docs-only PR is checked too. The input and output -tables in `docs/user/actions/`, the action table in `docs/user/README.md` and the README of every -action with a chapter are generated from each `action.yml` by `scripts/generate-docs.py`, and the -gate fails when a committed copy no longer matches: run the script and commit what it writes. It -also checks every example in `docs/user` and `templates/` against the `action.yml` files, for -`permissions:`, and with actionlint. [docs/dev/CHAPTER-TEMPLATE.md](docs/dev/CHAPTER-TEMPLATE.md) +The `gate` job also checks the user manual, so a docs-only PR is checked too. Every action has a +chapter in `docs/user/actions/`. The chapters' input and output tables, the action table in +`docs/user/README.md` and every action's README are generated from each `action.yml` by +`scripts/generate-docs.py`, and the gate fails when a committed copy no longer matches, or when an +action has no chapter: run the script and commit what it writes. It also checks every example in +`docs/user` and `templates/` against the `action.yml` files, for `permissions:`, and with +actionlint. [docs/dev/CHAPTER-TEMPLATE.md](docs/dev/CHAPTER-TEMPLATE.md) lists the checks and how to run them locally. ### Breaking Changes @@ -175,8 +176,8 @@ Update these docs when adding features: | Doc | Update When | |-----|-------------| | The action's `action.yml` descriptions, then run `python3 scripts/generate-docs.py` | Any input or output change | -| The action's chapter, `docs/user/actions/.md`, or its `README.md` while it has no chapter | A change in what the action does | -| `docs/QUICK-REFERENCE.md` | New inputs added | +| The action's chapter, `docs/user/actions/.md` | A change in what the action does | +| A chapter for it, from `docs/dev/CHAPTER-TEMPLATE.md`, and its place in `ORDER` in `scripts/generate-docs.py` | A new action | | `docs/MIGRATION-GUIDE.md` | Workflow patterns change | | `docs/dev/PLAN.md` | A backlog item or tracked issue opens, closes or changes scope; a consumer changes the ref it pins | diff --git a/README.md b/README.md index 6520110..6b57b1b 100644 --- a/README.md +++ b/README.md @@ -191,7 +191,6 @@ We're in the `0.x` development phase (pre-1.0.0). Reference the actions with: - **[docs/user/](./docs/user/README.md)** - User manual: what each action does, how to set it up, and every input and output - **[docs/CONTAINER-GUIDE.md](./docs/CONTAINER-GUIDE.md)** - Quick start with containers - **[docs/MIGRATION-GUIDE.md](./docs/MIGRATION-GUIDE.md)** - Migrating lecture repositories -- **[docs/QUICK-REFERENCE.md](./docs/QUICK-REFERENCE.md)** - Action reference - **[docs/dev/](./docs/dev/README.md)** - Developer docs: design, testing, container validation, the GPU AMI, and the work plan ## Getting Started diff --git a/build-jupyter-cache/README.md b/build-jupyter-cache/README.md index 4158ba8..0c18c66 100644 --- a/build-jupyter-cache/README.md +++ b/build-jupyter-cache/README.md @@ -1,232 +1,11 @@ -# Build Jupyter Cache Action + -Performs a fresh build of all lecture formats and saves to GitHub cache. Designed to run on the main branch (typically weekly) to generate the cache that PR workflows restore. +# Build Jupyter Cache -## Design Philosophy +Builds the lectures from scratch in each format you list, and saves the result as the cache that pull requests and publishing start from, but only if every build passes. A failed run keeps the last good cache and files an issue. -This action follows the **"build first, save only on success"** pattern: +How to set it up, and every input and output: [the `build-jupyter-cache` chapter of the user manual](../docs/user/actions/build-jupyter-cache.md). -1. Build all requested formats (jupyter, pdflatex, html) -2. Verify ALL builds succeeded -3. Only then save to cache -4. If any build fails: existing cache is preserved, issue is created +Every action: [the user manual](../docs/user/README.md). -This ensures PRs always have a working cache to restore, even when the weekly build encounters errors. - -## Inputs - -| Input | Description | Required | Default | -|-------|-------------|----------|---------| -| `builders` | Comma-separated builders: jupyter, pdflatex, html (whitespace also separates; an unknown name fails the run) | No | `html` | -| `environment` | Path to environment.yml (non-container builds) | No | `environment.yml` | -| `environment-update` | Path to delta environment.yml for container builds | No | `''` | -| `source-dir` | Source directory for lectures | No | `lectures` | -| `latex-requirements-file` | Path to the `latex-requirements.txt` passed to `setup-environment`. Used only in standard (non-container) mode when `pdflatex` is among the builders | No | `latex-requirements.txt` | -| `upload-artifact` | Upload `_build` as an artifact when a build fails (on success the cache already holds it) | No | `true` | -| `artifact-retention-days` | Days to retain artifact | No | `30` | -| `create-issue-on-failure` | Create GitHub issue on failure | No | `true` | -| `issue-assignees` | Comma-separated usernames for issue | No | `''` | -| `issue-labels` | Comma-separated labels for issue | No | `build-failure,automated` | -| `upload-failure-reports` | Pass through to the inner `build-lectures` calls so a failed build uploads its `reports/*.err.log` artifact. Defaults `true` here, unlike `build-lectures`' own `false` — this action runs unattended and the failure issue points at the report | No | `true` | - -## Outputs - -| Output | Description | -|--------|-------------| -| `cache-saved` | Whether a new cache was saved (`true` only if every requested build passed) | -| `build-success` | `true` only if every requested build passed, `false` for anything else — including a run that aborted during setup before any build started | -| `cache-key` | The cache key used | -| `jupyter-status` | Status of jupyter build (success/failure/skipped; empty if the run aborted before the builds) | -| `pdflatex-status` | Status of pdflatex build (success/failure/skipped; empty if the run aborted before the builds) | -| `html-status` | Status of html build (success/failure/skipped; empty if the run aborted before the builds) | -| `failure-issue-url` | URL of the failure issue filed or commented on (empty when there was no failure, or alerting is off) | - -## Cache Key Strategy - -**Save key:** `build-{hash(environment.yml)}-{hash(environment-update.yml)}-{run_id}` - -Each successful build creates a new cache entry. Old caches expire automatically after 7 days of no access (GitHub's default). - -> **Why no lecture-content hash?** The key is intentionally environment-only. `_build` is a -> warm-start baseline; freshness is handled by jupyter-cache (per-notebook, content-addressed), -> Sphinx incremental rebuilds, and this weekly cold rebuild. Adding a content hash would miss the -> cache on nearly every PR and force a cold rebuild for no correctness gain. See the *Cache Key -> Strategy* section of [restore-jupyter-cache](../restore-jupyter-cache/) for the full rationale. - -**Restore key pattern** (used by `restore-jupyter-cache`): -```yaml -restore-keys: | - build-{hash(environment.yml)}-{hash(environment-update.yml)}- - build-{hash(environment.yml)}- - build- -``` - -This prefix matching ensures PRs always get the most recently saved cache. - -## Usage - -### Basic Usage (in cache.yml workflow) - -```yaml -name: Build Cache -on: - schedule: - - cron: '0 0 * * 0' # Weekly Sunday midnight UTC - workflow_dispatch: - push: - branches: [main] - paths: ['environment.yml'] - -jobs: - cache: - runs-on: ubuntu-latest - container: - image: ghcr.io/quantecon/quantecon:latest - permissions: - contents: read - issues: write # required while create-issue-on-failure is true - packages: read # required to pull the container image - steps: - - uses: actions/checkout@v7 - - uses: quantecon/actions/build-jupyter-cache@v0 -``` - -> **`issues: write` is not optional** unless you set `create-issue-on-failure: false`. Without it -> the action now fails the job with an explicit error instead of skipping the alert — a cache -> build that fails unnoticed is the exact failure mode this alerting exists to prevent (#83). - -### All Builders with PDF - -```yaml -- uses: quantecon/actions/build-jupyter-cache@v0 - with: - builders: 'jupyter,pdflatex,html' -``` - -### Custom Issue Settings - -```yaml -- uses: quantecon/actions/build-jupyter-cache@v0 - with: - builders: 'jupyter,html' - create-issue-on-failure: true - issue-assignees: 'maintainer1,maintainer2' - issue-labels: 'build-failure,urgent' -``` - -### Without Issue Creation - -```yaml -- uses: quantecon/actions/build-jupyter-cache@v0 - with: - create-issue-on-failure: false -``` - -## Build Flow - -``` -┌─────────────────────────────────────────────────────────────┐ -│ 1. Setup Environment │ -│ └── Auto-detects container vs standard runner │ -├─────────────────────────────────────────────────────────────┤ -│ 2. Build All Formats (continue-on-error) │ -│ ├── jupyter (if in builders) │ -│ ├── pdflatex (if in builders) │ -│ └── html (if in builders, copies pdf/notebooks) │ -├─────────────────────────────────────────────────────────────┤ -│ 3. Verify Results │ -│ └── Check all requested builders succeeded │ -├─────────────────────────────────────────────────────────────┤ -│ 4a. ALL PASSED │ -│ ├── Save build cache │ -│ └── Save execution cache │ -├─────────────────────────────────────────────────────────────┤ -│ 4b. ANY FAILED │ -│ ├── DO NOT save cache (preserve existing) │ -│ ├── Upload artifact (for debugging) │ -│ ├── Create/update GitHub issue │ -│ └── Fail workflow │ -└─────────────────────────────────────────────────────────────┘ -``` - -## Failure Handling - -When a build fails: - -1. **Cache preserved** - Old cache remains for PRs -2. **Issue created** - Automatic GitHub issue with: - - Link to failed workflow run - - Table showing which builders failed - - Debug instructions -3. **Artifacts uploaded** - the per-builder `execution-reports-` artifacts holding the - `reports/*.err.log` tracebacks (`upload-failure-reports`, default `true`), plus the full - `_build` directory (`upload-artifact`, default `true`). The issue body names whichever were - actually produced instead of telling you to go hunting. -4. **Workflow fails** - Clear signal that action is needed -5. **Alert verified** - the action asserts an issue was really filed and fails loudly if not, so - alerting cannot silently no-op. The URL is exposed as the `failure-issue-url` output. - -### Duplicate Issue Prevention - -The action checks for existing open issues with the `build-failure` label: -- If found: Adds comment to existing issue -- If not found: Creates new issue - -This prevents issue spam from repeated failures. - -## Container vs Standard Runner - -The action works in both environments via `setup-environment`: - -### Container (Recommended) - -```yaml -jobs: - cache: - runs-on: ubuntu-latest - container: - image: ghcr.io/quantecon/quantecon:latest - steps: - - uses: actions/checkout@v7 - - uses: quantecon/actions/build-jupyter-cache@v0 -``` - -Benefits: -- Faster setup (LaTeX pre-installed) -- Consistent with PR builds -- Reliable (no network installs) - -### Standard Runner - -```yaml -jobs: - cache: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - uses: quantecon/actions/build-jupyter-cache@v0 - with: - builders: 'jupyter,pdflatex,html' -``` - -The `setup-environment` step auto-detects the environment and installs LaTeX if needed. - -## Build Summary - -The action generates a GitHub Actions summary showing: - -- Cache key used -- Builder results (✅/❌/⏭️) -- Build output sizes -- Container mode status - -## Workflow Templates - -See [templates/](../templates/) for complete workflow examples: -- `cache.yml` - Container-based cache generation workflow - -## Related Actions - -- **[restore-jupyter-cache](../restore-jupyter-cache/)** - Restore cache in PR workflows -- **[build-lectures](../build-lectures/)** - Individual build step (used internally) -- **[setup-environment](../setup-environment/)** - Environment setup (used internally) + diff --git a/build-jupyter-cache/action.yml b/build-jupyter-cache/action.yml index 274032a..25d23d6 100644 --- a/build-jupyter-cache/action.yml +++ b/build-jupyter-cache/action.yml @@ -1,22 +1,42 @@ name: 'Build Jupyter Cache' -description: 'Fresh build of all lecture formats and save to GitHub cache (runs on main branch, typically weekly)' +description: >- + Builds the lectures from scratch in each format you list, and saves the result as the cache that + pull requests and publishing start from, but only if every build passes. A failed run keeps the + last good cache and files an issue. author: 'QuantEcon' +# Each description below is copied verbatim into the user manual +# (docs/user/actions/build-jupyter-cache.md) by scripts/generate-docs.py, so it +# is written for a reader. "Container mode" and "standard mode" are +# setup-environment's. inputs: builders: - description: 'Comma-separated list of builders to run: jupyter, pdflatex, html (whitespace also separates; an unknown name fails the run)' + description: >- + The formats to build and cache: any of `jupyter`, `pdflatex` and `html`, separated by commas + or whitespace, as in `jupyter,pdflatex,html`. They run in that order whatever order they are + listed in, and the `html` build copies in the PDF and the notebooks when those are listed + too. An unknown name, or none at all, fails the run before anything is built. required: false default: 'html' environment: - description: 'Path to environment.yml for full environment (non-container builds)' + description: >- + Path to the Conda environment file. In standard mode `setup-environment` builds the + environment from it, and fails if it does not exist. In either mode its hash is part of the + build cache's key, so give `restore-jupyter-cache` the same file. required: false default: 'environment.yml' environment-update: - description: 'Path to delta environment.yml for container builds (empty = use pre-installed packages only)' + description: >- + Container mode: path to a Conda environment file of packages to add to the image's + environment, which `setup-environment` installs. Empty, the default, adds none. In either + mode its hash is part of the build cache's key, so give `restore-jupyter-cache` the same + file. required: false default: '' source-dir: - description: 'Source directory containing lectures' + description: >- + The book to build, as `build-lectures` takes it: the directory that holds its `_config.yml` + and `_toc.yml`. The execution cache's key hashes every `.md` file under it. required: false default: 'lectures' # The default repeats setup-environment's own, and must not be '': the runner @@ -24,67 +44,90 @@ inputs: # default, which would fail every standard-mode pdflatex build. latex-requirements-file: description: >- - Path to the latex-requirements.txt listing the apt packages to install, - passed through to setup-environment. Used only in standard (non-container) - mode when pdflatex is among the builders; the container images ship LaTeX. + Standard mode, with `pdflatex` among the builders: path to the list of apt packages that + `setup-environment` installs for LaTeX. The job fails if the file is missing or names no + package. Ignored in container mode, whose images carry LaTeX. required: false default: 'latex-requirements.txt' upload-artifact: - description: 'Upload the _build directory as an artifact when a build fails (for debugging; on success the cache already holds it). Set false to suppress even that.' + description: >- + `'true'`, the default, uploads the whole `_build` directory as the artifact + `build-cache-` when a build fails, since no cache is saved then. `'false'` uploads + nothing. A run whose builds all pass uploads no artifact either way: the cache holds + `_build`. required: false default: 'true' artifact-retention-days: - description: 'Number of days to retain the build artifact' + description: >- + How many days to keep the artifact that `upload-artifact` uploads, up to the repository's + retention limit. required: false default: '30' create-issue-on-failure: - description: 'Create GitHub issue when build fails' + description: >- + `'true'`, the default, files an issue when the run fails, or comments on the open one that + carries the first of `issue-labels`, and fails the job if neither worked. It needs + `issues: write`. `'false'` files nothing. required: false default: 'true' issue-assignees: - description: 'Comma-separated GitHub usernames to assign to failure issues' + description: >- + Comma-separated GitHub usernames to assign a new failure issue to. Empty, the default, + assigns nobody. required: false default: '' issue-labels: - description: 'Comma-separated labels for failure issues' + description: >- + Comma-separated labels for a new failure issue, created in the repository if they do not + exist. The first also finds an open failure issue: a later failure comments on it instead of + filing another. Empty files a new issue for every failure. required: false default: 'build-failure,automated' upload-failure-reports: description: >- - Pass upload-failure-reports through to the inner build-lectures calls, so a - failed build uploads its execution-report artifact. Defaults to true here, - unlike build-lectures' own default of false: that action is usually driven by - a human watching a PR, whereas this one runs unattended (typically weekly), - and the auto-filed failure issue tells the maintainer to go read a report. - Set to false if re-uploading reports on every failure is too costly. + `'true'`, the default, makes each failed build upload its execution reports as + `execution-reports-`, as the `build-lectures` input of the same name does, and the + failure issue names them. `'false'` uploads none. The default differs from + `build-lectures`' because this action runs unattended. required: false default: 'true' outputs: cache-saved: - description: 'Whether new cache was saved (only true if all builds passed)' + description: >- + `'true'` when every requested build passed, which is when the build and execution caches + are saved; `'false'` otherwise, including a run that stopped before any build. Always set. value: ${{ steps.status.outputs.all-passed }} build-success: description: >- - Whether the cache build succeeded. 'true' only if every requested build - passed; 'false' for anything else, including a run that aborted during - setup before any build started (it used to be empty in that case, which - no caller could distinguish from success — see #123). + The same value as `cache-saved`: `'true'` only when every requested build passed; `'false'` + for anything else, including a run that stopped during setup, before any build. Always + set. value: ${{ steps.status.outputs.all-passed }} cache-key: - description: 'The cache key used for saving' + description: >- + The key the build cache is saved under, `build---`. Set even when a build fails and nothing is saved. value: ${{ steps.cache-key.outputs.key }} jupyter-status: - description: 'Status of jupyter build (success/failure/skipped)' + description: >- + `success` or `failure` for a `jupyter` build that ran, `skipped` when `builders` does not + list it. Empty when the run stopped before the builds. value: ${{ steps.verify-builds.outputs.jupyter-status }} pdflatex-status: - description: 'Status of pdflatex build (success/failure/skipped)' + description: >- + `success` or `failure` for a `pdflatex` build that ran, `skipped` when `builders` does not + list it. Empty when the run stopped before the builds. value: ${{ steps.verify-builds.outputs.pdflatex-status }} html-status: - description: 'Status of html build (success/failure/skipped)' + description: >- + `success` or `failure` for an `html` build that ran, `skipped` when `builders` does not list + it. Empty when the run stopped before the builds. value: ${{ steps.verify-builds.outputs.html-status }} failure-issue-url: - description: 'URL of the failure issue filed or commented on (empty when no failure, or alerting disabled)' + description: >- + The URL of the failure issue filed or commented on. Empty when every build passed, or with + `create-issue-on-failure: 'false'`. value: ${{ steps.create-failure-issue.outputs.issue-url }} runs: diff --git a/build-lectures/README.md b/build-lectures/README.md index 44901ad..2e6a216 100644 --- a/build-lectures/README.md +++ b/build-lectures/README.md @@ -1,309 +1,11 @@ -# Build Lectures Action + -Builds QuantEcon lectures using Jupyter Book. +# Build Lectures -> **Note:** For caching, use the dedicated cache actions: -> - [`build-jupyter-cache`](../build-jupyter-cache) - Weekly cache generation on main branch -> - [`restore-jupyter-cache`](../restore-jupyter-cache) - Read-only restore for PR workflows +Builds the lectures with Jupyter Book, as the website, the PDF or notebooks. When the build fails it prints each failing notebook's traceback, and can upload the execution reports. -## Features +How to set it up, and every input and output: [the `build-lectures` chapter of the user manual](../docs/user/actions/build-lectures.md). -- 📚 **Multi-builder support** (HTML, PDF, Jupyter notebooks) -- 📦 **Asset assembly** - copy PDFs and notebooks into HTML build -- 🔍 **Execution reports** - upload reports on build failure -- ⚙️ **Configurable build options** via extra arguments -- 📊 **Build summary reporting** with artifact paths -- 🎯 **Output path detection** based on builder type +Every action: [the user manual](../docs/user/README.md). -## Inputs - -| Input | Description | Required | Default | -|-------|-------------|----------|---------| -| `builder` | Jupyter Book builder (html/pdflatex/jupyter) | No | `html` | -| `source-dir` | Directory containing lecture files | No | `lectures` | -| `output-dir` | Base output directory | No | `.` | -| `extra-args` | Extra jupyter-book build arguments (passed unquoted, so multiple flags word-split; quoted args with spaces are not supported) | No | `-W --keep-going` | -| `html-copy-pdf` | Copy PDFs to `_build/html/_pdf/` (HTML only) | No | `false` | -| `html-copy-notebooks` | Copy notebooks to `_build/html/_notebooks/` (HTML only) | No | `false` | -| `upload-failure-reports` | Upload execution reports on failure | No | `false` | -| `failure-artifact-name` | Custom name for the failure-report artifact | No | `''` (uses `execution-reports-{builder}`) | - -## Outputs - -| Output | Description | -|--------|-------------| -| `build-path` | Full path to build artifacts | - -## Usage - -### HTML Build (Default) - -```yaml -- uses: quantecon/actions/build-lectures@v0 -``` - -### PDF Build - -```yaml -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'pdflatex' -``` - -### Jupyter Notebook Build - -```yaml -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'jupyter' -``` - -### Custom Build Arguments - -```yaml -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'html' - extra-args: '-W --keep-going -v' -``` - -### Using Build Output - -```yaml -- uses: quantecon/actions/build-lectures@v0 - id: build - -- name: Upload artifacts - uses: actions/upload-artifact@v4 - with: - name: html-build - path: ${{ steps.build.outputs.build-path }} -``` - -### Multi-Format Build with Asset Assembly - -Build PDF and notebooks first, then HTML with asset assembly: - -```yaml -# Build PDF -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'pdflatex' - -# Build notebooks -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'jupyter' - -# Build HTML and assemble all assets -- uses: quantecon/actions/build-lectures@v0 - id: build - with: - builder: 'html' - html-copy-pdf: true - html-copy-notebooks: true -``` - -Result: -``` -_build/html/ -├── index.html -├── _pdf/ -│ └── quantecon-lectures.pdf -└── _notebooks/ - ├── intro.ipynb - └── ... -``` - -### Upload Reports on Build Failure - -```yaml -- uses: quantecon/actions/build-lectures@v0 - with: - upload-failure-reports: true -``` - -On failure, uploads: -- `_build/*/reports/` - Jupyter Book execution reports -- `_build/.jupyter_cache/` - Cache state for debugging - -Artifact name: `execution-reports-{builder}` - -## Caching - -For caching notebook execution and build outputs, use the dedicated cache actions: - -- **[`build-jupyter-cache`](../build-jupyter-cache)** - Weekly cache generation on main branch -- **[`restore-jupyter-cache`](../restore-jupyter-cache)** - Read-only restore for PR workflows - -See the [cache actions documentation](../build-jupyter-cache/README.md) for setup instructions. - -## Builder Types - -### HTML Builder - -**Command:** `jb build lectures --path-output ./ -W --keep-going` - -**Output:** `_build/html/` - -**Use for:** -- Website deployment -- Netlify previews -- GitHub Pages - -### PDF Builder - -**Command:** `jb build lectures --builder pdflatex --path-output ./ -n -W --keep-going` - -**Output:** `_build/latex/` - -**Requirements:** -- LaTeX packages: pre-installed in the QuantEcon containers; on a standard runner, use `setup-environment` with `install-latex: 'true'` -- ~30-45 minutes build time - -**Use for:** -- PDF downloads -- Print versions - -### Jupyter Builder - -**Command:** `jb build lectures --path-output ./ --builder=custom --custom-builder=jupyter -n -W --keep-going` - -**Output:** `_build/jupyter/` - -**Use for:** -- Downloadable notebooks -- Direct execution -- Binder integration - -## Build Arguments - -### Default Arguments - -`-W --keep-going` - -- `-W`: Treat warnings as errors -- `--keep-going`: meaningful only alongside `-W` — collect every warning-as-error instead of - stopping at the first, then exit non-zero at the end of the build. Sphinx ANDs the two flags - (`self.keep_going = warningiserror and keep_going`), so without `-W` this flag has no effect - at all. - -> **Warning:** keep `-W` in any `extra-args` you set. -> -> A code cell that raises is not an error to Jupyter Book. `myst-nb` logs the failed execution as -> a *warning* and carries on — `raise_on_error` under `execute:` in `_config.yml` (Sphinx: -> `nb_execution_raise_on_error`) defaults to `false`. `-W` is the only thing that turns that -> warning into a non-zero exit, and `extra-args` **replaces** the default rather than adding to it. -> -> So a build with `extra-args` set but `-W` omitted publishes the site, writes -> `_build//reports/.err.log`, and exits **0** — a green CI run over a broken -> lecture. -> -> Setting `raise_on_error: true` in `_config.yml` is an alternative, but a blunter one: myst-nb -> then raises instead of warning, so the build aborts at the first failing notebook and never -> writes the `reports/` tracebacks that `upload-failure-reports` collects. Prefer `-W`. - -### Common Extra Arguments - -**Verbose output:** -```yaml -extra-args: '-W --keep-going -v' -``` - -**Nitpick mode:** -```yaml -extra-args: '-W --keep-going -n' -``` - -**Quiet mode:** -```yaml -extra-args: '-W --keep-going -q' -``` - -**Fresh build (ignore cache):** -```yaml -extra-args: '-W --keep-going --all' -``` - -## Troubleshooting - -### Build Failures - -**Symptom:** Build fails with notebook execution error - -**Solutions:** -1. Check specific notebook in build logs -2. Run locally: `jb build lectures` -3. Re-run locally the way CI does — `jb build lectures -W --keep-going` — to see every error that - fails the build (`--keep-going` is inert without `-W`, since Sphinx only honours it when - warnings are already being treated as errors) -4. Enable `upload-failure-reports: true` for detailed reports - -### Missing Artifacts - -**Symptom:** `build-path` directory empty or missing - -**Solutions:** -1. Check builder output in logs -2. Verify source directory exists -3. Check for build errors in previous step - -### Slow Builds - -**Symptom:** Builds taking too long - -**Solutions:** -1. Use cache actions (`restore-jupyter-cache`) to restore previous builds -2. Consider splitting large notebooks -3. Check if notebooks are re-executing unnecessarily - -## Examples - -### Full CI Workflow - -```yaml -jobs: - build: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - - uses: quantecon/actions/setup-environment@v0 - with: - install-latex: 'true' - - - uses: quantecon/actions/build-lectures@v0 - id: build - - - uses: actions/upload-artifact@v4 - with: - name: html - path: ${{ steps.build.outputs.build-path }} -``` - -### Multi-Format Build - -```yaml -jobs: - build-html: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - uses: quantecon/actions/setup-environment@v0 - - uses: quantecon/actions/build-lectures@v0 - with: - builder: 'html' - - build-pdf: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - - uses: quantecon/actions/setup-environment@v0 - with: - install-latex: 'true' - - uses: quantecon/actions/build-lectures@v0 - with: - builder: 'pdflatex' -``` - -See [docs/MIGRATION-GUIDE.md](../docs/MIGRATION-GUIDE.md) for complete workflow examples. + diff --git a/build-lectures/action.yml b/build-lectures/action.yml index a3ef27a..ea632d3 100644 --- a/build-lectures/action.yml +++ b/build-lectures/action.yml @@ -1,48 +1,84 @@ name: 'Build Lectures' -description: 'Builds QuantEcon lectures using Jupyter Book' +description: >- + Builds the lectures with Jupyter Book, as the website, the PDF or notebooks. When the build + fails it prints each failing notebook's traceback, and can upload the execution reports. author: 'QuantEcon' -# NOTE: For caching, use the dedicated cache actions: -# - build-jupyter-cache: Weekly cache generation on main branch -# - restore-jupyter-cache: Read-only restore for PR workflows - +# Each description below is copied verbatim into the user manual +# (docs/user/actions/build-lectures.md) by scripts/generate-docs.py, so it is +# written for a reader. inputs: builder: - description: 'Jupyter Book builder to use (html, pdflatex, custom --custom-builder=jupyter)' + description: >- + What to build: `html`, the default, builds the website into `/_build/html`; + `pdflatex` builds the PDF into `/_build/latex`, and needs LaTeX; `jupyter` + builds notebooks into `/_build/jupyter`. Any other value is passed to `jb build` + as `--builder `. required: false default: 'html' source-dir: - description: 'Source directory containing lectures' + description: >- + The book to build: the directory that holds its `_config.yml` and `_toc.yml`, relative to + the workspace. required: false default: 'lectures' output-dir: - description: 'Output directory for build artifacts' + description: >- + The directory the build is written under, passed to `jb build` as `--path-output`, so the + output lands in `/_build`. The cache actions read and write `_build` at the + workspace root, so keep the default, `.`, in a job that uses them. required: false default: '.' extra-args: - description: 'Extra arguments to pass to jupyter-book build. Passed unquoted so multiple flags word-split (e.g. "-W --keep-going"); quoted arguments containing spaces are not supported.' + description: >- + Further arguments for `jb build`. They are split on whitespace, so a quoted value that + contains a space cannot be passed. Setting this replaces the default, so keep `-W` in it: + without `-W`, a notebook that raises an exception is only a warning, and the build + succeeds. required: false default: '-W --keep-going' html-copy-pdf: - description: 'Stage PDF from _build/latex/ into _build/html/_pdf/ before HTML build so theme can detect and enable downloads (requires prior pdflatex build)' + description: >- + With `builder: html`: `'true'` copies each PDF under `/_build/latex` into + `/_build/html/_pdf/` before the build, so the site offers it for download. The + PDF comes from a `pdflatex` build earlier in the job, or from a restored build cache; if + that directory does not exist, a warning says so and the site is built without it. Ignored + by the other builders. required: false default: 'false' html-copy-notebooks: - description: 'Stage notebooks from _build/jupyter/ into _build/html/_notebooks/ before HTML build so theme can detect and enable downloads (requires prior jupyter build)' + description: >- + With `builder: html`: `'true'` copies each notebook under `/_build/jupyter` into + `/_build/html/_notebooks/` before the build, so the site offers them for + download. The notebooks come from a `jupyter` build earlier in the job, or from a restored + build cache; if that directory does not exist, a warning says so and the site is built + without them. Ignored by the other builders. required: false default: 'false' upload-failure-reports: - description: 'Upload execution reports as artifacts when build fails' + description: >- + `'true'` uploads an artifact when the build fails, holding the `reports` directories in + `_build/html`, `_build/latex` and `_build/jupyter`, and `_build/.jupyter_cache`, all under + `output-dir`, kept for 7 days. `'false'`, the default, uploads nothing. For the `html`, + `pdflatex` and `jupyter` builders the log shows each failing notebook's traceback either + way. required: false default: 'false' failure-artifact-name: - description: 'Custom name for failure report artifact (default: execution-reports-{builder})' + description: >- + The name of the artifact that `upload-failure-reports` uploads. Empty, the default, names it + `execution-reports-`. A name can be used only once in a workflow run, so set this + when more than one job in a run builds with the same builder. required: false default: '' outputs: build-path: - description: 'Path to the build output directory' + description: >- + The directory the builder wrote to: `/_build/html`, `_build/latex` or + `_build/jupyter` for those three builders, and `/_build` for any other. With the + default `output-dir` it starts with `./`, as in `./_build/html`. Set whether or not the + build succeeds. value: ${{ steps.build.outputs.build-path }} runs: diff --git a/deploy-cloudflare/README.md b/deploy-cloudflare/README.md index 613a67c..2a82050 100644 --- a/deploy-cloudflare/README.md +++ b/deploy-cloudflare/README.md @@ -1,202 +1,11 @@ -# Deploy Cloudflare Action + -Publishes a built site to an existing Cloudflare Worker behind Cloudflare Access, for sites only members should see, and fails the job unless the site is proven gated. +# Deploy Cloudflare -GitHub Pages stays the route for everything public ([`publish-gh-pages`](../publish-gh-pages)). What Pages cannot do on the org's Team plan is serve a site to members only, since Pages access control needs Enterprise Cloud. Cloudflare Access in front of a Worker's static assets does, for free at this scale. PR previews are a different action ([`preview-cloudflare`](../preview-cloudflare)). +Publishes a built site to an existing Cloudflare Worker behind Cloudflare Access, for a site only members may see, and fails unless the site is proven gated before and after the deploy. -## Features +How to set it up, and every input and output: [the `deploy-cloudflare` chapter of the user manual](../docs/user/actions/deploy-cloudflare.md). -- 🔒 **Proves the gate before uploading.** If an anonymous request to the Worker is not redirected to your Access team's login, nothing is uploaded. That refuses a Worker that does not exist, one that is public, and one gated by the wrong Access organisation. -- 🔁 **Proves it again after deploying**, on production and on the new version's own preview URL, each at the site root and at one real file from the build. A site that is silently public fails the job instead of passing it. -- 🗂️ **Optional preview alias**, for example `report-2026-08`: a permanent URL for this build alongside the moving production URL. The alias is gate-checked too. -- 📌 **Pinned wrangler**, installed with `npm ci` from a committed lockfile and kept current by Dependabot. -- 📝 **Job summary.** On `push`, `schedule` and `workflow_dispatch` there is no PR to comment on, so the result (including "refused" or "deployed but not gated") goes to the job summary. +Every action: [the user manual](../docs/user/README.md). -## Usage - -```yaml -name: Publish members dashboard -on: - push: - branches: [main] - schedule: - - cron: '0 6 * * *' - workflow_dispatch: - -# One deploy at a time per site; a queued run deploys the newest build. -concurrency: - group: deploy-cloudflare-${{ github.workflow }} - cancel-in-progress: false - -jobs: - deploy: - runs-on: ubuntu-latest - permissions: - contents: read - steps: - - uses: actions/checkout@v7 - - - name: Build - run: ./build.sh # writes the site to _site/ - - - uses: quantecon/actions/deploy-cloudflare@v0 - with: - cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} - cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} - worker-name: members-dashboard - account-subdomain: my-subdomain # *.my-subdomain.workers.dev - team-domain: my-team.cloudflareaccess.com - build-dir: _site -``` - -A monthly report that also keeps a permanent URL per month: - -```yaml - - name: Name this month's alias - id: month - run: echo "alias=report-$(date -u +%Y-%m)" >> "$GITHUB_OUTPUT" - - - uses: quantecon/actions/deploy-cloudflare@v0 - with: - cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} - cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} - worker-name: monthly-report - account-subdomain: my-subdomain - team-domain: my-team.cloudflareaccess.com - build-dir: _build/html - alias: ${{ steps.month.outputs.alias }} # report-2026-08 -``` - -> **New in v0.12.0.** Pin `@v0.12.0` or later; `@v0` carries it once that release has moved the floating tag. - -## Requirements - -- **A Worker that already exists and is already behind Access.** The action never creates a Worker or turns Access on. See the [setup checklist](#setup-checklist). -- **Node.js 22 or later.** wrangler refuses to run below 22. If the runner has an older Node or none, the action sets up Node 24 itself. The QuantEcon containers (`ghcr.io/quantecon/quantecon`, `ghcr.io/quantecon/quantecon-build`) already carry Node 24. -- **`bash`, `curl` and `npm`**, all present on GitHub-hosted runners and in the QuantEcon containers. -- **Secrets** `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID`. Secrets are not passed to workflows triggered from forks or by Dependabot. The action then fails in validation, before any network call. - -## Inputs - -| Input | Description | Required | Default | -|-------|-------------|----------|---------| -| `cloudflare-api-token` | Account-owned token with **Editor on this Worker only** (see the checklist) | Yes | - | -| `cloudflare-account-id` | Cloudflare account ID | Yes | - | -| `worker-name` | The Worker to deploy to, one per site. Lowercase letters, digits and dashes | Yes | - | -| `account-subdomain` | The account's `workers.dev` subdomain: `my-subdomain` for `*.my-subdomain.workers.dev` (`my-subdomain.workers.dev` is accepted too) | Yes | - | -| `team-domain` | The Access team's login domain, `.cloudflareaccess.com` (a bare `` is accepted) | Yes | - | -| `build-dir` | Directory with the built site | Yes | - | -| `alias` | Also upload this build as a named preview alias. Lowercase letters, digits and dashes, **starting with a letter** (`report-2026-08`, not `2026-08`), and `-` must fit in 63 characters | No | `''` | -| `require-access` | Check the gate before uploading, after deploying (production and the new version's preview URL) and on the alias. `false` skips every check, with a warning. Never set it for private content | No | `true` | - -The output URLs are **constructed** from `worker-name`, `account-subdomain` and `alias`, never parsed from wrangler's output, following the same approach as `preview-cloudflare` (#131). The one address read from wrangler's output is the new version's preview URL, which is only probed and never handed out. - -## Outputs - -| Output | Description | -|--------|-------------| -| `deploy-url` | `https://{worker}.{subdomain}.workers.dev`. Set once the production deploy succeeds (even if the gate check after it then fails the job) | -| `alias-url` | `https://{alias}-{worker}.{subdomain}.workers.dev`. Empty when no alias is given or the upload did not happen | - -## What the gate check proves - -The check is [`scripts/check-access-gate.sh`](../scripts/check-access-gate.sh), taken from the probe verified in the status-projects pilot (QuantEcon/status-projects#35). It sends an unauthenticated request and **does not follow redirects**: following one would land on the login page and return `200`, making a gated site and a public one look the same. - -| Response | Verdict | -|----------|---------| -| `301/302/303/307/308` to exactly `team-domain` | ✅ gated | -| Redirect to a different `*.cloudflareaccess.com` | ❌ gated by the **wrong Access organisation** | -| Redirect whose `Location` carries userinfo or a backslash | ❌ the real host is ambiguous, and Access never sends one | -| `2xx` | ❌ **the site is public** | -| `404` | ❌ no Worker answers on that hostname, or its `workers.dev` route is off | -| Anything else, or unreachable | ❌ the gate could not be verified. The check fails closed | - -Every check probes the site root **and one non-HTML file from `build-dir`** (the first in sorted order with a URL-safe path, skipping dotfiles). A gate that protects only the entry point would be an easy mistake to make and invisible from the root. In the pilot, `data/latest.json` got the same redirect as `/`, as it should: Access covers every path on the hostname. - -The action runs the check at three points: - -1. **Before uploading.** This is prevention. Deploying content first and turning Access on afterwards would leave private data on a public hostname for the length of the setup. -2. **After deploying**, on production and on the new version's own preview URL. For production this is a regression guard: the pilot showed Access survives a redeploy, but the check is one request, and it guards against a private site silently going public. The preview URL is checked for the first time here. Every deploy gets one, `https://-{worker}.{subdomain}.workers.dev`, because the config enables preview URLs, and an Access setup that covers only the production hostname would leave it public. Its address comes from the `Current Version ID` line in wrangler's output; if that line is missing, the check fails rather than being skipped. The check also runs when `wrangler deploy` fails, because wrangler can exit with an error after the new version is already live. -3. **After the alias upload.** The alias is one more preview URL, so it gets its own check. - -You can run the same check by hand: - -```bash -bash scripts/check-access-gate.sh my-team.cloudflareaccess.com \ - https://members-dashboard.my-subdomain.workers.dev/ -# PASS 302 -> my-team.cloudflareaccess.com https://members-dashboard.my-subdomain.workers.dev/ -``` - -## Setup checklist - -### Once per Cloudflare account - -1. **Zero Trust organisation.** The team name you choose gives the login domain `.cloudflareaccess.com`, which is the `team-domain` input. The free tier covers 50 users. -2. **GitHub as the login method.** Create an OAuth App under the **organisation's** developer settings (GitHub → the org → Settings → Developer settings → OAuth Apps). An app created under the organisation itself needs no approval under the org's OAuth app access restrictions, while one under a personal account does. - - Homepage URL: `https://.cloudflareaccess.com` - - Authorization callback URL: `https://.cloudflareaccess.com/cdn-cgi/access/callback` - - Then add it in Zero Trust under Settings → Authentication → Login methods → GitHub. -3. **Make GitHub the only login method.** New Zero Trust accounts default to *Cloudflare account membership*, not one-time PIN. Remove that default. -4. **One reusable policy per audience.** Under Access controls → Policies, create an Allow policy with the **GitHub Organization** selector, the organisation `QuantEcon` and a **named team**, and reference it from each application. Use a team, not the whole organisation: org membership counted 82 accounts across 23 teams, including translation collaborators and course teams, which is the wrong audience for a grants dashboard. The pilot's policy is `QuantEcon Dashboards` (`QuantEcon` + team `dashboards`). Sessions can last up to one month. - -### Once per site - -1. **Create the Worker by hand, under its final name**, as a placeholder that holds no data (for example the dashboard's Hello World template). Keep its `workers.dev` route enabled: the action deploys to and checks `https://{worker}.{subdomain}.workers.dev`. Creating a Worker needs Admin at the Workers product scope, which the deploy token deliberately lacks ([Cloudflare docs](https://developers.cloudflare.com/workers/authorization/)). -2. **Turn Access on for the Worker with *All traffic*,** not *Previews only*. This protects the `workers.dev` hostname, every preview URL (so every alias), and any custom domain attached later. Attach the reusable policy and turn on instant authentication. Use this per-Worker setting rather than the account-wide "Protect all Workers" switch, because the same account hosts public lecture previews. -3. **Prove the gate on the placeholder** before the first real deploy: - ```bash - bash scripts/check-access-gate.sh .cloudflareaccess.com https://..workers.dev/ - ``` - The action repeats this check before every deploy and refuses if it fails, but proving it here first catches a setup mistake before any workflow depends on it. -4. **Create the deploy token:** an account-owned API token with **Editor on this one Worker** (Cloudflare's per-Worker roles), not the legacy account-wide *Workers Scripts: Edit*. It can deploy this Worker but cannot create one, so a mistyped `worker-name` fails instead of creating a new, ungated Worker. The gate check before uploading refuses that case too, whatever the token can do. -5. **Add the secrets to the consumer repository** (Settings → Secrets and variables → Actions): - - | Secret | Value | - |--------|-------| - | `CLOUDFLARE_API_TOKEN` | The per-Worker token from step 4 | - | `CLOUDFLARE_ACCOUNT_ID` | Dashboard URL `https://dash.cloudflare.com/{account-id}/...`, or Workers & Pages → Overview | - -### Why the action does not do this setup itself - -The Access application is one-time per Worker, and the identity provider is one-time per account. Provisioning either from CI would put an `Access: Apps and Policies: Edit` token into every consumer repository for a step that runs once. For the same reason the action cannot create the Worker: its token can only deploy to one that exists. If the org ever wants the account under code, the Terraform resources `zero_trust_access_application` and `zero_trust_access_policy` cover it. - -## Notes - -- **Custom domains** are a per-Worker setting in the dashboard, and the Worker's Access application picks them up automatically. The action does not manage domains: its generated config declares no routes, so a deploy leaves dashboard-attached domains alone, and it always checks the `workers.dev` URL. -- **The generated config** ([`write-config.js`](write-config.js)) sets the name, a fixed `compatibility_date`, `assets.directory`, `workers_dev: true` and `preview_urls: true`. It is written to `RUNNER_TEMP` for each run, so consumer repositories need no wrangler config. `preview_urls: true` turns preview URLs on for the Worker at every deploy, even if they were turned off in the dashboard; the check after deploying covers the new version's preview URL for that reason. wrangler running in CI overwrites settings changed in the dashboard (such as the placeholder's script) without prompting. Dotfiles in `build-dir` (Sphinx's `.buildinfo`, for example) are uploaded like any other file unless listed in a `.assetsignore`. -- **A blank page after access is granted.** On the first load after access is granted or restored, cached Access redirects for `style.css`, `app.js` and data files can be served in place of the real assets. A plain refresh fixes it. -- **A user added to the team after being denied** may stay denied until they revoke the OAuth app in their GitHub settings and log in again. This is documented Cloudflare behaviour, though it did not reproduce on the path the pilot tested. -- **Limits.** Workers static assets allow 20,000 files and 25 MiB per file per version on the free plan (100,000 files on paid). The 1,000 most recent preview aliases are kept per Worker. Lecture-sized Jupyter Books are a few thousand files. - -## Troubleshooting - -### "Refusing to deploy: … is not behind Access" - -Nothing was uploaded. The line above it says why: -- **`answered 404`**: no Worker by that name under that `account-subdomain`, or its `workers.dev` route is off. Check both, and create the Worker first if it is new. -- **`THE SITE IS PUBLIC`**: Access is off for the Worker, or set to *Previews only*. Turn it on with *All traffic*. -- **`wrong Access organisation`**: the Worker's Access application belongs to a different Zero Trust team, or `team-domain` is wrong. -- **`not to the Access login domain`**: the site redirected somewhere other than `team-domain`, or sent no `Location`, so Access is not answering for this hostname. Check that Access is on with *All traffic* and that `team-domain` is right. -- **`expected a redirect to the Access login domain`**: an unexpected status such as `401`, `403` or `5xx`. Check the Worker in the dashboard, then re-run the job. -- **`could not be reached`**: a network problem between the runner and Cloudflare. Re-run the job. - -### "wrangler deploy failed" - -wrangler's own error is printed above the annotation. An authentication or permission error usually means the token lacks Editor on this Worker, the account ID is wrong, or the token has expired. wrangler can fail after the new version is already live, so the job summary reports the gate check that ran after the failure. - -### "A hostname serving this Worker is NOT behind Access" - -This is the urgent one: the new build is live, or may be, and anonymous requests to production or to the new version's preview URL are not being redirected. The `FAIL` line above it names the hostname. Turn Access on for the Worker with *All traffic*, which covers production and every preview URL, or disable its `workers.dev` route. Then re-run the job to confirm. - -### "wrangler's output named no 'Current Version ID'" - -The new version's preview URL could not be worked out, so it was not checked, and the check fails. wrangler prints the ID only once every step after the upload has succeeded, so this follows most `wrangler deploy` failures; after a successful deploy it means a wrangler upgrade changed its output. Check the preview URL by hand with `scripts/check-access-gate.sh`, using the version ID from the dashboard. - -### "The alias is uploaded but … is NOT behind Access" - -Production is gated but previews are not. Set the Worker's Access to *All traffic*, which covers preview URLs. - -### An alias is rejected before anything runs - -Aliases must start with a lowercase letter and use only lowercase letters, digits and dashes, and `-` must fit in 63 characters. Use `report-2026-08`, not `2026-08`. + diff --git a/deploy-cloudflare/action.yml b/deploy-cloudflare/action.yml index 0425b11..b80b62a 100644 --- a/deploy-cloudflare/action.yml +++ b/deploy-cloudflare/action.yml @@ -1,61 +1,74 @@ name: 'Deploy Cloudflare' -description: 'Publishes a built site to an existing Cloudflare Worker behind Cloudflare Access, and fails unless the site is proven gated before and after the deploy' +description: >- + Publishes a built site to an existing Cloudflare Worker behind Cloudflare Access, for a site only + members may see, and fails unless the site is proven gated before and after the deploy. author: 'QuantEcon' +# Each description below is copied verbatim into the user manual +# (docs/user/actions/deploy-cloudflare.md) by scripts/generate-docs.py, so it +# is written for a reader. The repository is public: no account details here. inputs: cloudflare-api-token: description: >- - Cloudflare API token (use secrets.CLOUDFLARE_API_TOKEN). Account-owned, - with Editor on the named Worker only: it can deploy that Worker but - cannot create one, so a mistyped worker-name fails instead of creating a - new, ungated Worker. + An account-owned Cloudflare API token with Editor on this one Worker, from a repository + secret such as `secrets.CLOUDFLARE_API_TOKEN`. It can deploy the Worker but not create one, + so a mistyped `worker-name` fails instead of creating a new Worker that nothing gates. The + job fails if the token is empty, as it is on runs from forks and by Dependabot. required: true cloudflare-account-id: - description: 'Cloudflare account ID (use secrets.CLOUDFLARE_ACCOUNT_ID)' + description: >- + The ID of the Cloudflare account that owns the Worker, from a repository secret such as + `secrets.CLOUDFLARE_ACCOUNT_ID`. The job fails if it is empty. required: true worker-name: description: >- - The Worker to deploy to, one per site. It must already exist and already - be behind Access; see the README setup checklist. + The Worker to deploy to, one per site: lowercase letters, digits and dashes, at most 63 + characters. It must already exist, with its `workers.dev` route on, and already be behind + Access. required: true account-subdomain: description: >- - The account's workers.dev subdomain, e.g. `quantecon` for - `*.quantecon.workers.dev`. The URLs are constructed from it, never parsed - from wrangler output. + The account's `workers.dev` subdomain: `my-subdomain` for `*.my-subdomain.workers.dev`, which + `my-subdomain.workers.dev` also gives. The URLs are built from it, never read from + wrangler's output. required: true team-domain: description: >- - The Access team's login domain, `.cloudflareaccess.com` (a bare - `` is accepted). The gate check requires the redirect to land on - exactly this host. + The Access team's login domain, `.cloudflareaccess.com`; a bare `` is accepted + too. Every gate check requires an anonymous request to be redirected to exactly this host. required: true build-dir: - description: 'Directory containing the built site' + description: >- + The directory that holds the built site. The job fails if it is missing or holds no files, + and warns if it has no `index.html`, since the site's root would then answer 404. required: true alias: description: >- - Also upload the same build as a named preview alias, e.g. `report-2026-08`, - for a permanent URL alongside the moving production one. Lowercase - letters, digits and dashes, starting with a letter; `-` - must fit in 63 characters. Empty for none. + Also uploads the same build as a named preview alias, such as `report-2026-08`, for a + permanent URL beside the production one, which moves with each deploy. Lowercase letters, + digits and dashes, starting with a letter and not ending with a dash, and + `-` must fit in 63 characters. Empty, the default, uploads no alias. required: false default: '' require-access: description: >- - Prove the site is gated: before uploading, after deploying (production - and the new version's preview URL) and on the alias after its upload, an - unauthenticated request must be redirected to team-domain. `false` skips - every one of these checks and should never be used for private content. + `'true'`, the default, proves the site is gated: before anything is uploaded, after the + deploy, on production and on the new version's own preview URL, and on the alias after its + upload, an anonymous request must be redirected to `team-domain`. `'false'` skips every + check, with a warning: never use it for private content. required: false default: 'true' outputs: deploy-url: - description: 'Production URL, https://{worker}.{subdomain}.workers.dev. Set once the deploy succeeds.' + description: >- + The production URL, `https://..workers.dev`. Set once the + deploy succeeds, even if the gate check after it then fails the job. value: ${{ steps.deploy.outputs.deploy-url }} alias-url: - description: 'Preview alias URL, https://{alias}-{worker}.{subdomain}.workers.dev. Empty when no alias is given.' + description: >- + The alias's URL, `https://-..workers.dev`. Empty when + no `alias` is given, or when its upload failed or did not run. value: ${{ steps.alias.outputs.alias-url }} runs: @@ -153,7 +166,7 @@ runs: # Prevention, not detection. Before anything is uploaded, the Worker must # already answer an anonymous request with a redirect to this team's login. # That refuses a Worker that does not exist (a mistyped name, which a - # broader token than the README's would otherwise create, ungated), one + # broader token than the manual's would otherwise create, ungated), one # that is public, and one gated by the wrong Access organisation. - name: Check the Access gate before deploying id: gate-before @@ -168,7 +181,7 @@ runs: urls=("$URL/") [ -z "$PROBE_PATH" ] || urls+=("$URL$PROBE_PATH") if ! bash "$GITHUB_ACTION_PATH/../scripts/check-access-gate.sh" "$TEAM" "${urls[@]}"; then - echo "::error::Refusing to deploy: $URL is not behind Access for $TEAM. Nothing was uploaded. An admin must create the Worker and turn Access on (All traffic) before its first deploy; see the deploy-cloudflare README setup checklist." + echo "::error::Refusing to deploy: $URL is not behind Access for $TEAM. Nothing was uploaded. An admin must create the Worker and turn Access on (All traffic) before its first deploy; see the setup checklist at https://github.com/QuantEcon/actions/blob/main/docs/user/actions/deploy-cloudflare.md#setup-checklist" exit 1 fi diff --git a/docs/QUICK-REFERENCE.md b/docs/QUICK-REFERENCE.md deleted file mode 100644 index ad0a824..0000000 --- a/docs/QUICK-REFERENCE.md +++ /dev/null @@ -1,395 +0,0 @@ -# QuantEcon Actions - Quick Reference - -A cheat sheet for using QuantEcon composite actions in your workflows. - -## 📦 Available Actions - -| Action | Purpose | Time Savings | -|--------|---------|--------------| -| `setup-environment` | Conda + Python + LaTeX | ~5-6 min (cached) | -| `build-lectures` | Jupyter Book builds | Varies (cached execution) | -| `build-jupyter-cache` | Weekly cache generation (main branch) | Enables 80% faster CI | -| `restore-jupyter-cache` | Cache restore for PRs (read-only by default; optional `save-cache`) | ~14 min (avoids full rebuild) | -| `preview-netlify` | PR preview deployment (Netlify) | ~1 min | -| `preview-cloudflare` | PR preview deployment (Cloudflare) | ~1 min | -| `deploy-cloudflare` | Members-only site on a Cloudflare Worker behind Access, gate-checked | ~1 min | -| `publish-gh-pages` | GitHub Pages deployment | ~30 sec | - -## 🚀 Quick Start - -### Container CI Workflow (Recommended - Fastest) - -Two container options: -- `ghcr.io/quantecon/quantecon:latest` (3.33 GB compressed pull, 8.60 GB on disk) - Full Anaconda, max compatibility -- `ghcr.io/quantecon/quantecon-build:latest` (2.93 GB compressed pull, 7.32 GB on disk) - Lean: no Anaconda metapackage, a modestly smaller pull - -```yaml -name: CI -on: [pull_request] - -jobs: - build: - runs-on: ubuntu-latest - container: - image: ghcr.io/quantecon/quantecon-build:latest # Lean container for CI - permissions: - contents: read - pull-requests: write # preview-netlify's PR comment - packages: read - steps: - - uses: actions/checkout@v7 - with: - fetch-depth: 0 - - uses: quantecon/actions/setup-environment@v0 - with: - environment-update: 'environment-update.yml' # Optional - delta packages for container - # Auto-detects container, installs only lecture-specific packages - - uses: quantecon/actions/restore-jupyter-cache@v0 - with: - cache-type: 'build' - - uses: quantecon/actions/build-lectures@v0 - id: build - - uses: quantecon/actions/preview-netlify@v0 - with: - netlify-auth-token: ${{ secrets.NETLIFY_AUTH_TOKEN }} - netlify-site-id: ${{ secrets.NETLIFY_SITE_ID }} - build-dir: ${{ steps.build.outputs.build-path }} -``` - -### Standard CI Workflow (No Container) - -```yaml -name: CI -on: [pull_request] - -jobs: - build: - runs-on: ubuntu-latest - permissions: - contents: read - pull-requests: write # preview-netlify's PR comment - steps: - - uses: actions/checkout@v7 - with: - fetch-depth: 0 - - uses: quantecon/actions/setup-environment@v0 - with: - install-latex: 'true' - - uses: quantecon/actions/build-lectures@v0 - id: build - - uses: quantecon/actions/preview-netlify@v0 - with: - netlify-auth-token: ${{ secrets.NETLIFY_AUTH_TOKEN }} - netlify-site-id: ${{ secrets.NETLIFY_SITE_ID }} - build-dir: ${{ steps.build.outputs.build-path }} -``` - -### Minimal Publish Workflow - -```yaml -name: Publish -on: - push: - tags: ['publish-*'] - -permissions: - contents: read - pages: write - id-token: write - -concurrency: - group: "pages" - cancel-in-progress: false - -jobs: - publish: - runs-on: ubuntu-latest - environment: - name: github-pages - url: ${{ steps.deploy.outputs.page-url }} - steps: - - uses: actions/checkout@v7 - - uses: quantecon/actions/setup-environment@v0 - with: - install-latex: 'true' - - uses: quantecon/actions/build-lectures@v0 - id: build - - uses: quantecon/actions/publish-gh-pages@v0 - id: deploy - with: - build-dir: ${{ steps.build.outputs.build-path }} -``` - -Set a custom domain in **Settings → Pages**: this deploy ignores a CNAME file, so the `cname` input has no effect. - -## 🔧 Common Customizations - -### Build PDF - -```yaml -- uses: quantecon/actions/setup-environment@v0 - with: - install-latex: 'true' - -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'pdflatex' -``` - -### Build Jupyter Notebooks - -```yaml -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'jupyter' -``` - -### Fast PR Builds (with Execution Cache) - -Add `restore-jupyter-cache` before `build-lectures` to restore cached execution state. The restore is **read-only** by default; set `save-cache: true` to also save an updated cache at job end, scoped to the PR branch (speeds up later runs on the same PR): - -```yaml -- uses: quantecon/actions/restore-jupyter-cache@v0 - with: - cache-type: 'build' - -- uses: quantecon/actions/build-lectures@v0 - id: build -``` - -**Note:** Requires a `cache.yml` workflow to generate the cache. See [MIGRATION-GUIDE.md](MIGRATION-GUIDE.md#step-5-update-cacheyml). - -### Preview URL - -`preview-netlify` has no alias input: it always deploys to the `pr-{number}` alias, so a PR keeps one preview URL across pushes. Read it from the `deploy-url` output. - -### Force Cache Rebuild - -```yaml -- uses: quantecon/actions/setup-environment@v0 - with: - cache-version: 'v2' # Bump from v1 -``` - -## 💾 Cache Keys Reference - -| Action | Cache Key | Invalidates On | -|--------|-----------|----------------| -| `setup-environment` (container) | No caching | N/A | -| `setup-environment` (standard) | `conda-{OS}-{env-name}-{cache-version}-py{python-version}-{hash(env.yml)}`, path `$CONDA/envs/{env-name}` | env.yml, env name or Python version changes, manual bump | -| `build-jupyter-cache` | `build-{hash(env.yml)}-{hash(env-update.yml)}-{run-id}` | env file changes, each run | -| `restore-jupyter-cache` | `build-{hash(env.yml)}-{hash(env-update.yml)}-` (prefix) | env file changes | - -## 🎯 Inputs Quick Reference - -### setup-environment - -```yaml -python-version: '3.13' # Python version (ignored in container mode) -environment: 'environment.yml' # Conda env file (non-container mode) -environment-update: '' # Delta env file for container mode (empty = skip) -environment-name: 'quantecon' # Conda env name -cache-version: 'v1' # Manual cache control -install-latex: 'false' # Install LaTeX (auto-disabled in container) -latex-requirements-file: 'latex-requirements.txt' # LaTeX packages list -``` - -**Outputs:** `container-mode`, `conda-cache-hit` - -### build-lectures - -```yaml -builder: 'html' # html|pdflatex|jupyter -source-dir: 'lectures' # Source directory -output-dir: '.' # Output base -extra-args: '-W --keep-going' # JB arguments -html-copy-pdf: 'false' # Copy PDFs to _build/html/_pdf/ -html-copy-notebooks: 'false' # Copy notebooks to _build/html/_notebooks/ -upload-failure-reports: 'false' # Upload reports on failure -failure-artifact-name: '' # Custom name for the failure-report artifact -``` - -**Note:** Caching is handled separately via `build-jupyter-cache` and `restore-jupyter-cache`. - -### preview-netlify - -```yaml -netlify-auth-token: ${{ secrets.NETLIFY_AUTH_TOKEN }} # Required -netlify-site-id: ${{ secrets.NETLIFY_SITE_ID }} # Required -build-dir: '_build/html' # Required -lectures-dir: 'lectures' # For change detection (default) -``` - -### preview-cloudflare - -```yaml -cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} # Required -cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} # Required -project-name: 'my-lectures' # Required - Cloudflare Pages project name -build-dir: '_build/html' # Required -lectures-dir: 'lectures' # For change detection (default) -``` - -### deploy-cloudflare - -```yaml -cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} # Required - Editor on this Worker only -cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} # Required -worker-name: 'members-dashboard' # Required - must already exist and be behind Access -account-subdomain: 'my-subdomain' # Required - *.my-subdomain.workers.dev -team-domain: 'my-team.cloudflareaccess.com' # Required - the gate must redirect here -build-dir: '_site' # Required -alias: '' # Optional permanent preview alias, e.g. report-2026-08 -require-access: 'true' # Gate check before and after deploying (default) -``` - -### publish-gh-pages - -```yaml -build-dir: '_build/html' # Required -cname: '' # No effect on this deploy: set a custom domain in Settings → Pages -``` - -**Note:** Uses native GitHub Pages deployment. Requires workflow permissions: -```yaml -permissions: - pages: write - id-token: write -``` - -## 📊 Outputs Quick Reference - -### build-lectures - -```yaml -- id: build - uses: quantecon/actions/build-lectures@v0 - -# Access: ${{ steps.build.outputs.build-path }} -``` - -### preview-netlify - -```yaml -- id: netlify - uses: quantecon/actions/preview-netlify@v0 - -# Access: -# - ${{ steps.netlify.outputs.deploy-url }} -# - ${{ steps.netlify.outputs.changed-files }} -``` - -### deploy-cloudflare - -```yaml -- id: private - uses: quantecon/actions/deploy-cloudflare@v0 - -# Access: -# - ${{ steps.private.outputs.deploy-url }} -# - ${{ steps.private.outputs.alias-url }} -``` - -### publish-gh-pages - -```yaml -- id: pages - uses: quantecon/actions/publish-gh-pages@v0 - -# Access: ${{ steps.pages.outputs.page-url }} -``` - -## 🔍 Debugging Tips - -### Check Cache Hits - -Look for in logs: -``` -Conda: Restored from cache ✅ (saved ~5-6 minutes) # setup-environment, standard mode -✅ Cache restored successfully # restore-jupyter-cache -``` - -### Common Issues - -**Cache not working?** -```yaml -# Bump cache version -cache-version: 'v2' -``` - -**Build too slow?** -```yaml -# Use restore-jupyter-cache before build-lectures -- uses: quantecon/actions/restore-jupyter-cache@v0 - with: - cache-type: 'build' -``` - -**Netlify auth failing?** -```bash -# Verify secrets exist -gh secret list -``` - -**Pages 404?** -```yaml -# The native Pages deploy needs these; a permissions block drops every scope it omits -permissions: - contents: read # contents: write only if you set create-release-assets: 'true' - pages: write - id-token: write -``` - -## 📚 Full Documentation - -- **README.md** - Repository overview -- **dev/ARCHITECTURE.md** - Architecture overview -- **CONTAINER-GUIDE.md** - Container usage guide -- **MIGRATION-GUIDE.md** - Migration steps -- **{action}/README.md** - Detailed action docs - -## 🎓 Repository-Specific Notes - -### lecture-python.myst (GPU) - -```yaml -# ML packages (JAX, PyTorch) specified in repo's environment.yml -- uses: quantecon/actions/setup-environment@v0 - with: - environment-update: 'environment-update.yml' -``` - -### lecture-python-programming.myst - -```yaml -# Standard setup -- uses: quantecon/actions/setup-environment@v0 - with: - install-latex: 'true' -``` - -### lecture-python-intro - -```yaml -# Netlify only (no GH Pages) -- uses: quantecon/actions/preview-netlify@v0 -``` - -### lecture-python-advanced.myst - -```yaml -# Same as programming (standard setup) -- uses: quantecon/actions/setup-environment@v0 - with: - install-latex: 'true' -``` - -## 🔗 Links - -- **Repository:** https://github.com/quantecon/actions -- **Issues:** https://github.com/quantecon/actions/issues -- **Releases:** https://github.com/quantecon/actions/releases - ---- - -**💡 Pro Tip:** Start with the minimal workflow and add customizations as needed! diff --git a/docs/dev/ARCHITECTURE.md b/docs/dev/ARCHITECTURE.md index ca8d0fe..a1e7ddc 100644 --- a/docs/dev/ARCHITECTURE.md +++ b/docs/dev/ARCHITECTURE.md @@ -328,7 +328,7 @@ quantecon/actions/ └── docs/ ├── CONTAINER-GUIDE.md # Container build and usage guide ├── MIGRATION-GUIDE.md # How to migrate lecture repos - ├── QUICK-REFERENCE.md # Quick reference for all actions + ├── user/ # The user manual: a chapter for each action └── dev/ # Developer docs: this file, testing, plan, GPU AMI ``` @@ -481,7 +481,7 @@ lecture-python-intro/ - [CONTAINER-GUIDE.md](../CONTAINER-GUIDE.md) - Container build and usage guide - [MIGRATION-GUIDE.md](../MIGRATION-GUIDE.md) - How to migrate lecture repos -- [QUICK-REFERENCE.md](../QUICK-REFERENCE.md) - Quick reference for all actions +- [User manual](../user/README.md) - What each action does, how to set it up, and every input and output --- diff --git a/docs/dev/CHAPTER-TEMPLATE.md b/docs/dev/CHAPTER-TEMPLATE.md index 3b588be..06250cd 100644 --- a/docs/dev/CHAPTER-TEMPLATE.md +++ b/docs/dev/CHAPTER-TEMPLATE.md @@ -12,7 +12,7 @@ Each action has one chapter in the user manual, at `docs/user/actions/.m python3 scripts/generate-docs.py ``` - It fills the chapter's Inputs and Outputs tables, rewrites the action's `README.md` as a signpost to the chapter, and updates the action table in `docs/user/README.md`. The harness `gate` job runs it with `--check`, and fails when a committed table no longer matches. + It fills the chapter's Inputs and Outputs tables, rewrites the action's `README.md` as a signpost to the chapter, and updates the action table in `docs/user/README.md`. The harness `gate` job runs it with `--check`, and fails when a committed table no longer matches, or when an action has no chapter. - **Examples.** The minimal example is a complete workflow: `name:`, `on:`, `permissions:` and `jobs:`. Anything shorter is a fragment of steps, with no `jobs:` key. The realistic example is a link to the template that uses the action, not a copy of it. ## What the gate checks diff --git a/docs/dev/PLAN.md b/docs/dev/PLAN.md index 4882274..2b4ba09 100644 --- a/docs/dev/PLAN.md +++ b/docs/dev/PLAN.md @@ -2,7 +2,7 @@ Working plan for `QuantEcon/actions`: current state, prioritized backlog, dependency policy, and rollout status. -**Last updated:** 2026-09-28 — #206 fixes #205: bumping `setup-environment`'s `cache-version` builds a fresh Conda environment, because the cache's fallback no longer crosses versions. Before that, the same day, the user manual's generator, its drift checks and its first chapter, `setup-environment`, are up as #204 (#189). Before that, the same day, #102's fix is up as #203: the lecture matrix retries a failed HTML stage once with `--all`, and `lecture-python-programming` is back in it. Before that, 2026-09-25 — both container images move to the Anaconda 2026.07 baseline the lecture repos pin (#199), and #27's home is settled: a restore action built here (#200). Before that, 2026-09-24 — after **v0.12.0**, which delivered the rest of the July 2026 audit (#105–#109, #99; the #110 tracker is closed): items 7 and 8 are done, item 10 is absorbed by the user manual (#178), item 15 is down to one cosmetic fix, the closed issues' dispositions say so, and two consumer follow-ups are recorded under Consumers in production. Before that, 2026-09-23 — the #106/#109 docs sweep: the consumers table is current again (all eight exact pins, across six repos, are on `v0.11.1`), the issues opened since the July review have refreshed dispositions (#135 added), and item 10's size-figure half is done. Before that, 2026-08-11 — release gating added as P0 (#135, #136) and the consumers table corrected: five lecture repos are still on exact pins, which this document previously said did not exist. Before that, 2026-08-07 — after the **v0.11.0** and **v0.11.1** releases and the move of `lecture-dp` and `lecture-python.myst` to `@v0`, which is the first to carry both alerting fixes (#122, #127) to consumers: `build-jupyter-cache` reaches its siblings through the pinned `@v0` ref, so neither fix existed for any consumer until `v0` moved to this release. The backlog below is still the July 2026 review; individual items carry their own closure notes. +**Last updated:** 2026-09-28 — the user manual has a chapter for every action, the harness gate fails an action without one, and `docs/QUICK-REFERENCE.md` is gone (#190). Before that, the same day, #206 fixes #205: bumping `setup-environment`'s `cache-version` builds a fresh Conda environment, because the cache's fallback no longer crosses versions. Before that, the same day, the user manual's generator, its drift checks and its first chapter, `setup-environment`, are up as #204 (#189). Before that, the same day, #102's fix is up as #203: the lecture matrix retries a failed HTML stage once with `--all`, and `lecture-python-programming` is back in it. Before that, 2026-09-25 — both container images move to the Anaconda 2026.07 baseline the lecture repos pin (#199), and #27's home is settled: a restore action built here (#200). Before that, 2026-09-24 — after **v0.12.0**, which delivered the rest of the July 2026 audit (#105–#109, #99; the #110 tracker is closed): items 7 and 8 are done, item 10 is absorbed by the user manual (#178), item 15 is down to one cosmetic fix, the closed issues' dispositions say so, and two consumer follow-ups are recorded under Consumers in production. Before that, 2026-09-23 — the #106/#109 docs sweep: the consumers table is current again (all eight exact pins, across six repos, are on `v0.11.1`), the issues opened since the July review have refreshed dispositions (#135 added), and item 10's size-figure half is done. Before that, 2026-08-11 — release gating added as P0 (#135, #136) and the consumers table corrected: five lecture repos are still on exact pins, which this document previously said did not exist. Before that, 2026-08-07 — after the **v0.11.0** and **v0.11.1** releases and the move of `lecture-dp` and `lecture-python.myst` to `@v0`, which is the first to carry both alerting fixes (#122, #127) to consumers: `build-jupyter-cache` reaches its siblings through the pinned `@v0` ref, so neither fix existed for any consumer until `v0` moved to this release. The backlog below is still the July 2026 review; individual items carry their own closure notes. --- @@ -84,7 +84,7 @@ One correction to how that closure was written up: "cannot silently no-op" was t | 7 | ~~**Delete the dead `asset-url` output** in `publish-gh-pages`.~~ Done (#173, v0.12.0) — deleted with its README row, not repointed at `fromJSON(…assets)[0].browser_download_url`, which errors whenever the release step is skipped, the default. | #107, #173 | | 8 | ~~**`preview-netlify`: move the auth token into `env:`.**~~ Done (#174, v0.12.0) — with the step's four other interpolated values. `--auth` and `--site` are gone: netlify-cli reads `NETLIFY_AUTH_TOKEN` and `NETLIFY_SITE_ID` from the environment. | #105, #174 | | 9 | ~~**CI coverage for standard-mode conda caching.**~~ Done — `test-actions.yml` (the #100 stage-1 harness) runs the two-run miss→hit chain on every PR touching `setup-environment`, plus a build on the restored env. | #29, #33, #100 | -| 10 | **Docs surplus trim.** ~4,900 doc lines for ~1,600 lines of action code, with four overlapping indexes. Absorbed by the user manual (#178, decision 4). Its PR 1 (#182) moved the developer docs to `docs/dev/` and deleted `docs/README.md`. PR 2 (#189) generates the manual's input and output tables from each `action.yml` and checks them in the harness gate, so the READMEs' drift (#109) cannot recur in the chapters; the rest of this item (`QUICK-REFERENCE.md`, MIGRATION-GUIDE's per-repo notes, the copilot-instructions boilerplate, ARCHITECTURE's stale blocks) is its PRs 3–5. The corrected container-size figures were propagated in the #106 sweep (#175). | #40, #178 | +| 10 | **Docs surplus trim.** ~4,900 doc lines for ~1,600 lines of action code, with four overlapping indexes. Absorbed by the user manual (#178, decision 4). Its PR 1 (#182) moved the developer docs to `docs/dev/` and deleted `docs/README.md`. PR 2 (#189) generates the manual's input and output tables from each `action.yml` and checks them in the harness gate, so the READMEs' drift (#109) cannot recur in the chapters. PR 3 (#190) gives every action a chapter, fails the gate for an action without one, and deletes `QUICK-REFERENCE.md`; the rest of this item (MIGRATION-GUIDE's per-repo notes, the copilot-instructions boilerplate, ARCHITECTURE's stale blocks) is its PRs 4 and 5. The corrected container-size figures were propagated in the #106 sweep (#175). | #40, #178 | | 11 | **Document composite action vs reusable workflow.** Add the short decision rule to CONTRIBUTING.md or ARCHITECTURE.md so new CI lands at the right altitude. | #29 | | 12 | **Environment manifest v1.** The publish-time manifest is a stub (name/tag/commit/size). Define a versioned schema, capture the effective environment (resolved `conda list`/`pip freeze`, container digest, build metadata), and `repository_dispatch` to `status-lectures`. | #30, meta#321 | diff --git a/docs/dev/TESTING.md b/docs/dev/TESTING.md index 529bd9f..bba4df8 100644 --- a/docs/dev/TESTING.md +++ b/docs/dev/TESTING.md @@ -22,7 +22,7 @@ ## Action-Level PR Harness (`test-actions.yml`) -`.github/workflows/test-actions.yml` tests the composite actions themselves — via `uses: ./` local paths, so it exercises **the code on the PR**, not a released ref. The workflow runs on **every** PR, on pushes to `main`, and on manual dispatch; a `gate` job then decides whether the jobs themselves apply, skipping them for changes that cannot affect the actions. It works that way because a workflow suppressed by a `paths:` filter never publishes a check run at all, which would leave a required `Action harness: all checks` pending forever. This is stage 1 of the two-part design in issue #100; it is the permanent form of the throwaway harness that verified the #104 fix. The gate also runs the checks a docs-only PR must still pass, since it skips every other job: `templates/` pins the same third-party actions as `.github/workflows/` (#109), and the user manual matches each `action.yml`, with its examples passing the input, `permissions:` and actionlint checks of `scripts/generate-docs.py` (#178). +`.github/workflows/test-actions.yml` tests the composite actions themselves — via `uses: ./` local paths, so it exercises **the code on the PR**, not a released ref. The workflow runs on **every** PR, on pushes to `main`, and on manual dispatch; a `gate` job then decides whether the jobs themselves apply, skipping them for changes that cannot affect the actions. It works that way because a workflow suppressed by a `paths:` filter never publishes a check run at all, which would leave a required `Action harness: all checks` pending forever. This is stage 1 of the two-part design in issue #100; it is the permanent form of the throwaway harness that verified the #104 fix. The gate also runs the checks a docs-only PR must still pass, since it skips every other job: `templates/` pins the same third-party actions as `.github/workflows/` (#109), and the user manual has a chapter for every action and matches each `action.yml`, with its examples passing the input, `permissions:` and actionlint checks of `scripts/generate-docs.py` (#178). Fixtures are salted with `run_id`-`run_attempt` so cache keys are unique per run and miss assertions cannot be polluted by earlier runs. The committed fixture (`.github/fixtures/mini-lectures/`) executes a real code cell, so builds populate a genuine `_build/.jupyter_cache`. diff --git a/docs/user/README.md b/docs/user/README.md index d8e9294..f54fb29 100644 --- a/docs/user/README.md +++ b/docs/user/README.md @@ -15,13 +15,13 @@ A workflow step uses an action as `quantecon/actions/@v0`. | Action | What it does | |---|---| | [`setup-environment`](actions/setup-environment.md) | Sets up the Python environment for a lecture build. Inside a QuantEcon container it uses the image's environment, adding any extra packages you list; on a standard runner it builds a cached Conda environment and can install LaTeX. | -| [`build-lectures`](https://github.com/QuantEcon/actions/tree/main/build-lectures) | Builds QuantEcon lectures using Jupyter Book | -| [`build-jupyter-cache`](https://github.com/QuantEcon/actions/tree/main/build-jupyter-cache) | Fresh build of all lecture formats and save to GitHub cache (runs on main branch, typically weekly) | -| [`restore-jupyter-cache`](https://github.com/QuantEcon/actions/tree/main/restore-jupyter-cache) | Restores Jupyter Book build cache from GitHub Actions cache, with optional save for PR-scoped caching | -| [`preview-netlify`](https://github.com/QuantEcon/actions/tree/main/preview-netlify) | Deploys lecture builds to Netlify for PR previews with smart comments showing changed pages | -| [`preview-cloudflare`](https://github.com/QuantEcon/actions/tree/main/preview-cloudflare) | Deploys lecture builds to Cloudflare Pages for PR previews with smart comments showing changed pages | -| [`publish-gh-pages`](https://github.com/QuantEcon/actions/tree/main/publish-gh-pages) | Publishes lecture builds to GitHub Pages using native GitHub Pages deployment (no gh-pages branch needed) | -| [`deploy-cloudflare`](https://github.com/QuantEcon/actions/tree/main/deploy-cloudflare) | Publishes a built site to an existing Cloudflare Worker behind Cloudflare Access, and fails unless the site is proven gated before and after the deploy | +| [`build-lectures`](actions/build-lectures.md) | Builds the lectures with Jupyter Book, as the website, the PDF or notebooks. When the build fails it prints each failing notebook's traceback, and can upload the execution reports. | +| [`build-jupyter-cache`](actions/build-jupyter-cache.md) | Builds the lectures from scratch in each format you list, and saves the result as the cache that pull requests and publishing start from, but only if every build passes. A failed run keeps the last good cache and files an issue. | +| [`restore-jupyter-cache`](actions/restore-jupyter-cache.md) | Restores the cache that `build-jupyter-cache` saved, so a pull request or publish build re-executes only the notebooks that changed. It only restores by default, and can also save a cache for the later runs of the same pull request. | +| [`preview-netlify`](actions/preview-netlify.md) | Deploys a pull request's built site to Netlify as a preview, at a URL that stays the same for every push, and comments on the pull request with it and with links to the lectures it changes. | +| [`preview-cloudflare`](actions/preview-cloudflare.md) | Deploys a pull request's built site to Cloudflare Pages as a preview, at a URL that stays the same for every push, and comments on the pull request with it and with links to the lectures it changes. | +| [`publish-gh-pages`](actions/publish-gh-pages.md) | Publishes a built site to GitHub Pages with GitHub's own Pages deploy, so no gh-pages branch is needed. On a tag it can also attach the site to the release, as an archive with its checksum and a manifest. | +| [`deploy-cloudflare`](actions/deploy-cloudflare.md) | Publishes a built site to an existing Cloudflare Worker behind Cloudflare Access, for a site only members may see, and fails unless the site is proven gated before and after the deploy. | diff --git a/docs/user/actions/build-jupyter-cache.md b/docs/user/actions/build-jupyter-cache.md new file mode 100644 index 0000000..9e094a4 --- /dev/null +++ b/docs/user/actions/build-jupyter-cache.md @@ -0,0 +1,152 @@ +# build-jupyter-cache + +> This describes `main`. For the version you pin, open the manual at that tag. Changes not yet released are under `[Unreleased]` in the [CHANGELOG](https://github.com/QuantEcon/actions/blob/main/CHANGELOG.md). + +Builds the lectures from scratch, in each format listed in `builders`, and saves the result as the cache that pull-request and publish builds restore with [`restore-jupyter-cache`](restore-jupyter-cache.md). It saves two caches: + +- **The build cache**: the whole `_build` directory, with the HTML, the PDF, the notebooks and the execution cache. +- **The execution cache**: `_build/.jupyter_cache` alone, which holds each notebook's executed outputs. + +Both are saved only when every build passes. A failed run saves nothing, so the builds that restore keep the last good cache, and it files an issue, so that the failure is seen. + +## When to use it + +Use it in a workflow of its own, on the default branch: weekly, on demand, and when the environment file changes, as [`cache.yml`](https://github.com/QuantEcon/actions/blob/main/templates/cache.yml) does. + +- **On the default branch.** A cache saved there can be restored by every branch and pull request. One saved on another branch serves only that branch, and the pull requests that target it. +- **On its own.** The action runs [`setup-environment`](setup-environment.md) and [`build-lectures`](build-lectures.md) itself, so the job needs neither. +- **From scratch.** Do not restore a cache before it. Its fresh build is what clears out what incremental builds leave behind, such as the pages of a lecture that has been removed. + +## Requirements + +- **Runner.** A QuantEcon container, or a GitHub-hosted Ubuntu runner, where `setup-environment` builds the Conda environment and, with `pdflatex` among the builders, installs LaTeX. +- **Node.** None. +- **Permissions.** `issues: write`, to file the failure issue, unless `create-issue-on-failure` is `'false'`. The job also needs `contents: read` for `actions/checkout`, and the templates grant `packages: read` in a `container:` job, for the image pull. Saving caches and uploading artifacts need no permission. +- **Secrets.** None: the failure issue is filed with the job's own token. +- **Files in your repository.** The book in `source-dir`. In standard mode, the environment file, and with `pdflatex` the LaTeX package list; see [`setup-environment`](setup-environment.md#requirements). In container mode, the `environment-update` file if you set one. Check out with `fetch-depth: 0`, so that the cached HTML dates each page from its own history. +- **Setup outside GitHub.** None. + +## Inputs + + + +| Input | Required | Default | Description | +|---|---|---|---| +| `builders` | no | `html` | The formats to build and cache: any of `jupyter`, `pdflatex` and `html`, separated by commas or whitespace, as in `jupyter,pdflatex,html`. They run in that order whatever order they are listed in, and the `html` build copies in the PDF and the notebooks when those are listed too. An unknown name, or none at all, fails the run before anything is built. | +| `environment` | no | `environment.yml` | Path to the Conda environment file. In standard mode `setup-environment` builds the environment from it, and fails if it does not exist. In either mode its hash is part of the build cache's key, so give `restore-jupyter-cache` the same file. | +| `environment-update` | no | `''` | Container mode: path to a Conda environment file of packages to add to the image's environment, which `setup-environment` installs. Empty, the default, adds none. In either mode its hash is part of the build cache's key, so give `restore-jupyter-cache` the same file. | +| `source-dir` | no | `lectures` | The book to build, as `build-lectures` takes it: the directory that holds its `_config.yml` and `_toc.yml`. The execution cache's key hashes every `.md` file under it. | +| `latex-requirements-file` | no | `latex-requirements.txt` | Standard mode, with `pdflatex` among the builders: path to the list of apt packages that `setup-environment` installs for LaTeX. The job fails if the file is missing or names no package. Ignored in container mode, whose images carry LaTeX. | +| `upload-artifact` | no | `true` | `'true'`, the default, uploads the whole `_build` directory as the artifact `build-cache-` when a build fails, since no cache is saved then. `'false'` uploads nothing. A run whose builds all pass uploads no artifact either way: the cache holds `_build`. | +| `artifact-retention-days` | no | `30` | How many days to keep the artifact that `upload-artifact` uploads, up to the repository's retention limit. | +| `create-issue-on-failure` | no | `true` | `'true'`, the default, files an issue when the run fails, or comments on the open one that carries the first of `issue-labels`, and fails the job if neither worked. It needs `issues: write`. `'false'` files nothing. | +| `issue-assignees` | no | `''` | Comma-separated GitHub usernames to assign a new failure issue to. Empty, the default, assigns nobody. | +| `issue-labels` | no | `build-failure,automated` | Comma-separated labels for a new failure issue, created in the repository if they do not exist. The first also finds an open failure issue: a later failure comments on it instead of filing another. Empty files a new issue for every failure. | +| `upload-failure-reports` | no | `true` | `'true'`, the default, makes each failed build upload its execution reports as `execution-reports-`, as the `build-lectures` input of the same name does, and the failure issue names them. `'false'` uploads none. The default differs from `build-lectures`' because this action runs unattended. | + + + +## Outputs + + + +| Output | Description | +|---|---| +| `cache-saved` | `'true'` when every requested build passed, which is when the build and execution caches are saved; `'false'` otherwise, including a run that stopped before any build. Always set. | +| `build-success` | The same value as `cache-saved`: `'true'` only when every requested build passed; `'false'` for anything else, including a run that stopped during setup, before any build. Always set. | +| `cache-key` | The key the build cache is saved under, `build---`. Set even when a build fails and nothing is saved. | +| `jupyter-status` | `success` or `failure` for a `jupyter` build that ran, `skipped` when `builders` does not list it. Empty when the run stopped before the builds. | +| `pdflatex-status` | `success` or `failure` for a `pdflatex` build that ran, `skipped` when `builders` does not list it. Empty when the run stopped before the builds. | +| `html-status` | `success` or `failure` for an `html` build that ran, `skipped` when `builders` does not list it. Empty when the run stopped before the builds. | +| `failure-issue-url` | The URL of the failure issue filed or commented on. Empty when every build passed, or with `create-issue-on-failure: 'false'`. | + + + +## Examples + +### Minimal + +A weekly cache build in a container, which can also be started by hand: + +```yaml +name: Build cache +on: + schedule: + - cron: '0 0 * * 0' # Sundays at 00:00 UTC + workflow_dispatch: + +jobs: + build-cache: + runs-on: ubuntu-latest + container: + image: ghcr.io/quantecon/quantecon:latest + permissions: + contents: read + issues: write # the failure issue + packages: read # the image pull, as the templates grant it + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 # each page's "Last changed" date comes from git log + - uses: quantecon/actions/build-jupyter-cache@v0 +``` + +### Every format + +For a site that offers the PDF and the notebooks as downloads: + +```yaml +- uses: quantecon/actions/build-jupyter-cache@v0 + with: + builders: jupyter,pdflatex,html +``` + +The cached `_build/html` then holds the PDF and the notebooks too, so a build that restores it can offer them without building them again: see [`build-lectures`](build-lectures.md#downloads-on-the-site). + +### On a standard runner + +Leave out the `container:` block. `setup-environment` then builds the Conda environment from `environment.yml`, and with `pdflatex` among the builders installs the LaTeX packages listed in `latex-requirements.txt`. Both files must be at the repository root, or named with `environment` and `latex-requirements-file`. + +### In the templates + +[`cache.yml`](https://github.com/QuantEcon/actions/blob/main/templates/cache.yml) runs this action weekly, on manual dispatch, and on pushes to `main` that change `environment.yml`, one run at a time. It builds `html`, lists the other formats as comments, files an issue on failure, and keeps the failed `_build` for 30 days. + +## Behaviour + +1. **Builders.** `builders` is split on commas and whitespace, and every name must be `jupyter`, `pdflatex` or `html`. +2. **Environment.** `setup-environment` runs with `environment`, `environment-update` and `latex-requirements-file`, and installs LaTeX only when `pdflatex` is among the builders. Its other inputs keep their defaults, so in standard mode the environment is named `quantecon`, uses Python 3.13, and is cached under `cache-version` `v1`. +3. **Builds.** `build-lectures` runs once for each requested builder, always in the order `jupyter`, `pdflatex`, `html`, with its default `extra-args`, `-W --keep-going`, and `output-dir`, `.`. The `html` build copies in the PDF when `pdflatex` ran, and the notebooks when `jupyter` ran. A failed build does not stop the later ones, so each run reports on every builder. +4. **Caches.** If every build passed, both caches are saved: + + | Cache | Path | Key | + |---|---|---| + | Build | `_build` | `build---` | + | Execution | `_build/.jupyter_cache` | `jupyter-cache--` | + + A file that does not exist hashes to an empty string, so with no `environment-update` the build cache's key is `build---`. Every successful run saves new entries, and `restore-jupyter-cache` restores the newest through a prefix match. GitHub removes a cache that has not been restored for 7 days, and the oldest caches first when the repository's cache storage is full. +5. **Job summary.** "Jupyter Cache Build Summary" gives the outcome, the cache key, the trigger, the commit, the mode `setup-environment` ran in, each builder's result and the size of each directory in `_build`. + +The action runs `setup-environment` and `build-lectures` at `@v0`, whichever version of `build-jupyter-cache` the workflow pins. + +### When a build fails + +1. **Nothing is saved.** The last good caches stay, and the builds that restore them carry on as before. +2. **Artifacts.** With `upload-artifact`, the whole `_build` directory as `build-cache-`. With `upload-failure-reports`, each failed build's reports as `execution-reports-`. +3. **The failure issue.** With `create-issue-on-failure`, the action files an issue titled `🔴 Cache Build Failed - `, with the labels in `issue-labels` and the assignees in `issue-assignees`. If an open issue already carries the first of those labels, it adds a comment there instead. Either way the text links to the run, gives each builder's result, names the artifacts the run actually uploaded, and gives a command to reproduce each failed build locally. The action then checks that the issue was filed, and fails if it was not. +4. **The job fails**, with `One or more builds failed - see summary above`. + +A run can also stop before any lecture is built: on an invalid `builders`, or when `setup-environment` fails. It then fails with `The cache build aborted during …, before any lecture was built`, and the issue says that no lecture was built, so that the failure is not mistaken for a broken lecture. Every builder then reads `not run`. + +## Troubleshooting + +**`Unknown builder '…' in the builders input. Valid builders: jupyter, pdflatex, html`.** Fix the name: the PDF builder is `pdflatex`, not `pdf`. + +**`Could not file the failure issue: the token lacks issue permissions`.** Add `issues: write` to the job's `permissions:`, or set `create-issue-on-failure: 'false'`. + +**`create-issue-on-failure is enabled but no failure issue was filed (…)`.** The step that files the issue failed, and the reason is in its log, just above. The run's own failure is in the job summary. + +**`The cache build aborted during setup, before any lecture was built`.** `setup-environment` failed: its log, and the [`setup-environment` troubleshooting](setup-environment.md#troubleshooting), give the cause. + +**New failures comment on an old issue.** The action comments on the open issue that carries the first label. Close the issue once the build is fixed; the next failure files a new one. + +**Pull requests do not restore the new cache.** A cache saved on another branch does not reach them: run the workflow on the default branch. Check also that the runs that restore pass the same `environment` and `environment-update`, since both files' hashes are in the key. diff --git a/docs/user/actions/build-lectures.md b/docs/user/actions/build-lectures.md new file mode 100644 index 0000000..e75d1b9 --- /dev/null +++ b/docs/user/actions/build-lectures.md @@ -0,0 +1,187 @@ +# build-lectures + +> This describes `main`. For the version you pin, open the manual at that tag. Changes not yet released are under `[Unreleased]` in the [CHANGELOG](https://github.com/QuantEcon/actions/blob/main/CHANGELOG.md). + +Runs `jb build`, Jupyter Book 1, on the book in `source-dir`, with one builder per step: + +- **`html`**, the default: the website, in `_build/html`. +- **`pdflatex`**: the PDF, in `_build/latex`. +- **`jupyter`**: the lectures as notebooks, in `_build/jupyter`. + +An `html` build can also copy in the PDF and the notebooks from earlier builds, so the site offers them for download. When a build fails, the action prints the traceback of each notebook that failed to execute, and can upload the execution reports as an artifact. + +## When to use it + +Use it in any job that builds lectures, after [`setup-environment`](setup-environment.md), and after [`restore-jupyter-cache`](restore-jupyter-cache.md) in a job that starts from the build cache. A preview or publish step then deploys the directory in its `build-path` output. + +- **One step per format.** For a site that offers the PDF and the notebooks, build `jupyter` and `pdflatex` first, then `html` with `html-copy-pdf` and `html-copy-notebooks`, all in one job. See [Downloads on the site](#downloads-on-the-site). +- **Not in the weekly cache build.** [`build-jupyter-cache`](build-jupyter-cache.md) runs this action itself, once for each of its builders. + +## Requirements + +- **Runner.** Any job where a login shell finds `jb`: a QuantEcon container, whose images carry Jupyter Book 1 and LaTeX, or a GitHub-hosted runner after `setup-environment`, whose environment file must then install `jupyter-book` 1.x and every extension the book's `_config.yml` loads. The `pdflatex` builder needs LaTeX: on a standard runner, pass `install-latex: 'true'` to `setup-environment`. +- **Node.** None. +- **Permissions.** None of its own: the action calls no GitHub API. The job needs `contents: read` for `actions/checkout`. +- **Secrets.** None. +- **Files in your repository.** A Jupyter Book 1 project in `source-dir`: its `_config.yml`, its `_toc.yml` and the pages they name. The `html` builder dates each page from the repository's git history, so check out with `fetch-depth: 0`. +- **Setup outside GitHub.** None. + +## Inputs + + + +| Input | Required | Default | Description | +|---|---|---|---| +| `builder` | no | `html` | What to build: `html`, the default, builds the website into `/_build/html`; `pdflatex` builds the PDF into `/_build/latex`, and needs LaTeX; `jupyter` builds notebooks into `/_build/jupyter`. Any other value is passed to `jb build` as `--builder `. | +| `source-dir` | no | `lectures` | The book to build: the directory that holds its `_config.yml` and `_toc.yml`, relative to the workspace. | +| `output-dir` | no | `.` | The directory the build is written under, passed to `jb build` as `--path-output`, so the output lands in `/_build`. The cache actions read and write `_build` at the workspace root, so keep the default, `.`, in a job that uses them. | +| `extra-args` | no | `-W --keep-going` | Further arguments for `jb build`. They are split on whitespace, so a quoted value that contains a space cannot be passed. Setting this replaces the default, so keep `-W` in it: without `-W`, a notebook that raises an exception is only a warning, and the build succeeds. | +| `html-copy-pdf` | no | `false` | With `builder: html`: `'true'` copies each PDF under `/_build/latex` into `/_build/html/_pdf/` before the build, so the site offers it for download. The PDF comes from a `pdflatex` build earlier in the job, or from a restored build cache; if that directory does not exist, a warning says so and the site is built without it. Ignored by the other builders. | +| `html-copy-notebooks` | no | `false` | With `builder: html`: `'true'` copies each notebook under `/_build/jupyter` into `/_build/html/_notebooks/` before the build, so the site offers them for download. The notebooks come from a `jupyter` build earlier in the job, or from a restored build cache; if that directory does not exist, a warning says so and the site is built without them. Ignored by the other builders. | +| `upload-failure-reports` | no | `false` | `'true'` uploads an artifact when the build fails, holding the `reports` directories in `_build/html`, `_build/latex` and `_build/jupyter`, and `_build/.jupyter_cache`, all under `output-dir`, kept for 7 days. `'false'`, the default, uploads nothing. For the `html`, `pdflatex` and `jupyter` builders the log shows each failing notebook's traceback either way. | +| `failure-artifact-name` | no | `''` | The name of the artifact that `upload-failure-reports` uploads. Empty, the default, names it `execution-reports-`. A name can be used only once in a workflow run, so set this when more than one job in a run builds with the same builder. | + + + +Paths are relative to the workspace, which is the repository root after `actions/checkout`. + +## Outputs + + + +| Output | Description | +|---|---| +| `build-path` | The directory the builder wrote to: `/_build/html`, `_build/latex` or `_build/jupyter` for those three builders, and `/_build` for any other. With the default `output-dir` it starts with `./`, as in `./_build/html`. Set whether or not the build succeeds. | + + + +## Examples + +### Minimal + +Builds the website in a container, and keeps it as an artifact of the run: + +```yaml +name: Build lectures +on: + pull_request: + +jobs: + build: + runs-on: ubuntu-latest + container: + image: ghcr.io/quantecon/quantecon-build:latest + permissions: + contents: read + packages: read # the image pull, as the templates grant it + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 # each page's "Last changed" date comes from git log + - uses: quantecon/actions/setup-environment@v0 + - uses: quantecon/actions/build-lectures@v0 + id: build + with: + upload-failure-reports: 'true' + - uses: actions/upload-artifact@v7 + with: + name: site + path: ${{ steps.build.outputs.build-path }} +``` + +### Downloads on the site + +Build the notebooks and the PDF first, in the same job, then the website with both copied in: + +```yaml +- uses: quantecon/actions/build-lectures@v0 + with: + builder: jupyter +- uses: quantecon/actions/build-lectures@v0 + with: + builder: pdflatex +- uses: quantecon/actions/build-lectures@v0 + id: build + with: + builder: html + html-copy-pdf: 'true' + html-copy-notebooks: 'true' +``` + +The website then holds both, and the theme links to them: + +```text +_build/html/ +├── index.html +├── _pdf/ +│ └── +└── _notebooks/ + ├── .ipynb + └── … +``` + +After `restore-jupyter-cache`, the copies can also come from the build cache, if its cache build ran those builders: `_build/latex` and `_build/jupyter` are then already in place, as the last cache build left them. + +### Other arguments for `jb build` + +Keep `-W --keep-going` in any value you set, because it replaces the default: + +```yaml +- uses: quantecon/actions/build-lectures@v0 + with: + extra-args: '-W --keep-going --all' # rebuild every page, not only the changed ones +``` + +`-v` makes the log more verbose, `-q` quieter, and `-n` turns on Sphinx's nitpicky mode, which warns about every reference it cannot resolve. The `pdflatex` and `jupyter` builders already pass `-n`. + +### In the templates + +The [workflow templates](https://github.com/QuantEcon/actions/tree/main/templates) build the website with this action, with `upload-failure-reports: 'true'`: [`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml) for a pull request's preview, and [`publish.yml`](https://github.com/QuantEcon/actions/blob/main/templates/publish.yml) before it publishes to GitHub Pages. `publish.yml` carries the `jupyter` and `pdflatex` steps, and the two copy inputs, as comments. [`cache.yml`](https://github.com/QuantEcon/actions/blob/main/templates/cache.yml) builds through `build-jupyter-cache` instead. + +## Behaviour + +1. **Downloads.** For an `html` build with `html-copy-pdf: 'true'`, every `.pdf` anywhere under `_build/latex` is copied into `_build/html/_pdf/`. With `html-copy-notebooks: 'true'`, every `.ipynb` under `_build/jupyter` is copied into `_build/html/_notebooks/`. Both copies are flat: files from subdirectories land side by side. A missing source directory is a warning, not a failure. +2. **Git history.** For every builder but `pdflatex` and `jupyter`, the action checks that git can read the repository, because [quantecon-book-theme](https://github.com/QuantEcon/quantecon-book-theme) dates each page and builds its changelog from `git log`, and drops both without a word when git fails. + - In a container job git refuses the work tree, reporting "dubious ownership": the runner creates the workspace as its own user, and the container's steps run as root. The action then trusts the work tree for the rest of the job, by adding `GIT_CONFIG_COUNT`, `GIT_CONFIG_KEY_` and `GIT_CONFIG_VALUE_` to the job's environment, so later steps can run git too. Nothing is written to disk, and nothing outlasts the job. + - A warning says when the build will lack the dates: git is missing, `source-dir` is not in a git work tree, or the clone is shallow. A shallow clone dates every page to the checked-out commit. +3. **The build.** The action runs, in a login shell, so that on a standard runner the Conda environment from `setup-environment` is active: + + ```text + jb build --path-output + ``` + + | Builder | Builder flags | Output | + |---|---|---| + | `html` | none | `/_build/html` | + | `pdflatex` | `--builder pdflatex -n` | `/_build/latex` | + | `jupyter` | `--builder=custom --custom-builder=jupyter -n` | `/_build/jupyter` | + | any other | `--builder ` | `/_build` | + + The log shows the full command in its "Build Command" group. +4. **On failure**, the action prints a summary of the builder, source and output. For the `html`, `pdflatex` and `jupyter` builders it then prints each failed notebook's traceback, from `reports/*.err.log` in the build's output directory: the last 200 lines of each, in a log group named after the report. With `upload-failure-reports: 'true'` it uploads the reports and the execution cache as an artifact. +5. **On success**, the log ends with a "Build Summary" group and a listing of the output directory, in "Build Artifacts". Nothing is uploaded. + +### What fails the job + +`jb build` exiting with an error. With `-W`, which the default `extra-args` passes, that includes every warning, so a notebook cell that raises an exception fails the build. + +> [!WARNING] +> Keep `-W` in any `extra-args` you set. A cell that raises is not an error to Jupyter Book: myst-nb logs it as a warning and carries on, and only `-W` turns that into a failed build. Without it the build exits 0 over a broken lecture, and the job goes on to deploy it. `--keep-going` makes the build report every such warning before it fails, instead of stopping at the first; without `-W` it does nothing. +> +> Setting `raise_on_error: true` under `execute:` in `_config.yml` also fails the build, but at the first failing notebook, before any `reports/*.err.log` is written. Prefer `-W`. + +## Troubleshooting + +**`❌ BUILD FAILED - Jupyter Book build encountered errors`.** The groups below it hold each failed notebook's traceback. To reproduce the failure locally, run the command from the "Build Command" group, for example `jb build lectures -W --keep-going`. + +**The build passed, but a notebook failed.** `extra-args` is set without `-W`. Add it back. + +**`jb: command not found`.** On a standard runner, `setup-environment` has not run, or its environment file does not install `jupyter-book`. In a container, check the job's `container:` image. + +**`PDF source directory not found: …` or `Notebook source directory not found: …`.** An `html` build asked for a copy that nothing produced. Run the `pdflatex` or `jupyter` build first, in the same job, or restore a build cache whose cache build ran it. + +**`… is a shallow clone, so every page's 'Last changed' date will be the checked-out commit's date`.** Set `fetch-depth: 0` on `actions/checkout`. + +**`git cannot find the work tree for …`.** `source-dir` is not inside a git checkout: check that `actions/checkout` runs first, and where it checks out to. + +**The failure-report upload fails because `an artifact with this name already exists`.** Another job in the run uploaded reports for the same builder. Give each job its own `failure-artifact-name`. diff --git a/docs/user/actions/deploy-cloudflare.md b/docs/user/actions/deploy-cloudflare.md new file mode 100644 index 0000000..4a8fb8e --- /dev/null +++ b/docs/user/actions/deploy-cloudflare.md @@ -0,0 +1,221 @@ +# deploy-cloudflare + +> This describes `main`. For the version you pin, open the manual at that tag. Changes not yet released are under `[Unreleased]` in the [CHANGELOG](https://github.com/QuantEcon/actions/blob/main/CHANGELOG.md). + +Publishes a built site to a Cloudflare Worker that sits behind Cloudflare Access, for a site that only members of a GitHub team may see, such as a dashboard or a report. Access sends every visitor to a GitHub login first, and lets in only those its policy allows. + +The Worker must already exist and already be behind Access: the action never creates one, and never turns Access on. What it does is prove the gate works, and fail the job when it cannot: + +- **Before anything is uploaded**, an anonymous request to the Worker must be redirected to your Access team's login. If it is not, nothing is uploaded. +- **After the deploy**, the same must hold for production and for the new version's own preview URL, and for the alias after its upload, if you give one. + +The result goes to the job summary, since a deploy on a push, a schedule or a manual run has no pull request to comment on. + +## When to use it + +Use it to publish a site that must not be public. + +- **Not for a public site.** [`publish-gh-pages`](publish-gh-pages.md) publishes to GitHub Pages, which serves every site publicly unless the organisation is on GitHub Enterprise Cloud. A Worker behind Access can be private on Cloudflare's free plans. +- **Not for pull-request previews.** [`preview-cloudflare`](preview-cloudflare.md) gives each pull request a preview on Cloudflare Pages. + +## Requirements + +- **Runner.** Any with `bash`, `curl` and `npm`, as GitHub-hosted runners and the QuantEcon images have. The job must be able to reach `registry.npmjs.org`, from which `wrangler` is installed each run. +- **Node.** 22 or later, for `wrangler`. The action sets up Node 24 itself when the runner has an older one or none. The QuantEcon images carry Node 24. +- **Permissions.** None of its own: the action calls no GitHub API. The job needs `contents: read` for `actions/checkout`. +- **Secrets.** `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID`, from the [setup checklist](#setup-checklist). Runs from forks and by Dependabot get no secrets, and the action then fails before it contacts anything. +- **Files in your repository.** None beyond the built site: the action writes the Worker's wrangler configuration itself, for each run. +- **Setup outside GitHub.** A Worker behind Access, set up once by a Cloudflare admin: see the [setup checklist](#setup-checklist). + +## Setup checklist + +### Once per Cloudflare account + +1. **A Zero Trust organisation.** The team name you choose gives the login domain `.cloudflareaccess.com`, which is the `team-domain` input. The free plan covers up to 50 users. +2. **GitHub as the login method.** Create an OAuth App under the GitHub organisation's own settings, **Settings**, **Developer settings**, **OAuth Apps**, rather than under a personal account: an app that the organisation owns needs no approval under the organisation's OAuth app access restrictions. Give it: + - Homepage URL: `https://.cloudflareaccess.com` + - Authorization callback URL: `https://.cloudflareaccess.com/cdn-cgi/access/callback` + + Then add it in Zero Trust, under **Settings**, **Authentication**, **Login methods**, **GitHub**. +3. **Make GitHub the only login method.** A new Zero Trust account starts with another login method switched on. Remove it. +4. **One reusable policy per audience.** Under **Access controls**, **Policies**, create an Allow policy with the **GitHub Organization** selector, your organisation, and a named team. Reference it from each application. Use a team rather than the whole organisation: an organisation's members usually include people a private site is not meant for, such as outside collaborators. A session can last up to a month. + +### Once per site + +1. **Create the Worker by hand, under its final name,** as a placeholder that holds no data, such as the dashboard's Hello World template. Keep its `workers.dev` route on: the action deploys to `https://..workers.dev`, and checks that URL. Creating a Worker needs Admin on the Workers product, which the deploy token deliberately lacks; see Cloudflare's [authorization docs](https://developers.cloudflare.com/workers/authorization/). +2. **Turn Access on for the Worker with *All traffic*,** not *Previews only*. That covers the `workers.dev` hostname, every preview URL, and so every alias, and any custom domain you attach later. Attach the reusable policy, and turn on instant authentication. Use this setting on the Worker, not the account-wide "Protect all Workers" switch, if the account also serves public sites. +3. **Prove the gate on the placeholder** before the first real deploy, with [`check-access-gate.sh`](https://github.com/QuantEcon/actions/blob/main/scripts/check-access-gate.sh) from a clone of this repository: + + ```bash + bash scripts/check-access-gate.sh .cloudflareaccess.com https://..workers.dev/ + ``` + + The action repeats this check before every deploy, and refuses to deploy if it fails, but proving it now catches a setup mistake before any workflow depends on the site. +4. **Create the deploy token**: an account-owned API token with **Editor** on this one Worker, from Cloudflare's per-Worker roles, not the account-wide *Workers Scripts: Edit*. It can deploy this Worker but cannot create one, so a mistyped `worker-name` fails instead of creating a new Worker with nothing in front of it. The check before the upload refuses that case too, whatever the token can do. +5. **Add the secrets to the repository** that deploys the site, under **Settings**, **Secrets and variables**, **Actions**: + + | Secret | Value | + |---|---| + | `CLOUDFLARE_API_TOKEN` | the per-Worker token from step 4 | + | `CLOUDFLARE_ACCOUNT_ID` | from the dashboard's URL, `https://dash.cloudflare.com//…`, or the **Workers & Pages** overview | + +### Why the action does not do this setup itself + +The Access application is set up once per Worker, and the login method once per account. Doing either from a workflow would put a token that can edit Access applications and policies into every repository that deploys, for a step that runs once. For the same reason the action cannot create the Worker: its token can deploy only to one that exists. An organisation that wants its Cloudflare account under code can manage the same settings with Terraform, whose resources `zero_trust_access_application` and `zero_trust_access_policy` cover them. + +## Inputs + + + +| Input | Required | Default | Description | +|---|---|---|---| +| `cloudflare-api-token` | yes | none | An account-owned Cloudflare API token with Editor on this one Worker, from a repository secret such as `secrets.CLOUDFLARE_API_TOKEN`. It can deploy the Worker but not create one, so a mistyped `worker-name` fails instead of creating a new Worker that nothing gates. The job fails if the token is empty, as it is on runs from forks and by Dependabot. | +| `cloudflare-account-id` | yes | none | The ID of the Cloudflare account that owns the Worker, from a repository secret such as `secrets.CLOUDFLARE_ACCOUNT_ID`. The job fails if it is empty. | +| `worker-name` | yes | none | The Worker to deploy to, one per site: lowercase letters, digits and dashes, at most 63 characters. It must already exist, with its `workers.dev` route on, and already be behind Access. | +| `account-subdomain` | yes | none | The account's `workers.dev` subdomain: `my-subdomain` for `*.my-subdomain.workers.dev`, which `my-subdomain.workers.dev` also gives. The URLs are built from it, never read from wrangler's output. | +| `team-domain` | yes | none | The Access team's login domain, `.cloudflareaccess.com`; a bare `` is accepted too. Every gate check requires an anonymous request to be redirected to exactly this host. | +| `build-dir` | yes | none | The directory that holds the built site. The job fails if it is missing or holds no files, and warns if it has no `index.html`, since the site's root would then answer 404. | +| `alias` | no | `''` | Also uploads the same build as a named preview alias, such as `report-2026-08`, for a permanent URL beside the production one, which moves with each deploy. Lowercase letters, digits and dashes, starting with a letter and not ending with a dash, and `-` must fit in 63 characters. Empty, the default, uploads no alias. | +| `require-access` | no | `true` | `'true'`, the default, proves the site is gated: before anything is uploaded, after the deploy, on production and on the new version's own preview URL, and on the alias after its upload, an anonymous request must be redirected to `team-domain`. `'false'` skips every check, with a warning: never use it for private content. | + + + +## Outputs + + + +| Output | Description | +|---|---| +| `deploy-url` | The production URL, `https://..workers.dev`. Set once the deploy succeeds, even if the gate check after it then fails the job. | +| `alias-url` | The alias's URL, `https://-..workers.dev`. Empty when no `alias` is given, or when its upload failed or did not run. | + + + +Both URLs are built from `worker-name`, `account-subdomain` and `alias`, never read from wrangler's output. + +## Examples + +### Minimal + +Rebuilds a members-only dashboard on each push to `main` and every morning, one deploy at a time: + +```yaml +name: Publish members dashboard +on: + push: + branches: [main] + schedule: + - cron: '0 6 * * *' + workflow_dispatch: + +concurrency: + group: deploy-cloudflare-${{ github.workflow }} + cancel-in-progress: false + +jobs: + deploy: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@v7 + - name: Build + run: ./build.sh # writes the site to _site/ + - uses: quantecon/actions/deploy-cloudflare@v0 + with: + cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} + cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + worker-name: members-dashboard + account-subdomain: my-subdomain # *.my-subdomain.workers.dev + team-domain: my-team.cloudflareaccess.com + build-dir: _site +``` + +### A permanent URL for each month + +A monthly report can keep each month's build at a URL of its own, beside the production URL, which always shows the latest: + +```yaml +- name: Name this month's alias + id: month + run: echo "alias=report-$(date -u +%Y-%m)" >> "$GITHUB_OUTPUT" +- uses: quantecon/actions/deploy-cloudflare@v0 + with: + cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} + cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + worker-name: monthly-report + account-subdomain: my-subdomain + team-domain: my-team.cloudflareaccess.com + build-dir: _build/html + alias: ${{ steps.month.outputs.alias }} # report-2026-08, for example +``` + +### In the templates + +No template uses this action: a private site is a choice made per site, not the default for a lecture repository. + +## Behaviour + +1. **Inputs.** Every input is checked before anything else runs: the secrets must be set, the names must be valid hostname labels, `team-domain` must be an Access login domain, and `build-dir` must hold at least one file. The action also picks the file to probe besides the site's root: see [The gate check](#the-gate-check). +2. **The check before the deploy.** With `require-access: 'true'`, the production URL must be gated. If it is not, the job fails with nothing uploaded. That refuses a Worker that does not exist, as a mistyped name would be, one that is public, and one gated by another Access organisation. +3. **wrangler.** The action sets up Node 24 if the runner lacks Node 22 or later, and installs the `wrangler` version pinned in its [`package.json`](https://github.com/QuantEcon/actions/blob/main/deploy-cloudflare/package.json) with `npm ci`, into a temporary directory. +4. **The configuration.** It writes a wrangler configuration for this one run, with the Worker's name, a fixed compatibility date, `build-dir` as the assets directory, and `workers_dev` and `preview_urls` both on. It declares no routes. +5. **The deploy.** `wrangler deploy` publishes `build-dir` to production. Every version also gets its own preview URL, `https://-..workers.dev`, read from the `Current Version ID` line of wrangler's output. +6. **The check after the deploy**, on production and on the new version's preview URL. It runs even when `wrangler deploy` fails, because wrangler can fail after the new version is already live. +7. **The alias**, if given: `wrangler versions upload --preview-alias ` uploads the same build as a preview, then its URL is checked too. +8. **The job summary** gives each URL and the result of its check, or says why nothing was deployed. + +With `require-access: 'false'`, none of the three checks runs, and a warning says that the site is not checked. + +### The gate check + +Each check is [`check-access-gate.sh`](https://github.com/QuantEcon/actions/blob/main/scripts/check-access-gate.sh), which sends an anonymous request and does not follow redirects: following one would land on the login page, which answers `200`, and a gated site would look the same as a public one. The answer decides: + +| Response | Verdict | +|---|---| +| `301`, `302`, `303`, `307` or `308`, to exactly `team-domain` | gated | +| A redirect to another `*.cloudflareaccess.com` | fails: gated by the wrong Access organisation | +| A redirect whose `Location` carries a user name or a backslash | fails: the real host is ambiguous, and Access never sends one | +| `2xx` | fails: **the site is public** | +| `404` | fails: no Worker answers on that hostname, or its `workers.dev` route is off | +| Anything else, or no answer | fails: the gate could not be verified | + +Each check requests the site's root and one file from `build-dir` that is not HTML: the first, in sorted order, with a URL-safe path, skipping dotfiles and the `_headers`, `_redirects` and `_worker.js` control files. A gate that protected only the root would otherwise pass. Access covers every path on a hostname, so a correct gate redirects both. + +### What the generated configuration means for the Worker + +- **Preview URLs are on** after every deploy, even if they were turned off in the dashboard, which is why the check after the deploy covers the new version's preview URL. +- **The deploy replaces the Worker's code**, such as the placeholder's script, without asking. Other settings changed in the dashboard can be overwritten too. +- **Custom domains** are a setting on the Worker in the dashboard, and its Access application covers them. The configuration declares no routes, so a deploy leaves them alone, and the action checks only the `workers.dev` URL. +- **Dotfiles** in `build-dir`, such as Sphinx's `.buildinfo`, are uploaded like any other file, unless a `.assetsignore` file in `build-dir` lists them. + +### Limits + +On the free plan a Worker's static assets can hold 20,000 files per version, and 25 MiB per file; paid plans allow 100,000 files. A Worker keeps its 1,000 most recent preview aliases. A lecture-sized Jupyter Book is a few thousand files. + +## Troubleshooting + +**`Refusing to deploy: … is not behind Access for …`.** Nothing was uploaded. The `FAIL` line above it says why: + +- `answered 404`: no Worker by that name answers under `account-subdomain`, or its `workers.dev` route is off. Check both, and create the Worker first if it is new. +- `THE SITE IS PUBLIC`: Access is off for the Worker, or set to *Previews only*. Turn it on with *All traffic*. +- `the Worker is attached to the wrong Access organisation`: the Worker's Access application belongs to another Zero Trust team, or `team-domain` is wrong. +- `not to the Access login domain`: the site redirected elsewhere, or sent no `Location`, so Access is not answering for this hostname. Check that Access is on with *All traffic*, and that `team-domain` is right. +- `expected a redirect to the Access login domain`: another status, such as `401`, `403` or a `5xx`. Check the Worker in the dashboard, then re-run the job. +- `could not be reached`: a network problem between the runner and Cloudflare. Re-run the job. + +**`wrangler deploy failed (exit …)`.** wrangler's own error is printed above it. An authentication or permission error usually means the token lacks Editor on this Worker, the account ID is wrong, or the token has expired. wrangler can fail after the new version is live, so the job summary gives the check that ran after the failure. + +**`A hostname serving this Worker is NOT behind Access for …`.** The urgent one: the new build is live, or may be, and an anonymous request to production or to the new version's preview URL was not redirected. The `FAIL` line above names the hostname. Turn Access on for the Worker with *All traffic*, which covers production and every preview URL, or turn off its `workers.dev` route. Then re-run the job to confirm. + +**`wrangler's output named no 'Current Version ID'`.** The new version's preview URL could not be worked out, so it was not checked, and the check fails rather than pass unseen. wrangler prints the ID only once every step after the upload has succeeded, so this usually follows a failed `wrangler deploy`. After a successful one it means a wrangler update changed its output. Check the preview URL by hand with `check-access-gate.sh`, with the version ID from the dashboard. + +**`The alias is uploaded but … is NOT behind Access for …`.** Production is gated but previews are not. Set the Worker's Access to *All traffic*, which covers preview URLs. + +**An alias is refused before anything runs.** It must start with a lowercase letter and hold only lowercase letters, digits and dashes, and `-` must fit in 63 characters. Use `report-2026-08`, not `2026-08`. + +**`cloudflare-api-token is empty`.** The secret is missing, or the run came from a fork or from Dependabot, which get no secrets. + +**A blank page after access is granted.** On the first load after Access lets a visitor in, cached redirects can be served in place of the page's stylesheets, scripts and data. Reloading the page fixes it. + +**A person added to the team is still refused.** Cloudflare documents that a user refused before joining can stay refused until they revoke the OAuth app in their GitHub settings, under **Applications**, and log in again. diff --git a/docs/user/actions/preview-cloudflare.md b/docs/user/actions/preview-cloudflare.md new file mode 100644 index 0000000..59bb1cc --- /dev/null +++ b/docs/user/actions/preview-cloudflare.md @@ -0,0 +1,157 @@ +# preview-cloudflare + +> This describes `main`. For the version you pin, open the manual at that tag. Changes not yet released are under `[Unreleased]` in the [CHANGELOG](https://github.com/QuantEcon/actions/blob/main/CHANGELOG.md). + +Deploys a pull request's built site to a Cloudflare Pages project as a preview, on the branch `pr-`, so the preview's URL, `https://pr-..pages.dev`, stays the same for every push to the pull request. Each deployment also gets a URL of its own, which goes on showing that one push. The action then posts one comment on the pull request, and updates it on each later push, with: + +- the preview URL and the commit it shows; +- a link to the page of each lecture the pull request adds or modifies, under "Changed Lectures"; +- the URL of this push's own deployment, under "Build Info". + +It deploys only on `pull_request` events, and skips pull requests from forks and from Dependabot, whose runs get no secrets. The project's production site is never touched. + +## When to use it + +Use it in a pull request's build job, after [`build-lectures`](build-lectures.md), in place of the Netlify step in [`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml), which carries this step as a comment. + +- **Or on Netlify.** [`preview-netlify`](preview-netlify.md) does the same on Netlify, and is the one `ci.yml` uses. [Netlify or Cloudflare?](preview-netlify.md#netlify-or-cloudflare) compares the two. +- **Not to publish the site.** [`publish-gh-pages`](publish-gh-pages.md) publishes to GitHub Pages, and [`deploy-cloudflare`](deploy-cloudflare.md) publishes a site that only members may see. + +## Requirements + +- **Runner.** Any, with `bash`, `git` and `npm`, as GitHub-hosted runners and the QuantEcon images have. The job must be able to reach `registry.npmjs.org`: `wrangler` is installed each run from the lockfile beside the action. +- **Node.** 22 or later, which the pinned `wrangler` needs. The QuantEcon images carry Node 24. On a standard runner, set it up before this action with `actions/setup-node`, as in [preview-netlify's example](preview-netlify.md#on-a-standard-runner). +- **Permissions.** `pull-requests: write`, for the comment. The job also needs `contents: read` for `actions/checkout`. +- **Secrets.** `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID`, from [Setting up Cloudflare Pages](#setting-up-cloudflare-pages). +- **Files in your repository.** None beyond the built site. The comment's lecture links need the pull request's base and head commits, which the action fetches if the checkout lacks them. +- **Setup outside GitHub.** A Cloudflare Pages project, once: see [Setting up Cloudflare Pages](#setting-up-cloudflare-pages). + +## Setting up Cloudflare Pages + +1. **A Cloudflare account.** A free one is enough to start. +2. **Create a project that Cloudflare does not build.** In the dashboard, under **Workers & Pages**, create a Pages project with **Upload assets**, which is direct upload, and upload any file as a placeholder. Its name is the `project-name` input and part of every URL, so choose it with care. Such a project has no link to the repository, so Cloudflare neither builds your pull requests nor comments on them: this action does both. +3. **Find the account ID.** It is in the dashboard's URL, `https://dash.cloudflare.com//…`, and on the **Workers & Pages** overview. +4. **Create an API token** under **My Profile**, **API Tokens**, as a custom token with a single permission, **Account**, **Cloudflare Pages**, **Edit**, for the account that owns the project. Copy it when it is shown: Cloudflare shows it only once. +5. **Add both as repository secrets**, under the repository's **Settings**, **Secrets and variables**, **Actions**: + + | Secret | Value | + |---|---| + | `CLOUDFLARE_API_TOKEN` | the token from step 4 | + | `CLOUDFLARE_ACCOUNT_ID` | the account ID from step 3 | + +The previews then appear at: + +| Deployment | URL | +|---|---| +| Pull request 5, newest push | `https://pr-5..pages.dev` | +| One push | `https://..pages.dev` | +| The project's production site, which the action leaves alone | `https://.pages.dev` | + +## Inputs + + + +| Input | Required | Default | Description | +|---|---|---|---| +| `cloudflare-api-token` | yes | none | A Cloudflare API token that can edit Cloudflare Pages in the account, from a repository secret such as `secrets.CLOUDFLARE_API_TOKEN`. | +| `cloudflare-account-id` | yes | none | The ID of the Cloudflare account that owns the Pages project, from a repository secret such as `secrets.CLOUDFLARE_ACCOUNT_ID`. | +| `project-name` | yes | none | The Cloudflare Pages project to deploy to, which must already exist. Its name is part of every preview URL, as in `.pages.dev`. | +| `build-dir` | yes | none | The directory that holds the built site, such as `_build/html`, which `build-lectures` gives in its `build-path` output. | +| `lectures-dir` | no | `lectures` | The directory of lecture `.md` files that change detection looks in: each one the pull request adds or modifies gets a link in the comment, and is listed in `changed-files`. Empty turns change detection off, and the comment gives only the preview URL. | + + + +## Outputs + + + +| Output | Description | +|---|---| +| `deploy-url` | The pull request's preview URL, `https://pr-..pages.dev`, which always shows its newest push, so use it for anything a person will follow. Empty when nothing was deployed: on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | +| `deployment-url` | The URL of this one deployment, `https://..pages.dev`, which goes on showing this push after later ones. Empty when wrangler's output does not name it, and when nothing was deployed. | +| `changed-files` | The lecture files that the pull request adds or modifies under `lectures-dir`, one path per line. Empty when there are none, with `lectures-dir: ''`, on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | + + + +## Examples + +### Minimal + +A pull request's build in a container, deployed as a preview: + +```yaml +name: Preview +on: + pull_request: + +jobs: + preview: + runs-on: ubuntu-latest + container: + image: ghcr.io/quantecon/quantecon-build:latest + permissions: + contents: read + pull-requests: write # the preview comment + packages: read # the image pull, as the templates grant it + steps: + - uses: actions/checkout@v7 + - uses: quantecon/actions/setup-environment@v0 + - uses: quantecon/actions/build-lectures@v0 + id: build + - uses: quantecon/actions/preview-cloudflare@v0 + with: + cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} + cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + project-name: my-lectures + build-dir: ${{ steps.build.outputs.build-path }} +``` + +### In the templates + +[`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml) carries this action as a commented alternative to its `preview-netlify` step. To use it, replace that step with the comment's, set `project-name`, and add the two secrets. + +## Behaviour + +1. **Who gets a preview.** A run by Dependabot, or for a pull request from a fork, stops here: it prints `⚠️ Cloudflare deployment skipped (untrusted actor or fork PR)`, deploys nothing, and passes. +2. **Changed lectures.** On a `pull_request` event with `lectures-dir` set, the action lists each `.md` file under `lectures-dir` that the pull request adds or modifies, as [preview-netlify](preview-netlify.md#behaviour) does. +3. **The CLI.** The `wrangler` version pinned in the action's [`package.json`](https://github.com/QuantEcon/actions/blob/main/preview-cloudflare/package.json) is installed with `npm ci`, into a temporary directory, and its `wrangler` command is added to the `PATH` of the job's later steps. +4. **The deploy**, on `pull_request` events only: + + ```text + wrangler pages deploy --project-name --branch pr- --commit-hash --commit-message "PR # ()" + ``` + + The token and the account ID reach it through the environment. A branch other than the project's production branch makes the deployment a preview, so the production site is untouched. wrangler's output is in the "Deployment Output" log group. +5. **The URLs.** `deploy-url` is not read from wrangler's output but built from the inputs: `https://pr-..pages.dev` is the alias Cloudflare gives the newest deployment of the branch `pr-`. `deployment-url` is the first other `pages.dev` address in wrangler's output. +6. **The comment.** The action updates the pull request's comment that starts `## ☁️ Cloudflare Preview Ready!`, or posts one, with the lecture links built as in preview-netlify's. + +On any event but `pull_request`, the action installs the CLI and does nothing else. + +### What fails the job + +- `npm ci` failing to install the pinned CLI. +- `wrangler pages deploy` failing. +- The comment failing to post, for want of `pull-requests: write`. + +## Security + +Previews of pull requests from forks are not supported, and the action skips them. Do not run it from a `pull_request_target` workflow to get around that. A workflow triggered that way builds and runs the fork's notebooks with `CLOUDFLARE_API_TOKEN` and a write-scoped `GITHUB_TOKEN` within their reach, the pattern known as a "pwn request". It would not produce a preview either: the action deploys only on `pull_request` events. + +## Troubleshooting + +**`npm ci of the pinned wrangler failed — see the npm output above (it needs registry.npmjs.org)`.** The runner cannot reach the npm registry, or npm is missing. + +**`wrangler installed but does not run — see the output above`.** Usually a Node older than 22. Set up Node 24 before this action. + +**`wrangler pages deploy failed (exit …) — see the Deployment Output group above`.** wrangler's own message is in that group. + +- An authentication error: the token lacks **Cloudflare Pages**, **Edit**, has expired, or belongs to another account than `CLOUDFLARE_ACCOUNT_ID`. +- A project that cannot be found: `project-name` must match the project's name exactly, in the account that `CLOUDFLARE_ACCOUNT_ID` names. + +**The preview URL answers 404.** A new alias can take a few seconds to appear. If it lasts, check that `build-dir` holds an `index.html`. + +**`Resource not accessible by integration`, when commenting.** The job lacks `pull-requests: write`. + +**The comment lists no changed lectures, or a link leads to a missing page.** See [preview-netlify's troubleshooting](preview-netlify.md#troubleshooting): the two actions detect and link lectures the same way. + +**The comment has no "This deployment" line.** wrangler's output named no per-deployment URL, so `deployment-url` is empty. The preview URL is not affected. diff --git a/docs/user/actions/preview-netlify.md b/docs/user/actions/preview-netlify.md new file mode 100644 index 0000000..9a44e71 --- /dev/null +++ b/docs/user/actions/preview-netlify.md @@ -0,0 +1,176 @@ +# preview-netlify + +> This describes `main`. For the version you pin, open the manual at that tag. Changes not yet released are under `[Unreleased]` in the [CHANGELOG](https://github.com/QuantEcon/actions/blob/main/CHANGELOG.md). + +Deploys a pull request's built site to a Netlify site as a preview, under the alias `pr-`, so the preview's URL, `https://pr---.netlify.app`, stays the same for every push to the pull request. It then posts one comment on the pull request, and updates it on each later push, with: + +- the preview URL and the commit it shows; +- a link to the page of each lecture the pull request adds or modifies, under "Changed Lectures". + +It deploys only on `pull_request` events, and skips pull requests from forks and from Dependabot, whose runs get no secrets. The Netlify site's production deploy is never touched. + +## When to use it + +Use it in a pull request's build job, after [`build-lectures`](build-lectures.md), as [`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml) does. + +- **Not to publish the site.** [`publish-gh-pages`](publish-gh-pages.md) publishes to GitHub Pages, and [`deploy-cloudflare`](deploy-cloudflare.md) publishes a site that only members may see. +- **Or on Cloudflare.** [`preview-cloudflare`](preview-cloudflare.md) does the same on Cloudflare Pages. The table below compares the two. + +### Netlify or Cloudflare? + +| | `preview-netlify` | `preview-cloudflare` | +|---|---|---| +| Preview URL | `https://pr---.netlify.app` | `https://pr-..pages.dev` | +| A URL for each push | no | yes, in `deployment-url` and in the comment | +| Repository secrets | `NETLIFY_AUTH_TOKEN`, `NETLIFY_SITE_ID` | `CLOUDFLARE_API_TOKEN`, `CLOUDFLARE_ACCOUNT_ID` | +| Other inputs it needs | none | `project-name` | +| CLI, installed each run | `netlify-cli`, which needs Node 22.13 or later | `wrangler`, which needs Node 22 or later | +| In `ci.yml` | the preview step | a commented alternative | + +Both deploy the files `build-lectures` built, with no build on the provider's side, and neither needs the provider to be linked to the repository. For what each provider's plans allow, see its pricing page. + +## Requirements + +- **Runner.** Any, with `bash`, `git`, `python3` and `npm`, as GitHub-hosted runners and the QuantEcon images have. The job must be able to reach `registry.npmjs.org`: `netlify-cli` is installed each run from the lockfile beside the action. +- **Node.** 22.13 or later, which the pinned `netlify-cli` needs. The QuantEcon images carry Node 24. On a standard runner, set it up before this action with `actions/setup-node`; see [On a standard runner](#on-a-standard-runner). +- **Permissions.** `pull-requests: write`, for the comment. The job also needs `contents: read` for `actions/checkout`. +- **Secrets.** `NETLIFY_AUTH_TOKEN` and `NETLIFY_SITE_ID`, from [Setting up Netlify](#setting-up-netlify). +- **Files in your repository.** None beyond the built site. The comment's lecture links need the pull request's base and head commits, which the action fetches if the checkout lacks them. +- **Setup outside GitHub.** A Netlify site, once: see [Setting up Netlify](#setting-up-netlify). + +## Setting up Netlify + +1. **Create a site that Netlify does not build.** In Netlify, add a new site with **Deploy manually**, and drop any folder on it as a placeholder. Such a site has no link to the repository, so Netlify neither builds your pull requests nor comments on them: this action does both. +2. **Create a personal access token**, under **User settings**, **Applications**, **Personal access tokens**. Give it a name that says what it is for, such as `github-actions`, and copy it when it is shown: Netlify shows it only once. +3. **Find the site ID**, under the site's **Site configuration**, **General**, **Site details**. It is a UUID, such as `a1b2c3d4-e5f6-…`. +4. **Add both as repository secrets**, under the repository's **Settings**, **Secrets and variables**, **Actions**: + + | Secret | Value | + |---|---| + | `NETLIFY_AUTH_TOKEN` | the token from step 2 | + | `NETLIFY_SITE_ID` | the site ID from step 3 | + +A site that is already linked to the repository builds each pull request itself, and comments on it, beside this action. To stop that, do one of these under the site's **Site configuration**: + +- **Unlink the repository**, under **Build & deploy**, **Continuous deployment**. The site keeps serving, but Netlify no longer builds or comments. +- **Turn off Netlify's pull-request comments** only, under **Notifications**: remove its GitHub commit and pull-request comments. +- **Turn off deploy previews**, under **Build & deploy**, **Continuous deployment**, **Branches and deploy contexts**: set **Deploy Previews** to **None**. + +## Inputs + + + +| Input | Required | Default | Description | +|---|---|---|---| +| `netlify-auth-token` | yes | none | A Netlify personal access token, from a repository secret such as `secrets.NETLIFY_AUTH_TOKEN`. It reaches `netlify-cli` through the environment, never on its command line. | +| `netlify-site-id` | yes | none | The ID of the Netlify site to deploy to, from a repository secret such as `secrets.NETLIFY_SITE_ID`. | +| `build-dir` | yes | none | The directory that holds the built site, such as `_build/html`, which `build-lectures` gives in its `build-path` output. | +| `lectures-dir` | no | `lectures` | The directory of lecture `.md` files that change detection looks in: each one the pull request adds or modifies gets a link in the comment, and is listed in `changed-files`. Empty turns change detection off, and the comment gives only the preview URL. | + + + +## Outputs + + + +| Output | Description | +|---|---| +| `deploy-url` | The preview's URL, `https://pr---.netlify.app`, the same for every push to the pull request. Empty when nothing was deployed: on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | +| `changed-files` | The lecture files that the pull request adds or modifies under `lectures-dir`, one path per line. Empty when there are none, with `lectures-dir: ''`, on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | + + + +## Examples + +### Minimal + +A pull request's build in a container, deployed as a preview: + +```yaml +name: Preview +on: + pull_request: + +jobs: + preview: + runs-on: ubuntu-latest + container: + image: ghcr.io/quantecon/quantecon-build:latest + permissions: + contents: read + pull-requests: write # the preview comment + packages: read # the image pull, as the templates grant it + steps: + - uses: actions/checkout@v7 + - uses: quantecon/actions/setup-environment@v0 + - uses: quantecon/actions/build-lectures@v0 + id: build + - uses: quantecon/actions/preview-netlify@v0 + with: + netlify-auth-token: ${{ secrets.NETLIFY_AUTH_TOKEN }} + netlify-site-id: ${{ secrets.NETLIFY_SITE_ID }} + build-dir: ${{ steps.build.outputs.build-path }} +``` + +### On a standard runner + +Set up Node first: + +```yaml +- uses: actions/setup-node@v7 + with: + node-version: '24' +- uses: quantecon/actions/preview-netlify@v0 + with: + netlify-auth-token: ${{ secrets.NETLIFY_AUTH_TOKEN }} + netlify-site-id: ${{ secrets.NETLIFY_SITE_ID }} + build-dir: _build/html +``` + +### In the templates + +[`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml) deploys each pull request's build with this action, with `preview-cloudflare` as a commented alternative. + +## Behaviour + +1. **Who gets a preview.** A run by Dependabot, or for a pull request from a fork, stops here: it prints `⚠️ Netlify deployment skipped (untrusted actor or fork PR)`, deploys nothing, and passes. +2. **Changed lectures.** On a `pull_request` event with `lectures-dir` set, the action compares the pull request's base and head commits. It lists each `.md` file under `lectures-dir` that the pull request adds or modifies, except `/intro.md` and files whose path within `lectures-dir` starts with `_`. A renamed lecture counts as added; a deleted one is not listed. +3. **The CLI.** The `netlify-cli` version pinned in the action's [`package.json`](https://github.com/QuantEcon/actions/blob/main/preview-netlify/package.json) is installed with `npm ci`, into a temporary directory, and its `netlify` command is added to the `PATH` of the job's later steps. +4. **The deploy**, on `pull_request` events only: + + ```text + netlify deploy --no-build --dir --alias pr- --message "PR # ()" --json + ``` + + The token and the site ID reach it through the environment. `--no-build` stops Netlify from running a build of its own, and without `--prod` the deploy is a preview, so the site's production deploy is untouched. Netlify's output is in the "Deployment Output" log group. +5. **The comment.** The action updates the pull request's comment that starts `## 📖 Netlify Preview Ready!`, or posts one. Each changed lecture links to `/.html`. `` is the file's path without `.md`, and also without the `lectures-dir/` in front when `_toc.yml` is in `lectures-dir`, because Jupyter Book then builds the pages relative to it. + +On any event but `pull_request`, the action installs the CLI and does nothing else. + +### What fails the job + +- `npm ci` failing to install the pinned CLI. +- `netlify deploy` failing, or printing no deploy URL. +- The comment failing to post, for want of `pull-requests: write`. + +## Security + +Previews of pull requests from forks are not supported, and the action skips them. Do not run it from a `pull_request_target` workflow to get around that. A workflow triggered that way builds and runs the fork's notebooks with `NETLIFY_AUTH_TOKEN` and a write-scoped `GITHUB_TOKEN` within their reach, the pattern known as a "pwn request". It would not produce a preview either: the action deploys only on `pull_request` events. + +## Troubleshooting + +**`npm ci of the pinned netlify-cli failed — see the npm output above (it needs registry.npmjs.org)`.** The runner cannot reach the npm registry, or npm is missing. + +**`netlify-cli installed but does not run — see the output above`.** Usually a Node older than 22.13. Set up Node 24 before this action. + +**`netlify deploy failed (exit …) — see the Deployment Output group above`.** Netlify's own message is in that group. An authorisation error means the token is wrong, revoked or expired; a site that cannot be found means `NETLIFY_SITE_ID` is wrong. Check also that `build-dir` exists. + +**`Resource not accessible by integration`, when commenting.** The job lacks `pull-requests: write`. + +**The comment lists no changed lectures.** The pull request adds or modifies no `.md` file under `lectures-dir`, other than `intro.md` and files starting with `_`, or `lectures-dir` names the wrong directory. In a container job, git must also be able to read the checkout: `build-lectures` makes git trust it for the rest of the job, so run this action after it, or trust the workspace yourself with `git config --global --add safe.directory "$GITHUB_WORKSPACE"`. + +**A lecture link leads to a missing page.** The links assume Jupyter Book's layout: pages relative to `lectures-dir` when `_toc.yml` is inside it, and relative to the repository root when it is not. + +**Two comments on each pull request, one of them from Netlify.** The Netlify site is linked to the repository: see the end of [Setting up Netlify](#setting-up-netlify). + +**A pull request from a fork got no preview.** Expected: its runs get no secrets, so the action skips them. See [Security](#security). diff --git a/docs/user/actions/publish-gh-pages.md b/docs/user/actions/publish-gh-pages.md new file mode 100644 index 0000000..68501be --- /dev/null +++ b/docs/user/actions/publish-gh-pages.md @@ -0,0 +1,236 @@ +# publish-gh-pages + +> This describes `main`. For the version you pin, open the manual at that tag. Changes not yet released are under `[Unreleased]` in the [CHANGELOG](https://github.com/QuantEcon/actions/blob/main/CHANGELOG.md). + +Publishes a built site to the repository's GitHub Pages site, through GitHub's own Pages deploy: [configure-pages](https://github.com/actions/configure-pages), [upload-pages-artifact](https://github.com/actions/upload-pages-artifact) and [deploy-pages](https://github.com/actions/deploy-pages). The site goes to Pages as an artifact of the run, so there is no `gh-pages` branch, and the repository does not grow with every publish. + +On a run triggered by a tag it can also attach the site to that tag's GitHub release, as an archive, its SHA-256 checksum and a manifest. Their format is described in [Release assets](#release-assets), and is a contract that other tools can rely on. + +## When to use it + +Use it to publish a public site, after [`build-lectures`](build-lectures.md), as [`publish.yml`](https://github.com/QuantEcon/actions/blob/main/templates/publish.yml) does for each push to `main`. + +- **Not for pull requests.** [`preview-netlify`](preview-netlify.md) or [`preview-cloudflare`](preview-cloudflare.md) gives each pull request a preview of its own. +- **Not for a site only members may see.** Every GitHub Pages site is public, unless the organisation is on GitHub Enterprise Cloud. [`deploy-cloudflare`](deploy-cloudflare.md) publishes behind a login instead. + +## Requirements + +- **Runner.** Any. +- **Node.** None. +- **Permissions.** `pages: write` and `id-token: write`, for the deploy. The job also needs `contents: read` for `actions/checkout`, or `contents: write` with `create-release-assets: 'true'`, for the release. +- **Secrets.** None. The deploy authenticates with the job's OIDC token, and the release upload with the `github-token` you pass, which can be the job's own `GITHUB_TOKEN`. +- **Files in your repository.** None beyond the built site. +- **Repository settings.** + - Under **Settings**, **Pages**, set **Source** to **GitHub Actions**, not **Deploy from a branch**. + - Run the job in the `github-pages` environment, with `environment: github-pages`, as the templates do. The environment's protection rules decide which branches and tags may deploy, and by default only the default branch may. + - A custom domain is set under **Settings**, **Pages**, **Custom domain**; see [Custom domain](#custom-domain). +- **Setup outside GitHub.** Only for a custom domain: its DNS records, at the domain's provider. + +## Inputs + + + +| Input | Required | Default | Description | +|---|---|---|---| +| `build-dir` | yes | none | The directory that holds the built site, such as `_build/html`, which `build-lectures` gives in its `build-path` output. The job fails if it does not exist. | +| `cname` | no | `''` | A domain to write into `build-dir` as a `CNAME` file, which also lands in the release archive. It does not set the site's custom domain, and a warning says so: a GitHub Actions Pages deploy ignores `CNAME` files, so set the domain under Settings, Pages. Empty, the default, writes no file. | +| `create-release-assets` | no | `false` | `'true'` also attaches the site to the GitHub release of the tag the run was triggered by, creating the release if there is none: an archive, its SHA-256 checksum and a manifest. It needs `github-token`, and a run triggered by a tag: any other run skips it, with a warning. `'false'`, the default, attaches nothing. | +| `asset-name` | no | `''` | With `create-release-assets: 'true'`: the start of each asset's file name. Empty, the default, uses the repository's name followed by `-html`, as in `-html`. | +| `github-token` | no | `''` | With `create-release-assets: 'true'`: a token that can write the repository's releases, such as `secrets.GITHUB_TOKEN` in a job with `contents: write`. The job fails if it is empty. The Pages deploy does not use it. | + + + +## Outputs + + + +| Output | Description | +|---|---| +| `page-url` | The URL of the published site, as GitHub Pages reports it: `https://.github.io//`, or the custom domain's. Set once the deploy succeeds. | + + + +## Examples + +### Minimal + +Publishes each push to `main`, one run at a time: + +```yaml +name: Publish +on: + push: + branches: [main] + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + publish: + runs-on: ubuntu-latest + container: + image: ghcr.io/quantecon/quantecon:latest + permissions: + contents: read + pages: write # the Pages deploy + id-token: write # the Pages deploy's OIDC token + packages: read # the image pull, as the templates grant it + environment: + name: github-pages + url: ${{ steps.publish.outputs.page-url }} + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 # each page's "Last changed" date comes from git log + - uses: quantecon/actions/setup-environment@v0 + - uses: quantecon/actions/build-lectures@v0 + id: build + - uses: quantecon/actions/publish-gh-pages@v0 + id: publish + with: + build-dir: ${{ steps.build.outputs.build-path }} +``` + +`cancel-in-progress: false` lets a publish that has started finish, while a newer one waits its turn. + +### With release assets + +Publishes on each tag that starts with `publish`, and attaches the site to the tag's release: + +```yaml +name: Publish +on: + push: + tags: ['publish*'] + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + publish: + runs-on: ubuntu-latest + container: + image: ghcr.io/quantecon/quantecon:latest + permissions: + contents: write # the release assets + pages: write + id-token: write + packages: read + environment: + name: github-pages + url: ${{ steps.publish.outputs.page-url }} + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 + - uses: quantecon/actions/setup-environment@v0 + - uses: quantecon/actions/build-lectures@v0 + id: build + - uses: quantecon/actions/publish-gh-pages@v0 + id: publish + with: + build-dir: ${{ steps.build.outputs.build-path }} + create-release-assets: 'true' + github-token: ${{ secrets.GITHUB_TOKEN }} +``` + +The `github-pages` environment must also allow the tags to deploy: under **Settings**, **Environments**, **github-pages**, **Deployment branches and tags**, add the pattern `publish*`. + +### In the templates + +[`publish.yml`](https://github.com/QuantEcon/actions/blob/main/templates/publish.yml) publishes each push to `main`, and runs on manual dispatch too, after restoring the build cache and building the site. Release assets are there as comments, with the three changes a workflow needs before it can make them: a tag trigger, `contents: write`, and the tag pattern allowed in the `github-pages` environment. + +## Behaviour + +1. **Checks.** `build-dir` must exist. The action prints how many files it holds and its size. +2. **`cname`.** When set, the domain is written to `/CNAME`, with a warning that the deploy ignores it. +3. **The deploy.** configure-pages reads the repository's Pages settings, upload-pages-artifact packs `build-dir` as the run's `github-pages` artifact, and deploy-pages publishes it and reports the URL in `page-url`. +4. **Release assets**, with `create-release-assets: 'true'`, after the deploy: + - On a run not triggered by a tag, a warning says they are skipped, and the job carries on. + - With `github-token` empty, the job fails. + - Otherwise the action writes the three files in [Release assets](#release-assets), and uploads them with [action-gh-release](https://github.com/softprops/action-gh-release) to the release named after the tag, which it creates if there is none. +5. **Summary.** A "Deployment Summary" log group gives the page URL, the `CNAME` domain if any, and the release's URL if assets were uploaded. + +The site is live before the release assets are made, so a failure in making or uploading them leaves the new site published. + +## Release assets + +With `create-release-assets: 'true'`, a run triggered by the tag `` attaches three files to that tag's release, where `` is the `asset-name` input or, when that is empty, `-html`: + +| File | Contents | +|---|---| +| `-.tar.gz` | The whole of `build-dir`, including any `CNAME` file, as a gzip-compressed tar whose paths are relative to `build-dir`, as in `./index.html`. | +| `-checksum.txt` | One line in `sha256sum`'s format: the archive's SHA-256 in hexadecimal, two spaces, and the archive's file name, with no directory. | +| `-manifest.json` | A JSON object that describes the archive, with the fields below. | + +| Manifest field | Type | Value | +|---|---|---| +| `name` | string | `` | +| `tag` | string | `` | +| `commit` | string | The full SHA of the commit the run built. | +| `timestamp` | string | When the assets were made: ISO 8601 date and time to the second, with the UTC offset, as in `2026-09-28T10:04:05+00:00`. | +| `size_mb` | number | The disk space `build-dir` takes, in mebibytes, as `du -sm` reports it: a whole number, rounded up. | +| `file_count` | number | The number of files in `build-dir`, not counting a `CNAME` file that `cname` writes. | +| `repository` | string | `/`. | + +For example: + +```json +{ + "name": "lecture-python-html", + "tag": "publish-2026-09-28", + "commit": "", + "timestamp": "2026-09-28T10:04:05+00:00", + "size_mb": 164, + "file_count": 1342, + "repository": "/" +} +``` + +To check an archive, run `sha256sum` in the directory that holds both files: + +```bash +sha256sum --check lecture-python-html-checksum.txt +``` + +This is the contract. A release of this project that changes a file name, the archive's layout, the checksum's format, or a manifest field's name or meaning says so in its CHANGELOG entry. Read the manifest as JSON, and ignore any field you do not know, so that a field added later does no harm. The checksum's and the manifest's names do not carry the tag, so keep each release's files in a directory of their own. + +## Custom domain + +1. **Set the domain** under **Settings**, **Pages**, **Custom domain**. A GitHub Actions deploy reads it only from there, and ignores any `CNAME` file in the site, which is why `cname` has no effect on it. +2. **Point the domain's DNS at GitHub Pages**, at the domain's provider: for a subdomain such as `lectures.example.org`, a `CNAME` record pointing at `.github.io`. GitHub's [custom domain guide](https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site) gives the records for an apex domain. +3. **Turn on Enforce HTTPS** in the same settings, once GitHub has issued the certificate, which can take a while after the DNS change. + +## Moving from a `gh-pages` branch + +A repository that publishes by pushing to a branch, with peaceiris/actions-gh-pages or a version of this action that took a `target-branch` input, moves over in five steps: + +1. Under **Settings**, **Pages**, change **Source** from **Deploy from a branch** to **GitHub Actions**. +2. Give the job `pages: write` and `id-token: write`, and run it in the `github-pages` environment. It no longer needs `contents: write`, unless it makes release assets. +3. Replace the deploy step with this action, passing only `build-dir`. The deploy needs no token. +4. Set the custom domain under **Settings**, **Pages**: a `CNAME` file in the site no longer applies. +5. Once the new deploy is live, delete the `gh-pages` branch, if you like, to shrink the repository. + +## Troubleshooting + +**`❌ Error: Build directory not found: …`.** `build-dir` is wrong, or the build did not run. Pass `build-lectures`' `build-path` output. + +**`Get Pages site failed`, from configure-pages.** GitHub Pages is not enabled with the GitHub Actions source. Set **Settings**, **Pages**, **Source** to **GitHub Actions**. + +**The deploy fails asking for `id-token: write`, or for `pages: write`.** A `permissions:` block drops every scope it does not list: add both. + +**`Tag "…" is not allowed to deploy to github-pages due to environment protection rules`.** Allow the tag pattern under **Settings**, **Environments**, **github-pages**, **Deployment branches and tags**. A branch other than the default is refused the same way. + +**`create-release-assets is enabled but this run was not triggered by a tag …; skipping release assets`.** Release assets need a tag trigger, such as `tags: ['publish*']` under `on.push`. + +**`create-release-assets is enabled but 'github-token' is empty`.** Pass `github-token: ${{ secrets.GITHUB_TOKEN }}`. + +**The release upload fails with `Resource not accessible by integration`, or a 403.** The job lacks `contents: write`. + +**A warning that `cname has no effect on the GitHub Pages deployment`.** Set the domain under **Settings**, **Pages**, and drop `cname`, unless you want the `CNAME` file in the release archive. + +**The site answers 404.** Check that the Pages source is **GitHub Actions**, and that `build-dir` has an `index.html` at its top. A new site can take a minute or two to appear. + +**The custom domain does not resolve.** Check the DNS records, and allow for DNS to propagate, which can take up to a day. The domain must be set under **Settings**, **Pages**. diff --git a/docs/user/actions/restore-jupyter-cache.md b/docs/user/actions/restore-jupyter-cache.md new file mode 100644 index 0000000..17ae1fd --- /dev/null +++ b/docs/user/actions/restore-jupyter-cache.md @@ -0,0 +1,171 @@ +# restore-jupyter-cache + +> This describes `main`. For the version you pin, open the manual at that tag. Changes not yet released are under `[Unreleased]` in the [CHANGELOG](https://github.com/QuantEcon/actions/blob/main/CHANGELOG.md). + +Restores a cache that [`build-jupyter-cache`](build-jupyter-cache.md) saved, before a build, so the build starts from the last cache build instead of from nothing. Jupyter Book then re-executes only the notebooks whose code has changed, and Sphinx rewrites only the pages that have. It restores one of two caches, chosen with `cache-type`: + +- **`build`**, the default: the whole `_build` directory, with the HTML, the PDF, the notebooks and the execution cache. +- **`execution`**: only `_build/.jupyter_cache`, which holds each notebook's executed outputs. It is much smaller, and saves the execution time, but not the rest of the build. + +By default it only restores. With `save-cache: 'true'` it also saves a cache at the end of the job, which only later runs of the same pull request can restore. + +## When to use it + +Use it in a job that builds lectures, after `setup-environment` and before [`build-lectures`](build-lectures.md): in the pull-request builds of [`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml), and in the publish builds of [`publish.yml`](https://github.com/QuantEcon/actions/blob/main/templates/publish.yml). + +- **Not in the cache build itself.** `build-jupyter-cache` builds from scratch on purpose; see its chapter. +- **Once per job.** Every `build-lectures` step after it builds on the restored `_build`. + +## Requirements + +- **Runner.** Any runner or container. +- **Node.** None. +- **Permissions.** None: restoring and saving a cache need no `permissions:`. The job needs `contents: read` for `actions/checkout`. +- **Secrets.** None. +- **Files in your repository.** Only what the key hashes: for the build cache, the same `environment` and `environment-update` files that `build-jupyter-cache` was given. +- **A cache to restore.** `build-jupyter-cache` must have succeeded on the default branch, or on the branch a pull request targets. GitHub removes a cache that goes 7 days without being restored. +- **Setup outside GitHub.** None. + +## Inputs + + + +| Input | Required | Default | Description | +|---|---|---|---| +| `cache-type` | no | `build` | What to restore. `build`, the default, restores the whole `_build` directory the last cache build saved: its HTML, PDF and notebooks, and the execution cache in `_build/.jupyter_cache`. `execution` restores only `_build/.jupyter_cache`, which holds each notebook's executed outputs. | +| `path` | no | `_build` | The build directory to restore into. The path is part of what identifies a cache, so only `_build`, the default, finds the caches that `build-jupyter-cache` saves; any other path finds only what this action saved with `save-cache` and the same `path`. With `cache-type: execution` the cache goes into `/.jupyter_cache`. | +| `source-dir` | no | `lectures` | With `cache-type: execution`: the book's directory. The key hashes every `.md` file under it, as `build-jupyter-cache` does with its own `source-dir`. Ignored for the build cache. | +| `environment` | no | `environment.yml` | With `cache-type: build`: the Conda environment file whose hash is part of the key. Give the same file that `build-jupyter-cache` was given, or no cache matches. Ignored for the execution cache. | +| `environment-update` | no | `''` | With `cache-type: build`: the container delta file whose hash is part of the key. Give the same file that `build-jupyter-cache` was given; with another, only the fallback matches, restoring the newest cache built from the same `environment`. Ignored for the execution cache. | +| `key` | no | `''` | A key to restore in place of the generated ones. It is matched as a prefix, so the newest cache whose key starts with it is restored, and with `save-cache: 'true'` the cache is saved under exactly this key. Empty, the default, uses the generated keys. | +| `fail-on-miss` | no | `false` | `'true'` fails the job when nothing is restored. `'false'`, the default, carries on without a cache, and the build starts from scratch. | +| `save-cache` | no | `false` | `'true'` also saves the directory at the end of the job, if the job succeeds, under a key that ends in the run id. GitHub scopes a cache to the branch or pull request that saved it, so only later runs of the same pull request restore it. `'false'`, the default, only restores. | + + + +## Outputs + + + +| Output | Description | +|---|---| +| `cache-hit` | `'true'` when a cache was restored, whether its key matched exactly or only as a prefix; `'false'` when nothing was restored. Always set. Unlike the `actions/cache` output of the same name, it counts a prefix match, which is how the generated keys always match. | +| `cache-key` | The key of the cache that was restored: a saved key such as `build---`, not the prefix that found it. Empty when nothing was restored. | + + + +## Examples + +### Minimal + +A pull-request build in a container that starts from the build cache: + +```yaml +name: Build lectures +on: + pull_request: + +jobs: + build: + runs-on: ubuntu-latest + container: + image: ghcr.io/quantecon/quantecon:latest + permissions: + contents: read + packages: read # the image pull, as the templates grant it + steps: + - uses: actions/checkout@v7 + - uses: quantecon/actions/setup-environment@v0 + - uses: quantecon/actions/restore-jupyter-cache@v0 + - uses: quantecon/actions/build-lectures@v0 +``` + +### Saving for later runs of a pull request + +```yaml +- uses: quantecon/actions/restore-jupyter-cache@v0 + with: + save-cache: 'true' +``` + +At the end of a successful job, `_build` is saved under a key that ends in the run id. The pull request's later runs restore it, so each one re-executes only what changed since its previous run, not since the last cache build. No other pull request, and no branch, can restore it. + +### Only the execution cache + +```yaml +- uses: quantecon/actions/restore-jupyter-cache@v0 + with: + cache-type: execution +``` + +### Requiring a cache + +```yaml +- uses: quantecon/actions/restore-jupyter-cache@v0 + with: + fail-on-miss: 'true' +``` + +The job fails when no cache is found, for a build that would take too long without one. + +### In the templates + +[`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml) restores the build cache before a pull request's build, with `save-cache` and `fail-on-miss` as comments. [`publish.yml`](https://github.com/QuantEcon/actions/blob/main/templates/publish.yml) restores it before the build it publishes. Both restore what [`cache.yml`](https://github.com/QuantEcon/actions/blob/main/templates/cache.yml) saves. + +## Behaviour + +### Keys + +The action restores the newest cache whose key starts with the first of these prefixes to match any cache, and then the next: + +| `cache-type` | Restored to | Key | Fallback | +|---|---|---|---| +| `build` | `` | `build---` | `build--` | +| `execution` | `/.jupyter_cache` | `jupyter-cache--` | `jupyter-cache-` | + +`build-jupyter-cache` saves its caches under the same prefixes, followed by its run id. A file that does not exist hashes to an empty string. With `save-cache: 'true'` the key ends in this run's id instead, and is the one the cache is saved under. With `key` set, that key is the only prefix. + +GitHub looks for a match among the caches of the branch or pull request the job runs for before it looks at those of the default branch, or of the branch a pull request targets. So a pull request that has saved its own cache restores that one, even when a newer cache build has run since. + +### Why the keys are what they are + +- **The build cache never falls back across environments.** `_build` does not record the packages that produced it. Restored after a change to the environment file, it would hand the build pages and outputs made with the old packages. An edited environment file is a miss on purpose, and the build starts from scratch. +- **The execution cache does fall back to any execution cache.** Jupyter Book checks each cached notebook against the notebook's current code, and re-executes any that no longer match, so an unrelated cache can only save time. +- **The build cache's key does not hash the lectures.** The key is the same for every pull request, so every pull request restores the last cache build and re-executes only its own changes. A key that hashed the lectures would change with every edit, and miss on nearly every pull request. The cost is that a lecture removed since the last cache build keeps its old pages, which nothing links to, until the next cache build replaces `_build`. + +### Status report + +The action prints what it asked for and what it found. On a typical read-only restore of the build cache: + +```text +╔════════════════════════════════════════════════════════════════╗ +║ BUILD CACHE STATUS ║ +╚════════════════════════════════════════════════════════════════╝ + + Cache Type: build + Requested Key: build--- + Cache Hit: false (exact key match) + Matched Key: build--- + Save Cache: false + +════════════════════════════════════════════════════════════════════ + ✅ Cache restored successfully +════════════════════════════════════════════════════════════════════ +``` + +`Cache Hit` is `actions/cache`'s exact-match flag, which the generated keys never set; `Matched Key` shows what was restored, and the action's own `cache-hit` output is `'true'`. A collapsed "Cache Contents" group follows, with the size of the cache and of each directory in it. On a miss, the report says so, and the build carries on from nothing. + +## Troubleshooting + +**`⚠️ No cache found`.** In order of likelihood: + +- The pull request changes the environment file. That is a miss on purpose: the build runs from scratch, and the cache build after the merge saves a new cache. +- The default branch has no cache: the cache build has not succeeded yet, or its cache went 7 days without being restored. Run the cache workflow by hand, for example with `gh workflow run cache.yml`. +- `environment` or `environment-update` differs from what `build-jupyter-cache` was given, or `path` is not `_build`. +- The cache build and this job compress differently. `actions/cache` compresses with `zstd` where it is installed and with `gzip` where it is not, and finds only caches made the same way. Run both jobs in the same image, or both on the runner. + +**`Cache miss with fail-on-miss enabled`.** As above; the job failed because `fail-on-miss` is `'true'`. + +**The site shows pages of a lecture that was removed.** They come from the build cache, and the next cache build clears them. To clear them now, run the cache workflow by hand. + +**The restored build is broken.** Delete the repository's `build-` caches on the Caches page of its Actions tab, then run the cache workflow. diff --git a/docs/user/actions/setup-environment.md b/docs/user/actions/setup-environment.md index 1f29155..9859d00 100644 --- a/docs/user/actions/setup-environment.md +++ b/docs/user/actions/setup-environment.md @@ -11,11 +11,11 @@ The same step works in both modes, so a job can move between a container and a s ## When to use it -Use it in any job that builds lectures, after `actions/checkout` and before [`build-lectures`](https://github.com/QuantEcon/actions/tree/main/build-lectures). +Use it in any job that builds lectures, after `actions/checkout` and before [`build-lectures`](build-lectures.md). - **Prefer a container.** The images already hold the scientific stack, Jupyter Book and LaTeX, which standard mode has to install or restore from its cache. The [containers README](https://github.com/QuantEcon/actions/tree/main/containers) compares the two images. - **Keep the step in a container job even with nothing to add.** It then installs nothing, but it records the mode in the log, and it is what lets the job run on a standard runner too. -- **Do not add it before [`build-jupyter-cache`](https://github.com/QuantEcon/actions/tree/main/build-jupyter-cache).** That action runs `setup-environment` itself, and asks it for LaTeX when `pdflatex` is among its builders. +- **Do not add it before [`build-jupyter-cache`](build-jupyter-cache.md).** That action runs `setup-environment` itself, and asks it for LaTeX when `pdflatex` is among its builders. ## Requirements diff --git a/preview-cloudflare/README.md b/preview-cloudflare/README.md index 94ffb8a..a9267ce 100644 --- a/preview-cloudflare/README.md +++ b/preview-cloudflare/README.md @@ -1,214 +1,11 @@ -# Preview Cloudflare Action + -Deploys QuantEcon lecture builds to Cloudflare Pages for PR previews with smart comments showing direct links to changed pages. +# Preview Cloudflare -## Features +Deploys a pull request's built site to Cloudflare Pages as a preview, at a URL that stays the same for every push, and comments on the pull request with it and with links to the lectures it changes. -- 🔍 **PR preview deployments** with predictable URLs (`pr-{number}.{project}.pages.dev`) -- 📚 **Changed lecture detection** - Direct links to modified pages in PR comments -- 💬 **Smart PR comments** - Updates existing comment instead of creating duplicates -- 🔒 **Security-aware** - Skips deployment for forks and dependabot -- ☁️ **Cloudflare Pages** - Fast global CDN, free tier supports private repos +How to set it up, and every input and output: [the `preview-cloudflare` chapter of the user manual](../docs/user/actions/preview-cloudflare.md). -## Usage +Every action: [the user manual](../docs/user/README.md). -```yaml -- uses: quantecon/actions/preview-cloudflare@v0 - with: - cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} - cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} - project-name: my-lectures - build-dir: _build/html -``` - -That's it! Changed lecture detection works automatically for files in the `lectures/` directory. - -> **Note:** For changed lecture detection to work, your workflow must check out the repository with full git history using `actions/checkout@v7` with `fetch-depth: 0`. Without this, only the preview URL will be shown (no direct links to changed pages). - -## Requirements - -- **Node.js/npm:** Required for `wrangler` CLI installation - - The pinned `wrangler` needs Node.js 22 or later - - The QuantEcon containers (`ghcr.io/quantecon/quantecon`, `ghcr.io/quantecon/quantecon-build`) include Node.js 24 LTS - - For other runners, use `actions/setup-node@v7` with `node-version: '24'` before this action -- **`wrangler` version:** pinned exactly in [`package.json`](package.json) and installed per job with `npm ci` from [`package-lock.json`](package-lock.json), so the runner needs access to `registry.npmjs.org`. Dependabot bumps the pin -- **Git history:** Use `fetch-depth: 0` in checkout for change detection -- **Cloudflare secrets:** `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID` -- **Cloudflare Pages project:** Must be created beforehand - -## Inputs - -| Input | Description | Required | Default | -|-------|-------------|----------|---------| -| `cloudflare-api-token` | Cloudflare API token | Yes | - | -| `cloudflare-account-id` | Cloudflare Account ID | Yes | - | -| `project-name` | Cloudflare Pages project name | Yes | - | -| `build-dir` | Directory with built site | Yes | - | -| `lectures-dir` | Lectures directory for change detection | No | `lectures` | - -To disable changed file detection, set `lectures-dir: ''`. - -## Outputs - -| Output | Description | -|--------|-------------| -| `deploy-url` | Stable preview URL — `https://pr-{number}.{project}.pages.dev`. The branch alias, so it keeps pointing at the newest deployment as the PR gains commits. Use it for anything a human will follow | -| `deployment-url` | Immutable URL of this one deployment — `https://{hash}.{project}.pages.dev`. Pinned to the commit that produced it; use it to look at a specific revision. Empty if wrangler's output did not name it | -| `changed-files` | List of changed lecture files | - -## Example PR Comment - -When a PR modifies lecture files, the action posts a comment like: - -> ## ☁️ Cloudflare Preview Ready! -> -> **Preview URL:** https://pr-5.my-lectures.pages.dev -> -> **Commit:** [`abc1234`](https://github.com/...) -> -> ### 📚 Changed Lectures -> -> - [aiyagari](https://pr-5.my-lectures.pages.dev/aiyagari.html) -> - [mccall_model](https://pr-5.my-lectures.pages.dev/mccall_model.html) -> -> --- ->
Build Info -> -> - **Workflow:** [Build Preview](...) ->
- -## Complete Workflow Example - -```yaml -name: Build Preview -on: - pull_request: - -jobs: - preview: - runs-on: ubuntu-latest - permissions: - contents: read - pull-requests: write # Required for the PR preview comment - steps: - - uses: actions/checkout@v7 - with: - fetch-depth: 0 - - - name: Setup Environment - uses: quantecon/actions/setup-environment@v0 - with: - environment: environment.yml - - - name: Build Lectures - run: jb build lectures --path-output ./ - - - name: Deploy Preview - uses: quantecon/actions/preview-cloudflare@v0 - with: - cloudflare-api-token: ${{ secrets.CLOUDFLARE_API_TOKEN }} - cloudflare-account-id: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} - project-name: my-lectures - build-dir: _build/html -``` - -## Cloudflare Setup Guide - -### 1. Create Cloudflare Account - -1. Go to [Cloudflare](https://dash.cloudflare.com/sign-up) and create a free account -2. Verify your email address - -### 2. Create Cloudflare Pages Project - -Create a project that uses **direct upload** (no Git integration): - -1. Go to **Workers & Pages** in the Cloudflare dashboard -2. Click **Create** → **Pages** → **Upload assets** -3. Enter a project name (e.g., `my-lectures`) - this becomes part of your URL -4. Upload any placeholder file to create the project -5. Note the project name for the `project-name` input - -This creates a project without GitHub integration, meaning: -- ✅ No automatic builds on push/PR -- ✅ No duplicate PR comments from Cloudflare -- ✅ Full control via our GitHub Action - -### 3. Get Credentials - -**CLOUDFLARE_ACCOUNT_ID:** -1. Cloudflare Dashboard → Any page → look at the URL -2. The URL format is: `https://dash.cloudflare.com/{account-id}/...` -3. Or go to **Workers & Pages** → **Overview** → Account ID is shown in the right sidebar - -**CLOUDFLARE_API_TOKEN:** -1. Cloudflare Dashboard → **My Profile** (top right) → **API Tokens** -2. Click **Create Token** -3. Use the **Edit Cloudflare Workers** template, or create custom with: - - **Permissions:** Account → Cloudflare Pages → Edit - - **Account Resources:** Include → Your Account -4. Click **Continue to summary** → **Create Token** -5. Copy the token immediately (it won't be shown again) - -### 4. Add GitHub Secrets - -In your repository: **Settings** → **Secrets and variables** → **Actions** → **New repository secret** - -| Secret | Value | -|--------|-------| -| `CLOUDFLARE_API_TOKEN` | Your API token | -| `CLOUDFLARE_ACCOUNT_ID` | Your account ID | - -## Preview URL Structure - -Cloudflare Pages creates predictable URLs based on the branch name: - -| Deployment | URL | -|------------|-----| -| PR #5 | `https://pr-5.{project-name}.pages.dev` | -| PR #123 | `https://pr-123.{project-name}.pages.dev` | -| Production | `https://{project-name}.pages.dev` | - -## Cloudflare vs Netlify - -| Feature | Cloudflare Pages | Netlify | -|---------|------------------|---------| -| **Private repo support** | ✅ Free | ❌ Paid plans only | -| **Free tier** | Unlimited sites, 500 builds/month | 100GB bandwidth/month | -| **CLI tool** | `wrangler` | `netlify-cli` | -| **Preview URLs** | `pr-N.project.pages.dev` | `pr-N--site.netlify.app` | - -Use `preview-cloudflare` for private repositories, `preview-netlify` for public repositories with existing Netlify setup. - -## Security - -This action automatically skips deployment for: -- **Dependabot PRs** - Can't access secrets -- **Fork PRs** - Can't access secrets - -A notification is logged when deployment is skipped. - -> **Warning:** previews of pull requests from forks are not supported. Do not run this action -> from `pull_request_target` to get around that. A workflow triggered that way builds and runs -> the fork's notebooks with `CLOUDFLARE_API_TOKEN` and a write-scoped `GITHUB_TOKEN` in reach, which is -> the "pwn request" pattern. It would not produce a preview either: the action deploys only -> on `pull_request` events, so under `pull_request_target` its deploy step is skipped. - -## Troubleshooting - -### "Authentication error" or "Unauthorized" - -- Verify `CLOUDFLARE_API_TOKEN` has Cloudflare Pages Edit permission -- Check that `CLOUDFLARE_ACCOUNT_ID` is correct -- Ensure the token hasn't expired - -### "Project not found" - -- Verify the `project-name` matches exactly (case-sensitive) -- Ensure the project was created in the correct Cloudflare account - -### Preview URL returns 404 - -- Wait a few seconds - Cloudflare may take a moment to propagate -- Check the workflow logs for the actual deployed URL -- Verify the `build-dir` contains an `index.html` + diff --git a/preview-cloudflare/action.yml b/preview-cloudflare/action.yml index 09d0ebd..2483c4a 100644 --- a/preview-cloudflare/action.yml +++ b/preview-cloudflare/action.yml @@ -1,40 +1,61 @@ name: 'Preview Cloudflare' -description: 'Deploys lecture builds to Cloudflare Pages for PR previews with smart comments showing changed pages' +description: >- + Deploys a pull request's built site to Cloudflare Pages as a preview, at a URL that stays the + same for every push, and comments on the pull request with it and with links to the lectures it + changes. author: 'QuantEcon' +# Each description below is copied verbatim into the user manual +# (docs/user/actions/preview-cloudflare.md) by scripts/generate-docs.py, so it +# is written for a reader. inputs: cloudflare-api-token: - description: 'Cloudflare API token (use secrets.CLOUDFLARE_API_TOKEN)' + description: >- + A Cloudflare API token that can edit Cloudflare Pages in the account, from a repository + secret such as `secrets.CLOUDFLARE_API_TOKEN`. required: true cloudflare-account-id: - description: 'Cloudflare Account ID (use secrets.CLOUDFLARE_ACCOUNT_ID)' + description: >- + The ID of the Cloudflare account that owns the Pages project, from a repository secret such + as `secrets.CLOUDFLARE_ACCOUNT_ID`. required: true project-name: - description: 'Cloudflare Pages project name' + description: >- + The Cloudflare Pages project to deploy to, which must already exist. Its name is part of + every preview URL, as in `.pages.dev`. required: true build-dir: - description: 'Directory containing built site' + description: >- + The directory that holds the built site, such as `_build/html`, which `build-lectures` gives + in its `build-path` output. required: true lectures-dir: - description: 'Directory containing lecture .md files for change detection. Set to empty string to disable.' + description: >- + The directory of lecture `.md` files that change detection looks in: each one the pull + request adds or modifies gets a link in the comment, and is listed in `changed-files`. + Empty turns change detection off, and the comment gives only the preview URL. required: false default: 'lectures' outputs: deploy-url: description: >- - Stable preview URL for this PR — https://pr-{number}.{project}.pages.dev. - This is the branch alias, so it keeps pointing at the newest deployment as - the PR gets more commits. Use it for anything a human will follow. + The pull request's preview URL, `https://pr-..pages.dev`, which always + shows its newest push, so use it for anything a person will follow. Empty when nothing was + deployed: on an event other than `pull_request`, and for a pull request from a fork or by + Dependabot. value: ${{ steps.deploy.outputs.deploy-url }} deployment-url: description: >- - Immutable URL of this one deployment — https://{hash}.{project}.pages.dev. - Pinned to the commit that produced it, so use it to compare a specific - revision. Empty if wrangler's output did not name it. + The URL of this one deployment, `https://..pages.dev`, which goes on + showing this push after later ones. Empty when wrangler's output does not name it, and when + nothing was deployed. value: ${{ steps.deploy.outputs.deployment-url }} changed-files: - description: 'List of changed lecture files' + description: >- + The lecture files that the pull request adds or modifies under `lectures-dir`, one path per + line. Empty when there are none, with `lectures-dir: ''`, on an event other than + `pull_request`, and for a pull request from a fork or by Dependabot. value: ${{ steps.detect-changes.outputs.changed-files }} runs: diff --git a/preview-netlify/README.md b/preview-netlify/README.md index 1befb03..eced32a 100644 --- a/preview-netlify/README.md +++ b/preview-netlify/README.md @@ -1,187 +1,11 @@ -# Preview Netlify Action + -Deploys QuantEcon lecture builds to Netlify for PR previews with smart comments showing direct links to changed pages. +# Preview Netlify -## Features +Deploys a pull request's built site to Netlify as a preview, at a URL that stays the same for every push, and comments on the pull request with it and with links to the lectures it changes. -- 🔍 **PR preview deployments** with predictable URLs (`pr-{number}`) -- 📚 **Changed lecture detection** - Direct links to modified pages in PR comments -- 💬 **Smart PR comments** - Updates existing comment instead of creating duplicates -- 🔒 **Security-aware** - Skips deployment for forks and dependabot -- ⚡ **Reliable** - Uses JSON output for accurate URL extraction +How to set it up, and every input and output: [the `preview-netlify` chapter of the user manual](../docs/user/actions/preview-netlify.md). -## Usage - -```yaml -- uses: quantecon/actions/preview-netlify@v0 - with: - netlify-auth-token: ${{ secrets.NETLIFY_AUTH_TOKEN }} - netlify-site-id: ${{ secrets.NETLIFY_SITE_ID }} - build-dir: _build/html -``` - -That's it! Changed lecture detection works automatically for files in the `lectures/` directory. - -> **Note:** For changed lecture detection to work, your workflow must check out the repository with full git history using `actions/checkout@v7` with `fetch-depth: 0`. Without this, only the preview URL will be shown (no direct links to changed pages). - -## Requirements - -- **Node.js/npm:** Required for `netlify-cli` installation - - The pinned `netlify-cli` needs Node.js 22.13 or later - - The QuantEcon containers (`ghcr.io/quantecon/quantecon`, `ghcr.io/quantecon/quantecon-build`) include Node.js 24 LTS - - For other runners, use `actions/setup-node@v7` with `node-version: '24'` before this action -- **`netlify-cli` version:** pinned exactly in [`package.json`](package.json) and installed per job with `npm ci` from [`package-lock.json`](package-lock.json), so the runner needs access to `registry.npmjs.org`. Dependabot bumps the pin -- **python3:** For parsing Netlify JSON output (included in ubuntu-latest and QuantEcon container) -- **Git history:** Use `fetch-depth: 0` in checkout for change detection -- **Netlify secrets:** `NETLIFY_AUTH_TOKEN` and `NETLIFY_SITE_ID` - -## Inputs - -| Input | Description | Required | Default | -|-------|-------------|----------|---------| -| `netlify-auth-token` | Netlify auth token | Yes | - | -| `netlify-site-id` | Netlify site ID | Yes | - | -| `build-dir` | Directory with built site | Yes | - | -| `lectures-dir` | Lectures directory for change detection | No | `lectures` | - -To disable changed file detection, set `lectures-dir: ''`. - -## Outputs - -| Output | Description | -|--------|-------------| -| `deploy-url` | URL of deployed preview | -| `changed-files` | List of changed lecture files | - -## Example PR Comment - -When a PR modifies lecture files, the action posts a comment like: - -> ## 📖 Netlify Preview Ready! -> -> **Preview URL:** https://pr-5--site.netlify.app -> -> **Commit:** [`abc1234`](https://github.com/...) -> -> ### 📚 Changed Lectures -> -> - [aiyagari](https://pr-5--site.netlify.app/aiyagari.html) -> - [mccall_model](https://pr-5--site.netlify.app/mccall_model.html) -> -> --- ->
Build Info -> -> - **Workflow:** [Build Preview](...) ->
- -## Complete Workflow Example - -```yaml -name: Build Preview -on: - pull_request: - -jobs: - preview: - runs-on: ubuntu-latest - permissions: - contents: read - pull-requests: write # Required for the PR preview comment - steps: - - uses: actions/checkout@v7 - with: - fetch-depth: 0 - - - name: Setup Environment - uses: quantecon/actions/setup-environment@v0 - with: - environment: environment.yml - - - name: Build Lectures - run: jb build lectures --path-output ./ - - - name: Deploy Preview - uses: quantecon/actions/preview-netlify@v0 - with: - netlify-auth-token: ${{ secrets.NETLIFY_AUTH_TOKEN }} - netlify-site-id: ${{ secrets.NETLIFY_SITE_ID }} - build-dir: _build/html -``` - -## Netlify Setup Guide - -### 1. Create Netlify Site (CLI-only mode) - -For best results, create a site that uses **CLI deployment only** (no automatic builds): - -1. Go to [Netlify](https://app.netlify.com) and click **Add new site** → **Deploy manually** -2. Drag any folder to create the site (this creates an empty placeholder) -3. Note the site name (e.g., `jade-tarsier-d98a19`) - -This creates a site without GitHub integration, meaning: -- ✅ No automatic builds on push/PR -- ✅ No duplicate PR comments from Netlify -- ✅ Full control via our GitHub Action - -### 2. Get Credentials - -**NETLIFY_AUTH_TOKEN:** -1. Netlify → User Settings → Applications → Personal access tokens -2. Create new token with descriptive name (e.g., `github-actions`) -3. Copy the token immediately (it won't be shown again) - -**NETLIFY_SITE_ID:** -1. Netlify → Your Site → Site configuration → General → Site details -2. Copy the **Site ID** (a UUID like `a1b2c3d4-e5f6-...`) - -### 3. Add GitHub Secrets - -In your repository: **Settings** → **Secrets and variables** → **Actions** → **New repository secret** - -| Secret | Value | -|--------|-------| -| `NETLIFY_AUTH_TOKEN` | Your personal access token | -| `NETLIFY_SITE_ID` | Your site ID | - -### 4. (Optional) Disable Netlify GitHub Integration - -If you previously linked your repo to Netlify and are seeing **duplicate PR comments**, disable Netlify's built-in integration: - -**Option A: Unlink Repository (Recommended)** -1. Netlify → Your Site → **Site configuration** → **Build & deploy** → **Continuous deployment** -2. Click **Unlink** to disconnect the GitHub repo -3. Your site remains active, but Netlify won't auto-build or comment - -**Option B: Disable PR Comments Only** -1. Netlify → Your Site → **Site configuration** → **Notifications** -2. Find **GitHub notifications** section -3. Delete or disable: - - "GitHub commit comments" - - "GitHub PR comments" - -**Option C: Disable Deploy Previews** -1. Netlify → Your Site → **Site configuration** → **Build & deploy** → **Continuous deployment** -2. Under **Branches and deploy contexts**, set **Deploy Previews** to "None" - -### Which setup to use? - -| Scenario | Recommendation | -|----------|----------------| -| New project | Create site via "Deploy manually" (no GitHub link) | -| Existing linked site | Unlink repository in Netlify settings | -| Want Netlify builds + our action | Disable Netlify PR comments only | - -## Security - -This action automatically skips deployment for: -- **Dependabot PRs** - Can't access secrets -- **Fork PRs** - Can't access secrets - -A notification is logged when deployment is skipped. - -> **Warning:** previews of pull requests from forks are not supported. Do not run this action -> from `pull_request_target` to get around that. A workflow triggered that way builds and runs -> the fork's notebooks with `NETLIFY_AUTH_TOKEN` and a write-scoped `GITHUB_TOKEN` in reach, which is -> the "pwn request" pattern. It would not produce a preview either: the action deploys only -> on `pull_request` events, so under `pull_request_target` its deploy step is skipped. +Every action: [the user manual](../docs/user/README.md). + diff --git a/preview-netlify/action.yml b/preview-netlify/action.yml index 984c97f..fa036e0 100644 --- a/preview-netlify/action.yml +++ b/preview-netlify/action.yml @@ -1,28 +1,49 @@ name: 'Preview Netlify' -description: 'Deploys lecture builds to Netlify for PR previews with smart comments showing changed pages' +description: >- + Deploys a pull request's built site to Netlify as a preview, at a URL that stays the same for + every push, and comments on the pull request with it and with links to the lectures it changes. author: 'QuantEcon' +# Each description below is copied verbatim into the user manual +# (docs/user/actions/preview-netlify.md) by scripts/generate-docs.py, so it is +# written for a reader. inputs: netlify-auth-token: - description: 'Netlify authentication token (use secrets.NETLIFY_AUTH_TOKEN)' + description: >- + A Netlify personal access token, from a repository secret such as + `secrets.NETLIFY_AUTH_TOKEN`. It reaches `netlify-cli` through the environment, never on its + command line. required: true netlify-site-id: - description: 'Netlify site ID (use secrets.NETLIFY_SITE_ID)' + description: >- + The ID of the Netlify site to deploy to, from a repository secret such as + `secrets.NETLIFY_SITE_ID`. required: true build-dir: - description: 'Directory containing built site' + description: >- + The directory that holds the built site, such as `_build/html`, which `build-lectures` gives + in its `build-path` output. required: true lectures-dir: - description: 'Directory containing lecture .md files for change detection. Set to empty string to disable.' + description: >- + The directory of lecture `.md` files that change detection looks in: each one the pull + request adds or modifies gets a link in the comment, and is listed in `changed-files`. + Empty turns change detection off, and the comment gives only the preview URL. required: false default: 'lectures' outputs: deploy-url: - description: 'URL of the deployed preview site' + description: >- + The preview's URL, `https://pr---.netlify.app`, the same for every push to the + pull request. Empty when nothing was deployed: on an event other than `pull_request`, and + for a pull request from a fork or by Dependabot. value: ${{ steps.deploy.outputs.deploy-url }} changed-files: - description: 'List of changed lecture files' + description: >- + The lecture files that the pull request adds or modifies under `lectures-dir`, one path per + line. Empty when there are none, with `lectures-dir: ''`, on an event other than + `pull_request`, and for a pull request from a fork or by Dependabot. value: ${{ steps.detect-changes.outputs.changed-files }} runs: diff --git a/publish-gh-pages/README.md b/publish-gh-pages/README.md index a5e3802..e8f954e 100644 --- a/publish-gh-pages/README.md +++ b/publish-gh-pages/README.md @@ -1,282 +1,11 @@ -# Publish to GitHub Pages Action + -Publishes QuantEcon lecture builds to GitHub Pages using native GitHub Pages deployment. +# Publish to GitHub Pages -## Features +Publishes a built site to GitHub Pages with GitHub's own Pages deploy, so no gh-pages branch is needed. On a tag it can also attach the site to the release, as an archive with its checksum and a manifest. -- 📄 **Native GitHub Pages deployment** - No gh-pages branch needed -- 🌐 **Custom domains** - set in Settings → Pages; the optional `cname` input only writes a `CNAME` file (see [Custom Domain Setup](#custom-domain-setup)) -- � **Release assets** - Optional HTML archive, checksum, and manifest -- �📊 **Deployment statistics** (file count, size) -- 🔗 **Automatic URL generation** from GitHub -- ⚡ **No repo bloat** - Eliminates gh-pages branch history issues +How to set it up, and every input and output: [the `publish-gh-pages` chapter of the user manual](../docs/user/actions/publish-gh-pages.md). -## Why Native Deployment? +Every action: [the user manual](../docs/user/README.md). -This action uses GitHub's native Pages deployment (via artifacts) instead of pushing to a gh-pages branch: - -| Aspect | Native (this action) | Branch-based (old) | -|--------|---------------------|-------------------| -| gh-pages branch | ❌ Not needed | ✅ Required | -| Repo size growth | ❌ None | ⚠️ Grows over time | -| GitHub support | ✅ First-party | Third-party | -| Concurrency | ✅ Built-in | Manual | - -## Inputs - -| Input | Description | Required | Default | -|-------|-------------|----------|---------| -| `build-dir` | Directory with built site | Yes | - | -| `cname` | Writes a `CNAME` file into `build-dir`. Does **not** set the custom domain: the Pages deploy ignores `CNAME` files, so set it in Settings → Pages (see [Custom Domain Setup](#custom-domain-setup)) | No | - | -| `create-release-assets` | Create and upload release assets | No | `false` | -| `asset-name` | Base name for assets (e.g., "lecture-python-html") | No | `-html` | -| `github-token` | Token for uploading release assets | If creating assets | - | - -**Note:** `github-token` is only needed when `create-release-assets: 'true'`. - -## Outputs - -| Output | Description | -|--------|-------------| -| `page-url` | URL of deployed site | - -## Usage - -### Basic Deployment - -```yaml -- uses: quantecon/actions/publish-gh-pages@v0 - with: - build-dir: '_build/html' -``` - -### With Custom Domain - -Set the domain itself in Settings → Pages → Custom domain; the Pages deploy does not read a `CNAME` file. `cname` only writes one into `build-dir` and the release archive, and the action warns that it has no effect on the deploy. - -```yaml -- uses: quantecon/actions/publish-gh-pages@v0 - with: - build-dir: '_build/html' - cname: 'python.quantecon.org' -``` - -### Using Page URL - -```yaml -- uses: quantecon/actions/publish-gh-pages@v0 - id: pages - with: - build-dir: '_build/html' - -- name: Verify deployment - run: | - echo "Site deployed to: ${{ steps.pages.outputs.page-url }}" -``` - -### With Release Assets - -Create downloadable archives attached to GitHub releases: - -```yaml -- uses: quantecon/actions/publish-gh-pages@v0 - with: - build-dir: '_build/html' - cname: 'python.quantecon.org' - create-release-assets: 'true' - github-token: ${{ secrets.GITHUB_TOKEN }} -``` - -This creates and uploads to the release: -- `-html-.tar.gz` - Complete site archive -- `-html-checksum.txt` - SHA256 checksum -- `-html-manifest.json` - Metadata (commit, timestamp, size) - -The asset name defaults to `-html` (e.g., `lecture-python.myst-html`). Override with `asset-name` if needed: - -```yaml -create-release-assets: 'true' -asset-name: 'lecture-python-html' # Custom name without .myst -github-token: ${{ secrets.GITHUB_TOKEN }} -``` - -**Note:** Requires `contents: write` permission for release uploads. - -## Setup Requirements - -### Required Workflow Permissions - -The workflow must have these permissions: - -```yaml -permissions: - contents: write # For release assets (read is enough without assets) - pages: write - id-token: write -``` - -### Enable GitHub Pages - -1. Go to repository Settings → Pages -2. Set Source to **"GitHub Actions"** (not "Deploy from a branch") -3. Save - -### Concurrency (Recommended) - -Add to your workflow to prevent deployment conflicts: - -```yaml -concurrency: - group: "pages" - cancel-in-progress: false -``` - -### Custom Domain Setup - -If using custom domain: - -1. Set the domain in repository Settings → Pages → Custom domain. This is the only - place a GitHub Actions Pages deploy reads it from: it ignores `CNAME` files, so the - `cname` input only writes one into `build-dir` (and the release archive) and warns - that it has no effect on the deploy. -2. Configure DNS records at your domain provider: - - CNAME record: `python → quantecon.github.io` - - Or A records to GitHub's IPs -3. Enable HTTPS in repository settings (automatic after DNS propagation) - -## Troubleshooting - -### Deployment Fails with Permissions Error - -**Solutions:** -1. Ensure workflow has `pages: write` and `id-token: write` permissions -2. Verify GitHub Pages source is set to "GitHub Actions" in repo settings - -### Build Directory Not Found - -**Symptom:** `❌ Error: Build directory not found` - -**Solutions:** -1. Verify `build-dir` path is correct -2. Check build step completed successfully -3. Use output from build action: `${{ steps.build.outputs.build-path }}` - -### Page Not Loading - -**Symptom:** 404 error on GitHub Pages URL - -**Solutions:** -1. Wait 1-2 minutes for propagation -2. Verify GitHub Pages source is "GitHub Actions" in settings -3. Verify index.html exists in root - -### Custom Domain Not Working - -**Symptom:** CNAME configured but domain not resolving - -**Solutions:** -1. Verify DNS records at domain provider -2. Wait for DNS propagation (up to 24 hours) -3. Check the domain is set in Settings → Pages → Custom domain (a `CNAME` file in the site is not read) -4. Enable HTTPS in repository settings - -## Performance - -| Step | Time | -|------|------| -| Prepare build | ~2 seconds | -| Upload artifact | ~10-30 seconds | -| Deploy to Pages | ~30-60 seconds | -| **Total** | **~45-90 seconds** | - -## Examples - -### Complete Publish Workflow - -```yaml -name: Publish - -on: - push: - tags: ['publish-*'] - -permissions: - contents: write # For release assets - pages: write - id-token: write - -concurrency: - group: "pages" - cancel-in-progress: false - -jobs: - publish: - runs-on: ubuntu-latest - environment: - name: github-pages - url: ${{ steps.deploy.outputs.page-url }} - steps: - - uses: actions/checkout@v7 - - - uses: quantecon/actions/setup-environment@v0 - with: - install-latex: 'true' - - - uses: quantecon/actions/build-lectures@v0 - id: build - - - uses: quantecon/actions/publish-gh-pages@v0 - id: deploy - with: - build-dir: ${{ steps.build.outputs.build-path }} - cname: 'python.quantecon.org' - create-release-assets: 'true' - asset-name: 'lecture-python-html' - github-token: ${{ secrets.GITHUB_TOKEN }} -``` - -### Conditional Deployment - -Deploy only on main branch: - -```yaml -- uses: quantecon/actions/publish-gh-pages@v0 - if: github.ref == 'refs/heads/main' - with: - build-dir: '_build/html' -``` - -## Migration from Branch-Based Deployment - -If migrating from peaceiris/actions-gh-pages or similar: - -1. **Update workflow permissions:** - ```yaml - permissions: - contents: read - pages: write - id-token: write - ``` - -2. **Change GitHub Pages source:** - - Go to Settings → Pages - - Change from "Deploy from a branch" to "GitHub Actions" - -3. **Update action usage:** - ```yaml - # Before - - uses: quantecon/actions/publish-gh-pages@v0 - with: - build-dir: '_build/html' - github-token: ${{ secrets.GITHUB_TOKEN }} - target-branch: 'gh-pages' - - # After - - uses: quantecon/actions/publish-gh-pages@v0 - with: - build-dir: '_build/html' - ``` - -4. **Optional: Delete gh-pages branch** to reclaim repo space - -See [docs/MIGRATION-GUIDE.md](../docs/MIGRATION-GUIDE.md) for complete workflow examples. + diff --git a/publish-gh-pages/action.yml b/publish-gh-pages/action.yml index 2f6f542..2a04950 100644 --- a/publish-gh-pages/action.yml +++ b/publish-gh-pages/action.yml @@ -1,31 +1,54 @@ name: 'Publish to GitHub Pages' -description: 'Publishes lecture builds to GitHub Pages using native GitHub Pages deployment (no gh-pages branch needed)' +description: >- + Publishes a built site to GitHub Pages with GitHub's own Pages deploy, so no gh-pages branch is + needed. On a tag it can also attach the site to the release, as an archive with its checksum and + a manifest. author: 'QuantEcon' +# Each description below is copied verbatim into the user manual +# (docs/user/actions/publish-gh-pages.md) by scripts/generate-docs.py, so it is +# written for a reader. inputs: build-dir: - description: 'Directory containing built site' + description: >- + The directory that holds the built site, such as `_build/html`, which `build-lectures` gives + in its `build-path` output. The job fails if it does not exist. required: true cname: - description: 'Domain to write into build-dir as a CNAME file (it also lands in the release archive). Has no effect on the Pages deploy, which ignores CNAME files: set the custom domain in Settings -> Pages.' + description: >- + A domain to write into `build-dir` as a `CNAME` file, which also lands in the release + archive. It does not set the site's custom domain, and a warning says so: a GitHub Actions + Pages deploy ignores `CNAME` files, so set the domain under Settings, Pages. Empty, the + default, writes no file. required: false default: '' create-release-assets: - description: 'Create and upload release assets (HTML archive, checksum, manifest)' + description: >- + `'true'` also attaches the site to the GitHub release of the tag the run was triggered by, + creating the release if there is none: an archive, its SHA-256 checksum and a manifest. It + needs `github-token`, and a run triggered by a tag: any other run skips it, with a warning. + `'false'`, the default, attaches nothing. required: false default: 'false' asset-name: - description: 'Base name for release assets (e.g., "lecture-python-html")' + description: >- + With `create-release-assets: 'true'`: the start of each asset's file name. Empty, the + default, uses the repository's name followed by `-html`, as in `-html`. required: false default: '' github-token: - description: 'GitHub token for uploading release assets (only needed if create-release-assets is true)' + description: >- + With `create-release-assets: 'true'`: a token that can write the repository's releases, + such as `secrets.GITHUB_TOKEN` in a job with `contents: write`. The job fails if it is + empty. The Pages deploy does not use it. required: false default: '' outputs: page-url: - description: 'URL of the deployed GitHub Pages site' + description: >- + The URL of the published site, as GitHub Pages reports it: `https://.github.io//`, + or the custom domain's. Set once the deploy succeeds. value: ${{ steps.deployment.outputs.page_url }} runs: diff --git a/restore-jupyter-cache/README.md b/restore-jupyter-cache/README.md index 48031fc..3dc48cb 100644 --- a/restore-jupyter-cache/README.md +++ b/restore-jupyter-cache/README.md @@ -1,223 +1,11 @@ -# Restore Jupyter Cache Action + -Restores Jupyter Book build cache from GitHub Actions cache. Designed for PR workflows that need to restore cache generated by main branch builds. +# Restore Jupyter Cache -## Design Philosophy +Restores the cache that `build-jupyter-cache` saved, so a pull request or publish build re-executes only the notebooks that changed. It only restores by default, and can also save a cache for the later runs of the same pull request. -By default this action is **read-only** - it restores cache but never saves. This ensures: +How to set it up, and every input and output: [the `restore-jupyter-cache` chapter of the user manual](../docs/user/actions/restore-jupyter-cache.md). -- **Single source of truth**: Cache comes from the main-branch `build-jupyter-cache` build -- **No conflicts**: Multiple PRs don't overwrite each other's cache -- **Predictable state**: You always know where cache came from +Every action: [the user manual](../docs/user/README.md). -For cache generation on main, use the `build-jupyter-cache` action in your cache.yml workflow. - -### Optional PR-scoped saving (`save-cache`) - -Set `save-cache: true` to also **save** an updated cache at the end of the job. GitHub Actions -scopes caches by ref, so the saved cache is visible only to subsequent runs on the same PR -branch — it cannot affect other PRs or `main`. Leave it `false` (the default) to keep the strict -read-only behaviour above. - -## Inputs - -| Input | Description | Required | Default | -|-------|-------------|----------|---------| -| `cache-type` | `build` (full `_build`) or `execution` (`.jupyter_cache` only) | No | `build` | -| `path` | Directory to restore the cache into. Must match where `build-lectures` reads (`_build`) — pointing it elsewhere restores the cached content to a different directory, so the build steps won't see the restored state | No | `_build` | -| `source-dir` | Source directory for lectures (used for content hash in execution cache) | No | `lectures` | -| `environment` | Path to environment file (used for cache key hash) | No | `environment.yml` | -| `environment-update` | Path to delta environment file for container builds (used for cache key hash) | No | `''` | -| `key` | Custom cache key (auto-generated if not specified) | No | Auto | -| `fail-on-miss` | Fail workflow if no cache found | No | `false` | -| `save-cache` | Also save an updated cache at job end, scoped to the PR branch (see Design Philosophy) | No | `false` | - -## Outputs - -| Output | Description | -|--------|-------------| -| `cache-hit` | `true` if a cache was restored, **including via a prefix match**. This is deliberately broader than `actions/cache`'s own `cache-hit`, which reports exact key matches only — under this action's key scheme an exact match is the uncommon case, so an exact-match-only signal would read as a miss on almost every successful restore. | -| `cache-key` | The key that was actually matched, which on a prefix match is not the key that was requested. Empty on a miss. | - -Both outputs, and `fail-on-miss`, behave identically with `save-cache` on or off. - -## Cache Key Strategy - -The action uses **prefix matching** to find the most recently saved cache: - -**Build cache** (default): -- Saves as: `build-{env-hash}-{update-hash}-{run-id}` -- Restores using prefix: `build-{env-hash}-{update-hash}-` → finds most recent -- Falls back to: `build-{env-hash}-` → same base environment, different delta - -There is deliberately **no bare `build-` fallback**. A `_build` produced under a different environment is a miss, not a warm start: `_build` carries no record of what produced it, so restoring one across an `environment.yml` change would hand the build outputs generated by the old package set — the failure mode behind #28. An environment change is meant to be a miss, which is what the status report tells the user. - -**Execution cache**: -- Saves as: `jupyter-cache-{content-hash}-{run-id}` -- Restores using prefix: `jupyter-cache-{content-hash}-` → finds most recent -- Falls back to: `jupyter-cache-` → any execution cache - -The execution cache **keeps** its bare fallback, and the asymmetry is deliberate. `.jupyter_cache` is content-addressed per notebook: jupyter-cache revalidates every entry against the notebook's own hash and re-executes anything that no longer matches, so a cache restored from unrelated content can only help. `_build` has no equivalent self-check. - -This ensures PRs always get the latest successful cache without conflicts. - -### Why the build-cache key is keyed on the environment, not lecture content - -The build-cache key intentionally hashes only the environment, **not** `lectures/**/*.md`. The -`_build` cache is a stable **warm-start baseline** from `main`, and freshness is handled -downstream — so a content hash would only hurt: - -- **jupyter-cache** (inside `_build/.jupyter_cache`) is content-addressed per notebook — it - re-executes only notebooks whose code changed, regardless of the cache key. -- **Sphinx** rebuilds incrementally, re-rendering only the pages that changed. -- The weekly cold `build-jupyter-cache` run rebuilds `_build` from scratch, clearing any drift. - -Hashing lecture content into the key would miss the cache on essentially every PR (each PR edits -some `.md`) and force a cold, full re-execution — defeating the warm-start for no correctness gain. -The one side effect of incremental builds — Sphinx not deleting output for removed/renamed sources -— leaves only **orphaned** pages (absent from `_toc.yml`, unreachable in navigation), which the -weekly rebuild clears. - -## Cache Types - -### Build Cache (`cache-type: build`) - Default - -Restores entire `_build` directory including HTML, PDF, notebook outputs, and .jupyter_cache. - -**Best for**: Fast PR previews with full incremental builds - -### Execution Cache (`cache-type: execution`) - -Restores only `.jupyter_cache` directory containing cached notebook execution outputs. - -**Best for**: Lighter cache when you only need notebook execution state - -## Usage - -### Basic Usage (Build Cache) - -```yaml -- uses: quantecon/actions/restore-jupyter-cache@v0 - -- uses: quantecon/actions/build-lectures@v0 -``` - -### Execution Cache Only - -```yaml -- uses: quantecon/actions/restore-jupyter-cache@v0 - with: - cache-type: 'execution' -``` - -### Multi-Builder Workflow - -```yaml -# Restore once at start -- uses: quantecon/actions/restore-jupyter-cache@v0 - -# All builds share restored state -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'jupyter' - -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'pdflatex' - -- uses: quantecon/actions/build-lectures@v0 - with: - builder: 'html' - html-copy-pdf: true - html-copy-notebooks: true -``` - -### Require Cache (Fail if Missing) - -```yaml -- uses: quantecon/actions/restore-jupyter-cache@v0 - with: - fail-on-miss: true -``` - -## Cache Status Report - -The action prints a cache status report for debugging. This is a read-only restore of a build -cache that found an earlier save. With the generated key, saved keys end in the saving run's id, -so a read-only restore never matches its key exactly. The report's `Cache Hit` line is -`actions/cache`'s exact-match flag and reads `false`; `Matched Key` shows what was restored, and the -action's own `cache-hit` output is `true`. `{update-hash}` is empty when `environment-update` is -unset, so the key then reads `build-{env-hash}--`. The lines from `Path:` down sit in a collapsible -"Cache Contents" group. - -``` -╔════════════════════════════════════════════════════════════════╗ -║ BUILD CACHE STATUS ║ -╚════════════════════════════════════════════════════════════════╝ - - Cache Type: build - Requested Key: build-{env-hash}-{update-hash}- - Cache Hit: false (exact key match) - Matched Key: build-{env-hash}-{update-hash}-35963034770 - Save Cache: false - -════════════════════════════════════════════════════════════════════ - ✅ Cache restored successfully -════════════════════════════════════════════════════════════════════ - -Path: _build - -Total Size: 164M - -Files: 1342 -Directories: 89 - -── Directory Sizes ── -120M _build/html -7.0M _build/jupyter -30M _build/latex - -── Build Directories ── - html/: 120M (1100 files) - latex/: 30M (45 files) - jupyter/: 7.0M (95 files) - .jupyter_cache/: 6.0M -``` - -## Setting Up Cache Generation - -See [templates/cache.yml](../templates/cache.yml) for a complete cache generation workflow. - -The cache generation workflow should: -1. Run on main branch (schedule + push) -2. Build from scratch (no cache restore) -3. Save cache using `actions/cache/save` - -## Troubleshooting - -### Cache Miss on PR - -**Symptom**: PR builds show "No cache found" - -**Solutions**: -1. Ensure cache.yml workflow has run on main -2. Check if `environment.yml` changed (invalidates build cache) -3. Run cache workflow manually: `gh workflow run cache.yml` - -### Stale Cache - -**Symptom**: Build uses old outputs - -**Solutions**: -1. Clear cache: `gh cache delete "build-*" --repo your-org/repo` -2. Re-run cache generation workflow -3. Verify cache key matches - -### Cache Too Large - -**Symptom**: Cache restore is slow or fails - -**Solutions**: -1. Use `execution` cache type (smaller, ~6 MB vs ~150 MB) -2. Clean build artifacts before saving cache -3. Check GitHub cache storage limits (10 GB per repo) + diff --git a/restore-jupyter-cache/action.yml b/restore-jupyter-cache/action.yml index 8bc19ef..59c0e44 100644 --- a/restore-jupyter-cache/action.yml +++ b/restore-jupyter-cache/action.yml @@ -1,47 +1,84 @@ name: 'Restore Jupyter Cache' -description: 'Restores Jupyter Book build cache from GitHub Actions cache, with optional save for PR-scoped caching' +description: >- + Restores the cache that `build-jupyter-cache` saved, so a pull request or publish build re-executes + only the notebooks that changed. It only restores by default, and can also save a cache for the + later runs of the same pull request. author: 'QuantEcon' +# Each description below is copied verbatim into the user manual +# (docs/user/actions/restore-jupyter-cache.md) by scripts/generate-docs.py, so +# it is written for a reader. inputs: cache-type: - description: 'Type of cache to restore: execution (.jupyter_cache only) or build (full _build directory)' + description: >- + What to restore. `build`, the default, restores the whole `_build` directory the last cache + build saved: its HTML, PDF and notebooks, and the execution cache in `_build/.jupyter_cache`. + `execution` restores only `_build/.jupyter_cache`, which holds each notebook's executed + outputs. required: false default: 'build' path: - description: 'Directory to restore the cache into. Must match where build-lectures reads/writes (_build); pointing it elsewhere restores the cached _build to a directory the build steps will not use.' + description: >- + The build directory to restore into. The path is part of what identifies a cache, so only + `_build`, the default, finds the caches that `build-jupyter-cache` saves; any other path + finds only what this action saved with `save-cache` and the same `path`. With + `cache-type: execution` the cache goes into `/.jupyter_cache`. required: false default: '_build' source-dir: - description: 'Source directory containing lectures (used for content hash in execution cache)' + description: >- + With `cache-type: execution`: the book's directory. The key hashes every `.md` file under + it, as `build-jupyter-cache` does with its own `source-dir`. Ignored for the build cache. required: false default: 'lectures' environment: - description: 'Path to environment file (used for cache key hash)' + description: >- + With `cache-type: build`: the Conda environment file whose hash is part of the key. Give the + same file that `build-jupyter-cache` was given, or no cache matches. Ignored for the + execution cache. required: false default: 'environment.yml' environment-update: - description: 'Path to delta environment file for container builds (used for cache key hash)' + description: >- + With `cache-type: build`: the container delta file whose hash is part of the key. Give the + same file that `build-jupyter-cache` was given; with another, only the fallback matches, + restoring the newest cache built from the same `environment`. Ignored for the execution + cache. required: false default: '' key: - description: 'Cache key to look for (auto-generated if not specified)' + description: >- + A key to restore in place of the generated ones. It is matched as a prefix, so the newest + cache whose key starts with it is restored, and with `save-cache: 'true'` the cache is saved + under exactly this key. Empty, the default, uses the generated keys. required: false default: '' fail-on-miss: - description: 'Fail the workflow if no cache is found' + description: >- + `'true'` fails the job when nothing is restored. `'false'`, the default, carries on without a + cache, and the build starts from scratch. required: false default: 'false' save-cache: - description: 'Save build cache at end of job for faster subsequent PR runs. Cache is scoped to the PR branch and cannot affect other PRs or main.' + description: >- + `'true'` also saves the directory at the end of the job, if the job succeeds, under a key + that ends in the run id. GitHub scopes a cache to the branch or pull request that saved it, + so only later runs of the same pull request restore it. `'false'`, the default, only + restores. required: false default: 'false' outputs: cache-hit: - description: 'Whether the cache was successfully restored (includes prefix match)' + description: >- + `'true'` when a cache was restored, whether its key matched exactly or only as a prefix; + `'false'` when nothing was restored. Always set. Unlike the `actions/cache` output of the + same name, it counts a prefix match, which is how the generated keys always match. value: ${{ steps.restore.outputs.cache-matched-key != '' }} cache-key: - description: 'The cache key that was matched (empty if no hit)' + description: >- + The key of the cache that was restored: a saved key such as `build---`, + not the prefix that found it. Empty when nothing was restored. value: ${{ steps.restore.outputs.cache-matched-key }} runs: diff --git a/scripts/generate-docs.py b/scripts/generate-docs.py index 242543c..fc3d043 100644 --- a/scripts/generate-docs.py +++ b/scripts/generate-docs.py @@ -10,12 +10,15 @@ inputs, outputs in each chapter, docs/user/actions/.md actions the action table in docs/user/README.md - signpost the whole of /README.md, once the action has a chapter + signpost the whole of /README.md + +Every action must have a chapter, and every chapter an action. Usage: python3 scripts/generate-docs.py rewrite every generated block - python3 scripts/generate-docs.py --check print the diff and fail if a block is out of date + python3 scripts/generate-docs.py --check print the diff and fail if a block is out of date, + or an action has no chapter python3 scripts/generate-docs.py --check-examples [--actionlint PATH] --check-examples reads every yaml block in docs/user and every workflow in templates/: @@ -49,7 +52,6 @@ CHAPTERS = MANUAL / "actions" INDEX = MANUAL / "README.md" TEMPLATES = ROOT / "templates" -REPO_URL = "https://github.com/QuantEcon/actions" # The order the manual lists the actions in, which is the order a lecture repo meets them. # A new action goes here too; the script refuses to guess where. @@ -189,11 +191,7 @@ def outputs_table(action): def actions_table(actions): rows = ["| Action | What it does |", "|---|---|"] for action in actions: - if action.chapter.is_file(): - link = f"actions/{action.name}.md" - else: - # No chapter yet (#190): its README is still the reference. - link = f"{REPO_URL}/tree/main/{action.name}" + link = f"actions/{action.name}.md" rows.append(f"| [`{action.name}`]({link}) | {cell(action.description, action.path, 'description')} |") return "\n".join(rows) @@ -274,27 +272,22 @@ def generate(actions): if chapter.stem not in names: message = f"{rel(CHAPTERS)} holds one chapter per action, and there is no {chapter.stem}/action.yml" error(chapter, 0, message) - missing = [] for action in actions: source = f"{action.name}/action.yml" - if action.chapter.is_file(): - text = fill(action.chapter, {"inputs": inputs_table(action), "outputs": outputs_table(action)}, source) - if text is not None: - wanted[action.chapter] = text - wanted[action.readme] = "\n".join(wrap("signpost", source, signpost(action))) + "\n" - else: - missing.append(action.name) - if action.readme.is_file() and "BEGIN GENERATED: signpost" in action.readme.read_text(encoding="utf-8"): - error(action.readme, 1, f"a generated signpost, but {rel(action.chapter)} does not exist") + if not action.chapter.is_file(): + template = rel(ROOT / "docs" / "dev" / "CHAPTER-TEMPLATE.md") + error(action.path, 0, f"{action.name} has no chapter: write {rel(action.chapter)}, starting from {template}") + continue + text = fill(action.chapter, {"inputs": inputs_table(action), "outputs": outputs_table(action)}, source) + if text is not None: + wanted[action.chapter] = text + wanted[action.readme] = "\n".join(wrap("signpost", source, signpost(action))) + "\n" if INDEX.is_file(): text = fill(INDEX, {"actions": actions_table(actions)}, "each action.yml") if text is not None: wanted[INDEX] = text else: error(INDEX, 0, "the manual's index is missing") - if missing: - # #190 makes this an error, once every action has its chapter. - print(f"no chapter yet, so the README is still the reference: {', '.join(missing)}") return wanted From bbc542bf2f94426265edf2e5e0aa3d6d2f6917bd Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 28 Sep 2026 11:29:54 +0000 Subject: [PATCH 2/4] docs(manual): correct the preview and restore chapters against the code A fact-check of the chapters against the actions, their scripts and the pinned CLIs' sources found these. Preview actions: - Change detection compares the tip of the base branch with the pull request's head, so it lists lectures changed on the base branch since the pull request branched off. The chapters and the lectures-dir and changed-files descriptions now say so; the code fix is #215. - lectures-dir is matched against paths from the repository root, so ./lectures and lectures/ match nothing. - preview-cloudflare builds its URLs from project-name, which is wrong when Cloudflare gives the project another pages.dev address. The setup guide now says to check it; the code fix is #216. - Netlify's dashboard calls a site a project: the setup steps use its words. The README's recommendation to unlink, and its table of which setup to use, are back, with an example of the comment. - The comment is looked up among the first 30 comments; the missing failure cases and messages (an unset token, no deploy URL, a missing Pages project) have troubleshooting entries; both QuantEcon images trust every directory for git, so the safe.directory advice is for other images. restore-jupyter-cache: - The caches save execution time only if the book sets execute_notebooks: cache. Sphinx re-reads every page of a fresh checkout, so the claim that it rewrites only changed pages is gone. - A different environment-update still hits through the fallback, and neither key records the container image. - save-cache: a cache saved from a branch reaches every pull request, so it is for pull-request builds only; a re-run matches its own key exactly and saves nothing; a custom key is saved once. - The execution cache's fallback can hand back outputs made with old packages; caches are also evicted when storage is full. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01FeKTxWtkzVMyPbqeSgM2Tb --- docs/user/actions/preview-cloudflare.md | 34 ++++---- docs/user/actions/preview-netlify.md | 92 +++++++++++++++++----- docs/user/actions/restore-jupyter-cache.md | 44 ++++++----- preview-cloudflare/action.yml | 12 +-- preview-netlify/action.yml | 12 +-- restore-jupyter-cache/action.yml | 15 ++-- 6 files changed, 138 insertions(+), 71 deletions(-) diff --git a/docs/user/actions/preview-cloudflare.md b/docs/user/actions/preview-cloudflare.md index 2eeadfb..8acd656 100644 --- a/docs/user/actions/preview-cloudflare.md +++ b/docs/user/actions/preview-cloudflare.md @@ -5,8 +5,8 @@ Deploys a pull request's built site to a Cloudflare Pages project as a preview, on the branch `pr-`, so the preview's URL, `https://pr-..pages.dev`, stays the same for every push to the pull request. Each deployment also gets a URL of its own, which goes on showing that one push. The action then posts one comment on the pull request, and updates it on each later push, with: - the preview URL and the commit it shows; -- a link to the page of each lecture the pull request adds or modifies, under "Changed Lectures"; -- the URL of this push's own deployment, under "Build Info". +- under "Changed Lectures", a link to the page of each lecture that differs between the pull request and its base branch; +- under "Build Info", a link to the workflow run, and the URL of this push's own deployment. It deploys only on `pull_request` events, and skips pull requests from forks and from Dependabot, whose runs get no secrets. The project's production site is never touched. @@ -29,7 +29,9 @@ Use it in a pull request's build job, after [`build-lectures`](build-lectures.md ## Setting up Cloudflare Pages 1. **A Cloudflare account.** A free one is enough to start. -2. **Create a project that Cloudflare does not build.** In the dashboard, under **Workers & Pages**, create a Pages project with **Upload assets**, which is direct upload, and upload any file as a placeholder. Its name is the `project-name` input and part of every URL, so choose it with care. Such a project has no link to the repository, so Cloudflare neither builds your pull requests nor comments on them: this action does both. +2. **Create a project that Cloudflare does not build.** In the dashboard, under **Workers & Pages**, create a Pages project with **Upload assets**, which is direct upload, and upload any file as a placeholder. Such a project has no link to the repository, so Cloudflare neither builds your pull requests nor comments on them: this action deploys and comments instead. + - Its name is the `project-name` input. + - **Check its address.** The action builds the preview URL, and every lecture link in the comment, as `https://pr-..pages.dev`. Cloudflare can give a project a `pages.dev` address other than its name, as it does when that name is already taken, and those URLs would then be wrong. After creating the project, check that it is served at exactly `https://.pages.dev`; if not, create one under a name that is free. See [#216](https://github.com/QuantEcon/actions/issues/216). 3. **Find the account ID.** It is in the dashboard's URL, `https://dash.cloudflare.com//…`, and on the **Workers & Pages** overview. 4. **Create an API token** under **My Profile**, **API Tokens**, as a custom token with a single permission, **Account**, **Cloudflare Pages**, **Edit**, for the account that owns the project. Copy it when it is shown: Cloudflare shows it only once. 5. **Add both as repository secrets**, under the repository's **Settings**, **Secrets and variables**, **Actions**: @@ -57,7 +59,7 @@ The previews then appear at: | `cloudflare-account-id` | yes | none | The ID of the Cloudflare account that owns the Pages project, from a repository secret such as `secrets.CLOUDFLARE_ACCOUNT_ID`. | | `project-name` | yes | none | The Cloudflare Pages project to deploy to, which must already exist. Its name is part of every preview URL, as in `.pages.dev`. | | `build-dir` | yes | none | The directory that holds the built site, such as `_build/html`, which `build-lectures` gives in its `build-path` output. | -| `lectures-dir` | no | `lectures` | The directory of lecture `.md` files that change detection looks in: each one the pull request adds or modifies gets a link in the comment, and is listed in `changed-files`. Empty turns change detection off, and the comment gives only the preview URL. | +| `lectures-dir` | no | `lectures` | The directory of lecture `.md` files that change detection looks in, as a path from the repository root such as `lectures`, with no `./` or trailing slash: each lecture that differs from the base branch gets a link in the comment, and is listed in `changed-files`. Empty turns change detection off, and the comment gives only the preview URL. | @@ -69,7 +71,7 @@ The previews then appear at: |---|---| | `deploy-url` | The pull request's preview URL, `https://pr-..pages.dev`, which always shows its newest push, so use it for anything a person will follow. Empty when nothing was deployed: on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | | `deployment-url` | The URL of this one deployment, `https://..pages.dev`, which goes on showing this push after later ones. Empty when wrangler's output does not name it, and when nothing was deployed. | -| `changed-files` | The lecture files that the pull request adds or modifies under `lectures-dir`, one path per line. Empty when there are none, with `lectures-dir: ''`, on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | +| `changed-files` | The lecture files under `lectures-dir` that were added or modified between the tip of the pull request's base branch and its head, one path per line, so a lecture changed on the base branch since the pull request branched off is listed too. Empty when there are none, with `lectures-dir: ''`, on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | @@ -113,23 +115,23 @@ jobs: ## Behaviour 1. **Who gets a preview.** A run by Dependabot, or for a pull request from a fork, stops here: it prints `⚠️ Cloudflare deployment skipped (untrusted actor or fork PR)`, deploys nothing, and passes. -2. **Changed lectures.** On a `pull_request` event with `lectures-dir` set, the action lists each `.md` file under `lectures-dir` that the pull request adds or modifies, as [preview-netlify](preview-netlify.md#behaviour) does. +2. **Changed lectures.** On a `pull_request` event with `lectures-dir` set, the action lists the lectures that differ between the pull request's base, which is the tip of its base branch, and its head, by the same rules as [preview-netlify](preview-netlify.md#behaviour). That includes a lecture changed on the base branch since the pull request branched off ([#215](https://github.com/QuantEcon/actions/issues/215)). 3. **The CLI.** The `wrangler` version pinned in the action's [`package.json`](https://github.com/QuantEcon/actions/blob/main/preview-cloudflare/package.json) is installed with `npm ci`, into a temporary directory, and its `wrangler` command is added to the `PATH` of the job's later steps. 4. **The deploy**, on `pull_request` events only: ```text - wrangler pages deploy --project-name --branch pr- --commit-hash --commit-message "PR # ()" + wrangler pages deploy --project-name --branch pr- --commit-hash --commit-message "PR # ()" ``` - The token and the account ID reach it through the environment. A branch other than the project's production branch makes the deployment a preview, so the production site is untouched. wrangler's output is in the "Deployment Output" log group. -5. **The URLs.** `deploy-url` is not read from wrangler's output but built from the inputs: `https://pr-..pages.dev` is the alias Cloudflare gives the newest deployment of the branch `pr-`. `deployment-url` is the first other `pages.dev` address in wrangler's output. -6. **The comment.** The action updates the pull request's comment that starts `## ☁️ Cloudflare Preview Ready!`, or posts one, with the lecture links built as in preview-netlify's. + `` is the first 7 characters of the head commit. The token and the account ID reach the CLI through the environment. A branch other than the project's production branch makes the deployment a preview, so the production site is untouched. wrangler's output is in the "Deployment Output" log group. +5. **The URLs.** `deploy-url` is not read from wrangler's output but built from the inputs: `https://pr-..pages.dev` is the alias Cloudflare gives the newest deployment of the branch `pr-`, as long as the project's own address is `.pages.dev`. `deployment-url` is the first other `pages.dev` address in wrangler's output. +6. **The comment.** Among the pull request's first 30 comments, the action looks for one that contains `## ☁️ Cloudflare Preview Ready!`, and updates it; if there is none, it posts a new one. Its lecture links are built as in preview-netlify's. -On any event but `pull_request`, such as `push`, `workflow_dispatch` or `schedule`, the action installs the CLI and does nothing else, and nothing in the log says so: the step succeeds, with empty `deploy-url` and `deployment-url` outputs. To know whether a preview was deployed, check `deploy-url`. +On any event but `pull_request`, such as `push`, `workflow_dispatch` or `schedule`, the action installs the CLI and does nothing else, and nothing in the log says so: the step succeeds, with empty `deploy-url` and `deployment-url` outputs. To know whether a preview was deployed, check `deploy-url`. The one exception is a run by Dependabot, which prints the skip message and installs nothing. ### What fails the job -- `npm ci` failing to install the pinned CLI. +- `npm ci` failing to install the pinned CLI, or the installed CLI failing to run. - `wrangler pages deploy` failing. - The comment failing to post, for want of `pull-requests: write`. @@ -146,12 +148,14 @@ Previews of pull requests from forks are not supported, and the action skips the **`wrangler pages deploy failed (exit …) — see the Deployment Output group above`.** wrangler's own message is in that group. - An authentication error: the token lacks **Cloudflare Pages**, **Edit**, has expired, or belongs to another account than `CLOUDFLARE_ACCOUNT_ID`. -- A project that cannot be found: `project-name` must match the project's name exactly, in the account that `CLOUDFLARE_ACCOUNT_ID` names. +- `The Pages project "…" does not exist.`: `project-name` must match the project's name exactly, in the account that `CLOUDFLARE_ACCOUNT_ID` names. The action never creates a project. -**The preview URL answers 404.** A new alias can take a few seconds to appear. If it lasts, check that `build-dir` holds an `index.html`. +**The preview URL answers 404, or shows another site.** Check first that the project is served at exactly `https://.pages.dev`: if Cloudflare gave it another address, the URLs the action builds are wrong, as [Setting up Cloudflare Pages](#setting-up-cloudflare-pages) explains. The comment's "This deployment" link, read from wrangler's output, is right either way. Otherwise, a new alias can take a few seconds to appear; if the 404 lasts, check that `build-dir` holds an `index.html`. **`Resource not accessible by integration`, when commenting.** The job lacks `pull-requests: write`. -**The comment lists no changed lectures, or a link leads to a missing page.** See [preview-netlify's troubleshooting](preview-netlify.md#troubleshooting): the two actions detect and link lectures the same way. +**The comment lists no changed lectures, lists one the pull request did not change, or a link leads to a missing page.** See [preview-netlify's troubleshooting](preview-netlify.md#troubleshooting): the two actions detect and link lectures the same way. + +**A new preview comment on every push.** The pull request has more than 30 comments before the preview's, so the action does not find its own comment, and posts another. **The comment has no "This deployment" line.** wrangler's output named no per-deployment URL, so `deployment-url` is empty. The preview URL is not affected. diff --git a/docs/user/actions/preview-netlify.md b/docs/user/actions/preview-netlify.md index b0c855d..cfa4397 100644 --- a/docs/user/actions/preview-netlify.md +++ b/docs/user/actions/preview-netlify.md @@ -5,10 +5,13 @@ Deploys a pull request's built site to a Netlify site as a preview, under the alias `pr-`, so the preview's URL, `https://pr---.netlify.app`, stays the same for every push to the pull request. It then posts one comment on the pull request, and updates it on each later push, with: - the preview URL and the commit it shows; -- a link to the page of each lecture the pull request adds or modifies, under "Changed Lectures". +- under "Changed Lectures", a link to the page of each lecture that differs between the pull request and its base branch; +- under "Build Info", a link to the workflow run. It deploys only on `pull_request` events, and skips pull requests from forks and from Dependabot, whose runs get no secrets. The Netlify site's production deploy is never touched. +Netlify's dashboard calls a site a *project*. The action's input and secret keep the word *site*: `netlify-site-id` and `NETLIFY_SITE_ID` take the project's ID. + ## When to use it Use it in a pull request's build job, after [`build-lectures`](build-lectures.md), as [`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml) does. @@ -21,7 +24,7 @@ Use it in a pull request's build job, after [`build-lectures`](build-lectures.md | | `preview-netlify` | `preview-cloudflare` | |---|---|---| | Preview URL | `https://pr---.netlify.app` | `https://pr-..pages.dev` | -| A URL for each push | no | yes, in `deployment-url` and in the comment | +| A URL for each push | not reported, though Netlify keeps one for each deploy | yes, in `deployment-url` and in the comment | | Repository secrets | `NETLIFY_AUTH_TOKEN`, `NETLIFY_SITE_ID` | `CLOUDFLARE_API_TOKEN`, `CLOUDFLARE_ACCOUNT_ID` | | Other inputs it needs | none | `project-name` | | CLI, installed each run | `netlify-cli`, which needs Node 22.13 or later | `wrangler`, which needs Node 22 or later | @@ -36,25 +39,33 @@ Both deploy the files `build-lectures` built, with no build on the provider's si - **Permissions.** `pull-requests: write`, for the comment. The job also needs `contents: read` for `actions/checkout`. - **Secrets.** `NETLIFY_AUTH_TOKEN` and `NETLIFY_SITE_ID`, from [Setting up Netlify](#setting-up-netlify). - **Files in your repository.** None beyond the built site. The comment's lecture links need the pull request's base and head commits, which the action fetches if the checkout lacks them. -- **Setup outside GitHub.** A Netlify site, once: see [Setting up Netlify](#setting-up-netlify). +- **Setup outside GitHub.** A Netlify project, once: see [Setting up Netlify](#setting-up-netlify). ## Setting up Netlify -1. **Create a site that Netlify does not build.** In Netlify, add a new site with **Deploy manually**, and drop any folder on it as a placeholder. Such a site has no link to the repository, so Netlify neither builds your pull requests nor comments on them: this action does both. +1. **Create a project that Netlify does not build.** In Netlify, add a new project with **Deploy manually**, and drop any folder on it as a placeholder. Such a project has no link to the repository, so Netlify neither builds your pull requests nor comments on them: this action deploys and comments instead. 2. **Create a personal access token**, under **User settings**, **Applications**, **Personal access tokens**. Give it a name that says what it is for, such as `github-actions`, and copy it when it is shown: Netlify shows it only once. -3. **Find the site ID**, under the site's **Site configuration**, **General**, **Site details**. It is a UUID, such as `a1b2c3d4-e5f6-…`. +3. **Find the project ID**, under the project's configuration, **General**, in its details. It is a UUID, such as `a1b2c3d4-e5f6-…`. 4. **Add both as repository secrets**, under the repository's **Settings**, **Secrets and variables**, **Actions**: | Secret | Value | |---|---| | `NETLIFY_AUTH_TOKEN` | the token from step 2 | - | `NETLIFY_SITE_ID` | the site ID from step 3 | + | `NETLIFY_SITE_ID` | the project ID from step 3 | + +A project that is already linked to the repository builds each pull request itself, and comments on it, beside this action. What to do depends on the project: -A site that is already linked to the repository builds each pull request itself, and comments on it, beside this action. To stop that, do one of these under the site's **Site configuration**: +| The Netlify project | Do this | +|---|---| +| New | Create it with **Deploy manually**, as above. | +| Linked to the repository | Unlink it. This is the usual choice. | +| Linked, and you want to keep Netlify's own builds | Turn off only its pull-request comments. | -- **Unlink the repository**, under **Build & deploy**, **Continuous deployment**. The site keeps serving, but Netlify no longer builds or comments. -- **Turn off Netlify's pull-request comments** only, under **Notifications**: remove its GitHub commit and pull-request comments. -- **Turn off deploy previews**, under **Build & deploy**, **Continuous deployment**, **Branches and deploy contexts**: set **Deploy Previews** to **None**. +All three settings are in the project's configuration: + +- **Unlink the repository**, under **Build & deploy**, **Continuous deployment**. The project keeps serving, but Netlify no longer builds or comments. +- **Turn off Netlify's pull-request comments**, under **Notifications**: remove its GitHub commit and pull-request comments. +- **Turn off deploy previews** as well, if Netlify should not build pull requests at all: under **Build & deploy**, **Continuous deployment**, **Branches and deploy contexts**, set **Deploy Previews** to **None**. ## Inputs @@ -65,7 +76,7 @@ A site that is already linked to the repository builds each pull request itself, | `netlify-auth-token` | yes | none | A Netlify personal access token, from a repository secret such as `secrets.NETLIFY_AUTH_TOKEN`. It reaches `netlify-cli` through the environment, never on its command line. | | `netlify-site-id` | yes | none | The ID of the Netlify site to deploy to, from a repository secret such as `secrets.NETLIFY_SITE_ID`. | | `build-dir` | yes | none | The directory that holds the built site, such as `_build/html`, which `build-lectures` gives in its `build-path` output. | -| `lectures-dir` | no | `lectures` | The directory of lecture `.md` files that change detection looks in: each one the pull request adds or modifies gets a link in the comment, and is listed in `changed-files`. Empty turns change detection off, and the comment gives only the preview URL. | +| `lectures-dir` | no | `lectures` | The directory of lecture `.md` files that change detection looks in, as a path from the repository root such as `lectures`, with no `./` or trailing slash: each lecture that differs from the base branch gets a link in the comment, and is listed in `changed-files`. Empty turns change detection off, and the comment gives only the preview URL. | @@ -76,7 +87,7 @@ A site that is already linked to the repository builds each pull request itself, | Output | Description | |---|---| | `deploy-url` | The preview's URL, `https://pr---.netlify.app`, the same for every push to the pull request. Empty when nothing was deployed: on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | -| `changed-files` | The lecture files that the pull request adds or modifies under `lectures-dir`, one path per line. Empty when there are none, with `lectures-dir: ''`, on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | +| `changed-files` | The lecture files under `lectures-dir` that were added or modified between the tip of the pull request's base branch and its head, one path per line, so a lecture changed on the base branch since the pull request branched off is listed too. Empty when there are none, with `lectures-dir: ''`, on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. | @@ -134,22 +145,45 @@ Set up Node first: ## Behaviour 1. **Who gets a preview.** A run by Dependabot, or for a pull request from a fork, stops here: it prints `⚠️ Netlify deployment skipped (untrusted actor or fork PR)`, deploys nothing, and passes. -2. **Changed lectures.** On a `pull_request` event with `lectures-dir` set, the action compares the pull request's base and head commits. It lists each `.md` file under `lectures-dir` that the pull request adds or modifies, except `/intro.md` and files whose path within `lectures-dir` starts with `_`. A renamed lecture counts as added; a deleted one is not listed. +2. **Changed lectures.** On a `pull_request` event with `lectures-dir` set, the action compares two commits: the pull request's base, which is the tip of its base branch, and its head. It lists each `.md` file under `lectures-dir` that was added or modified between them, except `/intro.md` and files whose path within `lectures-dir` starts with `_`. A renamed lecture counts as added; a deleted one is not listed. + - Because the comparison starts from the base branch's tip, not from where the pull request branched off, a lecture changed on the base branch since then is listed too, although the pull request did not touch it. See [#215](https://github.com/QuantEcon/actions/issues/215). + - `lectures-dir` is matched against paths from the repository root, as git writes them: give it as `lectures`, not `./lectures` or `lectures/`, which match nothing. Lectures at the repository root cannot be listed. 3. **The CLI.** The `netlify-cli` version pinned in the action's [`package.json`](https://github.com/QuantEcon/actions/blob/main/preview-netlify/package.json) is installed with `npm ci`, into a temporary directory, and its `netlify` command is added to the `PATH` of the job's later steps. 4. **The deploy**, on `pull_request` events only: ```text - netlify deploy --no-build --dir --alias pr- --message "PR # ()" --json + netlify deploy --no-build --dir --alias pr- --message "PR # ()" --json ``` - The token and the site ID reach it through the environment. `--no-build` stops Netlify from running a build of its own, and without `--prod` the deploy is a preview, so the site's production deploy is untouched. Netlify's output is in the "Deployment Output" log group. -5. **The comment.** The action updates the pull request's comment that starts `## 📖 Netlify Preview Ready!`, or posts one. Each changed lecture links to `/.html`. `` is the file's path without `.md`, and also without the `lectures-dir/` in front when `_toc.yml` is in `lectures-dir`, because Jupyter Book then builds the pages relative to it. + `` is the first 7 characters of the pull request's head commit. The token and the site ID reach the CLI through the environment. `--no-build` stops Netlify from running a build of its own, and without `--prod` the deploy is a preview, so the site's production deploy is untouched. Netlify's output is in the "Deployment Output" log group. +5. **The comment.** Among the pull request's first 30 comments, the action looks for one that contains `## 📖 Netlify Preview Ready!`, and updates it; if there is none, it posts a new one. Each changed lecture links to `/.html`. `` is the file's path without `.md`, and also without the `lectures-dir/` in front when `_toc.yml` is in `lectures-dir`, because Jupyter Book then builds the pages relative to it. + + For a pull request that changes two lectures, the comment's Markdown is: + + ````markdown + ## 📖 Netlify Preview Ready! + + **Preview URL:** https://pr-5--.netlify.app + + **Commit:** [`abc1234`](https://github.com///commit/) + + ### 📚 Changed Lectures -On any event but `pull_request`, such as `push`, `workflow_dispatch` or `schedule`, the action installs the CLI and does nothing else, and nothing in the log says so: the step succeeds, with an empty `deploy-url`. To know whether a preview was deployed, check `deploy-url`. + - [aiyagari](https://pr-5--.netlify.app/aiyagari.html) + - [mccall_model](https://pr-5--.netlify.app/mccall_model.html) + + --- +
Build Info + + - **Workflow:** [](https://github.com///actions/runs/) +
+ ```` + +On any event but `pull_request`, such as `push`, `workflow_dispatch` or `schedule`, the action installs the CLI and does nothing else, and nothing in the log says so: the step succeeds, with an empty `deploy-url`. To know whether a preview was deployed, check `deploy-url`. The one exception is a run by Dependabot, which prints the skip message and installs nothing. ### What fails the job -- `npm ci` failing to install the pinned CLI. +- `npm ci` failing to install the pinned CLI, or the installed CLI failing to run. - `netlify deploy` failing, or printing no deploy URL. - The comment failing to post, for want of `pull-requests: write`. @@ -163,14 +197,30 @@ Previews of pull requests from forks are not supported, and the action skips the **`netlify-cli installed but does not run — see the output above`.** Usually a Node older than 22.13. Set up Node 24 before this action. -**`netlify deploy failed (exit …) — see the Deployment Output group above`.** Netlify's own message is in that group. An authorisation error means the token is wrong, revoked or expired; a site that cannot be found means `NETLIFY_SITE_ID` is wrong. Check also that `build-dir` exists. +**`netlify deploy failed (exit …) — see the Deployment Output group above`.** Netlify's own message is in that group: + +- `Authentication required. NETLIFY_AUTH_TOKEN is not set`: the secret is missing, or empty in this repository. +- Any other authorisation error: the token is wrong, revoked or expired. +- A project that cannot be found: `NETLIFY_SITE_ID` is wrong. + +Check also that `build-dir` exists. + +**`no deploy URL in netlify's --json output — see the raw output above`.** The deploy ran, but its output had no `deploy_url`, so the action cannot comment. The raw output is printed above the error. It usually means a new `netlify-cli` changed its output: please report it as an issue in this repository. **`Resource not accessible by integration`, when commenting.** The job lacks `pull-requests: write`. -**The comment lists no changed lectures.** The pull request adds or modifies no `.md` file under `lectures-dir`, other than `intro.md` and files starting with `_`, or `lectures-dir` names the wrong directory. In a container job, git must also be able to read the checkout: `build-lectures` makes git trust it for the rest of the job, so run this action after it, or trust the workspace yourself with `git config --global --add safe.directory "$GITHUB_WORKSPACE"`. +**The comment lists no changed lectures.** Check, in this order: + +- The pull request adds or modifies no lecture that is listed: see [Behaviour](#behaviour) for the files it skips, `/intro.md` and paths within `lectures-dir` that start with `_`. +- `lectures-dir` names the wrong directory, or is written as `./lectures` or `lectures/`: give it as `lectures`. +- git cannot read the checkout, which makes change detection find nothing without a word. That happens in a container job whose image does not trust every directory; both QuantEcon images do. `build-lectures` makes git trust the checkout for the rest of the job, so run this action after it, or trust the workspace yourself with `git config --global --add safe.directory "$GITHUB_WORKSPACE"`. + +**The comment lists a lecture the pull request did not change.** The lecture was changed on the base branch after the pull request branched off: see [Behaviour](#behaviour). **A lecture link leads to a missing page.** The links assume Jupyter Book's layout: pages relative to `lectures-dir` when `_toc.yml` is inside it, and relative to the repository root when it is not. -**Two comments on each pull request, one of them from Netlify.** The Netlify site is linked to the repository: see the end of [Setting up Netlify](#setting-up-netlify). +**Two comments on each pull request, one of them from Netlify.** The Netlify project is linked to the repository: see the end of [Setting up Netlify](#setting-up-netlify). + +**A new preview comment on every push.** The pull request has more than 30 comments before the preview's, so the action does not find its own comment, and posts another. **A pull request from a fork got no preview.** Expected: its runs get no secrets, so the action skips them. See [Security](#security). diff --git a/docs/user/actions/restore-jupyter-cache.md b/docs/user/actions/restore-jupyter-cache.md index 17ae1fd..8bd2c0c 100644 --- a/docs/user/actions/restore-jupyter-cache.md +++ b/docs/user/actions/restore-jupyter-cache.md @@ -2,12 +2,14 @@ > This describes `main`. For the version you pin, open the manual at that tag. Changes not yet released are under `[Unreleased]` in the [CHANGELOG](https://github.com/QuantEcon/actions/blob/main/CHANGELOG.md). -Restores a cache that [`build-jupyter-cache`](build-jupyter-cache.md) saved, before a build, so the build starts from the last cache build instead of from nothing. Jupyter Book then re-executes only the notebooks whose code has changed, and Sphinx rewrites only the pages that have. It restores one of two caches, chosen with `cache-type`: +Restores a cache that [`build-jupyter-cache`](build-jupyter-cache.md) saved, before a build, so the build starts from the last cache build instead of from nothing. Jupyter Book then re-executes only the notebooks whose code has changed, which is where most of a lecture build's time goes. That needs the book to execute its notebooks through that cache, with `execute_notebooks: cache` in its `_config.yml`. -- **`build`**, the default: the whole `_build` directory, with the HTML, the PDF, the notebooks and the execution cache. +It restores one of two caches, chosen with `cache-type`: + +- **`build`**, the default: the whole `_build` directory, as the cache build left it. That is the HTML and the execution cache, and the PDF and notebooks if the cache build made them. - **`execution`**: only `_build/.jupyter_cache`, which holds each notebook's executed outputs. It is much smaller, and saves the execution time, but not the rest of the build. -By default it only restores. With `save-cache: 'true'` it also saves a cache at the end of the job, which only later runs of the same pull request can restore. +By default it only restores. With `save-cache: 'true'` it also saves a cache at the end of the job, which, saved from a pull request's run, only that pull request's later runs can restore. ## When to use it @@ -18,12 +20,12 @@ Use it in a job that builds lectures, after `setup-environment` and before [`bui ## Requirements -- **Runner.** Any runner or container. +- **Runner.** Any runner or container with `bash`. A job on Windows never finds a cache saved on Linux, since `actions/cache` keeps the two apart. - **Node.** None. - **Permissions.** None: restoring and saving a cache need no `permissions:`. The job needs `contents: read` for `actions/checkout`. - **Secrets.** None. -- **Files in your repository.** Only what the key hashes: for the build cache, the same `environment` and `environment-update` files that `build-jupyter-cache` was given. -- **A cache to restore.** `build-jupyter-cache` must have succeeded on the default branch, or on the branch a pull request targets. GitHub removes a cache that goes 7 days without being restored. +- **Files in your repository.** A `_config.yml` whose `execute:` section sets `execute_notebooks: cache`. With Jupyter Book's default, `auto`, notebooks are executed without jupyter-cache, so there is no execution cache to save or restore. For the build cache's key, the same `environment` file that `build-jupyter-cache` was given. +- **A cache to restore.** `build-jupyter-cache` must have succeeded on the default branch, or on the branch a pull request targets. GitHub removes a cache that goes 7 days without being restored, and the least recently used caches first when the repository's cache storage is full. - **Setup outside GitHub.** None. ## Inputs @@ -37,9 +39,9 @@ Use it in a job that builds lectures, after `setup-environment` and before [`bui | `source-dir` | no | `lectures` | With `cache-type: execution`: the book's directory. The key hashes every `.md` file under it, as `build-jupyter-cache` does with its own `source-dir`. Ignored for the build cache. | | `environment` | no | `environment.yml` | With `cache-type: build`: the Conda environment file whose hash is part of the key. Give the same file that `build-jupyter-cache` was given, or no cache matches. Ignored for the execution cache. | | `environment-update` | no | `''` | With `cache-type: build`: the container delta file whose hash is part of the key. Give the same file that `build-jupyter-cache` was given; with another, only the fallback matches, restoring the newest cache built from the same `environment`. Ignored for the execution cache. | -| `key` | no | `''` | A key to restore in place of the generated ones. It is matched as a prefix, so the newest cache whose key starts with it is restored, and with `save-cache: 'true'` the cache is saved under exactly this key. Empty, the default, uses the generated keys. | +| `key` | no | `''` | A key to restore in place of the generated ones. It is matched as a prefix, so the newest cache whose key starts with it is restored. With `save-cache: 'true'` the cache is saved under exactly this key, and only once: a cache is never overwritten, and a later run that matches the key exactly saves nothing. Empty, the default, uses the generated keys. | | `fail-on-miss` | no | `false` | `'true'` fails the job when nothing is restored. `'false'`, the default, carries on without a cache, and the build starts from scratch. | -| `save-cache` | no | `false` | `'true'` also saves the directory at the end of the job, if the job succeeds, under a key that ends in the run id. GitHub scopes a cache to the branch or pull request that saved it, so only later runs of the same pull request restore it. `'false'`, the default, only restores. | +| `save-cache` | no | `false` | `'true'` also saves the directory at the end of the job, if the job succeeds and its key did not match exactly. Saved from a pull request's run, the cache can be restored only by that pull request's later runs; saved from a run on a branch, it reaches every pull request that targets the branch. So use it in pull-request builds only. `'false'`, the default, only restores. | @@ -49,7 +51,7 @@ Use it in a job that builds lectures, after `setup-environment` and before [`bui | Output | Description | |---|---| -| `cache-hit` | `'true'` when a cache was restored, whether its key matched exactly or only as a prefix; `'false'` when nothing was restored. Always set. Unlike the `actions/cache` output of the same name, it counts a prefix match, which is how the generated keys always match. | +| `cache-hit` | `'true'` when a cache was restored, whether its key matched exactly or only as a prefix; `'false'` when nothing was restored. Always set. Unlike the `actions/cache` output of the same name, it counts a prefix match, which is how a read-only restore with the generated keys always matches. | | `cache-key` | The key of the cache that was restored: a saved key such as `build---`, not the prefix that found it. Empty when nothing was restored. | @@ -88,7 +90,11 @@ jobs: save-cache: 'true' ``` -At the end of a successful job, `_build` is saved under a key that ends in the run id. The pull request's later runs restore it, so each one re-executes only what changed since its previous run, not since the last cache build. No other pull request, and no branch, can restore it. +At the end of a successful job, `_build` is saved under a key that ends in the run id. The pull request's later runs restore it, so each one re-executes only what changed since its previous run, not since the last cache build. + +- **Only in pull-request builds.** Saved from a pull request's run, the cache is visible only to that pull request's later runs. Saved from a run on a branch, as in `publish.yml`, it would reach every pull request that targets the branch, and as the newest match it would win over the cache build's. +- **One save per run.** A re-run of the job, or a second job in the same run, finds this run's own key as an exact match, and saves nothing. +- **Storage.** Each run saves a whole `_build`, and GitHub removes the least recently used caches when the repository's cache storage is full. ### Only the execution cache @@ -116,21 +122,21 @@ The job fails when no cache is found, for a build that would take too long witho ### Keys -The action restores the newest cache whose key starts with the first of these prefixes to match any cache, and then the next: +The action looks up a key, and then a fallback: | `cache-type` | Restored to | Key | Fallback | |---|---|---|---| | `build` | `` | `build---` | `build--` | | `execution` | `/.jupyter_cache` | `jupyter-cache--` | `jupyter-cache-` | -`build-jupyter-cache` saves its caches under the same prefixes, followed by its run id. A file that does not exist hashes to an empty string. With `save-cache: 'true'` the key ends in this run's id instead, and is the one the cache is saved under. With `key` set, that key is the only prefix. +`build-jupyter-cache` saves its caches under the same prefixes, followed by its run id. A file that does not exist hashes to an empty string. With `save-cache: 'true'` the key ends in this run's id instead, and is the one the cache is saved under. With `key` set, that key is both the key and the only fallback. -GitHub looks for a match among the caches of the branch or pull request the job runs for before it looks at those of the default branch, or of the branch a pull request targets. So a pull request that has saved its own cache restores that one, even when a newer cache build has run since. +GitHub looks first among the caches of the branch or pull request the job runs for, trying the key and then the fallback, and then the same among the caches of the base and default branches. For each, an exact match wins, and otherwise the newest cache whose key starts with it is restored. So a pull request that has saved its own cache restores that one, even when a newer cache build has run since. ### Why the keys are what they are -- **The build cache never falls back across environments.** `_build` does not record the packages that produced it. Restored after a change to the environment file, it would hand the build pages and outputs made with the old packages. An edited environment file is a miss on purpose, and the build starts from scratch. -- **The execution cache does fall back to any execution cache.** Jupyter Book checks each cached notebook against the notebook's current code, and re-executes any that no longer match, so an unrelated cache can only save time. +- **The build cache never falls back across `environment` files.** `_build` does not record the packages that produced it. Restored after a change to the environment file, it would hand the build pages and outputs made with the old packages. An edited environment file is a miss on purpose, and the build starts from scratch. The fallback does cross `environment-update` files, and neither key records the container image, so a job on a newer image restores a `_build` made on the old one. +- **The execution cache does fall back to any execution cache.** Jupyter Book checks each cached notebook against the notebook's current code, and re-executes any that no longer match, so a cache made from other lectures is safe as far as the code goes. It does not check the packages: the key hashes only the lectures, so after a change to the environment the execution cache hands back outputs made with the old packages, until the next cache build replaces it. - **The build cache's key does not hash the lectures.** The key is the same for every pull request, so every pull request restores the last cache build and re-executes only its own changes. A key that hashed the lectures would change with every edit, and miss on nearly every pull request. The cost is that a lecture removed since the last cache build keeps its old pages, which nothing links to, until the next cache build replaces `_build`. ### Status report @@ -153,15 +159,15 @@ The action prints what it asked for and what it found. On a typical read-only re ════════════════════════════════════════════════════════════════════ ``` -`Cache Hit` is `actions/cache`'s exact-match flag, which the generated keys never set; `Matched Key` shows what was restored, and the action's own `cache-hit` output is `'true'`. A collapsed "Cache Contents" group follows, with the size of the cache and of each directory in it. On a miss, the report says so, and the build carries on from nothing. +`Cache Hit` is `actions/cache`'s exact-match flag, which a read-only restore with the generated keys never sets; `Matched Key` shows what was restored, and the action's own `cache-hit` output is `'true'`. A collapsed "Cache Contents" group follows, with the size of the cache and of each directory in it. On a miss, the report says so, and the build carries on from nothing. ## Troubleshooting -**`⚠️ No cache found`.** In order of likelihood: +**`No cache found`.** In order of likelihood: - The pull request changes the environment file. That is a miss on purpose: the build runs from scratch, and the cache build after the merge saves a new cache. -- The default branch has no cache: the cache build has not succeeded yet, or its cache went 7 days without being restored. Run the cache workflow by hand, for example with `gh workflow run cache.yml`. -- `environment` or `environment-update` differs from what `build-jupyter-cache` was given, or `path` is not `_build`. +- The default branch has no cache: the cache build has not succeeded yet, or GitHub has removed its cache, after 7 days without a restore or to make room. Run the cache workflow by hand, for example with `gh workflow run cache.yml`. +- `environment` differs from what `build-jupyter-cache` was given, or `path` is not `_build`. - The cache build and this job compress differently. `actions/cache` compresses with `zstd` where it is installed and with `gzip` where it is not, and finds only caches made the same way. Run both jobs in the same image, or both on the runner. **`Cache miss with fail-on-miss enabled`.** As above; the job failed because `fail-on-miss` is `'true'`. diff --git a/preview-cloudflare/action.yml b/preview-cloudflare/action.yml index 2483c4a..a828f2f 100644 --- a/preview-cloudflare/action.yml +++ b/preview-cloudflare/action.yml @@ -31,9 +31,10 @@ inputs: required: true lectures-dir: description: >- - The directory of lecture `.md` files that change detection looks in: each one the pull - request adds or modifies gets a link in the comment, and is listed in `changed-files`. - Empty turns change detection off, and the comment gives only the preview URL. + The directory of lecture `.md` files that change detection looks in, as a path from the + repository root such as `lectures`, with no `./` or trailing slash: each lecture that differs + from the base branch gets a link in the comment, and is listed in `changed-files`. Empty + turns change detection off, and the comment gives only the preview URL. required: false default: 'lectures' @@ -53,8 +54,9 @@ outputs: value: ${{ steps.deploy.outputs.deployment-url }} changed-files: description: >- - The lecture files that the pull request adds or modifies under `lectures-dir`, one path per - line. Empty when there are none, with `lectures-dir: ''`, on an event other than + The lecture files under `lectures-dir` that were added or modified between the tip of the + pull request's base branch and its head, one path per line, so a lecture changed on the base + branch since the pull request branched off is listed too. Empty when there are none, with `lectures-dir: ''`, on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. value: ${{ steps.detect-changes.outputs.changed-files }} diff --git a/preview-netlify/action.yml b/preview-netlify/action.yml index fa036e0..48291c3 100644 --- a/preview-netlify/action.yml +++ b/preview-netlify/action.yml @@ -26,9 +26,10 @@ inputs: required: true lectures-dir: description: >- - The directory of lecture `.md` files that change detection looks in: each one the pull - request adds or modifies gets a link in the comment, and is listed in `changed-files`. - Empty turns change detection off, and the comment gives only the preview URL. + The directory of lecture `.md` files that change detection looks in, as a path from the + repository root such as `lectures`, with no `./` or trailing slash: each lecture that differs + from the base branch gets a link in the comment, and is listed in `changed-files`. Empty + turns change detection off, and the comment gives only the preview URL. required: false default: 'lectures' @@ -41,8 +42,9 @@ outputs: value: ${{ steps.deploy.outputs.deploy-url }} changed-files: description: >- - The lecture files that the pull request adds or modifies under `lectures-dir`, one path per - line. Empty when there are none, with `lectures-dir: ''`, on an event other than + The lecture files under `lectures-dir` that were added or modified between the tip of the + pull request's base branch and its head, one path per line, so a lecture changed on the base + branch since the pull request branched off is listed too. Empty when there are none, with `lectures-dir: ''`, on an event other than `pull_request`, and for a pull request from a fork or by Dependabot. value: ${{ steps.detect-changes.outputs.changed-files }} diff --git a/restore-jupyter-cache/action.yml b/restore-jupyter-cache/action.yml index 59c0e44..f420a80 100644 --- a/restore-jupyter-cache/action.yml +++ b/restore-jupyter-cache/action.yml @@ -49,8 +49,9 @@ inputs: key: description: >- A key to restore in place of the generated ones. It is matched as a prefix, so the newest - cache whose key starts with it is restored, and with `save-cache: 'true'` the cache is saved - under exactly this key. Empty, the default, uses the generated keys. + cache whose key starts with it is restored. With `save-cache: 'true'` the cache is saved + under exactly this key, and only once: a cache is never overwritten, and a later run that + matches the key exactly saves nothing. Empty, the default, uses the generated keys. required: false default: '' fail-on-miss: @@ -61,9 +62,10 @@ inputs: default: 'false' save-cache: description: >- - `'true'` also saves the directory at the end of the job, if the job succeeds, under a key - that ends in the run id. GitHub scopes a cache to the branch or pull request that saved it, - so only later runs of the same pull request restore it. `'false'`, the default, only + `'true'` also saves the directory at the end of the job, if the job succeeds and its key did + not match exactly. Saved from a pull request's run, the cache can be restored only by that + pull request's later runs; saved from a run on a branch, it reaches every pull request that + targets the branch. So use it in pull-request builds only. `'false'`, the default, only restores. required: false default: 'false' @@ -73,7 +75,8 @@ outputs: description: >- `'true'` when a cache was restored, whether its key matched exactly or only as a prefix; `'false'` when nothing was restored. Always set. Unlike the `actions/cache` output of the - same name, it counts a prefix match, which is how the generated keys always match. + same name, it counts a prefix match, which is how a read-only restore with the generated + keys always matches. value: ${{ steps.restore.outputs.cache-matched-key != '' }} cache-key: description: >- From 6a1328a4edf2bd63f4e7f87f9f08842a846b14b8 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 28 Sep 2026 11:32:26 +0000 Subject: [PATCH 3/4] docs(manual): correct the publish-gh-pages and deploy-cloudflare chapters From the fact-check of both chapters against the actions, the gate probe and wrangler 4.139.0's source. deploy-cloudflare: - On a hostname found ungated, the chapter and the action's error message said to turn off the Worker's workers.dev route. With the route off, preview URLs stay on, and wrangler itself warns they may be public. Both now say to turn off the route and the preview URLs. - The Worker's Access setting is not shown to cover custom domains, and the action never checks them: the chapter now says to check a custom domain by hand with check-access-gate.sh. - The alias is uploaded only after the deploy and the check after it pass; the probe skips dot-directories too; worker-name cannot start or end with a dash; a require-access value other than true or false fails. - The Terraform resource names carry their cloudflare_ prefix; outside collaborators are not members, so the audience example is a member who joined for a translation or a course; the default login method is named. check-access-gate.sh's usage example uses a placeholder team. publish-gh-pages: - A tag containing / fails the release assets, after the site is live. - size_mb, like file_count, is measured before cname writes its file. - The github-token check applies only to tag-triggered runs; a release is found by its tag and created if missing; the deploy uses the job's token as well as its OIDC token. - Moving from a gh-pages branch gains the environment step for tag deploys and a link to the migration guide, and drops a reference to an input no released version had. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01FeKTxWtkzVMyPbqeSgM2Tb --- deploy-cloudflare/action.yml | 10 ++++----- docs/user/actions/deploy-cloudflare.md | 24 ++++++++++----------- docs/user/actions/publish-gh-pages.md | 30 ++++++++++++++++---------- publish-gh-pages/action.yml | 9 ++++---- scripts/check-access-gate.sh | 2 +- 5 files changed, 42 insertions(+), 33 deletions(-) diff --git a/deploy-cloudflare/action.yml b/deploy-cloudflare/action.yml index b80b62a..9f00c9a 100644 --- a/deploy-cloudflare/action.yml +++ b/deploy-cloudflare/action.yml @@ -22,9 +22,9 @@ inputs: required: true worker-name: description: >- - The Worker to deploy to, one per site: lowercase letters, digits and dashes, at most 63 - characters. It must already exist, with its `workers.dev` route on, and already be behind - Access. + The Worker to deploy to, one per site: lowercase letters, digits and dashes, not starting or + ending with a dash, at most 63 characters. It must already exist, with its `workers.dev` + route on, and already be behind Access. required: true account-subdomain: description: >- @@ -55,7 +55,7 @@ inputs: `'true'`, the default, proves the site is gated: before anything is uploaded, after the deploy, on production and on the new version's own preview URL, and on the alias after its upload, an anonymous request must be redirected to `team-domain`. `'false'` skips every - check, with a warning: never use it for private content. + check, with a warning: never use it for private content. Any other value fails the job. required: false default: 'true' @@ -315,7 +315,7 @@ runs: failed=1 fi if ! bash "$GITHUB_ACTION_PATH/../scripts/check-access-gate.sh" "$TEAM" "${urls[@]}"; then - echo "::error::A hostname serving this Worker is NOT behind Access for $TEAM (see above). Treat the site as public until this is fixed: turn Access on for the Worker with All traffic, which also covers its preview URLs, or disable its workers.dev route." + echo "::error::A hostname serving this Worker is NOT behind Access for $TEAM (see above). Treat the site as public until this is fixed: turn Access on for the Worker with All traffic, which also covers its preview URLs, or turn off both its workers.dev route and its preview URLs." failed=1 fi exit "$failed" diff --git a/docs/user/actions/deploy-cloudflare.md b/docs/user/actions/deploy-cloudflare.md index 4a8fb8e..660b301 100644 --- a/docs/user/actions/deploy-cloudflare.md +++ b/docs/user/actions/deploy-cloudflare.md @@ -37,13 +37,13 @@ Use it to publish a site that must not be public. - Authorization callback URL: `https://.cloudflareaccess.com/cdn-cgi/access/callback` Then add it in Zero Trust, under **Settings**, **Authentication**, **Login methods**, **GitHub**. -3. **Make GitHub the only login method.** A new Zero Trust account starts with another login method switched on. Remove it. -4. **One reusable policy per audience.** Under **Access controls**, **Policies**, create an Allow policy with the **GitHub Organization** selector, your organisation, and a named team. Reference it from each application. Use a team rather than the whole organisation: an organisation's members usually include people a private site is not meant for, such as outside collaborators. A session can last up to a month. +3. **Make GitHub the only login method.** A new Zero Trust account starts with *Cloudflare account membership* as a login method. Remove it. +4. **One reusable policy per audience.** Under **Access controls**, **Policies**, create an Allow policy with the **GitHub Organization** selector, your organisation, and a named team. Reference it from each application. Use a team rather than the whole organisation: an organisation's members usually include people a private site is not meant for, such as members who joined for a translation or a course. A session can last up to a month. ### Once per site 1. **Create the Worker by hand, under its final name,** as a placeholder that holds no data, such as the dashboard's Hello World template. Keep its `workers.dev` route on: the action deploys to `https://..workers.dev`, and checks that URL. Creating a Worker needs Admin on the Workers product, which the deploy token deliberately lacks; see Cloudflare's [authorization docs](https://developers.cloudflare.com/workers/authorization/). -2. **Turn Access on for the Worker with *All traffic*,** not *Previews only*. That covers the `workers.dev` hostname, every preview URL, and so every alias, and any custom domain you attach later. Attach the reusable policy, and turn on instant authentication. Use this setting on the Worker, not the account-wide "Protect all Workers" switch, if the account also serves public sites. +2. **Turn Access on for the Worker with *All traffic*,** not *Previews only*. That covers the `workers.dev` hostname and every preview URL, and so every alias. A custom domain needs a check of its own: see [What the generated configuration means for the Worker](#what-the-generated-configuration-means-for-the-worker). Attach the reusable policy, and turn on instant authentication. Use this setting on the Worker, not the account-wide "Protect all Workers" switch, if the account also serves public sites. 3. **Prove the gate on the placeholder** before the first real deploy, with [`check-access-gate.sh`](https://github.com/QuantEcon/actions/blob/main/scripts/check-access-gate.sh) from a clone of this repository: ```bash @@ -61,7 +61,7 @@ Use it to publish a site that must not be public. ### Why the action does not do this setup itself -The Access application is set up once per Worker, and the login method once per account. Doing either from a workflow would put a token that can edit Access applications and policies into every repository that deploys, for a step that runs once. For the same reason the action cannot create the Worker: its token can deploy only to one that exists. An organisation that wants its Cloudflare account under code can manage the same settings with Terraform, whose resources `zero_trust_access_application` and `zero_trust_access_policy` cover them. +The Access application is set up once per Worker, and the login method once per account. Doing either from a workflow would put a token that can edit Access applications and policies into every repository that deploys, for a step that runs once. For the same reason the action cannot create the Worker: its token can deploy only to one that exists. An organisation that wants its Cloudflare account under code can manage the same settings with Terraform, whose resources `cloudflare_zero_trust_access_application` and `cloudflare_zero_trust_access_policy` cover them. ## Inputs @@ -71,12 +71,12 @@ The Access application is set up once per Worker, and the login method once per |---|---|---|---| | `cloudflare-api-token` | yes | none | An account-owned Cloudflare API token with Editor on this one Worker, from a repository secret such as `secrets.CLOUDFLARE_API_TOKEN`. It can deploy the Worker but not create one, so a mistyped `worker-name` fails instead of creating a new Worker that nothing gates. The job fails if the token is empty, as it is on runs from forks and by Dependabot. | | `cloudflare-account-id` | yes | none | The ID of the Cloudflare account that owns the Worker, from a repository secret such as `secrets.CLOUDFLARE_ACCOUNT_ID`. The job fails if it is empty. | -| `worker-name` | yes | none | The Worker to deploy to, one per site: lowercase letters, digits and dashes, at most 63 characters. It must already exist, with its `workers.dev` route on, and already be behind Access. | +| `worker-name` | yes | none | The Worker to deploy to, one per site: lowercase letters, digits and dashes, not starting or ending with a dash, at most 63 characters. It must already exist, with its `workers.dev` route on, and already be behind Access. | | `account-subdomain` | yes | none | The account's `workers.dev` subdomain: `my-subdomain` for `*.my-subdomain.workers.dev`, which `my-subdomain.workers.dev` also gives. The URLs are built from it, never read from wrangler's output. | | `team-domain` | yes | none | The Access team's login domain, `.cloudflareaccess.com`; a bare `` is accepted too. Every gate check requires an anonymous request to be redirected to exactly this host. | | `build-dir` | yes | none | The directory that holds the built site. The job fails if it is missing or holds no files, and warns if it has no `index.html`, since the site's root would then answer 404. | | `alias` | no | `''` | Also uploads the same build as a named preview alias, such as `report-2026-08`, for a permanent URL beside the production one, which moves with each deploy. Lowercase letters, digits and dashes, starting with a letter and not ending with a dash, and `-` must fit in 63 characters. Empty, the default, uploads no alias. | -| `require-access` | no | `true` | `'true'`, the default, proves the site is gated: before anything is uploaded, after the deploy, on production and on the new version's own preview URL, and on the alias after its upload, an anonymous request must be redirected to `team-domain`. `'false'` skips every check, with a warning: never use it for private content. | +| `require-access` | no | `true` | `'true'`, the default, proves the site is gated: before anything is uploaded, after the deploy, on production and on the new version's own preview URL, and on the alias after its upload, an anonymous request must be redirected to `team-domain`. `'false'` skips every check, with a warning: never use it for private content. Any other value fails the job. | @@ -162,8 +162,8 @@ No template uses this action: a private site is a choice made per site, not the 4. **The configuration.** It writes a wrangler configuration for this one run, with the Worker's name, a fixed compatibility date, `build-dir` as the assets directory, and `workers_dev` and `preview_urls` both on. It declares no routes. 5. **The deploy.** `wrangler deploy` publishes `build-dir` to production. Every version also gets its own preview URL, `https://-..workers.dev`, read from the `Current Version ID` line of wrangler's output. 6. **The check after the deploy**, on production and on the new version's preview URL. It runs even when `wrangler deploy` fails, because wrangler can fail after the new version is already live. -7. **The alias**, if given: `wrangler versions upload --preview-alias ` uploads the same build as a preview, then its URL is checked too. -8. **The job summary** gives each URL and the result of its check, or says why nothing was deployed. +7. **The alias**, if given, and only once the deploy and the check after it have passed: `wrangler versions upload --preview-alias ` uploads the same build as a preview, then its URL is checked too. +8. **The job summary** gives each URL and the result of its check, or says why nothing was deployed. Production and the new version's preview URL share one result, that of the check after the deploy. With `require-access: 'false'`, none of the three checks runs, and a warning says that the site is not checked. @@ -180,13 +180,13 @@ Each check is [`check-access-gate.sh`](https://github.com/QuantEcon/actions/blob | `404` | fails: no Worker answers on that hostname, or its `workers.dev` route is off | | Anything else, or no answer | fails: the gate could not be verified | -Each check requests the site's root and one file from `build-dir` that is not HTML: the first, in sorted order, with a URL-safe path, skipping dotfiles and the `_headers`, `_redirects` and `_worker.js` control files. A gate that protected only the root would otherwise pass. Access covers every path on a hostname, so a correct gate redirects both. +Each check requests the site's root and one file from `build-dir` that is not HTML: the first in byte order with a URL-safe path, skipping any path with a part that starts with a dot, and the `_headers`, `_redirects` and `_worker.js` control files. If no file qualifies, only the root is requested. A gate that protected only the root would otherwise pass. Access covers every path on a hostname, so a correct gate redirects both. ### What the generated configuration means for the Worker - **Preview URLs are on** after every deploy, even if they were turned off in the dashboard, which is why the check after the deploy covers the new version's preview URL. - **The deploy replaces the Worker's code**, such as the placeholder's script, without asking. Other settings changed in the dashboard can be overwritten too. -- **Custom domains** are a setting on the Worker in the dashboard, and its Access application covers them. The configuration declares no routes, so a deploy leaves them alone, and the action checks only the `workers.dev` URL. +- **Custom domains** are a setting on the Worker in the dashboard. The configuration declares no routes, so a deploy leaves them alone. The action checks only the `workers.dev` hostname and preview URLs, so check a custom domain yourself with `check-access-gate.sh`, once it is attached and after any change to Access. - **Dotfiles** in `build-dir`, such as Sphinx's `.buildinfo`, are uploaded like any other file, unless a `.assetsignore` file in `build-dir` lists them. ### Limits @@ -200,13 +200,13 @@ On the free plan a Worker's static assets can hold 20,000 files per version, and - `answered 404`: no Worker by that name answers under `account-subdomain`, or its `workers.dev` route is off. Check both, and create the Worker first if it is new. - `THE SITE IS PUBLIC`: Access is off for the Worker, or set to *Previews only*. Turn it on with *All traffic*. - `the Worker is attached to the wrong Access organisation`: the Worker's Access application belongs to another Zero Trust team, or `team-domain` is wrong. -- `not to the Access login domain`: the site redirected elsewhere, or sent no `Location`, so Access is not answering for this hostname. Check that Access is on with *All traffic*, and that `team-domain` is right. +- `not to the Access login domain`: the site redirected elsewhere, sent no `Location`, or sent one whose host is ambiguous, so Access is not answering for this hostname. Check that Access is on with *All traffic*, and that `team-domain` is right. - `expected a redirect to the Access login domain`: another status, such as `401`, `403` or a `5xx`. Check the Worker in the dashboard, then re-run the job. - `could not be reached`: a network problem between the runner and Cloudflare. Re-run the job. **`wrangler deploy failed (exit …)`.** wrangler's own error is printed above it. An authentication or permission error usually means the token lacks Editor on this Worker, the account ID is wrong, or the token has expired. wrangler can fail after the new version is live, so the job summary gives the check that ran after the failure. -**`A hostname serving this Worker is NOT behind Access for …`.** The urgent one: the new build is live, or may be, and an anonymous request to production or to the new version's preview URL was not redirected. The `FAIL` line above names the hostname. Turn Access on for the Worker with *All traffic*, which covers production and every preview URL, or turn off its `workers.dev` route. Then re-run the job to confirm. +**`A hostname serving this Worker is NOT behind Access for …`.** The urgent one: the new build is live, or may be, and an anonymous request to production or to the new version's preview URL was not redirected. The `FAIL` line above names the hostname. Turn Access on for the Worker with *All traffic*, which covers production and every preview URL, then re-run the job to confirm. Until you can, turn off both the Worker's `workers.dev` route and its preview URLs in the dashboard: with the route off, preview URLs stay on, and wrangler itself warns that they may be public. A re-run is then refused, with `answered 404`, until the route is back on. **`wrangler's output named no 'Current Version ID'`.** The new version's preview URL could not be worked out, so it was not checked, and the check fails rather than pass unseen. wrangler prints the ID only once every step after the upload has succeeded, so this usually follows a failed `wrangler deploy`. After a successful one it means a wrangler update changed its output. Check the preview URL by hand with `check-access-gate.sh`, with the version ID from the dashboard. diff --git a/docs/user/actions/publish-gh-pages.md b/docs/user/actions/publish-gh-pages.md index 68501be..3dae4f2 100644 --- a/docs/user/actions/publish-gh-pages.md +++ b/docs/user/actions/publish-gh-pages.md @@ -18,7 +18,7 @@ Use it to publish a public site, after [`build-lectures`](build-lectures.md), as - **Runner.** Any. - **Node.** None. - **Permissions.** `pages: write` and `id-token: write`, for the deploy. The job also needs `contents: read` for `actions/checkout`, or `contents: write` with `create-release-assets: 'true'`, for the release. -- **Secrets.** None. The deploy authenticates with the job's OIDC token, and the release upload with the `github-token` you pass, which can be the job's own `GITHUB_TOKEN`. +- **Secrets.** None. The deploy authenticates with the job's own token and its OIDC token, and the release upload with the `github-token` you pass, which can be the job's own `GITHUB_TOKEN`. - **Files in your repository.** None beyond the built site. - **Repository settings.** - Under **Settings**, **Pages**, set **Source** to **GitHub Actions**, not **Deploy from a branch**. @@ -36,7 +36,7 @@ Use it to publish a public site, after [`build-lectures`](build-lectures.md), as | `cname` | no | `''` | A domain to write into `build-dir` as a `CNAME` file, which also lands in the release archive. It does not set the site's custom domain, and a warning says so: a GitHub Actions Pages deploy ignores `CNAME` files, so set the domain under Settings, Pages. Empty, the default, writes no file. | | `create-release-assets` | no | `false` | `'true'` also attaches the site to the GitHub release of the tag the run was triggered by, creating the release if there is none: an archive, its SHA-256 checksum and a manifest. It needs `github-token`, and a run triggered by a tag: any other run skips it, with a warning. `'false'`, the default, attaches nothing. | | `asset-name` | no | `''` | With `create-release-assets: 'true'`: the start of each asset's file name. Empty, the default, uses the repository's name followed by `-html`, as in `-html`. | -| `github-token` | no | `''` | With `create-release-assets: 'true'`: a token that can write the repository's releases, such as `secrets.GITHUB_TOKEN` in a job with `contents: write`. The job fails if it is empty. The Pages deploy does not use it. | +| `github-token` | no | `''` | With `create-release-assets: 'true'`: a token that can write the repository's releases, such as `secrets.GITHUB_TOKEN` in a job with `contents: write`. On a run triggered by a tag, the job fails if it is empty. The Pages deploy does not use it. | @@ -46,7 +46,7 @@ Use it to publish a public site, after [`build-lectures`](build-lectures.md), as | Output | Description | |---|---| -| `page-url` | The URL of the published site, as GitHub Pages reports it: `https://.github.io//`, or the custom domain's. Set once the deploy succeeds. | +| `page-url` | The URL of the published site, as GitHub Pages reports it, such as `https://.github.io//`, or the custom domain's. Set once the deploy succeeds. | @@ -150,7 +150,8 @@ The `github-pages` environment must also allow the tags to deploy: under **Setti 4. **Release assets**, with `create-release-assets: 'true'`, after the deploy: - On a run not triggered by a tag, a warning says they are skipped, and the job carries on. - With `github-token` empty, the job fails. - - Otherwise the action writes the three files in [Release assets](#release-assets), and uploads them with [action-gh-release](https://github.com/softprops/action-gh-release) to the release named after the tag, which it creates if there is none. + - Otherwise the action writes the three files in [Release assets](#release-assets), and uploads them with [action-gh-release](https://github.com/softprops/action-gh-release) to the tag's release. If the tag has no release, it creates one, named after the tag. + - A tag whose name contains `/` fails here, after the site is live: the tag is part of the archive's file name, and the slash makes it a path that does not exist. 5. **Summary.** A "Deployment Summary" log group gives the page URL, the `CNAME` domain if any, and the release's URL if assets were uploaded. The site is live before the release assets are made, so a failure in making or uploading them leaves the new site published. @@ -169,12 +170,14 @@ With `create-release-assets: 'true'`, a run triggered by the tag `` attache |---|---|---| | `name` | string | `` | | `tag` | string | `` | -| `commit` | string | The full SHA of the commit the run built. | +| `commit` | string | The full SHA of the commit the run built, `GITHUB_SHA`. | | `timestamp` | string | When the assets were made: ISO 8601 date and time to the second, with the UTC offset, as in `2026-09-28T10:04:05+00:00`. | | `size_mb` | number | The disk space `build-dir` takes, in mebibytes, as `du -sm` reports it: a whole number, rounded up. | -| `file_count` | number | The number of files in `build-dir`, not counting a `CNAME` file that `cname` writes. | +| `file_count` | number | The number of files in `build-dir`. | | `repository` | string | `/`. | +`size_mb` and `file_count` are measured before `cname` writes its `CNAME` file, which the archive then includes. + For example: ```json @@ -195,7 +198,7 @@ To check an archive, run `sha256sum` in the directory that holds both files: sha256sum --check lecture-python-html-checksum.txt ``` -This is the contract. A release of this project that changes a file name, the archive's layout, the checksum's format, or a manifest field's name or meaning says so in its CHANGELOG entry. Read the manifest as JSON, and ignore any field you do not know, so that a field added later does no harm. The checksum's and the manifest's names do not carry the tag, so keep each release's files in a directory of their own. +This is the contract. A release of this project that changes a file name, the archive's layout, the checksum's format, or a manifest field's name or meaning says so in its CHANGELOG entry. Read the manifest as JSON, and ignore any field you do not know, so that a field added later does no harm. The checksum's and the manifest's names do not carry the tag, so keep each release's files in a directory of their own. Tags must not contain `/`, as [Behaviour](#behaviour) explains. ## Custom domain @@ -205,13 +208,16 @@ This is the contract. A release of this project that changes a file name, the ar ## Moving from a `gh-pages` branch -A repository that publishes by pushing to a branch, with peaceiris/actions-gh-pages or a version of this action that took a `target-branch` input, moves over in five steps: +A repository that publishes by pushing to a branch, with peaceiris/actions-gh-pages or a similar action, moves over in six steps: 1. Under **Settings**, **Pages**, change **Source** from **Deploy from a branch** to **GitHub Actions**. 2. Give the job `pages: write` and `id-token: write`, and run it in the `github-pages` environment. It no longer needs `contents: write`, unless it makes release assets. -3. Replace the deploy step with this action, passing only `build-dir`. The deploy needs no token. -4. Set the custom domain under **Settings**, **Pages**: a `CNAME` file in the site no longer applies. -5. Once the new deploy is live, delete the `gh-pages` branch, if you like, to shrink the repository. +3. If the workflow publishes on tags, allow their pattern in that environment, under **Settings**, **Environments**, **github-pages**, **Deployment branches and tags**. By default only the default branch may deploy. +4. Replace the deploy step with this action, passing only `build-dir`. The deploy needs no token. +5. Set the custom domain under **Settings**, **Pages**: a `CNAME` file in the site no longer applies. +6. Once the new deploy is live, delete the `gh-pages` branch, if you like, to shrink the repository. + +The [migration guide](../../MIGRATION-GUIDE.md) moves a whole lecture repository onto these actions, publishing included. ## Troubleshooting @@ -225,6 +231,8 @@ A repository that publishes by pushing to a branch, with peaceiris/actions-gh-pa **`create-release-assets is enabled but this run was not triggered by a tag …; skipping release assets`.** Release assets need a tag trigger, such as `tags: ['publish*']` under `on.push`. +**`Cannot open: No such file or directory`, when making the release assets.** The tag's name contains `/`. Use a tag without one, such as `publish-2026-09-28`. + **`create-release-assets is enabled but 'github-token' is empty`.** Pass `github-token: ${{ secrets.GITHUB_TOKEN }}`. **The release upload fails with `Resource not accessible by integration`, or a 403.** The job lacks `contents: write`. diff --git a/publish-gh-pages/action.yml b/publish-gh-pages/action.yml index 2a04950..0843fb6 100644 --- a/publish-gh-pages/action.yml +++ b/publish-gh-pages/action.yml @@ -39,16 +39,17 @@ inputs: github-token: description: >- With `create-release-assets: 'true'`: a token that can write the repository's releases, - such as `secrets.GITHUB_TOKEN` in a job with `contents: write`. The job fails if it is - empty. The Pages deploy does not use it. + such as `secrets.GITHUB_TOKEN` in a job with `contents: write`. On a run triggered by a tag, + the job fails if it is empty. The Pages deploy does not use it. required: false default: '' outputs: page-url: description: >- - The URL of the published site, as GitHub Pages reports it: `https://.github.io//`, - or the custom domain's. Set once the deploy succeeds. + The URL of the published site, as GitHub Pages reports it, such as + `https://.github.io//`, or the custom domain's. Set once the deploy + succeeds. value: ${{ steps.deployment.outputs.page_url }} runs: diff --git a/scripts/check-access-gate.sh b/scripts/check-access-gate.sh index daff7ca..5ab3ef6 100755 --- a/scripts/check-access-gate.sh +++ b/scripts/check-access-gate.sh @@ -6,7 +6,7 @@ # (#163). # # check-access-gate.sh [ ...] -# check-access-gate.sh quantecon.cloudflareaccess.com \ +# check-access-gate.sh .cloudflareaccess.com \ # https://..workers.dev/ \ # https://..workers.dev/data/latest.json # From 1f03040e6d156bdfe41295d409073341ddbfdee9 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 28 Sep 2026 11:36:23 +0000 Subject: [PATCH 4/4] docs(manual): correct the build chapters against the code; CHANGELOG From the fact-check of build-lectures and build-jupyter-cache against the actions and the sources of Jupyter Book 1.0.4, Sphinx 7.4.7, myst-nb and quantecon-book-theme 0.22.0. build-lectures: - The theme links a page's notebook with the page's directory in the path, but html-copy-notebooks copies the notebooks flat, so a nested page's link leads nowhere (#217). Both download links also start at the site's root, which breaks them on a site served under a path. - Both QuantEcon images trust every directory for git, so the dubious ownership handling applies to other images only, and it writes no git configuration file. - A rebuild over the same _build after a failed -W build reads nothing and passes, so --all is needed to see the failures again; pdflatex and jupyter run with -n, so under -W an unresolved reference fails them; ci.yml's shallow checkout dates every preview page to one commit; for another builder, build-path is the directory above its output. build-jupyter-cache: - The builders share _build/.doctrees, so a failing notebook usually fails only the first builder to run it. - The @v0 siblings behave as in the latest release, not as main does. - The failure issue names execution-report artifacts that a build which failed early never uploaded; cache-saved reports the build result, and a failed save only warns; execute_notebooks: cache is required. CHANGELOG: the corrections, the three issues they found, and the deploy-cloudflare message fix. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01FeKTxWtkzVMyPbqeSgM2Tb --- CHANGELOG.md | 19 ++++++++++++++- build-jupyter-cache/action.yml | 6 +++-- build-lectures/action.yml | 8 +++--- docs/user/actions/build-jupyter-cache.md | 18 +++++++------- docs/user/actions/build-lectures.md | 31 ++++++++++++++++-------- 5 files changed, 56 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0e1d9f4..b9add98 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,7 +57,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - The `preview-cloudflare` setup guide now asks for a token with Cloudflare Pages Edit alone, not the broader "Edit Cloudflare Workers" template. - `deploy-cloudflare`'s `account-subdomain` example no longer names a real account, and its - refusal message points at the chapter's setup checklist instead of the README. (#190) + refusal message points at the chapter's setup checklist instead of the README. + - The caches save execution time only when the book sets `execute_notebooks: cache`, which the + `build-jupyter-cache` and `restore-jupyter-cache` chapters now require. Sphinx re-reads every + page of a fresh checkout, so the old claim that it rewrites only the changed pages is gone. + - `restore-jupyter-cache`'s `save-cache` reaches every pull request when saved from a branch, + so the chapter keeps it to pull-request builds, and a re-run, which matches its own key + exactly, saves nothing. + - Three limits in the code are documented, and filed: the preview actions' change detection + lists lectures changed on the base branch after the pull request branched off (#215); + `preview-cloudflare` builds its URLs from `project-name`, which is wrong when Cloudflare gives + the project another `pages.dev` address (#216); and `html-copy-notebooks` copies the notebooks + flat, so the download link of a page in a subdirectory leads nowhere (#217). + - Both QuantEcon images trust every directory for git, so the `build-lectures` chapter's + "dubious ownership" handling is for other images only. (#190) - **`preview-netlify`, `preview-cloudflare` READMEs**: the Security sections said a notification is logged whenever the deploy is skipped. That holds only for Dependabot and fork PRs. On any event other than `pull_request`, both actions skip change detection, the deploy and the PR @@ -108,6 +121,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `cname:` in six consumers, a permanent `preview-cloudflare` step in the canary). (#187) ### Fixed +- **`deploy-cloudflare`**: when a hostname serving the Worker is found ungated, the error said to + turn Access on or turn off the Worker's `workers.dev` route. With the route off its preview URLs + stay on, which wrangler itself warns may leave them public, so the message now says to turn off + both the route and the preview URLs. (#190) - **`setup-environment`**: bumping `cache-version` builds a fresh Conda environment, as it was documented to. It was in the exact cache key but in neither `restore-keys` fallback, and the first fallback matched the same environment file under any version, so a bump restored the diff --git a/build-jupyter-cache/action.yml b/build-jupyter-cache/action.yml index 25d23d6..f4e850c 100644 --- a/build-jupyter-cache/action.yml +++ b/build-jupyter-cache/action.yml @@ -95,8 +95,10 @@ inputs: outputs: cache-saved: description: >- - `'true'` when every requested build passed, which is when the build and execution caches - are saved; `'false'` otherwise, including a run that stopped before any build. Always set. + `'true'` when every requested build passed, which is when the action saves the build and + execution caches; `'false'` otherwise, including a run that stopped before any build. Always + set. A save that fails only logs a warning, as on a re-run of a run that passed, whose keys + already exist. value: ${{ steps.status.outputs.all-passed }} build-success: description: >- diff --git a/build-lectures/action.yml b/build-lectures/action.yml index ea632d3..f867fdf 100644 --- a/build-lectures/action.yml +++ b/build-lectures/action.yml @@ -13,7 +13,7 @@ inputs: What to build: `html`, the default, builds the website into `/_build/html`; `pdflatex` builds the PDF into `/_build/latex`, and needs LaTeX; `jupyter` builds notebooks into `/_build/jupyter`. Any other value is passed to `jb build` - as `--builder `. + as `--builder `, which writes into a directory under `/_build`. required: false default: 'html' source-dir: @@ -76,9 +76,9 @@ outputs: build-path: description: >- The directory the builder wrote to: `/_build/html`, `_build/latex` or - `_build/jupyter` for those three builders, and `/_build` for any other. With the - default `output-dir` it starts with `./`, as in `./_build/html`. Set whether or not the - build succeeds. + `_build/jupyter` for those three builders. For any other it is `/_build`, the + directory above the one that builder wrote to. With the default `output-dir` it starts + with `./`, as in `./_build/html`. Set whether or not the build succeeds. value: ${{ steps.build.outputs.build-path }} runs: diff --git a/docs/user/actions/build-jupyter-cache.md b/docs/user/actions/build-jupyter-cache.md index 9e094a4..9cb4d9d 100644 --- a/docs/user/actions/build-jupyter-cache.md +++ b/docs/user/actions/build-jupyter-cache.md @@ -4,7 +4,7 @@ Builds the lectures from scratch, in each format listed in `builders`, and saves the result as the cache that pull-request and publish builds restore with [`restore-jupyter-cache`](restore-jupyter-cache.md). It saves two caches: -- **The build cache**: the whole `_build` directory, with the HTML, the PDF, the notebooks and the execution cache. +- **The build cache**: the whole `_build` directory, with the output of each builder in `builders` and the execution cache. - **The execution cache**: `_build/.jupyter_cache` alone, which holds each notebook's executed outputs. Both are saved only when every build passes. A failed run saves nothing, so the builds that restore keep the last good cache, and it files an issue, so that the failure is seen. @@ -23,7 +23,7 @@ Use it in a workflow of its own, on the default branch: weekly, on demand, and w - **Node.** None. - **Permissions.** `issues: write`, to file the failure issue, unless `create-issue-on-failure` is `'false'`. The job also needs `contents: read` for `actions/checkout`, and the templates grant `packages: read` in a `container:` job, for the image pull. Saving caches and uploading artifacts need no permission. - **Secrets.** None: the failure issue is filed with the job's own token. -- **Files in your repository.** The book in `source-dir`. In standard mode, the environment file, and with `pdflatex` the LaTeX package list; see [`setup-environment`](setup-environment.md#requirements). In container mode, the `environment-update` file if you set one. Check out with `fetch-depth: 0`, so that the cached HTML dates each page from its own history. +- **Files in your repository.** The book in `source-dir`, whose `_config.yml` sets `execute_notebooks: cache` under `execute:`. With Jupyter Book's default, `auto`, notebooks are executed without jupyter-cache, and there is no execution cache to save. In standard mode, the environment file, and with `pdflatex` the LaTeX package list; see [`setup-environment`](setup-environment.md#requirements). In container mode, the `environment-update` file if you set one. Check out with `fetch-depth: 0`, so that the cached HTML dates each page from its own history. - **Setup outside GitHub.** None. ## Inputs @@ -52,7 +52,7 @@ Use it in a workflow of its own, on the default branch: weekly, on demand, and w | Output | Description | |---|---| -| `cache-saved` | `'true'` when every requested build passed, which is when the build and execution caches are saved; `'false'` otherwise, including a run that stopped before any build. Always set. | +| `cache-saved` | `'true'` when every requested build passed, which is when the action saves the build and execution caches; `'false'` otherwise, including a run that stopped before any build. Always set. A save that fails only logs a warning, as on a re-run of a run that passed, whose keys already exist. | | `build-success` | The same value as `cache-saved`: `'true'` only when every requested build passed; `'false'` for anything else, including a run that stopped during setup, before any build. Always set. | | `cache-key` | The key the build cache is saved under, `build---`. Set even when a build fails and nothing is saved. | | `jupyter-status` | `success` or `failure` for a `jupyter` build that ran, `skipped` when `builders` does not list it. Empty when the run stopped before the builds. | @@ -115,7 +115,7 @@ Leave out the `container:` block. `setup-environment` then builds the Conda envi 1. **Builders.** `builders` is split on commas and whitespace, and every name must be `jupyter`, `pdflatex` or `html`. 2. **Environment.** `setup-environment` runs with `environment`, `environment-update` and `latex-requirements-file`, and installs LaTeX only when `pdflatex` is among the builders. Its other inputs keep their defaults, so in standard mode the environment is named `quantecon`, uses Python 3.13, and is cached under `cache-version` `v1`. -3. **Builds.** `build-lectures` runs once for each requested builder, always in the order `jupyter`, `pdflatex`, `html`, with its default `extra-args`, `-W --keep-going`, and `output-dir`, `.`. The `html` build copies in the PDF when `pdflatex` ran, and the notebooks when `jupyter` ran. A failed build does not stop the later ones, so each run reports on every builder. +3. **Builds.** `build-lectures` runs once for each requested builder, always in the order `jupyter`, `pdflatex`, `html`, with its default `extra-args`, `-W --keep-going`, and `output-dir`, `.`. The `html` build copies in the PDF when `pdflatex` ran, and the notebooks when `jupyter` ran. A failed build does not stop the later ones, but they build on what it read: all the builders share `_build/.doctrees`, and a later builder rereads only the pages that changed. So a notebook that fails usually fails only the first builder to run it, and the later ones pass. The run still fails, and saves nothing; start from the first builder that failed. 4. **Caches.** If every build passed, both caches are saved: | Cache | Path | Key | @@ -123,16 +123,16 @@ Leave out the `container:` block. `setup-environment` then builds the Conda envi | Build | `_build` | `build---` | | Execution | `_build/.jupyter_cache` | `jupyter-cache--` | - A file that does not exist hashes to an empty string, so with no `environment-update` the build cache's key is `build---`. Every successful run saves new entries, and `restore-jupyter-cache` restores the newest through a prefix match. GitHub removes a cache that has not been restored for 7 days, and the oldest caches first when the repository's cache storage is full. -5. **Job summary.** "Jupyter Cache Build Summary" gives the outcome, the cache key, the trigger, the commit, the mode `setup-environment` ran in, each builder's result and the size of each directory in `_build`. + A file that does not exist hashes to an empty string, so with no `environment-update` the build cache's key is `build---`. Every successful run saves new entries, and `restore-jupyter-cache` restores the newest through a prefix match. GitHub removes a cache that has not been restored for 7 days, and the least recently used caches first when the repository's cache storage is full. +5. **Job summary.** "Jupyter Cache Build Summary" gives the outcome, the cache key, the trigger, the commit, the mode `setup-environment` ran in, each builder's result, and the size of `_build` and of each builder's directory in it. -The action runs `setup-environment` and `build-lectures` at `@v0`, whichever version of `build-jupyter-cache` the workflow pins. +The action runs `setup-environment` and `build-lectures` at `@v0`, whichever version of `build-jupyter-cache` the workflow pins. So inside it they behave as in the latest release, not as this manual describes `main`: a change listed under `[Unreleased]` in the CHANGELOG reaches them only when a release moves `v0`. ### When a build fails 1. **Nothing is saved.** The last good caches stay, and the builds that restore them carry on as before. 2. **Artifacts.** With `upload-artifact`, the whole `_build` directory as `build-cache-`. With `upload-failure-reports`, each failed build's reports as `execution-reports-`. -3. **The failure issue.** With `create-issue-on-failure`, the action files an issue titled `🔴 Cache Build Failed - `, with the labels in `issue-labels` and the assignees in `issue-assignees`. If an open issue already carries the first of those labels, it adds a comment there instead. Either way the text links to the run, gives each builder's result, names the artifacts the run actually uploaded, and gives a command to reproduce each failed build locally. The action then checks that the issue was filed, and fails if it was not. +3. **The failure issue.** With `create-issue-on-failure`, the action files an issue titled `🔴 Cache Build Failed - `, with the labels in `issue-labels` and the assignees in `issue-assignees`. If an open issue already carries the first of those labels, it adds a comment there instead. Either way the text links to the run, gives each builder's result, and gives a command to reproduce each failed build locally. It names the `_build` artifact only if the run uploaded one. With `upload-failure-reports`, it also names an `execution-reports-` artifact for each failed builder, although a build that failed before writing any report uploads none. The action then checks that the issue was filed, and fails if it was not. 4. **The job fails**, with `One or more builds failed - see summary above`. A run can also stop before any lecture is built: on an invalid `builders`, or when `setup-environment` fails. It then fails with `The cache build aborted during …, before any lecture was built`, and the issue says that no lecture was built, so that the failure is not mistaken for a broken lecture. Every builder then reads `not run`. @@ -149,4 +149,4 @@ A run can also stop before any lecture is built: on an invalid `builders`, or wh **New failures comment on an old issue.** The action comments on the open issue that carries the first label. Close the issue once the build is fixed; the next failure files a new one. -**Pull requests do not restore the new cache.** A cache saved on another branch does not reach them: run the workflow on the default branch. Check also that the runs that restore pass the same `environment` and `environment-update`, since both files' hashes are in the key. +**Pull requests do not restore the new cache.** A cache saved on another branch does not reach them: run the workflow on the default branch. Check also that the runs that restore pass the same `environment`, whose hash is in the key, and that they run where `zstd` is installed if the cache build did, or the other way round: see [`restore-jupyter-cache`](restore-jupyter-cache.md#troubleshooting). diff --git a/docs/user/actions/build-lectures.md b/docs/user/actions/build-lectures.md index e75d1b9..a41f0ea 100644 --- a/docs/user/actions/build-lectures.md +++ b/docs/user/actions/build-lectures.md @@ -32,7 +32,7 @@ Use it in any job that builds lectures, after [`setup-environment`](setup-enviro | Input | Required | Default | Description | |---|---|---|---| -| `builder` | no | `html` | What to build: `html`, the default, builds the website into `/_build/html`; `pdflatex` builds the PDF into `/_build/latex`, and needs LaTeX; `jupyter` builds notebooks into `/_build/jupyter`. Any other value is passed to `jb build` as `--builder `. | +| `builder` | no | `html` | What to build: `html`, the default, builds the website into `/_build/html`; `pdflatex` builds the PDF into `/_build/latex`, and needs LaTeX; `jupyter` builds notebooks into `/_build/jupyter`. Any other value is passed to `jb build` as `--builder `, which writes into a directory under `/_build`. | | `source-dir` | no | `lectures` | The book to build: the directory that holds its `_config.yml` and `_toc.yml`, relative to the workspace. | | `output-dir` | no | `.` | The directory the build is written under, passed to `jb build` as `--path-output`, so the output lands in `/_build`. The cache actions read and write `_build` at the workspace root, so keep the default, `.`, in a job that uses them. | | `extra-args` | no | `-W --keep-going` | Further arguments for `jb build`. They are split on whitespace, so a quoted value that contains a space cannot be passed. Setting this replaces the default, so keep `-W` in it: without `-W`, a notebook that raises an exception is only a warning, and the build succeeds. | @@ -51,7 +51,7 @@ Paths are relative to the workspace, which is the repository root after `actions | Output | Description | |---|---| -| `build-path` | The directory the builder wrote to: `/_build/html`, `_build/latex` or `_build/jupyter` for those three builders, and `/_build` for any other. With the default `output-dir` it starts with `./`, as in `./_build/html`. Set whether or not the build succeeds. | +| `build-path` | The directory the builder wrote to: `/_build/html`, `_build/latex` or `_build/jupyter` for those three builders. For any other it is `/_build`, the directory above the one that builder wrote to. With the default `output-dir` it starts with `./`, as in `./_build/html`. Set whether or not the build succeeds. | @@ -108,7 +108,7 @@ Build the notebooks and the PDF first, in the same job, then the website with bo html-copy-notebooks: 'true' ``` -The website then holds both, and the theme links to them: +The website then holds both, and quantecon-book-theme links every page to them: ```text _build/html/ @@ -120,6 +120,11 @@ _build/html/ └── … ``` +Two limits apply to the links: + +- **Lectures in subdirectories.** The theme links a page to `/_notebooks/.ipynb`, with the page's directory in ``, but the notebooks are copied into `_notebooks/` side by side. So the notebook link of a page in a subdirectory leads nowhere, and two notebooks with the same name overwrite each other ([#217](https://github.com/QuantEcon/actions/issues/217)). A book whose lectures all sit at the top of `source-dir` is not affected. +- **Sites served under a path.** Both links start at the site's root, `/_pdf/…` and `/_notebooks/…`, so they lead nowhere on a site that is not at the root of its domain, such as a GitHub Pages project site with no custom domain. The theme's `download_nb_path` option puts a prefix in front of the notebook links; the PDF link has no such option. + After `restore-jupyter-cache`, the copies can also come from the build cache, if its cache build ran those builders: `_build/latex` and `_build/jupyter` are then already in place, as the last cache build left them. ### Other arguments for `jb build` @@ -132,18 +137,18 @@ Keep `-W --keep-going` in any value you set, because it replaces the default: extra-args: '-W --keep-going --all' # rebuild every page, not only the changed ones ``` -`-v` makes the log more verbose, `-q` quieter, and `-n` turns on Sphinx's nitpicky mode, which warns about every reference it cannot resolve. The `pdflatex` and `jupyter` builders already pass `-n`. +`-v` makes the log more verbose, `-q` quieter, and `-n` turns on Sphinx's nitpicky mode, which warns about every reference it cannot resolve. The `pdflatex` and `jupyter` builders already pass `-n`, so with `-W` every reference they cannot resolve fails the build, even where the `html` build passes. ### In the templates -The [workflow templates](https://github.com/QuantEcon/actions/tree/main/templates) build the website with this action, with `upload-failure-reports: 'true'`: [`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml) for a pull request's preview, and [`publish.yml`](https://github.com/QuantEcon/actions/blob/main/templates/publish.yml) before it publishes to GitHub Pages. `publish.yml` carries the `jupyter` and `pdflatex` steps, and the two copy inputs, as comments. [`cache.yml`](https://github.com/QuantEcon/actions/blob/main/templates/cache.yml) builds through `build-jupyter-cache` instead. +The [workflow templates](https://github.com/QuantEcon/actions/tree/main/templates) build the website with this action, with `upload-failure-reports: 'true'`: [`ci.yml`](https://github.com/QuantEcon/actions/blob/main/templates/ci.yml) for a pull request's preview, and [`publish.yml`](https://github.com/QuantEcon/actions/blob/main/templates/publish.yml) before it publishes to GitHub Pages. `publish.yml` carries the `jupyter` and `pdflatex` steps, and the two copy inputs, as comments. [`cache.yml`](https://github.com/QuantEcon/actions/blob/main/templates/cache.yml) builds through `build-jupyter-cache` instead. `publish.yml` and `cache.yml` check out the full history, with `fetch-depth: 0`; `ci.yml` does not, so a preview dates every page to the pull request's commit, and its log carries the shallow-clone warning. ## Behaviour -1. **Downloads.** For an `html` build with `html-copy-pdf: 'true'`, every `.pdf` anywhere under `_build/latex` is copied into `_build/html/_pdf/`. With `html-copy-notebooks: 'true'`, every `.ipynb` under `_build/jupyter` is copied into `_build/html/_notebooks/`. Both copies are flat: files from subdirectories land side by side. A missing source directory is a warning, not a failure. +1. **Downloads.** For an `html` build with `html-copy-pdf: 'true'`, every `.pdf` anywhere under `_build/latex` is copied into `_build/html/_pdf/`. With `html-copy-notebooks: 'true'`, every `.ipynb` under `_build/jupyter` is copied into `_build/html/_notebooks/`. Both copies are flat: files from subdirectories land side by side, which the notebook links do not expect (see [Downloads on the site](#downloads-on-the-site)). A missing source directory is a warning, not a failure. 2. **Git history.** For every builder but `pdflatex` and `jupyter`, the action checks that git can read the repository, because [quantecon-book-theme](https://github.com/QuantEcon/quantecon-book-theme) dates each page and builds its changelog from `git log`, and drops both without a word when git fails. - - In a container job git refuses the work tree, reporting "dubious ownership": the runner creates the workspace as its own user, and the container's steps run as root. The action then trusts the work tree for the rest of the job, by adding `GIT_CONFIG_COUNT`, `GIT_CONFIG_KEY_` and `GIT_CONFIG_VALUE_` to the job's environment, so later steps can run git too. Nothing is written to disk, and nothing outlasts the job. - - A warning says when the build will lack the dates: git is missing, `source-dir` is not in a git work tree, or the clone is shallow. A shallow clone dates every page to the checked-out commit. + - In a container whose image does not already trust every directory, git refuses the work tree, reporting "dubious ownership": the runner creates the workspace as its own user, and the container's steps run as root. Both QuantEcon images trust every directory, so there git reads the tree and nothing is added. Elsewhere the action trusts the work tree for the rest of the job, by adding `GIT_CONFIG_COUNT`, `GIT_CONFIG_KEY_` and `GIT_CONFIG_VALUE_` to the job's environment, so later steps can run git too. No git configuration file is written, and the entries do not outlast the job. + - A warning says when the build will lack the dates: git is missing, `source-dir` is not in a git work tree, git cannot read the work tree, or the clone is shallow. A shallow clone dates every page to the checked-out commit. 3. **The build.** The action runs, in a login shell, so that on a standard runner the Conda environment from `setup-environment` is active: ```text @@ -155,7 +160,7 @@ The [workflow templates](https://github.com/QuantEcon/actions/tree/main/template | `html` | none | `/_build/html` | | `pdflatex` | `--builder pdflatex -n` | `/_build/latex` | | `jupyter` | `--builder=custom --custom-builder=jupyter -n` | `/_build/jupyter` | - | any other | `--builder ` | `/_build` | + | any other | `--builder ` | a directory under `/_build`, which is what `build-path` gives | The log shows the full command in its "Build Command" group. 4. **On failure**, the action prints a summary of the builder, source and output. For the `html`, `pdflatex` and `jupyter` builders it then prints each failed notebook's traceback, from `reports/*.err.log` in the build's output directory: the last 200 lines of each, in a log group named after the report. With `upload-failure-reports: 'true'` it uploads the reports and the execution cache as an artifact. @@ -165,6 +170,8 @@ The [workflow templates](https://github.com/QuantEcon/actions/tree/main/template `jb build` exiting with an error. With `-W`, which the default `extra-args` passes, that includes every warning, so a notebook cell that raises an exception fails the build. +A second build over the same `_build`, after one that failed with `-W --keep-going`, reads nothing, warns about nothing and passes: Sphinx's saved state counts every page as up to date. To see the failures again, rebuild with `--all` in `extra-args`. + > [!WARNING] > Keep `-W` in any `extra-args` you set. A cell that raises is not an error to Jupyter Book: myst-nb logs it as a warning and carries on, and only `-W` turns that into a failed build. Without it the build exits 0 over a broken lecture, and the job goes on to deploy it. `--keep-going` makes the build report every such warning before it fails, instead of stopping at the first; without `-W` it does nothing. > @@ -184,4 +191,8 @@ The [workflow templates](https://github.com/QuantEcon/actions/tree/main/template **`git cannot find the work tree for …`.** `source-dir` is not inside a git checkout: check that `actions/checkout` runs first, and where it checks out to. -**The failure-report upload fails because `an artifact with this name already exists`.** Another job in the run uploaded reports for the same builder. Give each job its own `failure-artifact-name`. +**The failure-report upload fails because `an artifact with this name already exists`.** Another job in the run, or an earlier step in the same job, uploaded reports for the same builder. Give each build its own `failure-artifact-name`. + +**A `pdflatex` or `jupyter` build fails on a reference the `html` build accepts.** Those two builders run with `-n`, so every reference they cannot resolve is a warning, and `-W` makes it an error. Fix the reference. + +**A rebuild passes after a failed build, with no warnings.** It read nothing: see [What fails the job](#what-fails-the-job). Rebuild with `--all`.