feat(static-sites): let project sites declare Bunny edge rules
All checks were successful
Test / test (push) Successful in 6s
All checks were successful
Test / test (push) Successful in 6s
Project sites can pass a TOML file of edge rules (redirects, status codes, blocking, response headers, cache times) to the deploy action. The gateway validates them when a deployment starts and reconciles its own rules on the site's Pull Zones when it finalizes, deleting dropped rules and purging when a rule changed. Actions that could reach beyond the site, such as origin changes, are not available. The purge retry is limited to recorded purge failures, so a deployment whose rules failed to apply cannot be completed by a purge alone.
This commit is contained in:
parent
411010ef5a
commit
4490557f13
4 changed files with 263 additions and 9 deletions
79
README.md
79
README.md
|
|
@ -54,6 +54,83 @@ Interrupted uploads are aborted best-effort; abandoned leases expire after 30
|
|||
minutes. The dedicated Storage Zone password authorizes this workflow without
|
||||
exposing an account-wide API key to the project.
|
||||
|
||||
## Edge rules
|
||||
|
||||
Bunny Storage has no `_headers` or `_redirects` file and picks a file's
|
||||
`Content-Type` from its extension. With the gateway, a site declares Bunny edge
|
||||
rules in a TOML file kept in its repository, outside the build directory:
|
||||
|
||||
```yaml
|
||||
with:
|
||||
directory: dist
|
||||
storage-zone: example-org
|
||||
purge-endpoint: https://gandalf.tionis.dev
|
||||
edge-rules: bunny.toml
|
||||
```
|
||||
|
||||
```toml
|
||||
[[edge_rules]]
|
||||
name = "markdown"
|
||||
url = "*.md"
|
||||
response_headers = { "Content-Type" = "text/markdown; charset=utf-8" }
|
||||
|
||||
[[edge_rules]]
|
||||
name = "old-blog"
|
||||
url = ["*/blog/*", "*/news/*"]
|
||||
redirect = { to = "https://example.org{{path}}", status = 301 }
|
||||
|
||||
[[edge_rules]]
|
||||
name = "retired"
|
||||
url = "*/alt/*"
|
||||
status_code = 410
|
||||
|
||||
[[edge_rules]]
|
||||
name = "hashed-assets"
|
||||
url = "*/_astro/*"
|
||||
cache_seconds = 31536000
|
||||
browser_cache_seconds = 31536000
|
||||
```
|
||||
|
||||
Each rule has a unique `name` (lowercase letters, digits, `-`), at least one
|
||||
trigger and at least one action. Rules run in file order; there are at most 25.
|
||||
|
||||
Triggers take one pattern or a list of up to 10, any of which may match:
|
||||
|
||||
| Key | Matches |
|
||||
| --- | --- |
|
||||
| `url` | The full request URL, e.g. `*.md` or `*/docs/*` (`*` also spans `/`). |
|
||||
| `method` | `GET`, `HEAD`, `POST`, `PUT`, `PATCH`, `DELETE`, `OPTIONS`. |
|
||||
| `status` | The response status code, e.g. `[404]`. |
|
||||
| `country` | Two-letter country codes such as `"DE"`. |
|
||||
| `request_header` | `{ name = "X-Debug", values = ["1"] }`. |
|
||||
|
||||
With several triggers, `match` decides: `"all"` (default), `"any"` or `"none"`.
|
||||
|
||||
Actions:
|
||||
|
||||
| Key | Effect |
|
||||
| --- | --- |
|
||||
| `redirect` | `{ to = "https://…", status = 301 }`; `status` is 301, 302 (default), 307 or 308. `to` may use Bunny variables such as `{{path}}` (path with query string). |
|
||||
| `status_code` | Replace the response status, 200-599. |
|
||||
| `block` | `true`: refuse the request. |
|
||||
| `response_headers` | Header names to values. Framing and hop-by-hop headers such as `Content-Length` are rejected. |
|
||||
| `cache_seconds` | Edge cache time in seconds. |
|
||||
| `browser_cache_seconds` | Browser cache time in seconds. |
|
||||
| `ignore_query_string` | `true`: cache regardless of the query string. |
|
||||
| `force_download` | `true`: serve as a download. |
|
||||
|
||||
`redirect`, `status_code` and `block` exclude each other. Bunny stops at the
|
||||
first matching redirect or block; headers from all matching rules apply.
|
||||
Actions that could reach beyond the site, such as changing the origin or
|
||||
switching off security features, are not available.
|
||||
|
||||
The gateway checks the rules when the deployment starts, so a mistake fails
|
||||
before anything is uploaded, and its error names the rule. A dry run only checks
|
||||
the TOML structure. On finalization the gateway applies the rules to the site's
|
||||
Pull Zones, removes rules dropped from the file, and purges the cache when a
|
||||
rule changed. A deployment through the gateway without `edge-rules` removes the
|
||||
rules of earlier deployments; rules set up by other means are never touched.
|
||||
|
||||
## Command line
|
||||
|
||||
The script can also be used directly with Python 3.11 or newer:
|
||||
|
|
@ -70,6 +147,8 @@ asset-before-HTML and manifest-last publication order. Set
|
|||
`BUNNY_TRANSFER_WORKERS` or pass `--workers` to tune the limit.
|
||||
|
||||
Set `BUNNY_STORAGE_ENDPOINT` when the Storage Zone is outside Frankfurt.
|
||||
Pass edge rules with `--edge-rules bunny.toml` (or `STATIC_SITE_EDGE_RULES`);
|
||||
`--dry-run` checks only the file's TOML structure.
|
||||
|
||||
## Development
|
||||
|
||||
|
|
|
|||
|
|
@ -17,6 +17,13 @@ inputs:
|
|||
description: Optional trusted Gandalf deployment gateway base URL.
|
||||
required: false
|
||||
default: ""
|
||||
edge-rules:
|
||||
description: >-
|
||||
Optional TOML file with [[edge_rules]] (redirects, status codes, headers,
|
||||
cache times) for the site's Pull Zones. Needs purge-endpoint. Through the
|
||||
gateway, leaving it empty removes rules from earlier deployments.
|
||||
required: false
|
||||
default: ""
|
||||
dry-run:
|
||||
description: Report changes without uploading or deleting files.
|
||||
required: false
|
||||
|
|
@ -39,6 +46,7 @@ runs:
|
|||
STATIC_SITE_REVISION: ${{ github.sha }}
|
||||
STATIC_SITE_DRY_RUN: ${{ inputs.dry-run }}
|
||||
STATIC_SITE_WORKERS: ${{ inputs.workers }}
|
||||
STATIC_SITE_EDGE_RULES: ${{ inputs.edge-rules }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
args=(bunny "$STATIC_SITE_DIRECTORY" --workers "$STATIC_SITE_WORKERS")
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ import subprocess
|
|||
import sys
|
||||
import tarfile
|
||||
import time
|
||||
import tomllib
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
|
@ -27,7 +28,6 @@ MANIFEST_VERSION = 1
|
|||
BUNNY_RETRY_DELAYS = (1, 2, 4, 8, 16)
|
||||
BUNNY_TRANSIENT_STATUS_CODES = {401, 408, 429, 500, 502, 503, 504}
|
||||
DEFAULT_BUNNY_TRANSFER_WORKERS = 16
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
|
||||
|
|
@ -35,6 +35,34 @@ class DeployError(RuntimeError):
|
|||
"""A static-site deployment could not be completed safely."""
|
||||
|
||||
|
||||
def load_edge_rules(path: Path) -> list[dict[str, Any]]:
|
||||
"""Read the [[edge_rules]] tables of a TOML file.
|
||||
|
||||
Only the structure is checked here; the Gandalf gateway validates the rules
|
||||
when a deployment starts, before anything is uploaded.
|
||||
"""
|
||||
try:
|
||||
data = tomllib.loads(path.read_text(encoding="utf-8"))
|
||||
except OSError as exc:
|
||||
raise DeployError(f"cannot read edge rules {path}: {exc}") from exc
|
||||
except tomllib.TOMLDecodeError as exc:
|
||||
raise DeployError(f"edge rules {path} are not valid TOML: {exc}") from exc
|
||||
unknown = set(data) - {"edge_rules"}
|
||||
if unknown:
|
||||
raise DeployError(
|
||||
f"edge rules {path} has unknown keys: {', '.join(sorted(unknown))}"
|
||||
)
|
||||
rules = data.get("edge_rules", [])
|
||||
if not isinstance(rules, list) or not all(isinstance(rule, dict) for rule in rules):
|
||||
raise DeployError(f"edge rules {path}: edge_rules must be [[edge_rules]] tables")
|
||||
try:
|
||||
# Dates and times have no JSON form and no meaning in a rule.
|
||||
json.dumps(rules)
|
||||
except TypeError as exc:
|
||||
raise DeployError(f"edge rules {path} contain an unsupported value: {exc}") from exc
|
||||
return rules
|
||||
|
||||
|
||||
def positive_int(value: str) -> int:
|
||||
try:
|
||||
parsed = int(value)
|
||||
|
|
@ -363,14 +391,19 @@ def start_bunny_deployment(
|
|||
password: str,
|
||||
idempotency_key: str,
|
||||
revision: str | None,
|
||||
edge_rules: list[dict[str, Any]] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
payload: dict[str, Any] = {"idempotency_key": idempotency_key, "revision": revision}
|
||||
if edge_rules is not None:
|
||||
# Sent here as well so the gateway rejects broken rules before the upload.
|
||||
payload["edge_rules"] = edge_rules
|
||||
result = deployment_gateway_request(
|
||||
endpoint,
|
||||
zone,
|
||||
password,
|
||||
"POST",
|
||||
"deployments",
|
||||
{"idempotency_key": idempotency_key, "revision": revision},
|
||||
payload,
|
||||
)
|
||||
deployment = result.get("deployment")
|
||||
if not isinstance(deployment, dict) or not deployment.get("deployment_id"):
|
||||
|
|
@ -386,23 +419,27 @@ def finalize_bunny_deployment(
|
|||
idempotency_key: str,
|
||||
revision: str | None,
|
||||
manifest_sha256: str,
|
||||
edge_rules: list[dict[str, Any]] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
payload: dict[str, Any] = {
|
||||
"idempotency_key": idempotency_key,
|
||||
"revision": revision,
|
||||
"manifest_sha256": manifest_sha256,
|
||||
}
|
||||
if edge_rules is not None:
|
||||
payload["edge_rules"] = edge_rules
|
||||
result = deployment_gateway_request(
|
||||
endpoint,
|
||||
zone,
|
||||
password,
|
||||
"POST",
|
||||
f"deployments/{urllib.parse.quote(deployment_id, safe='')}/finalize",
|
||||
{
|
||||
"idempotency_key": idempotency_key,
|
||||
"revision": revision,
|
||||
"manifest_sha256": manifest_sha256,
|
||||
},
|
||||
payload,
|
||||
)
|
||||
deployment = result.get("deployment")
|
||||
if not isinstance(deployment, dict):
|
||||
raise DeployError("Bunny deployment gateway returned an invalid finalization")
|
||||
return deployment
|
||||
return {**deployment, "edge_rules_changed": bool(result.get("edge_rules_changed"))}
|
||||
|
||||
|
||||
def abort_bunny_deployment(
|
||||
|
|
@ -427,12 +464,20 @@ def deploy_bunny(args: argparse.Namespace) -> dict[str, Any]:
|
|||
raise DeployError(
|
||||
"BUNNY_PURGE_ENDPOINT and BUNNY_PULL_ZONE_ID are mutually exclusive"
|
||||
)
|
||||
edge_rules_path = getattr(args, "edge_rules", None)
|
||||
if edge_rules_path and not args.purge_endpoint:
|
||||
raise DeployError("edge rules can only be applied through BUNNY_PURGE_ENDPOINT")
|
||||
# Through the gateway, no file means no rules: it removes rules from earlier deploys.
|
||||
edge_rules = load_edge_rules(edge_rules_path) if edge_rules_path else []
|
||||
files = collect_files(args.directory)
|
||||
manifest = manifest_for(files)
|
||||
manifest_sha256 = hashlib.sha256(manifest_bytes(manifest)).hexdigest()
|
||||
revision = args.revision.strip() or None
|
||||
rules_sha256 = hashlib.sha256(
|
||||
json.dumps(edge_rules, sort_keys=True).encode()
|
||||
).hexdigest()
|
||||
idempotency_key = args.idempotency_key.strip() or hashlib.sha256(
|
||||
f"{args.zone}\0{revision or ''}\0{manifest_sha256}".encode()
|
||||
f"{args.zone}\0{revision or ''}\0{manifest_sha256}\0{rules_sha256}".encode()
|
||||
).hexdigest()
|
||||
storage = BunnyStorage(args.zone, args.password, args.endpoint)
|
||||
remote = parse_manifest(storage.get_manifest())
|
||||
|
|
@ -445,6 +490,7 @@ def deploy_bunny(args: argparse.Namespace) -> dict[str, Any]:
|
|||
"delete": delete,
|
||||
"purged": False,
|
||||
"deployment": None,
|
||||
"edge_rules": [rule.get("name") for rule in edge_rules],
|
||||
}
|
||||
if args.dry_run:
|
||||
return result
|
||||
|
|
@ -456,6 +502,7 @@ def deploy_bunny(args: argparse.Namespace) -> dict[str, Any]:
|
|||
args.password,
|
||||
idempotency_key,
|
||||
revision,
|
||||
edge_rules,
|
||||
)
|
||||
try:
|
||||
if result["changed"]:
|
||||
|
|
@ -468,6 +515,7 @@ def deploy_bunny(args: argparse.Namespace) -> dict[str, Any]:
|
|||
idempotency_key,
|
||||
revision,
|
||||
manifest_sha256,
|
||||
edge_rules,
|
||||
)
|
||||
except Exception:
|
||||
try:
|
||||
|
|
@ -559,6 +607,12 @@ def parser() -> argparse.ArgumentParser:
|
|||
or os.environ.get("CI_COMMIT_SHA", "")
|
||||
),
|
||||
)
|
||||
bunny.add_argument(
|
||||
"--edge-rules",
|
||||
type=Path,
|
||||
default=os.environ.get("STATIC_SITE_EDGE_RULES") or None,
|
||||
help="TOML file with [[edge_rules]] for the site's Pull Zones (needs the gateway).",
|
||||
)
|
||||
bunny.add_argument(
|
||||
"--idempotency-key",
|
||||
default=os.environ.get("STATIC_SITE_IDEMPOTENCY_KEY", ""),
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ import sys
|
|||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
from typing import Any
|
||||
from unittest import mock
|
||||
|
||||
MODULE_PATH = Path(__file__).resolve().parents[1] / "static_site.py"
|
||||
|
|
@ -292,5 +293,117 @@ class StaticSiteTests(unittest.TestCase):
|
|||
)
|
||||
|
||||
|
||||
MARKDOWN_RULES = '''
|
||||
[[edge_rules]]
|
||||
name = "markdown"
|
||||
url = "*.md"
|
||||
response_headers = { "Content-Type" = "text/markdown; charset=utf-8" }
|
||||
|
||||
[[edge_rules]]
|
||||
name = "old-blog"
|
||||
url = ["*/blog/*"]
|
||||
redirect = { to = "https://example.org{{path}}", status = 301 }
|
||||
'''
|
||||
|
||||
|
||||
def gateway_args(root: Path, edge_rules: Path | None) -> Any:
|
||||
return static_site.argparse.Namespace(
|
||||
directory=root,
|
||||
zone="example-org",
|
||||
password="storage-password",
|
||||
endpoint="storage.bunnycdn.com",
|
||||
purge_endpoint="https://purge.example",
|
||||
pull_zone_id="",
|
||||
api_key="",
|
||||
dry_run=False,
|
||||
workers=4,
|
||||
revision="abc1234",
|
||||
idempotency_key="",
|
||||
edge_rules=edge_rules,
|
||||
)
|
||||
|
||||
|
||||
class EdgeRuleTests(unittest.TestCase):
|
||||
def write_rules(self, directory: str, text: str) -> Path:
|
||||
path = Path(directory) / "bunny.toml"
|
||||
path.write_text(text)
|
||||
return path
|
||||
|
||||
def test_loads_declared_rules(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
rules = static_site.load_edge_rules(
|
||||
self.write_rules(directory, MARKDOWN_RULES)
|
||||
)
|
||||
self.assertEqual(
|
||||
rules,
|
||||
[
|
||||
{
|
||||
"name": "markdown",
|
||||
"url": "*.md",
|
||||
"response_headers": {
|
||||
"Content-Type": "text/markdown; charset=utf-8"
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "old-blog",
|
||||
"url": ["*/blog/*"],
|
||||
"redirect": {"to": "https://example.org{{path}}", "status": 301},
|
||||
},
|
||||
],
|
||||
)
|
||||
|
||||
def test_rejects_malformed_files(self) -> None:
|
||||
# Rule contents are validated by the gateway; see handler_test.ts.
|
||||
cases = {
|
||||
"unknown keys": 'headers = "x"\n' + MARKDOWN_RULES,
|
||||
"must be \\[\\[edge_rules\\]\\] tables": 'edge_rules = ["x"]\n',
|
||||
"unsupported value": MARKDOWN_RULES + "since = 2026-10-02\n",
|
||||
"not valid TOML": "[[edge_rules]\n",
|
||||
}
|
||||
for message, text in cases.items():
|
||||
with self.subTest(message), tempfile.TemporaryDirectory() as directory:
|
||||
with self.assertRaisesRegex(DeployError, message):
|
||||
static_site.load_edge_rules(self.write_rules(directory, text))
|
||||
|
||||
def test_edge_rules_need_the_gateway(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
write_site(root)
|
||||
args = gateway_args(root, self.write_rules(directory, MARKDOWN_RULES))
|
||||
args.purge_endpoint = ""
|
||||
with self.assertRaisesRegex(DeployError, "BUNNY_PURGE_ENDPOINT"):
|
||||
static_site.deploy_bunny(args)
|
||||
|
||||
@mock.patch.object(static_site, "finalize_bunny_deployment")
|
||||
@mock.patch.object(static_site, "start_bunny_deployment")
|
||||
@mock.patch.object(static_site, "BunnyStorage")
|
||||
def test_finalization_sends_declared_rules_or_none(
|
||||
self,
|
||||
storage_class: mock.Mock,
|
||||
start_deployment: mock.Mock,
|
||||
finalize_deployment: mock.Mock,
|
||||
) -> None:
|
||||
storage_class.return_value.get_manifest.return_value = None
|
||||
start_deployment.return_value = {"deployment_id": "deployment-id"}
|
||||
finalize_deployment.return_value = {"purged": True}
|
||||
keys = []
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory) / "dist"
|
||||
root.mkdir()
|
||||
write_site(root)
|
||||
rules = self.write_rules(directory, MARKDOWN_RULES)
|
||||
for path, expected in ((rules, ["markdown", "old-blog"]), (None, [])):
|
||||
result = static_site.deploy_bunny(gateway_args(root, path))
|
||||
for sent in (
|
||||
start_deployment.call_args.args[5],
|
||||
finalize_deployment.call_args.args[7],
|
||||
):
|
||||
self.assertEqual([rule["name"] for rule in sent], expected)
|
||||
self.assertEqual(result["edge_rules"], expected)
|
||||
keys.append(start_deployment.call_args.args[3])
|
||||
# Changing only the rules must not replay the previous deployment.
|
||||
self.assertNotEqual(keys[0], keys[1])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
|
|||
Loading…
Reference in a new issue