diff --git a/README.md b/README.md index 33f638f..1a3147b 100644 --- a/README.md +++ b/README.md @@ -263,15 +263,15 @@ Flags: - `--host` - Address to listen on (default: `127.0.0.1`). Any other address makes the site and its KV data reachable on the network. - `--functions` - Functions manifest (default: `quant/edge-functions.json`). - `--rules` - Rules manifest (default: `quant/edge-rules.json`). -- `--meta` - Template metadata with the KV stores (default: `quant/meta.json`). +- `--meta` - Template metadata with the KV stores (default: `meta` in `quant/serve.json`, then `quant/meta.json`). - `--fixtures` - Directory of KV fixtures (default: `quant/kv-fixtures`). -- `--org` - Organisation (default: the `quant.json` clientid, then `local`). -- `--project` - Project (default: the `quant.json` project, then `dev`). +- `--org` - Organisation (default: `org` in `quant/serve.json`, the `quant.json` clientid, then `local`). +- `--project` - Project (default: `project` in `quant/serve.json`, the `quant.json` project, then `dev`). `quant serve` has no `--project` option of its own: it uses the global `--project` option of the CLI, so `quant serve --help` lists it with the global options. - `--domain` - Host for the domain match of rules (default: the Host of the request). - `--user-name` - Value for `{user_name}` in KV seed values (default: `git config user.name`, then `Owner`). - `--build` - Command to run once before the server starts. - `--watch` - Reload when files change. With `--build`, run the build again when files under `src/` or `public/` change. -- `--run` - Long-running command, for example a dev server, to start beside the server. Quant serve stops it when it stops. +- `--run` - Long-running command, for example a dev server, to start beside the server. Quant serve stops it when it stops. On macOS and Linux the command runs in its own process group, and quant serve stops the whole group. Some dev servers move to the background and the command exits 0 at once. If the process group is still alive, quant serve prints one notice, keeps serving and stops the group when it stops. If only the `--proxy` URL still answers (the dev server left the process group), quant serve says that it cannot stop that dev server. On Windows there is no process group: quant serve stops only the command, and it cannot stop a dev server that moved to the background. - `--proxy` - Dev server URL. Requests that no rule ends go to this URL. WebSocket upgrades (for example, HMR) also go to this URL, after the auth rules of the path. Function rules do not run for an upgrade. - `--geo` - Geolocation fields for functions, as a JSON object. - `--client-ip` - Client address for functions. @@ -299,6 +299,7 @@ quant kv list notes --local --index by_author --equals author=ana quant kv get notes post:1 --local quant kv set notes post:1 '{"title":"Hello"}' --local quant kv set notes post:1 '{"title":"New"}' --local --if-version 3 +quant kv set notes api:token '"abc123"' --local --secret quant kv delete notes post:1 --local ``` @@ -310,9 +311,12 @@ Flags: - `--index` - `list`: index name. - `--equals` - `list`: `field=value` for the index. Repeat the flag for more fields. - `--if-version` - `set`: write only if the key has this version. -- `--org` - Organisation (default: the `quant.json` clientid, then `local`). -- `--project` - Project (default: the `quant.json` project, then `dev`). -- `--meta` - Template metadata with the KV stores (default: `quant/meta.json`). +- `--secret` - `set`: store the value encrypted, as `set(key, value, { secret: true })` in a function does. A function reads the value as usual. `list` shows `encrypted: true` and the value `[ENCRYPTED]`. `set` prints the entry with the value `[ENCRYPTED]`, so the value does not go into the terminal output or CI logs. `get --local` shows the value, because the local store is yours. The value is a command argument, so it goes into your shell history; the interactive menu asks for a secret value with hidden input. With `get`, `list` or `delete`, `--secret` is ignored and the command prints one notice. A key in an indexed collection cannot hold a secret (`INDEX_FIELD_ENCRYPTED`). The local store uses local keys only: do not put production secrets in it. +- `--org` - Organisation (default: `org` in `quant/serve.json`, the `quant.json` clientid, then `local`). +- `--project` - Project (default: `project` in `quant/serve.json`, the `quant.json` project, then `dev`). +- `--meta` - Template metadata with the KV stores (default: `meta` in `quant/serve.json`, then `quant/meta.json`). + +To set these defaults one time for a project, put them in `quant/serve.json` (see [Local project settings](#local-project-settings-quantservejson)). ## Local ledgers @@ -446,10 +450,20 @@ Each field is optional. - `globalHeaders` - Headers for every response from the site: pages, files, function responses, the 404 page and the basic authentication 401. A header that a function sets replaces the global header of the same name. `quant-server` and `age` cannot be set. - `basicAuth` - With a username and a password, every path needs these credentials. A request without them gets 401 with `WWW-Authenticate: Basic`. The check applies before the rules, so functions and redirects need the credentials too. WebSocket upgrades also need them. The `/_quant_serve/` control routes do not need them: they are local only and need the `x-quant-serve: 1` header (see [Safety](#safety)). - `trailingSlash` - `"redirect"`: a path that ends with `/` (except `/`) gets a 301 to the path without the trailing slashes, with the query string. A path that a rule answers is not redirected. `"none"` (the default): no redirect. +- `org`, `project` - The organisation and the project for the local KV store ids, for `quant serve` and `quant kv --local`. Use them when the project has no `quant.json`. Each value is 1 to 100 letters, digits, `.`, `_` or `-`. +- `meta` - The template metadata file with the KV stores (default: `quant/meta.json`). The path is relative to the project directory and must stay in it. + +For `org`, `project` and `meta`, the order is: the flag, then `quant/serve.json`, then `quant.json` (org and project only), then the default (`local`, `dev`, `quant/meta.json`). `quant serve` reads these three fields only at start-up, also with `--watch`. If `quant/serve.json` and `quant.json` set a different org (or project) and no flag chooses, `quant serve` and `quant kv --local` print one warning line that names the value they use. + +The local store ids contain the org and the project (for example `local-dev-orbit`). If you add or change `org` or `project` in `quant/serve.json`, the store ids change (for example to `acme-dev-orbit`), and the local data that you wrote before seems to be missing. The data is still in `.quant/kv/`. Set the old values again to see it. + +```json +{ "org": "acme", "project": "marketing-site" } +``` The order is the order of the edge: global headers, basic authentication, rules, the trailing-slash redirect, then static content. -`quant/serve.json` is local only. No deploy command reads it, and `quant deploy` never uploads it. `quant serve` never prints the password. Do not put a production password in the file. A file with the wrong shape stops `quant serve` at start-up, with a message that names the field. With `--watch`, `quant serve` reads the file again when it changes. If the new file has the wrong shape, `quant serve` prints the message and keeps the last valid settings. +`quant/serve.json` is local only. No deploy command reads it, and `quant deploy` never uploads it. `quant serve` never prints the password. Do not put a production password in the file. A file with the wrong shape stops `quant serve` at start-up, with a message that names the field. It also stops `quant kv --local` (exit code 1), because that command reads the `org`, `project` and `meta` defaults from the same file. With `--watch`, `quant serve` reads the file again when it changes. If the new file has the wrong shape, `quant serve` prints the message and keeps the last valid settings. ## Edge libraries in quant serve @@ -597,6 +611,14 @@ exits non-zero and lists each changed field (values only for matchers and weight Re-run with `--force` to overwrite those rules with the manifest. `--force` never takes over a rule this manifest did not create (a rule ID collision). +When the rules API refuses a deploy with 409 or 422 and gives a message, `quant rules` +prints that message and exits non-zero. Examples: + +- 409: the change may already be live. Reload the rules and check them. A retry is safe. +- 422: the requested rule order cannot be kept. A retry does not help. Contact support. + +Without a message, a 409 means another request is changing the rules (retry shortly). + Portal-issued template tokens expire after one year; rotate the token and update `QUANT_API_TOKEN` before expiry. diff --git a/docs/adopting-quant-serve.md b/docs/adopting-quant-serve.md index 2aaa3ee..8a1d517 100644 --- a/docs/adopting-quant-serve.md +++ b/docs/adopting-quant-serve.md @@ -49,6 +49,13 @@ bundles function sources into the committed function bodies. Give the dev server `--host 127.0.0.1` when it listens only on IPv6 by default, or the proxy cannot reach it. + In some environments the dev server moves to the background and the `--run` + command exits 0 at once. quant serve then prints that the command moved to the + background and keeps serving. On macOS and Linux, it stops the dev server when + it stops, but only while the dev server stays in the process group of the + `--run` command. On Windows, quant serve cannot stop a dev server that moved to + the background. See the `--run` flag in the README. + 3. Move the seed data into fixtures. Each store gets one file, `quant/kv-fixtures/.json`, with a JSON array of entries: diff --git a/package-lock.json b/package-lock.json index e1a671e..5d7b0dc 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@quantcdn/quant-cli", - "version": "7.2.0", + "version": "7.3.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@quantcdn/quant-cli", - "version": "7.2.0", + "version": "7.3.0", "license": "ISC", "dependencies": { "@clack/core": "^1.0.0", diff --git a/package.json b/package.json index 7628e0b..05b5d28 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@quantcdn/quant-cli", - "version": "7.2.0", + "version": "7.3.0", "description": "Deploy tools for QuantCDN", "type": "module", "main": "cli.js", diff --git a/src/commands/kv.js b/src/commands/kv.js index 99fa068..2e42a1f 100644 --- a/src/commands/kv.js +++ b/src/commands/kv.js @@ -6,7 +6,7 @@ import fs from 'node:fs'; import path from 'node:path'; -import { select, text, isCancel } from '@clack/prompts'; +import { select, text, password, confirm, isCancel } from '@clack/prompts'; import { openLocalKv } from '../serve/kv/wire.js'; import { pendingLedgerWarnings } from '../serve/kv/ledgers.js'; import { DEFAULT_FILES } from '../serve/project.js'; @@ -16,6 +16,8 @@ import { resolveOptions } from './serve.js'; const REMOTE_MESSAGE = 'Remote KV commands are not available yet. Use --local.'; const ACTIONS = ['list', 'get', 'set', 'delete']; const MAX_PAGES = 100; +// The mask of an encrypted value, the same as in a list. +const ENCRYPTED = '[ENCRYPTED]'; /** A failure with a message for the user and no stack. */ class UsageError extends Error {} @@ -91,14 +93,24 @@ function need(value, label) { return value; } +// The set options: ifVersion, and secret (the value is stored encrypted, as +// with `secret: true` in a function: a function reads it, a list shows it masked). +function setOptions(args) { + const options = {}; + if (args['if-version'] !== undefined) options.ifVersion = args['if-version']; + if (args.secret === true) options.secret = true; + return Object.keys(options).length > 0 ? options : undefined; +} + const OPERATIONS = { list: (store, args) => listAll(store, args), get: (store, args) => store.getEntry(need(args.key, 'The key')), async set(store, args) { const value = parseJson(need(args.value, 'The JSON value')); - const ifVersion = args['if-version']; - await store.set(need(args.key, 'The key'), value, ifVersion === undefined ? undefined : { ifVersion }); - return store.getEntry(args.key); + await store.set(need(args.key, 'The key'), value, setOptions(args)); + const entry = await store.getEntry(args.key); + // Never print a secret value after a write: it would go to logs and scrollback. + return entry && entry.encrypted ? { ...entry, value: ENCRYPTED } : entry; }, async delete(store, args) { await store.delete(need(args.key, 'The key')); @@ -113,6 +125,7 @@ function describeError(err) { async function execute(args, root, error) { if (!ACTIONS.includes(args.action)) throw new UsageError(`The action must be one of: ${ACTIONS.join(', ')}`); + if (args.secret === true && args.action !== 'set') error(`--secret applies only to set. It is ignored for ${args.action}.`); const options = resolveOptions({ org: args.org, project: args.project, meta: args.meta }, { root, gitUserName: () => null }); const id = idOf(options, args.store); if (id === null) throw new UsageError(`FORBIDDEN: Access to KV store ${args.store} is not allowed`); @@ -120,7 +133,8 @@ async function execute(args, root, error) { const stores = readStores(root, options.meta).map((store) => ({ ...store, indexes: [] })); const local = await openLocalKv({ root, org: options.org, project: options.project, stores }); // The warnings come before the operation, so a failed operation keeps them. - for (const warning of [...local.warnings, ...pendingLedgerWarnings(local.backend, local.backend.storeIds())]) error(warning); + const warnings = [...options.configWarnings, ...local.warnings, ...pendingLedgerWarnings(local.backend, local.backend.storeIds())]; + for (const warning of warnings) error(warning); return OPERATIONS[args.action](local.kvStore.open(id), args); } @@ -144,18 +158,35 @@ export async function runKv(args, { root = process.cwd(), print = console.log, e } } -// Ask for the action, the store and what the action needs. Null on cancel. -async function promptArgs() { - const action = await select({ message: 'Operation', options: ACTIONS.map((value) => ({ value, label: value })) }); - if (isCancel(action)) return null; - const store = await text({ message: 'Store name (from quant/meta.json)' }); - if (isCancel(store)) return null; +const PROMPTS = { select, text, password, confirm, isCancel }; + +// set: the secret question (default no), then the JSON value. A secret +// value is typed with hidden input. Null on cancel. +async function askValue(prompts) { + const secret = await prompts.confirm({ message: 'Store the value as a secret (encrypted)?', initialValue: false }); + if (prompts.isCancel(secret)) return null; + const ask = secret === true ? prompts.password : prompts.text; + const value = await ask({ message: 'JSON value' }); + return prompts.isCancel(value) ? null : { value, secret: secret === true }; +} + +/** + * The menu: ask for the action, the store and what the action needs. + * @param {{ select, text, password, confirm, isCancel }} [prompts] The prompt functions (for tests). + * @returns {Promise} The args, or null on cancel. + */ +export async function askKv(prompts = PROMPTS) { + const action = await prompts.select({ message: 'Operation', options: ACTIONS.map((value) => ({ value, label: value })) }); + if (prompts.isCancel(action)) return null; + const store = await prompts.text({ message: 'Store name (from quant/meta.json)' }); + if (prompts.isCancel(store)) return null; const args = { local: true, action, store }; if (action === 'list') return args; - const key = await text({ message: 'Key' }); - if (isCancel(key)) return null; - const value = action === 'set' ? await text({ message: 'JSON value' }) : undefined; - return isCancel(value) ? null : { ...args, key, value }; + const key = await prompts.text({ message: 'Key' }); + if (prompts.isCancel(key)) return null; + if (action !== 'set') return { ...args, key, value: undefined }; + const answers = await askValue(prompts); + return answers === null ? null : { ...args, key, ...answers }; } export default { @@ -172,10 +203,11 @@ export default { .option('index', { type: 'string', describe: 'list: index name' }) .option('equals', { type: 'string', array: true, describe: 'list: field=value for the index (repeat)' }) .option('if-version', { type: 'string', describe: 'set: write only if the key has this version' }) - .option('org', { type: 'string', describe: 'Organisation (default: quant.json clientid, then local)' }) - .option('project', { type: 'string', describe: 'Project (default: quant.json project, then dev)' }) - .option('meta', { type: 'string', default: DEFAULT_FILES.meta, describe: 'Template metadata with KV stores' }), - promptArgs, + .option('secret', { type: 'boolean', describe: 'set: store the value encrypted (a list shows it masked)' }) + .option('org', { type: 'string', describe: 'Organisation (default: quant/serve.json org, quant.json clientid, then local)' }) + .option('project', { type: 'string', describe: 'Project (default: quant/serve.json project, quant.json project, then dev)' }) + .option('meta', { type: 'string', describe: `Template metadata with KV stores (default: quant/serve.json meta, then ${DEFAULT_FILES.meta})` }), + promptArgs: () => askKv(), async handler(args) { const code = await runKv(args); if (code !== 0) process.exit(code); diff --git a/src/commands/rules.js b/src/commands/rules.js index 5c29756..9bb38cf 100644 --- a/src/commands/rules.js +++ b/src/commands/rules.js @@ -59,12 +59,38 @@ function refusalError(rules) { return new Error('Managed rules were changed outside code; nothing was written. Re-run with --force to overwrite them with this manifest.'); } +const MESSAGE_STATUSES = [409, 422]; +const MAX_MESSAGE = 300; + +// A string as one short line, or null when it is not text. +function oneLine(value) { + if (typeof value !== 'string' || value.trim() === '') return null; + const line = value.replace(/[\u0000-\u001f\u007f-\u009f]+/g, ' ').replace(/\s+/g, ' ').trim(); + return line.length > MAX_MESSAGE ? `${line.slice(0, MAX_MESSAGE - 3)}...` : line; +} + +function adviceText(status, code) { + if (status === 409) return 'Rules are being changed by another request (409). Retry shortly.'; + return `Rules deployment failed (${status || code || 'network error'}). Check portal URL, token scopes and project access.`; +} + +// A 409 or 422 `message` replaces the advice: the API writes it for the user +// (for example, that the change may already be live, or that the rule order +// cannot be kept). A string `error` alone is printed above the advice. +function statusText(status, data, code) { + const advice = adviceText(status, code); + if (!MESSAGE_STATUSES.includes(status)) return advice; + const message = oneLine(data?.message); + if (message) return `Rules deployment failed (${status}): ${message}`; + const errorText = oneLine(data?.error); + return errorText ? `Rules deployment failed (${status}): ${errorText}\n${advice}` : advice; +} + // Never print Axios config/headers or an arbitrary upstream body containing credentials. function deployError(error) { - const status = error.response?.status; - if (status === 409 && Array.isArray(error.response?.data?.rules)) return refusalError(error.response.data.rules); - if (status === 409) return new Error('Rules are being changed by another request (409). Retry shortly.'); - return new Error(`Rules deployment failed (${status || error.code || 'network error'}). Check portal URL, token scopes and project access.`); + const { status, data } = error.response || {}; + if (status === 409 && Array.isArray(data?.rules)) return refusalError(data.rules); + return new Error(statusText(status, data, error.code)); } export default { diff --git a/src/commands/serve.js b/src/commands/serve.js index 1225478..a3f50b5 100644 --- a/src/commands/serve.js +++ b/src/commands/serve.js @@ -14,6 +14,7 @@ import { startServer } from '../serve/server.js'; import { DEFAULT_FILES, loadProject } from '../serve/project.js'; import { openLocalKv } from '../serve/kv/wire.js'; import { LEDGERS_FILE, readLedgerFile } from '../serve/kv/ledgers.js'; +import { readServeConfig } from '../serve/project-config.js'; const DEFAULTS = Object.freeze({ dir: 'dist', port: 8787, host: '127.0.0.1', org: 'local', project: 'dev', userName: 'Owner' }); const LATER_RELEASE = ['pull-rules', 'kv-remote']; @@ -23,6 +24,8 @@ const POSIX = process.platform !== 'win32'; /** The signals that stop quant serve. SIGHUP comes when the terminal closes. */ export const STOP_SIGNALS = Object.freeze(['SIGINT', 'SIGTERM', 'SIGHUP']); export const RUN_GRACE_MS = 2000; +/** How long to wait for the --proxy URL after the --run command exits 0. */ +export const RUN_BACKGROUND_WAIT_MS = 5000; /** The first value that is set (not undefined, null or ''), else the fallback. */ function first(values, fallback) { @@ -96,23 +99,38 @@ function clockOf(value) { return clock; } -function siteOptions(args) { +function siteOptions(args, local) { return { dir: first([args.dir], DEFAULTS.dir), port: portOf(args.port), host: first([args.host], DEFAULTS.host), functions: first([args.functions], DEFAULT_FILES.functions), rules: first([args.rules], DEFAULT_FILES.rules), - meta: first([args.meta], DEFAULT_FILES.meta), + meta: first([args.meta, local.meta], DEFAULT_FILES.meta), fixtures: first([args.fixtures], DEFAULT_FILES.fixtures) }; } -function identityOptions(args, root, gitUserName) { +// A warning for each of org and project that quant/serve.json and quant.json +// set to different values, when no flag chooses: quant/serve.json wins. +function precedenceWarnings(args, local, saved) { + const fields = [ + { name: 'org', flag: first([args.org, args.clientid], undefined), saved: saved.clientid, field: 'clientid' }, + { name: 'project', flag: first([args.project], undefined), saved: saved.project, field: 'project' } + ]; + return fields + .filter((item) => item.flag === undefined && local[item.name] !== undefined && first([item.saved], undefined) !== undefined && + String(item.saved) !== local[item.name]) + .map((item) => `quant/serve.json ${item.name} "${local[item.name]}" is used, not quant.json ${item.field} "${item.saved}": ` + + `quant/serve.json wins over quant.json. Pass --${item.name} to choose.`); +} + +function identityOptions(args, root, gitUserName, local) { const saved = readQuantJson(root); return { - org: first([args.org, args.clientid, saved.clientid], DEFAULTS.org), - project: first([args.project, saved.project], DEFAULTS.project), + configWarnings: precedenceWarnings(args, local, saved), + org: first([args.org, args.clientid, local.org, saved.clientid], DEFAULTS.org), + project: first([args.project, local.project, saved.project], DEFAULTS.project), domain: first([args.domain], undefined), userName: first([args['user-name'], gitUserName(root)], DEFAULTS.userName) }; @@ -148,10 +166,12 @@ function devOptions(args) { */ export function resolveOptions(args, { root = process.cwd(), gitUserName = readGitUserName } = {}) { refuseLaterFlags(args); + // The org, project and meta defaults of quant/serve.json (a flag wins). + const local = readServeConfig(root).defaults; return { root, - ...siteOptions(args), - ...identityOptions(args, root, gitUserName), + ...siteOptions(args, local), + ...identityOptions(args, root, gitUserName, local), ...kvOptions(args), ...devOptions(args) }; @@ -229,7 +249,7 @@ function rebuild(options, print) { // On POSIX the --run command gets its own process group, so that a stop // also ends the processes that the shell starts. Its stdin is not the // terminal, because a background process group cannot read the terminal. -function startRun(command, root, print) { +function startRun(command, root, print, proxy) { const child = spawn(command, { shell: true, cwd: root, @@ -237,13 +257,19 @@ function startRun(command, root, print) { stdio: POSIX ? ['ignore', 'inherit', 'inherit'] : 'inherit' }); child.on('error', (err) => print(`The --run command could not start: ${err.message}`)); + // stopRun aborts the wait for the --proxy URL, so that it never keeps Node alive. + child.runAbort = new AbortController(); child.on('exit', (code, signal) => { - if (!child.stopping) print(`The --run command stopped (${signal || `exit code ${code}`}).`); + if (child.stopping) return; + runExitLine(child, { code, signal, proxy, abort: child.runAbort.signal }).then((line) => { + if (!child.stopping) print(line); + }); }); return child; } function groupAlive(pid) { + if (!POSIX) return false; try { process.kill(-pid, 0); return true; @@ -252,6 +278,64 @@ function groupAlive(pid) { } } +function requestSignal(abort) { + const timeout = AbortSignal.timeout(500); + return abort && AbortSignal.any ? AbortSignal.any([timeout, abort]) : timeout; +} + +async function answers(url, abort) { + try { + await fetch(url, { redirect: 'manual', signal: requestSignal(abort) }); + return true; + } catch (_err) { + return false; + } +} + +// A delay that ends at once when abort fires. +function pause(ms, abort) { + return new Promise((resolve) => { + const timer = setTimeout(done, ms); + function done() { + clearTimeout(timer); + if (abort) abort.removeEventListener('abort', done); + resolve(); + } + if (abort) abort.addEventListener('abort', done, { once: true }); + }); +} + +// True when the dev server answers within waitMs (it can still be starting). +// False at once when abort fires. +async function reachable(url, waitMs, abort) { + const deadline = Date.now() + waitMs; + while (!(abort && abort.aborted)) { + if (await answers(url, abort)) return true; + if (Date.now() >= deadline) return false; + await pause(200, abort); + } + return false; +} + +/** + * The line to print when the --run command exits by itself. Some dev servers + * move to the background and the command exits 0. When its process group is + * still alive, quant serve stops that group on exit. When only the --proxy + * URL still answers (the dev server left the group, or on Windows), quant + * serve cannot stop it. + */ +export async function runExitLine(child, { code, signal, proxy, waitMs = RUN_BACKGROUND_WAIT_MS, abort }) { + const stopped = `The --run command stopped (${signal || `exit code ${code}`}).`; + if (signal || code !== 0) return stopped; + if (groupAlive(child.pid)) { + return 'The --run command moved to the background (exit code 0). quant serve keeps serving and stops its process group when quant serve stops.'; + } + if (proxy && await reachable(proxy, waitMs, abort)) { + return `The --run command exited (exit code 0), but the dev server at ${proxy} still answers. quant serve keeps serving, but it cannot stop that dev server. Stop it yourself.`; + } + return stopped; +} + function signalGroup(child, signal) { try { if (POSIX) process.kill(-child.pid, signal); @@ -273,11 +357,16 @@ async function waitForGroupEnd(pid, graceMs) { return !groupAlive(pid); } +function abortExitWait(child) { + if (child && child.runAbort) child.runAbort.abort(); +} + /** * Stop the --run command: SIGTERM to its process group, then SIGKILL when * the group is still alive after the grace period. */ export async function stopRun(child, { graceMs = RUN_GRACE_MS } = {}) { + abortExitWait(child); if (!child || !child.pid || (!POSIX && !running(child))) return; child.stopping = true; signalGroup(child, 'SIGTERM'); @@ -401,9 +490,10 @@ export async function runServe(options, { stop, print = console.log, log, kvStor if (options.build) runBuild(options.build, options.root); const kv = await kvFor(options, kvStore, log, print); const server = await startOrShowKeys(serverOptions(options, { kvStore: kv.kvStore, backend: kv.backend, log }, print), kv, print); - const lines = startupLines({ url: server.url, host: options.host, clock: options.clock, ...startupSummary(server.summary, kv) }); + const summary = { ...server.summary, warnings: [...(options.configWarnings || []), ...server.summary.warnings] }; + const lines = startupLines({ url: server.url, host: options.host, clock: options.clock, ...startupSummary(summary, kv) }); for (const line of lines) print(line); - const child = options.run ? startRun(options.run, options.root, print) : null; + const child = options.run ? startRun(options.run, options.root, print, options.proxy) : null; watchRun(server, child); const onExit = () => killRun(child); process.once('exit', onExit); @@ -469,9 +559,9 @@ export default { .option('host', { type: 'string', default: DEFAULTS.host, describe: 'Address to listen on' }) .option('functions', { type: 'string', default: DEFAULT_FILES.functions, describe: 'Functions manifest' }) .option('rules', { type: 'string', default: DEFAULT_FILES.rules, describe: 'Rules manifest' }) - .option('meta', { type: 'string', default: DEFAULT_FILES.meta, describe: 'Template metadata with KV stores' }) + .option('meta', { type: 'string', describe: `Template metadata with KV stores (default: quant/serve.json meta, then ${DEFAULT_FILES.meta})` }) .option('fixtures', { type: 'string', default: DEFAULT_FILES.fixtures, describe: 'KV fixtures directory' }) - .option('org', { type: 'string', describe: 'Organisation (default: quant.json clientid, then local)' }) + .option('org', { type: 'string', describe: 'Organisation (default: quant/serve.json org, quant.json clientid, then local)' }) .option('domain', { type: 'string', describe: 'Host for rule domain matching (default: the request Host)' }) .option('user-name', { type: 'string', describe: '{user_name} value (default: git config user.name, then Owner)' }) .option('reset', { type: 'boolean', describe: 'Delete the local KV state and seed again' }) diff --git a/src/serve/project-config.js b/src/serve/project-config.js index ae2dd08..3adf43e 100644 --- a/src/serve/project-config.js +++ b/src/serve/project-config.js @@ -6,6 +6,9 @@ * * The request order is the order of the Quant edge: global headers, basic * authentication, rules, the trailing-slash redirect, then static content. + * + * The file can also give the defaults of org, project and meta for + * `quant serve` and `quant kv --local`. A flag wins over the file. */ import crypto from 'node:crypto'; @@ -14,7 +17,9 @@ import path from 'node:path'; export const SERVE_CONFIG = 'quant/serve.json'; -const FIELDS = ['globalHeaders', 'basicAuth', 'trailingSlash']; +const IDENTITY_FIELDS = ['org', 'project', 'meta']; +const FIELDS = ['globalHeaders', 'basicAuth', 'trailingSlash', ...IDENTITY_FIELDS]; +const SLUG = /^[A-Za-z0-9._-]{1,100}$/; const TRAILING_SLASH = ['redirect', 'none']; const HEADER_NAME = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/; const HEADER_VALUE = /^[\t\x20-\x7e]*$/; @@ -55,9 +60,34 @@ function trailingSlash(value, source) { return value; } +function slug(name, value, source) { + if (typeof value !== 'string' || !SLUG.test(value)) { + throw new Error(`${source}: ${name} must be letters, digits, ".", "_" or "-" (1 to 100 characters).`); + } + return value; +} + +function metaFile(_name, value, source) { + const inside = isText(value) && !path.isAbsolute(value) && !path.normalize(value).startsWith('..'); + if (!inside) throw new Error(`${source}: meta must be a path to a file in the project, relative to the project directory.`); + return value; +} + +const IDENTITY_CHECKS = { org: slug, project: slug, meta: metaFile }; + +// The org, project and meta defaults that the document gives. +function identityDefaults(document, source) { + const defaults = {}; + for (const name of IDENTITY_FIELDS.filter((field) => field in document)) { + defaults[name] = IDENTITY_CHECKS[name](name, document[name], source); + } + return defaults; +} + /** * Check a parsed quant/serve.json document. - * @returns {{ globalHeaders: Array<[string, string]>, basicAuth: string|null, trailingSlash: string, warnings: string[] }} + * @returns {{ globalHeaders: Array<[string, string]>, basicAuth: string|null, trailingSlash: string, + * defaults: { org?: string, project?: string, meta?: string }, warnings: string[] }} */ export function parseServeConfig(document, source = SERVE_CONFIG) { if (!isObject(document)) throw new Error(`${source} must be a JSON object.`); @@ -68,6 +98,7 @@ export function parseServeConfig(document, source = SERVE_CONFIG) { globalHeaders: 'globalHeaders' in document ? globalHeaders(document.globalHeaders, source, warnings) : [], basicAuth: 'basicAuth' in document ? basicAuth(document.basicAuth, source) : null, trailingSlash: 'trailingSlash' in document ? trailingSlash(document.trailingSlash, source) : 'none', + defaults: identityDefaults(document, source), warnings }; } diff --git a/tests/fixtures/serve-run/background.mjs b/tests/fixtures/serve-run/background.mjs new file mode 100644 index 0000000..592530f --- /dev/null +++ b/tests/fixtures/serve-run/background.mjs @@ -0,0 +1,11 @@ +// A --run command that moves the dev server to the background and exits 0, +// as some dev servers do. With "own-group", the dev server also leaves the +// process group. +import { spawn } from 'node:child_process'; +import { fileURLToPath } from 'node:url'; + +const [port, pidFile, mode] = process.argv.slice(2); +const server = fileURLToPath(new URL('./dev-server.mjs', import.meta.url)); +const child = spawn(process.execPath, [server, port, pidFile], { detached: mode === 'own-group', stdio: 'ignore' }); +child.unref(); +process.exit(0); diff --git a/tests/fixtures/serve-run/dev-server.mjs b/tests/fixtures/serve-run/dev-server.mjs new file mode 100644 index 0000000..f99fe9d --- /dev/null +++ b/tests/fixtures/serve-run/dev-server.mjs @@ -0,0 +1,9 @@ +// A small dev server for the --run tests: it listens on the given port and +// writes its pid to the given file. +import fs from 'node:fs'; +import http from 'node:http'; + +const [port, pidFile] = process.argv.slice(2); +http.createServer((_req, res) => res.end('dev')).listen(Number(port), '127.0.0.1', () => { + fs.writeFileSync(pidFile, String(process.pid)); +}); diff --git a/tests/fixtures/serve-run/fail.mjs b/tests/fixtures/serve-run/fail.mjs new file mode 100644 index 0000000..45a4a3f --- /dev/null +++ b/tests/fixtures/serve-run/fail.mjs @@ -0,0 +1,2 @@ +// A --run command that fails at once. +process.exit(1); diff --git a/tests/unit/commands/kv.test.mjs b/tests/unit/commands/kv.test.mjs index cdd5304..9d568e8 100644 --- a/tests/unit/commands/kv.test.mjs +++ b/tests/unit/commands/kv.test.mjs @@ -8,7 +8,7 @@ import yargs from 'yargs'; import { createVirtualClock } from '@quantcdn/edge-kv'; import { openLocalKv } from '../../../src/serve/kv/wire.js'; import { loadProject } from '../../../src/serve/project.js'; -import kv, { runKv } from '../../../src/commands/kv.js'; +import kv, { runKv, askKv } from '../../../src/commands/kv.js'; import { runServe, resolveOptions } from '../../../src/commands/serve.js'; import { getCommand, getCommandOptions } from '../../../src/commandLoader.js'; @@ -51,6 +51,47 @@ describe('kv command', () => { expect(getCommandOptions()).to.deep.include({ value: 'kv', label: 'Read and write the local KV store' }); }); + describe('menu', () => { + const CANCEL = Symbol('cancel'); + function prompts(answers) { + const asked = []; + const next = (kind) => async (question) => { + asked.push({ kind, ...question }); + return answers.shift(); + }; + return { + asked, + prompts: { select: next('select'), text: next('text'), password: next('password'), confirm: next('confirm'), isCancel: (value) => value === CANCEL } + }; + } + + it('asks set --secret as a yes/no question, default no, then the value with hidden input', async () => { + const fake = prompts(['set', 'orbit', 'api:token', true, '"x"']); + expect(await askKv(fake.prompts)).to.deep.equal({ local: true, action: 'set', store: 'orbit', key: 'api:token', value: '"x"', secret: true }); + const question = fake.asked.find((item) => item.kind === 'confirm'); + expect(question.initialValue).to.equal(false); + expect(question.message).to.match(/secret/i); + expect(fake.asked[fake.asked.length - 1].kind).to.equal('password'); + }); + + it('gives secret false and a visible value prompt when the answer is no', async () => { + const fake = prompts(['set', 'orbit', 'a', false, '1']); + expect(await askKv(fake.prompts)).to.include({ secret: false, value: '1' }); + expect(fake.asked[fake.asked.length - 1].kind).to.equal('text'); + }); + + it('does not ask about a secret for get', async () => { + const fake = prompts(['get', 'orbit', 'a']); + expect(await askKv(fake.prompts)).to.deep.equal({ local: true, action: 'get', store: 'orbit', key: 'a', value: undefined }); + expect(fake.asked.some((item) => item.kind === 'confirm')).to.equal(false); + }); + + it('returns null when the secret question is cancelled', async () => { + expect(await askKv(prompts(['set', 'orbit', 'a', CANCEL]).prompts)).to.equal(null); + expect(await askKv(prompts(['set', 'orbit', 'a', true, CANCEL]).prompts)).to.equal(null); + }); + }); + it('refuses a run without --local', async () => { expect(await run(['get', 'orbit', 'a'])).to.equal(1); expect(err).to.deep.equal(['Remote KV commands are not available yet. Use --local.']); @@ -107,6 +148,70 @@ describe('kv command', () => { expect(lastJson().map((item) => item.key)).to.deep.equal(['workspace:w1:project:p1']); }); + describe('set --secret', () => { + it('stores the value encrypted: get reads it, list masks it, the state file does not hold it', async () => { + expect(await run(['--local', 'set', 'orbit', 'api:token', '"s3cr3t-value"', '--secret'])).to.equal(0); + expect(lastJson()).to.include({ key: 'api:token', value: '[ENCRYPTED]', encrypted: true }); + expect(out.join('\n')).not.to.include('s3cr3t-value'); + expect(await run(['--local', 'get', 'orbit', 'api:token'])).to.equal(0); + expect(lastJson()).to.include({ value: 's3cr3t-value', encrypted: true }); + expect(await run(['--local', 'list', 'orbit', '--prefix', 'api:'])).to.equal(0); + expect(lastJson()).to.have.length(1); + expect(lastJson()[0]).to.include({ key: 'api:token', encrypted: true, value: '[ENCRYPTED]' }); + const state = fs.readFileSync(path.join(root, '.quant', 'kv', 'state.json'), 'utf8'); + expect(state).not.to.include('s3cr3t-value'); + }); + + it('stores a plain value without --secret', async () => { + expect(await run(['--local', 'set', 'orbit', 'plain', '1'])).to.equal(0); + expect(lastJson().encrypted).to.equal(false); + }); + + it('works with --if-version', async () => { + await run(['--local', 'set', 'orbit', 'a', '1']); + const { version } = lastJson(); + expect(await run(['--local', 'set', 'orbit', 'a', '2', '--secret', '--if-version', version])).to.equal(0); + expect(lastJson()).to.include({ value: '[ENCRYPTED]', encrypted: true }); + }); + + it('prints one notice that --secret is ignored on get, list and delete', async () => { + await run(['--local', 'set', 'orbit', 'a', '1']); + for (const action of ['get', 'list', 'delete']) { + err.length = 0; + const argv = action === 'list' ? ['--local', action, 'orbit', '--secret'] : ['--local', action, 'orbit', 'a', '--secret']; + expect(await run(argv)).to.equal(0); + expect(err).to.deep.equal([`--secret applies only to set. It is ignored for ${action}.`]); + } + }); + + it('refuses a secret in an indexed collection', async () => { + const { stores } = loadProject({ root }); + await openLocalKv({ root, org: 'local', project: 'dev', stores, indexState: 'active' }); + expect(await run(['--local', 'set', 'orbit', 'workspace:w1:project:p1', '{"workspace":"w1"}', '--secret'])).to.equal(1); + expect(err[err.length - 1]).to.match(/^INDEX_FIELD_ENCRYPTED: /); + }); + + it('a function on a running server reads the secret value', async () => { + let stop; + const stopped = new Promise((resolve) => { stop = resolve; }); + let url; + const options = resolveOptions({ dir: 'dist', port: 0 }, { root, gitUserName: () => null }); + const running = runServe(options, { stop: stopped, print: () => {}, log: () => {}, onListening: (server) => { url = server.url; } }); + await new Promise((resolve) => { + const check = () => (url ? resolve() : setTimeout(check, 20)); + check(); + }); + try { + expect(await run(['--local', 'set', 'orbit', 'workspace:studio:settings', '{"name":"Secret name"}', '--secret'])).to.equal(0); + const res = await fetch(`${url}/api/workspace`); + expect((await res.json()).settings.name).to.equal('Secret name'); + } finally { + stop(); + await running; + } + }); + }); + it('refuses an undeclared store', async () => { expect(await run(['--local', 'get', 'ghost', 'a'])).to.equal(1); expect(err).to.deep.equal(['FORBIDDEN: Access to KV store local-dev-ghost is not allowed']); @@ -118,6 +223,39 @@ describe('kv command', () => { expect(JSON.stringify(state)).to.include('acme-site-orbit'); }); + describe('defaults from quant/serve.json', () => { + const serveJson = (document) => fs.writeFileSync(path.join(root, 'quant', 'serve.json'), JSON.stringify(document)); + const state = () => fs.readFileSync(path.join(root, '.quant', 'kv', 'state.json'), 'utf8'); + + it('uses org, project and meta from quant/serve.json', async () => { + fs.mkdirSync(path.join(root, 'config')); + fs.renameSync(path.join(root, 'quant', 'meta.json'), path.join(root, 'config', 'meta.json')); + serveJson({ org: 'acme', project: 'site', meta: 'config/meta.json' }); + expect(await run(['--local', 'set', 'orbit', 'a', '1'])).to.equal(0); + expect(state()).to.include('acme-site-orbit'); + }); + + it('lets the flags win', async () => { + serveJson({ org: 'acme', project: 'site', meta: 'missing.json' }); + expect(await run(['--local', '--org', 'other', '--project', 'web', '--meta', 'quant/meta.json', 'set', 'orbit', 'a', '1'])).to.equal(0); + expect(state()).to.include('other-web-orbit').and.not.to.include('acme-site-orbit'); + }); + + it('warns once when quant/serve.json and quant.json name a different org', async () => { + fs.writeFileSync(path.join(root, 'quant.json'), JSON.stringify({ clientid: 'saved', project: 'dev' })); + serveJson({ org: 'acme' }); + expect(await run(['--local', 'set', 'orbit', 'a', '1'])).to.equal(0); + expect(err).to.deep.equal(['quant/serve.json org "acme" is used, not quant.json clientid "saved": quant/serve.json wins over quant.json. Pass --org to choose.']); + expect(state()).to.include('acme-dev-orbit'); + }); + + it('refuses an invalid quant/serve.json', async () => { + serveJson({ project: '' }); + expect(await run(['--local', 'get', 'orbit', 'a'])).to.equal(1); + expect(err[0]).to.match(/^quant\/serve\.json: project must be/); + }); + }); + it('leaves a BUILDING index BUILDING', async () => { const { stores } = loadProject({ root }); const first = await openLocalKv({ root, org: 'local', project: 'dev', stores, indexState: 'BUILDING' }); diff --git a/tests/unit/commands/rules.test.mjs b/tests/unit/commands/rules.test.mjs index d8227d0..3e99910 100644 --- a/tests/unit/commands/rules.test.mjs +++ b/tests/unit/commands/rules.test.mjs @@ -97,4 +97,50 @@ describe('Rules deployment', () => { try { await run({ force: true }); expect.fail('must fail'); } catch (error) { expect(error.message).to.include('does not own'); } try { await run(); expect.fail('must fail'); } catch (error) { expect(error.message).to.include('Retry'); } }); + describe('409 and 422 responses', () => { + async function failure(response) { + files(); + sinon.stub(axios, 'post').rejects({ response }); + sinon.stub(console, 'log'); + try { await run(); } catch (error) { return error.message; } + throw new Error('must fail'); + } + it('keeps the refusal for a 409 with a rules list', async () => { + const message = await failure({ status: 409, data: { message: 'Some managed rules changed outside code.', rules: [{ id: 'api', status: 'drift', diff: {} }] } }); + expect(message).to.include('--force').and.not.to.include('Some managed rules'); + }); + it('prints the server message of a 409', async () => { + const text = 'The change may already be live. Reload the rules and check them; a retry is safe.'; + const message = await failure({ status: 409, data: { error: 'conflict', message: text } }); + expect(message).to.equal(`Rules deployment failed (409): ${text}`); + }); + it('keeps the retry text for a 409 without a body', async () => { + expect(await failure({ status: 409 })).to.equal('Rules are being changed by another request (409). Retry shortly.'); + }); + it('prints the server message of a 422 and does not tell the user to retry', async () => { + const text = 'The requested rule order cannot be kept. Contact support.'; + const message = await failure({ status: 422, data: { error: 'rule_order', message: text } }); + expect(message).to.equal(`Rules deployment failed (422): ${text}`); + expect(message).not.to.match(/retry/i); + }); + it('prints the error text of a 422 and keeps the advice when the body has no message', async () => { + expect(await failure({ status: 422, data: { error: 'rule_order' } })).to.equal( + 'Rules deployment failed (422): rule_order\nRules deployment failed (422). Check portal URL, token scopes and project access.'); + }); + it('prints the error text of a 409 and keeps the retry advice when the body has no message', async () => { + expect(await failure({ status: 409, data: { error: 'conflict' } })).to.equal( + 'Rules deployment failed (409): conflict\nRules are being changed by another request (409). Retry shortly.'); + }); + it('uses the message, not the error text, when the body has both', async () => { + expect(await failure({ status: 409, data: { error: 'conflict', message: 'Reload and check.' } })).to.equal('Rules deployment failed (409): Reload and check.'); + }); + it('keeps the generic text for a 422 without a body', async () => { + expect(await failure({ status: 422 })).to.equal('Rules deployment failed (422). Check portal URL, token scopes and project access.'); + }); + it('prints one line of a long or multi-line message', async () => { + const message = await failure({ status: 422, data: { message: `first\nsecond\u0007${'x'.repeat(600)}` } }); + expect(message).not.to.match(/[\n\u0007]/); + expect(message.length).to.be.below(400); + }); + }); }); diff --git a/tests/unit/commands/serve.test.mjs b/tests/unit/commands/serve.test.mjs index 228fe07..08b45f5 100644 --- a/tests/unit/commands/serve.test.mjs +++ b/tests/unit/commands/serve.test.mjs @@ -11,6 +11,7 @@ import serve, { runServe, startupLines, stopRun, + runExitLine, versionInfo, STOP_SIGNALS } from '../../../src/commands/serve.js'; @@ -106,6 +107,66 @@ describe('serve command', () => { expect(JSON.stringify(options)).not.to.include('secret'); }); + describe('defaults from quant/serve.json', () => { + function serveJson(document) { + fs.mkdirSync(path.join(tmp, 'quant'), { recursive: true }); + fs.writeFileSync(path.join(tmp, 'quant', 'serve.json'), JSON.stringify(document)); + } + + it('reads org, project and meta from quant/serve.json', () => { + serveJson({ org: 'acme', project: 'site', meta: 'config/meta.json' }); + expect(resolveOptions(parse([]), { root: tmp, gitUserName: noGit })).to.include({ org: 'acme', project: 'site', meta: 'config/meta.json' }); + }); + + it('lets the flags win', () => { + serveJson({ org: 'acme', project: 'site', meta: 'config/meta.json' }); + const options = resolveOptions(parse(['--org', 'flag-org', '--project', 'flag-project', '--meta', 'flag/meta.json']), { root: tmp, gitUserName: noGit }); + expect(options).to.include({ org: 'flag-org', project: 'flag-project', meta: 'flag/meta.json' }); + }); + + it('wins over quant.json', () => { + fs.writeFileSync(path.join(tmp, 'quant.json'), JSON.stringify({ clientid: 'saved', project: 'saved-project' })); + serveJson({ org: 'acme' }); + expect(resolveOptions(parse([]), { root: tmp, gitUserName: noGit })).to.include({ org: 'acme', project: 'saved-project' }); + }); + + it('warns when quant/serve.json and quant.json set a different org or project', () => { + fs.writeFileSync(path.join(tmp, 'quant.json'), JSON.stringify({ clientid: 'saved', project: 'saved-project' })); + serveJson({ org: 'acme', project: 'saved-project' }); + expect(resolveOptions(parse([]), { root: tmp, gitUserName: noGit }).configWarnings).to.deep.equal([ + 'quant/serve.json org "acme" is used, not quant.json clientid "saved": quant/serve.json wins over quant.json. Pass --org to choose.' + ]); + serveJson({ project: 'site' }); + expect(resolveOptions(parse([]), { root: tmp, gitUserName: noGit }).configWarnings).to.deep.equal([ + 'quant/serve.json project "site" is used, not quant.json project "saved-project": quant/serve.json wins over quant.json. Pass --project to choose.' + ]); + }); + + it('does not warn when a flag chooses, or when the values are the same', () => { + fs.writeFileSync(path.join(tmp, 'quant.json'), JSON.stringify({ clientid: 'saved', project: 'p' })); + serveJson({ org: 'acme', project: 'p' }); + expect(resolveOptions(parse(['--org', 'x']), { root: tmp, gitUserName: noGit }).configWarnings).to.deep.equal([]); + serveJson({ org: 'saved' }); + expect(resolveOptions(parse([]), { root: tmp, gitUserName: noGit }).configWarnings).to.deep.equal([]); + }); + + it('prints the warning in the start-up output', async () => { + const root = path.join(tmp, 'project'); + fs.cpSync(ORBIT, root, { recursive: true, filter: (src) => !src.includes(`${path.sep}.quant`) }); + fs.writeFileSync(path.join(root, 'quant.json'), JSON.stringify({ clientid: 'saved' })); + fs.writeFileSync(path.join(root, 'quant', 'serve.json'), JSON.stringify({ org: 'acme' })); + const opts = resolveOptions({ dir: 'dist', port: 0, memory: true }, { root, gitUserName: noGit }); + await runServe(opts, { stop: Promise.resolve(), print: (line) => output.push(line), log: () => {} }); + const warnings = output.filter((line) => line.includes('wins over quant.json')); + expect(warnings).to.have.length(1); + }); + + it('refuses an invalid value', () => { + serveJson({ org: 'a b' }); + expect(() => resolveOptions(parse([]), { root: tmp, gitUserName: noGit })).to.throw('quant/serve.json: org must be'); + }); + }); + it('reads quant.json softly', () => { fs.writeFileSync(path.join(tmp, 'quant.json'), '{ broken'); const options = resolveOptions(parse([]), { root: tmp, gitUserName: noGit }); @@ -304,6 +365,126 @@ describe('serve command', () => { }); }); + describe('--run command that exits', function () { + this.timeout(15000); + const RUN_FIXTURES = path.join(HERE, '..', '..', 'fixtures', 'serve-run'); + + function freePort() { + return new Promise((resolve) => { + const probe = net.createServer().listen(0, '127.0.0.1', () => { + const { port } = probe.address(); + probe.close(() => resolve(port)); + }); + }); + } + + async function until(check, what, timeoutMs = 8000) { + const started = Date.now(); + while (!check()) { + if (Date.now() - started > timeoutMs) throw new Error(`timed out: ${what}`); + await new Promise((resolve) => setTimeout(resolve, 50)); + } + } + + function alive(pid) { + try { + process.kill(pid, 0); + return true; + } catch (_err) { + return false; + } + } + + // Serve the Orbit fixture with the --run command until the returned stop() is called. + function serveWith(run, proxy) { + const lines = []; + let stop; + const stopped = new Promise((resolve) => { stop = resolve; }); + const opts = resolveOptions({ dir: 'dist', port: 0, memory: true, run, proxy }, { root: ORBIT, gitUserName: noGit }); + const running = runServe(opts, { stop: stopped, print: (line) => lines.push(line), log: () => {} }); + return { lines, stop: async () => { stop(); await running; } }; + } + + function fixture(name, ...args) { + return [process.execPath, path.join(RUN_FIXTURES, name), ...args].map((part) => JSON.stringify(String(part))).join(' '); + } + + it('keeps serving when the command moves the dev server to the background, and stops it on exit', async function () { + if (process.platform === 'win32') this.skip(); + const port = await freePort(); + const pidFile = path.join(tmp, 'pid'); + const served = serveWith(fixture('background.mjs', port, pidFile), `http://127.0.0.1:${port}`); + let pid = null; + try { + await until(() => served.lines.some((line) => line.includes('moved to the background')), 'the background notice'); + await until(() => fs.existsSync(pidFile) && fs.readFileSync(pidFile, 'utf8') !== '', 'the dev server'); + pid = Number(fs.readFileSync(pidFile, 'utf8')); + const response = await fetch(`http://127.0.0.1:${port}/`); + expect(await response.text()).to.equal('dev'); + } finally { + await served.stop(); + } + await until(() => !alive(pid), 'the dev server to stop', 4000); + const notices = served.lines.filter((line) => line.startsWith('The --run command')); + expect(notices).to.have.length(1); + expect(notices[0]).to.include('moved to the background').and.not.to.include('stopped (exit code 0)'); + }); + + it('says so when the background dev server left the process group', async function () { + if (process.platform === 'win32') this.skip(); + const port = await freePort(); + const pidFile = path.join(tmp, 'pid'); + const served = serveWith(fixture('background.mjs', port, pidFile, 'own-group'), `http://127.0.0.1:${port}`); + let pid = null; + try { + await until(() => served.lines.some((line) => line.includes('cannot stop')), 'the notice'); + pid = Number(fs.readFileSync(pidFile, 'utf8')); + const notice = served.lines.find((line) => line.includes('cannot stop')); + expect(notice).to.include(`http://127.0.0.1:${port}`); + } finally { + await served.stop(); + if (pid === null && fs.existsSync(pidFile)) pid = Number(fs.readFileSync(pidFile, 'utf8')); + if (pid) process.kill(pid, 'SIGKILL'); + } + }); + + it('stops the wait for the --proxy URL at once when it is aborted', async () => { + const port = await freePort(); + const abort = new AbortController(); + const started = Date.now(); + setTimeout(() => abort.abort(), 100); + const line = await runExitLine({ pid: 999999 }, { code: 0, signal: null, proxy: `http://127.0.0.1:${port}`, waitMs: 5000, abort: abort.signal }); + expect(Date.now() - started).to.be.below(1000); + expect(line).to.equal('The --run command stopped (exit code 0).'); + }); + + it('aborts the wait for the --proxy URL when quant serve stops', async function () { + if (process.platform === 'win32') this.skip(); + const port = await freePort(); + const served = serveWith(`${JSON.stringify(process.execPath)} -e "process.exit(0)"`, `http://127.0.0.1:${port}`); + await new Promise((resolve) => setTimeout(resolve, 400)); + const started = Date.now(); + await served.stop(); + await new Promise((resolve) => setTimeout(resolve, 100)); + expect(Date.now() - started).to.be.below(1500); + expect(served.lines.filter((line) => line.startsWith('The --run command'))).to.deep.equal([]); + }); + + it('reports a failed command as before', async () => { + const served = serveWith(fixture('fail.mjs'), undefined); + await until(() => served.lines.some((line) => line.startsWith('The --run command')), 'the exit notice'); + await served.stop(); + expect(served.lines.filter((line) => line.startsWith('The --run command'))).to.deep.equal(['The --run command stopped (exit code 1).']); + }); + + it('reports a command that exits 0 and leaves nothing behind as stopped', async () => { + const served = serveWith(`${JSON.stringify(process.execPath)} -e "process.exit(0)"`, undefined); + await until(() => served.lines.some((line) => line.startsWith('The --run command')), 'the exit notice'); + await served.stop(); + expect(served.lines.filter((line) => line.startsWith('The --run command'))).to.deep.equal(['The --run command stopped (exit code 0).']); + }); + }); + describe('runServe', () => { function options(over = {}) { return resolveOptions({ dir: 'dist', port: 0, memory: true, ...over }, { root: ORBIT, gitUserName: noGit }); diff --git a/tests/unit/serve/project-config.test.mjs b/tests/unit/serve/project-config.test.mjs index 4311cb9..ae08559 100644 --- a/tests/unit/serve/project-config.test.mjs +++ b/tests/unit/serve/project-config.test.mjs @@ -18,7 +18,7 @@ function settings(document) { describe('serve project settings (quant/serve.json)', () => { describe('parseServeConfig', () => { it('gives the defaults for an empty document', () => { - expect(settings({})).to.deep.equal({ globalHeaders: [], basicAuth: null, trailingSlash: 'none', warnings: [] }); + expect(settings({})).to.deep.equal({ globalHeaders: [], basicAuth: null, trailingSlash: 'none', defaults: {}, warnings: [] }); }); it('reads every field', () => { @@ -28,6 +28,12 @@ describe('serve project settings (quant/serve.json)', () => { expect(result.trailingSlash).to.equal('redirect'); }); + it('reads the org, project and meta defaults for local KV commands', () => { + const result = settings({ org: 'acme', project: 'site-1', meta: 'config/meta.json' }); + expect(result.defaults).to.deep.equal({ org: 'acme', project: 'site-1', meta: 'config/meta.json' }); + expect(settings({ project: 'p' }).defaults).to.deep.equal({ project: 'p' }); + }); + it('skips the headers that a project cannot set, with a warning', () => { const result = settings({ globalHeaders: { 'quant-server': 'x', Age: '1', 'x-a': '1' } }); expect(result.globalHeaders).to.deep.equal([['x-a', '1']]); @@ -38,7 +44,7 @@ describe('serve project settings (quant/serve.json)', () => { it('refuses a document with the wrong shape, with a clear message', () => { const cases = [ [[], 'quant/serve.json must be a JSON object.'], - [{ headers: {} }, 'quant/serve.json: headers is not supported (supported: globalHeaders, basicAuth, trailingSlash).'], + [{ headers: {} }, 'quant/serve.json: headers is not supported (supported: globalHeaders, basicAuth, trailingSlash, org, project, meta).'], [{ globalHeaders: [] }, 'quant/serve.json: globalHeaders must be an object of header names and values.'], [{ globalHeaders: { 'bad name': 'x' } }, 'quant/serve.json: globalHeaders: "bad name" is not a valid header name.'], [{ globalHeaders: { 'x-a': 1 } }, 'quant/serve.json: globalHeaders.x-a must be printable ASCII text on one line.'], @@ -46,7 +52,13 @@ describe('serve project settings (quant/serve.json)', () => { [{ basicAuth: { username: 'u' } }, /basicAuth must be \{ "username": "\.\.\.", "password": "\.\.\." \}/], [{ basicAuth: { username: 'u', password: '' } }, /basicAuth must be/], [{ basicAuth: { username: 'u:v', password: 'p' } }, 'quant/serve.json: basicAuth.username cannot contain ":".'], - [{ trailingSlash: 'preserve' }, 'quant/serve.json: trailingSlash must be "redirect" or "none".'] + [{ trailingSlash: 'preserve' }, 'quant/serve.json: trailingSlash must be "redirect" or "none".'], + [{ org: '' }, 'quant/serve.json: org must be letters, digits, ".", "_" or "-" (1 to 100 characters).'], + [{ project: 'my site' }, 'quant/serve.json: project must be letters, digits, ".", "_" or "-" (1 to 100 characters).'], + [{ org: 7 }, /org must be letters/], + [{ meta: '' }, 'quant/serve.json: meta must be a path to a file in the project, relative to the project directory.'], + [{ meta: '/etc/meta.json' }, /meta must be a path to a file in the project/], + [{ meta: '../other/meta.json' }, /meta must be a path to a file in the project/] ]; for (const [document, message] of cases) expect(() => settings(document), JSON.stringify(document)).to.throw(message); });