CLI Reference¶
Every hasteward command and flag, generated from the CLI itself — never hand-edited.
The same content is available at the terminal via hasteward <command> --help.
HASteward safely triages, repairs, backs up, and restores database clusters managed by CNPG (PostgreSQL) and MariaDB Operator (Galera).
Backups are stored in restic repositories with block-level dedup, encryption, and compression.
Global Flags¶
| Flag | Type | Default | Description |
|---|---|---|---|
--backups-path |
string |
`` | Restic repository path or URL |
--cluster, -c |
string |
`` | Database cluster CR name |
--debug |
bool |
false |
Enable debug output |
--delete-timeout |
int |
300 |
Delete wait timeout in seconds |
--dry-run |
bool |
false |
Show planned actions without executing (destructive commands) |
--engine, -e |
string |
`` | Database engine: cnpg or galera |
--force, -f |
bool |
false |
Override automatic safety refusal for targeted repair. In ambiguous Galera recovery states (divergent UUIDs, split-brain, no clear primary), --donor is required to declare the authoritative source node. |
--instance, -i |
string |
`` | Target specific instance number |
--kubeconfig |
string |
`` | Path to kubeconfig file |
--namespace, -n |
string |
`` | Kubernetes namespace |
--no-color |
bool |
false |
Disable color output |
--no-escrow |
bool |
false |
Skip pre-repair escrow backup |
--output |
string |
auto |
Output format: auto, human, json, jsonl |
--restic-password |
string |
`` | Restic repository encryption password |
--verbose, -v |
bool |
false |
Verbose output (debug logging) |
Commands¶
hasteward backup¶
Manage cluster backups (create, list, restore, export, prune, policies, repositories)
Usage: hasteward backup
hasteward backup create¶
Create a backup of a database cluster
Usage: hasteward backup create [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--method, -m |
string |
dump |
Backup method: dump or native |
hasteward backup export¶
Extract a backup snapshot to a local .sql.gz file
Exports a database dump from a restic snapshot to a local gzipped SQL file.
For diverged snapshots, use -i to specify the instance ordinal.
Examples:
hasteward backup export -e cnpg -c zitadel-postgres -n zeldas-lullaby --snapshot latest -o dump.sql.gz
hasteward backup export -e cnpg -c zitadel-postgres -n zeldas-lullaby --snapshot abc123 -i 2 -o instance2.sql.gz
Usage: hasteward backup export [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--file, -o |
string |
`` | Output file path (e.g., dump.sql.gz) |
--snapshot |
string |
latest |
Restic snapshot ID or 'latest' (for restore) |
hasteward backup list¶
List restic backup snapshots
Usage: hasteward backup list [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--all-namespaces, -A |
bool |
false |
List across all namespaces |
--type, -t |
string |
all |
Snapshot type filter: backup, diverged, or all |
hasteward backup policies¶
List BackupPolicy resources
Usage: hasteward backup policies [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--all-namespaces, -A |
bool |
false |
List across all namespaces |
--type, -t |
string |
all |
Snapshot type filter: backup, diverged, or all |
hasteward backup prune¶
Apply retention policy and remove old backup snapshots
Prunes old backup snapshots from restic repositories according to the
configured retention policy (keep-last, keep-daily, keep-weekly, keep-monthly).
By default, only type=backup snapshots are pruned. Use -t diverged to prune
only diverged snapshots, or -t all to prune both types.
For diverged snapshots, retention is group-aware: snapshots sharing the same
job tag (from one repair operation) are kept or removed as a unit. So
--keep-last 3 means "keep the 3 most recent repair jobs" regardless of how
many instances each job captured.
Examples:
hasteward backup prune -e cnpg -c zitadel-postgres -n zeldas-lullaby --backups-path /backups
hasteward backup prune -e cnpg -c zitadel-postgres -n zeldas-lullaby --backups-path /backups \
--keep-last 7 --keep-daily 30 --keep-weekly 12 --keep-monthly 24
hasteward backup prune -e cnpg -c zitadel-postgres -n zeldas-lullaby --backups-path /backups \
-t diverged --keep-last 3
Usage: hasteward backup prune [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--keep-daily |
int |
30 |
Keep N daily snapshots (or jobs for diverged) |
--keep-last |
int |
7 |
Keep the last N snapshots (or jobs for diverged) |
--keep-monthly |
int |
24 |
Keep N monthly snapshots (or jobs for diverged) |
--keep-weekly |
int |
12 |
Keep N weekly snapshots (or jobs for diverged) |
--type, -t |
string |
backup |
Snapshot type to prune: backup, diverged, or all |
hasteward backup repositories¶
List BackupRepository resources
Usage: hasteward backup repositories [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--all-namespaces, -A |
bool |
false |
List across all namespaces |
--type, -t |
string |
all |
Snapshot type filter: backup, diverged, or all |
hasteward backup restore¶
Restore a database cluster from a restic snapshot
Usage: hasteward backup restore [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--method, -m |
string |
dump |
Backup method: dump or native |
--snapshot |
string |
latest |
Restic snapshot ID or 'latest' (for restore) |
hasteward bootstrap¶
Bootstrap a Galera cluster by declaring which node the operator bootstraps from
Bootstrap a Galera cluster by declaring which node the operator should bootstrap
from. There are two situations, one goal (force the operator to bootstrap the
authoritative node); bootstrap picks the path from the cluster's state:
ALL NODES DOWN (offline) — DANGEROUS. Identifies the highest-seqno node, establishes
authority offline (scale to 0 -> wsrep_recover), sets safe_to_bootstrap=1, patches the
CR with forceClusterBootstrapInPod, and brings the cluster back from total failure.
OPERATOR RECOVERY DEADLOCK (online) — when triage diagnoses
'galera-operator-recovery-deadlock' (the operator is stuck in a recovery it cannot
resolve because no node yields a seqno, while the DATA is healthy), force-bootstraps the
already-synced authority on the LIVE cluster (no scale-to-0), deletes the stuck recovery
jobs, and lets the operator reform. Non-destructive.
Safety gates:
- Refuses on a healthy cluster UNLESS triage diagnosed a recovery deadlock (else use 'repair')
- Refuses if seqno is ambiguous across nodes (unless --force)
- Refuses if split-brain is detected (unless --force)
- Supports --dry-run to preview the plan without mutation
Use --dry-run --output json for automation to inspect the decision
before approving execution.
Examples:
hasteward bootstrap -e galera -c kimai-mariadb -n hyrule-castle
hasteward bootstrap -e galera -c kimai-mariadb -n hyrule-castle --dry-run
hasteward bootstrap -e galera -c kimai-mariadb -n hyrule-castle --dry-run --output json
hasteward bootstrap -e galera -c kimai-mariadb -n hyrule-castle --force
Usage: hasteward bootstrap
hasteward docs¶
Documentation generation commands
Usage: hasteward docs [flags]
hasteward docs generate¶
Generate reference documentation fragments from code
Generate reference documentation as markdown fragments — one per registered
generator (cli-reference, env-reference, ...) — into --output-dir. The fragments are
assembled into the docs site's reference pages by the pipeline's narrate step.
Run against the freshly-built binary so the output reflects the CLI and environment
bindings actually being shipped, never a stale copy.
Usage: hasteward docs generate [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--help, -h |
bool |
false |
help for generate |
--output-dir |
string |
docs/assets/modules |
output directory for generated fragments |
hasteward prune-wal¶
Clear accumulated WAL from a disk-full CNPG instance (storage-pressure recovery)
Clears accumulated WAL segments from a disk-full PostgreSQL primary.
This is a DESTRUCTIVE storage-pressure RECOVERY operation — not backup retention.
It deletes WAL files from the instance's PVC to free disk space when the primary is
stuck in a WAL-accumulation deadlock (disk full -> can't start -> replicas can't
connect -> replication slots hold WAL -> disk stays full).
Safety: Only operates on CNPG clusters. Requires --instance to target a specific
instance. Runs triage first and verifies each ready replica is caught up before it
deletes anything.
Flow: triage -> safety check -> fence -> mount PVC -> clear pg_wal -> unfence
Examples:
hasteward prune-wal -e cnpg -c nextcloud-postgres -n temple-of-time -i 2
hasteward prune-wal -e cnpg -c grafana-postgres -n gossip-stone -i 1
Usage: hasteward prune-wal
hasteward repair¶
Heal unhealthy database instances
Usage: hasteward repair [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--donor, -d |
string |
`` | Explicit donor instance ordinal (declares authoritative source for repair) |
--heal-timeout |
int |
600 |
Heal wait timeout in seconds |
--unwedge |
bool |
false |
CNPG deadlock breaker: clear a disposable replica's datadir offline (escrow-gated) to un-freeze a disk-full cluster. Use --dry-run first. |
--wipe-datadir |
bool |
false |
Wipe entire datadir on target instance (not just grastate). Forces full SST reseed from donor. Use when local data is irrecoverably corrupted. Requires --force and --instance. |
hasteward reset-authority¶
Reset a Galera cluster's authority when its metadata is inconsistent (cluster-wide restart)
Reset the cluster-wide authority on a Galera cluster. Use this when the nodes'
authority metadata is inconsistent — no node can be trusted as the source of truth — and
it must be forcibly corrected. It intentionally stops ALL database pods, rewrites the
authority metadata on the target instance, and restarts the cluster; every node
experiences downtime.
This is NOT repair (instance-scoped data heal) and NOT bootstrap (declaring authority on a
DOWN cluster). Reach for it when the cluster is up but its authority metadata is corrupt.
Requires --force, --instance, and at least one action flag (--fix-bootstrap).
Usage: hasteward reset-authority [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--fix-bootstrap |
bool |
false |
reset-authority: clear grastate and remove bootstrap config on target instance. Prevents stale local bootstrap behavior during cluster restart. |
--heal-timeout |
int |
600 |
Heal wait timeout in seconds |
hasteward serve¶
Run the hasteward operator (controller + scheduler)
Starts the hasteward operator which watches CNPG Cluster and MariaDB CRs
for clinic.hasteward.prplanit.com/policy annotations and automatically runs scheduled backups
and triage/repair operations based on BackupPolicy configuration.
Endpoints:
:8080/metrics Prometheus metrics
:8081/healthz Liveness probe
:8081/readyz Readiness probe
Usage: hasteward serve
hasteward status¶
Show the current state of managed database clusters
Reports the current, factual state of the managed clusters — the "what is?"
question: engine, namespace, whether HASteward manages it, last triage, last backup.
For interpretation — what is WRONG and what to do next — use 'triage', which runs the
diagnosis catalog and recommends a remedy.
Usage: hasteward status
hasteward triage¶
Read-only diagnostics for a database cluster
Usage: hasteward triage
hasteward update¶
Update this hasteward CLI binary in place from the published image
Pull the HASteward image and atomically replace the running binary with the one
inside it.
hasteward update docker.io/prplanit/hasteward:latest
hasteward update --dev docker.io/prplanit/hasteward:latest-dev
hasteward update --image <ref>
The image binary is static (CGO_ENABLED=0), so it runs on any linux host; it is
verified to run here before the swap, and the swap is atomic — the running process
is unaffected.
Scope: this updates your LOCAL CLI binary only. The in-cluster operator (`serve`) is
updated by changing its image tag (GitOps), NOT by this command.
Usage: hasteward update [flags]
| Flag | Type | Default | Description |
|---|---|---|---|
--dev |
bool |
false |
update from the latest-dev image instead of the latest release |
--image |
string |
`` | image ref to update from (overrides default and --dev) |
hasteward version¶
Print the hasteward version and build info
Usage: hasteward version