Progressive Delivery Rollouts

License: pro tier and above. See Licensing.

A rollout is a timed canary weight ramp on a single pool member: instead of flipping a member's weight straight to its target value, it steps through a schedule (e.g. 10% → 50% → 100%), holding each step for a duration, with automatic rollback if health or RTT regresses partway through.

This is DNS-layer progressive delivery — there's no sidecar, no service mesh, and no extra process. The same gslbd background controller that already holds cluster state advances the schedule and writes weight through the exact path the WebUI weight slider uses.


How it works

  1. Create a rollout on a member with an ordered list of steps (weightPct, holdSeconds) and, optionally, a regression policy.

  2. The controller picks it up within one tick (every 5s) and applies the first step's weight.

  3. Every tick, it checks the regression policy against current signals. If nothing trips, it advances to whichever step should be active given elapsed time — a controller that was briefly down catches up in one sweep rather than replaying every step.

  4. A step with holdSeconds: 0 holds indefinitely — put that on the final step to require a manual promote instead of auto-completing.

  5. The rollout ends in one of two terminal states: promoted (all steps done, or promoted early) or rolled_back (regression tripped, or manually aborted). Either way the member's weight is left in a stable, final value.

Because the controller recomputes purely from persisted state (startedAt + the step schedule) rather than tracking progress in memory, it's safe to run identically on every node — no leader election, no coordinator.

Regression policy signals

All three are optional; a zero-value field disables that signal. An unmeasured signal (no health data yet, no RTT baseline yet) never trips a rollback on its own — only signals with actual data can regress.

Field

Trips rollback when

minHealthScore

The member's own health score drops below this.

minHealthyMembers

The pool's healthy member count drops below this.

maxRttDeltaMs

The member's RTT rises more than this many ms above the baseline captured when the rollout started running.

A regression rollback restores the member's pre-rollout weight (captured at creation time), marks the rollout rolled_back, and fires an alert through whatever channels are configured (webhook, email, PagerDuty, Opsgenie — see Alerting) with the tripped reason in the message.


WebUI

On the topology page, a member with an active rollout shows a progress bar under its weight badge — current step / total steps, the step's weight, and the rollout's state — plus quick-action buttons:

  • ⏸ Pause — running rollouts only; holds the current weight and stops step advancement.

  • ⏭ Promote — running or paused; jumps straight to the final step's weight and finishes immediately.

  • ✕ Abort — any non-terminal state; restores the pre-rollout weight.

Click the member to open its edit panel for the full Progressive Delivery section:

  • No active rollout → a step-schedule builder (add/remove steps, set each step's weight % and hold seconds) plus regression-policy thresholds, and a Start Rollout button.

  • Active rollout → the same status and pause/promote/abort controls as the canvas, with more room for the regression policy detail.


gslbctl

# List all rolloutsgslbctl rollouts list# Start one: ramp 10% → 50% → 100%, holding 5 min at each of the first two# steps and indefinitely at the last (so it waits for a manual promote)gslbctl rollouts start <member-id> --steps "10:300,50:300,100:0" \ --min-health 0.5 --max-rtt-delta 50 --min-healthy 1# Check statusgslbctl rollouts status <id># Pause / promote / abortgslbctl rollouts pause <id>gslbctl rollouts promote <id>gslbctl rollouts abort <id>

--steps is a comma-separated list of weightPct:holdSeconds pairs. All subcommands accept --server URL / $GSLB_SERVER_URL and $GSLB_API_KEY like the rest of gslbctl.


REST API

Method

Path

Notes

GET

/api/v1/rollouts

List all rollouts. Requires rollout:read.

POST

/api/v1/rollouts

Start a rollout. Requires rollout:write and a pro license.

GET

/api/v1/rollouts/{id}

Get one rollout. Requires rollout:read.

POST

/api/v1/rollouts/{id}/pause

Running only → paused. 409 otherwise. Requires rollout:write.

POST

/api/v1/rollouts/{id}/promote

Running or paused → promoted at the final step's weight. 409 otherwise. Requires rollout:write.

POST

/api/v1/rollouts/{id}/abort

Any non-terminal state → rolled_back, weight restored. 409 if already finished. Requires rollout:write.

curl -X POST https://nexus-api.example.com/api/v1/rollouts \ -H "Authorization: Bearer $GSLB_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "memberId": "<member-id>", "steps": [ {"weightPct": 10, "holdSeconds": 300}, {"weightPct": 50, "holdSeconds": 300}, {"weightPct": 100, "holdSeconds": 0} ], "regressionPolicy": {"minHealthScore": 0.5, "maxRttDeltaMs": 50, "minHealthyMembers": 1} }'

Response is a Rollout object (state starts pending; the background controller picks it up within one tick). See docs/swagger.yaml for the full schema.


Terraform

resource "nexus_rollout" "canary" {
member_id = nexus_member.web01.id
steps = [
{ weight_pct = 10, hold_seconds = 300 },
{ weight_pct = 50, hold_seconds = 300 },
{ weight_pct = 100, hold_seconds = 0 },
]
regression_policy = {
min_health_score = 0.5
max_rtt_delta_ms = 50
min_healthy_members = 1
}
}

Fire-and-forget semantics. A rollout completes or auto-rolls-back on its own schedule, independent of Terraform — terraform apply does not step, pause, or promote it, and terraform plan will not show drift from the schedule's own progress (only member_id, steps, and regression_policy are tracked as configuration; state, current_step, etc. are read-only computed attributes). There is no API to edit a rollout's schedule in place, so changing any of those three fields forces replacement — Terraform destroys the old rollout (aborting it, restoring the member's weight) and starts a new one. To intervene in a running rollout without touching the schedule, use gslbctl rollouts pause/promote/abort (or the REST API / WebUI) instead.

terraform destroy aborts the rollout if it's still pending, running, or paused. Destroying a rollout that already finished (promoted or rolled back) is a no-op against the API — there's nothing left to abort.


Permissions

Permission

What it grants

rollout:read

List and view rollouts.

rollout:write

Start, pause, promote, and abort rollouts.

Default grants:

Role

rollout:read

rollout:write

tenant_admin

✓

✓

operator

✓

✓

viewer

✓

—


Related


Was this article helpful?