Guard deploys against a broken migration
CI / build-and-push (push) Successful in 46s
CI / deploy (push) Successful in 9s

Program.cs: wrap the boot-time MigrateAsync in try/catch (was an unhandled
exception into a restart:unless-stopped crash-loop), log critical and exit(1)
on failure, and add a --migrate-only flag that applies migrations then exits
0 without starting the web host.

deploy.sh: run migrations as a preflight via the new --migrate-only image
against the live novelly-data volume, before the running (old-image) stack
is touched. A failing migration now aborts the deploy with the old
containers still serving traffic, instead of swapping to a crash-looping
new container first and finding out from the health-check timeout.
This commit is contained in:
James Wampler
2026-08-19 14:23:34 -07:00
parent 7a1c726af8
commit 8f93dce065
3 changed files with 42 additions and 1 deletions
+14
View File
@@ -34,6 +34,20 @@ services:
ports: ports:
- "${WEB_PORT:-6173}:80" - "${WEB_PORT:-6173}:80"
# Preflight migration check, run by deploy.sh via `--profile tools run --rm migrate`
# against the newly pulled image before the running stack is touched. Excluded from
# `up -d` by the tools profile.
migrate:
image: ${API_IMAGE}:latest
command: ["dotnet", "Novelly.Api.dll", "--migrate-only"]
environment:
ConnectionStrings__Novel: "Data Source=/data/novel.db"
volumes:
- novelly-data:/data
networks:
- novelly
profiles: ["tools"]
networks: networks:
novelly: novelly:
name: novelly-net name: novelly-net
+9
View File
@@ -19,6 +19,15 @@ COMPOSE="docker compose -f deploy/qa/docker-compose.qa.yml"
log "Pulling latest :latest images" log "Pulling latest :latest images"
$COMPOSE pull $COMPOSE pull
# Applies pending migrations against the live novelly-data volume using the new image,
# before the running (old-image) stack is touched. If a migration is broken, this fails
# here and the old containers keep serving traffic — `down`/`up` below never runs, so
# there is nothing to roll back.
log "Running preflight migration check"
if ! $COMPOSE --profile tools run --rm migrate; then
fail "migration failed against the new image; old novelly stack left running untouched"
fi
log "Recreating the novelly stack (data volume preserved)" log "Recreating the novelly stack (data volume preserved)"
$COMPOSE down $COMPOSE down
$COMPOSE up -d $COMPOSE up -d
+18
View File
@@ -41,12 +41,30 @@ builder.Services.AddCors(options => options.AddDefaultPolicy(policy => policy
.AllowAnyMethod() .AllowAnyMethod()
.AllowCredentials())); .AllowCredentials()));
var migrateOnly = args.Contains("--migrate-only");
var app = builder.Build(); var app = builder.Build();
using (var scope = app.Services.CreateScope()) using (var scope = app.Services.CreateScope())
{ {
var db = scope.ServiceProvider.GetRequiredService<NovelDbContext>(); var db = scope.ServiceProvider.GetRequiredService<NovelDbContext>();
try
{
await db.Database.MigrateAsync(); await db.Database.MigrateAsync();
}
catch (Exception ex)
{
app.Logger.LogCritical(ex, "Database migration failed on startup");
Environment.Exit(1);
}
if (migrateOnly)
{
app.Logger.LogInformation("Migration complete, exiting ({MigrateOnlyFlag})", "--migrate-only");
Environment.Exit(0);
}
await ServiceUser.EnsureSeededAsync(db, builder.Configuration[ServiceApiKeyAuthenticationHandler.ConfigurationKey], app.Logger); await ServiceUser.EnsureSeededAsync(db, builder.Configuration[ServiceApiKeyAuthenticationHandler.ConfigurationKey], app.Logger);
} }