Fix CI build/deploy/smoke pipeline for the self-hosted qa runner #3

Merged

wamplerj merged 21 commits from build-runner-fix into main 2026-07-04 18:53:05 -07:00

Conversation 0 Commits 21 Files Changed 39 +1344 -14

wamplerj commented 2026-07-04 18:52:40 -07:00

Owner

Copy Link

Summary

• Fix a chain of QA CI issues: Docker build/health-check flakiness, NuGet vuln pins, then adds the post-deploy integration + Playwright smoke suite and works through everything needed to make it actually run on the self-hosted qa runner (musl/Alpine job container, no node/dotnet/curl preinstalled, docker-outside-of-docker networking).
• Adds a fixed dev/QA seed admin user + fixed Development environment API key so integration tests and e2e specs have a stable target.
• Pins Aspire's AppHost.cs ports/credentials to match the docker-compose local dev defaults.
• Adds tests/api/MicCheck.Api.Tests.Integration coverage for disabled flags, environment-document bootstrap, identity override precedence, auth login, and unauthorized access; adds a Playwright e2e suite under src/admin/e2e (login, nav, context selection, features CRUD/toggle).
• Adds a smoke-qa CI job that runs both suites against the just-deployed QA stack, working around: no curl/node/dotnet on the bare runner, musl vs glibc (Playwright browsers run via the official mcr.microsoft.com/playwright image instead), and the runner's job-container network isolation (reach the QA stack via the docker bridge gateway IP; docker cp instead of a bind mount to get files into the playwright container, since paths don't cross the docker-outside-of-docker boundary).

Test plan

• Unit tests pass (dotnet test tests/api/MicCheck.Api.Tests.Unit, 243 passed)
• API integration suite passes against the real QA stack in CI
• Playwright e2e suite passes against the real QA stack in CI (verified 3/3 locally against a real dev API + vite server for the flakiest spec)
• Full CI pipeline (build → deploy-qa → smoke-qa) green end to end on the qa runner

## Summary - Fix a chain of QA CI issues: Docker build/health-check flakiness, NuGet vuln pins, then adds the post-deploy integration + Playwright smoke suite and works through everything needed to make it actually run on the self-hosted `qa` runner (musl/Alpine job container, no node/dotnet/curl preinstalled, docker-outside-of-docker networking). - Adds a fixed dev/QA seed admin user + fixed Development environment API key so integration tests and e2e specs have a stable target. - Pins Aspire's `AppHost.cs` ports/credentials to match the docker-compose local dev defaults. - Adds `tests/api/MicCheck.Api.Tests.Integration` coverage for disabled flags, environment-document bootstrap, identity override precedence, auth login, and unauthorized access; adds a Playwright e2e suite under `src/admin/e2e` (login, nav, context selection, features CRUD/toggle). - Adds a `smoke-qa` CI job that runs both suites against the just-deployed QA stack, working around: no curl/node/dotnet on the bare runner, musl vs glibc (Playwright browsers run via the official `mcr.microsoft.com/playwright` image instead), and the runner's job-container network isolation (reach the QA stack via the docker bridge gateway IP; `docker cp` instead of a bind mount to get files into the playwright container, since paths don't cross the docker-outside-of-docker boundary). ## Test plan - [x] Unit tests pass (dotnet test tests/api/MicCheck.Api.Tests.Unit, 243 passed) - [x] API integration suite passes against the real QA stack in CI - [x] Playwright e2e suite passes against the real QA stack in CI (verified 3/3 locally against a real dev API + vite server for the flakiest spec) - [x] Full CI pipeline (build → deploy-qa → smoke-qa) green end to end on the qa runner

wamplerj added 21 commits 2026-07-04 18:52:40 -07:00

Auto-install .NET SDK in CI scripts when runner lacks it ... 37ac4b47f2

Gitea runner build failed with "dotnet: command not found". Added
ensure_dotnet() to lib.sh, using the vendored dotnet-install.sh to
bootstrap the SDK into .dotnet/ when not already on PATH, called from
build.sh/test.sh/prepush.sh. Also trigger CI on build-runner-fix to
verify the fix.

Pin Microsoft.OpenApi to patched 2.9.0 to fix NU1903 build failure ... 926af91389

Microsoft.AspNetCore.OpenApi 10.0.5 pulls in Microsoft.OpenApi 2.0.0
transitively, which has a known high-severity vulnerability
(GHSA-v5pm-xwqc-g5wc) and fails the build with TreatWarningsAsErrors.
Pinned a direct reference to the patched 2.9.0 (still 2.x, API-compatible).

Pin MessagePack to patched 2.5.302 in AppHost ... a81798cffe

Aspire.Hosting.PostgreSQL/JavaScript pull in MessagePack 2.5.192
transitively, which has multiple known vulnerabilities and fails the
build with TreatWarningsAsErrors. Pinned a direct reference to the
patched 2.5.302 (same major, no API break).

Add .dockerignore to stop host obj/bin leaking into CI image builds ... 2dc40f72ce

Dockerfile.ci COPYs full project directories after restoring inside
the container. Without a .dockerignore, the host's obj/bin (built
with a different local SDK than the container's) got copied over the
container's fresh restore output, corrupting project.assets.json and
crashing ResolvePackageAssets with a NullReferenceException on publish.

Replace actions/checkout with plain git in deploy-qa job ... 80d450207c

Self-hosted qa runner has no node in PATH, so the Node-based
actions/checkout@v4 action can't run there. Swap it for a bash git
fetch/checkout, matching the pipeline's no-marketplace-action convention.

Fix api container healthcheck failing with no error logs ... 8d0cef08b1

aspnet base image ships neither curl nor wget, so the compose
healthcheck's wget call failed silently at the exec level (visible only
in docker's health-check log, not app stdout). Install curl in the
Dockerfile.ci final stage and switch the healthcheck to use it.

Fix health check path mismatch causing 404s ... b32f5d56db

The app maps health checks at /health (ServiceDefaults), not /api/v1/health.
Compose healthcheck and deploy-qa.sh's wait-loop both hit the wrong path.
Also add an nginx location for /health since it lives outside the /api
prefix that the admin proxy otherwise forwards unchanged.

Fix deploy-qa health check unreachable from runner's network namespace ... 1b2ce263e5

The self-hosted runner runs in its own container on a separate bridge
network, so curling localhost:$QA_ADMIN_PORT hit the runner's own loopback
instead of the docker host's published port, never the QA stack.
Exec into the admin container and curl its own localhost instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Add integration + Playwright smoke suite for post-deploy QA validation ... 8fc0ebb724

Existing coverage was one integration test and zero e2e tests. Adds the thin,
high-value integration/e2e layer unit tests can't reach (real Postgres wire
contract, real browser flows) and wires it as a post-deploy smoke gate.

- Seed a deterministic dev/QA admin user and fixed Development env key so
  HTTP-only tests can authenticate without per-run registration.
- Expand the API integration suite: disabled-flag, unauthorized access,
  environment-document bootstrap, identity-override precedence, and auth
  login scenarios, reusing the existing black-box HTTP+Npgsql harness.
- Add a Playwright suite for the admin SPA: login, nav, project/environment
  context selection, and feature create/toggle, against the real API.
- Bind QA Postgres to 127.0.0.1 for direct seeding, add smoke-qa.sh and an
  ensure_node() bootstrap (the qa runner has no Node today), and wire a new
  smoke-qa CI job after deploy-qa.

Pin Aspire AppHost ports/credentials to match docker-compose defaults ... 404b1e9904

Aspire assigned random ports and an auto-generated Postgres password on each
run, so the fixed targets baked into local.runsettings and docker-compose
(localhost:5000/5173/5432, miccheck/password) didn't work when developing via
Aspire instead of docker compose. Pins Postgres user/password/port and the
api/admin HTTP endpoints so either workflow hits the same local addresses.

Claude.md tweaks 25504a8a85

Fall back to wget in ensure_node when curl is unavailable ... 2d6ee2197b

The self-hosted qa runner has neither curl nor wget guaranteed; ensure_node
previously assumed curl, breaking smoke-qa on that host.

Install native runtime deps for dotnet when missing on bare runners ... 8427e4c8a1

The self-hosted qa runner's minimal image lacked libstdc++/libgcc, which
dotnet fails on with an obscure symbol-relocation error rather than a
clear "missing dependency" message.

Add apk fallback for native dotnet deps on musl/Alpine runners ... ec6bb38902

dotnet-install.sh silently picks the linux-musl-x64 SDK on this runner,
which has no apt-get - only apk - so the previous apt-only fix still failed.

Reach QA's published ports from the smoke-qa job container via bridge gateway ... c001cff9ba

The runner executes each job in its own container on docker's default bridge
network, so "localhost" never resolves to the docker host - smoke-qa's direct
Postgres connection and Playwright/API HTTP calls need the bridge gateway IP
instead. Bind the db port to that same gateway IP (not 0.0.0.0) so it stays
reachable only from sibling containers, not off-box.

Install Node via apk on musl runners instead of nodejs.org's glibc tarball ... 2d81b7655d

Same runner dotnet-install.sh detects as musl and picks linux-musl-x64 for -
the glibc node tarball can't even exec there ("env: can't execute 'node'").

Run Playwright e2e in the official playwright container on musl runners ... aeb4f03627

Playwright's bundled Chromium is glibc-only and --with-deps assumes apt -
both fail on this musl/Alpine runner. Run the e2e leg in
mcr.microsoft.com/playwright (browsers preinstalled, pinned to the exact
resolved @playwright/test version) instead, reachable at the same bridge
gateway IP as the rest of the QA stack.

Use npm run e2e instead of npx playwright in the smoke container ... cff98b5df4

npx resolved the registry's generic "playwright" package instead of the
locally installed @playwright/test bin, triggering an unwanted fresh install
and "No tests found". npm run e2e uses the project's own node_modules/.bin
unambiguously.

Use docker cp instead of a bind mount to get repo files into the playwright container ... 37e00bf756

This script runs inside the runner's own job container and talks to the
host's docker daemon over a mounted socket (docker-outside-of-docker). A
bind mount path only exists inside this job container, not on the host the
daemon resolves it against, so docker run -v failed with ENOENT. docker cp
copies file contents instead, sidestepping the path mismatch.

Wait for the toggle PUT to land before reloading in the feature e2e test ... cfb6c5bde3

The click fires an async toggle-state update; reloading immediately after
raced it and read back the pre-toggle value, flaking the persistence check.

Assert on the real featurestate PATCH landing, not the switch's transient DOM flash ... 5f3fe81758

When the feature-state map isn't loaded yet for a row, the click handler
no-ops silently and the native checkbox briefly flashes toggled before Vue
snaps it back to the unchanged model-value - reading as success for an
instant even though nothing was sent. Wait for the actual PATCH response
so the test fails clearly instead of passing then flaking on reload.
Verified 3/3 locally against a real dev API + vite server.

wamplerj merged commit 1556b486d2 into main 2026-07-04 18:53:05 -07:00

wamplerj deleted branch build-runner-fix 2026-07-04 18:53:05 -07:00

wamplerj referenced this issue from a commit 2026-07-04 18:53:07 -07:00

Fix CI build/deploy/smoke pipeline for the self-hosted qa runner (#3)

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

wamplerj

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: wamplerj/mic-check#3