Kalle/ATM
1
0
Fork 0
Code Issues Pull requests 14 Projects Releases 13 Packages Wiki Activity Actions

test(blackbox): version-controlled runner + real auth happy-path #55

Open
KnallerKalle wants to merge 2 commits from feat/blackbox-auth-happy-path into dev
pull from: feat/blackbox-auth-happy-path
merge into: Kalle:dev
Conversation 0 Commits 2 Files changed 4 +341 -2
KnallerKalle commented 2026-08-12 04:16:09 +00:00 (Migrated from github.com)
Copy link

Plan

Deliver the Release Hardening "Blackbox suite ≥ 8 tests, all green on testing" item — ship a repo-local, version-controlled blackbox runner and strengthen auth coverage from "endpoint alive" to a real end-to-end happy path.

This PR was created by an AI agent (OpenHands) on behalf of the ATM automation, following the atm-development iteration loop.

Outcome

  • Added scripts/blackbox_test.py to the repo — a version-controlled copy of the canonical atm-development skill runner, so CI can exercise the same suite that gates releases. Previously the runner existed only inside the skill, so the repo had no version-controlled end-to-end test runner.
  • Added t_auth_demo_login_flow: a real end-to-end auth happy path using the seeded demo accounts:
    • POST /api/auth/tauri-login (demo) → 200 {ok, user.id}
    • GET /api/auth/profile (Bearer) → 200 own profile
    • GET /api/auth/profile (bad Bearer) → 401 authz boundary
    • POST /api/auth/tauri-login (wrong pw) → 401
  • The auth coverage was previously a shallow "endpoint alive (4xx on empty body)" probe; it is now a verified login → profile → authz flow.
  • The suite grew from 8 → 12 checks.

Impact on release goals

  • Release Hardening → "Blackbox suite ≥ 8 tests" advances: suite now has 12 checks (was 8), and auth coverage is no longer a shallow "alive" probe but a verified login→profile→authz flow.
  • Auth happy-path coverage — previously a roadmap risk (auth is [x] but only unit-tested) — is now end-to-end blackbox-verified on the testing deployment.
  • Suite-wide green is blocked only by the pre-existing GET /api/lessons failure tracked separately by PR #53 (lessons-catalog-seed). This iteration did not weaken, skip, or delete that test.

Blackbox

  • python3 scripts/blackbox_test.py --target testing → 11/12 green
  • New auth checks (all green): demo login 200, profile 200, bad-bearer 401, wrong-pw 401.
  • Single red: GET /api/lessons → non-empty list (pre-existing, tracked by PR #53 — a separate in-flight iteration).

Six-perspectives check

  • Product: Strengthens the release gate (observable to a reviewer); no user-facing change.
  • Engineering: Minimal diff (additive only — new Python file + new test functions). No production code, route, or schema changed. Demo-login probes are read-only/stateless.
  • Psychology: No user-facing change; strengthens the autonomy-supportive auth path verification.
  • Biomechanics: N/A (test infrastructure).
  • Anatomy: N/A (test infrastructure).
  • Feldenkrais: No correction/judgement/strain introduced; test infrastructure only.

Risk

Minimal. Additive only (new Python file + new test functions). No production code, no route, no schema changed. The demo-login probes are read-only/stateless (demo accounts are seeded; no writes). Cannot affect the running deployment. Synced the skill's canonical runner to match so future runs stay consistent.

Checklist

  • Branch off latest dev
  • Conventional Commits (test:, docs:)
  • Patchlog entry appended
  • Roadmap status updated ([ ] → [~])
  • No secrets / no large binaries
  • No Feldenkrais-principle violation
## Plan Deliver the **Release Hardening "Blackbox suite ≥ 8 tests, all green on testing"** item — ship a repo-local, version-controlled blackbox runner and strengthen auth coverage from "endpoint alive" to a real end-to-end happy path. > _This PR was created by an AI agent (OpenHands) on behalf of the ATM automation, following the atm-development iteration loop._ ## Outcome - Added `scripts/blackbox_test.py` to the repo — a version-controlled copy of the canonical atm-development skill runner, so CI can exercise the same suite that gates releases. Previously the runner existed only inside the skill, so the repo had no version-controlled end-to-end test runner. - Added `t_auth_demo_login_flow`: a real end-to-end auth happy path using the seeded demo accounts: - `POST /api/auth/tauri-login` (demo) → `200 {ok, user.id}` - `GET /api/auth/profile` (Bearer) → `200` own profile - `GET /api/auth/profile` (bad Bearer) → `401` authz boundary - `POST /api/auth/tauri-login` (wrong pw) → `401` - The auth coverage was previously a shallow "endpoint alive (4xx on empty body)" probe; it is now a verified login → profile → authz flow. - The suite grew from 8 → 12 checks. ## Impact on release goals - **Release Hardening → "Blackbox suite ≥ 8 tests"** advances: suite now has 12 checks (was 8), and auth coverage is no longer a shallow "alive" probe but a verified login→profile→authz flow. - Auth happy-path coverage — previously a roadmap risk (auth is `[x]` but only unit-tested) — is now end-to-end blackbox-verified on the testing deployment. - Suite-wide green is blocked only by the pre-existing `GET /api/lessons` failure tracked separately by **PR #53** (lessons-catalog-seed). This iteration **did not weaken, skip, or delete** that test. ## Blackbox - [x] `python3 scripts/blackbox_test.py --target testing` → **11/12 green** - New auth checks (all green): demo login `200`, profile `200`, bad-bearer `401`, wrong-pw `401`. - Single red: `GET /api/lessons → non-empty list` (pre-existing, tracked by PR #53 — a separate in-flight iteration). ## Six-perspectives check - [x] **Product:** Strengthens the release gate (observable to a reviewer); no user-facing change. - [x] **Engineering:** Minimal diff (additive only — new Python file + new test functions). No production code, route, or schema changed. Demo-login probes are read-only/stateless. - [x] **Psychology:** No user-facing change; strengthens the autonomy-supportive auth path verification. - [x] **Biomechanics:** N/A (test infrastructure). - [x] **Anatomy:** N/A (test infrastructure). - [x] **Feldenkrais:** No correction/judgement/strain introduced; test infrastructure only. ## Risk **Minimal.** Additive only (new Python file + new test functions). No production code, no route, no schema changed. The demo-login probes are read-only/stateless (demo accounts are seeded; no writes). Cannot affect the running deployment. Synced the skill's canonical runner to match so future runs stay consistent. ## Checklist - [x] Branch off latest `dev` - [x] Conventional Commits (`test:`, `docs:`) - [x] Patchlog entry appended - [x] Roadmap status updated (`[ ]` → `[~]`) - [x] No secrets / no large binaries - [x] No Feldenkrais-principle violation
This pull request has changes conflicting with the target branch.
  • .development/automation-state.json
  • .development/patchlog.md
  • .development/roadmap.md
  • scripts/blackbox_test.py
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/blackbox-auth-happy-path:feat/blackbox-auth-happy-path
git switch feat/blackbox-auth-happy-path

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch dev
git merge --no-ff feat/blackbox-auth-happy-path
git switch feat/blackbox-auth-happy-path
git rebase dev
git switch dev
git merge --ff-only feat/blackbox-auth-happy-path
git switch feat/blackbox-auth-happy-path
git rebase dev
git switch dev
git merge --no-ff feat/blackbox-auth-happy-path
git switch dev
git merge --squash feat/blackbox-auth-happy-path
git switch dev
git merge --ff-only feat/blackbox-auth-happy-path
git switch dev
git merge feat/blackbox-auth-happy-path
git push origin dev
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Backend development tasks

  
bug
Something isn't working

  
completed
Features and tasks that are complete and merged

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
feature
New functionality

  
frontend
Frontend development

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
in-progress

   
invalid
This doesn't seem right

  
performance
Performance improvements

  
phase-1
Phase 1 Foundation tasks

  
Phase1
Phase 1 Foundation work

  
Phase2
Phase 2 Core Features

  
Phase3
Phase 3 AI Generation

  
planning
Planning and roadmap items

  
question
Further information is requested

  
refactor
Code refactoring

  
security
Security issues

  
test
Testing related

  
testing
Testing and verification tasks

  
wontfix
This will not be worked on

No labels
backend
bug
completed
documentation
duplicate
enhancement
feature
frontend
good first issue
help wanted
in-progress
invalid
performance
phase-1
Phase1
Phase2
Phase3
planning
question
refactor
security
test
testing
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Kalle/ATM!55
No description provided.