ci: denní kontrola API digiarchivu a AMČR OAI

Plánovaný workflow api_monitor.yml (denně 05:17 UTC + ruční spuštění)
ověřuje kontrakt API, na kterém plugin závisí:

- tests/api_contract.py – stejné dotazy jako plugin, kontrola klíčů,
  typů a tvarů odpovědí (OK / DRIFT / FAIL / UNAVAILABLE)
- tests/api_plugin_live.py – vlastní funkce pluginu (fetch_set,
  load_amcr_data) proti živému API v qgis/qgis:ltr
- tests/api_monitor_report.py – jedno sledovací issue se štítkem
  api-monitor; čistý běh ho zavře, výpadek ho nemění

Neběží na PR, aby výpadek digiarchivu neshodil PR. Popis v AGENTS.md,
OpenSpec změna add-daily-api-monitor.

Připraveno s pomocí AI (Claude), ověřeno proti produkčnímu API.

openspec/changes/add-daily-api-monitor/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-10-02

openspec/changes/add-daily-api-monitor/design.md

@@ -0,0 +1,173 @@
+# Design
+
+## Context
+
+- What broke in #67: the facet item shape of `api/search/query`
+  (`json.nl` `arrntv` → `arrarr` with the Solr 10 migration in digiarchiv
+  v4.1.0). `fetch_set` caught the `TypeError`, logged a warning and returned
+  `[]`; nothing outside QGIS noticed. The plugin now accepts both shapes
+  (`amcr_codelists._facet_name`) and keeps previous codelist values when a
+  set comes back empty, which makes the next break of this kind even
+  quieter for users – only a check outside the plugin catches it.
+- What the plugin uses (from the code on `main`):
+  - `GET https://digiarchiv.aiscr.cz/api/assets/i18n/cs.json`
+    (`amcr_tools.load_translations`)
+  - `GET …/api/search/query` with `entity`, `mapa=true`, `sort=ident_cely
+    asc`, `rows=500`, `page`, `loc_rpt=minLat,minLon,maxLat,maxLon`,
+    filters as `key=value:or` lists, date ranges; response
+    `response.numFound` / `response.docs[]`, errors as HTTP 200 without
+    `response` (`amcr_tools.load_amcr_data`, `_api_get_json`)
+  - PIAN geometry requests in batches (`amcr_tools.load_amcr_data`)
+  - `…/api/search/query` with `rows=0&noFacets=false&onlyFacets=true`,
+    response `facet_counts.facet_fields.<f_…>[]` (`amcr_codelists.fetch_set`
+    for `vedouci`, `nalezce`)
+  - `GET https://api.aiscr.cz/2.2/oai?verb=ListRecords&metadataPrefix=oai_dc
+    &set=…` with `resumptionToken` pagination (`fetch_set`, all other sets
+    in `amcr_codelists.slovnicek`)
+  - `POST …/api/user/login`, `GET …/api/user/islogged`, `…/logout`
+    (`amcr_tools.login_to_api`, session check)
+- Live probe (2026-10-02, anonymous, Praha bbox `49.9,14.3,50.2,14.7`, one
+  page of 500): `akce` numFound 20 635 (1.0 MB, 0.6 s), `lokalita` 342
+  (0.8 MB, 0.4 s), `pian` 30 321 (0.8 MB, 0.5 s), `samostatny_nalez` 0
+  (anonymous SN are sparse – the test bbox must be chosen so that every
+  entity returns records). Bbox chosen: Mikulov `48.8,16.6,48.9,16.75`
+  – akce 185, lokalita 18, samostatny_nalez 2, pian 294. Only 12
+  anonymous map-enabled SN exist in the whole CZ, so 2 is accepted:
+  archive records do not disappear (maintainer decision); facet request 54 fields, items are lists. The
+  deployed version read from the web bundle: `v4.0.3-237-g96deec70-dirty`
+  (footer text is hard-coded and not reliable).
+- GitHub runs `schedule` only on the default branch (`main`) and disables
+  scheduled workflows after 60 days without repository activity.
+
+## Goals / Non-Goals
+
+**Goals:**
+- Daily, credential-free check that tells apart three outcomes: the API
+  changed in a way the plugin breaks on (fail), the API changed in a way
+  the plugin survives (drift), the API was not reachable (unavailable).
+- Messages specific enough to start a fix without re-probing (field, old
+  shape, new shape, request).
+- One tracking issue, no daily noise.
+
+**Non-Goals:**
+- Logged-in access, access levels B–D.
+- Full data validation (counts, contents of records).
+- Changing plugin code to make it more testable – if a function cannot be
+  called without UI, the live test provides a fake `iface` / canvas.
+
+## Decisions
+
+1. **Two scripts, two jobs.** `tests/api_contract.py` (job *API contract*,
+   `ubuntu-latest` + `actions/setup-python`, `requests` pinned in workflow
+   `env`) and `tests/api_plugin_live.py` (job *Plugin against live API*,
+   `docker run qgis/qgis:ltr`, same invocation style as the smoke test).
+   *Alternative:* one script in the QGIS container – rejected: a QGIS image
+   problem would hide the contract result, and the contract check would
+   pay the image pull every day.
+   Only `ltr` for the live job: the API path does not differ between Qt5
+   and Qt6, which the PR smoke test already covers on both.
+2. **Contract = recorded expectations in the script, not a stored
+   baseline file.** Each check states the expected keys/types/shapes
+   inline (e.g. facet item is `[str, int]`, `numFound` is `int`,
+   `docs[].ident_cely` is `str`). A mismatch the plugin tolerates is
+   reported as **DRIFT** (e.g. facet shape back to `{"name":…}`, an
+   extra type the parser accepts), a mismatch it does not tolerate as
+   **FAIL**. Updating an expectation is a reviewed code change.
+   *Alternative:* snapshot JSON baseline refreshed automatically – rejected:
+   a silent baseline refresh would accept the very change we want to see.
+3. **Statuses and exit codes.** Each check yields `OK` / `DRIFT` / `FAIL` /
+   `UNAVAILABLE` + detail. Both scripts write `results-<job>.json`
+   (check name, status, detail, request URL without secrets) and a Markdown
+   table to `$GITHUB_STEP_SUMMARY` when set; exit code 1 on any `FAIL` or
+   `DRIFT`, 0 otherwise (an all-`UNAVAILABLE` run is green but visible in
+   the summary). Locally the scripts print the same table.
+4. **Retries.** Network errors, timeouts and HTTP 5xx: 3 attempts with
+   backoff (2 s, 8 s). After that the check is `UNAVAILABLE`; checks that
+   depend on it are skipped as `UNAVAILABLE`, not `FAIL`. HTTP 4xx or a
+   200 with an `error` body is a real answer and is judged by the contract.
+   A per-host circuit breaker follows: once a host fails its full retry
+   cycle, later requests to it return "unreachable" without network I/O,
+   so an all-unreachable run ends in ~10 s instead of tens of minutes.
+5. **Test inputs from the live API, not from `heslar.csv`.** Filter values
+   are taken from facets of the same bbox in the same run; the test bbox is
+   a fixed small area where every entity (`akce`, `lokalita`,
+   `samostatny_nalez`, `pian`) returns a non-zero anonymous count below
+   one page, chosen during implementation by a probe and documented in
+   the script. Pagination is checked separately with `rows=100` on a larger
+   area (pages must not overlap; downloaded ≥ numFound when it is small
+   enough).
+6. **Live plugin test thresholds.** For each set in
+   `amcr_codelists.slovnicek`: `fetch_set` must return ≥ 1 item **and** at
+   least 50 % of the row count of that category in the bundled
+   `codelists/heslar.csv` (a shrunken codelist is the #67 symptom). For
+   each data type: `load_amcr_data` on the test bbox (fake `iface`, fake
+   canvas in EPSG:5514) must add at least one layer with ≥ 1 feature with
+   a valid geometry and the expected attribute fields. The plugin package
+   is imported as a package (`amcr_viewer.amcr_tools`), so its relative
+   imports work – a bare `spec_from_file_location` makes `load_amcr_data`
+   swallow the import error into "0 records".
+7. **Deployed version.** Fetch `https://digiarchiv.aiscr.cz/home`, scan the
+   referenced `*.js` bundles for `raw:"v…"` (git-describe) and report it;
+   not finding it is a `DRIFT` of its own check, never a `FAIL` of the run.
+8. **Reporting job** (`needs` both, `if: always()`, only when
+   `github.ref_name == github.event.repository.default_branch`; the
+   workflow has no other triggers than `schedule` and `workflow_dispatch`; `permissions: issues: write`
+   for this job only, `contents: read` elsewhere). It downloads both result
+   files (artifacts) and with `gh`:
+   - any `FAIL`/`DRIFT` → find the open issue with label `api-monitor`; if
+     none, create it (`gh label create api-monitor --force` first); if it
+     exists and the fingerprint (sorted names of FAIL/DRIFT checks – not
+     UNAVAILABLE, which would make it flap – stored as an HTML comment in
+     the issue body) differs, add a comment and update the
+     fingerprint; identical fingerprint → do nothing;
+   - every check `OK` → close the open issue with a comment linking the
+     run;
+   - no `FAIL`/`DRIFT` but some `UNAVAILABLE` → leave the issue as it is
+     (an outage proves neither break nor recovery);
+   - a job that did not produce its result file (crashed script, image
+     pull failure) counts as one `FAIL` check named after the job.
+   Issue text in Czech (repo convention for issues), unwrapped GFM: deployed
+   version, table of non-OK checks, run link, how to reproduce locally.
+   *Alternative:* `actions/github-script` or a marketplace action – rejected:
+   `gh` is preinstalled and needs no third-party action pin.
+9. **Schedule** `cron: "17 5 * * *"` (07:17 CEST) – off the full hour,
+   before the working day. Plus `workflow_dispatch`. Concurrency group per
+   ref, `cancel-in-progress: false`.
+10. **Not a PR check.** The new workflow does not run on `pull_request`: a
+    PR must not go red because digiarchiv is down. Contributors run the
+    scripts locally or dispatch the workflow on their branch.
+
+## Risks / Trade-offs
+
+- **False alarms from data changes** (a record deleted in the test bbox,
+  an entity count dropping to 0) → thresholds are "≥ 1" and "≥ 50 % of the
+  bundled codelist", not exact counts; the bbox is chosen with margin.
+- **Scheduled workflow auto-disabled after 60 days of inactivity** →
+  documented in `AGENTS.md`; any push to `main` (dependabot included)
+  resets the timer.
+- **Tests only `main`'s plugin code** – a fix waiting on a version branch
+  is not exercised by the schedule → manual dispatch on that branch.
+- **Anonymous only** → `pristupnost` B–D paths and login success are not
+  covered; the login *error* path is.
+- **Load on digiarchiv** – a few dozen small requests a day; negligible.
+
+## Verification
+
+- Both scripts run locally against the live API and pass
+  (`uv run -q --no-project --with requests python tests/api_contract.py`;
+  live test in `docker run qgis/qgis:ltr`).
+- **#67 regression check**: run the live test against the plugin as of the
+  commit before the #67 fix (`git archive`) – it must FAIL on `vedouci` /
+  `nalezce`; the contract test must flag a facet-shape DRIFT when its
+  expectation is temporarily set to the old `{"name":…}` shape.
+- Outage simulation: point the scripts at an unroutable host
+  (environment override of the base URLs) → all checks `UNAVAILABLE`,
+  exit 0.
+- Reporting logic tested with a dry-run mode (`API_MONITOR_DRY_RUN=1`
+  prints the `gh` commands instead of running them) for: new issue, same
+  fingerprint, changed fingerprint, recovery.
+- The full `AGENTS.md` check set passes on the new files (check_sources,
+  bandit, detect-secrets `--all-files`, flake8 `--isolated` on
+  `amcr_viewer/`, ruff on the repo); `actionlint` on the new workflow.
+- After merge: one manual `workflow_dispatch` on `main` and inspection of
+  the summary.

openspec/changes/add-daily-api-monitor/proposal.md

@@ -0,0 +1,78 @@
+# Proposal
+
+## Why
+
+Digiarchiv changes its API without notice to clients. Issue #67 showed the
+cost: digiarchiv v4.1.0 (Solr 10) changed the facet item shape from
+`{"name": …}` to `[value, count]`, the plugin swallowed the resulting
+exception and the person codelists (`vedouci`, `nalezce`) came out empty –
+found by hand, after users were already affected. Today nothing in the
+repository talks to the live API: `tests/smoke_test.py` is deliberately
+offline and CI runs only on pull requests and pushes, so an API change is
+noticed only when a user hits it.
+
+## What Changes
+
+- New scheduled GitHub Actions workflow `.github/workflows/api_monitor.yml`
+  that runs once a day (and on manual dispatch) against the production
+  digiarchiv and `api.aiscr.cz` OAI, without credentials.
+- New **contract test** `tests/api_contract.py` (plain `requests`, no QGIS):
+  sends the same requests the plugin sends and checks the shape of every
+  response the plugin reads – endpoints, keys, value types, facet item
+  shape, OAI sets, pagination, bbox restriction, PIAN geometry, filters,
+  login error path. Says *what* changed in the API.
+- New **live plugin test** `tests/api_plugin_live.py`, run in
+  `qgis/qgis:ltr`: calls the plugin's own functions (`fetch_set` for every
+  codelist set, `load_translations`, `load_amcr_data` per data type) against
+  the live API and checks that they produce non-empty, well-formed results.
+  Says *whether* the change breaks users.
+- Every run records the deployed digiarchiv version (git-describe string
+  from the web bundle) in its summary.
+- Outages (timeouts, HTTP 5xx, connection errors after retries) are
+  reported as *unavailable*, separately from contract breaks, and do not
+  open an issue.
+- A failing or drifting run opens **one** tracking issue (label
+  `api-monitor`) or updates the open one; the next clean run closes it.
+- `AGENTS.md` documents the monitor (what it runs, how to run it locally,
+  how to read the issue).
+
+What changes for a plugin user: nothing directly – no file under
+`amcr_viewer/` changes and the plugin version is not bumped. Indirectly,
+API breaks like #67 are found within a day of a digiarchiv release instead
+of by users.
+
+Out of scope:
+
+- Logged-in checks (variant D): no account secret is stored in the repo;
+  anonymous runs cover only `pristupnost=A` records.
+- Testing version branches on schedule: GitHub runs `schedule` only on the
+  default branch; other refs can be run by manual dispatch.
+- Availability monitoring of digiarchiv as a service.
+
+## Capabilities
+
+### New Capabilities
+
+- `api-monitoring`: periodic verification that the digiarchiv / AMČR OAI
+  API still satisfies the contract the plugin depends on, and reporting of
+  breaks through a tracking issue.
+
+### Modified Capabilities
+
+<!-- none – openspec/specs/ is not maintained (change-tracked) -->
+
+## Impact
+
+- New files: `.github/workflows/api_monitor.yml`, `tests/api_contract.py`,
+  `tests/api_plugin_live.py`; edited `AGENTS.md`.
+- Affected plugin modules (read, not changed): `amcr_viewer/amcr_tools.py`
+  (`load_translations`, `load_amcr_data`, `login_to_api`),
+  `amcr_viewer/amcr_codelists.py` (`slovnicek`, `fetch_set`).
+- API: one run ≈ a few dozen anonymous requests to
+  `digiarchiv.aiscr.cz/api/*` and `api.aiscr.cz/2.2/oai`, restricted to a
+  small bbox – negligible load for digiarchiv.
+- GitHub: scheduled workflow minutes (two short jobs per day), the
+  `api-monitor` label, `issues: write` permission for the reporting job
+  only. Depends on the digiarchiv repository
+  (`ARUP-CAS/aiscr-digiarchiv-2`) only as the source of the API under test.
+- PR targets `main` (repository tooling, no plugin behaviour change).

openspec/changes/add-daily-api-monitor/specs/api-monitoring/spec.md

@@ -0,0 +1,63 @@
+# Spec Delta
+
+## Purpose
+
+Detects changes of the digiarchiv / AMČR OAI API that break or alter what
+the AMČR Viewer plugin relies on, within a day of their deployment, and
+reports them where maintainers see them.
+
+## ADDED Requirements
+
+### Requirement: The API contract is checked daily
+The repository SHALL run, once a day and on manual dispatch, a check of
+every API endpoint, parameter and response field the plugin uses, without
+credentials, against the production API.
+
+#### Scenario: Scheduled run
+- **WHEN** the daily schedule fires on the default branch
+- **THEN** both the contract test and the live plugin test run against the production API and their results are published in the run summary
+
+#### Scenario: Manual run on another branch
+- **WHEN** a maintainer dispatches the workflow on a non-default branch
+- **THEN** the tests run against that branch's plugin code and no issue is opened, updated or closed
+
+### Requirement: Checks follow the plugin, not the API documentation
+The contract test SHALL send requests built the way the plugin builds them
+and check the response keys, value types and shapes the plugin reads. The
+live plugin test SHALL call the plugin's own codelist and download
+functions.
+
+#### Scenario: Facet shape changes
+- **WHEN** the API returns facet items in a shape different from the one recorded in the contract test
+- **THEN** the contract test reports a drift naming the facet field and the old and new shape
+
+#### Scenario: Plugin function returns nothing
+- **WHEN** a plugin codelist set or data download returns zero items for an input that returned items before
+- **THEN** the live plugin test fails and names the set or data type
+
+### Requirement: Outages are not reported as API changes
+A request that times out, fails to connect or returns HTTP 5xx SHALL be
+retried; if it still fails, the check SHALL be reported as unavailable,
+separately from failures and drifts.
+
+#### Scenario: Server maintenance
+- **WHEN** digiarchiv is unreachable during the whole run
+- **THEN** the run reports the affected checks as unavailable and no issue is opened
+
+### Requirement: Breaks are reported through one tracking issue
+A run with a failure or drift on the default branch SHALL open an issue
+labelled `api-monitor`, or update the open one, with the deployed
+digiarchiv version and the list of failing checks. A clean run SHALL close
+the open issue.
+
+#### Scenario: First failing run
+- **WHEN** a scheduled run fails and no open `api-monitor` issue exists
+- **THEN** a new issue is opened with the deployed version, failing checks and a link to the run
+
+#### Scenario: Repeated identical failure
+- **WHEN** a scheduled run fails with the same set of FAIL/DRIFT checks as the open issue already lists, regardless of which checks are unavailable
+- **THEN** no new issue and no new comment is created
+
+#### Scenario: Recovery
+- **WHEN** a scheduled run passes while an `api-monitor` issue is open
+- **THEN** the issue is closed with a comment linking the passing run

openspec/changes/add-daily-api-monitor/tasks.md

@@ -0,0 +1,57 @@
+# Tasks
+
+## 1. Contract test
+
+- [x] 1.1 Probe and fix the test inputs: a small bbox where `akce`,
+  `lokalita`, `samostatny_nalez` and `pian` all return 1–499 anonymous
+  records, and a larger area for pagination; record the probe numbers in
+  the script header; verify by a probe run in scratch
+- [x] 1.2 Write `tests/api_contract.py` with the status model, retries and
+  outputs from design.md (decisions 3, 4, 7) and checks for: i18n
+  `cs.json`, every OAI set in `amcr_codelists.slovnicek` (first page shape
+  + `resumptionToken` paging), facet fields `f_vedouci` / `f_nalezce` item
+  shape, main query per entity (keys and value types the plugin reads,
+  `numFound` int), pagination without overlap, bbox restriction, PIAN
+  batch geometry, every filter key the dialog builds (values taken from
+  live facets), date range, error answer for an invalid parameter (HTTP 200
+  without `response`), unknown entity, login with deliberately wrong
+  credentials; verify a local run is all OK
+- [x] 1.3 Verify the drift detection: temporarily set the facet
+  expectation to the old `{"name":…}` shape → DRIFT reported with the
+  field and both shapes; revert
+
+## 2. Live plugin test
+
+- [x] 2.1 Write `tests/api_plugin_live.py` (package import of
+  `amcr_viewer`, fake `iface` / canvas, thresholds from design.md
+  decision 6, same status model and outputs); verify it passes in
+  `qgis/qgis:ltr`
+- [x] 2.2 #67 regression: run it against the plugin from the commit before
+  the #67 fix (`git archive` into scratch) → FAIL on `vedouci` and
+  `nalezce`; verify and record the output
+
+## 3. Workflow and reporting
+
+- [x] 3.1 Write `.github/workflows/api_monitor.yml` per design.md
+  (decisions 1, 8, 9, 10): pinned action SHAs as in `code_quality.yml`,
+  pinned `requests`, artifacts with result files, reporting job with
+  `issues: write` only; verify with `actionlint`
+- [x] 3.2 Reporting script (inline step or `tests/api_monitor_report.py`)
+  with `API_MONITOR_DRY_RUN=1`; verify the four cases (new issue, same
+  fingerprint, changed fingerprint, recovery) and that a run with only
+  UNAVAILABLE leaves the issue untouched
+- [x] 3.3 Outage simulation (unroutable base URL override) → all
+  UNAVAILABLE, exit 0; verify
+
+## 4. Documentation and checks
+
+- [x] 4.1 `AGENTS.md`: new subsection on the API monitor (what it runs,
+  local commands, how to read the issue, 60-day schedule disable, manual
+  dispatch for version branches); verify by reading the diff
+- [x] 4.2 Run the `AGENTS.md` check set (check_sources, bandit,
+  detect-secrets `--all-files`, flake8 `--isolated` on `amcr_viewer/`,
+  ruff, smoke test in `qgis/qgis:ltr` and `:stable` – unchanged plugin
+  code, must stay green) and `openspec validate add-daily-api-monitor
+  --strict`; verify all clean
+- [ ] 4.3 After merge into `main`: manual `workflow_dispatch` on `main`,
+  inspect the summary and that no issue was opened on a clean run