Maintenance succeeds when users can finish the intended task, with current sources and correct access. Test the workflow end to end, including required network and identity dependencies, during the planned maintenance.
Key takeaways
- Measure logical user requests: record initial failures, retries, and final outcomes separately.
- Test the real boundary: shared identity, network, and application dependencies belong in the experiment.
- Record the test conditions: include topology, versions, load, and maintenance type with the result.
Define success before maintenance
Define a bounded claim: the named workflow meets its response, freshness, and access criteria during the specified supported maintenance. Record the data set, user roles, load, topology, versions, and change type. Derive pass criteria from workflow requirements; this plan sets no universal latency threshold or availability percentage.
Instrument the user result and the underlying work
| Measure | Why it belongs in the record |
|---|---|
| Deadline success across all logical requests | One logical request is one user task; include timeouts and failed requests in the denominator |
| First-attempt and retry outcomes | Reveals interruptions hidden by eventual success |
| End-to-end latency, from first attempt through final outcome | Includes retries and backoff; report latency with timeout and failure counts, not successful-request latency alone |
| Accepted source version | Identifies stale or inconsistent evidence after a change |
| Permission result | Confirms an allowed user succeeds and a restricted user stays restricted |
| Late backend completion | Exposes work continuing after a user timeout or cancellation |
Assign one logical request ID per user task and distinct IDs to attempts. Report baseline, maintenance, and recovery windows separately. Retain minimal test metadata, not raw protected document content.
Run cases that isolate different dependencies
Run the ordinary path, then the approved planned-maintenance case. If supported, separately remove a serving component and test remaining capacity under representative load. Add client-reconnection and architecture-relevant shared-dependency cases, isolating each event so failures can be attributed to the condition under test. Azure SQL's planned-maintenance documentation describes client interruption and recovery considerations for Azure SQL; use the tested engine's own guidance.
Apply the same test boundary to an integrated platform
Seahorse Cloud documents storage, parsing, vector synchronization, and managed-agent components. For the selected workflow, distinguish the serving path (retrieval, agent, and model components as applicable) from document updates through ingestion, parsing, and vector refresh; identify which dependencies are shared with the maintenance target. A parsing pause may affect freshness while searches continue against an existing index; test whether that applies to the selected architecture.
Compare provider-managed service with self-operation using the same workload and evidence. Keep the hosting arrangement, commercial terms, and operator duties visible. A feature list describes scope; the test shows the result for the selected setup.
Report what passed and what remains untested
Report deadline success across all logical requests, first-attempt failures, retries, timeouts, request counts, latency, freshness, and access results. Separate baseline, maintenance, and recovery results; record the maintenance event, recovery actions, and any differences from production topology or load.
Approve the tested setup, change a specific boundary, or retest after a change. Record any remaining gap as the next action.