Observability, delivery, and security
A service is not complete until teams can deploy it safely, trace its behaviour, protect its boundaries, and own incidents.
Correlate logs, metrics, and traces
Metrics reveal trends and alert conditions. Logs explain discrete events. Distributed traces show where time and errors travel through a request. Use the same service name, environment, trace identifier, and deployment version across all three.
Instrument boundaries first: inbound requests, outbound calls, broker publishing and consumption, database queries, and important business outcomes.
Make deployments boring and reversible
Build immutable artifacts once and promote the same artifact between environments. Automate health checks, progressive rollout, rollback, and database compatibility checks.
A deployment is not successful because containers started. It is successful when user-facing indicators remain healthy. Compare the new version against the previous one during canary or gradual traffic shifts.
Authenticate identity and authorise every operation
A trusted network is not sufficient. Validate service identity, user context, audience, expiry, and permission at the service performing the action. Keep credentials short-lived and rotate them automatically.
Minimise sensitive data in APIs, events, logs, and traces. Encrypt transport, isolate secrets from images and source control, and audit privileged actions.
Give every service a production owner
- Repository and owning team
- Runbook and escalation path
- Dashboard and alerts tied to an SLO
- Dependencies and data classification
- Deployment and rollback procedure
- Capacity, backup, and recovery expectations
Earn complexity gradually
Standardise common concerns through a small platform: service templates, identity, telemetry, deployment, and local tooling. Keep the paved road helpful rather than mandatory bureaucracy.
Regularly remove services whose boundary no longer pays for itself. Architecture is a living trade-off, not a one-way journey toward more boxes.
Can you explain “Observability, delivery, and security” without reaching for the API reference?
Describe the main trade-off, one failure mode, and the decision you would make in a real project.