ADR-007: Architecture Simplification & Cloud Resource Optimization

Status

Proposed — Pending engineering team review

Context

The platform accumulated significant infrastructure complexity across 191 repos, 35 services, 4 dedicated clusters, 140 databases, 48 Helm charts, 121 Terraform files, and 28 CI/CD workflows. ADRs 001 and 004 address service and cluster consolidation. ADR-005 addresses database consolidation. This ADR takes a holistic view of architecture simplification and cloud resource optimization to improve supportability, maintainability, and cost efficiency.

Current Complexity Metrics

Decision

Implement a five-pillar simplification strategy that reduces operational surface area, eliminates redundancy, and right-sizes cloud resources.

Pillar 1: Repository Consolidation

Result: From 191 repos → ~40 actively maintained repos.

Pillar 2: Infrastructure Right-Sizing

Pillar 3: Observability Consolidation

Replace the fragmented observability stack with a unified approach:

Consolidation benefit: Eliminate peeq-logging service, eliminate self-managed Elasticsearch/Kibana, reduce monitoring services from 5+ to 2 (Prometheus stack + Cloud Logging).

Pillar 4: CI/CD Simplification

Pillar 5: Dead Code & Redundancy Elimination

Combined Cost Impact Estimate

Hypothesis Background

Primary: Systematic simplification across infrastructure, code, and operations reduces total cost of ownership by 50-70% while improving system reliability and developer experience.

• Evidence: The platform was built for scale that hasn’t materialized — 4 separate clusters for 4 brands with identical code (H11 L2). The infrastructure was designed for a future that assumed many more tenants needing hard isolation.
• Evidence: Over-decomposition is documented — 6 services with <5 endpoints (tech-debt-inventory). Consolidation is straightforward because all services follow identical patterns (H13 L1).
• Evidence: Observability gaps (no alerting, APM disabled, no SLOs) mean the team is paying for infrastructure they can’t effectively monitor.

Alternative 1: Optimize incrementally without a unified strategy. - Rejected: Individual optimizations (just consolidate clusters, or just fix observability) miss compounding benefits. Database consolidation enables simpler PgBouncer which enables simpler Cloud SQL which enables simpler Terraform. The optimizations reinforce each other.

Alternative 2: Rewrite on a simpler platform (e.g., monolith on Cloud Run). - Rejected: H14 falsified — the application code is sound. The complexity is in infrastructure and operational overhead, not in business logic. A rewrite would duplicate proven integrations.

Falsifiability Criteria

• If combined optimizations don’t achieve at least 30% cloud cost reduction → reassess individual pillar assumptions
• If simplification causes more than 2 production incidents in the first quarter → pause and stabilize before continuing
• If developer experience surveys show increased cognitive load despite fewer components → investigate and adjust
• If GCP billing shows unexpected cost increases from any pillar → immediately investigate and revert if needed

Evidence Quality

Overall: L1 (WLNK capped by billing data and traffic patterns L0)

Bounded Validity

• Scope: All application services, infrastructure, CI/CD, and observability. Excludes external SaaS integrations (Stripe, Mux, Zoom, etc.) which are already optimized.
• Expiry: Re-evaluate cost targets annually. Re-evaluate architecture if tenant count exceeds 10 or if a fundamentally different workload type is added (e.g., ML training, large-scale data processing).
• Review trigger: If any single optimization causes a regression in availability (SLOs breached) or developer productivity.
• Monitoring: Monthly cloud cost reports by category. Quarterly developer experience survey. SLO dashboards for all services.

Consequences

Positive: - Estimated 50-70% cloud cost reduction - Dramatically simpler operational model (~40 repos, 1 cluster, 6 databases, unified observability) - Faster onboarding for new team members (less to learn) - Better incident response (alerting, tracing, SLOs exist) - Easier to add new brands (namespace + values file, not entire cluster stack)

Negative: - Large coordinated effort across infrastructure, application, and CI/CD - Shared infrastructure increases blast radius (mitigated by regional HA, namespace isolation) - Migration period has dual-state complexity (old + new running in parallel) - Some optimizations depend on others (database consolidation depends on service consolidation)

Mitigated by: Phased rollout following dependency order (ADR-001 services → ADR-005 databases → ADR-004 cluster → ADR-006 scale-to-zero). Each phase is independently valuable and reversible.

Execution Order

graph TD
    A[Phase 1: Dead Code Cleanup<br/>Archive repos, remove dead FE code] --> B[Phase 2: Service Consolidation<br/>ADR-001: 35 → 18 services]
    B --> C[Phase 3: Database Consolidation<br/>ADR-005: 35 → 6 databases]
    C --> D[Phase 4: Cluster Consolidation<br/>ADR-004: 4 → 1 cluster]
    D --> E[Phase 5: Compute Optimization<br/>ADR-006: Knative + scale-to-zero]

    F[Parallel: Observability<br/>Alerting, tracing, SLOs] --> D
    G[Parallel: CI/CD Simplification<br/>Enforce security gates] --> B
    H[Parallel: CastAI Audit<br/>Reduce on-demand overrides] --> D

Decision date: 2026-01-31 Review by: 2026-07-31