dotb_ai-platform/stack.md

Technology Stack

This document records technology decisions, default candidates, deferred options, and trade-offs. It does not override the approved Lead Scoring Customer Pilot specification or released executable contracts.

Project Context and Boundaries

• Purpose: Build the first Python-based Lead Scoring service in AI Platform for a controlled Customer Pilot.
• Shape: Workspace containing specifications, documentation, and the initialized baseline for the Lead Scoring service (src/lead_scoring/).
• Material constraints:
  • DotB EMS remains the CRM source of truth.
  • Omni owns EMS integration, identity resolution, conversation enrichment, Lead Scoring configuration, and HITL UI.
  • Lead Scoring owns scoring logic, versioned configuration, scoring-run history, and domain events.
  • AIHub owns model access, V-Key validation, quota, credit, and LLM audit.
  • Global runtimes and the shared mcpls configuration remain outside project bootstrap scope.

Active Decisions

Service runtime and HTTP framework

• Status: Implemented baseline in src/lead_scoring/.
• Primary decision owner: Lead Scoring service.
• Technology: Python 3.13, uv, FastAPI, and Pydantic.
• Rationale: The service is Python-first, Pydantic provides executable validation, and FastAPI derives the API contract from the same models.
• Baseline implementation: src/lead_scoring/ package, GET /health endpoint, async lifespan managing database engine and session factory.

API contract and generated client

• Status: Implemented baseline CLI for OpenAPI generation.
• Primary decision owner: Lead Scoring API contract.
• Technology: Pydantic models as the API source of truth, FastAPI-generated OpenAPI, CLI generator with --check stale detection.
• Rationale: One executable backend contract avoids handwritten DTO and frontend client drift.
• Constraints: Generated artifacts (openapi.json) are ignored derived artifacts produced deterministically by uv run generate-openapi and verified for drift via --check. Never edit manually.

Shared executable Contract Pack

• Status: Selected pre-release baseline for the Customer Pilot.
• Primary decision owner: AI Platform, with DotB-Omni review and consumer validation.
• Technology: Versioned committed directory at contracts/lead-scoring-contract-pack/v1/, generated from Pydantic models, OpenAPI and shared JSON fixtures.
• Rationale: One pinned artifact corpus lets provider and consumer repositories validate the same exact wire shapes without maintaining competing payload definitions.
• Versioning: Consumers pin contract_pack_version together with the AI Platform Git commit. Additive changes preserve the API major and existing required fields/enums; breaking changes publish a new major directory and retain the old pack.
• Verification: uv run generate-contract-pack --validate-fixtures validates positive and negative provider fixtures, while uv run generate-contract-pack --check detects missing, stale or unexpected artifacts.

Persistence engine and migration tooling

• Status: Adopted and implemented baseline.
• Primary decision owner: Lead Scoring persistence and delivery boundary.
• Technology: PostgreSQL 16+, SQLAlchemy 2 async (asyncpg), Alembic async migration lifecycle.
• Rationale: Production requires PostgreSQL-native features (transactional outbox, jsonb, concurrency safety). SQLite fallback is strictly prohibited to prevent dialect divergence.
• Verification: Database integration tests execute real Alembic upgrade head and downgrade base lifecycle against disposable PostgreSQL 16 containers via Docker testcontainers. SQLite and ambient database fallbacks are strictly prohibited.

Verification and repository command surface

• Status: Implemented baseline command surface.
• Primary decision owner: Root-level generation and verification workflow.
• Selected baseline: uv lockfile, Ruff, Pyright, ty for fast feedback, pytest (with pytest-asyncio and testcontainers[postgres]).
• Rationale: A small command surface serves local development, agents, hooks, and CI without adding a task orchestrator.
• Constraints: Pyright remains the authoritative type-checking reference. ty is non-authoritative fast feedback.

AI integration and credential boundary

• Status: Adopted by the approved specification.
• Primary decision owner: Lead Scoring and AIHub service boundary.
• Technology: Bounded fixed-pass scoring pipelines; caller-provided X-Tenant-VKey held in memory by Lead Scoring and forwarded to AIHub as a Bearer token.
• Rationale: AIHub remains the single owner of model access and quota while Lead Scoring avoids persistent raw V-Key storage and an extra resolver service.
• Constraints: Raw V-Keys must not enter databases, queues, caches, logs, crash dumps, or browser code. Autonomous tools and unbounded agent loops are outside the pilot.

Pilot durability and event delivery

• Status: Selected pattern; persistence baseline adopted with PostgreSQL.
• Primary decision owner: Lead Scoring persistence and delivery boundary.
• Technology: Durable scoring runs plus transactional outbox and registered webhook delivery on PostgreSQL; no dedicated message broker for the pilot.
• Rationale: This preserves run durability and retryable delivery without adding Kafka, RabbitMQ, or another operational subsystem before the workload requires one.
• Open work: Implement retry worker, outbox table schema, and webhook delivery on the adopted PostgreSQL persistence baseline.

Pilot scoring execution topology

• Status: Adopted and implemented for rule-only execution.
• Primary decision owner: Lead Scoring runtime lifecycle.
• Technology: In-process asyncio tasks with PostgreSQL conditional claims and expiring run leases.
• Rationale: This is the smallest topology that keeps the raw tenant V-Key in the live attempt only, prevents duplicate claims, and lets database recovery terminalize orphaned work without adding a broker.
• Constraints: A process or task loss cannot resume the attempt because the V-Key is not durable. Recovery commits worker_interrupted with retryable=true and a terminal outbox event; retry is a new admission action.

Frontend toolchain

• Status: Selected for the internal Dev Console; not implemented.
• Primary decision owner: AI Platform internal console, with Omni owning customer-facing configuration and HITL UI.
• Technology: Svelte 5, TypeScript, Bun, Vite, Vitest, @sveltejs/vite-plugin-svelte, svelte-check, @sveltejs/mcp, ESLint, and Prettier Svelte tooling.
• Rationale: This provides a small client for generated OpenAPI types without adding SvelteKit before routing, SSR, or deployment requires it.
• Constraints: The console must not duplicate Omni business UI. It uses a backend-only internal test V-Key separate from production Omni tenant credentials; raw V-Key never reaches browser code. Add a Svelte language server only if mcpls must serve .svelte files semantically.

Pilot trigger boundary

• Status: Adopted by the approved specification.
• Production initiation: Manual Omni action and Captain create_and_score_ems_lead tool invocation.
• Internal initiation: AI Platform Dev Console using a separate backend-only test V-Key.
• Deferred: Generic automatic input-event triggers such as message-created, conversation-resolved, or EMS lifecycle events.
• Constraint: Completion events from Lead Scoring to Omni remain required and are distinct from deferred input triggers.

Evaluated Options

• TypeScript 7 native compiler and LSP: candidate - Run project-locally as tsc --lsp --stdio or tsgo --lsp --stdio. Do not change global TypeScript, Node, or mcpls defaults before a project-local pilot.
• typescript-language-server: current - Existing mcpls backend for TypeScript and JavaScript. Its push-only diagnostics behavior does not satisfy the current mcpls pull-diagnostics path; upgrading the wrapper alone is not the fix.
• svelte-language-server: deferred - Add only if mcpls must provide semantic LSP support for .svelte.
• SvelteKit: deferred - Add only when production routing, SSR, or deployment requires it.
• oasdiff: deferred - Adopt as a blocking compatibility check after the first reviewed and versioned OpenAPI baseline exists.
• Schemathesis: deferred - Adopt when the real ASGI app, declared error responses, deterministic authentication, and external-service overrides exist.
• Dedicated broker such as Kafka or RabbitMQ: deferred - Transactional outbox and registered webhooks are sufficient for the Customer Pilot boundary.
• Monorepo orchestrator, second formatter or linter, generic agent framework, or custom build system: rejected - No current requirement justifies the maintenance and duplicate source-of-truth cost.

Unresolved Decisions

• Production deployment and service-to-service security: Pilot caller authentication is the approved per-app opaque Bearer service-token pattern; define hosting, network exposure, per-environment secret delivery, rotation/revocation, and operational ownership before Customer Pilot deployment.
• TypeScript 7 adoption: Decide whether TS7 is the project compiler only or also the mcpls language-server backend after a project-local pilot verifies build, typecheck, diagnostics, and compatibility with the selected Svelte tooling.