Architecture Overview

System Architecture

graph TB subgraph ControlPlane["Control Plane"] Browser["Browser/Client"] Ingress["Nginx Ingress<br/>SSL Termination"] Portal["Portal<br/>Angular 19 :80"] API["Cairn API<br/>Express/TS :3000"] ArgoCD["ArgoCD<br/>GitOps :443"] end subgraph DataPlane["Data Plane"] PG[("PostgreSQL<br/>entities")] Mongo[("MongoDB<br/>logs/metrics")] Redis[("Redis<br/>cache/pubsub")] MinIO[("MinIO/S3<br/>file/backups")] end subgraph UserApps["User Applications"] NodeSDK["Node.js SDK"] PySDK["Python SDK"] ReactSDK["React SDK"] AngularSDK["Angular SDK"] end Browser --> Ingress Ingress --> Portal Ingress --> API Ingress --> ArgoCD API --> PG API --> Mongo API --> Redis API --> MinIO API -.- NodeSDK API -.- PySDK API -.- ReactSDK API -.- AngularSDK

Component Overview

Cairn API

Express 4 + TypeScript + TypeORM + Mongoose — REST backend handling auth, CRUD, secrets, deployments, SDK endpoints, and audit logging. Connects to PostgreSQL for relational data and MongoDB for time-series/logs.

Portal

Angular 19 + Tailwind + Standalone Components — Web UI providing dashboards, project management, secret editor, and OAuth consent screens. Thin client with no direct data store.

PostgreSQL 16

Primary data store — Users, projects, secrets, roles, deployments, and audit logs. Schema managed by versioned TypeORM migrations, applied by a Job before each API rollout. Persistent volume.

MongoDB 7.x

Time-series & unstructured data — Logs, API metrics, bug reports, SDK events, and raw metrics. Non-blocking Mongoose connection. Persistent volume.

Redis 7

In-memory cache + pub/sub — Permission cache (60s TTL), session store, pub/sub for real-time events. Ephemeral with optional AOF persistence.

MinIO / S3

Object storage — File uploads, database backups, bug report screenshots, and SDK artifacts. S3-compatible API. Persistent volume.

API Server Structure

graph LR subgraph src["src/"] Server["server.ts"] Config["config/"] Entities["entities/<br/>18 TypeORM entities"] Routes["routes/<br/>18 route modules"] Lib["lib/<br/>10 service modules"] Middleware["middleware/<br/>JWT + RBAC"] Schemas["schemas/<br/>Mongoose schemas"] end Server --> Config Server --> Entities Server --> Routes Server --> Lib Routes --> Middleware Lib --> Middleware Config --> Entities Config --> Schemas
Directory Contents
server.ts Bootstrap — init DB, seed, mount routes, listen
config/ Database (TypeORM), Mongoose, connections (pg/Redis/Mongo), permissions (5 preset roles), K8s client wrappers, init/shutdown
entities/ 18 TypeORM entities — User, Role, Project, Environment, Deployment, Secret, SecretVersion, ServiceRegistration, SdkCredential, AuditLog, ClickupTaskLink, DbConnection, DbBackup, File, StorageProvider, Alert, SmtpConfig, ProjectConfig
routes/ 18 route modules — auth, sdk, secrets, deployments, projects, webhooks, cicd, metrics, bug-reports, db-provision, db-connections, storage, alerts, config, bootstrap, audit-logs, settings
lib/ 10 service modules — secrets-encryption (AES-256-GCM), gitlab, clickup, k8s, lokilog, database-service, preview, preview-decay (72h TTL), storage-service, smtp-service
middleware/ JWT verify, RBAC, permission cache (60s TTL), SDK token auth
schemas/ Mongoose schemas — Log, ApiMetric, MetricsRaw, BugReport, ErrorDoc, SdkEvent, FeatureFlag, MetricsHourly

Multi-SDK Architecture

Four SDKs sit inside customer applications. Each auto-registers with the Cairn API on startup, then talks to a fixed set of SDK endpoints for heartbeat, log shipping, metrics, bug reports, and config pull.

graph LR subgraph Apps["Customer applications"] direction TB NodeApp["Node.js<br/>@mpratyush54/sdk-node"] PyApp["Python<br/>platform-sdk"] ReactApp["React<br/>@mpratyush54/sdk-react"] AngularApp["Angular<br/>@mpratyush54/sdk-angular"] end subgraph API["Cairn API — /api/sdk/*"] direction TB Register["POST /register"] Heartbeat["POST /heartbeat"] Logs["POST /logs"] Metrics["POST /api-metrics"] Bug["POST /bug-report"] Cfg["GET /config"] DBC["GET /db-credentials"] end subgraph Data["Data plane"] direction TB PG[("PostgreSQL<br/>registrations, config")] Mongo[("MongoDB<br/>logs, metrics, bugs")] Redis[("Redis<br/>cache, heartbeats")] end Apps ==>|"SDK token"| API Register --> PG Cfg --> PG DBC --> PG Heartbeat--> Redis Logs --> Mongo Metrics --> Mongo Bug --> Mongo

Which SDK uses which endpoint:

Endpoint Node Python React Angular
POST /api/sdk/register Yes Yes Yes Yes
POST /api/sdk/heartbeat Yes Yes Yes Yes
POST /api/sdk/logs Yes Yes — —
POST /api/sdk/api-metrics Yes — Yes Yes
POST /api/sdk/bug-report Yes — Yes Yes
GET /api/sdk/config Yes Yes Yes Yes
GET /api/sdk/db-credentials Yes Yes — —

Node.js SDK — @mpratyush54/sdk-node

Auth: sdk-{projectId}:{secret} or sdk_live_{uuid}. Key feature: PlatformClient.init() auto-registers, sends heartbeats, provides Express metrics middleware, captures console logs, Winston/Pino transports, and database managers (pg, mongo, redis).

Python SDK — platform-sdk

Auth: SDK token. Key feature: PlatformClient() provides registration, metrics reporting, logging, and config fetching.

React SDK — @mpratyush54/sdk-react

Auth: SDK token. Key feature: <PlatformProvider> wraps your app with hook-based API access, <ErrorBoundary> for crash interception, <BugReporterWidget>, usePlatform(), and useBugReporter() hooks.

Angular SDK — @mpratyush54/sdk-angular

Auth: SDK token. Key feature: PlatformModule provides an HTTP interceptor for metrics, ErrorHandler for global error capture, and BugReporterComponent for user feedback.

External Integrations

graph LR API["Cairn API"] GitLab["GitLab CI/CD"] GitHub["GitHub Actions"] ClickUp["ClickUp Tasks"] API -- "Webhooks → cicd.ts" --> GitLab API -- "Webhooks → cicd.ts" --> GitHub API -- "postComment()" --> ClickUp GitLab -- "triggerPipeline()" --> API GitHub -- "triggerPipeline()" --> API

GitLab / GitHub CI/CD

  • Webhook receiver: POST /api/webhooks/gitlab / POST /api/webhooks/github
  • Validates X-Gitlab-Token / X-Hub-Signature, extracts branch, commit SHA, project
  • Creates/updates preview environments per branch, deploys to k3s
  • Posts ClickUp comments with preview URLs
  • Pipeline trigger: triggerPipeline(projectId, branch) fires GitLab pipeline via API token

ClickUp

  • Bug report → task: When an SDK submits a bug report, if the project has clickupListId, a task is created automatically
  • Preview env notification: formatPreviewComment() generates formatted comment with branch, URL, and expiry
  • Task extraction: extractTaskId(branch) parses CU-12345 from branch name

Secrets

  • Built-in encrypted secrets store: project secrets are stored in Cairn's own Secret entity (AES-256-GCM) and managed from the Secrets page or the secrets API — no external secret manager is required
  • fetchSecrets(projectId, environment) reads them and returns a plaintext map
  • Used by SDK config endpoint (GET /api/sdk/config) and DB credentials endpoint (GET /api/sdk/db-credentials)

Key Design Decisions

#1 Password sign-in with optional two-factor

POST /api/auth/login takes an email or username and a password (bcrypt hashes). Accounts with two-factor enabled get an MFA challenge before a session is issued, and admins can require two-factor for everyone. The installer creates the first admin account; there are no demo users (leftover demo users are removed at start-up). GitHub and GitLab sign-in can be added under Settings.

#2 JWT-based Sessions

Tokens are signed with JWT_SECRET, which the installer generates. The API refuses to start if it is missing, shorter than 32 characters, or a known default. Expiry: 24 hours for login JWT, 1 hour for OIDC access tokens. Payload: { id, email, name, role } — no password hash needed. The expressAuthenticate middleware verifies the JWT on every protected route and attaches req.user.

#3 RBAC with Cached Permissions

ROLE_PRESETS defines 5 roles (admin, devops, tech_lead, developer, viewer). An in-memory cache (Map<userId, { permissions, expiresAt }>) with 60s TTL avoids Redis latency. clearPermissionCache(userId?) is called on role mutation. requirePermission(...permissions) resolves user permissions and checks all required perms.

#4 Dual-Database Strategy

PostgreSQL (TypeORM) handles ACID-compliant relational data — users, projects, secrets, deployments, roles, audit logs. MongoDB (Mongoose) handles high-volume write throughput for time-series data — logs, API metrics, raw metrics, bug reports, SDK events. PostgreSQL schema changes ship as versioned TypeORM migrations; MongoDB connects non-blocking.

#5 In-Memory Permission Cache over Redis

Permissions are cached in a process-local Map rather than Redis to avoid network latency. The cache is cleared on role mutation; stale cache is tolerated for a maximum of 60s. Suitable for single-replica API deployment (horizontal scaling would need a distributed cache).

#6 SDK-First Auto-Provisioning

SDK init() triggers auto-registration which creates K8s Deployment, Service, Ingress, and ArgoCD Application resources. Databases are auto-provisioned via provisionPostgresDb. Preview environments are created per branch with a 72h TTL decay scheduler.