Architecture Overview
System Architecture
Component Overview
Cairn API
Express 4 + TypeScript + TypeORM + Mongoose — REST backend handling auth, CRUD, secrets, deployments, SDK endpoints, and audit logging. Connects to PostgreSQL for relational data and MongoDB for time-series/logs.
Portal
Angular 19 + Tailwind + Standalone Components — Web UI providing dashboards, project management, secret editor, and OAuth consent screens. Thin client with no direct data store.
PostgreSQL 16
Primary data store — Users, projects, secrets, roles, deployments, and audit logs. Schema managed by versioned TypeORM migrations, applied by a Job before each API rollout. Persistent volume.
MongoDB 7.x
Time-series & unstructured data — Logs, API metrics, bug reports, SDK events, and raw metrics. Non-blocking Mongoose connection. Persistent volume.
Redis 7
In-memory cache + pub/sub — Permission cache (60s TTL), session store, pub/sub for real-time events. Ephemeral with optional AOF persistence.
MinIO / S3
Object storage — File uploads, database backups, bug report screenshots, and SDK artifacts. S3-compatible API. Persistent volume.
API Server Structure
| Directory | Contents |
|---|---|
server.ts |
Bootstrap — init DB, seed, mount routes, listen |
config/ |
Database (TypeORM), Mongoose, connections (pg/Redis/Mongo), permissions (5 preset roles), K8s client wrappers, init/shutdown |
entities/ |
18 TypeORM entities — User, Role, Project, Environment, Deployment, Secret, SecretVersion, ServiceRegistration, SdkCredential, AuditLog, ClickupTaskLink, DbConnection, DbBackup, File, StorageProvider, Alert, SmtpConfig, ProjectConfig |
routes/ |
18 route modules — auth, sdk, secrets, deployments, projects, webhooks, cicd, metrics, bug-reports, db-provision, db-connections, storage, alerts, config, bootstrap, audit-logs, settings |
lib/ |
10 service modules — secrets-encryption (AES-256-GCM), gitlab, clickup, k8s, lokilog, database-service, preview, preview-decay (72h TTL), storage-service, smtp-service |
middleware/ |
JWT verify, RBAC, permission cache (60s TTL), SDK token auth |
schemas/ |
Mongoose schemas — Log, ApiMetric, MetricsRaw, BugReport, ErrorDoc, SdkEvent, FeatureFlag, MetricsHourly |
Multi-SDK Architecture
Four SDKs sit inside customer applications. Each auto-registers with the Cairn API on startup, then talks to a fixed set of SDK endpoints for heartbeat, log shipping, metrics, bug reports, and config pull.
Which SDK uses which endpoint:
| Endpoint | Node | Python | React | Angular |
|---|---|---|---|---|
POST /api/sdk/register |
Yes | Yes | Yes | Yes |
POST /api/sdk/heartbeat |
Yes | Yes | Yes | Yes |
POST /api/sdk/logs |
Yes | Yes | — | — |
POST /api/sdk/api-metrics |
Yes | — | Yes | Yes |
POST /api/sdk/bug-report |
Yes | — | Yes | Yes |
GET /api/sdk/config |
Yes | Yes | Yes | Yes |
GET /api/sdk/db-credentials |
Yes | Yes | — | — |
Node.js SDK — @mpratyush54/sdk-node
Auth: sdk-{projectId}:{secret} or sdk_live_{uuid}. Key feature: PlatformClient.init() auto-registers, sends heartbeats, provides Express metrics middleware, captures console logs, Winston/Pino transports, and database managers (pg, mongo, redis).
Python SDK — platform-sdk
Auth: SDK token. Key feature: PlatformClient() provides registration, metrics reporting, logging, and config fetching.
React SDK — @mpratyush54/sdk-react
Auth: SDK token. Key feature: <PlatformProvider> wraps your app with hook-based API access, <ErrorBoundary> for crash interception, <BugReporterWidget>, usePlatform(), and useBugReporter() hooks.
Angular SDK — @mpratyush54/sdk-angular
Auth: SDK token. Key feature: PlatformModule provides an HTTP interceptor for metrics, ErrorHandler for global error capture, and BugReporterComponent for user feedback.
External Integrations
GitLab / GitHub CI/CD
- Webhook receiver:
POST /api/webhooks/gitlab/POST /api/webhooks/github - Validates
X-Gitlab-Token/X-Hub-Signature, extracts branch, commit SHA, project - Creates/updates preview environments per branch, deploys to k3s
- Posts ClickUp comments with preview URLs
- Pipeline trigger:
triggerPipeline(projectId, branch)fires GitLab pipeline via API token
ClickUp
- Bug report → task: When an SDK submits a bug report, if the project has
clickupListId, a task is created automatically - Preview env notification:
formatPreviewComment()generates formatted comment with branch, URL, and expiry - Task extraction:
extractTaskId(branch)parsesCU-12345from branch name
Secrets
- Built-in encrypted secrets store: project secrets are stored in Cairn's own Secret entity (AES-256-GCM) and managed from the Secrets page or the secrets API — no external secret manager is required
fetchSecrets(projectId, environment)reads them and returns a plaintext map- Used by SDK config endpoint (
GET /api/sdk/config) and DB credentials endpoint (GET /api/sdk/db-credentials)
Key Design Decisions
#1 Password sign-in with optional two-factor
POST /api/auth/login takes an email or username and a password (bcrypt hashes). Accounts with two-factor enabled get an MFA challenge before a session is issued, and admins can require two-factor for everyone. The installer creates the first admin account; there are no demo users (leftover demo users are removed at start-up). GitHub and GitLab sign-in can be added under Settings.
#2 JWT-based Sessions
Tokens are signed with JWT_SECRET, which the installer generates. The API refuses to start if it is missing, shorter than 32 characters, or a known default. Expiry: 24 hours for login JWT, 1 hour for OIDC access tokens. Payload: { id, email, name, role } — no password hash needed. The expressAuthenticate middleware verifies the JWT on every protected route and attaches req.user.
#3 RBAC with Cached Permissions
ROLE_PRESETS defines 5 roles (admin, devops, tech_lead, developer, viewer). An in-memory cache (Map<userId, { permissions, expiresAt }>) with 60s TTL avoids Redis latency. clearPermissionCache(userId?) is called on role mutation. requirePermission(...permissions) resolves user permissions and checks all required perms.
#4 Dual-Database Strategy
PostgreSQL (TypeORM) handles ACID-compliant relational data — users, projects, secrets, deployments, roles, audit logs. MongoDB (Mongoose) handles high-volume write throughput for time-series data — logs, API metrics, raw metrics, bug reports, SDK events. PostgreSQL schema changes ship as versioned TypeORM migrations; MongoDB connects non-blocking.
#5 In-Memory Permission Cache over Redis
Permissions are cached in a process-local Map rather than Redis to avoid network latency. The cache is cleared on role mutation; stale cache is tolerated for a maximum of 60s. Suitable for single-replica API deployment (horizontal scaling would need a distributed cache).
#6 SDK-First Auto-Provisioning
SDK init() triggers auto-registration which creates K8s Deployment, Service, Ingress, and ArgoCD Application resources. Databases are auto-provisioned via provisionPostgresDb. Preview environments are created per branch with a 72h TTL decay scheduler.