Network Topology

Overall Network Architecture

graph TB INTERNET([INTERNET]) IP["YOUR_SERVER_IP<br/>(Public IP)"] FW["Host Firewall<br/>Open: 80, 443<br/>Closed: all others"] INGRESS["ingress-nginx Controller<br/>(hostNetwork: true)<br/>80 to redirect / 443 to TLS"] CAPS["caps<br/>(portal + api)"] MON["monitoring<br/>(grafana, prometheus)"] OPS["argocd / portainer / etc."] PG[("postgres-service:5432")] MG[("mongo-service:27017")] RD[("redis-service:6379")] MN[("minio-service:9000")] LK[("loki-service:3100")] INTERNET --> IP --> FW --> INGRESS INGRESS --> CAPS INGRESS --> MON INGRESS --> OPS CAPS --> PG CAPS --> MG CAPS --> RD CAPS --> MN CAPS --> LK

Ingress Controller: nginx-ingress on Host Network

# Helm values for ingress-nginx
controller:
  hostNetwork: true
  kind: DaemonSet
  service:
    type: ClusterIP  # or LoadBalancer
  publishService:
    enabled: false
  config:
    use-forwarded-headers: "true"
    compute-full-forwarded-for: "true"
    proxy-buffer-size: "128k"
    ssl-redirect: "true"
    force-ssl-redirect: "false"
  • Host network mode: The nginx-ingress controller binds directly to the host's ports 80 and 443, bypassing kube-proxy overhead
  • DaemonSet: Ensures one pod per node (single-node k3s, so one pod)
  • SSL termination: All traffic arrives as HTTPS; cert-manager provisions Let's Encrypt certificates

SSL Termination via cert-manager + Let's Encrypt

sequenceDiagram participant Client participant Ingress as nginx-ingress participant CM as cert-manager participant LE as Let's Encrypt Client->>Ingress: HTTPS :443 Ingress->>Ingress: TLS handshake Ingress->>Client: HTTP (in-cluster) Note over CM: Watches Ingress annotations CM->>LE: Request cert (prod ClusterIssuer) CM->>LE: Request cert (staging ClusterIssuer) LE-->>CM: Issue certificate CM-->>Ingress: Store in Kubernetes Secret

Certificate provisioning flow:

  1. Ingress created with annotation cert-manager.io/cluster-issuer: letsencrypt-prod
  2. cert-manager creates a Certificate resource
  3. cert-manager creates an HTTP-01 challenge pod/ingress
  4. Let's Encrypt validates domain ownership via HTTP-01
  5. Certificate stored in a Kubernetes Secret referenced in ingress.spec.tls[0].secretName

Domain-specific behavior (from src/routes/sdk.ts:221):

const isRealDomain = !domain.includes('sslip.io') && !domain.match(/^\d+\.\d+\.\d+\.\d+/);
  • sslip.io and IP-based domains: no cert-manager annotation, self-signed or no TLS
  • Real domains (e.g., app.example.com): full Let's Encrypt provisioning

Subpath Routing

graph LR HOST["https://YOUR_SERVER_IP.sslip.io"] PORTAL["portal-service:80<br/>Angular SPA<br/>try_files to /index.html"] API["api-service:3000<br/>/api/*"] ARGO["argocd-server:443<br/>(ssl-passthrough)"] PORT["portainer-service:9000"] GRAF["grafana-service:3000<br/>(subpath)"] HOST -- "/" --> PORTAL HOST -- "/api/*" --> API HOST -- "/argocd/*" --> ARGO HOST -- "/portainer/*" --> PORT HOST -- "/grafana/*" --> GRAF

Ingress Rules (Complete)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: platform-ingress
  namespace: caps
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "100m"
spec:
  tls:
  - hosts:
    - YOUR_SERVER_IP.sslip.io
    secretName: caps-tls
  rules:
  - host: YOUR_SERVER_IP.sslip.io
    http:
      paths:
      # Portal (root)
      - path: /
        pathType: Prefix
        backend:
          service:
            name: portal-service
            port:
              number: 80

      # API
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 3000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argocd-ingress
  namespace: argocd
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /argocd
spec:
  rules:
  - host: YOUR_SERVER_IP.sslip.io
    http:
      paths:
      - path: /argocd
        pathType: Prefix
        backend:
          service:
            name: argocd-server
            port:
              number: 443
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: portainer-ingress
  namespace: portainer
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: YOUR_SERVER_IP.sslip.io
    http:
      paths:
      - path: /portainer
        pathType: Prefix
        backend:
          service:
            name: portainer
            port:
              number: 9000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-ingress
  namespace: monitoring
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: YOUR_SERVER_IP.sslip.io
    http:
      paths:
      - path: /grafana
        pathType: Prefix
        backend:
          service:
            name: grafana
            port:
              number: 3000

Internal DNS Resolution

graph TB CD["CoreDNS<br/>kube-system"] SVC["Service DNS<br/>postgres.postgres<br/>.svc.cluster.local"] POD["Pod DNS (statefulset)<br/>postgres-0.postgres<br/>.svc.cluster.local"] EXT["External DNS<br/>sslip.io<br/>YOUR_IP.sslip.io"] CD --> SVC CD --> POD CD --> EXT

Internal DNS naming conventions:

Service DNS Name (in-cluster) Resolves To
PostgreSQL postgres.postgres.svc.cluster.local ClusterIP 10.43.x.x
MongoDB mongodb.mongo.svc.cluster.local ClusterIP 10.43.x.x
Redis redis.redis.svc.cluster.local ClusterIP 10.43.x.x
MinIO minio.minio.svc.cluster.local ClusterIP 10.43.x.x
Loki loki.loki.svc.cluster.local ClusterIP 10.43.x.x
API api-service.caps.svc.cluster.local ClusterIP 10.43.x.x

In-cluster API configuration (src/server.ts uses env vars):

POSTGRES_HOST=postgres.postgres.svc.cluster.local
MONGODB_URI=mongodb://mongodb.mongo.svc.cluster.local:27017/platform
REDIS_HOST=redis.redis.svc.cluster.local
LOKI_URL=http://loki.loki.svc.cluster.local:3100

External DNS: sslip.io Wildcard

The server's public IP YOUR_SERVER_IP is used with the sslip.io wildcard DNS service:

graph LR W["*.YOUR_SERVER_IP.sslip.io"] -- "A record" --> IP["YOUR_SERVER_IP"]

This allows multiple subdomain-based services without managing real DNS records:

Domain Resolves To Service
YOUR_SERVER_IP.sslip.io YOUR_SERVER_IP Root — Portal
api.YOUR_SERVER_IP.sslip.io YOUR_SERVER_IP API (if separated)
*.YOUR_SERVER_IP.sslip.io YOUR_SERVER_IP Wildcard — SDK auto-created services

SDK auto-created Ingress domains (src/routes/sdk.ts:232):

const domain = project.domain || process.env.DOMAIN || 'sslip.io';
// Resulting host: {serviceName}.{domain}
// e.g. "my-app.YOUR_SERVER_IP.sslip.io"

systemd-resolved Configuration

To ensure reliable DNS resolution within the k3s cluster, systemd-resolved is configured on the host:

# /etc/systemd/resolved.conf
[Resolve]
DNS=1.1.1.1 8.8.8.8
FallbackDNS=8.8.4.4
Domains=~.
DNSSEC=allow-downgrade
DNSOverTLS=opportunistic
Cache=yes
DNSStubListener=yes

Key settings:

  • Primary DNS: Cloudflare (1.1.1.1) + Google (8.8.8.8)
  • Domains=~.: Route all queries through these DNS servers (prevents search domain leakage)
  • DNSOverTLS=opportunistic: Encrypt DNS queries when the upstream supports it
  • Cache=yes: Local DNS caching reduces latency

Troubleshooting DNS issues:

# Check resolved status
resolvectl status

# Flush cache
resolvectl flush-caches

# Test resolution
resolvectl query YOUR_SERVER_IP.sslip.io

# Check k3s CoreDNS
kubectl -n kube-system logs -l k8s-app=kube-dns --tail=50

Common DNS issues & fixes (see troubleshooting docs):

Issue Cause Fix
IPv6 DNS timeout AAAA queries to non-IPv6 networks sysctl -w net.ipv6.conf.all.disable_ipv6=1
CoreDNS loop systemd-resolved interference Set Domains=~. in resolved.conf
cert-manager timeout DNS propagation delay Check dig resolution, increase ACME timeout

Network Policies (Future Consideration)

Currently no NetworkPolicy resources are enforced. The cluster relies on:

  • Namespace isolation (logical, not network-enforced)
  • Service-based access (only explicit connections through service DNS)
  • Ingress-only external access (no NodePort or LoadBalancer exposed besides ingress)

Recommended future network policies:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-allow-ingress-only
  namespace: caps
spec:
  podSelector:
    matchLabels:
      app: api-service
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: ingress-nginx
    ports:
    - protocol: TCP
      port: 3000

Port Summary

Service Protocol Port Source Destination Purpose
HTTP TCP 80 Internet Host Redirect to 443
HTTPS TCP 443 Internet Host → Ingress All web traffic
Ingress metrics TCP 10254 Internal Ingress Prometheus scraping
PostgreSQL TCP 5432 API pod PostgreSQL pod Data store
MongoDB TCP 27017 API pod MongoDB pod Logs/metrics
Redis TCP 6379 API pod Redis pod Cache
MinIO API TCP 9000 API pod MinIO pod File storage
MinIO Console TCP 9001 Internal MinIO pod Admin UI
Loki TCP 3100 API pod Loki pod Log aggregation
Grafana TCP 3000 Ingress Grafana pod Dashboards
ArgoCD TCP 443 Ingress ArgoCD pod GitOps UI
Portainer TCP 9000 Ingress Portainer pod Container mgmt
OAuth2 Proxy TCP 4180 Ingress OAuth2 proxy SSO

See K8s Infrastructure for Helm chart versions and resource limits, and Overview for the high-level system diagram.