Microservices and Components

Microservices and Components

SecuSiem is architected as 5 Python/FastAPI microservices deployed on Kubernetes, with an infrastructure layer comprising PostgreSQL, Redis, Grafana Loki, and Grafana OSS.

Data Flow Diagram

Log Sources (Apps, SDK, Syslog)
        |
        v
+-------------------+
|  ingestion-api    |  Bearer token authentication
|  (FastAPI)        |  Redis rate limiting
+--------+----------+  Loki stream transformation
         |
         v
+-------------------+
|  Grafana Loki 3.0 |  Multi-tenant (X-Scope-OrgID)
|  (SingleBinary)   |  Filesystem/S3 storage
+--------+----------+
         |
    +----+----+
    |         |
    v         v
+--------+ +----------+
| claude | | Grafana  |
| analyzer| | OSS      |
+--------+ +----------+
    |         Dashboards, Explore, Alerting
    v
+-------------------+
| tenant-manager    |  Tenant lifecycle management
| billing-service   |  Stripe billing
| report-generator  |  PDF reports
+-------------------+
         |
         v
+-------------------+
| PostgreSQL 16     |  6 tables: tenants, api_keys,
| Redis 7.2         |  usage_records, alerts,
+-------------------+  detection_rules, audit_logs

The 5 Microservices

1. tenant-manager

Main file: services/tenant-manager/main.py Responsibility: Complete tenant lifecycle management

• 15+ REST endpoints
• 6 SQLAlchemy data models (Tenant, APIKey, UsageRecord, Alert, DetectionRule, AuditLog)
• 5 external managers (S3Manager, LokiManager, GrafanaManager, AuthentikManager, StripeManager)
• Auto-provisioning: S3 bucket, Loki org, Grafana org, Authentik group, Stripe subscription
• API key generation with bcrypt hashing
• Pydantic validation for all requests/responses
• Request timing middleware with structured logging
• CORS middleware for cross-origin requests

2. ingestion-api

Main file: services/ingestion-api/main.py Responsibility: Log reception and forwarding to Loki

• Loki-compatible endpoint: POST /loki/api/v1/push
• Bearer token authentication (API key with sk_live_ prefix)
• Forward to Loki with X-Scope-OrgID header for multi-tenant isolation
• Readiness check with Loki connectivity verification
• Uses httpx.AsyncClient for asynchronous calls
• Pydantic models for log stream and push request validation

3. claude-analyzer

Main file: services/claude-analyzer/main.py Responsibility: AI-powered security log analysis

• Anthropic Claude API integration (Sonnet 4.5 for quick analysis, Opus 4.5 for deep analysis)
• 10 pre-configured regex detection rules with MITRE ATT&CK mapping
• 2-step analysis: regex preprocessing then Claude analysis
• Confidence scoring and false positive detection
• Periodic background analysis task (configurable, default every 300 seconds)

4. billing-service

Main file: services/billing-service/main.py Responsibility: Billing and usage tracking

• Stripe webhook handler for subscription events
• Per-tenant usage calculation endpoint
• Stripe SDK integration
• Health and readiness endpoints

5. report-generator

Main file: services/report-generator/main.py Responsibility: Security report generation

• Report types: daily, weekly, monthly, custom
• PDF generation pipeline (planned: ReportLab + Matplotlib)
• Per-tenant report generation with download URL
• Health and readiness endpoints

Infrastructure Components