Operations

Scalability & Limits

DB Audit is built to handle enterprise-scale deployments. Understand the limits, optimize for performance, and scale with confidence.

Performance at Scale

1M+

events/second

Per collector cluster

From event to alert

<100

P99 for recent data

10:1

compression

Average compression ratio

Plan Limits

Each plan has different limits based on your organization's needs. Enterprise plans offer unlimited capacity.

Limit	Starter	Professional	Enterprise
Databases	5	25	Unlimited
Events	10M/month	100M/month	Unlimited
Retention	30 days	1 year	Custom
Collectors	1	5	Unlimited
Users	5	25	Unlimited
API Calls	10K/day	100K/day	Unlimited

API Rate Limits

API endpoints have rate limits to ensure fair usage and system stability. Limits are applied per API key.

Endpoint	Rate Limit	Burst
`GET /v1/events`	1,000/min	100
`POST /v1/events/search`	500/min	50
`POST /v1/events/export`	10/hour	2
`GET /v1/events/stream`	10 concurrent	-
`* (other endpoints)`	1,000/min	100

Rate Limit Headers

# API rate limit headers
HTTP/1.1 200 OK
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 847
X-RateLimit-Reset: 1705330200

# Rate limit exceeded response
HTTP/1.1 429 Too Many Requests
{
  "error": {
    "code": "rate_limit_exceeded",
    "message": "API rate limit exceeded. Retry after 60 seconds.",
    "retry_after": 60
  }
}

Scaling Strategies

Horizontal Collector Scaling

Deploy multiple collectors to distribute load across high-traffic databases.

One collector can monitor up to 50 databases
Deploy collectors close to databases for lower latency
Use dedicated collectors for high-volume databases (>10K queries/sec)

Event Sampling

For extremely high-volume environments, enable intelligent sampling to reduce data volume while maintaining security visibility.

Sample by query pattern to deduplicate repetitive queries
Always capture 100% of security events regardless of sampling
Adjust sampling rates per database based on criticality

Query Optimization

Structure your audit queries for optimal performance at scale.

Use time-bounded queries to leverage hot storage tier
Filter by database_id before other fields for faster results
Use aggregations instead of fetching raw events for dashboards

Configuration Examples

Auto-Scaling Collectors (Kubernetes)

Configure horizontal pod autoscaling to automatically scale collectors based on load.

# Kubernetes HorizontalPodAutoscaler for collectors
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: dbaudit-collector-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: dbaudit-collector
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

Event Sampling Configuration

Configure intelligent sampling to reduce event volume while maintaining security coverage.

# config.yaml - Event sampling configuration
collector:
  sampling:
    enabled: true
    default_rate: 1.0  # 100% by default

    # Sample by query pattern
    patterns:
      - pattern: "SELECT .* FROM health_check"
        rate: 0.01  # 1% sampling for health checks

      - pattern: "SELECT .* FROM sessions WHERE"
        rate: 0.1   # 10% sampling for session queries

    # Never sample security events
    never_sample:
      - event_type: "security.*"
      - severity: ["critical", "high"]
      - policy_match: true

    # Per-database overrides
    databases:
      analytics_dw:
        rate: 0.05  # 5% for high-volume analytics DB

Batch API Operations

Use batch endpoints to reduce API calls and improve efficiency.

# Batch event retrieval for efficiency
POST /v1/events/batch
{
  "requests": [
    {
      "id": "req_1",
      "filters": {
        "database_id": "db_prod_users",
        "time_range": {"last": "1h"}
      }
    },
    {
      "id": "req_2",
      "filters": {
        "database_id": "db_prod_orders",
        "time_range": {"last": "1h"}
      }
    }
  ]
}

# Single response with all results
{
  "responses": [
    {"id": "req_1", "events": [...], "total": 1542},
    {"id": "req_2", "events": [...], "total": 3891}
  ]
}

Best Practices for Scale

Use time-bounded queries

Always specify time ranges to leverage hot storage tier.

Implement exponential backoff

Handle 429 responses gracefully with retry logic.

Use webhooks for real-time

Prefer webhooks over polling for real-time alerts.

Cache aggregated data

Cache dashboard metrics to reduce API load.

Filter at the source

Use collector-level filtering to reduce network traffic.

Monitor collector health

Set up alerts for collector lag and buffer overflow.