Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.getomni.co/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

Each connector runs as an independent service (a Docker container in self-hosted deployments, or an ECS task on AWS) and syncs data independently. The omni-connector-manager service orchestrates all connector operations, including scheduling syncs, tracking progress, and managing concurrent operations.
For instructions on adding a new connector, see the Connectors Overview.

​
Connector Manager

The connector-manager service is the central orchestrator for all connector operations. It provides:
CapabilityDescription
Sync OrchestrationTriggers syncs on-demand or on schedule
Progress TrackingReal-time sync progress via Server-Sent Events
Concurrency ControlLimits concurrent syncs (default: 10 global, 3 per type)
Stale Sync DetectionMarks hung syncs as failed after STALE_SYNC_TIMEOUT_MINUTES of inactivity
Interrupted Sync RecoveryPolls each connector’s /sync/{id} endpoint on every scheduler tick; if a connector has lost the run (typically due to a restart), re-triggers it so it resumes from the last persisted checkpoint (up to 3 attempts before failing the run)
SDK EndpointsAPI for custom connectors built with Python/TypeScript SDK

​
Configuration

Key connector-manager settings (via environment variables):
VariableDefaultDescription
MAX_CONCURRENT_SYNCS10Maximum concurrent syncs across all sources
MAX_CONCURRENT_SYNCS_PER_TYPE3Maximum concurrent syncs per connector type
SCHEDULER_POLL_INTERVAL_SECONDS60How often the scheduler checks for due syncs
STALE_SYNC_TIMEOUT_MINUTES60Timeout to mark a sync as stale/failed
See Configuration Reference for all options.

​
Document Conversion

Connectors that ingest binary files (PDFs, Office docs, images) all route their text extraction through the shared Docling service. Toggle it on or off, and pick a quality preset, from Settings → Document Conversion:
  • Fast — text-heavy docs, basic tables, no OCR
  • Balanced (default) — accurate tables, no OCR; good tradeoff for most deployments
  • Quality — OCR on, higher image scale, code/formula enrichment
Changes take effect immediately — no restart needed. See Configuration Reference → Document Conversion for environment-level tuning (CPU pinning, memory limits, concurrency).

​
Viewing Connector Status

Navigate to Settings → Integrations to see all configured connectors.
Omni Integrations
For each connector, you can see:
  • Enabled/Disabled status
  • Last sync timestamp
  • Sync progress when a sync is running (including document count)

​
Enabling and Disabling Connectors

Each connector can be enabled or disabled independently. A disabled connector:
  • Stops syncing new content
  • Retains previously indexed documents
  • Can be re-enabled at any time
To toggle a connector’s status, click Configure and change the enabled setting.

​
Sync Operations

​
Automatic Sync

Connectors automatically sync on a schedule. After the initial full sync, incremental syncs run periodically to capture new and updated content.

​
Manual Sync

To trigger a manual sync:
  1. Navigate to Settings → Integrations
  2. Select the connector
  3. Click Sync Now
You can only trigger a manual sync if no sync is currently running. If a sync is in progress, the button will be disabled.

​
Removing a Connector

To disconnect a data source:
  1. Navigate to Settings → Integrations
  2. Select the connector
  3. Click Remove
  4. Confirm the removal
When you remove a connector:
  • The connector service is stopped
  • All indexed documents from that source are deleted
  • Users will no longer see results from that source

​
Troubleshooting

​
Connector shows “Error” status

Check the connector logs. For Docker Compose deployments: 
# Check specific connector logs
docker logs omni-<connector-name>-connector

# Check connector-manager logs for orchestration issues
docker logs omni-connector-manager
For AWS deployments, check CloudWatch Logs for the connector’s ECS task. Common issues:
  • Expired or revoked credentials
  • API rate limiting
  • Network connectivity problems
  • Connector-manager unable to reach connector service

​
Sync seems stuck

If a sync has been running for an unusually long time:
  1. Check the connector-manager logs for heartbeat failures and auto-resume attempts: 
    docker logs omni-connector-manager | grep -iE "stale|timeout|lost|resume"
  2. Check the connector logs for errors
  3. If the connector restarted, connector-manager will detect the lost run on its next scheduler tick and re-trigger it from the last checkpoint (up to 3 resume attempts).
  4. Syncs that continue to make no progress are marked failed after STALE_SYNC_TIMEOUT_MINUTES of inactivity (default: 60 minutes).
  5. Restart the connector service if it appears hung rather than progressing.

​
Connector not receiving sync requests

Verify connector-manager can reach the connector:
  1. Check connector URL configuration: 
    # Ensure CONNECTOR_<TYPE>_URL is set correctly
echo $CONNECTOR_GOOGLE_URL
  2. Test connectivity from connector-manager: 
    docker exec omni-connector-manager curl http://<connector>:<port>/health
  3. Check connector-manager logs for connection errors

​
Too many concurrent syncs

If syncs are queuing or timing out:
  1. Check current sync count in connector-manager logs
  2. Adjust concurrency limits:
    • MAX_CONCURRENT_SYNCS (default: 10)
    • MAX_CONCURRENT_SYNCS_PER_TYPE (default: 3)
  3. Consider staggering sync schedules for different sources