> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getomni.co/llms.txt
> Use this file to discover all available pages before exploring further.
# Connector SDK Overview
> Build custom connectors to integrate any data source with Omni
The Omni Connector SDKs enable you to build custom connectors that integrate any data source with Omni's search and AI capabilities. Whether you're connecting an internal database, a SaaS application, or a proprietary system, the SDKs provide everything you need.
## Why Build a Custom Connector?
While Omni includes connectors for popular platforms (Google Workspace, Slack, Atlassian, HubSpot), you may need to integrate:
* **Internal systems** — Databases, wikis, or custom applications
* **Niche SaaS tools** — Industry-specific platforms not yet supported
* **Proprietary data sources** — File formats or APIs unique to your organization
* **Legacy systems** — Older systems with custom APIs
## SDK Features
Both SDKs provide the same core capabilities:
| Feature | Description |
| ------------------------ | ------------------------------------------------------ |
| **Connector Base Class** | Abstract base with sync lifecycle management |
| **Sync Context** | Utilities for emitting documents and tracking progress |
| **Content Storage** | Store document content for indexing |
| **SDK Client** | Communication with the connector-manager service |
| **HTTP Server** | Built-in server exposing standard connector endpoints |
| **Data Models** | Type-safe models for documents, events, and metadata |
## Available SDKs
Build connectors in Python with FastAPI
Build connectors in TypeScript with Express
Build connectors in Rust with Axum (`omni-connector-sdk` crate at `sdk/rust/`)
All three SDKs are functionally equivalent. Choose the language your team is most comfortable with. The Rust SDK doesn't yet have a dedicated doc page — the built-in `google`, `slack`, `atlassian`, `imap`, `nextcloud`, `fireflies`, `filesystem`, and `web` connectors are all built against it and serve as the canonical reference.
**The SDKs are not published to PyPI or npm.** All connector development happens **inside the Omni monorepo**, with the SDK consumed as a local path dependency (`{ path = "../../sdk/python" }` for Python, `"file:../../sdk/typescript"` for TypeScript). To build a connector, clone [omni](https://github.com/getomnico/omni) and add a new directory under `connectors/`. This keeps connectors and the SDK versioned together, so you never have to chase a mismatched release.
## Scaffolding with the `/build-connector` skill
The omni repository ships with a [Claude Code](https://claude.com/claude-code) skill called `build-connector` that scaffolds an entire new connector — sync logic, manifest, Dockerfile, package config, frontend wiring (icon, setup dialog, source type enum), Docker Compose service, AWS and GCP Terraform, and integration tests — following the conventions of the built-in connectors.
To use it, install Claude Code, open the `omni` repo, and run:
```
/build-connector Asana
```
(or any other service name). The skill walks through the full checklist:
1. Choose a language (Python, TypeScript, or Rust) and read the matching reference connector
2. Implement sync, manifest, and any custom actions
3. Add a database migration for the new source type
4. Add the variant to the Rust `SourceType` enum in `shared/src/models.rs`
5. Wire up the frontend (icon, setup dialog, integrations page entry)
6. Add the service to `docker/docker-compose.yml` and `.env.example`
7. Add AWS ECS and GCP Cloud Run definitions
8. Write integration tests against a real ParadeDB + Redis + connector-manager via testcontainers
The skill is not required — you can build a connector by hand by reading the existing ones — but it eliminates a lot of boilerplate and ensures nothing is missed.
## Architecture
Custom connectors integrate with Omni exclusively through HTTP communication with the connector-manager service. Connectors have **no direct database access** — all document emission, state management, and content storage is handled via the SDK client which communicates with connector-manager over HTTP.
```
┌─────────────────┐ ┌───────────────────┐ ┌─────────────┐
│ Your Connector │ ←HTTP→ │ connector-manager │ ←Queue→ │ omni-indexer│
│ (SDK-based) │ │ (orchestrator) │ │ (indexing) │
└─────────────────┘ └───────────────────┘ └─────────────┘
│ │
│ │
↓ ↓
┌─────────────────┐ ┌───────────────────┐
│ Data Source │ │ PostgreSQL │
│ (your API) │ │ (storage/queue) │
└─────────────────┘ └───────────────────┘
```
### Connector Lifecycle
1. **omni-connector-manager** triggers a sync via HTTP `POST /sync`
2. Your connector receives the request with source configuration and credentials
3. Your connector fetches data from the external source
4. Documents are emitted through the SDK, which sends them to connector-manager
5. connector-manager queues events for the indexer
6. Your connector reports completion or failure
### HTTP Endpoints
The SDK automatically exposes these endpoints:
| Endpoint | Method | Purpose |
| --------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `/health` | GET | Health check for container orchestration |
| `/manifest` | GET | Returns connector metadata and capabilities |
| `/sync` | POST | Triggers a sync operation |
| `/sync/{sync_run_id}` | GET | Reports whether a given sync run is still active in this process. connector-manager polls this to detect connector restarts and auto-resume interrupted syncs from the last checkpoint. |
| `/cancel` | POST | Cancels a running sync |
| `/action` | POST | Executes connector-specific actions |
### Document Conversion (Docling)
Binary documents (PDFs, Word, Excel, PowerPoint, images) are converted to text via the centralized Docling service rather than per-connector logic. The SDK exposes this through `ctx.content_storage.extract_text(data, mime_type, filename)` (and `extract_and_store_content(...)` when you want to extract and persist in one call). Under the hood the SDK calls `POST /sdk/extract-text` on connector-manager, which routes to Docling when the admin has enabled it and falls back to lightweight built-in extractors otherwise. All built-in connectors (IMAP, Gmail, Filesystem, Nextcloud, etc.) use this pathway — custom connectors should too.
## Quick Start Example
Here's a minimal connector implementation in Python:
```python theme={null}
from omni_connector import Connector, Document, DocumentMetadata, SyncContext
class MyConnector(Connector):
@property
def name(self) -> str:
return "my-connector"
@property
def version(self) -> str:
return "1.0.0"
@property
def sync_modes(self) -> list[str]:
return ["full"]
async def sync(
self,
source_config: dict,
credentials: dict,
state: dict | None,
ctx: SyncContext
) -> None:
# Fetch items from your data source
items = await self.fetch_items(credentials["api_key"])
for item in items:
# Store content
content_id = await ctx.content_storage.save(item["content"])
# Emit document
await ctx.emit(Document(
external_id=item["id"],
title=item["title"],
content_id=content_id,
metadata=DocumentMetadata(
url=item["url"],
created_at=item["created_at"],
),
))
await ctx.increment_scanned()
await ctx.complete()
if __name__ == "__main__":
MyConnector().serve(port=8000)
```
## Wrapping an MCP Server
All three SDKs can expose tools from a [Model Context Protocol](https://modelcontextprotocol.io) server to Omni's AI assistant. Override the `mcp_server` property on your connector to return either a `StdioMcpServer` (subprocess transport) or an `HttpMcpServer` (Streamable HTTP transport for remote MCP servers). The SDK opens a session per call, discovers the tools, and registers them with the connector-manager.
**Stdio (local subprocess):**
```python theme={null}
from omni_connector import Connector, StdioMcpServer
class MyConnector(Connector):
@property
def mcp_server(self) -> StdioMcpServer:
return StdioMcpServer(
command="my-mcp-server",
args=["stdio"],
)
def prepare_mcp_env(self, credentials: dict) -> dict[str, str]:
return {"API_TOKEN": credentials["api_key"]}
```
**Remote (Streamable HTTP):**
```python theme={null}
from omni_connector import Connector, HttpMcpServer
class MyConnector(Connector):
@property
def mcp_server(self) -> HttpMcpServer:
return HttpMcpServer(url="https://api.example.com/mcp")
def prepare_mcp_headers(self, credentials: dict) -> dict[str, str]:
return {"Authorization": f"Bearer {credentials['api_key']}"}
```
The `github` connector is a reference implementation that wraps the official GitHub MCP server over stdio. Return `None` (the default) to disable MCP support entirely for your connector.
## Example Connectors
Both SDKs include an RSS connector example that demonstrates:
* Fetching data from an external API
* Parsing and transforming content
* Emitting documents with metadata
* Implementing incremental sync
* Handling errors and cancellation
* Implementing custom actions
See the SDK-specific documentation for the complete example walkthrough.
## Deploying Your Connector
### Docker Deployment
Each connector ships its own `Dockerfile`. Copy from a reference connector in your chosen language (e.g. `connectors/notion/Dockerfile` for Python, `connectors/linear/Dockerfile` for TypeScript, `connectors/google/Dockerfile` for Rust).
Add the connector as a service in `docker/docker-compose.yml`, gated by a Compose profile so it only starts when the user opts in via `ENABLED_CONNECTORS`:
```yaml theme={null}
services:
my-connector:
build:
context: ../connectors/my-connector
profiles: ["my-connector"]
environment:
CONNECTOR_MANAGER_URL: ${CONNECTOR_MANAGER_URL}
PORT: ${MY_CONNECTOR_PORT}
networks:
- omni-network
```
Then add the port to `.env.example` and document the new profile name in the `ENABLED_CONNECTORS` comment.
### Registration with Connector Manager
You don't need to set per-connector URLs in connector-manager's environment. **Connectors auto-register their manifest with connector-manager on a 30-second heartbeat** (90s TTL) — as long as the container is reachable on the Docker network and `CONNECTOR_MANAGER_URL` is set, it will appear automatically.
## What's Next
Detailed Python SDK documentation
Detailed TypeScript SDK documentation