Data Connectors Overview
ZenSearch supports 17 data connectors across cloud storage, collaboration tools, CRM systems, databases, and web content. Connect your organization's data sources to create a unified, searchable knowledge base.
Connector Categories
Cloud Storage
Store and manage files in cloud platforms:
| Connector | Description | Auth Method |
|---|---|---|
| Amazon S3 | S3 and S3-compatible storage (MinIO) | IAM Role, Access Keys |
| Google Drive | Personal and Workspace drives | OAuth 2.0 |
| SharePoint | Microsoft SharePoint Online | OAuth 2.0 |
| Azure Blob | Azure Blob Storage containers | Connection String, SAS |
Collaboration Tools
Connect team collaboration platforms:
| Connector | Description | Auth Method |
|---|---|---|
| Confluence | Atlassian Confluence (Cloud/Server) | API Token, OAuth |
| Notion | Pages, databases, and blocks | OAuth 2.0 |
| Slack | Channels, threads, and files | OAuth 2.0 |
Development Tools
Index code and project management:
| Connector | Description | Auth Method |
|---|---|---|
| GitHub | Repositories, code, and issues | OAuth 2.0, PAT |
| Jira | Issues, comments, and worklogs | API Token, OAuth |
CRM Systems
Connect customer relationship management platforms:
| Connector | Description | Auth Method |
|---|---|---|
| Salesforce | Accounts, contacts, opportunities | OAuth 2.0 |
| HubSpot | Contacts, companies, deals | OAuth 2.0 |
| SAP | S/4HANA ERP data | Basic Auth, OAuth |
Databases
Query structured data sources:
| Connector | Description | Auth Method |
|---|---|---|
| PostgreSQL | PostgreSQL databases | Username/Password |
| MySQL | MySQL and MariaDB | Username/Password |
| ClickHouse | ClickHouse analytics DB | Username/Password |
| MS SQL Server | Microsoft SQL Server | SQL/Windows Auth |
Web Content
Crawl and index websites:
| Connector | Description | Auth Method |
|---|---|---|
| Web Crawler | Website crawling | None/Basic Auth |
Common Configuration
Basic Settings
All connectors share common configuration options:
| Setting | Description |
|---|---|
| Name | Display name for the connector |
| Collection | Target collection for documents |
| Enabled | Whether sync is active |
| Schedule | Sync frequency (if applicable) |
Authentication
OAuth Connectors
For OAuth-based connectors (Google Drive, Notion, Slack, etc.):
- Click Authorize during setup
- Sign in to the external service
- Grant requested permissions
- Return to ZenSearch automatically
API Key/Token
For API key-based connectors:
- Generate an API key in the source platform
- Enter the key in ZenSearch
- Test the connection
Username/Password
For database and basic auth connectors:
- Enter host and port
- Provide username and password
- Configure SSL if needed
- Test the connection
Adding a Connector
Step-by-Step
- Navigate to Knowledge → Data Sources
- Click Add Data Source
- Select connector type
- Configure authentication
- Set source-specific options
- Choose target collection
- Test connection
- Create connector
Configuration Wizard
The wizard guides you through:
┌─────────────────────────────────────────┐
│ Step 1: Select Connector Type │
│ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │
│ │ S3 │ │Drive│ │Slack│ │ ... │ │
│ └─────┘ └─────┘ └─────┘ └─────┘ │
├─────────────────────────────────────────┤
│ Step 2: Configure Connection │
│ [Authentication fields...] │
├─────────────────────────────────────────┤
│ Step 3: Source Options │
│ [Filters, paths, scopes...] │
├─────────────────────────────────────────┤
│ Step 4: Collection Assignment │
│ [Select or create collection] │
└─────────────────────────────────────────┘
Sync Behavior
Initial Sync
When a connector is created:
- Full content fetch from source
- Document parsing and analysis
- Embedding generation
- Index population
Incremental Sync
Subsequent syncs:
- Detect changed/new content
- Process only updates
- Remove deleted content
- Update search index
Webhook Sync (Real-time)
For supported connectors:
- Configure webhooks in source
- Receive instant update notifications
- Process changes immediately
- Near real-time search updates
Supported Sync Methods by Connector
| Connector | Scheduled | Incremental | Webhooks |
|---|---|---|---|
| S3 | Yes | Yes | Yes |
| Google Drive | Yes | Yes | Yes |
| SharePoint | Yes | Yes | Yes |
| Confluence | Yes | Yes | Yes |
| Notion | Yes | Yes | No |
| Slack | Yes | Yes | Yes |
| GitHub | Yes | Yes | Yes |
| Jira | Yes | Yes | Yes |
| Salesforce | Yes | Yes | Yes |
| HubSpot | Yes | Yes | Yes |
| SAP | Yes | Yes | No |
| PostgreSQL | Yes | No | No |
| MySQL | Yes | No | No |
| ClickHouse | Yes | No | No |
| MS SQL | Yes | No | No |
| Web Crawler | Yes | Yes | No |
Permission Sync
Supported Platforms
These connectors sync document-level permissions:
- Google Drive: File and folder sharing
- SharePoint: Site and document permissions
- Confluence: Space and page restrictions
- Salesforce: Record sharing rules
- Slack: Channel membership
Permission Modes
| Mode | Description |
|---|---|
| Strict | Only show documents user can access in source |
| Permissive | Show all documents (for internal use) |
Content Filtering
Path/Prefix Filters
For storage connectors:
Include: /documents/public/*
Exclude: /documents/archive/*
File Type Filters
Limit by file extension:
Include: .pdf, .docx, .txt, .md
Exclude: .exe, .zip
Date Filters
Sync content from specific periods:
Modified after: 2024-01-01
Created within: Last 90 days
Custom Filters
Connector-specific filtering:
- Slack: Specific channels
- Confluence: Selected spaces
- GitHub: Specific branches
- Jira: JQL filter queries
Best Practices
Security
- Use minimal required permissions
- Rotate API keys regularly
- Use OAuth when available
- Enable permission sync
Performance
- Start with smaller scopes
- Enable incremental sync
- Use webhooks when available
- Filter unnecessary content
Organization
- One connector per purpose
- Group related connectors in collections
- Use descriptive names
- Document filter settings
Troubleshooting
Connection Failed
- Verify credentials are correct
- Check network connectivity
- Ensure required permissions granted
- Review firewall/proxy settings
Sync Errors
- Check source accessibility
- Verify API rate limits
- Review error messages
- Try manual re-sync
Missing Content
- Check filter settings
- Verify permissions
- Wait for sync completion
- Review content types supported
Next Steps
Choose a connector guide for detailed setup instructions: