Skip to main content

Knowledge Management

The Knowledge page is your central hub for managing data sources, collections, and sync activities. Connect your organization's data and keep it indexed and up-to-date.

Overview​

Knowledge management in ZenSearch involves:

  • Data Sources: Connections to external platforms
  • Collections: Logical groupings of documents
  • Sync Jobs: Background processes that update content
  • Documents: Individual pieces of indexed content

Knowledge Page​

Access the Knowledge page from the left sidebar. It contains three tabs:

Data Sources Tab​

View and manage all connected data sources:

  • Source Cards: Visual representation of each connector
  • Status Indicators: Active, Paused, Error, Inactive
  • Sync Controls: Manual sync triggers
  • Options Menu: Edit, sync, delete

Collections Tab​

Organize your content into collections:

  • Collection Cards: Name, description, document count
  • Embedding Model: Which model indexes the collection
  • Statistics: Document and connector counts
  • Management: Create, edit, delete collections

Activity Tab​

Monitor sync jobs and processing:

  • Job List: All sync jobs with status
  • Progress Bars: Real-time processing progress
  • Actions: Cancel running jobs, retry failed ones
  • Details: Click to see processing breakdown

Data Sources​

Adding a Data Source​

  1. Click Add Data Source
  2. Select connector type from the wizard
  3. Configure authentication and settings
  4. Test the connection
  5. Create the connector

Connector Configuration​

Each connector has specific settings:

Common Settings​

SettingDescription
NameDisplay name for the connector
CollectionWhich collection to add documents to
EnabledWhether sync is active

Authentication Options​

Depending on the connector type:

  • OAuth: Click to authorize (Google, Notion, Slack, etc.)
  • API Key: Enter your API token
  • Basic Auth: Username and password
  • IAM Role: AWS IAM role ARN

Managing Sources​

Sync Now​

Trigger an immediate sync:

  1. Click the Options menu (three dots)
  2. Select Sync Now
  3. View progress in Activity tab

Edit Settings​

  1. Click the source card or Edit from menu
  2. Modify configuration
  3. Test connection
  4. Save changes

Pause/Resume​

  1. Open source settings
  2. Toggle the Enabled switch
  3. Paused sources won't sync

Delete Source​

  1. Click Delete from options menu
  2. Confirm deletion
  3. Documents may remain in collections

Source Status​

StatusDescription
ActiveConnected and syncing normally
SyncingCurrently running a sync job
PausedSync disabled by user
ErrorConnection or sync failure
InactiveNever synced or no content

Webhook Indicators​

Sources with webhook support show:

  • Real-time badge: Webhooks enabled
  • Last activity: Most recent webhook event
  • Activity log: View webhook history

Collections​

Creating a Collection​

  1. Go to Collections tab
  2. Click Create Collection
  3. Fill in details:
FieldDescription
NameUnique collection name
DescriptionWhat this collection contains
Embedding ModelModel for generating vectors
  1. Click Create

Collection Settings​

Edit collections to modify:

  • Name and description
  • Default embedding model
  • Associated connectors

Collection Statistics​

Each collection shows:

  • Document Count: Total indexed documents
  • Embedding Count: Vector representations
  • Connector Count: Data sources feeding this collection
  • Last Updated: Most recent sync

Deleting Collections​

warning

Deleting a collection removes all documents and embeddings. This action cannot be undone.

  1. Click Delete on the collection card
  2. Review the impact warning
  3. Confirm deletion

Sync Activity​

Understanding Jobs​

Sync jobs process content through multiple stages:

Collecting → Parsing → Analyzing → Indexing → Done

Stage Descriptions​

StageDescription
CollectingFetching files from source
ParsingExtracting text and metadata
AnalyzingBuilding document structure and creating semantic units
IndexingGenerating embeddings for search

Job Status​

StatusDescription
QueuedWaiting to start
RunningCurrently processing
CompletedFinished successfully
FailedError during processing
CancelledStopped by user

Viewing Job Details​

Click any job row to see:

  • Full job information
  • Document breakdown by stage
  • Error messages and counts
  • Processing timestamps

Cancelling Jobs​

For running jobs:

  1. Click the Cancel button
  2. Job will stop at current stage
  3. Partial results may be available

Retrying Failed Jobs​

For failed jobs:

  1. Click the Retry button
  2. New job starts from beginning
  3. Check error messages first

Document Management​

Viewing Documents​

From a data source detail page:

  1. Click Documents tab
  2. Browse indexed documents
  3. See processing status
  4. Search within source

Document Status​

StatusDescription
IndexedSuccessfully processed and searchable
PendingWaiting for processing
ProcessingCurrently being indexed
FailedError during indexing
DeletedRemoved from source

Viewing Files​

The Files tab shows raw files:

  • Original file names
  • File sizes
  • Collection dates
  • Delete options

Source Detail Page​

Click any data source to see its detail page with:

Files Tab​

Raw files from the source:

  • File listing with metadata
  • Size and collection date
  • Search and filter
  • Bulk delete

Documents Tab​

Processed documents:

  • Document titles
  • Processing status
  • View full content
  • Delete individual documents

Jobs Tab​

Sync history for this source:

  • All sync jobs
  • Status and timing
  • Error details

Settings Tab​

Full connector configuration:

  • Edit all settings
  • Change collection
  • Re-authenticate
  • Delete source

Webhooks Tab​

For webhook-enabled sources:

  • Event history
  • Activity timestamps
  • Event types

Best Practices​

Organizing Collections​

  1. By Department: Engineering, Sales, HR
  2. By Project: Project Alpha, Q4 Initiative
  3. By Type: Documentation, Communications, Data

Sync Scheduling​

  1. High-change sources: Enable webhooks
  2. Stable content: Daily or weekly syncs
  3. Large sources: Off-peak scheduling

Managing Growth​

  1. Monitor document counts
  2. Archive old collections
  3. Remove unused connectors
  4. Clean up failed documents

Troubleshooting​

Sync Failures​

  1. Check error messages in job details
  2. Verify credentials are valid
  3. Ensure source is accessible
  4. Check permission settings

Missing Documents​

  1. Verify source contains content
  2. Check filter settings
  3. Wait for sync completion
  4. Review collection assignment

Slow Syncs​

  1. Large sources take longer
  2. Check for processing bottlenecks
  3. Consider filtering content
  4. Break into smaller collections

Next Steps​