Knowledge-base ingestion¶

Concepts¶

Source connectors; synchronization; change detection; incremental ingestion
Document parsing; OCR; layout extraction; tables; captions; images
Normalization; deduplication; ingestion validation
Document IDs; versions; stable chunk IDs; source URLs
Metadata extraction; source permissions; access-control metadata
Fixed-size chunking; recursive chunking; semantic chunking; structure-aware chunking
Chunk overlap; parent-child chunks; document hierarchy
Failed-ingestion recovery; dead-letter ingestion; backfills
Freshness; expiration; deletion propagation; data lineage