Supported Formats & Limits
This page covers the technical details of how thinnestAI processes your knowledge sources — supported formats, size constraints, chunking strategy, and tips for getting the best results.File Format Reference
Source Type Limits
Knowledge Base Limits
Chunking and Processing
When you add a source, thinnestAI breaks the content into smaller pieces called chunks. This is essential for accurate retrieval — the agent finds the specific chunk that answers the question, rather than searching through an entire document.How Chunking Works
- Content extraction — Text is extracted from the source format (PDF parsing, HTML scraping, etc.).
- Cleaning — Headers, footers, navigation elements, and other noise are removed.
- Splitting — Content is split into chunks at natural boundaries:
- Headings and section breaks
- Paragraph boundaries
- Sentence boundaries (for very long paragraphs)
- Overlap — Adjacent chunks share a small overlap to preserve context across boundaries.
Chunk Parameters
Why Chunk Size Matters
- Too large — Chunks contain mixed topics, making retrieval less precise.
- Too small — Chunks lack context, making answers less comprehensive.
- Just right — Each chunk covers one topic or concept completely.
Embedding Models
thinnestAI converts each chunk into a vector embedding — a numerical representation that captures the semantic meaning of the text. These embeddings power semantic search.How Embeddings Work
Embedding Details
Vector Storage
Embeddings are stored in a PostgreSQL database with the pgvector extension, enabling fast similarity search at scale. For larger deployments, thinnestAI also supports LanceDB and ChromaDB as vector storage backends.Search & Retrieval
When your agent searches the knowledge base, the system runs a hybrid search:Retrieval Parameters
Best Practices for Knowledge Quality
Document Structure
Good structure produces better chunks and more accurate retrieval. Do:- Use clear headings (H1, H2, H3) to organize content.
- Write one topic per section.
- Use bullet points and numbered lists for key information.
- Include a table of contents for long documents.
- Put multiple unrelated topics in one section.
- Use images as the primary way to convey information (images aren’t indexed).
- Include large blocks of unformatted text without breaks.
Content Quality
- Be specific — “Our PAYG plan charges ₹1.50/min for voice and ₹0.50/message for chat” is better than “See our pricing page for details.”
- Be complete — Each section should be self-contained. Don’t rely on context from other sections.
- Be current — Remove outdated content. Old information can cause incorrect answers.
- Avoid contradictions — If two sources say different things, the agent may give inconsistent answers.
Organizing Knowledge Bases
Example organization for a SaaS company:
Testing Your Knowledge
After adding sources, test with real questions:- Ask direct questions — “What’s the refund policy?” Should return exact information.
- Ask rephrased questions — “Can I get my money back?” Should find the same content.
- Ask edge cases — “What happens if I cancel after 60 days?” Should find the closest relevant policy.
- Check for gaps — If the agent says “I don’t have information about that,” you may need to add more content.
Troubleshooting
Next Steps
- Knowledge Sources — Step-by-step guide for each source type.
- Knowledge Overview — Creating and assigning knowledge bases.