Uploading Knowledge
Add documents and web content to a knowledge base so agents can retrieve from it at runtime. Xagent parses, chunks, embeds, and indexes everything you upload.
Upload Methods
- File upload — add one or many files at once, or drag and drop them onto the upload area.
- Website import — crawl a site starting from a URL and import the pages it finds.
Supported File Types
| Group | Formats |
|---|---|
| Documents | .pdf, .doc, .docx, .pptx, .txt, .md, .html |
| Data | .xlsx, .xls, .csv, .json |
| Code | .py, .js, and other common source formats |
File size
Maximum 100 MB per file. Split larger files before uploading.
Processing Options
When you upload, you control how documents are turned into searchable chunks:
| Option | Choices / default |
|---|---|
| Parse method | default (recommended), pypdf, pdfplumber, unstructured, pymupdf, deepdoc |
| Chunk strategy | recursive (default), fixed_size, markdown |
| Chunk size | characters per chunk (default 1000) |
| Chunk overlap | overlapping characters between chunks (default 200) |
Uploaded documents move through pending → processing → completed (or failed). Each is parsed, chunked, embedded, and indexed automatically.
Website Import
Point the crawler at a start URL and bound the crawl so it stays focused and respectful:
- Max pages (default 100) and crawl depth (default 3) cap how much is fetched.
- Concurrent requests (default 3, max 10) and request interval (default 1s) throttle load on the target.
- Timeout (default 30s) bounds how long each page fetch may take — raise it for slow sites.
- URL / exclude patterns, same-domain only, and CSS content / remove selectors filter what is kept.
- Follow robots.txt keeps crawling within the site's stated policy.
Tune chunking to your content
Technical docs benefit from larger chunks with more overlap (~1500–2000 / 300–500). FAQs work better with smaller chunks (~500–800 / 100–200). For code, use the markdown strategy to preserve structure.
Next Steps
- Knowledge Retrieval → — search and use what you uploaded
- Knowledge Bases Overview →
- Embedding Models →