Document Analytics

Document Analytics is a troubleshooting tool for administrators that shows how documents are stored across the three backend systems: DynamoDB (metadata and chunks), OpenSearch (search index), and S3 (original files). Use it to identify sync mismatches, orphaned chunks, and processing failures.

Admin access required. This tool is for administrators who need to troubleshoot document processing issues.

Accessing Document Analytics

1. Navigate to the Document Analytics page from the admin menu
2. The page loads with filter controls, summary statistics, and a document table

Filters

Filter	Options	Notes
Status	All Statuses, indexed, processing, pending, failed, partial	Filter by document processing status
Limit	25, 50, 100, 250	Number of documents to display
Verify S3	Checkbox	Adds S3 existence checks — slower but shows if original files exist

Click Refresh after changing filters to reload the data.

Summary Statistics

Five cards at the top show the overall health of your document storage:

Stat	What It Shows
Total Documents	Total number of documents in the system
OpenSearch Docs	Documents indexed in the search engine
OpenSearch Chunks	Total text fragments across all indexed documents
Orphaned Chunks	Chunks in OpenSearch that have no matching DynamoDB record. Red if > 0 (indicates a problem), green if 0.
By Status	Color-coded pills showing count per status (indexed = green, processing = blue, pending = yellow, failed = red, partial = orange)

Orphaned Chunks Warning

If orphaned chunks are detected, a red warning box appears showing the document IDs that exist in OpenSearch but NOT in DynamoDB. Click any ID to copy it to your clipboard for further investigation.

Document Table

The main table lists all documents with these columns:

Column	Description
Filename	Document name (hover for full name) + truncated document ID (click to copy)
Status	Color-coded badge: indexed (green), processing (blue), pending (yellow), failed (red), partial (orange)
Sync	"Synced" (green) if DynamoDB and OpenSearch match, "Not indexed" (yellow) if not in OpenSearch yet, "Mismatch (X/Y)" (red) if chunk counts differ
Dynamo Chunks	Number of chunks stored in DynamoDB
OS Chunks	Number of chunks indexed in OpenSearch
Extraction	How the document was extracted (extraction source)
Actions	Details button to open the document detail modal

Document Detail Modal

Click Details on any document to see a deep cross-reference view:

Consistency Check

A banner at the top shows either:

• Green: "All storage systems consistent" — DynamoDB, OpenSearch, and S3 all agree
• Red: "Inconsistency detected" — with details showing chunk counts and S3 status

DynamoDB Section

• Primary key (PK) — click to copy
• Chunk count
• Expandable Metadata — shows all document metadata fields as formatted JSON
• Expandable Chunks — list of each chunk with sort key, chunk index, page number, type, and text preview

OpenSearch Section

• Chunk count and sync status
• Expandable Chunks — list with OpenSearch ID (click to copy), chunk index, page number, type, and text preview

S3 Section

• Exists: green checkmark or red X
• Size: file size in KB
• Modified: last modification timestamp

Troubleshooting

Document shows "Mismatch" sync status

Solutions:

• The document has different chunk counts in DynamoDB vs OpenSearch
• Try re-uploading the document to trigger reprocessing
• Check if the document is still "processing" — it may not be finished yet

Orphaned chunks detected

Solutions:

• Copy the orphaned document IDs from the warning box
• These chunks exist in OpenSearch but their parent document was deleted from DynamoDB
• Contact engineering to clean up the orphaned indices

Document stuck in "failed" status

Solutions:

• Click Details to see if there's an error message
• Common causes: corrupt file, unsupported format, file too large
• Try re-uploading the document in a supported format (PDF, DOCX, TXT)

Related Articles

• Documents and AI Chat
• Boost Categories

---

Need help? Contact our support team at [e-mail]