Virtual Search Assessor
Automate relevance assessment and query generation with the power of Large Language Models.
AI-Powered Relevance Analysis
The Virtual Assessor uses LLMs to perform tasks that traditionally require hours of manual human effort. This emerging paradigm allows you to scale your search evaluation process, create ground-truth data more efficiently, and gain deeper insights into your content.
Deep-Dive Analysis
Go beyond surface-level analysis. For any document in your search results, this feature initiates a multi-step pipeline:
- Downloads & Chunks: The full document content is fetched and broken into smaller, manageable chunks.
- Vectorizes: The query and each document chunk are converted into numerical vectors (embeddings).
- Evaluates: A similarity search finds the most relevant chunks, which are then scored by an LLM to produce an overall document relevance score.
Automated Query Generation
Instead of manually creating test queries, let our Virtual Assessor do it for you. You can trigger query generation:
- Content Analysis: An LLM analyzes the document's content to understand its key topics and concepts.
- Suggestion Engine: It then generates a list of both traditional keyword queries and natural language questions that the document would be a perfect answer for.
- Expand Coverage: This provides an efficient way to expand your test coverage and ensure your most important documents are discoverable.
LLM Judgement and Reporting
The platform uses LLMs not just for document analysis, but for meta-analysis of your test results. You can trigger an "LLM Judgement" on reports, where an LLM analyzes the comparative metrics and provides a qualitative summary of which search configuration performed best and why, acting as an automated data analyst.