SS

RAG Context Playgroundgpt-4o

Connected to Rust Gateway API

How does Contextra combine vector search results with keyword search, and what reranking algorithm is used?
14:32:05
SS
Contextra executes a **Hybrid Retrieval Pipeline** engineered for maximum precision and recall: 1. **Dual Retrieval**: - **Vector Search**: Performs cosine similarity search over vector embeddings in Qdrant (using HNSW index). - **Keyword Search**: Performs BM25 full-text match over chunk text in PostgreSQL / Redis payload index. 2. **Reciprocal Rank Fusion (RRF)**: Scores from both retrievers are merged using the RRF formula: $$\text{RRF\_Score}(d) = \sum_{m \in M} \frac{1}{k + r_m(d)}$$ where $k=60$ and $r_m(d)$ is the rank of document $d$ in system $m$. 3. **Reranking & Context Assembly**: The top 25 merged candidates are passed to a cross-encoder Reranker model, returning the final Top K chunks bounded by your token budget. ```rust // libs/retrieval/src/hybrid.rs pub async fn fuse_rankings( vector_results: &[ScoredChunk], keyword_results: &[ScoredChunk], k: f64, ) -> Vec<ScoredChunk> { // RRF implementation in Rust } ```

Retrieved Sources & Citations:

14:32:06138 ms485 tokens

Retrieval Inspector

Qdrant + BM25

Top Ranked Chunks (4)

system_architecture_spec.md96.2% match

Contextra retrieval pipeline: 1) Query normalization & entity extraction. 2) Concurrent Qdrant dense vector search & Postgres BM25 keyword search. 3) Reciprocal Rank Fusion (RRF k=60). 4) Cross-encoder neural reranking.

Collection: Core System DocsIndex #4
memory_scoring_algorithm.pdf89.4% match

Memory importance scoring uses exponential recency decay coupled with frequency-weighted keyword overlap to maintain a dynamic 4KB conversation summary buffer.

Collection: AI Research PapersIndex #12
gateway_api_reference.openapi.json84.1% match

POST /api/v1/conversations/{id}/messages/stream returns a Server-Sent Events (SSE) stream delivering token deltas, citation references, and final execution metrics.

Collection: API SpecificationsIndex #28
system_architecture_spec.md78.8% match

Context Assembler enforces a hard token limit by packing system prompt, long-term memory summary, and retrieved chunks into an optimized prompt tree.

Collection: Core System DocsIndex #9
Retrieval Pipeline142 ms
Embed: 18ms
Qdrant: 24ms
Rerank: 45ms