AI-Mugallym
On-premise AI platform for organizations — retrieval-augmented chat that runs entirely inside the client's network.
on-premise deploymentThe problem
Organizations wanted an AI assistant grounded in their own documents, but couldn't send that data to a third-party API. The platform had to run fully on-premise, on the client's own hardware, with zero external calls.
Key decisions
Hybrid retrieval, not just embeddings
Pure vector search missed exact-match queries — names, codes, short phrases. Dense retrieval (ChromaDB) is fused with sparse BM25 via Reciprocal Rank Fusion, then re-ranked with a cross-encoder for meaningfully better recall on short queries.
Query expansion before retrieval
HyDE (hypothetical document embeddings) and multi-query rephrasing widen the net before hybrid search runs, cached in Redis for 24h so repeat queries are instant.
Self-hosted models end to end
Ollama/vLLM for generation, Sentence Transformers for embeddings — nothing leaves the network. Rate limits are enforced per role (guest / user / moderator / admin) directly in Redis.
Built and measured a Turkmen-language benchmark
General LLM benchmarks don't cover Turkmen. Wrote an evaluation suite scoring language consistency, refusal avoidance, keyword coverage and latency, specific to this deployment.
Stack
Where it stands
Live admin panel with real-time CPU / RAM / Disk / GPU monitoring; every RAG behavior — reranking, MMR diversity, query expansion — is toggleable per deployment without a restart.
Need something like this built or run?