For the past few weeks I've been tinkering with language models (LLMs) outside the usual scope of my cybersecurity labs. The starting idea was simple: have a fast, sovereign AI assistant, without depending solely on the American giants of the sector. The result ended up drifting toward something a bit more ambitious — getting several models to collaborate with each other, like a small committee that proofreads and refines itself.
Here's the story of these two experiments, hiccups included — because that's usually where you learn the most.
Why host your own AI assistant
First instinct: try local inference on my server, with Ollama. On CPU-only hardware, even with relatively lightweight models (3 to 8 billion parameters), generation speed capped out between 2 and 8 tokens per second — far too slow for smooth day-to-day use. Verdict, no appeal: for a responsive assistant, I had to move away from all-local and go through an external API.
I settled on Infomaniak AI Tools, a Swiss service that hosts open-source models (Mixtral, Llama, Gemma, Mistral, and others) behind an OpenAI-compatible API. Two reasons for this choice: consistency with my existing infrastructure (I was already using Infomaniak for my encrypted backups), and an approach more respectful of GDPR than systematically sending data to American servers.
Building the assistant: Open WebUI and its pitfalls
Once the API was chosen, I deployed Open WebUI as a Docker container on my server, connected to the Infomaniak API rather than Ollama. The interface is clean, customizable, and works well once configured.
Two lessons worth remembering from this phase:
The exact model name matters. The Infomaniak API rejects a misspelled model name — but the error it returns lists every valid identifier, which makes fixing it much easier.
A reflex worth having with containers: check what they're actually exposing. A classic point of vigilance with Docker: depending on its configuration, a container can end up more exposed on the network than intended. It's exactly the kind of thing I systematically check after every deployment — and this time, the check paid off, with a quick adjustment to restrict access to the strict minimum. A good illustration of why post-deployment auditing is an integral part of the process, not an optional step.
After a dedicated subdomain, a Let's Encrypt certificate, and a bit of visual customization, the assistant was running over HTTPS, accessible from a private Discord bot for everyday questions.
What if several models corrected each other?
Once the simple assistant was in place, the next idea emerged: what if, instead of querying a single model, a question passed through several models in sequence, each one refining the previous one's answer? A sort of automated proofreading committee.
I built this with AnythingLLM's Agent Flows, a no-code visual editor that lets you chain API calls and LLM instructions without writing a line of backend code. The chain I settled on:
Gemma 4 → Mistral Small → Nemotron → Ministral
Each model receives the original question and the previous model's answer, with instructions to refine, correct, or complete it.
The technical obstacles (and what they reveal)
This project was a good lesson in the concrete limits of multi-model orchestration:
JSON is more fragile than it looks. A chat API's raw response is a structured JSON object, not ready-to-use text. As soon as you feed a response's text back into the next call, every unescaped line break, tab, or quote breaks the request. The fix was to force every intermediate link in the chain to produce plain text, with no formatting whatsoever — markdown, tables, and bullet lists included.
Long answers end up slowing everything down. Without a length cap, each link in the chain considerably bulked up the next one's prompt, eventually causing server timeouts (504 errors) on the last call. Capping response size (max_tokens) and enforcing a conciseness instruction solved the problem — at the cost of a trade-off: on a multi-part question, the chain can end up focusing on one part of the question and forgetting another, with no visible error to flag it.
A chain that "refines" can also hallucinate. This is the most interesting point, in my opinion. An intermediate model can confidently introduce incorrect or invented information, presented as a "clarification" or "enrichment" of the previous answer. Without an explicit safeguard, nothing stops that drift from propagating all the way to the final output. It's not a bug to fix, but a structural feature of this type of architecture — worth keeping in mind before blindly trusting an output "consolidated" by several AIs.
What I take away from this
Technically, both building blocks work: a responsive, sovereign assistant on one side, a multi-model refinement mechanism on the other. But the exercise was mostly useful for understanding where the real limits of this kind of system lie — the fragility of the exchange format between steps, the trade-off between robustness and thoroughness, and above all the idea that consensus among several AIs isn't automatically synonymous with reliability.
The next step will be to work on a multi-part question detector upstream of the chain, to keep part of the question from getting lost along the way. To be continued.
Article written as part of my personal experiments under the HIVE banner — Hardening, Investigation, Verification, Expertise.