Small,
and it really reads.
Lynx is a small LLM built for RAG. It answers a question in one sentence, using only the documents you give it. We fine-tuned the 0.8B Qwen3.5 for question answering over retrieved documents in 20 languages. It runs on an ordinary CPU, with no GPU.
- Parameters
- 0.8B
- File (Q4_K_M)
- 0.67GB
- Languages
- 20languages
- Hardware
- CPUis all it needs
ReleasedDownload the weights and try it host and license are provisional
1/146 the size,
93.8 points.
We compared Lynx with gpt-oss-120b (117 billion parameters) on a test of 13,061 questions in 20 languages. Averaged over four measures, Lynx 0.8B scores 93.8 and gpt-oss-120b scores 97.5. On saying "no answer" for a short document that doesn't contain one, Lynx scores 98.0, nearly the same as the 120B model.
Answering from a short document3,751 questions
Finding the answer in a long document (about 2,500 tokens)3,697 questions
Saying "no answer" for a short document without one3,650 questions
Saying "no answer" for a long document without one1,963 questions
Out of 7,448 questions whose answer was in the document, Lynx wrongly declined with "no answer" on only 25 (gpt-oss-120b: 32).
* Measured on our own test (November 2026, Q4_K_M). Graded by Qwen3-32B (within 0.4 points of grading by gpt-oss-120b). Detailed conditions are at the end of the page.
20 languages,
all the same way.
The prompt and the answer both stay in that language. The questions, too, were made from each language's own Wikipedia (not translated). Scroll down and the tiles flip over to show the score for each language (pressing a tile flips it, too).
* Figures are the average of the four measures (points). About 650 questions per language (about 480 for Hindi only).
Finding one sentence
in 2,500 tokens.
Lynx finds the one sentence that answers the question, in one document hidden in a set of 16 otherwise unrelated documents. As you scroll, Lynx reads on through the documents. This is one question from a separate test written in the style of internal documents, with the answer Lynx actually returned.
QuestionOn which day is the Aobano city pool closed?
Evidence found
Lynx's answer
* All the texts are fictional (written for testing).
CPU only,
about 7 seconds.
On a laptop CPU (no GPU), a chat window replays everything from reading a document of about 2,500 tokens to giving the answer. Playback is at 3x speed; the clock shows the real seconds.
Lynxrunning on this PC's CPU
On which day is the Aobano city pool closed?Attached: 16 documents (about 2,500 tokens)
0.0splaying at 3x speed
Read the documentsWrite the answer
- Reading
- 388 tokens/s
- Writing the answer
- 41 tokens/s
- Hardware
- Core i7-11800H, 8 threads
* Measured in-house (Q4_K_M, llama.cpp). The reading speed is at 2,048 tokens; the writing speed is after reading 4,000 tokens. Results vary with the hardware and the length of the document.
To use it,
one prompt.
Just put the document and the question into the same prompt used in training, and send it. No system prompt is needed.
The answer you get back
- Turn thinking mode off (
enable_thinking: false). The model was trained without thinking. - With
temperature 0, you get the same answer every time. - The answer is one sentence that echoes the wording of the question. It averages 13 to 26 tokens.
{"messages": [{"role": "user",
"content": "(the prompt above)"}],
"temperature": 0, "max_tokens": 300,
"chat_template_kwargs": {"enable_thinking": false}}Download it
and run it right away.
2
Start it with llama.cpp
Use a build that supports Qwen3.5 (confirmed with b11829).
llama-server -m lynx-0.8b-Q4_K_M.gguf \ -c 8192 --jinja -t 8 --port 8090
3
Send the prompt
Send it in the format above to the OpenAI-compatible /v1/chat/completions endpoint.
What it is good for
- Answering from company policies, manuals and FAQs, with the source paragraph shown
- Document search in environments where data can't leave
- Workplaces where documents in several languages are mixed
Where other models fit better
- Long summaries (the answer is a single sentence)
- Extracting information into JSON
- Questions about general knowledge that isn't in the document
Pick a document
and try it.
You can send the documents and questions we have prepared to Lynx 0.8B running on a server. Choose one document and one question, then press Run. Some of the questions have no answer in the document.
Live: the real model is answering right now, on this server's CPU alone
1Choose a document
2Choose a question
3Lynx's answer
Lynx 0.8Brunning on a server
The answer will appear here.
How we measured.
Then, let's talk.
- Questions: Qwen3-32B wrote questions and answers from Wikipedia paragraphs in each of the 20 languages. gpt-oss-120b checked the questions, answers and documents and removed the unsuitable ones, leaving 13,061 questions.
- The four measures: a paragraph that has the answer (80 to 400 tokens) / a long document in which that paragraph is mixed in with other articles (about 1,500 to 3,500 tokens) / a paragraph without the answer (another paragraph of the same article, or another article) / a long document with the answer paragraph taken out.
- Models: Lynx 0.8B (Q4_K_M, llama.cpp) and gpt-oss-120b (vLLM, reasoning effort medium). The prompt is exactly the same.
- Grading: two models, Qwen3-32B and gpt-oss-120b, judged whether each answer was correct (their results differ by no more than 0.4 points).
- The "long document" example is one question from a separate test modeled on company policies, notices and the like.
- Conducted: November 2026. Like the other figures on the home page, these will be reviewed before the official release.