Ornith 1.5 9B and 35B-A3B Benchmark on RTX 5090

ornith-ai released two models on Hugging Face: Ornith-1.5-9B-GGUF and Ornith-1.5-35B-A3B-GGUF. Both cards quote a 262,144 token context.

In this blog, we will compare both against Gemma 4-26B-A4B and Qwen 3.8 27B on one RTX 5090, on speed, memory, context, quality and tool use. We read the GGUF files first.

Gemma is a mixture of experts like Ornith 35B-A3B. In simple words, a mixture keeps many small expert networks inside one model and wakes only a few per token. Qwen 3.8 is dense, so it reads all of its weights every time, which makes it our control.

Setup: Four Models, One RTX 5090

Everything ran on one machine in one session.

• GPU: RTX 5090, 32 GB VRAM, full GPU offload
• Runtime: llama-server build 10448 on Windows 11
• Quantization: Q4_K_M for all four models
• KV cache: f16 with flash attention on
• Grading: pure Python functions, no judge model, meaning no AI grading another AI

Every model started the same way.

llama-server.exe -m Ornith-1.5-35B-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 8192 -ngl 999 -fa on -np 1 --jinja --no-warmup --metrics

Decode tests used the raw /completion endpoint with ignore_eos, so the model cannot stop early. Every model produced exactly 512 tokens.

What Is Inside the Ornith GGUF Files?

A GGUF file stores the model on disk, and its header lists every tensor inside. So we can count the blocks ourselves instead of trusting the card.

Ornith 9B reports the architecture qwen35 and Ornith 35B-A3B reports qwen35moe. Both sit on the Qwen 3.5 stack, so any gain comes from training rather than structure.

The header also names 256 experts, 8 of them active per token.

Grouping the tensors by block index shows the layout. Most Ornith blocks carry no attention weights at all. In the 35B, attention sits at block 3, 7, 11 and every fourth position after that.

The three blocks in between are Gated DeltaNet. They carry ssm_alpha, ssm_beta and ssm_out instead of key and value projections.

In simple words, Gated DeltaNet is a memory that stands in for attention. Attention keeps every past token and reads back over all of them for each new token. Gated DeltaNet keeps one running summary instead, and the gate decides how much of that summary to hold on to and how much the new token writes over. The summary is a fixed size whatever the context, so only the attention blocks hold a KV cache.

Gemma solves the same problem differently. All 30 of its blocks are attention blocks, but 25 use a 1,024 token sliding window.

Let me tabulate all four models for your better understanding.

The plus one is a draft head, a small helper block that guesses the next few tokens. It is not a layer the token passes through, so we count it apart. Gemma carries none.

Residual is the width of the model's running hidden state, the workspace every block writes into. Ornith is the finer mixture, with 256 experts against Gemma's 128. Both activate 8 per token.

Which Model Writes Tokens Fastest?

Decode speed is how fast a model writes its answer, in tokens per second. If it follows active parameters, both mixtures should beat Qwen by a wide margin.

The prediction holds across both vendors. Ornith leads Gemma by 18%. Against the dense Qwen the gap is 3.5 times.

So speed here belongs to sparsity, not to Ornith.

Content made almost no difference. Across counting, code, JSON, essay and fiction the spread stayed under 1.5% for every model.

How Fast Does Each Model Read a Long Prompt?

Prefill is how fast a model reads the prompt before answering. We measured it from 512 tokens to 32k.

All numbers in the table are tokens per second.

Both mixtures are slow on very short prompts, because picking which experts to use costs a fixed amount on every step. A 512 token prompt is too short to absorb it.

From 2k upward Gemma overtakes Ornith 35B and stays ahead, so on long prompts and short answers Gemma is the faster mixture.

How Much Memory Does the KV Cache Cost?

The KV cache holds one entry per token already in context. It decides how much context a card can hold.

Ornith and Gemma land on the same number by different routes. Ornith skips the cache on three blocks in four, and Gemma keeps 25 of its 30 blocks inside a 1,024 token window.

Ornith 9B looks odd here. It has the fewest attention blocks of the four, 8 against the 35B's 10, and it still costs more per token. Block count is only half the sum.

The other half is KV heads. Every attention block stores one key and one value for each KV head it keeps, and a key and a value take 256 numbers each at 2 bytes apiece. So a block with 4 KV heads costs 4 × 512 × 2 bytes, which is 4 KB.

Here, we can see that the 9B pays double per block, so its 8 blocks come to more than the 35B's 10.

Keeping fewer KV heads than query heads is grouped query attention, where several query heads share one KV entry. The 35B shares each KV head across 8 query heads and the 9B across 4. That single choice is what separates them.

So the Gated DeltaNet schedule is a real win over Qwen 3.8, which shares Ornith's family and still costs 64 KB. It is not a win over Gemma.

We can also store the cache in smaller numbers to save memory. That is KV quantization, and it behaved the same way for all four models.

How Much Context Can Each Model Really Use?

All four models advertise 262,144 tokens, and all four start at that size and report it back. So the number a server reports proves nothing.

So we measured the point where decode speed drops instead. The ceiling below is the largest context where a model still runs within 90% of its own short context rate.

Qwen 3.8 holds 78 tok/s at 245,760 and drops to 7.3 at 262,144. A 7% larger window costs it 91% of its speed. Its 64 KB per token puts it at the edge of the card.

The other three stay flat to the full window, and Gemma does it with the most room to spare.

How Fast Can Each Model Answer From a 128k Document?

We hid a fact inside documents of 16k, 64k and 128k tokens, then asked for it back.

Ornith 35B and Gemma finish about half a second apart. Qwen takes three times longer.

Every model found the hidden fact every time. Retrieval accuracy separated nothing.

Does the MTP Draft Head Make These Models Faster?

Both Ornith files carry a nextn block, a multi-token prediction head, or MTP. In simple words, a small helper inside the file guesses the next few tokens and the full model checks all the guesses in one go. Qwen 3.8 ships one too, and Gemma does not. We tuned the same mechanism flag by flag in the Qwen 3.8 27B speed settings post.

llama-server ignores those tensors unless we ask for them.

llama-server.exe -m Ornith-1.5-35B-Q4_K_M.gguf -c 8192 -ngl 999 -fa on --jinja --spec-type draft-mtp --spec-draft-n-max 4

MTP helps on structured text and hurts on prose.

Acceptance rate explains it. That is the share of guesses the full model agrees with, and a rejected guess is wasted work. The draft head guesses right 43% to 86% of the time on structured output. On prose that falls to 26% to 39%, and drafting stops paying for itself.

This one goes to Qwen 3.8. Its head reaches 2.34x on JSON where Ornith 35B manages 1.10x, and it stays above 1.0 on prose where both Ornith models turn slower.

Which Model Serves Many Users at Once?

We fired 1, 4 and 8 requests in parallel and measured total throughput. That is the tokens per second across all requests added together.

Ornith leads on a single stream. Gemma overtakes it at four slots and finishes 4% ahead at eight, because its lighter weights leave more VRAM for the extra caches that parallel requests need.

Which Model Answers Hard Problems Fastest?

Speed means nothing if the fast models answer worse. We ran 12 problems with checkable answers: eight in a base tier run once, and four harder ones run at two seeds each. A Python function grades every run by reading the final ANSWER: line.

Accuracy separated almost nothing. Three models scored perfectly on both tiers, and Qwen 3.8 missed one of its two runs on a single hard math problem, answering it correctly at the other seed. The table below is the hard tier, eight scored runs per model, because it is the only place the models had to work.

Gemma spends 9,759 tokens where Ornith spends 2,110, which is 4.6 times more thinking for the same score. That turns Gemma's prefill and concurrency wins into a 43.6 second answer. This gap comes from training, not from architecture.

Ornith also thinks proportionally less, at 70% of output against 89% for both rivals, so it commits to an answer sooner.

Qwen 3.8 spends the fewest tokens of anyone at 1,420, and still takes 19.2 seconds, because it decodes at 77 tok/s. Token count and wall clock point in opposite directions.

How Well Does Each Model Use Tools?

We built a tool-calling test with a virtual database and a small source tree. Ten tasks ran at two seeds each. The tools return fixed data, and nothing runs model-written code.

One of our own tasks was badly worded. It ended with "report the timezone of that region", so every model answered UTC+1 while our grader wanted the datacenter too. We excluded it and fixed the wording, which leaves 18 scored runs per model.

Both Ornith models solved everything and Gemma missed one. Nobody produced a malformed tool call in 72 runs.

The thinking pattern repeats: Gemma spends 5,573 characters per task against Ornith's 2,230.

What Does Ornith's Model Card Claim?

Every claim on the card, checked against what we measured.

Which Model Should We Run?

Ornith 35B-A3B and Gemma 4-26B-A4B are near twins on hardware: same KV cost, same full window, about half a second apart on a 128k question. Gemma is lighter on VRAM, faster on prompts above 2k, and faster under eight-way load.

Training separates them, not silicon. Ornith reaches the same answers in a fifth of the time because it does not keep re-checking itself. So take Ornith for question answering and agent loops, and Gemma for long prompts or many users, which also saves 3.3 GB.

Ornith 9B holds a full 262k window in 15.4 GB and reads short prompts faster than either mixture. Qwen 3.8 27B is the slowest here and the only model that cannot hold its own advertised context, but it keeps the best MTP head and the lowest token count per answer.

What We Did Not Test

• Vision. Both Ornith repositories ship an mmproj file. We tested text only.
• Quality cost of KV quantization. We measured the sizes, not the output.
• Quality cost of MTP. We proved the output changes, not that it gets worse.
• Agentic coding at SWE-bench scale, where Ornith's main claim lives.

Conclusion

This is how the Ornith 1.5 release measures up. We read the hybrid block layout out of the GGUF headers. Then we ran both models against Gemma 4-26B-A4B and Qwen 3.8 27B on one RTX 5090. The two mixtures came out near twins on hardware, at the same 20 KB per token and the same full 262k context, and training is what separates them.

Key takeaways:

• Decode speed follows active parameters, not total size. Both mixtures beat the dense Qwen 3.8 by a wide margin, so the speed belongs to sparsity, not to Ornith.
• Ornith 35B-A3B and Gemma 4-26B-A4B land on the same KV cache cost by different routes: skipped cache blocks against sliding windows.
• On Windows, "the context loads" proves nothing. Measure where decode speed falls off a cliff instead.
• Ornith's real edge is training. It answers hard problems in 8 seconds where Gemma needs 43.6, at the same accuracy.
• MTP speeds up structured output and can slow down prose. Qwen 3.8 carries the best head of the three.

Next steps:

• See these same models face IBM's latest release in the Granite 4.2 vs Gemma 4 vs Qwen 3.8 benchmark.
• Tune draft depth, KV cache type, and context size on your own card with the Qwen 3.8 27B speed settings on llama.cpp.
• Match a quantization and a context size to your GPU with the local LLMs technical reference guide.