+ 02 / THE TECHNOLOGY

Two engines.
One workload.↗

An optimised DSpark speculator that predicts how your outputs continue, and a custom KV backend that remembers what your traffic reuses. Both tuned on your workload, both measured against standard serving on the same GPUs.

ENGINE 01 / DSPARK SPECULATIVE DECODING

Draft a block.
Verify it at once.

DSpark, published by DeepSeek in July 2026, drafts a block of tokens in one pass and predicts how much of it will survive verification. We start from there and optimise it for your workload.

Block drafting

A parallel drafter proposes a whole block of tokens in one forward pass, instead of one token at a time.

Verified by the target

The target model checks the block in a single pass and keeps the longest accepted prefix. It is lossless: outputs follow the target model’s distribution exactly.

Optimised for your workload

Every deployment runs a DSpark optimised for its own workload, so your formats, phrasing and code conventions are predicted more often.

Scheduled for your load

Per-position confidence and your live concurrency decide how many tokens to verify, so the speed-up holds when the GPUs are busy.

BASELINE

Standard decoding

FORWARD PASSES: 0
TOKENS / PASS: —
vs STANDARD: …

The lighthouse keeper had not spoken aloud in eleven days, so when the knock came she answered with a voice that cracked like old paint. On the step stood a boy with salt in his hair and a letter sealed in green wax. “They said you would know what this means,” he whispered, and behind him the fog began, very slowly, to sing.Thanks for reaching out, Maya. I can see order #48213 was delivered to your previous address on 12 September. I have issued a free replacement to your new address, and it should arrive within 3–5 business days. You will receive a tracking link by email as soon as it ships. Is there anything else I can help you with today?def get_user(session, user_id: int) -> User | None: """Return the user, or None if they do not exist.""" stmt = select(User).where(User.id == user_id) return session.execute(stmt).scalar_one_or_none() def list_active_users(session, limit: int = 50) -> list[User]: stmt = select(User).where(User.active.is_(True)).limit(limit) return list(session.execute(stmt).scalars()){"invoice_id": "INV-20931", "vendor": "Northwind Traders", "currency": "USD", "total": 1840.50, "due_date": "2026-10-15", "line_items": [{"sku": "NW-114", "qty": 12, "unit_price": 96.25}, {"sku": "NW-208", "qty": 4, "unit_price": 171.37}]}The lighthouse keeper had not spoken aloud in eleven days, so when the knock came she answered with a voice that cracked like old paint. On the step stood a boy with salt in his hair and a letter sealed in green wax. “They said you would know what this means,” he whispered, and behind him the fog began, very slowly, to sing.

PUBLISHED DRAFTER

Stock DSpark

FORWARD PASSES: 0
TOKENS / PASS: —
vs STANDARD: …

WORKLOAD-TUNED

Our DSpark

FORWARD PASSES: 0
TOKENS / PASS: —
vs STANDARD: …

Token from the target model Drafted token, verified and accepted Draft awaiting verification

MEAN ACCEPTED TOKENS PER VERIFICATION · QWEN3-8B · T = 1

2468

5.30

5.33

6.17

7.05

GSM8K

4.33

4.64

5.52

6.48

HumanEval

2.66

3.11

3.72

4.86

MT-Bench

2.54

2.98

3.58

4.52

Alpaca

EAGLE-3DFlashStock DSparkOur DSpark

SPEED-UP OVER STANDARD DECODING · BY CONCURRENCY

Speed-up over standard decoding as concurrency grows
Concurrent requests	Our DSpark	DFlash	EAGLE-3
1	5.9	5.1	1.6
4	5.6	4.5	—
8	5.3	4.5	—
16	4.9	3.9	—
32	4.3	2.8	1

EAGLE-3, DFlash and stock DSpark acceptance from the DSpark paper, compared in the same harness. DFlash and EAGLE-3 concurrency curves from the DFlash paper on B200 with SGLang. At 32 concurrent requests our DSpark runs 4.3× faster than standard decoding.

ENGINE 02 / CUSTOM KV BACKEND

Remember everything
worth remembering.

Prefill is the most expensive way to reread a prompt you have already seen. Our KV backend makes reuse the default path, across every tier of memory in the server.

Tiered by temperature

Hot blocks stay in GPU memory, warm blocks move to host DRAM, cold blocks to local NVMe. Reuse promotes them back in milliseconds.

Workload-aware eviction

We keep what your traffic will reuse: system prompts, tool schemas and live sessions, rather than whatever was used most recently.

Hybrid-attention ready

Sliding-window, linear and sparse-attention layers need their own cache layouts. Stock engines often miss on these models; ours is built for them.

Cache-aware routing

Each request goes to the worker that already holds its context, so reuse becomes the default path instead of a lucky hit.

ONE SESSION · AGENT CONVERSATION, 8 TURNS

Recompute the whole context Reuse cached KV, prefill only new tokens

6K-token system and tool prefix · about 5K tokens added per turn · 12K tokens/s prefill · 400K tokens/s KV load from the cache tiers.

UNDER REAL AGENTIC LOAD

0× faster to the first token.

Average time to first token across 32 concurrent coding-agent sessions with 100K-token contexts. Baselines as published by LMCache, May 2026.

Recompute every turn0.0 s

vLLM prefix cache0.0 s

LMCache offload0.0 s

Our KV backend0.0 s

CACHE HIT RATE

0% of prompt tokens,
never recomputed.

Share of prompt tokens served from cache on a creative writing workload, next to hit rates reported across the industry.

DeepSeek API, 24 h of production traffic0.0%

Claude Code sessions via Requesty0.0%

Cline sessions via Requesty0.0%

Our KV backend · creative writing0.0%

HOW WE MEASURE

Your traffic.
Your GPUs. Your target.

Every result compares our stack with standard serving on identical hardware, at the same latency target, on replayed traffic.

Replay your traffic

A representative sample with your real mix of prompt lengths, reuse and output formats.

Same GPUs, same target

Standard serving runs on identical hardware at the same per-user speed and time-to-first-token target.

Check the outputs

DSpark is lossless, so outputs match standard decoding. Where precision changes, quality is evaluated too.

Report everything

Throughput per GPU, TTFT and inter-token latency at p50 and p99, cache hit rate, acceptance and cost per token.

WHERE STOCK METHODS STRUGGLE MOST

The harder the workload,
the bigger the gain.

Creative writing defeats generic speculative decoding and prefix caching: every reply is new prose, and readers pause long enough between turns for ordinary caches to forget the story. It is also where optimising for the workload pays off most.

DSPARK · TOKENS ACCEPTED PER VERIFICATION Stock DSpark Ours

Creative writingBIGGEST GAIN

2.46.2

×2.6

Support chat

3.75.9

×1.6

Knowledge assistant

3.95.6

×1.4

Coding agent

5.58.1

×1.5

Document extraction

6.210.2

×1.6

KV BACKEND · PROMPT TOKENS FROM CACHE Standard prefix cache Ours

Creative writingBIGGEST GAIN

22%91%

×4.1

Support chat

61%93%

×1.5

Knowledge assistant

35%78%

×2.2

Coding agent

64%96%

×1.5

Document extraction

48%89%

×1.9

WORKLOAD PROFILE

Creative writing

Long stories and character dialogue at high temperature. Every reply is new prose, and readers pause long enough between turns for ordinary caches to forget the story.

MODEL: Qwen3.8-Flash-Next
HARDWARE: 4× H100
LATENCY TARGET: ≥ 40 tok/s per user

×0.0output throughput per GPU

Output throughput↑ 6.8×

STANDARD

760

TUNED

tokens / s / GPU

Cost per 1M output tokens↓ 6.8×

STANDARD

$1.46

TUNED

$0.00

USD · at $3.99/GPU-hour

Time to first token, p50↓ 7.9×

STANDARD

1.90

TUNED

0.00

seconds

Prompt tokens from cache↑ 4.1×

STANDARD

TUNED

% of prompt tokens

PUBLISHED BASELINES

Compared with
the best published work.

Baselines come from papers, engineering blogs and public price lists, accessed 24 September 2026.

DSpark paper, DeepSeek-AI (2026-07-06)DFlash paper (ICML 2026)EAGLE-3 paper LMCache blog: multi-turn agentic workloads on MI300X (2026-05-12)DeepSeek inference system overview (2025-02)Requesty coding-agent economy report SemiAnalysis InferenceX benchmarks (accessed 2026-09-24)DeepSeek API pricing (accessed 2026-09-24)Z.ai pricing (accessed 2026-09-24)Alibaba Cloud Model Studio pricing (accessed 2026-09-24)Claude API pricing (accessed 2026-09-24)Gemini API pricing (accessed 2026-09-24)Lambda GPU pricing (accessed 2026-09-24)

THE NEXT MOVE IS YOURS

Bring your workload.
We’ll bring the numbers.

Benchmark your workload

One benchmark.
A whole new cost curve.

Two engines.One workload.↗

Draft a block.Verify it at once.

Block drafting

Verified by the target

Optimised for your workload

Scheduled for your load

Remember everythingworth remembering.

Tiered by temperature

Workload-aware eviction

Hybrid-attention ready

Cache-aware routing

0× faster to the first token.

0% of prompt tokens,never recomputed.

Your traffic.Your GPUs. Your target.

Replay your traffic

Same GPUs, same target

Check the outputs

Report everything

The harder the workload,the bigger the gain.

Creative writing

Compared withthe best published work.

Bring your workload.We’ll bring the numbers.

Two engines.
One workload.↗

Draft a block.
Verify it at once.

Remember everything
worth remembering.

0% of prompt tokens,
never recomputed.

Your traffic.
Your GPUs. Your target.

The harder the workload,
the bigger the gain.

Compared with
the best published work.

Bring your workload.
We’ll bring the numbers.