Pareto DeepSeek V4.1 Flash review: only 5 hits in 18 hot requests, answers indistinguishable from official

In the 2026-11-03/04 APB Full run, Pareto's four scores are Q 44.58, F 93.42, P 57.69 and R 100, with an overall of 68.72. The 8k hot requests hit 2 of 9 and 32k hit 3 of 9.

On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Profile covers Pareto only: what each section measured and how it came out.

The numbers below are scores, not a ranking or a provider recommendation. The four scores, Q, F, P and R, come first; the overall is a summary index, not a certification, and it has no confidence interval. The scoring rules are defined in AIHubMix Provider Benchmark Standard v2.0.

Scores across six providers

Providers are listed in a fixed order, not a ranking. K is a sub-score of P, accounting for 20% of it, and is listed separately for reference. The overall score is computed with the composition rules of AIHubMix Provider Benchmark Standard, which also take in the comparison with the official channel and evidence completeness; it is not a direct weighting of the four scores in the table. It is a summary index with no confidence interval. The DeepSeek official channel and Alibaba Cloud differ by only 0.37 overall, which is not a distinguishable difference.

At a glance

• Answer quality: Q 44.58, with BFCL 85.27%, HLE 19.32% and AA-LCR 52.00%. The gap to the official channel is +2.12 percentage points, with a 95% interval of −0.02 to +4.18: not distinguishable.
• Function support: F 93.42, with 29/33 original scored checks passing. Tool calls and tool-result round-trip, parallel and named calls, JSON mode and conversation history have additional successful streaming records; scores retain their original values.
• Response speed: P 57.69. Median first body token 5.85 s at single concurrency and 6.13 s at 10-way, TPOT 4.63 ms; 110 of 180 speed requests were valid.
• Cache: K 27.58. The 8k hot requests hit 2 of 9 and the 32k hot requests 3 of 9; no cold control read the cache.
• Service stability: R 100. All 300 dedicated requests succeeded on the first attempt.

Test setup

Requests used the AIHubMix quick start route with a provider-specific model ID; the other providers' IDs are listed on the models page. The model's own parameters are described in DeepSeek's API documentation.

Q — answer quality: 44.58

Answer quality draws on three suites: 500 BFCL questions answered 3 times each, 250 HLE questions answered 10 times each, and 25 AA-LCR questions answered 10 times each. BFCL is scored by automatic rules; HLE and AA-LCR are judged right or wrong by the judge model deepseek-flash. All six providers received the same questions, and the three suites are pooled into one accuracy. Ten answers were removed because the question could not be scored, leaving 4,240 scored question-instances.

Every question-instance received a complete answer; there were no service failures.

Each BFCL question was answered 3 times; the table sums the 3 rounds.

Compared with the other five

Vs official: the accuracy gap in percentage points, with the 95% interval in brackets; an interval that crosses 0 is recorded as not distinguishable. The HLE and AA-LCR judge, deepseek-flash, belongs to the same DeepSeek family as the tested model and was not calibrated, so its judgments may be lenient.

F — function support: 93.42

The function cases check whether interface behaviour matches the protocol. F 93.42 comes from the 33 counted checks (29 passed), after renormalising the observed group weights; the other 147 planned check slots in the original suite were excluded from scoring.

• The listed features have successful streaming records; this does not award full marks to the whole group. No checks are added to scoring. F 93.42 and the overall of 68.72 retain their original values.

“Windows” is how many of the 3 windows failed the check.

The malformed-messages and malformed-stream-options cases returned 400 with an empty response body on several providers. Reasoning-token metering behaved differently across providers.

Compared with the other five

P — response speed: 57.69

Each request carried about 10,000 input tokens and asked for about 1,100 output tokens of body text. In each of three windows, 30 requests ran at single concurrency and 30 at 10-way concurrency: 180 requests in all.

Invalid requests are scored as infinite, so the medians in the table can be higher than valid-only medians. TPOT is estimated from streamed chunks, not measured per token on the hardware. Throughput is valid output tokens at 10-way concurrency divided by batch wall-clock time.

Compared with the other five

All six shared one machine, network and account. The three windows were scheduled provider by provider, so collection periods do not fully overlap; small differences should not be over-read. ∞ means more than half the requests were invalid, so the median counts as infinite.

K — cache: 27.58

Each group sends five requests. The first carries a fixed prefix of about 8k or 32k tokens and seeds the cache. Requests 2–4 keep the prefix and change only the final question, at gaps of 5 s, 30 s and 120 s. The fifth uses a new prefix as a cold control. Three windows × two lengths make 30 requests. K is the cache tokens read by hot requests divided by the tokens that should be reusable under the rules, with the two lengths weighted equally; K is 20% of P.

Values are cache-read tokens reported by the server; “Not returned” means the field was absent from the response.

The 8k hot requests hit 2 of 9, reading 7,936 tokens on each hit; the 32k hot requests hit 3 of 9, reading 32,000 tokens on each hit. Cold requests read no cache, so hot and cold separate cleanly.

Compared with the other five

R — service stability: 100

The 300 requests fall into four types: short text, long input, tool calls and multi-turn history. They ran at single concurrency across three windows. Only whether the first attempt returned a valid, complete response is checked; answers are not graded and nothing is retried. Failures in the answer-quality and speed sections are counted in Q and P, not again in R.

All 300 requests returned a valid, complete response on the first attempt. R = 100 describes these 300 dedicated requests only; it does not mean the whole chain had no failures. The lower bound of the 95% interval is 98.78%: 300 successes out of 300 can only say the failure rate is likely below 1.22%.

Compared with the other five

R covers only these 300 dedicated requests; failures in the answer-quality and speed sections are recorded in Q and P and are not counted again in R.

FAQ

Why is F 93.42 when only 33 checks were counted?
The original counted checks passed 29 of 33, producing F 93.42 after renormalising the observed group weights; the other 147 planned check slots were excluded from scoring. The streaming records describe specific working features and add no scored checks. The displayed overall remains 68.72.

Why is P 57.69 when TPOT scored 100.00?
Only 110 of 180 speed requests were valid; 70 returned 200 with None. Invalid requests count as infinite, so the median first body token is 5.85 s single and 6.13 s at 10-way (3.08 s and 4.92 s valid-only). TTFA scored 24.59 and throughput 72.05, for a response score of 65.22; with cache K 27.58 at 20%, P is 57.69.

Why is K only 27.58?
Hot requests hit 2 of 9 at 8k (21.89) and 3 of 9 at 32k (33.27), and the two lengths weigh equally. When a hit happened it read 7,936 or 32,000 tokens, and no cold control read the cache.

Is Pareto's answer quality higher than the official channel's?
Not distinguishably. Combined accuracy is 44.58% against 42.45%, a gap of +2.12 points, but the 95% interval runs from −0.02 to +4.18 and crosses 0. The HLE and AA-LCR judge, deepseek-flash, was not calibrated and may be lenient.

Which F cases failed?
Two. The streaming stop sequence returned an empty body in all three windows, while the same input without stop worked. Exact text output had an extra final period in one of three windows.

Keep reading: the DeepSeek V4.1 Flash series

• Pareto's Q gap of +2.12 is measured against the official channel; the baseline's own run is in DeepSeek official channel DeepSeek V4.1 Flash review: comparison baseline, 15 invalid requests at single concurrency.
• Fireworks also returned empty bodies on stop requests, and hit 3 of 9 at 8k: Fireworks DeepSeek V4.1 Flash review: stop requests return empty body, 8k cache hot requests hit 3 of 9.
• Alibaba Cloud's hot requests hit 18 of 18, against Pareto's 5 of 18: Alibaba Cloud DeepSeek V4.1 Flash review: 1.34s first body token, all 18 hot requests hit cache, answers indistinguishable from official.
• ByteDance's speed requests were all valid, with a 2.12 s first body token: ByteDance DeepSeek V4.1 Flash review: answers indistinguishable from official, input counts don't match.
• Baidu also lost many speed requests, to no_available_channel: Baidu DeepSeek V4.1 Flash review: tool calling on par with official, frequent no-available-channel pulls down answer quality and speed.

Sources

1. DeepSeek V4.1 Flash 1017 model page (AIHubMix)
2. AIHubMix Provider Benchmark Standard v2.0 (AIHubMix docs)
3. Quick start (AIHubMix docs)
4. Models (AIHubMix)
5. DeepSeek API documentation (DeepSeek)