DeepSeek official channel DeepSeek V4.1 Flash review: comparison baseline, 15 invalid requests at single concurrency
In the 2026-11-03/04 APB Full run, the DeepSeek official channel is the answer-quality baseline. Its four scores are Q 42.45, F 94.00, P 86.14 and R 100, with an overall of 73.76, only 0.37 from Alibaba Cloud: not distinguishable. Fifteen single-concurrency speed requests in window 1 were invalid.
On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Profile covers the DeepSeek official channel only: what each section measured and how it came out.
The numbers below are scores, not a ranking or a provider recommendation. The four scores, Q, F, P and R, come first; the overall is a summary index, not a certification, and it has no confidence interval. The scoring rules are defined in AIHubMix Provider Benchmark Standard v2.0.
Scores across six providers

| Provider | Q quality | F function | P speed | K cache | R stability | Overall |
|---|---|---|---|---|---|---|
| ByteDance | 42.05 | 74.98 | 84.37 | 93.83 | 100 | 66.37 |
| Baidu | 38.00 | 88.39 | 37.77 | 68.85 | 100 | 56.48 |
| Alibaba Cloud | 41.75 | 89.78 | 88.58 | 99.37 | 100 | 73.38 |
| DeepSeek official channel (this post) | 42.45 | 94.00 | 86.14 | 98.38 | 100 | 73.76 |
| Fireworks | 41.42 | 71.78 | 79.51 | 60.68 | 100 | 64.10 |
| Pareto | 44.58 | 93.42 | 57.69 | 27.58 | 100 | 68.72 |
Providers are listed in a fixed order, not a ranking. K is a sub-score of P, accounting for 20% of it, and is listed separately for reference. The overall score is computed with the composition rules of AIHubMix Provider Benchmark Standard, which also take in the comparison with the official channel and evidence completeness; it is not a direct weighting of the four scores in the table. It is a summary index with no confidence interval. The DeepSeek official channel and Alibaba Cloud differ by only 0.37 overall, which is not a distinguishable difference.
At a glance
- Answer quality: Q 42.45, with BFCL 86.13%, HLE 15.06% and AA-LCR 53.20%. The other five providers' Q is compared against this one. HLE output hit the max_tokens ceiling 7 times, counted as wrong.
- Function support: F 94.00; 137 of 147 counted checks passed. Another 27 checks that depend on exact token counts were judged invalid and left out of the denominator.
- Response speed: P 86.14. In window 1, 15 of 30 single-concurrency requests were invalid (12 × 503, 2 at the deadline, 1 read timeout). With invalid requests counted as infinite, the median first body token is 1.87 s (1.72 s valid-only); 10-way 1.72 s, TPOT 2.84 ms.
- Cache: K 98.38. All 18 hot requests at 8k and 32k hit, and no cold control read the cache.
- Service stability: R 100. All 300 dedicated requests succeeded on the first attempt.
Test setup
| Item | Detail |
|---|---|
| Model | DeepSeek V4.1 Flash 1017 |
| Requested model ID | deepseek-flash |
| Returned model name | deepseek-flash |
| Access route | Through AIHubMix's DeepSeek official provider route, not a direct connection to the official API |
| Parameters | Fixed APB v2 request profile, temperature 1.0, max_tokens 16,384, 300 s timeout, no automatic retry |
| Test profile | The five providers other than Pareto shared one fixed APB v2 profile for Q/P/K/R; F also checked its parameter contract |
| Time | 2026-11-03 18:07 – 11-04 16:33; cache requests 11-04 22:10–22:27; all UTC+8. All six providers shared one Mac mini, one network and one account. |
| Samples | Q 4,250 question-instances; F 180 checks; P 180 requests; R 300; K 30 |
Requests used the AIHubMix quick start route with a provider-specific model ID; the other providers' IDs are listed on the models page. The model's own parameters are described in DeepSeek's API documentation.
Q — answer quality: 42.45
Answer quality draws on three suites: 500 BFCL questions answered 3 times each, 250 HLE questions answered 10 times each, and 25 AA-LCR questions answered 10 times each. BFCL is scored by automatic rules; HLE and AA-LCR are judged right or wrong by the judge model deepseek-flash. All six providers received the same questions, and the three suites are pooled into one accuracy. Ten answers were removed because the question could not be scored, leaving 4,240 scored question-instances.
| Suite | Questions × rounds | Correct | Accuracy | Service failures |
|---|---|---|---|---|
| BFCL tool calls, auto-scored | 500 × 3 | 1,292/1,500 | 86.13% | 0 |
| HLE hard QA, judge-scored | 250 × 10 | 375/2,490 | 15.06% | 7 |
| AA-LCR long-doc reasoning, judge-scored | 25 × 10 | 133/250 | 53.20% | 0 |
| Total · 95% interval 40.08%–44.73% | — | 1,800/4,240 | 42.45% | 7 |
Service failures (failed requests or truncated output) stay in the denominator as wrong answers: HLE output hit the max_tokens ceiling 7 times.
| BFCL category | Passed | Pass rate |
|---|---|---|
| simple_python: one function, right call and arguments | 334/360 | 92.78% |
| multiple: pick the right one of several functions | 268/300 | 89.33% |
| parallel: one function called several times at once | 271/300 | 90.33% |
| parallel_multiple: several functions at once | 207/240 | 86.25% |
| irrelevance: no relevant function, so no call | 113/150 | 75.33% |
| ahm_multi_turn: three-stage tool chain | 99/150 | 66.00% |
Each BFCL question was answered 3 times; the table sums the 3 rounds.
Compared with the other five


| Provider | BFCL | HLE | AA-LCR | Combined | Vs official |
|---|---|---|---|---|---|
| ByteDance | 85.20% | 15.34% | 49.20% | 42.05% | −0.40 (−1.46 to +0.66), not distinguishable |
| Baidu | 86.40% | 9.36% | 32.80% | 38.00% | −4.46 (−5.78 to −3.16), below official |
| Alibaba Cloud | 87.00% | 13.29% | 53.60% | 41.75% | −0.71 (−1.72 to +0.26), not distinguishable |
| DeepSeek official channel (this post) | 86.13% | 15.06% | 53.20% | 42.45% | Baseline |
| Fireworks | 85.53% | 14.38% | 46.00% | 41.42% | −1.04 (−2.14 to +0.05), not distinguishable |
| Pareto | 85.27% | 19.32% | 52.00% | 44.58% | +2.12 (−0.02 to +4.18), not distinguishable |
Vs official: the accuracy gap in percentage points, with the 95% interval in brackets; an interval that crosses 0 is recorded as not distinguishable. The HLE and AA-LCR judge, deepseek-flash, belongs to the same DeepSeek family as the tested model and was not calibrated, so its judgments may be lenient.
F — function support: 94.00
The function suite looks only at whether interface behaviour matches the protocol, not at whether the model answers well. Sixty cases in six groups ran once in each of three windows: 180 checks, judged against the format each provider documents. Checks that were not applicable, invalid or unobserved leave the denominator. DeepSeek official channel passed 137 of 147 counted checks and scored 94.00 on the group weights.
| Case group | Weight | Passed | Excluded | Group score |
|---|---|---|---|---|
| Streaming and message protocol | 20% | 27/27 | 3 | 100.00 |
| Tool interface | 20% | 30/30 | — | 100.00 |
| Output constraints and parameters | 20% | 30/30 | — | 100.00 |
| Context and state | 15% | 14/15 | 15 | 93.33 |
| Errors and recovery | 15% | 24/30 | — | 80.00 |
| Metering and cache protocol | 10% | 12/15 | 15 | 80.00 |
| Total · weighted score | — | 137/147 | 33 | 94.00 |
The weighted score applies the group weights; it is not simply passed checks divided by 147.
| Failed case | Windows | Reason |
|---|---|---|
| Malformed messages · errors and recovery | 3/3 | 400 with an empty body, no structured error |
| Malformed stream_options · errors and recovery | 3/3 | 400 with an empty body, no structured error |
| Reasoning-token metering · metering and cache protocol | 3/3 | No reasoning-token count returned |
| Isolation between requests · context and state | 1/3 | Step 1 did not reply only STORED, so isolation could not be checked |
“Windows” is how many of the 3 windows failed the check.
| Status | Case | Count | Reason |
|---|---|---|---|
| Unobserved | Streaming stop sequence | 3 | The request for this step was not sent after the previous step, so it could not be judged |
| Invalid | 9 cases (listed below the table) | 27 | The pre-fixed native-count template did not match this channel's usage, so exact-count checks were invalid |
| Not applicable | Explicit cache_control | 3 | Registered in advance as unsupported by this interface |
These checks leave the denominator; they count as neither a pass nor a failure.
The 9 invalid cases: 10k long input, within context limit, 32k long input, 100k long input, over context, native input count, tool definitions counted in input, input count stable with cache, streaming vs non-streaming input.
The malformed-messages and malformed-stream-options error cases failed on several providers: they returned 400 with an empty body, so the client gets no structured error explanation. Reasoning-token metering behaved differently from provider to provider.
Compared with the other five

| Provider | Passed | Failed | Excluded | Weighted score |
|---|---|---|---|---|
| ByteDance | 124 | 50 | 6 | 74.98 |
| Baidu | 148 | 26 | 6 | 88.39 |
| Alibaba Cloud | 155 | 22 | 3 | 89.78 |
| DeepSeek official channel (this post) | 137 | 10 | 33 | 94.00 |
| Fireworks | 122 | 55 | 3 | 71.78 |
| Pareto | 29 | 4 | 147 | 93.42 |
P — response speed: 86.14
Each request carried about 10,000 input tokens and asked for about 1,100 output tokens of body text. In each of three windows, 30 requests ran at single concurrency and 30 at 10-way concurrency: 180 requests in all.
| Item | Detail |
|---|---|
| First body token (TTFA) | Median 1.87 s single (valid-only 1.72 s), 1.72 s at 10-way |
| Generation speed (TPOT) | 2.84 ms/token single, 2.67 ms/token at 10-way |
| Effective throughput | 1,546.0 token/s at 10-way |
| Valid requests | 75/90 single (window 1: 15/30), 90/90 at 10-way; invalid: 12 returned 503 service_unavailable_error, 2 hit the deadline unfinished, 1 read timeout |
| Sub-scores | TTFA 51.65, TPOT 100.00, throughput 100.00; TTFA and TPOT take the lower tier, weighted 35% / 35% / 30%: response score 83.08 |
| P score | Response score 83.08 × 80% + cache K 98.38 × 20% = 86.14 |
One valid request waited 106.45 s for its first body token. Invalid requests are scored as infinite, so the medians in the table can be higher than valid-only medians. TPOT is estimated from streamed chunks, not measured per token on the hardware. Throughput is valid output tokens at 10-way concurrency divided by batch wall-clock time.
Compared with the other five

| Provider | Valid requests | TTFA single | TTFA 10-way | TPOT (slower) | Throughput tok/s | Response |
|---|---|---|---|---|---|---|
| ByteDance | 180/180 | 2.12 s | 2.10 s | 3.43 ms | 1,250 | 82.00 |
| Baidu | 118/180 | 2.26 s | ∞ | 50.00 ms | 1,329 | 30.00 |
| Alibaba Cloud | 180/180 | 1.34 s | 1.35 s | 4.69 ms | 1,314 | 85.88 |
| DeepSeek official channel (this post) | 165/180 | 1.87 s | 1.72 s | 2.84 ms | 1,546 | 83.08 |
| Fireworks | 179/180 | 1.63 s | 1.64 s | 3.29 ms | 1,505 | 84.22 |
| Pareto | 110/180 | 5.85 s | 6.13 s | 4.63 ms | 519 | 65.22 |
All six shared one machine, network and account. The three windows were scheduled provider by provider, so collection periods do not fully overlap; small differences should not be over-read. ∞ means more than half the requests were invalid, so the median counts as infinite.
K — cache: 98.38
Each group sends five requests. The first carries a fixed prefix of about 8k or 32k tokens and seeds the cache. Requests 2–4 keep the prefix and change only the final question, at gaps of 5 s, 30 s and 120 s. The fifth uses a new prefix as a cold control. Three windows × two lengths make 30 requests. K is the cache tokens read by hot requests divided by the tokens that should be reusable under the rules, with the two lengths weighted equally; K is 20% of P.
| Window | 1 · seed | 2 · +5 s | 3 · +30 s | 4 · +120 s | 5 · cold |
|---|---|---|---|---|---|
| 8k prefix · hot hits 9/9 · read 70,272 of 72,261 reusable · 97.25 | — | — | — | — | — |
| Window 1 | 0 | 7,808 | 7,808 | 7,808 | 0 |
| Window 2 | 0 | 7,808 | 7,808 | 7,808 | 0 |
| Window 3 | 0 | 7,808 | 7,808 | 7,808 | 0 |
| 32k prefix · hot hits 9/9 · read 286,848 of 288,270 reusable · 99.51 | — | — | — | — | — |
| Window 1 | 0 | 31,872 | 31,872 | 31,872 | 0 |
| Window 2 | 0 | 31,872 | 31,872 | 31,872 | 0 |
| Window 3 | 0 | 31,872 | 31,872 | 31,872 | 0 |
| K (8k and 32k weighted equally) | — | — | — | — | 98.38 |
Values are cache-read tokens reported by the server; “Not returned” means the field was absent from the response.
All nine 8k hot requests hit, each reading 7,808 tokens; all nine 32k hot requests hit, each reading 31,872 tokens. Cold requests read no cache, so hot and cold separate cleanly.
Compared with the other five

| Provider | 8k hits | 8k | 32k hits | 32k | K |
|---|---|---|---|---|---|
| ByteDance | 9/9 | 98.84 | 8/9 | 88.81 | 93.83 |
| Baidu | 5/9 | 49.60 | 8/9 | 88.10 | 68.85 |
| Alibaba Cloud | 9/9 | 98.83 | 9/9 | 99.90 | 99.37 |
| DeepSeek official channel (this post) | 9/9 | 97.25 | 9/9 | 99.51 | 98.38 |
| Fireworks | 3/9 | 32.81 | 8/9 | 88.54 | 60.68 |
| Pareto | 2/9 | 21.89 | 3/9 | 33.27 | 27.58 |
R — service stability: 100
The 300 requests fall into four types: short text, long input, tool calls and multi-turn history. They ran at single concurrency across three windows. Only whether the first attempt returned a valid, complete response is checked; answers are not graded and nothing is retried. Failures in the answer-quality and speed sections are counted in Q and P, not again in R.
| Request type | First-attempt success |
|---|---|
| Short text | 120/120 |
| 10k long input | 90/90 |
| Tool calls | 60/60 |
| Multi-turn history | 30/30 |
| Total · 95% interval 98.78%–100% | 300/300 |
All 300 requests returned a valid, complete response on the first attempt. R = 100 describes these 300 dedicated requests only; it does not mean the whole chain had no failures. The lower bound of the 95% interval is 98.78%: 300 successes out of 300 can only say the failure rate is likely below 1.22%.
Compared with the other five
| Provider | First-attempt success | 95% interval |
|---|---|---|
| ByteDance | 300/300 | 98.78%–100% |
| Baidu | 300/300 | 98.78%–100% |
| Alibaba Cloud | 300/300 | 98.78%–100% |
| DeepSeek official channel (this post) | 300/300 | 98.78%–100% |
| Fireworks | 300/300 | 98.78%–100% |
| Pareto | 300/300 | 98.78%–100% |
R covers only these 300 dedicated requests; failures in the answer-quality and speed sections are recorded in Q and P and are not counted again in R.
FAQ
Is this a direct connection to DeepSeek's API?
No. The requests go through AIHubMix's DeepSeek official provider route; they are not sent directly to the official API.
Why is the DeepSeek official channel the Q baseline?
The other five providers' answer-quality results are expressed as gaps from this channel, with 95% intervals. Being the baseline does not make it a ranking.
What happened in the first single-concurrency speed window?
Fifteen of 30 requests were invalid: 12 returned 503 service_unavailable_error, 2 reached the deadline unfinished and 1 hit a read timeout. Invalid requests count as infinite, so the median is 1.87 s; with valid requests only it is 1.72 s.
What are the 27 invalid function checks?
Nine cases that depend on exact token counts, such as long inputs, the context limit, native input counts and input consistency with cache, each in three windows. The test side's pre-fixed native-count template did not match the usage this channel reported, so these checks were left out of the denominator rather than scored as failures.
Does R 100 guarantee the route never fails?
No. R covers only the 300 dedicated stability requests, and failures in the answer-quality and speed runs are recorded in Q and P. With 300 of 300 successful, the lower bound of the 95% interval is 98.78%, so the failure rate is likely below 1.22%, not proven to be zero.
How does the cache test separate hot reads from cold?
Each group seeds an 8k or 32k prefix, repeats it at 5, 30 and 120 seconds with a new final question, then sends a new prefix as a cold control. Both lengths hit 9 of 9 hot requests, and every cold control read zero cached tokens.
Keep reading: the DeepSeek V4.1 Flash series
- Baidu is the only provider whose Q interval excludes 0 against this baseline: Baidu DeepSeek V4.1 Flash review: tool calling on par with official, frequent no-available-channel pulls down answer quality and speed.
- Alibaba Cloud's overall is only 0.37 from this channel: Alibaba Cloud DeepSeek V4.1 Flash review: 1.34s first body token, all 18 hot requests hit cache, answers indistinguishable from official.
- ByteDance's Q gap to this baseline is −0.40, and its metering checks failed: ByteDance DeepSeek V4.1 Flash review: answers indistinguishable from official, input counts don't match.
- Fireworks' Q gap to this baseline is −1.04, with empty replies on stop requests: Fireworks DeepSeek V4.1 Flash review: stop requests return empty body, 8k cache hot requests hit 3 of 9.
- Pareto's Q gap to this baseline is +2.12, still not distinguishable: Pareto DeepSeek V4.1 Flash review: only 5 hits in 18 hot requests, answers indistinguishable from official.
Sources
- DeepSeek V4.1 Flash 1017 model page (AIHubMix)
- AIHubMix Provider Benchmark Standard v2.0 (AIHubMix docs)
- Quick start (AIHubMix docs)
- Models (AIHubMix)
- DeepSeek API documentation (DeepSeek)