Alibaba Cloud DeepSeek V4.1 Flash review: 1.34s first body token, all 18 hot requests hit cache, answers indistinguishable from official
In the 2026-11-03/04 APB Full run, Alibaba Cloud's four scores are Q 41.75, F 89.78, P 88.58 and R 100, with an overall of 73.38. That is only 0.37 from the DeepSeek official channel's overall, and the 95% interval of the Q gap includes 0: the two cannot be told apart.
On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Profile covers Alibaba Cloud only: what each section measured and how it came out.
The numbers below are scores, not a ranking or a provider recommendation. The four scores, Q, F, P and R, come first; the overall is a summary index, not a certification, and it has no confidence interval. The scoring rules are defined in AIHubMix Provider Benchmark Standard v2.0.
Scores across six providers

| Provider | Q quality | F function | P speed | K cache | R stability | Overall |
|---|---|---|---|---|---|---|
| ByteDance | 42.05 | 74.98 | 84.37 | 93.83 | 100 | 66.37 |
| Baidu | 38.00 | 88.39 | 37.77 | 68.85 | 100 | 56.48 |
| Alibaba Cloud (this post) | 41.75 | 89.78 | 88.58 | 99.37 | 100 | 73.38 |
| DeepSeek official channel | 42.45 | 94.00 | 86.14 | 98.38 | 100 | 73.76 |
| Fireworks | 41.42 | 71.78 | 79.51 | 60.68 | 100 | 64.10 |
| Pareto | 44.58 | 93.42 | 57.69 | 27.58 | 100 | 68.72 |
Providers are listed in a fixed order, not a ranking. K is a sub-score of P, accounting for 20% of it, and is listed separately for reference. The overall score is computed with the composition rules of AIHubMix Provider Benchmark Standard, which also take in the comparison with the official channel and evidence completeness; it is not a direct weighting of the four scores in the table. It is a summary index with no confidence interval. The DeepSeek official channel and Alibaba Cloud differ by only 0.37 overall, which is not a distinguishable difference.
At a glance
- Answer quality: Q 41.75, with BFCL 87.00%, HLE 13.29% and AA-LCR 53.60%. The gap to the official channel is −0.71 percentage points, with a 95% interval of −1.72 to +0.26: not distinguishable.
- Function support: F 89.78; 155 of 177 counted checks passed. An invalid tool schema returned 503 instead of a parameter error, over-context did not return the documented error type, and an orphan tool result returned 200.
- Response speed: P 88.58. Median first body token 1.34 s at single concurrency and 1.35 s at 10-way, TPOT 4.69 ms; all 180 speed requests were valid.
- Cache: K 99.37. All 18 hot requests at 8k and 32k hit, and no cold control read the cache.
- Service stability: R 100. All 300 dedicated requests succeeded on the first attempt.
Test setup
| Item | Detail |
|---|---|
| Model | DeepSeek V4.1 Flash 1017 |
| Requested model ID | alicloud-deepseek-v4.1-flash-1017 |
| Returned model name | deepseek-v4.1-flash-1017 |
| Access route | Through AIHubMix with a provider-specific model ID, not a direct connection to the provider's account |
| Parameters | Fixed APB v2 request profile, temperature 1.0, max_tokens 16,384, 300 s timeout, no automatic retry |
| Test profile | The five providers other than Pareto shared one fixed APB v2 profile for Q/P/K/R; F also checked its parameter contract |
| Time | 2026-11-03 18:07 – 11-04 16:33; cache requests 11-04 22:10–22:28; all UTC+8. All six providers shared one Mac mini, one network and one account. |
| Samples | Q 4,250 question-instances; F 180 checks; P 180 requests; R 300; K 30 |
Requests used the AIHubMix quick start route with a provider-specific model ID; the other providers' IDs are listed on the models page. The model's own parameters are described in DeepSeek's API documentation.
Q — answer quality: 41.75
Answer quality draws on three suites: 500 BFCL questions answered 3 times each, 250 HLE questions answered 10 times each, and 25 AA-LCR questions answered 10 times each. BFCL is scored by automatic rules; HLE and AA-LCR are judged right or wrong by the judge model deepseek-flash. All six providers received the same questions, and the three suites are pooled into one accuracy. Ten answers were removed because the question could not be scored, leaving 4,240 scored question-instances.
| Suite | Questions × rounds | Correct | Accuracy | Service failures |
|---|---|---|---|---|
| BFCL tool calls, auto-scored | 500 × 3 | 1,305/1,500 | 87.00% | 0 |
| HLE hard QA, judge-scored | 250 × 10 | 331/2,490 | 13.29% | 8 |
| AA-LCR long-doc reasoning, judge-scored | 25 × 10 | 134/250 | 53.60% | 0 |
| Total · 95% interval 39.56%–43.85% | — | 1,770/4,240 | 41.75% | 8 |
Service failures (failed requests or truncated output) stay in the denominator as wrong answers: HLE output hit the max_tokens ceiling 8 times.
| BFCL category | Passed | Pass rate |
|---|---|---|
| simple_python: one function, right call and arguments | 332/360 | 92.22% |
| multiple: pick the right one of several functions | 270/300 | 90.00% |
| parallel: one function called several times at once | 273/300 | 91.00% |
| parallel_multiple: several functions at once | 212/240 | 88.33% |
| irrelevance: no relevant function, so no call | 113/150 | 75.33% |
| ahm_multi_turn: three-stage tool chain | 105/150 | 70.00% |
Each BFCL question was answered 3 times; the table sums the 3 rounds.
Compared with the other five


| Provider | BFCL | HLE | AA-LCR | Combined | Vs official |
|---|---|---|---|---|---|
| ByteDance | 85.20% | 15.34% | 49.20% | 42.05% | −0.40 (−1.46 to +0.66), not distinguishable |
| Baidu | 86.40% | 9.36% | 32.80% | 38.00% | −4.46 (−5.78 to −3.16), below official |
| Alibaba Cloud (this post) | 87.00% | 13.29% | 53.60% | 41.75% | −0.71 (−1.72 to +0.26), not distinguishable |
| DeepSeek official channel | 86.13% | 15.06% | 53.20% | 42.45% | Baseline |
| Fireworks | 85.53% | 14.38% | 46.00% | 41.42% | −1.04 (−2.14 to +0.05), not distinguishable |
| Pareto | 85.27% | 19.32% | 52.00% | 44.58% | +2.12 (−0.02 to +4.18), not distinguishable |
Vs official: the accuracy gap in percentage points, with the 95% interval in brackets; an interval that crosses 0 is recorded as not distinguishable. The HLE and AA-LCR judge, deepseek-flash, belongs to the same DeepSeek family as the tested model and was not calibrated, so its judgments may be lenient.
F — function support: 89.78
The function suite looks only at whether interface behaviour matches the protocol, not at whether the model answers well. Sixty cases in six groups ran once in each of three windows: 180 checks, judged against the format each provider documents. Checks that were not applicable, invalid or unobserved leave the denominator. Alibaba Cloud passed 155 of 177 counted checks and scored 89.78 on the group weights.
| Case group | Weight | Passed | Excluded | Group score |
|---|---|---|---|---|
| Streaming and message protocol | 20% | 30/30 | — | 100.00 |
| Tool interface | 20% | 30/30 | — | 100.00 |
| Output constraints and parameters | 20% | 30/30 | — | 100.00 |
| Context and state | 15% | 26/30 | — | 86.67 |
| Errors and recovery | 15% | 18/30 | — | 60.00 |
| Metering and cache protocol | 10% | 21/27 | 3 | 77.78 |
| Total · weighted score | — | 155/177 | 3 | 89.78 |
The weighted score applies the group weights; it is not simply passed checks divided by 177.
| Failed case | Windows | Reason |
|---|---|---|
| Over context · context and state | 3/3 | An error, but not the documented context-length error |
| Malformed messages · errors and recovery | 3/3 | 400 with an empty body, no structured error |
| Invalid tool schema · errors and recovery | 3/3 | Returned 503 instead of a parameter error |
| Orphan tool result · errors and recovery | 3/3 | Returned 200 instead of a parameter error |
| Malformed stream_options · errors and recovery | 3/3 | 400 with an empty body, no structured error |
| Reasoning-token metering · metering and cache protocol | 3/3 | No reasoning-token count returned |
| Tool definitions counted in input · metering and cache protocol | 3/3 | Input tokens did not rise when tool definitions were added |
| Isolation between requests · context and state | 1/3 | Step 1 did not reply only STORED, so isolation could not be checked |
“Windows” is how many of the 3 windows failed the check.
| Status | Case | Count | Reason |
|---|---|---|---|
| Not applicable | Explicit cache_control | 3 | Registered in advance as unsupported by this interface |
These checks leave the denominator; they count as neither a pass nor a failure.
The malformed-messages and malformed-stream-options error cases failed on several providers: they returned 400 with an empty body, so the client gets no structured error explanation. Reasoning-token metering behaved differently from provider to provider.
Compared with the other five

| Provider | Passed | Failed | Excluded | Weighted score |
|---|---|---|---|---|
| ByteDance | 124 | 50 | 6 | 74.98 |
| Baidu | 148 | 26 | 6 | 88.39 |
| Alibaba Cloud (this post) | 155 | 22 | 3 | 89.78 |
| DeepSeek official channel | 137 | 10 | 33 | 94.00 |
| Fireworks | 122 | 55 | 3 | 71.78 |
| Pareto | 29 | 4 | 147 | 93.42 |
P — response speed: 88.58
Each request carried about 10,000 input tokens and asked for about 1,100 output tokens of body text. In each of three windows, 30 requests ran at single concurrency and 30 at 10-way concurrency: 180 requests in all.
| Item | Detail |
|---|---|
| First body token (TTFA) | Median 1.34 s single, 1.35 s at 10-way |
| Generation speed (TPOT) | 4.63 ms/token single, 4.69 ms/token at 10-way |
| Effective throughput | 1,313.6 token/s at 10-way |
| Valid requests | 90/90 single, 90/90 at 10-way |
| Sub-scores | TTFA 59.65, TPOT 100.00, throughput 100.00; TTFA and TPOT take the lower tier, weighted 35% / 35% / 30%: response score 85.88 |
| P score | Response score 85.88 × 80% + cache K 99.37 × 20% = 88.58 |
TPOT is estimated from streamed chunks, not measured per token on the hardware. Throughput is valid output tokens at 10-way concurrency divided by batch wall-clock time.
Compared with the other five

| Provider | Valid requests | TTFA single | TTFA 10-way | TPOT (slower) | Throughput tok/s | Response |
|---|---|---|---|---|---|---|
| ByteDance | 180/180 | 2.12 s | 2.10 s | 3.43 ms | 1,250 | 82.00 |
| Baidu | 118/180 | 2.26 s | ∞ | 50.00 ms | 1,329 | 30.00 |
| Alibaba Cloud (this post) | 180/180 | 1.34 s | 1.35 s | 4.69 ms | 1,314 | 85.88 |
| DeepSeek official channel | 165/180 | 1.87 s | 1.72 s | 2.84 ms | 1,546 | 83.08 |
| Fireworks | 179/180 | 1.63 s | 1.64 s | 3.29 ms | 1,505 | 84.22 |
| Pareto | 110/180 | 5.85 s | 6.13 s | 4.63 ms | 519 | 65.22 |
All six shared one machine, network and account. The three windows were scheduled provider by provider, so collection periods do not fully overlap; small differences should not be over-read. ∞ means more than half the requests were invalid, so the median counts as infinite.
K — cache: 99.37
Each group sends five requests. The first carries a fixed prefix of about 8k or 32k tokens and seeds the cache. Requests 2–4 keep the prefix and change only the final question, at gaps of 5 s, 30 s and 120 s. The fifth uses a new prefix as a cold control. Three windows × two lengths make 30 requests. K is the cache tokens read by hot requests divided by the tokens that should be reusable under the rules, with the two lengths weighted equally; K is 20% of P.
| Window | 1 · seed | 2 · +5 s | 3 · +30 s | 4 · +120 s | 5 · cold |
|---|---|---|---|---|---|
| 8k prefix · hot hits 9/9 · read 71,424 of 72,270 reusable · 98.83 | — | — | — | — | — |
| Window 1 | 0 | 7,936 | 7,936 | 7,936 | 0 |
| Window 2 | 0 | 7,936 | 7,936 | 7,936 | 0 |
| Window 3 | 0 | 7,936 | 7,936 | 7,936 | 0 |
| 32k prefix · hot hits 9/9 · read 288,000 of 288,279 reusable · 99.90 | — | — | — | — | — |
| Window 1 | 0 | 32,000 | 32,000 | 32,000 | 0 |
| Window 2 | 0 | 32,000 | 32,000 | 32,000 | 0 |
| Window 3 | 0 | 32,000 | 32,000 | 32,000 | 0 |
| K (8k and 32k weighted equally) | — | — | — | — | 99.37 |
Values are cache-read tokens reported by the server; “Not returned” means the field was absent from the response.
All nine 8k hot requests hit, each reading 7,936 tokens; all nine 32k hot requests hit, each reading 32,000 tokens. Cold requests read no cache, so hot and cold separate cleanly.
Compared with the other five

| Provider | 8k hits | 8k | 32k hits | 32k | K |
|---|---|---|---|---|---|
| ByteDance | 9/9 | 98.84 | 8/9 | 88.81 | 93.83 |
| Baidu | 5/9 | 49.60 | 8/9 | 88.10 | 68.85 |
| Alibaba Cloud (this post) | 9/9 | 98.83 | 9/9 | 99.90 | 99.37 |
| DeepSeek official channel | 9/9 | 97.25 | 9/9 | 99.51 | 98.38 |
| Fireworks | 3/9 | 32.81 | 8/9 | 88.54 | 60.68 |
| Pareto | 2/9 | 21.89 | 3/9 | 33.27 | 27.58 |
R — service stability: 100
The 300 requests fall into four types: short text, long input, tool calls and multi-turn history. They ran at single concurrency across three windows. Only whether the first attempt returned a valid, complete response is checked; answers are not graded and nothing is retried. Failures in the answer-quality and speed sections are counted in Q and P, not again in R.
| Request type | First-attempt success |
|---|---|
| Short text | 120/120 |
| 10k long input | 90/90 |
| Tool calls | 60/60 |
| Multi-turn history | 30/30 |
| Total · 95% interval 98.78%–100% | 300/300 |
All 300 requests returned a valid, complete response on the first attempt. R = 100 describes these 300 dedicated requests only; it does not mean the whole chain had no failures. The lower bound of the 95% interval is 98.78%: 300 successes out of 300 can only say the failure rate is likely below 1.22%.
Compared with the other five
| Provider | First-attempt success | 95% interval |
|---|---|---|
| ByteDance | 300/300 | 98.78%–100% |
| Baidu | 300/300 | 98.78%–100% |
| Alibaba Cloud (this post) | 300/300 | 98.78%–100% |
| DeepSeek official channel | 300/300 | 98.78%–100% |
| Fireworks | 300/300 | 98.78%–100% |
| Pareto | 300/300 | 98.78%–100% |
R covers only these 300 dedicated requests; failures in the answer-quality and speed sections are recorded in Q and P and are not counted again in R.
FAQ
Is the 0.37 overall gap between Alibaba Cloud and the official channel meaningful?
No. The standard treats it as not a distinguishable difference, and the overall has no confidence interval. On answer quality, the Q gap is −0.71 percentage points with a 95% interval of −1.72 to +0.26, which includes 0.
What does the Q score combine?
Three suites: BFCL 500 questions × 3 rounds, scored automatically; HLE 250 × 10 and AA-LCR 25 × 10, judged by the model deepseek-flash. Combined accuracy is 41.75% over 4,240 scored question-instances.
Why is F 89.78 when 155 of 177 checks passed?
The score applies group weights rather than a simple pass ratio. Streaming, tool interface and output constraints each scored 100.00; context and state 86.67; errors and recovery 60.00; metering and cache protocol 77.78.
Why is R 100 if other sections had failed requests?
R covers only the 300 dedicated stability requests. Failures in the answer-quality and speed sections are recorded in Q and P and are not counted again in R.
Is K 99.37 a cache hit rate?
It is the cache tokens read on hot requests divided by the tokens that should be reusable, with 8k and 32k weighted half each. The 8k prefix scored 98.83 and the 32k prefix 99.90; all 18 hot requests hit.
Does a 1.34 s first token mean the whole answer arrives in 1.34 s?
No. It is the median wait for the first body token at single concurrency. Each request asked for about 1,100 output tokens, and TPOT in the slower tier was 4.69 ms per token.
Keep reading: the DeepSeek V4.1 Flash series
- Alibaba Cloud's overall is only 0.37 from the official channel; the baseline's own run is in DeepSeek official channel DeepSeek V4.1 Flash review: comparison baseline, 15 invalid requests at single concurrency.
- ByteDance also hit 9 of 9 at 8k but reported input tokens that miss the native count: ByteDance DeepSeek V4.1 Flash review: answers indistinguishable from official, input counts don't match.
- Baidu used the same suite but lost many requests to no_available_channel: Baidu DeepSeek V4.1 Flash review: tool calling on par with official, frequent no-available-channel pulls down answer quality and speed.
- Fireworks reached a 1.63 s first body token but hit only 3 of 9 at 8k: Fireworks DeepSeek V4.1 Flash review: stop requests return empty body, 8k cache hot requests hit 3 of 9.
- Pareto's first body token took 5.85 s at single concurrency, and its cache K was 27.58: Pareto DeepSeek V4.1 Flash review: only 5 hits in 18 hot requests, answers indistinguishable from official.
Sources
- DeepSeek V4.1 Flash 1017 model page (AIHubMix)
- AIHubMix Provider Benchmark Standard v2.0 (AIHubMix docs)
- Quick start (AIHubMix docs)
- Models (AIHubMix)
- DeepSeek API documentation (DeepSeek)