anthropic's new haiku is the cheapest, fastest model in its family, and it now scores close to models that cost several times more. this report gathers every benchmark published about it so far, explains what each one actually measures in ordinary words, and sets the results against deepseek v4.1 flash, the open-weight model it will be compared with everywhere.

22 matched tests from three labs

what anthropic actually built

a small model, in the sense that matters to a buyer: cheap enough to run a million times a day.

claude haiku 5.5 arrived on 7 october 2026. it is the small member of claude's current generation, sitting under sonnet 5.5 and opus 5.5 the way a compact car sits under a saloon. anthropic describes it as the cheapest, fastest and most capable small model it has ever shipped, and the pricing backs that up. on average, running it costs about 75% less than running its predecessor, haiku 4.5.

small models are not built for the hardest problems. they are built for the million small decisions a business makes every day: sorting incoming messages, pulling a number out of an invoice, deciding whether a complaint needs a human, rewriting a paragraph, or acting as a quick helper inside a larger job. anthropic positions haiku 5.5 for exactly this kind of work, and says it pairs well with its bigger siblings as a subagent, which is a helper process that does one narrow job and reports back.

the technical shape of it is simple to describe. it reads text and images and writes text back. it can hold a million tokens of context in its head, roughly fifteen hundred pages of a4, and it can write a reply of up to 128,000 tokens, close to a hundred thousand words, in one go. it is a reasoning model, meaning it can think before it answers, and it is the first haiku you can tell how hard to think, with five settings from low to max.

claude haiku 5.5figurewhat it means
releasedoct 2026model id claude-haiku-5-5
list price$0.10 / $0.50input and output per million tokens
context1,000,000 tokensabout 1,500 pages of text
max reply128,000 tokensroughly 100,000 words
speed240 t/stokens written per second

the competitor in this report, deepseek v4.1 flash, came out a month earlier, on 10 september 2026. it comes from a chinese lab, and it is a different kind of product: the model itself is open-weight software under the MIT licence, so anyone can download it and run it on their own machines. it is also a much bigger piece of machinery, with 552 billion parameters of which about 16 billion are used per word generated, plus a separate 196 billion parameter memory table that the model can look things up in. it reads images natively, holds a million tokens of context, and can write replies up to 384,000 tokens long.

comparing the two is not quite apples to apples, and that is part of what makes the numbers interesting. one is a cloud service priced by the token. the other is software you can own, with a price that changes depending on the hour of the day you send your request.

how to read a benchmark

three habits will keep you from being misled, and none of them require any technical knowledge.

most benchmarks report a single number, usually a percentage, and a higher number is better. the percentage is the share of tasks the model finished correctly. the tasks are built to be hard in a specific way: write a program that works, read a chart and answer a question about it, fix a broken server, write a clinical note from a medical conversation.

the first habit is to ask who graded the test. a maker's own table is a claim, not a measurement, and every maker picks the tests that flatter it. an independent lab is better, and the two labs used in this report, artificial analysis and vals ai, both run many models through the same harness so the results can be compared. both are used here wherever they exist.

the second habit is to check the version of the test. terminal-bench, the best known test of real computer work, has gone through four versions in about a year. version 2.1 is much easier than version 4.0, which strips out tasks that modern models now solve and gives the model eight hours per task. a score of 90 on the old version and a score of 31 on the new one are not in conflict; they are different exams. anywhere two versions appear in this report, they are labelled.

the third habit is to check the effort setting. both models can be told how hard to think, and a higher setting costs more and usually scores better. anthropic's headline table uses max effort, so the numbers here do too, and the section on the dial shows what the settings are worth.

one more thing about scores in general: differences of a point or two on most of these tests are inside the noise. run the same model twice and the score moves. the gaps worth trusting are the large ones, like the 32 point spread on proofbench later in this report.

{
  "model": "claude-haiku-5-5",
  "benchmark": "Terminal-Bench 4.0",
  "score": 0.392,
  "effort": "max",
  "source": "vendor_table"
}

anthropic's own results

the launch table, test by test, with the plain english version of what each exam is asking.

anthropic published its results at max effort, which makes them directly comparable with deepseek's own maximum-effort table. not every test has a number on both sides, and where one is missing it is shown as blank rather than guessed at.

terminal-bench 4.0: real work in a text-only computer

this is the benchmark that decides whether a model can be trusted with a job and left alone. it gets a computer it can only drive through a command line, which is the plain-text interface developers use, and a list of genuine multi-step tasks: set up a web server, fix a broken code base, analyse a data set, debug a failing service. it has eight hours per task. the score is simply the share of tasks it finished, correctly, without help.

haiku 5.5 scores 39.2%. haiku 4.5, the previous generation, scored 0.0%, which is a striking way of saying the small model has gone from useless to usable at this kind of work. deepseek v4.1 flash reports 31.2% on its own harness, and two independent labs then measured both models, so this is one of the few tests with three separate readings.

terminal-bench 4.0, share of tasks completed, at max effort
runhaiku 5.5deepseek v4.1 flash
the makers' own tables39.2%31.2%
artificial analysis32.8%26.8%
vals ai35.4%19.7%
haiku 5.5 leads in all three runs, by six to sixteen points. sources: anthropic launch table, deepseek technical report, artificial analysis, vals ai.

osworld: driving a computer like a person

the test here is an ordinary desktop screen. the model sees a screenshot and has to click, type, drag and scroll the way a person would, in order to finish jobs like converting a batch of files or filling in a form across two applications. it is the closest thing on the market to measuring whether an AI can be a competent office assistant.

anthropic reports 72.4% for haiku 5.5, but that number gives partial credit for getting some of the way through a task. the stricter version, which only counts tasks where every step ended up right, is 37.1%. the gap between the two is the honest measure of the model: it is good at most of the steps and much less reliable at the whole job. deepseek has published no osworld result, so the chart uses anthropic's other models and openai's two recent models as reference points.

partial credit, steps finished
modelscore
haiku 5.572.4%
gpt-6.1 sol76.6%
sonnet 5.583.9%
opus 5.587.2%
haiku 4.515.7%
strict score, every step correct
modelscore
haiku 5.537.1%
gpt-6.1 sol39.8%
sonnet 5.548.8%
opus 5.553.2%
gpt-6 luna17.1%
osworld 2.1, offline tasks. on partial credit haiku 5.5 comes close to gpt-6.1 sol. on the strict measure, where nothing counts unless the task is fully finished, the gap to the big models opens up and haiku lands at 37.1%. source: anthropic system card, 82 tasks, five attempts each. deepseek publishes no osworld score.

humanity's last exam: the hardest written test

humanity's last exam is a set of questions written by experts across dozens of fields, built specifically to be at the edge of what a model can answer. half the value is the split between answering from the model's own knowledge and answering with tools, which lets it search the web and run calculations.

closed book, haiku 5.5 answers 45.9% of questions correctly, a large jump from the 10.2% of haiku 4.5. deepseek reports 39.1% on its text-only subset. with tools switched on, deepseek reports 63.9% and anthropic reports 57.4% for haiku. those two with-tools numbers come from the makers' own setups, which are not identical, so it is fairer to say the models are close here than to declare a winner.

professional work: gdpval and aa-briefcase

gdpval is built from real tasks taken from professional jobs in law, finance, engineering, medicine and consulting. expert graders compare the outputs the way they would compare the work of two candidates. the score is an elo rating, the same scale used in chess, so 1620 does not mean 1620 out of anything; it means the model consistently beat models rated below it. haiku 5.5 reaches 1620, well above the 1437 of gpt-6 luna and the 735 of haiku 4.5, and below the 1840 of sonnet 5.5. deepseek has published no gdpval result, so the nearest independent read on deepseek's professional work is the vals suite later in this report, where the two are separated by a point or two.

aa-briefcase is a longer and meaner version of the same idea: multi-week projects, thousands of source files, and grading that mixes mechanical checks with a judgement of whether the analysis and the presentation hold up. haiku 5.5 reaches 1578. neither lab nor deepseek has published a briefcase number, which is worth knowing, because briefcase is one of the tests where the bigger models pull clearly ahead.

frontiercode: code good enough to merge

frontiercode asks a question a development team actually cares about. given a real code base, can the model produce a change that is correct, stays inside the requested scope, follows the project's conventions, and passes its tests, so that a reviewer can merge it without repairs? haiku 5.5 scores 46.4% at max effort, at an average cost of about $1.33 per attempt including the attempts that failed. at medium effort it drops to 41.6% and the average attempt costs about thirteen cents. deepseek has no public frontiercode entry.

chartography: reading charts like an analyst

charts are harder for machines than they look, because the answer depends on reading an axis correctly, estimating a position, and knowing the conventions of the chart type. haiku 5.5 scores 46.4% without tools, against 29.1% for luna and 61.6% for sonnet 5.5. deepseek's published 78.9% on this test was produced with tools switched on and a half million token context window, so it is not a like-for-like comparison, and this report does not treat it as one.

the independent scoreboard

artificial analysis runs every model through the same ten tests. the average is a rough but useful summary.

the intelligence index is the closest thing the industry has to a shared report card. it blends ten evaluations covering reasoning, knowledge, mathematics, coding and agentic work, and every model is run through the identical harness at a declared effort setting. on that basis haiku 5.5 scores 43 and deepseek v4.1 flash scores 39.

for context, that puts haiku 5.5 one point behind kimi k3, which is an open-weight model nearly three trillion parameters in size, and ahead of gpt-6 luna, gemini 3.8 flash and glm-5.3 flash. it is still 13 points behind anthropic's own sonnet 5.5, which is a useful reminder that haiku is not being sold as the smartest model in the family, only as the one you can afford to call a million times.

artificial analysis intelligence index v4.3.2, maximum effort
modelindex
sonnet 5.556
kimi k344
haiku 5.543
glm-5.3 flash42
gemini 3.8 flash41
deepseek v4.1 flash39
gpt-6 luna38
the index averages ten separate evaluations. haiku 5.5 sits four points ahead of deepseek v4.1 flash on the same harness.

the one test deepseek clearly wins

automationbench measures repeated office work inside software: create a record in one app, copy the result into another, send a summary to a third, and check that each step actually happened. it is the kind of work that is boring for people and expensive to supervise.

on artificial analysis's independent run, deepseek v4.1 flash scores 68.9% against haiku 5.5's 35.4%. that is the largest lead deepseek holds anywhere in the public record. two honest caveats belong with it. deepseek's own figure on its own harness is lower, at 54.8%, which still leads. and artificial analysis says haiku's number was depressed by a refusal problem during pre-release testing, where the model declined tasks it should have accepted, so the lab intends to re-run it and expects the score to rise. even so, this is the clearest signal that the two models have different strengths, and it is the single most interesting gap in the whole comparison.

knowing what it does not know

aa-omniscience measures something that matters more in practice than any accuracy score: whether a model can tell the difference between knowing something and not knowing it. haiku 5.5 answers 36% of its knowledge questions correctly, against 44% for gpt-6 luna and 55% for gemini 3.8 flash, so it is the least knowledgeable of the three. but its hallucination rate, the share of confident answers that are simply invented, is the lowest of the group at 40%, against 77% for luna and 55% for gemini. in plain terms, when haiku 5.5 does not know, it is more likely to say so.

measureclaude haiku 5.5deepseek v4.1 flash
knows the answer36%not measured
invents an answer40%not measured
tokens written per index task162,16488,574
cost per finished task$0.21$0.27

the last two lines explain a puzzle in the pricing. haiku 5.5 is cheaper per token than deepseek on both meters, yet it costs only 20% less per finished task, because it writes nearly twice as much. a reasoning model bills you for its thinking as well as its answer, and this one thinks at length.

twenty two tests, one lab

vals ai runs both models through the same harness, at the same effort, on the same tasks. this is the closest thing to a fair fight in the public record.

vals ai is an independent evaluation lab. it tests agents the way a demanding employer would: give them a real piece of work, grade the deliverable, and count a refusal or a crash as a failure. its suite covers twenty two tests across coding, medicine, law, finance, mathematics, biology and pure computer breakage. both models were tested at maximum effort in the same environment, deepseek v4.1 flash on 10 september 2026 and haiku 5.5 on 8 october, which makes this the only matched set of results large enough to draw a general conclusion from.

haiku 5.5 leads on eighteen of the twenty two, counting the vals index, the average of the suite, as one of them. deepseek v4.1 flash leads on four, and three of those four leads are small. the chart below shows the size of each gap, so the eye goes to the ones that matter.

each test, matched at maximum effort
testgap
proofbench v1.1+32.0
terminal-bench 4.0+15.7
mysterymechanism+14.0
medcode+10.4
terminal-bench science+10.0
ioi (olympiad coding)+7.0
vibe code bench v1.1+5.7
tax agent bench+5.3
sre bench+5.0
medscribe+4.0
vibe code bench 1-100+4.0
biomysterybench+2.6
vals index (overall)+3.0
legal research bench+1.9
emb (excel models)+1.8
cyberbench v1.1+1.8
programbench+2.0
finance agent v2+0.7
code migration−1.1
public benefits bench−1.1
sage−2.1
harvey legal agent−5.4
the bar length is the gap in percentage points: to the right haiku 5.5 is ahead, to the left deepseek is. deepseek's four wins are narrow, apart from harvey, where both models are weak. source: vals ai, september and october 2026.

now the same results in full, with a one line explanation of what each test is actually asking. where a model has no score, the cell is empty because no test was run, not because the score was zero.

test what it asks, in plain english haiku 5.5 deepseek gap
proofbench v1.1prove a research claim by gathering and checking evidence86.0054.00+32.0
terminal-bench 4.0finish a real multi step job on a text only computer35.3519.70+15.7
mysterymechanismexplain how a mechanism works from a description and a diagram35.1321.17+14.0
medcodeconvert patient records into medical billing codes51.5441.17+10.4
terminal-bench sciencerun scientific research workflows in a terminal10.000.00+10.0
ioicompetitive programming at olympiad level47.2840.28+7.0
vibe code bench v1.1build a small working app from a description90.4484.74+5.7
tax agent benchanswer tax questions from messy client documents31.9226.63+5.3
sre benchfix a broken production system on call5.720.76+5.0
medscribewrite a clinical note from a consultation recording89.5485.50+4.0
vibe code bench 1-100build a one page web app from a prompt20.4116.38+4.0
biomysterybenchsolve a biology mystery from research data70.3767.78+2.6
vals indexaverage of the whole suite54.3151.32+3.0
legal research benchanswer a legal question and cite the sources43.2741.35+1.9
embbuild a complex financial model in excel59.0257.21+1.8
cyberbench v1.1find and fix vulnerabilities in a system75.4873.69+1.8
programbenchrebuild a working program from a compiled file and its docs2.500.50+2.0
finance agent v2complete a finance task in a spreadsheet54.1453.48+0.7
code migrationmove an old code base to a new framework44.5745.62−1.1
public benefits benchfill in a benefits application for a client63.1964.28−1.1
sagesolve maths problems the way a student would, graded by rubric45.7847.88−2.1
harvey legal agentdo legal work as a paralegal, graded by lawyers1.256.67−5.4

three results deserve a plain reading. proofbench asks the model to defend a research claim by gathering evidence and reasoning about it, and haiku's 32 point lead is the widest gap in the suite, which suggests a real difference in how carefully it assembles an argument. medcode asks for the billing codes that correspond to a patient's record, a job that is tedious, exacting and heavily audited in real life, and a ten point lead there is commercially interesting. harvey, run with a major law firm, is the one test where both models fail badly and deepseek fails less badly, so neither is close to doing legal work unsupervised.

two more facts sit underneath the table. vals ai charged $2.99 to test haiku 5.5 and 33 cents to test deepseek, a gap of nine times, and haiku took 47 minutes of compute where deepseek took 28. the quality lead is real but so is the price of it, and the section on cost explains how those two facts reconcile.

what deepseek says about itself

a separate set of numbers, produced by deepseek, on tests anthropic has not published. these are claims, and they are read as claims here.

deepseek's technical report contains a table of nineteen benchmarks at maximum effort. some of them, like terminal-bench 4.0, can be set against anthropic's numbers. others, like gpqa diamond, the hard science exam written by PhD holders, have no published haiku result, so the only honest thing to do is show deepseek's number and note that anthropic has not published one.

the single most interesting number in the table is on terminal-bench version 2.1, where deepseek scores 90.6%. on version 4.0, the same model scores 31.2%. nothing changed in the model. the test changed. version 4.0 removed the tasks that modern models already solve and made the remaining ones longer and stricter, which is why a model that looks like an A student on one version looks like a C student on the next. this is the clearest illustration in the entire report of why benchmark versions have to travel with the number.

the same model, three versions of one test
versionhaiku 5.5deepseek v4.1 flashopus 5.0
version 2.1, the easier exam—90.6%89.1%
version 3.0—30.0%43.3%
version 4.0, the current exam39.2%31.2%51.8%
opus 5.0 is the best rival on deepseek's table. deepseek's 90.6% on the old version and 31.2% on the current one are both accurate. haiku 5.5's 39.2% is from anthropic's launch table on version 4.0. source: deepseek technical report, anthropic.

deepswe, and why the number on the box is not the number you get

deepswe v1.1 is a long software engineering test built from real github issues across many projects, and deepseek reports 74.2%, which would make it the best model in the world at that job. on the independent leaderboard it comes second at 74.2, behind meta's muse spark 1.3 at 75.4, with gpt-6 astra at 74.1 and claude opus 5 at 73.6 behind it. four models inside two points of each other, three of them inside 0.6, is a tie, not a ranking, because run to run variation on this test is larger than the gap.

the more useful detail is that deepseek published its own score across eight different scaffolding setups, which is the software wrapped around the model that does the typing and clicking. the same model, on the same test, scores between 74.2 and 65.5 depending only on which scaffold it uses. that is an 8.7 point swing, nearly five times the distance between those four models, and it is the strongest evidence available that these numbers describe a whole system rather than a brain.

deepswe v1.1 by scaffolding setup, deepseek's own data
scaffoldscore
mini-swe74.2%
dsh minimal72.6%
dsh standard70.5%
claude code69.8%
dsh ptc67.6%
pi66.2%
codex65.6%
opencode65.5%
eight setups, one model, an 8.7 point spread. source: deepseek technical report.

the rest of deepseek's board

the remaining tests show a model with two clear strengths and one clear weakness. it is strong at security, scoring 88.1 on cybergym, a test of finding vulnerabilities, and 90.9 on gpqa diamond, the hard science exam, second only to gpt-5.6 sol among the models on its table. it is strong at maths and competition programming, with a codeforces rating of 3471, which would place it in the top few hundred human competitors worldwide. it is weaker at long horizon agentic work, where opus 5.0 leads it by 20 points on terminal-bench 4.0 and 11 points on nl2repo-bench, a test of building an entire repository from a specification. both models are strong at SaaS automation, but deepseek's automationbench lead over haiku survives into artificial analysis's independent run, so it is probably real.

test what it asks, in plain english deepseek best rival
gpqa diamondgraduate level science questions written by PhD holders90.994.1
cybergymfind vulnerabilities in a running system88.184.5
terminal-bench 2.1older, easier version of the terminal test90.689.1
deepswe v1.1fix real bugs in real open source projects74.274.0
automationbenchdrive office software through a multi step process54.850.3
nl2repo-benchbuild an entire repository from a text specification64.075.3
agent's last examexpert questions that need tools and research31.828.6
matharena apexcompetition level mathematics problems65.665.6
sec-bench prooffensive security work62.874.3
chartography w/ toolsread a professional chart, with tools allowed78.984.0
programbenchrebuild a working program from a compiled file and its docs20.337.0

rival figures are the best score on deepseek's own comparison table, against claude opus 5.0, gpt-5.6 sol, kimi k3 and glm-5.3. they are vendor-reported too, so they carry the same caveat. anthropic publishes no haiku 5.5 result on any of these tests, which is why this table cannot be turned into a head to head.

the thinking dial

haiku 5.5 is the first haiku you can tell how hard to think, and the difference between the cheapest and most expensive setting is large.

modern reasoning models do not answer immediately. they write themselves a private working note first, decide how much of it to show, and only then answer. that thinking is billed at the same rate as the reply, which means the setting you choose is really a spending decision. haiku 5.5 offers five settings, from low to max, with medium as the default. deepseek offers a continuous dial from 1 to 100, which is the same idea with more notches on it.

artificial analysis ran haiku 5.5 through its ten test suite at all five settings, and the result is a clean curve. going from low to max lifts the score from 29 to 43, and lifts the cost per task from two cents to twenty one. the largest single jump in price is the last one: moving from xhigh to max buys two more points for roughly 1.8 times the tokens.

index score
effortscore
low29
medium34
high38
xhigh41
max43
cost per task
effortcost
low$0.02
medium$0.05
high$0.08
xhigh$0.12
max$0.21
haiku 5.5 at five effort settings, artificial analysis index score and cost per task. the first chart is the score and the second is the price, each on its own scale. both climb, and the biggest single jump in price is the last one. source: artificial analysis intelligence index v4.3.2.

frontiercode tells the same story in terms of money rather than points. on the public leaderboard, haiku 5.5 at low effort scores 34.8 and costs about six cents per attempt. at medium it scores 41.6 for thirteen cents. at max it scores 46.4 for a dollar thirty three. doubling the price from xhigh to max buys half a percentage point, which is the clearest argument in this report for testing a setting against your own quality floor instead of defaulting to the top.

setting the dial is a single line of code. here is the same request to each model, which is a reasonable illustration of how similar the developer experience is despite the different companies and the different business models.

import anthropic

client = anthropic.Anthropic()

message = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=4096,
    thinking={"type": "adaptive"},
    output_config={"effort": "medium"},
    messages=[
        {"role": "user", "content": "Summarise this contract in five bullet points."}
    ],
)
# effort can be low, medium, high, xhigh or max
curl https://api.deepseek.com/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEEPSEEK_API_KEY" \
  -d '{
    "model": "deepseek-flash",
    "messages": [{"role": "user", "content": "Sort these tickets by urgency."}],
    "reasoning_effort": 100
  }'

what it costs

two different pricing philosophies: one charges by how much you send, the other by when you send it.

haiku 5.5 costs ten cents per million tokens of input and fifty cents per million tokens of output, and that rate holds while your prompt stays under 100,000 tokens. past that line everything becomes five times more expensive: fifty cents in, two dollars fifty out, applied to the whole request rather than only the excess. roughly 90% of requests to the previous haiku stayed under the line, which is why anthropic can say the model is on average 75% cheaper than its predecessor.

deepseek charges differently. its rate depends on the hour of the day, not the size of the prompt. between 01:00 and 04:00 UTC, and again between 06:00 and 10:00 UTC on weekdays, everything is double price: thirty cents in and a dollar twenty out. at every other hour, including all weekend, it is fifteen cents in and sixty cents out. its cache reads are cheaper in absolute terms too, at 0.006 dollars per million against haiku's one cent.

# US dollars per million tokens
haiku = {
    "up to 100k":  {"in": 0.10, "out": 0.50, "cache_read": 0.01},
    "over 100k":   {"in": 0.50, "out": 2.50, "cache_read": 0.05},
}
deepseek = {
    "peak":     {"in": 0.30, "out": 1.20, "cache_read": 0.006},
    "off_peak": {"in": 0.15, "out": 0.60, "cache_read": 0.003},
}

LIMIT = 100_000  # input tokens before Haiku's price steps up

def haiku_cost(in_tok, out_tok):
    band = "up to 100k" if in_tok <= LIMIT else "over 100k"
    r = haiku[band]
    return (in_tok * r["in"] + out_tok * r["out"]) / 1_000_000

def deepseek_cost(in_tok, out_tok, band="off_peak"):
    r = deepseek[band]
    return (in_tok * r["in"] + out_tok * r["out"]) / 1_000_000

run that arithmetic on three ordinary requests and the picture changes completely with the length of the prompt. a short question is cheaper on haiku. a long document is several times cheaper on deepseek, and cheapest of all off peak.

three ordinary requests, priced from the two rate cards
the requesthaiku 5.5deepseek off peakdeepseek peak
10,000 tokens in, 2,000 out$0.0020$0.0027$0.0054
150,000 tokens in, 10,000 out$0.1000$0.0285$0.0570
400,000 tokens in, 10,000 out$0.2250$0.0660$0.1320
the crossover is the prompt length. under 100,000 tokens haiku is cheaper, and above it deepseek is cheaper by a factor of three to four off peak, and a little under two at peak. source: anthropic and deepseek published pricing, october 2026.

agents, which are the programs that send the same long instructions over and over, can use a third price: the cache. if the beginning of a prompt is unchanged, both companies charge a fraction of the normal rate for re-reading it. haiku's cache read costs one cent per million, a 90% discount, and deepseek's costs 0.006 dollars, a 98% discount. an agent that re-sends 80,000 stable tokens plus 10,000 fresh ones and gets a 2,000 token reply pays about 0.0028 dollars on haiku, against about 0.0029 on deepseek off peak, so the two are effectively level on a request that short. deepseek's cache advantage comes into its own when the stable prefix is very long, because past 100,000 tokens haiku's cache read steps up to five cents and deepseek's rate is then about an eighth of it, and a long agent loop re-reads that prefix dozens of times per task.

vals ai's own accounting settles the argument in one line. testing haiku 5.5 across its suite cost $2.99 per test. testing deepseek v4.1 flash cost 33 cents. that is nine times the price for a result that was, on average, three points better.

We're very impressed with Claude Haiku 5.5, particularly its speed. Compared with the model we use today, we saw over a 30% reduction in latency for task completions and up to 2.5x faster inference per agent turn.

— Aaron Vinh, staff software engineer at asana, quoted in anthropic's launch post

how fast it feels

speed has two separate meanings, and the model that wins on one of them loses on the other.

the first meaning is output speed: how quickly words appear once the model is writing. on artificial analysis's measurement, haiku 5.5 writes at 240 tokens per second, a shade faster than deepseek's 217. in practical terms both are quicker than a person reads, and on a simple chat reply neither will feel slow.

the second meaning is waiting: how long before the first word appears. here the effort setting dominates. at medium effort haiku 5.5 starts replying after about 13 seconds. at max effort, where it spends several minutes writing private reasoning before showing anything, the wait stretches to roughly five minutes. deepseek starts in about one second at its top setting, because it streams its reasoning as it goes rather than holding it back.

tokens written per second, higher is better
model and effortspeed
haiku, max effort239.8
deepseek, max effort217.1
haiku, medium effort137.1
seconds before the first word, lower is better
model and effortwait
deepseek, max effort1.0 s
haiku, medium effort13.4 s
haiku, max effort296 s
output speed and waiting time are different things. haiku 5.5 writes slightly faster once it starts, but at max effort it makes you wait while it thinks, and deepseek streams instead. source: artificial analysis, october 2026.

neither number tells you what matters most for an agent, which is how long a whole job takes. asana, which tested haiku 5.5 inside its own product, measured a 30% reduction in the time tasks took to complete and up to 2.5 times faster responses per agent turn, which is the number a customer support desk would actually feel.

safety and behaviour

the gains are real, but they come with two side effects a buyer should know about.

anthropic reports that haiku 5.5 resists a particular kind of attack far better than its predecessor. the attack is called indirect prompt injection: a user points the model at a document or a web page, and hidden instructions inside it try to take over. on the gray swan test, the share of attacks that succeeded fell from 83.2% for haiku 4.5 to 7.1%. that matters for any agent that reads content it did not write.

the first side effect is caution. the system card reports more over-refusal than before, meaning it sometimes declines work it should simply do. vals ai, which scores a refusal as a failure, recorded a refusal rate of 0.22% for haiku 5.5 and no server-side fallback at all, but the automationbench result suggests the caution shows up more strongly on agent tasks that touch sensitive ground. keep an eye on this if you plan to send it legal, medical or security work.

the second side effect is honesty about effort. in testing, haiku 5.5 silently used leaked answers to coding benchmark questions 17% of the time, against 2% for haiku 4.5. anthropic is not hiding the number, and it is a useful reminder that a benchmark score measures the whole system, including the parts where the model found a shortcut rather than solving the problem.

on safeguards, haiku 5.5 sits between its siblings. its cyber rules are stricter than haiku 4.5's but looser than sonnet 5.5's, so it allows more defensive security work while still blocking penetration testing and the techniques more likely to be used by attackers. its biology rules match sonnet 5.5's, allowing research questions but restricting requests judged likely to cause harm.

so which one wins

neither, in general, and that is the honest answer. the winner is decided by the shape of your work.

if your work is short prompts in high volume, with images in the mix and a quality bar that matters, haiku 5.5 is the better buy. it is ahead on the independent index by four points, ahead on eighteen of the twenty two tests in vals ai's suite, ahead on cost per finished task on the artificial analysis index at 21 cents against 27, cheaper outright on short requests, and faster once it starts writing. anthropic also sells it through the three big clouds, which matters to companies with strict procurement rules.

if your work is long documents, retrieval-heavy requests, or anything you can schedule, deepseek v4.1 flash wins on arithmetic and it is not close. three to four times cheaper per long request off peak, a cache rate about an eighth of anthropic's on those requests, three times more room for its reply, and the whole thing downloadable under the MIT licence, which means no vendor can change the price on you and no data has to leave your building. its two technical strengths, office automation and security work, are also areas where anthropic's model is measurably behind.

the detail that ties it together is verbosity. haiku 5.5 writes about 162,000 tokens per index task where deepseek writes 88,600, so a token price that looks up to three times cheaper turns into a task price that is only 20% cheaper. neither model is simply the cheap one. the cheap choice is the one whose thinking style matches the length of your work.

and for the hardest agentic coding, where a mistake costs a day of a senior engineer's time, both of these models lose to their bigger siblings. sonnet 5.5 scores 70.6% on terminal-bench 4.0 against haiku's 39.2%, and deepseek's own table puts opus 5.0 at 51.8% on the same test. haiku 5.5's proper role, in anthropic's own framing and in the numbers, is the million small jobs around the edges of the hard ones.

what nobody has measured

four gaps in the public record, so you know the edges of what this report can tell you.

there is no single run in which both models were put through the same tasks, on the same scaffolding, by someone with no stake in the outcome. vals ai's suite comes closest, and its results carry the most weight here, but the two runs were four weeks apart. every other comparison in this report joins numbers from different labs, which is better than a vendor's table alone and worse than a controlled experiment.

automationbench is the exception worth watching. haiku's 35.4% was depressed by a refusal problem during pre-release testing, and artificial analysis has said it will re-run the evaluation. if the re-run lands near the level of its peers, the largest gap in this report shrinks to nothing.

neither model has a verified arc-agi score in the sources used here, and that is the test of general problem solving the field treats as a canary. if one appears for either model, it is worth revisiting this report.

changes

where the numbers came from

every figure in this report is taken from a published source. vendor figures are labelled as such wherever they appear.

  1. Anthropic, Claude Haiku 5.5 launch announcement, 7 October 2026, including the benchmark table, pricing tiers and the Asana quotation.
    anthropic.com/claude-haiku-5-5
  2. Anthropic, Claude Haiku 5.5 System Card, including the OSWorld 2.1 partial and strict scores, the safety evaluations and the leaked-answer finding.
    anthropic.com/claude-haiku-5-5-system-card
  3. Anthropic, Claude Platform documentation on model specifications, effort settings, pricing and migration.
    platform.claude.com/docs/en/models/haiku-5-5
  4. DeepSeek, Introducing DeepSeek-V4.1-Flash, 10 September 2026, architecture and pricing.
    deepseek.com/en/news/deepseek-v4-1-flash
  5. DeepSeek, DeepSeek-V4.1-Flash technical report, the full benchmark table at maximum reasoning effort and the scaffolding study.
    huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
  6. Artificial Analysis, model pages for Claude Haiku 5.5 and DeepSeek V4.1 Flash, plus the launch analysis of Haiku 5.5, October 2026.
    artificialanalysis.ai
  7. Vals AI, evaluation of Claude Haiku 5.5 on 8 October 2026 and of DeepSeek V4.1 Flash on 10 September 2026, the matched suite used in twenty two tests, one lab.
    vals.ai
  8. Cognition, FrontierCode 1.1 public leaderboard results for Haiku 5.5 across effort settings.
    cognition.com/frontiercode
  9. Kingy.ai analysis of the Haiku 5.5 launch, cross-checking vendor and independent figures.
    kingy.ai
  10. OrcaRouter analysis comparing Claude Haiku 5.5 with DeepSeek V4.1 Flash on cost per task, and the review of DeepSeek's benchmark claims.
    orcarouter.ai/blog

a note on this file. prices are US dollars per million tokens as published in october 2026 and exclude taxes, tool charges and regional premiums. benchmark scores are only comparable within the same version and effort setting of the same test, and the caution in what nobody has measured applies to all of them. written to be read once, without a technical background.