the first part of this series proposed arbor, an operating system rebuilt from the bottom up for ai agents and the people who work with them. before building any of it, i wanted to know where the memory and the time really go on a computer that agents already use every day. so i measured mine: its memory and its disk, how long it takes to start and to find a file, and 201 sessions in which my coding agents made 47,700 calls to a model. the numbers say what arbor should build first, what it should take out, and what it should refuse to do even where refusing looks slower.
the operating system is not where most of the memory, the time or the money goes. the agents and the software around them are.
five measurements shaped everything that follows.
an agent's model reads its whole conversation again at every step. a cache makes that cheap, and on my computer it is still three quarters of the cost.
an agent works in steps. at each one the model reads everything so far, its instructions, the descriptions of the tools it may use, the conversation and the output of every command, and then writes a little more: a thought, a command, an edit. the next step reads all of that again, along with whatever the command printed. providers keep what a model read at the last step in a cache, and charge a tenth of the usual price to read it again (anthropic, prompt caching). my agents found 98.7% of what they read in that cache, which cut what their calls would cost by 87%.1 what is left is still mostly rereading:
| part of the bill | share |
|---|---|
| reading again, from the cache | 74.3% |
| the model's own writing | 13.8% |
| putting what was read in the cache | 11.9% |
so the cheapest token is one that is never read, and arbor's first changes are about reading less, not writing less.
my sessions opened with a median of 70,000 tokens before the first word of the task: instructions, and a description of every tool the agent might call. that opening is read again at every step, and it makes up 23% of everything my agents reread, about a sixth of the whole bill. anthropic measured the other side of it: when agents searched for the tools they needed instead of loading all of them, the descriptions shrank from 77,000 tokens to 8,700, and accuracy in their tests of tool use rose from 49% to 74% (anthropic, advanced tool use). in arbor every service describes itself, and an agent asks for the descriptions it needs when it needs them.
16% of my agents' steps did nothing but read, straight after another step that did nothing but read, and they cost 13% of the bill. each could have been folded into the step before it. in arbor one step can ask for many things at once, and a short script can call tools in a loop without a trip back to the model each time. anthropic reports 37% fewer tokens for the same work when the model writes such a script (anthropic, advanced tool use), and an example in which 150,000 tokens of tool results passed through the model needed 2,000 once code handled them (anthropic, code execution).
a long session fills up with the output of commands run an hour before. dropping old tool results while keeping the recent ones cut tokens by 84% in anthropic's tests, and raised the score on the task by 29% (anthropic, managing context). researchers at jetbrains found that simply masking old observations halved what an agent cost, at about the same rate of solved problems (lindenbauer and others 2025). in arbor every result stays in the ledger, and the model sees the recent ones.
what the model does read should be shorter. tools print for people: columns lined up, decoration, and every field they have. i printed the same listing of 100 files in eight ways, and counted the tokens:
| printed as | tokens |
|---|---|
| json, indented | 6,283 |
| yaml | 4,957 |
| json, on one line | 4,163 |
| a markdown table | 3,285 |
| toon, a table made for models | 2,916 |
ls -la | 2,776 |
| columns split by tabs | 2,690 |
| the names alone | 401 |
the format matters less than asking only for what is needed. git log printed 18,928 tokens for the last 20 changes in one of my projects, and 530 with one line for each. ps aux printed 9,900 tokens for the 30 largest processes, and 286 when asked for their ids, memory and names alone. the authors of toon measured 42.6% fewer tokens than json, with slightly higher accuracy (toon). code is the exception: wrapped in json, it scored at or below plain text for every model aider tested (gauthier 2024). so in arbor a result is a typed object, printed as a table of the fields the agent asked for, and code stays plain text.
a screenshot is the most expensive way to look. my agents took 3,461 of them, about 2.9 to 3.9 million tokens of reading, and one screenshot of a 1080p screen costs 1,560 tokens once it is scaled down, or 2,691 for a model that reads it at full size (anthropic, vision). in arbor an app says what is on its screen as data, and a screenshot is for checking how something looks.
the cache becomes part of the system. arbor puts the parts of a prompt that never change first, in the same order for every agent, and keeps timestamps and other details that change out of them, so that a provider's cache, or a local model's, can use them again. a cached prompt of 100,000 tokens began its answer in 2.4 seconds instead of 11.5 (anthropic 2024).
not every step needs the largest model, thinking for as long as it can. at medium effort, opus 4.5 matched sonnet 4.5's best score on a coding benchmark with 76% fewer output tokens (anthropic, opus 4.5), and a cascade that tries a small model first, and a large one only when the answer fails a check, cost 59 to 98% less at the same accuracy in one study (chen and others 2023). arbor's verifiers are what make the cascade safe: a cheap answer that passes its check is a finished answer.
two ideas turned out to be worth little here. my agents rarely read the same unchanged part of a file twice, 2% of their reads, and only 1.8% of their tool calls failed, so there is little to gain from avoiding retries.
the system takes little memory. the agents and the apps around them take gigabytes, and an agent's share holds text that would fit in a megabyte.
at one reading that day, with six sessions of my coding agent open, memory was spent like this: the six sessions held 1,568 MiB, the desktop app i run them from 1,354 MiB, a browser the agents were driving 2,054 MiB, and the whole gnome desktop 513 MiB. the machine had pushed 9.6 GiB out to swap, 6 of them to disk, and had 0.9 GiB left. a second reading that evening, with no browser open, found about half as much in swap. how much each part takes changes by the hour, but the desktop came last both times.
each session held 250 to 334 MiB to keep a conversation whose text, at the median of 266,000 tokens, comes to about a megabyte. the rest is the runtime around it: a javascript engine, its heap and its libraries. even doing nothing, runtimes differ a great deal:
| runtime | memory |
|---|---|
| node, for javascript | 41.7 |
| python | 9.6 |
| bash | 3.8 |
| rust | 2.2 |
| c, linked statically | 0.6 |
arbor runs one native agent service, and each agent is a handle inside it rather than a process of its own: a conversation, its keys and its budget, in a few mebibytes rather than hundreds. that is an estimate, and one of the first things the prototype has to prove.
when memory runs short, this computer writes pages to an 8 GiB file on disk, beside 4 GiB of swap kept compressed in memory, called zram. memory compresses well: zram held twice its size in pages here, and the heap of an agent compressed 2.15 times with zstd. arbor swaps to compressed memory only, never to disk, and compresses pages that have gone cold before memory runs short, not after. meta does this across its fleet, and saved 20 to 32% of its memory without a noticeable slowdown (meta 2022).
the apps are the other half. a desktop app built on a browser engine, a separate browser for the agents and the gnome desktop each bring an engine of their own, and gnome's shell used 2.1% of a core to show a screen on which nothing moved. moving such an app to the system's own web view saves only 11 to 31% of its memory in tauri's benchmarks (tauri), so the gain has to come from having fewer engines. in arbor the compositor draws the system's own screens, and one shared web engine draws only web pages. video and still layers go straight to the display's hardware planes, which cut the power of playing a video by 7 to 14% in kde's compositor (hugl 2025).
two things linux already does here stay as they are: its newer way of choosing which pages to reclaim, the multi-generational lru (zhao 2022), and huge pages given only to the programs that ask for them.
every command an agent runs should run in a sandbox of its own. that is only practical if a sandbox costs almost nothing to start.
in arbor, work happens in cells: sandboxes that see only the world and the keys they were given, as part one describes. the usual sandbox today is a docker container, which takes 125 milliseconds to start here, too slow to give one to every command. the kernel features docker is built from are far faster on their own:
| sandbox | milliseconds |
|---|---|
docker run | 125 |
| bubblewrap, with every namespace | 3.08 |
| four namespaces | 1.28 |
an empty sandbox is rarely what an agent needs. it needs python with its libraries loaded, or node, or a compiler, and loading them is the slow part. that can be done once: arbor keeps a warm template of each kind of cell and forks it, the way unix copies a process, so the copy shares the template's memory until it changes something.
| started | time | memory of its own |
|---|---|---|
| from nothing | 46.4 ms | 15.2 MiB |
| as a fork of a warm template | 0.73 ms | 2.1 MiB |
a study of containers for short tasks found the same shape: a container stripped of what slowed it started 18 times faster than docker, and templates made it 3 times faster again (oakes and others 2018).
code the agent did not write, or downloaded, deserves a stronger wall than namespaces, which all share one kernel. two kinds of cell give it one cheaply. webassembly runs code in a sandbox that its compiler enforces: faasm started such a cell in 0.5 milliseconds where docker took 2.8 seconds, in 90 KB of memory instead of 1.3 MB (shillaker and pietzuch 2020), and wasmtime makes a new instance in 5 microseconds instead of 2 milliseconds, by mapping a prepared image of its memory (fallin 2022). whole programs get a micro virtual machine: firecracker's monitor needs 3 MB of memory where qemu's needs 131 (agache and others 2020), a snapshot of one is restored in 4 to 10 milliseconds (brooker 2022), and alibaba runs such sandboxes in under 20 MB each by sharing their read-only images (li and others 2022).
agents work best on their own copy of everything, and today a copy costs time and disk. it does not have to.
a third of my projects folder, 22.9 GiB of 70.5 GiB, is files identical to others in it: 6.8 GiB in the separate checkouts my agents work in, 7.2 GiB in build outputs, and the rest elsewhere. arbor stores every piece of content once, under a fingerprint of what it holds, and every world that contains it points to that one copy. doing this for whole files alone gets about three quarters of what finer-grained deduplication saves (meyer and bolosky 2011), so arbor starts there.
the larger gain is time. today an agent gets its own copy of a project with git worktree, which took 79 milliseconds here for a checkout of 19 MB. that checkout then had to rebuild 1,774 steps, because build outputs are not part of it. a copy-on-write snapshot copies nothing until something changes: one of my projects, 28 GiB with all its build outputs, forked in 3 to 4 milliseconds, and the system call that does it took 27 microseconds of that, the rest going to start the program that makes the call. worlds can be fetched lazily too, which matters: when containers start, pulling their images takes 76% of the time, yet only 6.4% of the data pulled is ever read (harter and others 2016).
searching by walking the disk is a habit from when disks were small. an index kept current as things change answers thousands of times faster.
agents search all the time: for a file by its name, for a word in the code, for the one function they need in a long file. today each search walks the disk. my home folder holds 2.2 million files and folders, and find takes 1.4 seconds to look for a name among them, with all of them already in memory. arbor keeps a live catalog of every object instead, updated by the store itself after each write, so it needs no crawler, no nightly updatedb and no watches on folders.
| looking for | today | with an index |
|---|---|---|
| a file, by name | find, among 2.2 million: 1.4 seconds | a catalog: 0.32 milliseconds |
| a word in code | ripgrep, through 248 MiB: 33 to 40 milliseconds, or 174 with part of the code on disk | a trigram index: 1 to 5 milliseconds |
| two functions | the whole file read, 40,000 tokens at the median | its outline and the two functions: 8% of the tokens |
the catalog costs 90 bytes an object, 191 MiB for my 2.2 million, and 2 microseconds to update when something changes. it is updated just after a write, from the store's log of changes, and never inside one: the be file system kept its indexes current inside every write, and paid for it in speed on work heavy in metadata (giampaolo 1999). the idea works at scale already. plocate finds a name among 27 million files in 8 milliseconds where the older locate took 20 seconds (gunderson), and git, told what changed by a monitor of the file system, answered git status in 0.75 seconds instead of 85 on a large repository (hostetler 2022).
words in code need another kind of index, of trigrams: every sequence of three characters, with the files that contain it. it can answer even a regular expression, by narrowing the files to read before reading any (cox 2012). mine was quick to build with sqlite, and it is three times the size of the code it covers. google's code search kept its index to a fifth of the code (cox 2012), so there is room to do better.
long files are where agents waste the most reading. in 40 of my largest source files, 40,000 tokens each at the median, an outline of a file's types and functions is 4% of the file, and the outline with the two functions an agent wants is 8%. arbor stores code as objects that know their own structure, so an agent asks for the outline first, and then for functions by name.
searching by meaning needs an index of vectors, and that is fast too: faiss finds the nearest among a million vectors in 0.02 milliseconds, the true nearest 95% of the time (faiss), and diskann searches a billion on one machine with 64 GB of memory in under 3 milliseconds (subramanya and others 2019). arbor builds it in the background, away from the path of a write. small objects, by the million, are packed into large files with an index in memory, as facebook did with photos: one read from disk for each instead of about three, and 10 bytes of memory for each where a file system spent 536 on an inode (beaver and others 2010).
nearly half the time my computer takes to start passes before linux is even loaded.
| stage | from | to |
|---|---|---|
| the board's firmware | 0 | 11.3 |
| the boot loader | 11.3 | 13.0 |
| the kernel and its first image | 13.0 | 19.5 |
| linux's programs, up to the desktop | 19.5 | 25.2 |
systemd-analyze measured the start my measurements ran in. the firmware tests the hardware and starts it, the boot loader chooses a system and loads it, and the kernel then starts from a small image of drivers and tools, the initramfs, before handing over to the system on the drive.the firmware is the board maker's, and the fastest way past it is not to go through it. arbor resumes instead of starting: for daily use it sleeps in suspend-to-idle, which wakes without the firmware, and it moves into a new kernel with kexec, which starts the new kernel from the running one and skips the 13.0 seconds of firmware and boot loader. even the older kind of sleep spends little time in firmware: on a machine intel measured, it took 340 of the 2,087 milliseconds of a resume (brown 2015).
where a board accepts other firmware, firmware that starts only what the system uses is far faster. a coreboot laptop reached the kernel 1.1 seconds after power-on (menzel 2023), and a server whose firmware took eight minutes to start it took 20 seconds with linux in its place (edge 2017).
the image the kernel starts from, the initramfs, holds the drivers and tools needed to reach the real system. ubuntu builds it to start any pc, so on mine it is 90.7 MB, and most of it is for other computers:
| part | megabytes |
|---|---|
| kernel modules this pc never loads | 32.2 |
| programs that start the system, compressed | 24.3 |
| firmware, mostly for datacenter network cards | 17.2 |
| processor microcode, for every intel and amd model | 15.1 |
| kernel modules this pc loads | 1.6 |
arbor builds one kernel for the machine it runs on, with that machine's storage and file-system drivers built in and everything else loaded when it is needed. then there is no generic image to unpack and no boot menu: the firmware starts the kernel directly.
the last 5.7 seconds are linux starting its programs. the boot splash alone holds the desktop back by 2.04 seconds while it waits to finish, and jobs that could wait run during the start: ubuntu pro's timer for 3.9 seconds, the daily upgrade for 1.4, snap for 1.6, a check for new firmware for 0.5 and the manual's index for 0.3. in arbor the first screen starts first, and everything else waits its turn as background work.
three things that speed a start up are already done here, so nothing is left to gain from them: a kernel and image compressed with zstd, the processor's cores started in parallel, and a quiet start with no messages. with this board's firmware, i estimate arbor would start in about 13 seconds, nearly nine tenths of them in firmware, and with firmware like coreboot's in about 2.5. both estimates take the kernel at about a second, as on the coreboot laptop, and the first screen at half a second.
the same processor runs the same code faster when the code is laid out for how it really runs.
compilers guess which paths through a program are common. a profile-guided build measures them instead, and lays the code out to match. facebook's bolt, applied after profile-guided and link-time optimization, made data-center services up to 8.0% faster and the gcc and clang compilers up to 20.4% faster (panchenko and others 2018). the same kind of profile, as google applies it, made the linux kernel up to 5% faster on large benchmarks (xu and shen 2024). arbor ships its kernel and its busiest services built this way.
memory can be laid out better too. google's temeraire, a memory allocator built around huge pages, raised the requests its services answered each second by 7.7% and cut their memory by 2.4% (hunter and others 2021), and handing programs memory in pieces of 64 KB rather than 4 KB cut the time of a kernel build by 5% (roberts 2023).
so can calls into the kernel. flexsc let programs queue their system calls without stopping to enter the kernel for each one, and made apache up to 116% faster and mysql up to 40% (soares and stumm 2010). linux's version of the idea is io_uring, and arbor uses it in its own store, catalog and outbox. never from inside a cell, though: 60% of the kernel exploits submitted to google's reward program went through io_uring, and google turned it off on its production servers (google 2023).
two more are smaller. drives that only ever write forward, zoned drives, sustained 1,010 MiB/s of writes where ordinary drives managed 370 to 590 MiB/s, and wrote each byte about once instead of two or more times (bjørling and others 2021). arbor wants them once it has hardware of its own. and building every package for the newer instructions this processor has gains about 1% for most of them (canonical 2025), not yet worth a second set of packages.
a computer that waits should cost almost nothing. most of the saving comes from waking less, and from doing background work where it is cheapest.
nothing in arbor polls. the kernel stops its clock on idle cores, as linux already can, and without that a battery drains two to three times as fast (the kernel's documentation). services wake on events, never on a timer that asks whether anything happened, and the timers that remain are gathered by class, so that a hundred small wakeups become one. apple found that gathering timers cut a laptop's idle processor activity by 72% (cohen 2013).
background work becomes a class of its own: indexing, syncing and compacting the ledger run in batches, on the efficient cores. microsoft's ecoqos, which does this on windows, cut the processor's power by up to 90% for such work (li 2021), and android's doze, which holds such work back while a phone's screen is off, made phones last 30% longer with the screen off (sd times 2015).
each cell gets an energy preference too. this computer runs every core at the preference called performance, all the time. intel measured that its setting for saving power cut the power of playing a video by 52% on its laptop processors (wysocki 2024). choosing between big and little cores by energy used 8 to 27% less energy, for work that took 0.7 to 2.4% longer (perret 2018), and lavd, a scheduler written for valve's steam deck, used 16% less energy than linux's default in one of two game benchmarks, and the same in the other (min 2024).
a screen on which nothing changes should cost nothing to draw. gnome's shell here used 2.1% of a core to show one. panels that drop their refresh rate when nothing moves used 22% less power in samsung's phones, and 60% less for a still image shown at 10 Hz (samsung 2020), and a browser tab put to sleep used 37% less processor time and 32% less memory (microsoft edge team 2020).
devices sleep too, when they are allowed to. keeping the power saving that the firmware had turned on for the links to expansion cards, instead of switching it off everywhere, saved about 5 W on an idle laptop (garrett 2011); nvme drives that power themselves down saved nearly 1 W (lutomirski 2016); and sata links that sleep saved 0.5 to 3 W across 15 laptops (fedora 2018). arbor turns them on, with the settings known to be safe.
the rest is a matter of using the right part of the machine. a graphics chip's fixed-function decoder plays video on 4.4 to 8.9% of the energy the best software decoders use (kränzler and others 2024). if arbor runs models on the machine itself, batching their requests saves the most: 12.5 times less energy for each request, and up to 100 times with arrivals spaced to fill each batch (delavande and others 2026), and a phone's neural processor read prompts on 1.85 to 4.3 times less energy than its graphics processor (xu and others 2024). on a battery, or a phone network, background traffic waits to travel in batches, for 35 to 52% less energy spent on the network (balasubramanian and others 2009).
agent traffic is many small calls to a few services. the gain is in not setting up the same connection twice.
here, a call to the model's api took 185 milliseconds, 12 of them spent setting up a new connection. arbor's outbox, through which every request leaves, keeps one pool of open connections for all its cells, so that a cell borrows one instead of opening its own. cloudflare's proxy, doing the same, raised the share of requests that reused a connection from 87.1% to 99.92%, opened 160 times fewer, and delivered its slowest 5% of first bytes 80 milliseconds sooner (cloudflare 2022).
a connection that must be new costs less when it can resume an earlier session with tls 1.3: setting connections up took 190.1 milliseconds instead of 237.9 over a path 50 milliseconds long, and the client's cryptography 2.38 milliseconds instead of 9.22 (sy and others 2019). a cache of names in the outbox saves 2.5 to 34 milliseconds a lookup, or 24 to 156 when the resolver itself has to ask (al-dalky and rabinovich 2020). and large transfers need large windows: a window of 3 MB holds a path 100 milliseconds long to 240 megabits a second, whatever the line could carry.
http/3 and bbr help only on lossy or long paths: over quic, google search answered its slowest 1% of queries 14 to 17% sooner (langley and others 2017). techniques built for datacenters, such as bypassing the kernel or never copying data, change nothing at the rates agents send.
some of today's costs need no clever replacement. they only need to go.
updatedb and the file indexers: replaced by the store's log of changes.git log, and a 70,000-token opening.some optimizations buy speed with safety, or with the quality of the work. arbor turns them down, for these reasons.
all ten can be built on linux, in arbor's first step, and they are in order of what they save on my computer.
| change | today | with it |
|---|---|---|
| 1a shorter context | a 70,000-token opening, read again at every step, and old output kept to the end | tools described when asked for, 8,700 tokens instead of 77,000, and stale output dropped, for 84% fewer tokens |
| 2results for agents | 6,283 tokens for 100 files and 18,928 for a git log; 16% of steps only follow another read | tables of the fields asked for, 401 tokens and 530; many reads in one step |
| 3worlds | a third of my projects folder is copies; a checkout takes 79 ms, and then a rebuild | copy-on-write snapshots over one store of content: a 28 GiB project forked with its builds in 3 to 4 ms |
| 4warm cells | 125 ms to start docker; 46.4 ms and 15.2 MiB for a python cell | 1.28 ms for namespaces; 0.73 ms and 2.1 MiB forked from a template |
| 5one agent service | 250 to 334 MiB an agent, and 1.8 GiB for the desktop app and the desktop | agents as handles, a few mebibytes each, an estimate; one compositor and one web engine |
| 6compressed memory | 6 GiB swapped to disk at one reading | nothing on disk; zram holds twice its size here, and meta saved 20 to 32% of its memory |
| 7a live catalog | 1.4 s to find a name, up to 174 ms to find a word, whole files read | 0.32 ms, 1 to 5 ms with a trigram index, and 4 to 8% of a long file with its outline |
| 8a faster start | 25.2 s, through a generic image, a splash and jobs at boot | one kernel for the machine, and resume or kexec: about 13 s with this board's firmware, about 2.5 s with coreboot's, both estimates |
| 9waking less | every core at the performance preference, all the time | no polling, background work batched on efficient cores, devices asleep: up to 90% less processor power for that work, about 5 W from the links to expansion cards |
| 10profiled builds | code laid out by the compiler's guesses | services up to 8% faster, compilers up to 20%, the kernel up to 5% |
one computer, one day, and a set of scripts that can take every number again.
every number i call mine was taken on the computer described at the top, on 16 october 2026, by a suite of small scripts. it needs no administrator's rights, only reads what it measures, keeps its temporary files in memory, and runs in a little over two minutes. memory is the resident set of each process, from linux's own accounts in /proc, and where processes share pages, their proportional set. times are medians of repeated runs, with the files already in memory except where the text says otherwise. the start is as systemd-analyze reported the boot the measurements ran in, and the boot image was unpacked and compared with the modules the running kernel had loaded. copies on disk were found by hashing every file in the projects folder.
tokens come from two places. what calls cost comes from the usage the api reports with every call, which each session's transcript keeps: tokens read from the cache, written to it, read new, and written by the model. those are exact. counts of text, for formats and for tool results, use the o200k tokenizer, because claude's is not public.
i ran the whole suite a second time that evening, while another job kept the processor busy, at a load average of 11. of the 149 numbers it could compare, 115 came within 15% of the first run. most of the other 34 were start-up times that slowed under the load, and readings of memory that depend on what is open at the moment. code search was faster the second time, because during the first, part of the code had been pushed out of memory. none failed.
every other number comes from the source cited beside it. the list below marks the makers writing about their own products, the preprints nobody has reviewed yet, and the measurements anyone can run again. the estimates say so where they appear.