The Tab Ran Out of Memory

The first browser demo ran out of memory before the model finished downloading.

It was embarrassing for about five minutes, and then it became more useful than a successful demo would have been.

The first version of the pipeline still thought like a server: fetch the model, hold large pieces in memory, build an in-memory representation, compile it, and only then worry about durable storage. Moving the code into WebAssembly had changed the target, but not the ownership model.

The tab made the mistake impossible to ignore.

A real browser implementation could not be a server pipeline squeezed behind a web interface. Model shards had to stream. Tensors needed durable local identities before compilation. The compiled graph could not contain a second copy of weights already stored on the device. Memory admission had to happen before the browser transferred gigabytes it might never be able to use.

That is the design pressure behind hologram-ai: download, compile, materialize, and run the actual model pipeline inside a static browser application.

The browser is not a remote control for the machine.

For this product, the browser is the machine.

Porting the code was not enough

A native service can assume a large 64-bit address space, ordinary filesystem access, threads, process-level memory controls, long-lived daemons, and enough headroom to survive temporary copies that nobody is proud of.

A static browser application has a very different environment:

• WebAssembly commonly works within a 32-bit linear address space
• storage is origin-scoped rather than an ambient filesystem
• expensive work belongs in workers, not on the UI thread
• page lifecycle interruptions are normal
• the user can close the tab midway through any operation
• the WebAssembly program cannot assume native JIT behavior or arbitrary host services

Compiling the same Rust source to WASM does not erase those constraints.

The pipeline itself has to change:

flowchart LR
    subgraph Server-shaped pipeline
        S1[Download full model] --> S2[Load weights into memory]
        S2 --> S3[Compile graph + weights]
        S3 --> S4[Run]
    end

    subgraph Browser-shaped pipeline
        B1[Stream one tensor] --> B2[Hash incrementally]
        B2 --> B3[Persist by address in OPFS]
        B3 --> B4[Compile weightless graph]
        B4 --> B5[Materialize only when running]
    end

The second path does more than save memory. It moves ownership boundaries. Persistent storage exists before the full in-memory model, and the graph refers to weights rather than swallowing them.

That is what made the browser architecture coherent.

Put the tensor in storage before asking memory to own it

The important browser storage primitive here is OPFS, the Origin Private File System.

A download worker can stream safetensors shards, parse one tensor at a time, hash its bytes incrementally, and store it under a content-derived path:

tensors/<address>.bin

sequenceDiagram
    participant H as Hugging Face
    participant D as Download worker
    participant O as OPFS store
    participant C as Compiler

    H-->>D: streamed shard bytes
    loop tensor by tensor
        D->>D: parse tensor header
        D->>D: hash bytes incrementally
        D->>O: persist tensors/address.bin
    end
    D->>C: tensor manifest + model config
    C-->>O: weightless .holo archive

The browser no longer needs to retain the complete shard, the complete model, and the final local-store representation at the same time.

This reverses a common assumption. On a server, persistent storage may look like an optimization added after an in-memory representation is already working. In the browser, the content-addressed store is what makes the in-memory representation possible at all.

The object enters durable local storage as soon as it has an identity. Later stages can resolve it when they actually need it.

The graph should not become another model-sized object

The next memory problem was duplication.

A conventional compiled-model artifact often bundles structure and weights together. That is convenient for distribution, but wasteful when those weights already exist in an addressable local store.

The browser pipeline instead produces a weightless .holo archive. The archive describes the graph and records the tensor addresses it requires. When an inference session starts, the materializer resolves those addresses in OPFS and re-hashes each buffer before use.

flowchart TD
    M[Model config] --> G[Structural graph]
    T[Tensor manifest] --> G
    G --> H[Weightless .holo]

    H --> R[Runtime materializer]
    O[OPFS address store] --> R
    R --> V{Re-hash matches address?}
    V -- yes --> E[Inference session]
    V -- no --> X[Refuse corrupted buffer]

The graph becomes a structural object rather than a transport suitcase containing every byte it may ever reference.

That has useful side effects. Duplicate tensors can be stored once. Several compiled views can share the same weights. Corrupt local data fails materialization. The graph remains small enough to move independently, while its dependency closure stays explicit.

This is the kind of architecture the memory failure forced us to see.

Refuse before transferring the impossible model

A browser should not discover that a model cannot fit after downloading it.

The configuration and tensor metadata already reveal enough to estimate the working set: parameter counts, element widths, layer and hidden dimensions, vocabulary size, selected context length, activation buffers, runtime overhead, and the materialization strategy.

A simplified estimate is:

$$M_{\text{required}}(c) = M_{\text{weights}} + M_{\text{runtime}} + M_{\text{activations}}(c) + M_{\text{margin}}$$

where $c$ is the chosen context length.

Admission succeeds only when:

$$M_{\text{required}}(c) \le M_{\text{budget}}$$

flowchart TD
    C[Fetch config and tensor metadata] --> E[Estimate memory by context]
    E --> B{Fits browser budget?}
    B -- no --> R[Refuse before weight transfer]
    B -- yes --> S[Choose safe context]
    S --> D[Begin streamed download]

The product can reduce context, recommend a smaller model, or explain the environmental limit. What it should not do is optimistically transfer gigabytes, allocate until the tab disappears, and leave the user to infer what happened.

A resource guard is only real when it prevents the operation. A warning displayed beside an inevitable crash is not admission control.

One origin still contains a distributed system

Once the pipeline moved off the main thread, the browser stopped looking like one program.

The download worker owns persistent ingestion and compilation. The generation worker owns materialization and inference. The main thread owns interaction and presentation.

flowchart LR
    UI[Main thread UI] -->|commands| D[Download worker]
    UI -->|generate| G[Generation worker]
    D --> O[OPFS]
    G --> O
    D -->|progress / result| UI
    G -->|streamed tokens| UI

That separation keeps the interface responsive, but it creates familiar systems problems inside one browser origin: versioned message schemas, cancellation, partial-download recovery, worker restart, progress reporting, OPFS ownership, serialized errors, and compatibility between the TypeScript adapter and the WASM binding.

“Running in the browser” does not mean “one component.” It means the distributed boundaries are running on the user’s computer.

Treating those boundaries explicitly is better than letting them emerge as ad hoc postMessage calls after the product grows.

The browser should run the real engine

A browser product can drift easily if the native path uses one runtime while JavaScript implements a simplified approximation.

Then the CLI and browser share a name but not a contract.

The stronger design compiles the same Rust core to WebAssembly and keeps TypeScript as an adapter around the real binding. Native and browser paths call the same conceptual operations: compile, describe, materialize, run, and derive addresses.

flowchart TB
    Core[Rust hologram-ai core]
    Core --> N[Native CLI / library]
    Core --> W[WASM binding]
    W --> B[Browser command adapter]
    N --> C[Same archive and address contracts]
    B --> C

This does not make performance identical. It makes semantic drift harder. A .holo dependency or address should not acquire a browser-specific meaning merely because it crossed a WASM boundary.

That shared core also changes how the browser path has to be tested.

A unit test cannot prove the tab works

A Rust test can validate the memory estimator. Another can verify archive materialization. Neither proves that Chromium can complete the full journey through workers, OPFS, WASM, and the browser’s actual lifecycle rules.

The target-platform test has to do the target-platform work:

1. resolve a model
2. stream its tensors
3. persist them in OPFS
4. compile the graph
5. materialize the archive
6. run generation
7. verify the protocol behavior

That is why the browser journey belongs in the deployment gate, not as a demo somebody remembers to run before a release.

The browser is a supported machine. Its conformance tests should exercise the machine we claim to support.

Static hosting keeps the architecture honest

The application can be deployed as static assets.

I like that constraint because it removes a common ambiguity. If inference succeeds, an unmentioned backend did not quietly perform the expensive part. The network is used to retrieve model artifacts; compilation and inference remain local.

Static hosting also makes the limitations harder to hide. There is no server memory to rescue a bad estimate, no backend session to preserve state after the tab closes, no server-side secret manager, and no invisible fallback for an unsupported model.

Those limits force the product to tell the truth about which work happens where.

Local execution still does not automatically mean complete privacy. The application may contact a model registry, CDN, update service, or telemetry endpoint if one is added. A local-AI product should say exactly which bytes leave the origin and why.

A meaningful claim would be:

After model retrieval, prompts and generated tokens remain inside the browser origin.

That statement belongs in a network-boundary test. It should not be inferred merely from the presence of a .wasm file.

The address-space limit improved the native design too

The common browser WASM target remains constrained. A large desktop with abundant physical memory does not automatically give a WebAssembly module an unlimited address space.

That boundary pushes the architecture toward streaming rather than staging, deduplication rather than copying, addressed tensors rather than monolithic bundles, configurable context rather than one advertised maximum, and up-front admission rather than optimistic allocation.

Try the tab-sized version first

Before moving a large model into a browser, ask the browser to prove it can afford the job:

• measure the largest artifact you will transfer
• reserve storage before allocating the full in-memory representation
• stream one tensor or shard at a time
• make the memory budget visible to the user
• refuse the model before the tab starts a transfer it cannot finish

That turns “the browser crashed” into a useful admission result.

Those are good properties on a native host too. The browser simply refuses to let us postpone them.

The initial out-of-memory failure changed the question I was asking. I stopped asking how to squeeze the desktop pipeline into a tab.

It became, “What would this pipeline look like if the user’s browser were the primary computer from the beginning?”

The answer is a content-addressed local store, a weightless graph, worker-owned streaming, guarded admission, and the same runtime contract used natively.

That is not a remote shell. It is a computer architecture that happens to live behind a URL bar, with all the constraints that implies.