Tokens and costing

Inspect a recorded model call and separate reusable context from the information that changes on every tick.

• Distinguish input, output, cached, and total tokens where the provider exposes them.
• Identify context that grows each tick and decide what belongs in persistent incident state instead.
• Record model IDs and provider-reported runtime tokens; the course accepts the group’s report on the honor system.
• Inspect the recorded stable prefix and per-tick tail supplied by the instructor to see where prompt caching can apply.

Seeing token boundaries

Ordinary text splits into surprising pieces: subwords, leading spaces, an emoji, and punctuation. Try your own text at the tokenizer playground.

Token hygiene in Claude Code

A bloated context costs money on every turn and makes the model worse. Three commands show what fills the window, trim it, and track your limits (costs docs):

• /context shows what your session is carrying. Run it after a long working block and see how much is still earning its place. The Padlet is full of your /context screenshots from the skills session - open it and compare: what fills a window varies wildly between two people doing the same task.
• /compact summarizes the conversation so far. Running it yourself gives you an exact point where you can afford the model losing detail, instead of letting it happen mid-task.
• /usage shows how close you are to your plan limits. For a per-session and per-day breakdown from your local logs, run npx ccusage.

The # shortcut is the counterpart for keeping context lean across sessions: start a message with # and Claude offers to save the learning to memory, so bulky notes live in a file instead of being retyped into every session (memory docs).

Usage vs Context Limits

You will hit two different ceilings; do not confuse them:

• Context window caps one request, denominated in tokens: everything the model reads plus writes in a single call. It is the LLM’s short-term reasoning window - the only “memory” it has while thinking; anything outside it may as well not exist. Hit it and the call fails or gets compacted. Money cannot raise it.
• Usage limit caps your spending over time, denominated in dollars: your plan or API budget across all calls. Hit it and you wait or pay. Trimming one prompt barely moves it.

A lean context serves both: fewer tokens per call keeps you under the window, and tokens times calls is exactly what the dollar meter counts.

Token Pricing

Tokens only become cost once you know the per-token price, and prices vary by orders of magnitude across models. Compare them before you commit a model to a loop: the OpenCode Go model list covers the runtime models your agent calls, and OpenRouter lists prices across most providers.

The most important takeaway of this is there is a huge variety of models to choose from, at many different price-points.

Prices are $ per million tokens, input / output. Columns are capability tiers, not price bands. Kimi K3 shown here is an INT4 route, against $3.00 / $15.00 on Moonshot’s full-precision API. Sonnet 5 at $2.00 / $10.00 is introductory and reverts to $3.00 / $15.00 after 2026-08-31.

The cheap extreme is self-hosting: laptop-grade (runs on a consumer laptop) and toaster-grade (4B params or under, runs on embedded systems) are open weights you run yourself. GLM 4.7 Flash appears twice on purpose: cheaper to self-host. * are popular third-party distillations.

Tiers: artificialanalysis.ai; prices: openrouter.ai; as of 2026-08-09.

How to pick a model

For our class, the candidate runtime models are:

• intake: DeepSeek V4 Flash or MiMo V2.5
• dispatch: MiniMax M3, MiMo V2.5 Pro, DeepSeek V4 Pro, or Qwen 3.7 Plus.

A tremendous source of wastage in AI utility design is over-specifying models relative to their utility - a frontier model on a job a cheap one does as well burns budget on every single call.

Part of this class is deciding if the expensive pick actually earns its price on your task, by benchmarking it on data.

Token and caching strategies

Prefix caching Providers cache already-processed requests; a following call that begins with a byte-identical prefix pays only a re-read price (typically about 10% of the fresh-read price) until the cache expires from inactivity (typically 5 minutes, but up to an hour on a Claude subscription).

An agent loop resends the whole conversation on every call, so in an ongoing session nearly all input tokens are cheap cache reads. You can see this with npx ccusage, which breaks down the usage by cached and new reads.

Here we show you what that looks like when you run a 40k-token prefix and 6k of new tokens per turn at Sonnet 5 prices ($2.00 / M input, $0.20 / M cache read). The pale bar is cached, costing $0.20/M tokens. The dark bar is a fresh read, costing $2.00/M tokens. When the cache is used properly and the prefix is stable, five turns cost $0.18. If we make a small mistake (such as including a clock in the system prompt), the whole read is billed anew and five turns cost $0.58.

By the time this conversation extends to 200k tokens the price difference is huge: $0.40 vs $5.60.

You will exploit exactly this in Stage 1: dispatcher to cut the time and cost of your dispatch bot: its per-tick calls should share a long stable prefix, depending on how you design the prompt.