rw-book-cover

Metadata

Highlights

  • Every CEO Dan Shipper consumes more than three times as many tokens as the next-highest user at the company. That gap worries him: Are the rest of us keeping our AI bills low by thinking too small? (View Highlight)
  • A low token spend isn’t necessarily a problem, Dan says, “but it should be a smoke signal, especially on the engineering team, to be like, ‘Maybe there’s more we can be doing here.’” (View Highlight)
  • The goal isn’t tokenmaxxing—torching tokens for sport. Instead, Dan wants to give people room to experiment with radical new strategies and ways of working that, should they pan out, could deliver productivity gains that more than justify the token costs. (View Highlight)
  • Dan doesn’t set a universal spending limit for experimentation. Instead, after an expensive run, he expects the team to ask: Was the result worth what we spent, and could we get a similar outcome more efficiently next time? A costly experiment can be worthwhile if it reveals a useful new capability or limitation. A modest payoff warrants further investment only if it can be reproduced at a lower cost. (View Highlight)
  • Head of operations Arielle Shipper translates that philosophy into spending decisions. For now, she evaluates expensive runs case by case, using the results to decide whether to keep funding the work or change course. (View Highlight)
  • Her early-warning system is the company card. She keeps the ChatGPT automatic credit refill at $500—and gets a Slack push notification every time the card is topped up, which happens multiple times a day. An unusually rapid succession of charges signals that someone is building or testing something big, prompting her to check the usage leaderboard and message the people at the top. (View Highlight)
  • “Then Dan always responds, ‘It’s me running Ultra,’” she says, which explains why he’s at the top of the leaderboard. (View Highlight)
  • She wants answers to three questions: What did the run cost? What did it buy us? And what did we learn? (View Highlight)
  • Every is embedding those questions in its infrastructure. The team is developing a skill that will let employees create and share their own AI spending profit-and-loss statements, while head of evals Mike Taylor is creating personal benchmarks for every employee to show us which models—including less popular, cheaper ones—can handle many of our recurring tasks. The goal is to make spending and model choices more transparent without discouraging ambitious experiments. (View Highlight)
  • Many people on the team already answer those questions proactively. After a string of Astra experiments racked up billions of tokens, head of video Randy Counsman messaged Arielle detailing what had worked, what hadn’t, and what he’d learned—which was a lot. (View Highlight)
  • That’s how many OpenAI tokens Randy burned through trying to make a 3D model of his face with Astra.
    The project started innocently enough: Inspired by social media posts of flashy AI-generated demos made with Blender, a free 3D tool, he wanted to make a 3D model himself. Experimenting with AI, after all, is an important part of his job.
    Using strategies shared on X, he set up an orchestrator agent to maintain the plan, an implementer to assign tasks, and subagents to execute them, and instructed Codex to keep improving the model until it was “done extremely well,” Randy says. In retrospect, “It was an ambiguous goal.” (View Highlight)
  • An AI audit of his Codex sessions showed how his setup had created a compute-hungry “unruly swarm” of agents. Randy had told an orchestrator to deploy agents “as needed,” without limiting their number, and added an implementer agent to assign work to the subagents. As the run continued, those layers passed growing amounts of context back and forth—even for simple status checks, such as whether a subagent had completed a task—burning tokens on coordination. (View Highlight)
  • Randy has since rebuilt his setup for complex projects. He dropped the implementer altogether, limited the orchestrator to five Sol subagents to control costs, and now tells the orchestrator when to pause and ask for his feedback. For a recent project, he also generated an image of the design he wanted, so a judge agent could check the model’s work against a concrete target rather than follow an open-ended instruction to keep improving. (View Highlight)
  • A failed experiment is still a good investment if it shows where an AI system falls short. Randy now has a benchmark to run new models against—how well they turn 2D images of people, animals, and characters into 3D models—and a leaner setup to get a better result with fewer tokens. (View Highlight)
  • Spiral general manager Marcus Moretti’s token spend strategy is straightforward: Stay within the weekly usage limits for his Claude Code Max plan.
    It’s a simple goal that requires active management; Marcus selectively assigns work to Fable 5.1 or a cheaper model. (View Highlight)
  • Step 1. Learn what each model is good at. Marcus has used Anthropic’s models long enough to tell whether a job needs Sonnet, Opus, or Fable-level intelligence.
    Step 2. Match the model to the assignment. Basic tasks—analytics checks, for example—go to Sonnet. Work with a clear objective, such as a tightly scoped product change, goes to Opus. For a larger feature (like the new billing implementation he’s working on), Marcus uses Fable 5.1 to specify what needs to be built and how it should work.
    Step 3. For complex work, let the model delegate. Once the plan is ready, Marcus tells Fable, (View Highlight)
  • “Kick this off, and for all coding tasks, use your judgment about delegating to a lower model.”
    Fable assigns tasks to Opus and Sonnet, then checks their work. This lets the strongest model manage the project without executing every task. (View Highlight)