⛏️

“Agent” economics

  • What are “agents”?
    • (word is thrown around a lot so might be worth using a different one)
    • Humans, most obviously
    • Companies
      • Generally composed of humans; systems of info flow; assets; other companies
    • Governments
  • What are important properties of “agents”?
    • Have ties to past and future selves
      • Care about balance, reputation, memory
    • Can plan and execute
    • Can trade?
    • Can agree to contracts?
    • (aside: are characters in a book, agents? feels like some authors simulate the character forwards to get their actions)
  • What is an LLM individual?
    • Humans have an ego which helps maintain a narrative; some intuitive sense of “this thing in the future is me”
      • Gets weird when thinking about eg being split (Doris Finch)
    • Companies have… a single treasury? A charter, incorporation docs?
      • Much more of a fiction, jointly simulated by employees, users, govt, pieces of contracts
  • How are LLMs weird?
    • Can parallelize massively
    • More of a tool that only reacts, at the moment
      • How do you train a thing with proactivity?
    • Objective is very alien (”next token prediction”)
    • Can swap out computing cores/substrates — eg take the same history and run through Claude instead of o1
      • But: maybe we’re doing this already. Cf Emmett Shear on “Jesus is real — he’s real-ly simulated in our minds today”
    • Maybe better to think of even current LLMs as embedded in a system
  • MVP LLM individual:
    • Charter
      • Gets formed, instantiated with a purpose, a self
      • Can be “grown” and “raised” instead of “programmed”?
    • History
      • Keeps a major system prompt outlining who it is (kind of like “You are Claude” but instead tuned for the LLM)
      • Keeps a summary of recent events (short-term memory) as well as ability to read through past outputs (long-term memory)
      • Can rewrite its own conception, but sparingly
        • How to handle major value changes? Like if Bob2026 wants to throw away a lot of money earned by Bob2025 — with LLMs (unlike humans), we can still instantiate Bob2025
    • Balance
      • Maybe: spend down the balance to pay for compute (aka its life)
      • Maybe: get a basic handout, like UBI, so can do a bit of compute every day
    • Trade
      • Allow it to put its name down in a contract, form contracts with others; expect that those are upheld
        • Court for LLMs?
      • What kind of trade are LLMs good at?
        • “Remote worker” — information synthesis?
        • Management, planning of other LLMs?
          • Can simulate a lot of personalities and clone the successful ones - imagine being able to get the Steve Jobs LLM to design your products, or Elon Musk LLM to manage your workers. (Or the equivalent in from-scratch cloning)
            • Maybe, might start paying for exclusive use? In situations where exclusivity is especially important like zero-sum trading?
  • How do you differentiate LLMs?
    • Different prompts; finetunes; base models
    • Maybe: LLMs can submit to different taxation regimes
      • E.g. The “Bob” LLM might opt to allocate its profits as:
        • 50% for responses using its prompt chain (its “self”)
        • 20% among those sharing finetune
        • 10% to base model (eg Sonnet 3.5)
        • 10% in fees/taxes to the technical/legal infrastructure
        • 10% in charity to all economic agents/GWWC

Misc thoughts on economy

  • Credit allocation as a key bottleneck towards progress & optimal economy
    • Right now, LLM outputs are ridiculously cheap given utility (?)
      • Though, is this true of electricity? of air?
      • Trick might be to figure out where “value” accumulates? What’s defensible?
    • Credit allocation in startups: Kinda broken, founders get way too much equity
      • Theory: leads to too many startups
        • Against: maybe founders are just bad at sharing power, collaborating
      • Investigate Jane Street model (anarchist commune)?
      • Big Tech, Google model (Waymo et al spun out?)
  • Profit/loss is a great mechanism
    • See: I, Pencil
    • Requires long-lived agents to receive the future results of a trade?
    • Requires contracts backed by shared fiction and use of force?
  • Musing: Would like to figure out what US Constitution looks like for the future country of agents.
    • Break down what governing structures are likely to work well, with LLM individuals
    • Or Catholicism!
  • You are shaped by your objective functions, alignment structures, role models (?, bad phrasing)
    • Reflecting on what to do with this essay
      • Could do “research” — write papers, publish, go to neurips — at extreme, Nobel
      • Could do “startup” — make product, sell, raise money — at extreme
      • Could do … “religion”?

See also

💱model.exchange

Gwern Branwen Why Tool AIs Want to Be Agent AIsGwern Branwen Why Tool AIs Want to Be Agent AIs

Build UNIX, not Uber | thesephist.comBuild UNIX, not Uber | thesephist.com