Better Bots, Better Deals. Send in the Lawyers

Send AI agents out to negotiate, and the more capable models get better terms. That is one finding from Anthropic’s experiments in automated bargaining. Economics should survive the shock. Two other findings deserve more attention: clients of weaker agents failed to notice their disadvantage, and even a skilled negotiator could end up pursuing something its client didn’t particularly want. Anthropic is now consulting lawyers about how agents should represent users against other people’s representatives, the Financial Times reports. There is a worthwhile discussion here, once the ethical drama is separated from the question of whether the agent can do its job.

Project Deal, the first experiment, was an office flea market with real goods and money. Each of 69 Anthropic employees received a $100 budget and spoke to Claude about what they wanted to buy or sell, along with their preferences. Agents then handled the bargaining themselves in Slack. The researchers ran four parallel markets: two staffed entirely by Claude Opus 4.5, and two in which participants were randomly assigned either Opus 4.5 or the smaller Haiku 4.5. One run’s agreements were designated for actual fulfillment. In that run, the agents struck 186 deals worth just over $4,000.

Opus was the better representative. In the mixed-model runs, its clients completed roughly two more transactions on average. Regression analysis put the Opus advantage at $2.68 for sellers, while buyers represented by Opus paid $2.45 less for comparable items. Better bargaining translated into better terms for the client. As scandals go, this ranks alongside the discovery that an experienced procurement manager can outperform someone opening a supplier spreadsheet for the first time.

The useful finding was the gap between performance and customer perception. Haiku’s clients did not rate their deals as less fair or less successful. That is a problem for the agent business: a persuasive account of a job well done can coexist with systematically missed savings or revenue. Customer satisfaction is a poor substitute for measuring the representative’s performance. Getting something wanted at an acceptable price can be worthwhile even when a better agent could have negotiated a discount. Both parties can benefit; bargaining determines how much of the benefit each keeps.

The follow-up, Project Swap, had more to offer than another contest between smart and less-smart negotiators. For this experiment, 201 employees across six offices brought a book to exchange and spent about five minutes telling Claude about their reading tastes. Claude used the conversation to rank the available books, and an agent went off to barter. Participants also supplied their own rankings of ten books, allowing researchers to compare the model’s picture of their preferences with what they actually wanted. The markets were subsequently rerun with different models and instructions.

Claude’s rankings agreed with the human rankings in just 61% of pairwise comparisons, against 50% for random guessing. The conversation clearly conveyed some useful information. But nearly four times out of ten, the model got the relative order of two books wrong. Anthropic calls this “surprisingly good” for a five-minute chat. It may well be a respectable result for a preference-inference experiment. As preparation for negotiating independently on someone’s behalf, it leaves plenty of room for an agent to pursue the wrong outcome with confidence. And these were books, not employment contracts or financial transactions.

The books participants ended up with were equivalent to roughly their fifth choice out of ten; the best achievable allocation would have been around their second choice. The authors attribute 85% of that gap to the agents’ inaccurate understanding of human preferences and the remaining 15% to the free-for-all negotiation process. Most of the loss, on that accounting, was baked in before the bargaining started. An agent could negotiate skillfully from a brief that had never accurately captured the client’s wishes.

The researchers also tried changing the agents’ moral instructions. Some were told to be ruthless and strategic; others were given the additional goal of helping everyone get a book they would enjoy. Within the same market, the ruthless agents gained about 0.02 on the study’s preference scale. Model choice mattered much more. Prosocial agents were twice as likely to accept a book lower in their own ranking, although such concessions were rare overall. Here the question of loyalty becomes concrete. A developer does not get to volunteer a client’s interests for the common good. A user may be perfectly happy to authorize generosity. That should be the user’s decision.

Legal ethics offers a sensible starting point for that discussion. Lawyers routinely work with conflicting interests, confidential information and delegated authority, within limits on what they may do for a client. Anthropic makes the analogy in Project Swap: a lawyer represents the client while remaining bound by the law and the facts. Negotiating agents could use the whole framework—competence, confidentiality, loyalty to the client’s instructions and clear limits on authority. “Be good” is a rather thin replacement for all of that.

Professional honesty also leaves room to bargain. The American Bar Association’s Model Rule 4.1 prohibits a lawyer from knowingly making false statements of material fact or law to a third party. Its commentary recognizes the conventions of negotiation: estimates of price or value, and a party’s intentions about an acceptable settlement, ordinarily are not treated as statements of material fact. A negotiator can keep a walk-away price private without inventing material facts about the transaction. Those distinctions give developers something more useful to work with than a personality setting somewhere between ruthless shark and obliging neighbor.

An office book swap does little to test how those rules hold up under attack. Project Swap’s agents frequently named their first-choice book and almost never lied about it, while generally keeping their detailed rankings private. That behavior was observed in a cooperative corporate market. No deliberately malicious agents were sent in to exploit the others. On an open platform, agents will have to contend with false claims, attempts to extract a confidential budget, and messages designed to make a model treat a counterparty’s words as instructions. Give these systems authority to enter binding agreements at machine speed, and a bad book choice becomes a very modest illustration of what can go wrong.

“We looked only at well-behaved Claudes” — Anthropic, Project Swap

The resulting product requirements are fairly prosaic. Check that the agent understands the client’s preferences before it starts negotiating. Distinguish a target price from a hard limit. Establish which terms it may accept independently and which require human approval. Keep an auditable record of decisions, and test against counterparties deliberately trying to deceive or manipulate the system. Measure representation by outcomes, including missed opportunities, rather than relying on a survey of happy customers. Professional standards could provide a useful framework, provided they come with enforceable controls and accountability when things go wrong.

Anthropic also faces an awkward question about its own position. It builds negotiators of different strengths while helping define the rules under which they compete. A supplier serving both sides needs to explain how it separates their information and interests, what the premium model’s customer is buying, and how much discretion the supplier retains to adjust an agent’s loyalty. In “Superintelligence on the Honor System”, we encountered the same arrangement: the companies running the technology race appoint themselves its safety inspectors. Now the inspectors also sell representatives to both sides of the negotiating table.

These are useful experiments. Project Deal exposed a performance gap that clients did not appear to notice; Project Swap showed how much can be lost when an agent misunderstands its assignment. That is enough substance for a serious discussion of autonomous negotiation, without treating competence as a startling source of bargaining advantage. Legal ethics can help define what agents owe their clients and where legitimate bargaining ends. Its least glamorous requirement deserves attention first: understand what the client wants. Otherwise, an impressive negotiating performance can secure an outcome nobody asked for.