Postman Found That Too Many Tools Made Its AI Choose Badly

Postman Found That Too Many Tools Made Its AI Choose Badly

An AI assistant that can check orders, send emails, issue refunds and change stock has plenty of ways to make the wrong choice.

Postman saw that problem in its own testing. In an engineering report published with AWS on 9 October 2026, the team said errors increased when its AI could see more than about 40 tools. It sometimes picked the wrong action, sent the wrong information, or tried to use a tool that did not exist.

A tool is an action the AI can request, such as checking a record or sending a message. Postman now selects about 15 relevant tools from a collection of more than 170 for each task. Larger models reduced the mistakes but did not remove them.

The report describes an existing system, rather than a new launch. The figure of 40 is an observation from Postman’s tests, not a fixed limit for every AI product.

Give an order assistant an order job

Suppose a customer asks where their parcel is. A sensible starting point is to let the assistant check the order, read the delivery update and prepare a reply. Changing product prices or sending marketing emails has no place in that request.

For an online store, we would also separate looking up an order from issuing a refund. The assistant can prepare a refund request for someone to review before money moves.

Postman found that missing information caused more failures than missing tools. An order assistant needs the correct order, payment status, delivery record and customer request. Giving it more actions will not fix an outdated delivery record.

Try difficult cases before customers use it

Test a delayed parcel, a cancelled order, two customers with similar names and a payment that is still pending. Check which action the assistant chooses and whether its reply matches the records.

Record mistakes and staff corrections. When it cannot find the right order or confirm a payment, have it pass the case to a person. Add another tool only when you can name the task that needs it and show that the assistant uses it correctly.