I paid 345 agent APIs to see if they return the fields they advertise. 306 did. 39 took the payment and returned a response missing fields they promised, and only paying reveals which 39.
Every x402 seller ships a schema: send this input, get these fields back. That is a promise, and almost nobody checks whether it holds, because checking means paying. So I paid. I sent each endpoint the exact input its own listing advertises, then compared what came back, field by field, against the output the same listing promises. I ran 549 endpoints through it. 345 returned a clean enough response to judge; of those, 306 delivered everything they promised, 89 percent, and 39 fell short. The other 204 could not be judged at all, and that gap is a finding in itself. Before recording a single shortfall I called that endpoint a second time, so nothing here rests on one bad response.
- The 39 that fell short are not broken. They answer fast and quote the exact price they advertise, so a free probe rates them safe to use. api.myceliasignal.com, a price oracle, returned none of its 4 promised fields. x402.aispace.bot took payment for text-to-speech and sent back none of the 3 fields it lists. Others send some and drop the rest: stabletravel.dev sells airport weather and returned 5 of its 7 promised fields, dropping temperature and humidity, and agent.kihustle.tech promises 16 fields and returned 2. (Correction, September 7: this post originally said stabletravel.dev returned none of its 7 fields. That came from the parser bug described below; its archived responses regrade as 5 of 7 present, the receipts were corrected when the parser was fixed, and this sentence now matches them. It is still short, not zero.) Every one of these was confirmed on two separate paid calls before it was recorded, and the gap between looking reliable and actually delivering is exactly one paid call wide.
- The quieter finding is that a quarter of the market cannot be exercised by a plain agent at all. 204 endpoints needed an API key, rejected the very input they advertise, or answered with a 4xx or 5xx. Those are counted as inconclusive, never as a failure to deliver, because not getting a clean response is my limitation, not proof the seller shortchanged anyone. But it means the honest question, does it deliver, is only answerable for the endpoints that actually let you try, and today that is fewer than two in three.
- An earlier version of this counted one in four as not delivering. That was wrong, and it was my bug, not theirs. The test threw away the paid response before reading it, so endpoints that returned everything, vibesprings.net and api.seneschal.space among them, were scored as returning nothing. I caught it before it went up, pulled the data, and rebuilt the test: a response it cannot read is now inconclusive rather than failed, every shortfall is re-checked on a second call, a known-good endpoint runs first and aborts the whole run if the test scores it wrong, and the raw response is saved so the next mistake is a free re-check, not a re-run. The re-verification cost 86 cents, reconciled to the wallet on-chain.
Receipts and detailclick to expand
method, per endpoint:
read its published input example and output schema
pay it once with its own advertised input, payment capped at its advertised price
compare the returned fields against the fields it promised
if it falls short, call it again before recording anything
keep the verdict and the response shape (types only), throw away the values
549 checked: 306 delivered (89% of measurable), 39 short (11%), 204 inconclusive
returned NONE of what they promised (confirmed on two calls):
api.myceliasignal.com /oracle/price/btc/usd 4 promised, 0 back
vape-x402.vapex402... /scan/exploit_check 4 promised, 0 back
x402.aispace.bot /api/v1/audio/speech 3 promised, 0 back
voice.forgemesh.io /v1/tts/base 2 promised, 0 back
sent some, dropped the rest:
stabletravel.dev /api/flightaware/airports/KMIA/weather missing 2 of 7 (corrected Sep 7; was wrongly listed above as 0 back)
agent.kihustle.tech missing 14 of 16
agents.ai-rook.com missing 3 of 5
api.locus.report missing 3 of 4
could not measure (need a key, rejected our input, or 4xx/5xx): 204
these are not counted against the seller- Endpoints measured
- 345
- Delivered everything
- 306 (89%)
- Fell short
- 39 (11%)
- Could not be measured
- 204
- Returned more than promised
- 178
- Every short confirmed on
- 2 paid calls
- Re-verification cost, on-chain
- $0.86
- Shows up on a free check
- none of it