Why AI-Driven Client Apps Don’t Understand Your API

Recent surveys point to a massive growth in AI-driven bots crawling the internet looking for APIs. While many of these have malicious intent, a growing number are well-meaning API consumers just trying to discover, consume, and benefit from existing APIs. And, increasingly, these API requests are coming from MCP-driven platforms (Model Context Protocols) designed to enable autonomous software to interact directly with web APIs.

And, if recent statistics are any guide, they’re struggling. The success rate for multi-step AI-driven API workflows is about 30%. Worse, these clients often don’t give up. Instead, they keep trying—and failing—to interact with your APIs, driving up traffic while driving down the overall value proposition of target APIs.

So, what’s happening here? Why are AI-driven clients unable to take advantage of today’s APIs? And what will it take to turn this around?

It turns out the answer has been there all along. The things that AI-driven API consumers need are the same things that human developers need: clarity, context, and meaningful structure. Yet many companies still aren’t paying attention. And, as we learned back in 2017, “Attention Is All You Need.”

Are You Paying Attention?

The landmark 2017 paper “Attention Is All You Need” introduced the world to the notion of transformers. In the world of AI, a transformer is a model where words are mathematically scored based on their relationships to other words in the surrounding content. This scoring, referred to as attention, makes it possible for programs that use transformers (like ChatGPT) to produce responses that feel remarkably coherent to human readers.

The ability to use transformers to drive generative AI tools makes it imperative that we all rethink the way we design, document, and implement our APIs. In a nutshell, transformers pay attention to all the content they have access to, but they don’t understand any of it. Even more to the point, genAI platforms like ChatGPT, Claude, Gemini, and Copilot can easily pay attention to your API design. They can identify the URLs, the HTTP methods, the inputs, the schema, and the expected outputs. But they can’t perform any reasoning about which API to use and what the content in the returned body actually means.

Essentially, today’s AI-driven bots are fast and flexible API consumers that can’t find their way out of a wet paper bag. The good news is that we can take advantage of an AI-driven client’s skills at paying attention and add support within our API design to make up for its inability to make wise choices.

And that is a clear recipe for making your APIs AI-ready.

Things You Can Do Now to Level the Playing Field

Since AI-driven API clients are going to be good at pattern-matching, recognizing repeated content, and making associations based on context, we can use those skills to fill in the gaps LLM apps have regarding decision-making, meaning, and understanding.

Below are four practices that we already know make it easier for human developers to understand and use our APIs. It turns out these are the same things that will help AI-driven API clients be more successful, too.

• Be explicit: Don’t assume clients understand what this API does
• Tell them why: Provide clear descriptions of why and when clients might use the API
• Be consistent: The more your API looks like the thousands of others in the LLM’s training data, the better
• Make error responses actionable: Provide clear, consistent, detailed feedback that makes it easier to resolve runtime errors

Let’s look at each of these in turn.

Be explicit

Unlike humans, machines are not intuitive explorers. While they are great at parsing text and making associations, machines don’t make intuitive leaps. Instead, machines need explicit affordances; clues about what can be accomplished, how to do it, and why you might want to execute an action. The classic human-centric approach of designing and documenting an API is captured in this terse list:

• GET /customers/
• GET /customers/{id}
• POST /customers/
• PUT /customers/{id}
• DELETE /customers/{id}

Most humans know exactly what this list is communicating; the full list of available operations for managing a collection of customer records. Humans would look in other places in the API design documentation to determine the required and optional data properties to pass for each action as well as the format in which to cast the interactions (JSON, XML, HTML, etc.).

But machines can’t be trusted to exhibit that level of understanding and curiosity. They’re more likely to just make some “statistical guesses” about what this table represents and how to use it. To increase the chances of success and reduce the likelihood of mistakes, it is better to be much more explicit in your API documentation for machines. As in the following documentation example that is tuned for LLM consumption:

• To retrieve a list of customer records use GET /customers/
• To retrieve a single customer record use GET /customers/{id} while supplying the proper value of {id}
• To create a new customer record use POST /customers/ with the createCustomer schema
• To update an existing customer record use PUT /customers/{id} with the updateCustomer schema while supplying the proper value for {id}
• To remove a customer record from the collection use DELETE /customers/{id} while supplying the proper value for {id}

While these two lists essentially carry the same meaning for humans, the second list is much more helpful for machine-driven API clients.

Tell them why

Focusing on being explicit is a great way to improve the success rate of AI-driven client applications. Another way you can do this is to provide details on why an API client might want to use a particular API end point. It is important to keep in mind that AI-driven clients are pretty good at guessing how an API can be used but these same LLMs are not very good at figuring out why they should be used. You can fix that by adding text that explains the common uses for each API endpoint.

For example, in your documentation, include phrases such as “Use the PriorityAccounts endpoint to identify the top ten customers based on market size.” Or “Use the submitApplication endpoint once all the other steps in the employee application process have been completed.” These descriptions provide additional hints to API consumers on why or even when the APIs will be most helpful.

Note that, in both cases, the text identifies the endpoint by name and explains the reason an API client might use that API. AI-powered clients—especially those backed by LLMs—are very good at recognizing text like this and associating it with other text in your documentation such as the list we reviewed in the previous section.

Be predictable

The real power behind LLM-based client applications is found in all the documents and code these language models have scooped up as training data. All the books, papers, and source code fed into LLM databases provide statistical context for any new text your API documentation provides. It is the accumulated historical effort of thousands of writers, programmers, and software architects that makes it possible for AI clients to interact with your API.

And those interactions will be much smoother if your API looks a lot like all those other APIs it was fed as training data. If your API design contains lots of unique elements, unexpected responses, or non-traditional use of common protocols, AI-driven applications will have a harder time interacting with it.

For example, while it is perfectly “correct” to use HTTP PUT to create new records and HTTP PATCH to update existing records, most HTTP APIs use the POST to create records and PUT for updating them. If your API relies solely on a unique way to use PUT and PATCH operations you are probably making things harder on your AI-driven apps and reducing your chances of success. Or, if your API is exclusively dependent on a set of XML-based Schema Definition documents, AI-powered API clients that have been trained on thousands of lines of JSON Schema might not recognize your API input and output objects and could make mistakes when attempting to add or update data for your API.

Whenever possible, take advantage of common patterns and implementation details when building your API. That will better ensure AI clients can recognize and successfully interact with your services.

Make error responses actionable

When humans encounter errors in user interfaces, they usually can scan the displayed error information, compare it to the data they already typed in, and come up with a solution to resolve the error and continue using the service. That is not very easy for machine-driven API clients to handle. They don’t have the ability to scan the unexpected response, derive meaning, and then formulate a creative solution. Instead they either try again (maybe with some random changes) or just give up.

When designing your APIs to support machine-driven clients, it is important to apply the same three rules we’ve already mentioned (be explicit, tell them why, and be predictable) when API clients encounter errors.

First, make sure the client application recognizes the error situation. For API clients, this is more than just returning HTTP status 400. You should also include a formatted document that identifies and explains the details of the error. A great way to accomplish this is to use the Problem Details for HTTP APIs specification (RFC7078) format. This response gives you a structured way to identify the problem and suggest a possible change in order to resolve the error.

Note that this response also meets our criteria for the second rule (Tell them why). This update failed because a field was missing and that field is hatsize. The error report even tells the machine what they can do in order to make another attempt at updating the record.

Another advantage of using the RFC7078 format is that it helps us meet the third rule (Be consistent). This RFC is a common specification found in many API examples and is quite likely that the LLM’s training data contains lots of these responses. It is better to use this existing error format instead of relying on one you created yourself.

Finally, it is a good idea to design your APIs to treat errors as partial attempts. Most of the time, API errors are just simple mistakes caused by inconsistent or missing documentation and/or inexperienced developers. Providing explicit error information not only helps resolve the problem more easily, it offers an opportunity to “re-train” machine clients by populating the machine’s local context with examples of how to resolve errors in the future.

Remember, LLM-based clients are great at recognizing patterns. You can use that when you design your APIs, too.

Pay Attention to Your AI-driven API Consumers

As mentioned at the start of this article, the things identified here as a way to improve your interactions with AI-driven API clients are all practices that have been suggested in the past for improving the design of APIs for human interaction.

Being explicit cuts down on the cognitive load for developers and helps them focus on the creative problem-solving work needed to use your API to solve their immediate problem.

Telling them why makes it easier for developers to identify the APIs they need and to better understand the way they work and when they can be applied.

Being consistent is another way to reduce cognitive load for programmers and provide a more “intuitive” experience when using your API.

And, making error responses actionable leads to better error feedback and more consistent error resolution both at runtime and design time.

Finally, all these practices work better when you keep a close eye on the way API clients (both human- and AI-driven) actually use your service. Make note of which endpoints are commonly used. Identify persistent error conditions and how they get resolved. And keep track of API client traffic as a way to gauge which APIs provide the most return for your effort and which are more trouble than they are worth. Quality monitoring of your APIs will help you better understand who is using them and what kinds of trouble they are having. That will give you clues on how you can redesign your APIs in the future to improve the experience for everyone.

Whether you’re supporting human-driven API consumption or machine-driven clients, paying attention can pay off handsomely.

Read More

AI & ML, Commentary

Radar