The key word is "whole". @Retry from MicroProfile Fault Tolerance wraps a CDI bean method. On an AI service, that method is the entire conversation with the model: the first request, the tool calls the model asks for, the tool results sent back, the next request, and so on until the model produces a final answer. Every tool invocation in that loop runs inside the method call. When the method throws after the second round-trip, @Retry does exactly what it was built to do and starts the method over from the first message. The charge that went through during the first attempt is invisible to the retry. The model gets the same prompt and asks for the same tool, and the charge goes through a second time.

The guide's answer is a contract: "Tools are expected to be idempotent", and you are the one who has to "make sure your tool implementations tolerate being invoked multiple times for a single logical operation". The tool is yours, so the contract is yours. That note was not the first answer, though. The team implemented a build-time check for exactly this situation, then took it out and moved the guidance to the documentation.

The retry sits outside the loop, and you can see it in a stack trace

SmallRye Fault Tolerance, the MicroProfile Fault Tolerance implementation Quarkus ships, nests the strategies in a fixed chain around the method: fallback outermost, then retry, then circuit breaker, rate limit (a SmallRye addition, not part of the MicroProfile spec), timeout, bulkhead, and finally the call itself. Retry re-runs everything inside it. On a plain service method, "everything inside" is one operation. On an AI service with tools, it is a loop that talks to your database, your payment provider, your mail server.

Run the project below with a deliberately wrong API key and you can watch it happen. The log shows three POSTs to the chat completions endpoint with identical bodies: the same system message, the same user message, the same tool definition. One call, two retries. The stack trace of the final failure reads, from the outside in: io.smallrye.faulttolerance.core.retry.Retry.retryLoopIteration, then CheckoutAssistant$$QuarkusImpl_Subclass.checkout, then AiServiceMethodImplementationSupport.implement, the generated code that runs the model-and-tool loop. The retry is the outer frame. The loop, tools included, is the inner one.

The same trace shows a second, smaller retry: dev.langchain4j.internal.RetryUtils.withRetry inside OpenAiChatModel.doChat. Its unit is a single HTTP request to the model. That retry can repeat a network call; it cannot repeat a tool that already ran, because it wraps the request and never the loop. Two retries with different units, and the one on the interface has the wide blast radius.

With a bad key, nothing reaches the tool, so the three attempts are harmless. Move the failure one step later, after the model has called charge and the provider times out on the second round-trip, and the three attempts become three charges.

A build-time check, implemented and reverted

PR #2749 in quarkiverse/quarkus-langchain4j, merged on August 24, 2026, "reverts the build-time fault tolerance check (and its test)" and "keeps and expands the CAUTION note in guide-fault-tolerance.adoc, explaining that retrying an AI service method restarts the agent loop and that tools must be idempotent".

The description states the reasoning: after discussion with maintainers @geoand and @cescoffier, "instead of a build-time check, the guidance belongs in the documentation". In the review thread, @cescoffier wrote: "I would only write a warning in the doc", and: "For me it's normal that tools are retried, the developers must implement idempotency correctly." @geoand replied: "Exactly".

So the position is explicit: retrying tools is normal and idempotency is the developer's job. One caution note in the guide is all the framework gives you for it.

What the contract looks like in code

The project is small: an AI service with @Retry, one tool that charges an order, a gateway that refuses to charge the same key twice, and a REST endpoint to trigger it from Swagger UI.

<dependencies>
    <dependency>
        <groupId>io.quarkus</groupId>
        <artifactId>quarkus-rest-jackson</artifactId>
    </dependency>
    <dependency>
        <groupId>io.quarkus</groupId>
        <artifactId>quarkus-smallrye-openapi</artifactId>
    </dependency>
    <dependency>
        <groupId>io.quarkus</groupId>
        <artifactId>quarkus-smallrye-fault-tolerance</artifactId>
    </dependency>
    <dependency>
        <groupId>io.quarkiverse.langchain4j</groupId>
        <artifactId>quarkus-langchain4j-openai</artifactId>
    </dependency>
</dependencies>

The gateway stands in for a payment provider. The only property that matters is the map: charging a key that already has a receipt returns that receipt and does nothing else.

// src/main/java/com/example/checkout/Receipt.java
package com.example.checkout;

public record Receipt(String id, String idempotencyKey, double amount) {
}

// src/main/java/com/example/checkout/PaymentGateway.java
package com.example.checkout;

import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;

import jakarta.enterprise.context.ApplicationScoped;

/**
 * Stand-in for a real payment provider. The one property that matters here:
 * charging the same idempotency key twice returns the first receipt instead
 * of charging again.
 */
@ApplicationScoped
public class PaymentGateway {

    private final Map<String, Receipt> receipts = new ConcurrentHashMap<>();

    public Receipt charge(String idempotencyKey, double amount) {
        return receipts.computeIfAbsent(idempotencyKey,
                key -> new Receipt(UUID.randomUUID().toString(), key, amount));
    }
}

The tool uses the order id as the key. One order, one charge, however many times the loop asks.

// src/main/java/com/example/checkout/PaymentTools.java
package com.example.checkout;

import dev.langchain4j.agent.tool.P;
import dev.langchain4j.agent.tool.Tool;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;

@ApplicationScoped
public class PaymentTools {

    @Inject
    PaymentGateway gateway;

    @Tool("Charges the customer for an order. Calling it again for the same order does not charge twice.")
    public String charge(@P("the order id") String orderId,
                         @P("the amount to charge") double amount) {
        Receipt receipt = gateway.charge(orderId, amount);
        return "charged " + receipt.amount() + " for order " + orderId
                + ", receipt " + receipt.id();
    }
}

The AI service is where @Retry goes. Here it means: if checkout throws, run checkout again, tools and all, up to two more times.

// src/main/java/com/example/checkout/CheckoutAssistant.java
package com.example.checkout;

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.quarkiverse.langchain4j.RegisterAiService;
import org.eclipse.microprofile.faulttolerance.Retry;

@RegisterAiService(tools = PaymentTools.class)
public interface CheckoutAssistant {

    @SystemMessage("You are a checkout assistant. Use the charge tool to take the payment, then confirm it to the customer in one sentence.")
    @UserMessage("Charge order {orderId} the amount of {amount}.")
    @Retry(maxRetries = 2)
    String checkout(@V("orderId") String orderId, @V("amount") double amount);
}

// src/main/java/com/example/checkout/CheckoutResource.java
package com.example.checkout;

import jakarta.inject.Inject;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.QueryParam;
import jakarta.ws.rs.core.MediaType;

@Path("/checkout")
public class CheckoutResource {

    @Inject
    CheckoutAssistant assistant;

    @POST
    @Produces(MediaType.TEXT_PLAIN)
    public String checkout(@QueryParam("orderId") String orderId,
                           @QueryParam("amount") double amount) {
        return assistant.checkout(orderId, amount);
    }
}

# src/main/resources/application.properties
quarkus.langchain4j.openai.api-key=${OPENAI_API_KEY}
quarkus.langchain4j.openai.log-requests=true
quarkus.langchain4j.openai.log-responses=true

Export OPENAI_API_KEY, run ./mvnw quarkus:dev, open http://localhost:8080/q/swagger-ui and POST to /checkout with an order id and an amount. Request and response logging is on so you can watch each round-trip of the loop, including the tool call the model emits and the tool result that goes back.

With a real provider, the in-memory map is replaced by whatever the provider offers for the same purpose. If it accepts an idempotency key with the charge request, the key goes there. If it does not, the receipts table is yours to keep, and it has to survive a JVM restart, because the retry that hits you in production may come from a different instance than the one that made the first charge.

The key has to be minted outside the method

Charging is the easy case. An order id is a natural idempotency key: the business already agrees that one order is charged once. The model has the id in the prompt and passes it to the tool as an argument.

Sending an email has no natural key. "Send the customer a confirmation for order A-100" executed twice is two confirmations, both correct on their own terms. To make that tool idempotent, something has to identify the logical operation, and that something cannot be created inside the retried method, because the retried method is precisely what restarts. A UUID generated inside checkout is a new UUID on every attempt, which makes every attempt a new operation and defeats the purpose.

The caller mints the key once, before invoking the AI service. The tool reads it from the request. Tool arguments are whatever the model decides to write into the call; a key that has to travel through the prompt and come back intact is a key the model can drop, shorten or reinvent. The caller sets it and the tool reads it; the model never sees it.

This also shows in the gateway signature. charge(String idempotencyKey, double amount) cannot be called without a key. A reviewer reading the tool sees the contract in the parameter list. The tool description, "calling it again for the same order does not charge twice", is written for the model; the signature is written for the next developer. A model can ignore the description; the compiler will not let anyone skip the parameter.

@Timeout plus @Retry is the combination that produces the incident

The same guide has a second caution: "Use @Timeout carefully when using tools or multi-step function calls, as those may require more time to complete due to multiple background interactions with your application's tools or services."

Stack the two and the chain order does the rest. Timeout sits inside retry. A timeout that fires on the third round-trip, after the tool ran on the second, throws exactly the exception retry is waiting for. Retry restarts the method from the first message and the tool runs again. A slow provider and a tight timeout are enough; no bug in your tool is required.

The documentation note assumes you handled this scenario in advance by making the tool idempotent. Whether a note is the right place to ask for that is the next question.

Is a documentation note enough when the effect cannot be undone?

The maintainers' position has real arguments behind it.

A build-time check cannot tell a tool that reads from a tool that writes. A @Tool that searches a catalog and a @Tool that charges a card look identical to the build. The check either fires on every @Retry on every AI service with tools, which turns it into noise that gets silenced, or it needs a marker annotation on side-effecting tools, which people forget to add and which then produces a false sense of safety. Idempotency is a property of the tool's implementation and its backend, and the compiler cannot see either. And for read-only tools, retrying is correct: a catalog search run three times costs latency and nothing else. "It's normal that tools are retried" is a defensible description of the general case.

The counter-argument is about who reads the note and when. The person who writes the first tool reads the fault tolerance guide in week one, if they read it at all. @Retry gets added in month six, by someone else, during an incident, because the model provider is flaky and a retry is the obvious fix. At that moment, the note is not on the screen. A warning in the build log would be. The reverted check existed, and it had a test; the project decided the cost of that warning was higher than its value. For tools that search and summarize, I agree. For tools that move money or send mail, a line in the build output at the exact moment someone adds the annotation is cheaper than the refund.

What the framework's choice leaves you with is a floor, and you build the rest:

• Keep @Retry off AI service methods whose tools write. The per-request retry inside the provider client already covers the transient network failure, at the unit where retrying is safe.
• If the method-level retry is genuinely needed, every side-effecting tool takes a key minted outside the method.
• Treat @Timeout together with @Retry on an AI service with tools as a decision to be written down, with the double-execution case covered. Defaults copied from a REST client don't cover it.
• Put the contract in the signature. A gateway that requires an idempotency key is a check that runs on every call, in every codebase, without a plugin.

The sentence in the guide is a contract, and you sign it the moment you add the annotation. The framework tells you once and trusts you from there. Whether that trust holds is decided in the parameter list of your gateway, and nothing else in the build is going to check it for you.

That paragraph is the idea. The rest is about what the overlap looks like from the inside, and what to do when it seems to be missing.

The first time I said it out loud

There was a point in my career, in the hiring conversations with a company I was about to join, when I said it for the first time: let me use what I do, what I believe in, and what I like doing, to work for this company.

That sentence starts with me: what I already do, what I already believe, what I already like. The company comes in as the place where all of that gets put to work.

The magic of the whole thing, I think, is exactly this: you take what the company wants and cross it with what you want. Then you find the overlap. When you do that, you can literally do whatever you want inside the company.

The condition is delivery. The overlap is the region where the results you produce are, at the same time, what the company asked for and what you wanted to build. You deliver right there at the intersection. The freedom comes with the delivery, inside the work you were hired to do.

There's also a personal reason the overlap matters so much to me: I have a hard time talking about things I don't believe in. So, for me, the overlap has to hold what I believe in. When what the company needs me to say is the same thing I actually believe, that difficulty simply goes away.

When your audience changes the deal

My friend and Java Champion, Bruno Souza, once asked me:

"While you were working for that company, you built a community. You even built a business of your own, right? One that was able to survive all of those different things."

And that's where the relationship flips. Once you have an audience of your own, the invitation you get from a company changes. It used to be: come over here and talk to our audience. Now it's: come over here, we want to talk to your audience.

From that point on, the company is hiring you to talk to your audience. What it's after is the audience you built. The overlap is still there, and you still deliver at the intersection. What changed is who brings the audience to the table.

In practice, that means this: I have a freedom to do what I want that you probably can't even picture. The company wants me talking to the people I already talk to, about the things I already believe in. The overlap doesn't get any wider than that.

"But where I am now, there's no overlap"

The objection I hear at this point is: Ok, but where I am now, there's no overlap.

My answer is that there is. Sometimes it's hard to find. But hard to find and not there at all are two different things.

And if it really is impossible, then maybe it's time for one of two moves: you change what you want to do, or you change companies. And that's OK too.

Either way, the freedom has one source: the overlap between what the company wants and what you want, and the results you deliver right there. I'd like to know where yours is today.

Writing code was never the same thing as making software. The typing was always the cheap half, and the cheap half is the one that got cheaper.

Matt Pocock points to the other side: bad code is far more expensive today than it used to be. The reason is speed. The rate at which I can push bad code into production has no ceiling anymore. Whatever I used to get wrong once a sprint, I can now get wrong continuously.

Here is what I see day to day. AI does great work on a good codebase. Well structured, well written, documented, and it just flies. Working with it is a pleasure. On a bad codebase, it goes the other way and starts going nuts.

I once read Venkat Subramaniam stating the sharpest version of why. Developers say AI-generated code is bad. AI was trained on code written by developers. That, he says, is called karma. 🙂

So none of this is a new problem. We have been dealing with design, structure, and documentation for a very long time, and we already know what happens when they are missing. What changed is the clock. The same problems now arrive faster, and usually at a much bigger scale.

If you are running AI on a legacy codebase right now, I would like to hear which of the two you are getting: does it fly, or does it go nuts?

Then I read the code:

@Produces
@ApplicationScoped
public PromptGuardAgent promptGuardAgent() {
    ChatModel guardModel = OllamaChatModel.builder()
            .baseUrl(baseUrl)
            .modelName(guardModelName)
            .temperature(0.0)
            .think(Boolean.FALSE)
            .timeout(Duration.ofSeconds(timeoutSeconds))
            .build();

    return AiServices.builder(PromptGuardAgent.class)
            .chatModel(guardModel)
            .build();
}

This is real output from an agent in one of my projects. It even shows some care: temperature zero, an explicit timeout. But look at what it is: a producer assembling the ChatModel by hand and calling AiServices.builder. That's raw LangChain4j inside a Quarkus application. The Quarkus way is declarative: @RegisterAiService on the interface, the model named in application.properties. And the difference is expensive. With manual wiring you lose what the integration gives you for free: metrics and traces on every model call, declarative fault tolerance, guardrails you can plug in. None of that stops the app from running today. That's exactly where the danger lives: the problem doesn't show up on day one.

The bill always arrives

The gap between "works" and "well built" appears when the system lives on. It appears when someone has to maintain that code six months from now. It appears when traffic grows and resource usage becomes real money. And the pattern multiplies: a class with ten lines that could be one is a detail; ten thousand of them is an extra codebase you didn't need, polluting the context of every future agent session that touches it.

Markus Eisele, who writes The Main Thread and is on my team at IBM, compressed all of this into one line at his JCON keynote this year: "Code is cheap now. Software is not. You just get there a lot faster now." The cost of producing code collapsed. The cost of software (intent, correctness, maintenance, complexity) stayed exactly where it always was. He closes with two variations I keep coming back to: code is cheap, intent is not; code is cheap, verification is everything.

Matt Pocock inverted the same argument at the AI Engineer conference: bad code is the most expensive it has ever been. If your codebase is good, AI multiplies your speed. If it's bad, AI sinks with it and produces more bad code on top. Engineering fundamentals gained value in the age of agents. Good practices stopped being a reviewer's nitpick and became an economic argument.

And none of these pains are new. "It didn't do what I asked": Frederick Brooks, The Design of Design. "Too verbose": Eric Evans, Domain-Driven Design. "Done, but broken": Andy Hunt and Dave Thomas, The Pragmatic Programmer. "Hard to test": John Ousterhout, A Philosophy of Software Design. These books didn't break with AI. They became more important. We don't need new fundamentals; we need the usual ones, written in a format the agent can use.

Venkat Subramaniam summed up the irony: developers say AI-generated code is bad; AI was trained on code written by developers; that's called karma.

A good engineer with no memory

Why does the agent miss what a senior engineer gets right? Because everything an experienced engineer carries implicitly (the house patterns, the known traps, the right way to test) the agent doesn't carry between sessions. It's born again in every conversation. That knowledge has to be written somewhere the agent reads. If it isn't written, for the agent it doesn't exist.

Eisele has a phrase for the consequence, and I've adopted it: any structure you present to the model beats any clever prompt. A good prompt solves the task at hand. Structure (a conventions file, skills, MCP servers) solves every task that comes after. It's the difference between asking well and teaching once.

Fabio Akita put it in a way that lands: AI reflects who you are. If your engineering practice is messy, the agent amplifies the mess. If it's explicit and disciplined, the agent amplifies that instead. So the question stops being "does AI write good code?" and becomes: what am I giving it to read?

All of this lands on one thesis: the agent needs to read before it writes. The good news is that in our ecosystem, that work had already started.

What the Quarkus ecosystem already gives you

The Quarkus team attacked part of the problem with the Quarkus Agent MCP: a standalone Model Context Protocol (MCP) server that teaches your agent to code using Quarkus. It creates projects and manages the app lifecycle. It exposes skills per extension and runs semantic search over the whole documentation. And it carries a design detail I find brilliant: it runs outside the application process, so when the app crashes, the agent stays alive, reads the structured exception and goes fix the problem.

When I started using it, the quality of the Quarkus code my agents produced jumped immediately.

But a real application is bigger than the framework. There are the design patterns of the Java world, there are tests, there's persistence. And there's LangChain4j, which for me is the most critical case: a young ecosystem with a fast release pace, where the model has little training baggage. That's exactly where being specific and opinionated pays the most. The stakes are not small either: the State of Java survey by Azul found that among companies building AI features, half use Java in the implementation. This problem is ours.

Quarkus Agentic Scaffolding

So I built Quarkus Agentic Scaffolding. In one sentence: an installable artifact that puts Quarkus + LangChain4j best practices in front of the agent before it writes the first line.

It's open source under Apache 2.0. The templates are code that compiles, validated in CI. And it installs on practically any assistant that supports the Agent Skills format (Claude Code, Codex, Copilot, Cursor, IBM Bob and others) with one command:

npx skills add eldermoraes/quarkus-agentic-scaffolding

The architecture separates two things that usually come tangled together. On one side, an always-on conventions file (CLAUDE.md for Claude, AGENTS.md for the rest) that declares the rules the code must follow. On the other, three skills that execute procedures. The skills don't repeat the rules; they point to the file. One source of truth.

A sample of what the conventions declare:

• Java 25+ as the baseline
• Virtual threads by default for blocking work
• records, sealed types and pattern matching where they clarify intent
• Platform BOMs instead of pinned extension versions
• CDI-first
• Declarative AI services with @RegisterAiService
• Declarative guardrails
• Easy RAG first, portable to something heavier later
• Zero-code observability with Micrometer and OpenTelemetry

Nothing in that list is generic clean-code advice. It's the current way of building on this stack, and every line exists because it fixes a mistake agents make in the real world.

The plugin ships three skills.

/setup-agentic-scaffolding prepares the ground, and it carries a design decision I care about: the plugin doesn't try to be an island. It verifies your toolchain: JDK 25 / GraalVM, JBang, a container runtime. It registers the Quarkus Agent MCP and Context7, for library documentation fresher than the model's training data. And it recommends the Superpowers skill set for process. Each of these pieces expands what the agent can do well, and the plugin uses all of them along the way.

/scaffold-project covers both moments of creation: the new project end-to-end, and the new component inside a project that already exists (an AI service, tools, agents and multi-agent workflows, RAG, an MCP client or server, guardrails). The project skeleton comes from the Quarkus Agent MCP; the skill applies the layout, the base application.properties and the starter templates. You trigger it in natural language: ask for a RAG pipeline and it enters the flow on its own.

/audit-project closes the loop, because most projects won't be born now; they already exist. It compares the project against the conventions and returns a prioritized list: what's missing, where the evidence is, what the suggested fix looks like. It's read-only by default and only applies fixes with your confirmation. There's a foundation behind keeping this separate, and Andrej Karpathy named it: generating code and judging code are different capabilities. The audit puts the agent in the reviewer's seat with an explicit ruler in hand. The same kind of agent that wrote the code at the top of this post, now enforcing the practices it used to skip.

Does it save tokens?

A fair question: do the skills reduce token usage, or do they only improve quality? Honest answer: I haven't measured it yet. The runs were clearly faster, and my reading is that more upfront direction means less reasoning, because reasoning is largely the agent validating its own inferences. Fewer inferences to validate, fewer tokens burned. I intend to measure it properly and publish the numbers.

Skills: a new kind of open source asset

Conventions rot if nobody maintains them, so the project tracks the releases of Quarkus, Java and LangChain4j: when the stack moves, the target moves with it. Semantic versioning, a changelog, CI making sure the templates still compile.

There's a line from the DevStar working group, inside the Quarkus project, that frames this whole space: tools do things; skills teach how to do things well. I believe this becomes a new kind of open source contribution: library maintainers and platform teams writing knowledge for agents to read. And it works inside your company too. Your team's conventions can live in a file the agent reads before writing. The model is the same.

Try it on your own project

The sentence that made me start this project is the invitation I'll leave you with: I want to shorten your path to building applications that follow best practices while using AI.

Install it, scaffold a new project, audit one an agent already wrote. Then tell me what you find. Issues, pull requests and criticism are welcome: github.com/eldermoraes/quarkus-agentic-scaffolding.

The paper is Harness Engineering for Agentic AI Coding Tools: An Exploratory Study. It builds on work published in AIware 2026. The stated method: identify eight configuration mechanisms spanning from static context to executable and external integrations, then examine whether and how they are adopted across those 2,853 repositories, with a detailed analysis of Context Files, Skills and Subagents.

Two findings carry the paper.

First: Context Files dominate the configuration landscape and are often the sole mechanism in a repository, with AGENTS.md emerging as an interoperable standard across tools.

Second: few repositories adopt advanced mechanisms such as Skills and Subagents. Skills predominantly rely on static instructions rather than executable scripts.

Read those two sentences next to any vendor keynote from the last twelve months and you will notice the gap.

What "sole mechanism" means when you are the one maintaining the repo

The phrase doing the most work in that first finding is often the sole mechanism. Read it literally: configuration begins and ends there. A repository that has configured its coding agent has, in a large share of cases, written one markdown file and stopped.

If you use a coding agent daily on an enterprise Java codebase, that result is either deflating or vindicating depending on what you have been told to feel bad about. The narrative you have been sold says the sophisticated shop graduates from a context file to skills, then to subagents, then to a fleet of them coordinating. The observed behavior says the sophisticated shop writes a good markdown file and gets back to work.

There is a version of this finding that is just "adoption takes time, the advanced stuff is new." That reading is available and probably partly right. But it does not survive the second half of the second finding: Skills predominantly rely on static instructions rather than executable scripts.

The mechanism people adopted, minus the part that made it a mechanism

Skills, as a category, exist to hold something a context file cannot: procedure that runs. A skill can carry a script the agent executes: a deterministic step, a verification, a generator, something with an exit code. Executable scripts are optional in a skill (only the SKILL.md file is required), which is exactly why a skill can collapse into a longer paragraph of AGENTS.md with a folder around it.

The finding says the repositories that adopted Skills largely did not adopt that part. They used the skill container to hold more static instructions. So the file structure changed and the mechanism count went up. The configuration surface stayed exactly where it was: text that shapes what the model reads, with nothing that constrains what the system does.

That is a much more specific result than "advanced features have low adoption." It says people reached for the advanced mechanism, and then used it as if it were the basic one.

Three explanations, and the one you can test

The paper reports what it observed. Why executable configuration did not take is where the argument is, and from here I am reasoning past the data. Three explanations fit the shape of the finding:

Maintenance cost. An executable skill is code. It has dependencies and an owner. It breaks when the repo moves, and it needs updating every time the toolchain changes. A markdown file has none of those properties. It degrades silently where code fails loudly, which is worse in principle and much cheaper in practice. Anyone who has maintained a Makefile that only three people on the team can modify already knows this tradeoff and made the same call.

No visible return. When you improve AGENTS.md, the next agent run reflects it immediately and legibly: you read the output and see your instruction landed. When you add an executable skill, the benefit is conditional on the agent choosing to invoke it, in a situation that matches, on a run you may not observe. Static context pays you back in the same session. Developers optimize what they can see move.

No good way to test it. This is the explanation I would put money on, and it is the one with a concrete fix. We have decades of practice testing application code and roughly none testing agent configuration. What is the assertion for "this skill fires under the right conditions," and what does an error look like? A team with a mature CI pipeline can tell you in minutes whether a change broke the build. That same team cannot tell you whether last week's edit to their agent config made the agent better or worse, and they know it. Executable configuration without a test harness is a component you ship untested into a nondeterministic runtime. Java teams built an entire testing culture around not doing that.

The three are not exclusive. But notice which one you can act on: cost and visibility are properties of the ecosystem, and testability is a thing a team can build.

Claude Code users reach for more mechanisms than anyone else

One result deserves separate treatment: users of Claude Code employ the widest range of mechanisms among the tools studied.

In other words: a particular tool attracts more sophisticated practitioners. Read it as a fact about surfaces and you get something more useful. A mechanism that a tool does not expose cannot show up in a repository, no matter how sophisticated the person typing. If adoption breadth tracks the tool and not the team, what the study measures is the ceiling each tool sets.

What to actually do with this

The paper suggests AGENTS.md as a natural starting point, and the empirical result gives that suggestion teeth: it is the interoperable standard across tools, so the effort transfers when you switch. For most repositories it is also, apparently, where the work ends. Given the state of the tooling, that is a defensible place to stop.

The move that follows from the data has no glamour at all. Write the context file, make it genuinely good, and treat every step beyond it as a claim you have to justify. Before adding an executable skill, answer the question those 2,853 repositories answer with silence: how will you know it works, and how will you know when it stops working? If you have an answer, build it. If you do not, you would be adding a component to your build that apparently nobody will verify, which is the situation the static markdown file quietly avoids.

The market is selling multi-agent orchestration. The configuration files of thousands of repositories describe something plainer: one file, written carefully, doing most of the work. With a gap that wide between the pitch and the practice, the question worth asking is what the executable mechanisms still need before teams reach for them. Testability is the first item on that list.