An unsanctioned coding agent can become a company’s de facto standard before procurement notices. GitHub’s Copilot usage metrics API now reports third-party agent activity by individual agent, giving enterprises a way to compare what developers actually use with what the company officially approved.
The important evidence is no longer a few enthusiastic Slack messages or an expense claim buried in a team budget. Job starts and aggregated session counts can reveal that an approved agent has lost day-to-day adoption to another tool.
That discovery changes the question facing an engineering leader. The immediate issue is why developers chose the alternative, what access it received and how quickly the company can bring real usage under review.
Usage data exposes the gap between policy and practice
Procurement records describe authorized purchasing decisions. Usage data describes behavior.
Those records can diverge for ordinary reasons. The approved agent may perform poorly on the languages developers use most. Its authentication flow may add friction. It may lack access to the repositories, tools or workflows required for real work. An alternative may have spread through individual accounts, a limited trial or a team-level experiment that nobody formally closed.
The metrics alone do not establish which explanation is correct. A high session count does not prove better code, stronger security or lower total cost. Job starts can show that developers reached for a particular agent, but they cannot explain whether the resulting work was accepted, rewritten or abandoned.
Still, adoption data provides a stronger starting point than vendor claims. It lets engineering and procurement compare the approved choice with observed use, then investigate the difference.
The same principle applies to extensions, agent skills and other additions that enter development environments through a side door. The Unapproved Skill in the Build Log examines a related visibility problem at a smaller unit of deployment.
The first response should be investigation, not prohibition
A reflexive block can remove immediate exposure, but it can also erase evidence about why the approved option failed. Teams should preserve enough information to reconstruct the path of adoption before changing access.
Start by separating reported facts from assumptions. The metrics may establish which agents were used, how often sessions began and how activity was distributed. They do not establish developer intent, business value or acceptable risk.
Next, trace the operating context. Which repositories were involved? Did the alternative receive credentials, source code or customer information? Was its use concentrated in one team or spread across the organization? Did developers rely on personal accounts, company-managed identities or an integration already present in the toolchain?
Those questions turn a procurement surprise into a bounded review. They also prevent two common errors: treating every unsanctioned tool as equally dangerous, or assuming popularity proves suitability.
The review should include the developers who adopted the alternative. Their explanation may expose a practical failure in the approved setup, such as missing model access, slow responses, weak codebase context or a workflow that requires repeated manual steps. Those are hypotheses to test, not conclusions to announce.
Procurement needs evidence from real work
Coding agents are difficult to evaluate through feature tables alone. Vendors can list supported models, integrations and controls, while developers experience the product through a much narrower test: can it complete useful work inside this repository without creating more cleanup?
A stronger evaluation combines governance checks with measured trials. Give competing agents the same representative tasks. Record completion, developer intervention, review findings and any security or access concerns. Include routine maintenance work, unfamiliar code and tasks that require tool calls, since a polished demonstration may reveal little about daily use.
Session totals then become one input rather than the verdict. Adoption shows preference. Task evidence helps explain it. Security review determines whether the preferred tool can be used under acceptable controls.
This also gives procurement a better negotiating position. An approved vendor facing weak adoption should have to address the specific reasons teams avoid its product. An unsanctioned vendor with strong usage should still have to satisfy identity, data handling, audit and commercial requirements. Neither incumbent status nor developer enthusiasm should end the evaluation.
Build a process that can catch the next agent earlier
Agent adoption will keep moving faster than annual vendor reviews. The practical response is a recurring control loop built around inventory, usage and access.
Review agent-level activity on a fixed schedule. Compare the observed list with approved vendors and active trials. Assign an owner to every unexplained tool, then set a short deadline for classification: approved, restricted, under evaluation or blocked.
Publish a clear route for developers to request a trial. The route should identify what data the agent may access, how long the test runs and what evidence will decide the outcome. A usable exception process reduces the incentive to hide experiments.
Finally, connect usage review with access review. An unfamiliar agent with a handful of sessions and no sensitive permissions presents a different problem from one operating across private repositories with broad credentials. Counts reveal where to look. Permissions reveal what could happen.
The useful Monday-morning outcome is a named owner, a preserved record of activity and a review already on the calendar. By Friday, the organization should know why the alternative spread and whether to approve it, constrain it or remove it.
Sources
GitHub REST API documentation for Copilot usage
Comments
No comments yet.