# Is our AI spend paying off?

All squads. The last 90 days, compared with the 90 days before the rollout. Cursor, Claude Code, GitHub Copilot and Devin. Source: DevStats MCP. Illustrative example: the AI usage figures, seats and prices are simulated. The delivery metrics are real.

**Verdict: paying off, and the bill is coming in through review.** We pay $1,413 a month. The three people who work with agents every day went from 211 merged PRs to 662. In the same period rework doubled and reviews got much thinner. The spend is fine. What we do with the extra code is not.

## 1. What we pay for and what is used

| Tool | Seats | Active users | Utilization | Monthly cost | Cost per active user |
|---|---|---|---|---|---|
| Cursor | 7 | 5 | 71% | $280 | $56 |
| Claude Code | 5 | 4 | 80% | $500 | $125 |
| **GitHub Copilot** | **7** | **0** | **0%** | **$133** | no active user |
| Devin | shared | 3 | flat price | $500 | $167 |

We pay for 19 seats and 9 of them were used in the last 90 days. GitHub Copilot shows no activity at all: no active user, no accepted line, no PR reviewed. The team moved to Cursor and Claude Code and the Copilot seats stayed on the invoice. Two Cursor seats (dev-5, dev-6) and one Claude Code seat (dev-2) had no activity either.

Usage itself proves nothing. Cursor's 59.3k accepted lines and Claude Code's 89% tool acceptance say the tools are open on people's screens. They do not say anybody ships more. The next section does that.

## 2. What changed since the rollout

| | Before the rollout | Last 90 days | Change |
|---|---|---|---|
| PRs merged, everyone | 335 | 838 | +150% |
| PRs merged, the 3 daily agent users | 211 | 662 | +214% |
| PRs merged, autocomplete only (dev-2) | 85 | 109 | +28% |
| Coding time per PR | 16h 46m | 3h 23m | -80% |
| Coding + pickup + review per PR | 1d 13h 21m | 22h 12m | -41% |

The gain is concentrated where agents are used. dev-1, dev-3 and dev-4 had Claude Code or Cursor agent activity on most working days (59, 56 and 46 active days in their main tool). Together they went from 23.4 merged PRs per person per month to 73.6. dev-2 opened Cursor on 54 days, accepted 8,450 tab completions and made 0 agent requests: their output grew 28% and their PR cycle time went from 17h 45m to 1d 7h. Autocomplete alone did not move much.

Three caveats before anyone quotes these numbers:

- This is correlation. Two of the three agent users were already among the most active authors before the rollout. They may simply be the people who get the most out of any tool.
- PR size stayed flat (326 lines against 310), so the extra PRs are not one PR cut in three. But a PR count is still a count.
- The light users are not a control group. dev-5 and dev-6 merged 47 PRs before and 12 after because their work changed: dev-6 resolved 5 issues before and 213 after. dev-7 joined after the rollout and has no before. Deploy time is left out of the cycle time comparison because the earlier window has none recorded.

## 3. Side effects

The code is written five times faster and everything after it got worse.

- **Pickup time went up 67%**, from 8h 57m to 14h 57m. PRs are produced faster than reviewers pick them up. Part of the throughput gain sits in a queue.
- **Reviews got thinner.** Comments per review fell from 1.48 to 0.41 and review depth from 3.13 to 0.65 comments per PR. Review time dropped 67%, and reviews per merged PR fell from 2.1 to 1.6. Reviewers are approving 2.5 times more PRs by looking less at each one.
- **Rework doubled**, from 8.9% of changed lines to 17.7%. Code is being rewritten soon after it merges.
- **Stability slipped.** Change failure rate went from 2.1% to 3.1% and recovery time from 3d 10h to 5d 10h.

None of this proves AI writes worse code. It shows a review process sized for 335 PRs a quarter handling 838.

Devin is a separate case. Of its 131 sessions, 79 came from automations, almost all code review, and it completed 83 of the team's 1,315 reviews. As an author it opened 6 PRs and merged 4, and those took 5d 32m to merge against 1d 20h for everyone. Useful as a reviewer and investigator. Not as a PR author, at least not the way we use it today.

## 4. Recommendation per tool

| Tool | Decision | Monthly impact |
|---|---|---|
| GitHub Copilot | **Cancel all 7 seats.** Nobody used one in 90 days. | -$133 |
| Cursor | **Keep. Reclaim 2 seats** (dev-5, dev-6). Move dev-2 from autocomplete to agent mode. | -$80 |
| Claude Code | **Keep.** It is where the largest gains are. Give dev-2 30 days with dev-3 as a pair before deciding on the unused seat. | $0 now, $100 at stake |
| Devin | **Keep for review and investigation, stop assigning it PRs** until its merge time is near the team's. Review again next quarter: $500 for 3 people is the most expensive line per user. | $0 |

Reclaiming the unused seats saves $213 a month, $2,556 a year, 15% of the bill, with no effect on anyone's work. The bill goes to $1,200.

On the return: the agent users merge about 150 more PRs a month than before the rollout. If all of that is credited to the tools, each additional merged PR costs $9. If only a quarter is, $38. Either end of that range is cheap. I would not convert it into hours or headcount: the data does not support it.

**The money to spend is not on seats. It is on review.** Agree on a first look within the working day, keep PRs near the current 310 lines, and let Devin and AI review take the first pass so people spend their comments where it matters.

## 5. What to measure next quarter

1. **Rework share and change failure rate for agent users against everyone else.** If the quality cost is in AI-written code, it will show here. Today I can only see it for the whole team.
2. **Pickup time.** Target: back under 8h 57m with throughput holding near 838 PRs.
3. **dev-2 before and after agent mode**, 30 days each. It is the closest thing to an experiment we can run: same person, same work, one change.
4. **Devin merge time** on small, well-scoped tasks only. If it stays near 5 days, the author use case is closed.
5. **Seat utilization, monthly.** 19 seats paid and 9 used should never happen again.