The AI industry is spending hundreds of billions of dollars on models, chips, and data centers. Behind the money and positioning is a simpler question for engineering teams: can open-weight models do serious production work, or do they only look competitive on benchmarks?
We tested that question in a real workflow: reviewing pull requests.
Kilo Code Reviewer is an AI code review product that analyzes changes for bugs, security issues, reliability problems, and maintainability risks. Kilo users can select from 500+ models across 60+ providers, including frontier closed models and open-weight alternatives. That model choice also makes a useful experiment possible: use one model to write code, then a different model to review it.
TL;DR
The 30-second read
What the data actually says
Open weights reached the top tier
Two of the top three models for critical findings were open weight. All three leaders were within 5%.
One model created most of the security gap
Remove one closed-model outlier and the gap falls to 0.008 security findings per review.
Comparable review behavior at about 26x lower aggregate cost per token
Outside the Sol security outlier, open-weight reviewers were in the same behavioral range. Across broader traffic, non-closed models used 20% of cost for 86% of tokens; closed models used 80% for 13%. This is aggregate traffic, not same-task pricing.
The best reviewer may not be the best author
32.3% of attributed June reviews already used a different model to review the code.
Decision rule: choose a reviewer by measured behavior, not by license. When the stakes justify it, use a different model to review the one that wrote the code.
Open-weight models can surface critical issues
From June 22 to July 23, 2026, Kilo classified 10,643 completed Code Reviewer runs and 7,083 findings. Each finding received a severity and a category. We normalized results per review so that high-volume models did not win simply because they ran more often.
Who surfaced the most critical findings?
Critical findings per review, all 13 reviewer routes
Why do some model families appear twice? Versioned routes stay on a fixed snapshot; rolling aliases can move to a newer snapshot.
Takeaway
Open-weight models matched the strongest closed model on critical findings per review.
Open-weight models ranked first and third, and all three leaders were within 5%.
Kimi K2.7 Code led at 0.179 critical findings per review. Grok 4.5 followed at 0.176. Laguna M.1 reached 0.171. Kimi and Laguna are open-weight models; Grok is closed. The difference between first and third was under 5%.
That is the most reusable result in the dataset: open-weight models occupied two of the top three positions for surfacing critical findings. They can walk the walk in a production code review workflow.
It does not mean every open-weight model behaved the same way. GLM 5.2, for example, marked a much larger share of its output as suggestions. The spread inside the open group was wider than the average difference between open and closed groups.
Models disagree on what deserves escalation
The number of findings is only part of reviewer behavior. Models also differ in whether they frame feedback as a critical issue, a warning, or a suggestion.
What kind of feedback does each model give?
Share of each reviewer route's findings by severity
Laguna M.1 behaved like an escalation-focused reviewer: 28% of its findings were critical. GLM 5.2's rolling alias behaved more like a style and maintainability reviewer: 58% of its findings were suggestions. Both are open-weight models, reinforcing the central result that license category does not explain reviewer behavior.
The security gap mostly disappears without one outlier
At first glance, closed models found roughly twice as many security issues per review: 0.067 versus 0.034. That result looks decisive until the model-level data is visible.
The closed-model security lead is one outlier wide
Security findings per review, weighted by review volume
GPT 5.6 Sol reported security findings at 0.285 per review, far above every other model. Once it is removed, the closed-model average falls to 0.042. The open-weight average remains 0.034.
The takeaway is not “ignore security.” It is the opposite: if security coverage matters, choose a model with demonstrated security behavior. License category is a weak proxy for that behavior.
Use one model to code and another to review
A common misconception is that selecting an AI coding model means selecting one model for the entire workflow. It does not have to.
Author model
Write the change
Optimize for implementation speed, context, or cost.
Reviewer model
Challenge the change
Optimize for critical findings, security, or breadth.
The author model and reviewer model have different jobs. An author should produce a correct implementation efficiently. A reviewer should challenge assumptions, search for failure modes, and notice what the author missed. Using the same model for both can reproduce the same blind spots.
Kilo's model freedom lets teams match models to each stage. A lower-cost open-weight model can implement routine work, while a model with stronger security behavior reviews it. Or a frontier model can author a difficult change while an open-weight model with high critical intensity provides an independent pass.
In the analyzed June traffic, 32.3% of attributed reviews were already cross-model. The largest pairing used Step 3.7 Flash for authoring and Laguna M.1 for review.
The state of open-weight code review
The open-weight debate is often framed around ideology, sovereignty, or benchmark scores. Code review turns it into an operational question: can these models catch consequential issues in code produced by humans and other models?
In this sample, yes. Open-weight reviewers appeared among the strongest critical finders, handled most token volume at a minority of cost, and were already being used as independent reviewers in cross-model workflows. In the broader traffic mix, non-closed traffic had about one twenty-sixth the aggregate cost per token of closed traffic: (80% cost / 13% tokens) / (20% cost / 86% tokens) = 26.5. This is an aggregate traffic comparison, not a controlled same-task price comparison.
Three practical rules follow:
- Do not buy a license category. Buy a behavior. Compare critical intensity, security emphasis, false positives, latency, and cost for your repositories.
- Separate authoring from review. Give the reviewer a different objective and, when practical, a different model.
- Use open weights where they earn the work. The data supports treating them as production options, not fallback models.
What this data cannot prove
This dataset measures what models flagged, not whether every finding was correct. More findings can mean broader coverage, more false positives, or both. We do not yet have a consistent acceptance or dismissal signal for every finding, so this analysis should not be read as an accuracy leaderboard.
Repository mix can also affect results. Models were not randomly assigned to identical pull requests. Versioned routes appear separately from rolling aliases because a versioned route stays on a fixed model snapshot, while a rolling alias can move to a newer snapshot behind the same public model name.
The next useful measure is acceptance rate: which findings developers fix, dismiss, or dispute. That will turn a behavioral comparison into a stronger quality comparison.
Full model data
Click any column heading to sort the table. “Find/rev,” “Crit/rev,” and “Sec/rev” mean findings, critical findings, and security findings per completed review.
| Kimi K2.7 Code | open | 307 | 1.23 | 0.179 | 0.059 |
| Grok 4.5 | closed | 233 | 0.86 | 0.176 | 0.034 |
| Laguna M.1 | open | 346 | 0.60 | 0.171 | 0.014 |
| Claude Sonnet 5 - rolling alias | closed | 1,391 | 0.68 | 0.079 | 0.047 |
| Claude Sonnet 5 - versioned route | closed | 651 | 0.72 | 0.078 | 0.045 |
| Tencent Hy3 - July version | open | 607 | 1.18 | 0.051 | 0.058 |
| Step 3.7 Flash - rolling alias | open | 1,265 | 0.38 | 0.047 | 0.009 |
| GLM 5.2 - zai route | open | 739 | 0.60 | 0.041 | 0.047 |
| Step 3.7 Flash - versioned route | open | 884 | 0.50 | 0.040 | 0.011 |
| Tencent Hy3 - rolling alias | open | 703 | 0.81 | 0.038 | 0.041 |
| GPT 5.6 Sol | closed | 274 | 1.61 | 0.026 | 0.285 |
| GLM 5.2 - rolling alias | open | 2,186 | 0.56 | 0.020 | 0.043 |
| Claude Opus 4.8 | closed | 157 | 0.26 | 0.019 | 0.000 |
Methodology: 10,643 Code Reviewer runs were classified from June 22 through July 23, 2026, producing 7,083 findings. Per-model rates cover the 13 reviewer models with enough volume to normalize, accounting for 9,743 reviews. “Closed” covers Anthropic, OpenAI, Google, and xAI; all other models are grouped as open or non-closed for this analysis.
Where individual models stand out
The data reveals distinct reviewer profiles across critical intensity, security emphasis, and suggestion share. These highlights describe observed behavior, not an accuracy ranking.
Critical finding leader
Kimi K2.7 Code
0.179 critical findings / review
“Kimi K2.7 Code ranked first among 13 reviewer routes for critical findings per review in Kilo's analysis of 10,643 AI code reviews.”
Critical intensity leader
Laguna M.1
28% of findings classified critical
“Laguna M.1 had the highest critical share in Kilo's analysis: 28% of its code review findings were classified as critical.”
Security specialist
GPT 5.6 Sol
0.285 security findings / review
“GPT 5.6 Sol showed the strongest security emphasis in Kilo's analysis, reporting 0.285 security findings per code review.”
Suggestion-oriented reviewer
GLM 5.2 - rolling alias
58% of findings were suggestions
“GLM 5.2's rolling alias behaved like a suggestion-oriented reviewer in Kilo's analysis: 58% of its findings were suggestions.”