English
Review feedback
The last stage turns findings and verification results into comments. No agent is involved: workflow merges the findings that are one problem, labels and ranks them by rule, and workflow/report renders them as Markdown and SARIF.
Comment format
Every comment follows Conventional Comments:
text
<label> [decorations]: <subject>
[discussion]A label follows from three facts about a finding, each on a published scale:
| Fact | Scale | In eyeful |
|---|---|---|
| What it is about | Functional or evolvability, Mäntylä and Lassenius | A note is evolvability; the rest is functional |
| How it can be checked | Verification methods, ISO/IEC/IEEE 29148 | Evidence |
| How settled it is | kind, SARIF 2.1.0 | Kind |
Go takes the first row that fits. The names and their meanings are those of Conventional Comments; which facts give which label is eyeful's own rule (References).
| Label | Given to | Conventional Comments |
|---|---|---|
nitpick | Every finding behind the comment is a note | "trivial preference-based requests" |
issue | A finding whose kind is fail | "highlight specific problems" |
question | A finding that names a failing scenario, not yet shown: test or analysis | "a potential concern but … not quite sure if it's relevant" |
suggestion | A finding with no scenario, read against a standard: inspection | "propose improvements" |
Decorations say whether the comment blocks and which area it concerns: security, correctness, design, tests, ux, readability. Only an issue is blocking (A review). The subject and discussion are the expert's own words, taken from the most severe finding behind the comment.
text
issue (blocking, correctness): A session is still valid at exactly expires_at.
is_expired compares with >, so a request at the expiry second is accepted. The reproduction
test below fails on this change and passes with the suggested fix.Line comments
On a pull request, eyeful posts one review with the event COMMENT, never REQUEST_CHANGES or APPROVE. An issue that names a file and a line, and that the judge does not dispute, is posted on that line. So is a question whose reproduction passed the first four gates: a question to the author with the failing test attached. Other questions, suggestions and nitpicks are not posted, and each is folded on its own at the end of the summary under its whole subject. A summary longer than GitHub allows in one comment continues in the next. The summary counts the findings reproduced by a test and, of those, the ones that can block. Its heading counts the posted issues; with none, the reproduced questions, then the warnings and errors left to review, then says whether CI failed, nothing was reviewed or the review is incomplete. It reads "No issues found" only for a complete review with nothing left to review. A reproduced finding whose fix was not verified says why. GitHub takes a line comment only on a line its diff of the pull request shows, so an issue on any other line goes, in full, into the summary instead. When a posted comment has one fix for that same file, the fix is included as a suggested change. The eyeful commit status says whether a review was posted, not whether the change is right (Review lifecycle).
The ranked report
Everything else goes into one report. Findings that are one problem are merged, and the comment records how many experts reported it. Comments are sorted by:
- standing: not disputed by the judge, then disputed, then nitpicks;
- kind:
fail,review,open(Kind and severity); - the number of experts that reported it;
- label:
issue,question,suggestion,nitpick; - the plan's groups, then file and line.
Disputed comments and nitpicks are folded into collapsed sections at the end. Nothing is left out. The report closes with:
- the scopes of a split change, with their files and findings;
- the files triage set aside;
- the uncertainty checklist: absent experts, findings not reproduced, commands that did not run, and whether the budget ran out.
Outputs
| Output | For |
|---|---|
| Markdown | The terminal |
| Findings JSON | The desktop app, beside the diff |
| SARIF | Another agent, a script, GitHub code scanning, the evaluation |
| GitHub review | Line comments on the pull request |
| GitHub comment | A short summary: counts, the posted findings, the rest folded |
eyeful status | Whether a review was posted, linking to the run |
The GitHub posts happen in the review's job, and each is posted once (Architecture).
In the terminal
eyeful review prints, in order:
- the counts of findings by kind;
- what the change does: the planner's summary and its groups, core groups first;
- the tests the change breaks (Regressions);
- "No blocking issues", or a table of findings in report order:
text
| Severity | File:Line | Issue |
| ------------ | ------------------------- | ----------------------------------------- |
| 🔴 error | `app/session.py:11` | Session still valid at exactly expires_at |
| 🟠 warning | `web/src/Login.svelte:40` | The expiry is shown in UTC |- each finding: its severity, area and kind, a one-sentence summary, the failure scenario, the reasoning, how it was verified, the reproduction test and the fix as a diff;
- the disputed comments, the nitpicks, the files set aside and the uncertainty checklist.
Each run saves review.md, findings.json, results.sarif and result.json (Running locally).
Rounds
A re-review is a new review of the same subject: locally, the latest run in .eyeful/runs with the same base commit whose head is the current head or an ancestor of it. With execution, before any expert starts, the verifier runs every reproduction the last review saw fail, five times each, with no model:
| The reproduction now | The finding |
|---|---|
| passes every time | Resolved, and listed under "Resolved since the last review" |
fails with its fails text, not on the base commit | Still open; carried over with its test, and the experts are told |
| fails the same way on the base commit | Pre-existing; reported, never blocking |
| is flaky, fails another way or cannot run | Shown as unverified and listed in the uncertainty checklist |
Findings the last review did not reproduce are not carried: the experts read the change afresh. With read_only or none nothing is carried.
