Tools

Tools: How to Add Multi-Model AI Code Review to Your CI/CD Pipeline (2026)

2026-05-09 0 views admin
Tools: How to Add Multi-Model AI Code Review to Your CI/CD Pipeline (2026)

What "AI code review in CI" actually means

The minimum GitHub Actions config

Going from informational to blocking

GitLab CI

CircleCI

What to do when the models disagree

Cost and time, in case you're worried about either Running AI code review locally is fine for solo work. The moment you have a team, the question becomes: how do I make the AI an actual gate in the pipeline, not a thing one person remembers to run before they push? This is a walkthrough for wiring 2ndOpinion — the multi-model AI code review CLI — into a CI/CD pipeline. I'll show GitHub Actions in full, then sketch the same pattern for GitLab CI and CircleCI. The interesting decisions aren't where the YAML goes; they're around consensus thresholds, blocking vs informational mode, and what happens when Claude, Codex, and Gemini disagree on the same diff (which, from our review logs, is roughly 15% of the time). There are two shapes this takes, and the YAML is almost identical for either. The difference is the policy: I recommend starting in informational mode for the first week or two. AI reviewers — even three of them cross-examining each other — surface false positives. You want the team to learn the noise floor before the bot can block their merges, otherwise the first false-positive blocker generates a Slack thread that ends with "let's just turn this off." Here's the workflow file I use as a starting point. Drop it in .github/workflows/ai-review.yml: A few things worth calling out: fetch-depth: 0 is necessary because actions/checkout defaults to a shallow clone, and the CLI needs full history to compute the actual PR diff against the base branch. Skip this and your review runs against an empty diff, which produces a confidently empty review. Three API keys. Multi-model review means three providers. If you only set one, the CLI degrades to single-model mode and prints a warning. That's fine for a smoke test, but the whole reason you're doing this is the multi-model surface — the disagreement signal. --severity-threshold medium suppresses LOW findings in the PR comment. LOW is mostly nits and style preferences, and posting them on every PR trains your team to ignore the bot. Keep MEDIUM and HIGH visible; suppress LOW. To turn this into a merge gate, change one flag and one branch protection setting. The --fail-on high flag tells the CLI to exit with a non-zero status if any HIGH severity finding has consensus from at least 2 of 3 models. The 2-of-3 threshold matters — it's why you don't want to block on single-model verdicts. Any single model can confidently invent a critical bug. Two models independently flagging the same critical bug is meaningfully harder to fake. Then in Settings → Branches → Branch protection for your default branch, add the AI Code Review / review check to the required checks list. Now the merge button is gated. I'd hold this back for at least a week of informational-mode runs. Look at the false positive rate. If you're getting more than one false HIGH per ten PRs, tune the consensus threshold up to 3-of-3 instead of 2-of-3 before flipping the gate on: That's stricter — only blocks when all three models agree the finding is HIGH. False positive rate drops, false negative rate goes up. Tradeoff worth making early; you can loosen later once the team trusts the bot. Same pattern, different YAML. .gitlab-ci.yml: The CLI knows about GitLab's note format and uses CI_JOB_TOKEN automatically if it's available in the environment, so you don't need to set up a separate token unless you want bot-attributed comments. CircleCI's config doesn't have the same first-class PR concept, but the CLI handles it. .circleci/config.yml: CircleCI doesn't have a native PR-comment surface, so I store the review as a build artifact and add a separate small script to POST the JSON to the GitHub PR via a personal access token. Less elegant than the GitHub Actions path, but it works. The reason multi-model review is in CI in the first place is that disagreements are signal, not noise. The CLI's default behavior on a finding where models split: That last category is the most underrated output. About 8% of our diffs produce a 1-of-3 HIGH where exactly one model is convinced something is broken and the other two say it's fine. Most of those are false positives by the lone model. But about a quarter of them — by far the most interesting quarter — are real bugs that two models missed. You don't want those silently dropped, but you also don't want them blocking merges. NOTE-level surfacing is the right answer. Median review on a typical 200-line diff: about 40 seconds wall-clock and roughly $0.06 in combined API spend across the three providers. That's wall-clock time the developer doesn't spend; it runs in parallel with the rest of the CI matrix. The cost works out to less than a tenth of what most teams pay for any single human reviewer's hour, which is the right comparison — multi-model review doesn't replace human review, it replaces the human reviewer asking "did you check for race conditions" by hand. We've seen teams skip the AI review step for files larger than 1000 lines or generated files (lockfiles, schema dumps) — --exclude '**/*.lock' and --max-diff-lines 1000 handle both. If you want to try this on a real repo, the CLI is npm install -g 2ndopinion-cli and the docs for every flag mentioned above are at get2ndopinion.dev. The MCP server flavor (for plugging the same review engine into Claude Code or Cursor as an agent tool) is also there. We publish a weekly build-in-public update; this post is part of it. If you wire 2ndOpinion into your CI and one of your three models flags something the other two missed on a real diff, send the case over — those are the ones we use to tune the consensus thresholds. Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse

Command

Copy

$ name: AI Code Review on: pull_request: types: [opened, synchronize, reopened] jobs: review: runs-on: ubuntu-latest permissions: contents: read pull-requests: write steps: - uses: actions/checkout@v4 with: fetch-depth: 0 # need full history for diffs - name: Set up Node uses: actions/setup-node@v4 with: node-version: '20' - name: Install 2ndOpinion CLI run: -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - name: Run multi-model review env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | 2ndopinion review \ --base origin/${{ github.base_ref }} \ --head HEAD \ --format github-comment \ --severity-threshold medium \ --comment-pr ${{ github.event.pull_request.number }} name: AI Code Review on: pull_request: types: [opened, synchronize, reopened] jobs: review: runs-on: ubuntu-latest permissions: contents: read pull-requests: write steps: - uses: actions/checkout@v4 with: fetch-depth: 0 # need full history for diffs - name: Set up Node uses: actions/setup-node@v4 with: node-version: '20' - name: Install 2ndOpinion CLI run: -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - name: Run multi-model review env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | 2ndopinion review \ --base origin/${{ github.base_ref }} \ --head HEAD \ --format github-comment \ --severity-threshold medium \ --comment-pr ${{ github.event.pull_request.number }} name: AI Code Review on: pull_request: types: [opened, synchronize, reopened] jobs: review: runs-on: ubuntu-latest permissions: contents: read pull-requests: write steps: - uses: actions/checkout@v4 with: fetch-depth: 0 # need full history for diffs - name: Set up Node uses: actions/setup-node@v4 with: node-version: '20' - name: Install 2ndOpinion CLI run: -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - name: Run multi-model review env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | 2ndopinion review \ --base origin/${{ github.base_ref }} \ --head HEAD \ --format github-comment \ --severity-threshold medium \ --comment-pr ${{ github.event.pull_request.number }} 2ndopinion review \ --base origin/${{ github.base_ref }} \ --head HEAD \ --format github-comment \ --severity-threshold medium \ --fail-on high \ --comment-pr ${{ github.event.pull_request.number }} 2ndopinion review \ --base origin/${{ github.base_ref }} \ --head HEAD \ --format github-comment \ --severity-threshold medium \ --fail-on high \ --comment-pr ${{ github.event.pull_request.number }} 2ndopinion review \ --base origin/${{ github.base_ref }} \ --head HEAD \ --format github-comment \ --severity-threshold medium \ --fail-on high \ --comment-pr ${{ github.event.pull_request.number }} --fail-on high --consensus-required 3 --fail-on high --consensus-required 3 --fail-on high --consensus-required 3 ai-code-review: stage: test image: node:20 rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' variables: GIT_DEPTH: 0 script: - -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - 2ndopinion review --base origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME --head HEAD --format gitlab-note --severity-threshold medium --comment-mr $CI_MERGE_REQUEST_IID ai-code-review: stage: test image: node:20 rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' variables: GIT_DEPTH: 0 script: - -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - 2ndopinion review --base origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME --head HEAD --format gitlab-note --severity-threshold medium --comment-mr $CI_MERGE_REQUEST_IID ai-code-review: stage: test image: node:20 rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' variables: GIT_DEPTH: 0 script: - -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - 2ndopinion review --base origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME --head HEAD --format gitlab-note --severity-threshold medium --comment-mr $CI_MERGE_REQUEST_IID version: 2.1 jobs: ai-review: -weight: 500;">docker: - image: cimg/node:20.11 steps: - checkout - run: -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - run: name: Run review command: | 2ndopinion review \ --base origin/main \ --head HEAD \ --format json \ --output review.json - store_artifacts: path: review.json version: 2.1 jobs: ai-review: -weight: 500;">docker: - image: cimg/node:20.11 steps: - checkout - run: -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - run: name: Run review command: | 2ndopinion review \ --base origin/main \ --head HEAD \ --format json \ --output review.json - store_artifacts: path: review.json version: 2.1 jobs: ai-review: -weight: 500;">docker: - image: cimg/node:20.11 steps: - checkout - run: -weight: 500;">npm -weight: 500;">install -g 2ndopinion-cli - run: name: Run review command: | 2ndopinion review \ --base origin/main \ --head HEAD \ --format json \ --output review.json - store_artifacts: path: review.json - Informational mode. Every PR runs the review. Findings are posted as a comment or check annotation. Nothing blocks merge. Humans decide what to do. - Blocking mode. Review runs on every PR. If the consensus surface flags a HIGH severity finding, the check fails and merge is blocked until the author either fixes it or someone with override permission ships anyway. - 3-of-3 agree (HIGH): posted as a HIGH finding, blocks merge if --fail-on high is set. - 2-of-3 agree (HIGH): posted as a HIGH finding with the dissenting model's argument attached, blocks if --consensus-required 2. - 1-of-3 (HIGH): posted as a NOTE-level finding with the model's argument and the other two models' counter-arguments. Never blocks. Visible to humans.

🏷️ Tags

toolsutilitiessecurity toolsmultimodelreviewpipelineactually

More from Tools

Tools: Report: Custom Essay Writing Service Online

Tools: Report: Custom Essay Writing Service Online

2026-05-09 0
Tools: Fixing Database Import Failures After Moving Text Files from Windows to Linux (2026)

Tools: Fixing Database Import Failures After Moving Text Files from Windows to Linux (2026)

2026-05-09 0
Tools: How to Deploy Qwen2.5 72B with vLLM + FastAPI on a $20/Month DigitalOcean GPU Droplet: Production Inference at 1/90th Claude Cost (2026)

Tools: How to Deploy Qwen2.5 72B with vLLM + FastAPI on a $20/Month DigitalOcean GPU Droplet: Production Inference at 1/90th Claude Cost (2026)

2026-05-09 0
Tools: Create your first Docker Image (2026)

Tools: Create your first Docker Image (2026)

2026-05-09 0

Trending

1

CVE-2025-61481: Critical Remote Code Execution Vulnerability in MikroTik RouterOS & SwitchOS

2025-10-27 • 189 views
2

CVE-2025-43939: Dell Unity OS Command Injection (High)

2025-10-30 • 148 views
3

Google disputes false claims of massive Gmail data breach

2025-10-30 • 130 views
4

Microsoft: DNS outage impacts Azure and Microsoft 365 services

2025-10-30 • 88 views
5

3.5B Accounts, 1 Critical Flaw: Meta Closes WhatsApp Data-Harvesting

2025-11-25 • 81 views