Tools: How I Added AI Code Quality Checks to My CI Pipeline in 5 Minutes (And Found 47 Bugs on Day One)

Tools: How I Added AI Code Quality Checks to My CI Pipeline in 5 Minutes (And Found 47 Bugs on Day One)

How I Added AI Code Quality Checks to My CI Pipeline in 5 Minutes (And Found 47 Bugs on Day One)

The Problem

The Solution: Open Code Review

5-Minute Setup

Step 1: Install (30 seconds)

Step 2: Scan Locally (1 minute)

Step 3: Add to GitHub Actions (2 minutes)

Step 4: Generate SARIF Report (1 minute)

Step 5: Set Your Quality Threshold (30 seconds)

What I Found on Day One

L2 Mode: AI-Powered Deep Analysis

How It Compares

GitLab CI Integration

Tips from Production Use

Wrapping Up TL;DR: Set up Open Code Review — a free, open-source tool that catches AI-generated code defects — in your CI/CD pipeline. Here's the step-by-step guide. My team started using AI coding assistants (Copilot, Cursor, Claude Code) about 6 months ago. Velocity went up. But so did a new category of bugs: These bugs slip past ESLint, Prettier, and even TypeScript. They compile fine. They just break at runtime. I needed a quality gate that understood AI-generated code defects specifically. Open Code Review (OCR) is a free, open-source CLI that detects AI-specific code defects: Best part: L1 mode runs in under 10 seconds with no AI needed. You'll get a detailed report: Create .github/workflows/ocr.yml: For GitHub Code Scanning integration: The threshold parameter sets the minimum quality score (0-100). If the scan score falls below it, the CI check fails. That's it. Your CI pipeline now catches AI-generated code defects before they merge. Running OCR on our main codebase revealed: 47 issues that our existing toolchain (ESLint + TypeScript + SonarQube) completely missed. If you want deeper analysis, OCR has an L2 mode that uses local AI (Ollama) for: L2 takes a bit longer (~30 seconds for medium projects) but catches subtle issues that pattern matching alone can't. OCR complements your existing tools — it doesn't replace them. Keep ESLint for style. Add OCR for AI-specific defects. Not on GitHub? No problem: AI coding assistants are incredible productivity boosters. But they introduce a new category of bugs that traditional tools weren't designed to catch. Open Code Review fills that gap. It's free, open-source, runs locally, and takes 5 minutes to set up. GitHub · Portal · NPM What AI-generated code bugs have you encountered? I'd love to hear about your experience in the comments. Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse

Code Block

Copy

// Bug 1: Hallucinated import — package doesn't exist import { validateEmail } from 'email-validator-pro'; // ❌ Doesn't exist on npm // Bug 2: Deprecated API — worked fine in 2020 const parsed = url.parse(request.url); // ⚠️ Deprecated since Node 15 // Bug 3: Security anti-pattern const query = `SELECT * FROM users WHERE id = ${userId}`; // 🔴 SQL injection // Bug 1: Hallucinated import — package doesn't exist import { validateEmail } from 'email-validator-pro'; // ❌ Doesn't exist on npm // Bug 2: Deprecated API — worked fine in 2020 const parsed = url.parse(request.url); // ⚠️ Deprecated since Node 15 // Bug 3: Security anti-pattern const query = `SELECT * FROM users WHERE id = ${userId}`; // 🔴 SQL injection // Bug 1: Hallucinated import — package doesn't exist import { validateEmail } from 'email-validator-pro'; // ❌ Doesn't exist on npm // Bug 2: Deprecated API — worked fine in 2020 const parsed = url.parse(request.url); // ⚠️ Deprecated since Node 15 // Bug 3: Security anti-pattern const query = `SELECT * FROM users WHERE id = ${userId}`; // 🔴 SQL injection npm install -g @opencodereview/cli npm install -g @opencodereview/cli npm install -g @opencodereview/cli ocr scan src/ --sla L1 ocr scan src/ --sla L1 ocr scan src/ --sla L1 ╔══════════════════════════════════════════════════════════════╗ ║ Open Code Review V4 — Quality Report ║ ╚══════════════════════════════════════════════════════════════╝ Overall Score: 72/100 🟠 C Files Scanned: 48 | Duration: 6.3s 🔴 [error] api/handler.ts:45 — Possible hardcoded API key 🟡 [warn] utils/request.ts:12 — url.parse() deprecated → WHATWG URL API 🟡 [warn] services/auth.ts:67 — Cyclomatic complexity 22 (max: 15) ⚪ [info] types/index.ts:8 — Unused interface (context window artifact) ╔══════════════════════════════════════════════════════════════╗ ║ Open Code Review V4 — Quality Report ║ ╚══════════════════════════════════════════════════════════════╝ Overall Score: 72/100 🟠 C Files Scanned: 48 | Duration: 6.3s 🔴 [error] api/handler.ts:45 — Possible hardcoded API key 🟡 [warn] utils/request.ts:12 — url.parse() deprecated → WHATWG URL API 🟡 [warn] services/auth.ts:67 — Cyclomatic complexity 22 (max: 15) ⚪ [info] types/index.ts:8 — Unused interface (context window artifact) ╔══════════════════════════════════════════════════════════════╗ ║ Open Code Review V4 — Quality Report ║ ╚══════════════════════════════════════════════════════════════╝ Overall Score: 72/100 🟠 C Files Scanned: 48 | Duration: 6.3s 🔴 [error] api/handler.ts:45 — Possible hardcoded API key 🟡 [warn] utils/request.ts:12 — url.parse() deprecated → WHATWG URL API 🟡 [warn] services/auth.ts:67 — Cyclomatic complexity 22 (max: 15) ⚪ [info] types/index.ts:8 — Unused interface (context window artifact) name: AI Code Quality Check on: [pull_request] jobs: code-review: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install OCR run: npm install -g @opencodereview/cli - name: Scan changed files uses: raye-deng/open-code-review@v1 with: sla: L1 threshold: 60 scan-mode: diff github-token: ${{ secrets.GITHUB_TOKEN }} name: AI Code Quality Check on: [pull_request] jobs: code-review: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install OCR run: npm install -g @opencodereview/cli - name: Scan changed files uses: raye-deng/open-code-review@v1 with: sla: L1 threshold: 60 scan-mode: diff github-token: ${{ secrets.GITHUB_TOKEN }} name: AI Code Quality Check on: [pull_request] jobs: code-review: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install OCR run: npm install -g @opencodereview/cli - name: Scan changed files uses: raye-deng/open-code-review@v1 with: sla: L1 threshold: 60 scan-mode: diff github-token: ${{ secrets.GITHUB_TOKEN }} - name: Generate SARIF run: ocr scan src/ --sla L1 --format sarif -o ocr-results.sarif - name: Upload SARIF uses: github/codeql-action/upload-sarif@v3 with: sarif_file: ocr-results.sarif - name: Generate SARIF run: ocr scan src/ --sla L1 --format sarif -o ocr-results.sarif - name: Upload SARIF uses: github/codeql-action/upload-sarif@v3 with: sarif_file: ocr-results.sarif - name: Generate SARIF run: ocr scan src/ --sla L1 --format sarif -o ocr-results.sarif - name: Upload SARIF uses: github/codeql-action/upload-sarif@v3 with: sarif_file: ocr-results.sarif threshold: 70 # Adjust based on your team's tolerance threshold: 70 # Adjust based on your team's tolerance threshold: 70 # Adjust based on your team's tolerance # .ocrrc.yml sla: L2 ai: embedding: provider: ollama model: nomic-embed-text baseUrl: http://localhost:11434 llm: provider: ollama model: qwen3-coder endpoint: http://localhost:11434 # .ocrrc.yml sla: L2 ai: embedding: provider: ollama model: nomic-embed-text baseUrl: http://localhost:11434 llm: provider: ollama model: qwen3-coder endpoint: http://localhost:11434 # .ocrrc.yml sla: L2 ai: embedding: provider: ollama model: nomic-embed-text baseUrl: http://localhost:11434 llm: provider: ollama model: qwen3-coder endpoint: http://localhost:11434 code-review: script: - npx @opencodereview/cli scan src/ --sla L1 --threshold 60 --format json --output ocr-report.json artifacts: reports: codequality: ocr-report.json code-review: script: - npx @opencodereview/cli scan src/ --sla L1 --threshold 60 --format json --output ocr-report.json artifacts: reports: codequality: ocr-report.json code-review: script: - npx @opencodereview/cli scan src/ --sla L1 --threshold 60 --format json --output ocr-report.json artifacts: reports: codequality: ocr-report.json npm install -g @opencodereview/cli ocr scan src/ --sla L1 npm install -g @opencodereview/cli ocr scan src/ --sla L1 npm install -g @opencodereview/cli ocr scan src/ --sla L1 - Hallucinated imports — Verifies every import against npm/PyPI registries - Stale APIs — AST-based deprecated API detection - Security anti-patterns — Hardcoded secrets, eval(), SQL injection - Over-engineering — Cyclomatic complexity, nesting depth - Context artifacts — Unused interfaces, dead code from truncated generation - Cross-file coherence checking - Semantic duplication detection - AI confidence scoring - Start with L1. It's fast enough for every PR. Add L2 later for critical paths. - Use scan-mode: diff in CI to only scan changed files — keeps PR checks fast. - Don't set threshold too high initially. Start at 50-60 and raise it as you fix existing issues. - Use --format html for human-readable reports in pull request comments. - Run a full scan weekly (not just diffs) to catch gradual drift.