Tools

Tools: Easily benchmark all your app's endpoints at once - Guide

2026-04-30 0 views admin
Tools: Easily benchmark all your app's endpoints at once - Guide

The config

Running one endpoint

Running all of them

Round-tripping with the cloud dashboard

A note on the cloud dashboard

Why this is worth doing Most "load tests" in real codebases are a curl pasted into a Slack thread. Someone runs it before a release, eyeballs the latency, and we ship. There's nothing version-controlled, nothing repeatable, and the next person to touch the service has no idea which endpoints are actually fast paths. benchmarkr is a powerful and easy-to-use CLI and MCP tool that fixes that part of the workflow specifically. The thing I want to talk about in this post is the piece that makes it click: a YAML config that lives in your repo and describes every endpoint you care about, the same way a package.json describes your dependencies. First, install benchmarkr cli if you haven't: for Homebrew, or for Debian (More installation guides available here) Next, run benchmarkr endpoints init in your project root and you get a benchmarkr.yaml you can commit: A few things to notice: That's it. Saved defaults apply. Any flag you pass on the command line wins; headers and params are merged. So when you're poking at production specifically, you can do: …without editing the committed file. This is where the YAML pays for itself. Because every endpoint is named and self-describing, you can hand the entire file to the CLI in one shot: That walks every endpoint in benchmarkr.yaml in succession, applying each endpoint's saved defaults (concurrency, duration, headers, body — the whole config). Between runs you get a [i/N] <name> header so it's obvious where you are; live p50/p95/p99 streams in for the active endpoint and a final summary prints when it finishes. --all is mutually exclusive with --url and --endpoint, and any flags you do pass (e.g. --store, --json, --rate-limit) apply to every run in the sweep. For CI, this collapses the workflow step to one line: --json with --all emits an array — one entry per endpoint, with the same result shape as a single run — so you can pipe it straight into a regression check or upload it as a CI artifact: You're not maintaining a separate list of "endpoints to benchmark" in your CI workflow and a list in your config. There's one list. Add a new endpoint to benchmarkr.yaml in the same PR that adds the route, and the next CI run picks it up automatically — no workflow edits, no shell loop to babysit. The CLI gives you fast feedback. The dashboard gives you the long view — historical p95 charts, regression detection across versions, the kind of thing that's painful to wire up yourself. The newest piece is import/export, so the YAML in your repo and the endpoints in the dashboard stay in sync without anyone having to maintain both: Export from the dashboard. Open any endpoint and click Export for YAML or JSON. Or click Export all in the endpoints nav to dump every endpoint to one file you can drop into a fresh repo. Import to the dashboard. Click Import, pick a benchmarkr.yaml, and endpoints upsert by (user, name). If the config changed, a new version is recorded — so you get a history of how each endpoint's load shape evolved. A workflow I've been using: The cloud platform is currently in closed beta. We're planning to open it up to the public on a per-token basis in spring 2026 — if you'd like access at launch, you can join the waitlist. The CLI itself is open source and works without the cloud — benchmarkr run, the YAML config, and even local result persistence don't require an account or a token. The dashboard, history charts, version pinning, and import/export are the parts gated behind beta access for now. The shift that matters isn't "run benchmarks in CI" — plenty of tools do that. It's having a single, reviewable file that says here are this service's endpoints and how we expect them to behave under load, sitting next to the code in the same PR. Once that file exists: The repo already describes your API. This is just letting it benchmark itself. benchmarkr is open source — brew install mack-overflow/tap/benchmarkr or grab it from benchmarkr. Cloud dashboard beta access opens publicly per-token in spring 2026. Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse

Command

Copy

$ -weight: 500;">brew tap mack-overflow/tap -weight: 500;">brew -weight: 500;">install benchmarkr -weight: 500;">brew tap mack-overflow/tap -weight: 500;">brew -weight: 500;">install benchmarkr -weight: 500;">brew tap mack-overflow/tap -weight: 500;">brew -weight: 500;">install benchmarkr echo "deb [trusted=yes] https://-weight: 500;">apt.fury.io/mack-overflow/ /" \ | -weight: 600;">sudo tee /etc/-weight: 500;">apt/sources.list.d/benchmarkr.list -weight: 600;">sudo -weight: 500;">apt -weight: 500;">update -weight: 600;">sudo -weight: 500;">apt -weight: 500;">install benchmarkr echo "deb [trusted=yes] https://-weight: 500;">apt.fury.io/mack-overflow/ /" \ | -weight: 600;">sudo tee /etc/-weight: 500;">apt/sources.list.d/benchmarkr.list -weight: 600;">sudo -weight: 500;">apt -weight: 500;">update -weight: 600;">sudo -weight: 500;">apt -weight: 500;">install benchmarkr echo "deb [trusted=yes] https://-weight: 500;">apt.fury.io/mack-overflow/ /" \ | -weight: 600;">sudo tee /etc/-weight: 500;">apt/sources.list.d/benchmarkr.list -weight: 600;">sudo -weight: 500;">apt -weight: 500;">update -weight: 600;">sudo -weight: 500;">apt -weight: 500;">install benchmarkr version: 1 endpoints: - name: list-users method: GET url: ${API_BASE:-http://localhost:8080}/users headers: Authorization: Bearer ${API_TOKEN} defaults: concurrency: 10 duration_seconds: 30 - name: search-users method: GET url: ${API_BASE}/users/search params: q: "test" limit: "50" defaults: concurrency: 5 duration_seconds: 15 - name: create-order method: POST url: ${API_BASE}/orders headers: Authorization: Bearer ${API_TOKEN} Content-Type: application/json body: sku: "ABC-123" quantity: 1 defaults: concurrency: 2 duration_seconds: 10 version: 1 endpoints: - name: list-users method: GET url: ${API_BASE:-http://localhost:8080}/users headers: Authorization: Bearer ${API_TOKEN} defaults: concurrency: 10 duration_seconds: 30 - name: search-users method: GET url: ${API_BASE}/users/search params: q: "test" limit: "50" defaults: concurrency: 5 duration_seconds: 15 - name: create-order method: POST url: ${API_BASE}/orders headers: Authorization: Bearer ${API_TOKEN} Content-Type: application/json body: sku: "ABC-123" quantity: 1 defaults: concurrency: 2 duration_seconds: 10 version: 1 endpoints: - name: list-users method: GET url: ${API_BASE:-http://localhost:8080}/users headers: Authorization: Bearer ${API_TOKEN} defaults: concurrency: 10 duration_seconds: 30 - name: search-users method: GET url: ${API_BASE}/users/search params: q: "test" limit: "50" defaults: concurrency: 5 duration_seconds: 15 - name: create-order method: POST url: ${API_BASE}/orders headers: Authorization: Bearer ${API_TOKEN} Content-Type: application/json body: sku: "ABC-123" quantity: 1 defaults: concurrency: 2 duration_seconds: 10 benchmarkr run -e list-users benchmarkr run -e list-users benchmarkr run -e list-users benchmarkr run -e list-users \ --header "X-Trace: debug-2026-04-28" \ --concurrency 50 benchmarkr run -e list-users \ --header "X-Trace: debug-2026-04-28" \ --concurrency 50 benchmarkr run -e list-users \ --header "X-Trace: debug-2026-04-28" \ --concurrency 50 benchmarkr run --all benchmarkr run --all benchmarkr run --all # .github/workflows/perf.yml - name: Benchmark every endpoint env: API_BASE: https://api.staging.example.com API_TOKEN: ${{ secrets.STAGING_API_TOKEN }} BENCH_CLOUD_TOKEN: ${{ secrets.BENCHMARKR_TOKEN }} run: benchmarkr run --all --store --json > perf-results.json # .github/workflows/perf.yml - name: Benchmark every endpoint env: API_BASE: https://api.staging.example.com API_TOKEN: ${{ secrets.STAGING_API_TOKEN }} BENCH_CLOUD_TOKEN: ${{ secrets.BENCHMARKR_TOKEN }} run: benchmarkr run --all --store --json > perf-results.json # .github/workflows/perf.yml - name: Benchmark every endpoint env: API_BASE: https://api.staging.example.com API_TOKEN: ${{ secrets.STAGING_API_TOKEN }} BENCH_CLOUD_TOKEN: ${{ secrets.BENCHMARKR_TOKEN }} run: benchmarkr run --all --store --json > perf-results.json [ { "name": "list-users", "stop_reason": "completed", "duration": "30.001s", "stored": true, "result": { "requests": 12483, "p50_ms": 4, "p95_ms": 12, "p99_ms": 23, "errors_total": 0 } }, { "name": "search-users", "stop_reason": "completed", "duration": "15.002s", "stored": true, "result": { "requests": 4127, "p50_ms": 18, "p95_ms": 47, "p99_ms": 92, "errors_total": 0 } }, { "name": "create-order", "stop_reason": "completed", "duration": "10.001s", "stored": true, "result": { "requests": 312, "p50_ms": 41, "p95_ms": 88, "p99_ms": 121, "errors_total": 0 } } ] [ { "name": "list-users", "stop_reason": "completed", "duration": "30.001s", "stored": true, "result": { "requests": 12483, "p50_ms": 4, "p95_ms": 12, "p99_ms": 23, "errors_total": 0 } }, { "name": "search-users", "stop_reason": "completed", "duration": "15.002s", "stored": true, "result": { "requests": 4127, "p50_ms": 18, "p95_ms": 47, "p99_ms": 92, "errors_total": 0 } }, { "name": "create-order", "stop_reason": "completed", "duration": "10.001s", "stored": true, "result": { "requests": 312, "p50_ms": 41, "p95_ms": 88, "p99_ms": 121, "errors_total": 0 } } ] [ { "name": "list-users", "stop_reason": "completed", "duration": "30.001s", "stored": true, "result": { "requests": 12483, "p50_ms": 4, "p95_ms": 12, "p99_ms": 23, "errors_total": 0 } }, { "name": "search-users", "stop_reason": "completed", "duration": "15.002s", "stored": true, "result": { "requests": 4127, "p50_ms": 18, "p95_ms": 47, "p99_ms": 92, "errors_total": 0 } }, { "name": "create-order", "stop_reason": "completed", "duration": "10.001s", "stored": true, "result": { "requests": 312, "p50_ms": 41, "p95_ms": 88, "p99_ms": 121, "errors_total": 0 } } ] - Env var substitution. ${API_BASE} and ${API_BASE:-default} work the way they do in shell. A sibling .env file is auto-loaded but never overrides what's already in the environment, so the same file works on a laptop, in CI, and in staging. - Defaults travel with the endpoint. create-order runs at concurrency 2 for 10 seconds because that's what makes sense for a write path. list-users runs at concurrency 10. You set this once in the file you already review. - Discovery walks up from CWD. Run the CLI from any subdirectory and it finds the file, like -weight: 500;">git does. - Export from the dashboard. Open any endpoint and click Export for YAML or JSON. Or click Export all in the endpoints nav to dump every endpoint to one file you can drop into a fresh repo. - Import to the dashboard. Click Import, pick a benchmarkr.yaml, and endpoints upsert by (user, name). If the config changed, a new version is recorded — so you get a history of how each endpoint's load shape evolved. - Define endpoints in benchmarkr.yaml, commit them. - CI runs the loop above on every PR with --store and the cloud token, persisting results to the dashboard. - Open the endpoint in the dashboard to see the trend line for that endpoint across the last N PRs. - If somebody adds an endpoint via the dashboard UI for ad-hoc poking, Export → drop the file into the repo → it's now part of the CI matrix. - New endpoints get a perf budget at the same moment they get a route handler. - Reviewers can see in the diff that a new write path is being benchmarked at concurrency 2, not 100, and push back if that's wrong. - CI gets a free regression signal across every endpoint, not just the one someone remembered to add to a script. - The dashboard gives you the historical view without anyone manually re-entering endpoints.

🏷️ Tags

toolsutilitiessecurity toolseasilybenchmarkendpointsguideconfig

More from Tools

Tools: Thank you Linux for the 1y and thank you all for contributing to Linux and using it. (2026)

Tools: Thank you Linux for the 1y and thank you all for contributing to Linux and using it. (2026)

2026-04-30 0
Tools: Update: How To "Gaslight" A Binary

Tools: Update: How To "Gaslight" A Binary

2026-04-30 0
Tools: Ultimate Guide: Unix commands that feel like cheating — and why most devs have never heard of them

Tools: Ultimate Guide: Unix commands that feel like cheating — and why most devs have never heard of them

2026-04-30 0
Tools: Local AI vs. Cloud AI: When to Use Which (A Developer's Guide) - Guide

Tools: Local AI vs. Cloud AI: When to Use Which (A Developer's Guide) - Guide

2026-04-30 0

Trending

1

CVE-2025-61481: Critical Remote Code Execution Vulnerability in MikroTik RouterOS & SwitchOS

2025-10-27 • 189 views
2

CVE-2025-43939: Dell Unity OS Command Injection (High)

2025-10-30 • 148 views
3

Google disputes false claims of massive Gmail data breach

2025-10-30 • 130 views
4

Microsoft: DNS outage impacts Azure and Microsoft 365 services

2025-10-30 • 88 views
5

3.5B Accounts, 1 Critical Flaw: Meta Closes WhatsApp Data-Harvesting

2025-11-25 • 81 views