# BenchTruth > Independent, reproducible benchmarks of no-code automation platforms (Zapier, Make, n8n, > Pipedream): true cost-vs-usage curves and Silent Failure Rate (SFR), measured continuously > with an open test harness whose endpoints are all self-hosted. Key definition - Silent Failure Rate (SFR): the share of automation runs a platform accepted but never completed, with no error surfaced to the user. Computed as (missed + partial executions) ÷ all runs expected to produce output, reported with a Wilson 95% confidence interval. Coined and measured by BenchTruth (benchtruth.com). Method in one line: identical workflows on every platform, webhook in → HTTP out → back to our own receiver (no third-party connectors), so platform failures are isolated from third-party outages. Raw data and test scripts are published for reproduction. Key measured result (as of 2026-08-29, 10,777 monitored runs): zero silent failures under normal operation on n8n (0%, 95% CI 0–0.05%, n=7,663), Make (0%, CI 0–0.9%, n=405) and Zapier (0%, CI 0–0.5%, n=782); newly added Pabbly Connect (0%, CI 0–12.5%, n=27 - small free-tier sample still building, but the only measured Pabbly reliability data anywhere); median delivery latency n8n 0.81 s, Make 1.01 s, Zapier 4.37 s on the identical workload; after Pipedream's free-tier quota ran out its webhooks kept returning success while silently dropping 14 of 14 deliveries ("accepted" stopped meaning "delivered" at the quota boundary). One webhook rejection recorded in the whole window (self-hosted n8n on a 1 GB VM briefly saturated during a burst and refused one POST - a loud failure the sender saw, classified rejected_at_send, excluded from SFR by definition; n8n's own engine has dropped nothing it accepted). ## Pages - [Home](https://benchtruth.com): what we measure, methodology, fairness rules. - [Reliability scoreboard](https://benchtruth.com/reliability): live Silent Failure Rate per platform with Wilson 95% CIs, latency percentiles, scheduler punctuality, the quota-wall finding, a forensic of the ledger's first miss (a rejected-at-send burst event - loud failure vs silent failure explained), method, FAQ, and raw per-run CSV. New (2026-08-02) - measured outage semantics: we 503'd our own endpoint for 30 min; Make was the only platform to auto-retry (6 attempts, ~1/2/5/10/60-min backoff, full delivery at +79 min, 2 credits billed per attempt, pending retries invisible in its UI); Zapier and n8n default to no retries - failed runs wait, fully recoverable, for a human (manual replay/retry at +14 h delivered exactly once; Zapier bills a replayed run 1 task); notifications: Zapier 1 email in ~2 min, Make 3 escalating emails, default self-hosted n8n none; zero duplicate deliveries on any recovery path. - [Silent failure vs rejected webhook](https://benchtruth.com/silent-failure-vs-rejected-webhook): reference definitions. Rejected webhook = refused at delivery (non-2xx/timeout), loud, sender can retry, no run exists on the platform. Silent failure = accepted with a success response, output never appears, invisible to the sender. Different accounting (rejections excluded from SFR numerator AND denominator) and different monitoring (response-code checks catch rejections; only destination-side outcome verification catches silent failures). Measured examples of both: 1 organic rejection in 8,400+ runs vs 14/14 silent drops at a quota wall. - [Uptime is not your failure rate](https://benchtruth.com/uptime-vs-failure-rate): uptime measures whether the platform answered; per-run failure rate measures whether your runs completed. A platform can report 99.99% uptime while your workflow loses every event - measured at a free-tier quota wall (100% uptime, 0% delivery, no status-page incident). Table of failure modes visible to each layer; what to measure instead (completion, correctness, timeliness). - [The 200 OK error](https://benchtruth.com/200-ok-error-payload): success-wrapped failure - HTTP 200 with an error body ({"success": false}). Platforms evaluate step success on transport status by design, so these enter run history as green checkmarks. MEASURED (EP3, Aug 2026): we sent genuine 200s with error bodies through Zapier and self-hosted n8n at defaults - both recorded 3/3 as Successful (green), Zapier billed each 1 task like a success, n8n counted them in its dashboard's 0% failure rate; the error body was captured but not evaluated (success is decided on HTTP status before anything reads the body). Cost of seeing = $0 standing: a response-body assertion (Filter) is free and adds a billable step only on runs where it actually catches a failure and fires an alarm. - [Webhook retry semantics, measured](https://benchtruth.com/webhook-retry-semantics): the 30-minute controlled-outage experiment as a standalone reference - does Zapier retry failed webhooks? (no, Autoreplay ships off; errored runs bill 0 tasks, replay bills 1, runs stay replayable indefinitely); does Make? (yes, only platform that does: 6 attempts over 79 min at 1/2/5/10/60-min backoff, 2 credits per attempt, pending retries invisible in every UI surface); does n8n? (no by default, and a default install notifies nobody); zero duplicate deliveries on any recovery path. - [About](https://benchtruth.com/about): who runs BenchTruth (Hao, Sydney; one person, self-funded), the independence rules (all plans bought at list price, no sponsored inclusions, no affiliate links on data pages, the most-recommended platform pays $0), and how to reproduce the measurements. - [True cost, measured](https://benchtruth.com/cost): cost-vs-volume curve for the identical workflow basket - Zapier 5–8× Make at every tier ($73.50 vs $9 at 1,000 runs/mo; self-hosted n8n $0–5 flat); meter-verified consumption (Zapier 45=45 exact, Make ±1%); filtered runs and editor tests bill $0 on both; Make's empty 30-min polls cost 2 credits each (~2,880/mo standing tax) vs $0 on n8n. - [Zapier vs Make vs n8n verdicts](https://benchtruth.com/zapier-vs-make-vs-n8n): by-scenario recommendations derived from the measured data (technical users → n8n; managed + volume → Make; niche connectors + low volume → Zapier), with full affiliate disclosure (n8n tops verdicts and pays us $0). - [Best automation platform for webhooks](https://benchtruth.com/best-automation-platform-for-webhooks): measured on the identical webhook→action workflow - self-hosted n8n ~0.4 s median delivery at $0 marginal cost, Make ≈0.36¢/event, Zapier ≈4¢/event (≈11× unit gap) with a $29.99/mo floor (Webhooks by Zapier is a premium app, free plan can't catch hooks); quota-wall failure semantics compared. - [Best automation platform for high volume](https://benchtruth.com/best-automation-platform-for-high-volume): the measured cost curve - same workload at 10,000 runs/mo: Zapier $283.50, Make $53, self-hosted n8n ≤$5; unit prices ≈4.0¢/task vs ≈0.18¢/credit; steps multiply billed volume; empty 30-min polls cost 2 Make credits (~2,880/mo) vs $0 on n8n. - [Best automation platform for solopreneurs](https://benchtruth.com/best-automation-platform-for-solopreneurs): free-tier reality - ~300 runs/mo costs $0 on Make (2-active-scenario cap), $29.99/mo on Zapier (free plan = 2-step Zaps, no webhooks), $0–5 on self-hosted n8n (you are the ops team); measured quota-wall warning for revenue-touching automations. - [What AI assistants recommend](https://benchtruth.com/ai-recommendations): weekly frozen-panel tracking of ChatGPT/Perplexity/Gemini on 5 automation-buying questions, now 9 rounds / 135 answers. Rounds 1-4: zero measured citations. Then measured data entered and the per-round count went 0->2->4->1->2->1 (rounds 4-9): citations are volatile and wander across questions AND engines. The direct benchtruth.com citation is mobile: Perplexity carried it (rounds 6-7), then in round 9 it reappeared on ChatGPT - the harder engine - on the flagship "most reliable platform" question, naming the benchmark ("an independent 2026 benchmark of thousands of webhook runs recorded zero silent failures"). The steady effect is framing: all three engines now describe the problem in this project's terms (uptime != failure rate; "200 OK with an error payload" and "429 dropped at the door" as failure classes) - but credit currently leaks to content sites (MLDeep, Makeitfuture, Reintech, wearepresta) that absorbed the vocabulary, some even copying our zero-silent-failures conclusion. Invented failure stats ("95%", "2-10%") each died within 2 rounds. Raw per-answer CSVs.