BenchTruth

What we measure

We run identical workflows on Zapier, Make, n8n and Pipedream around the clock, with both endpoints under our own control (webhook in → HTTP out → back to our own receiver), so third-party app outages can't be mistaken for platform failures. Two numbers come out:

• Silent Failure Rate (SFR) — the share of runs a platform accepted but never completed, with no error surfaced. Computed as (missed + partial executions) ÷ all runs expected to produce output, reported with a Wilson 95% confidence interval. Rare failures need large samples, so this meter never stops running.
• True cost vs usage — what the same workload actually consumes (tasks / operations / credits) on each platform, published as a cost-vs-volume curve, not a single cherry-picked number.

How you can trust it

• Test harness and raw data are published — don't believe us, reproduce us.
• Rankings follow measurements only. No vendor pays to change a number.
• Flagship data pages carry no affiliate links.
• We test platforms that pay us nothing (Zapier has no affiliate program for publishers; self-hosted n8n earns us $0) and publish their results identically.
• Confidence intervals and limitations are stated, and every page shows its last-updated date.

Status

The measurement harness has been running continuously since July 2026. First public datasets — the reliability scoreboard and true-cost curves — are being accumulated now and will ship on this site. Larger samples first: reliability conclusions are only honest with tight confidence intervals.