Every launch checklist starts true and decays from the moment it is written. A redirect moves during a frontend cleanup. A signing secret gets rotated in one environment but not the other. A preview domain gets copied into a provider console and quietly becomes "production." The checklist still says "verify webhooks," but the thing it was describing no longer exists.
PreFlight is built on a simple distinction: a checklist records intent, while an evidence trail records what actually happened. Instead of asking a founder to trust five green dashboards, PreFlight probes the customer path the way a customer would — browser, provider, and database — and keeps a dated, comparable record of every handoff. This article explains what it checks, why the evidence model matters more than the score, and how to run the twenty-minute drill before real traffic arrives.
The Shift: From Checklist to Evidence Trail
What changes when launch readiness is recorded, not remembered
The Old Way: Dashboard Hopping
Founders who verify by hand open five provider dashboards, trust five green states, and discover the gap only after a customer reports it.
// the manual path...
Stripe dashboard shows green
Webhook logs say delivered
Homepage returns 200
The New Way: PreFlight
One probe follows the customer path across every provider and records what actually happened — including the side effect in your own database.
// you get a direct answer...
“A green dashboard is a claim. A recorded trace is evidence.”
A checklist is a snapshot. A launch is a movie.
A checklist answers "did someone think about this?" An evidence trail answers "what did the system do, when, and can we prove it again?" The difference shows up the first time something breaks after launch. With a checklist, the argument is about memory: someone checked webhooks, surely. With an evidence trail, the argument is over in ten seconds, because the last three runs are sitting there with timestamps, statuses, and the exact observed behavior.
The trail matters because modern SaaS failures are rarely single-system failures. They are handoff failures: the payment succeeded but the entitlement write failed; the webhook was delivered but the handler responded before verifying the signature; the deploy succeeded but the canonical domain still points at the old project. No single provider dashboard can see a handoff, because a handoff lives between providers. Only a probe that walks the whole path can record it.
So PreFlight stores five things for every check it runs: the status, a safe error code that is shareable in team chat without leaking provider internals, the observed behavior, the expected behavior, and the timestamp. Reruns append to the same history instead of replacing it. That is the entire product philosophy in one paragraph: keep the failed result next to the passing rerun, forever.
What PreFlight actually probes
The public surface comes first, because it is the only part of your product a crawler and a customer see identically: redirect chains and canonical host behavior, TLS and security headers, exposed assets and source maps, indexability signals like robots directives and canonical tags. A surprising number of launch disasters are visible from outside before anyone logs in — a source map shipping a secret is the canonical example, and it is detectable with nothing but a careful fetch of the same assets a browser receives.
Then the provider handoffs, for whatever you connect. For a Stripe-backed SaaS that means the checkout path in test mode, the webhook route's reachability and signature behavior, and — the part most teams skip — the success and cancel redirects on the canonical host, because those URLs rot silently every time a staging domain becomes production. The Stripe webhook guide is explicit that signature verification must happen against the raw payload (see Stripe's webhook documentation), and PreFlight's checks are shaped around exactly that contract.
For Supabase-backed products, the probes are behavioral rather than textual: anonymous reads, owner reads, cross-tenant reads, and the write paths your client actually performs. Row Level Security policy text is an input; the behavior under multiple roles is the evidence. The Supabase RLS documentation describes the policy model, but only a probe can tell you whether your policies, as deployed, hold the boundary you intended. Authorization failures of this kind are exactly what the OWASP Top Ten puts at the top of its list — broken access control that a green UI never reveals.
Auth callbacks, password reset flows, transactional email delivery, and — after the deploy, not before it — domain and environment parity on the canonical production host. A preview deployment can pass every check while the production host serves an old build with old environment variables. The post-deploy run against the canonical surface is not a nicety; it is the check that catches the release that succeeded at the platform but failed at the customer.
How Launch Evidence Reaches Your Decision
Provider state on one side, your database on the other
Provider Dashboards
Each integration reports its own health in its own vocabulary — and none of them can see your fulfillment write.
PARTIAL VIEW
PreFlight
One Evidence Trail
Status, safe error, observed behavior, expected behavior, and timestamp — one comparable record per handoff, kept across reruns.
RECORDED & COMPARABLE
Key Insight: The launch decision gets easier when every handoff leaves a receipt.
The five handoffs that fail quietly
After watching enough launches, the same five handoffs produce most of the "customer paid but cannot access" support queues. Each one has a specific fix, and each fix is cheap before traffic and expensive after.
Mistake #1: Checkout succeeds, entitlement write fails
The browser sees a success page, Stripe sees a completed session, and your database never gains the row that grants access. The handoff that failed is inside your own system, between the webhook handler and the fulfillment write.
Solution: assert the side effect, not the status code. A launch check should follow the session identifier into the database and confirm the entitlement, subscription, or order row exists with the right customer identity.
Mistake #2: The handler returns 200 before verification
A webhook route that parses JSON and responds before verifying the signature will happily accept forged events in production. The provider log says delivered; the security boundary was never exercised.
Solution: verify the signature against the raw body first, return bounded errors, and let the provider retry the failures it should retry. Then confirm the verified path writes exactly once per event identifier.
Mistake #3: Test-mode secrets validating live expectations
A test signing secret validating a test webhook proves almost nothing about the live endpoint. Environment swaps are among the most common webhook failures in the wild, and they surface only when the first real payment arrives.
Solution: verify test and live endpoints separately, with the secret that belongs to each, and label the evidence accordingly so nobody mistakes a test-mode pass for a live-mode pass.
Mistake #4: Redirects that point at a preview host
Success and cancel URLs copied from a preview deployment send paying customers to a domain that no longer exists after launch. The checkout works; the landing after payment does not.
Solution: exercise the exact success and cancel URLs on the canonical host as part of the drill, and treat a redirect chain that leaves your canonical domain as a failed check, not a warning.
Mistake #5: Source maps shipping with secrets
A bundle or source map that contains a private key or service-role credential is a public disclosure the moment it is fetched. The UI keeps working, which is why teams discover it last.
Solution: scan the same assets a browser receives — chunks and maps, not just HTML — and rotate anything that ever appeared in a public asset before running another check. Removal without rotation is not a fix.
Reading a failed check like an operator
A failed check is only useful if it tells you what to do next, so PreFlight results are structured as a path: status, then safe error code, then observed versus expected behavior, then the suggested fix, then the rerun. Success means the expected behavior was observed. Warning means the result needs review, a retry, or a manual confirmation. Failed means a hard blocker: rejected credentials, a missing side effect, invalid transport security, or an exposed secret.
Internet noise is real, and a provider endpoint can time out once and recover seconds later. The operator's discipline is to retry transient warnings before rotating keys or rewriting configuration — and to treat a warning that repeats as a reliability issue with an owner, not as noise. The companion discipline is to never delete the failed result after the rerun passes. The pair of results, failed then verified, is the artifact that proves the fix worked and the one you will want in a review, a diligence conversation, or a postmortem.
The twenty-minute pre-launch drill
The drill is short on purpose, because a ritual that takes an afternoon will not be repeated. Run it before launch, after every meaningful deploy, and after any provider configuration change.
- Confirm the canonical production host and the deploy commit you expect to be live.
- Run the public surface pass: redirects, headers, TLS, indexability, exposed assets and source maps.
- Run the payment rehearsal in test mode and assert the entitlement row, not just the session.
- Verify the webhook route with the correct signing secret for the environment under test.
- Probe data access as anonymous, owner, and a second tenant — reads and writes separately.
- Exercise success and cancel redirects on the canonical host.
- Exercise one browser journey end to end: signup, activation, and the first value moment.
- Keep the run beside the previous one and note what changed between them.
| Handoff | What "good" looks like | Evidence to keep |
|---|---|---|
| Checkout | Session created in test mode with the expected price and customer | Session identifier, timestamp, environment label |
| Webhook | Signature verified against raw body; bounded response; single write | Event identifier, verification result, response status |
| Fulfillment | Entitlement row exists for the right customer identity | Observed row, expected contract, delta if any |
| Data access | Anonymous denied, owner allowed, cross-tenant denied | Role matrix with observed outcomes per operation |
| Public surface | One canonical host, no secret-bearing assets, sane headers | Asset inventory, redirect map, header set, rerun diff |
Where PreFlight fits — and where it doesn't
Honesty about boundaries is part of the product. PreFlight is not a penetration test: it will not adversarially probe your business logic, and questions about injection depth, threat modeling, or tenant isolation at scale still deserve human review. It is not an accounting system: reconciliation of money against entitlements is an operational control it can surface, not a ledger it replaces. And a passing run is a dated statement about the system at that moment, not a certificate that the product is universally safe.
What it is, is the repeatable middle layer between "we checked it once in May" and a full security review: fast enough to run before every release, specific enough that a failed row names the provider, the observed behavior, and the next action, and durable enough that the history becomes your launch record. The intended first run is deliberately small — create a project, add one public URL, connect the provider your revenue path depends on, run the check, and fix the first failed row. Momentum, then coverage. See the feature overview for the full check surface, or the guide to reading a failure when a row comes back red.
Turn the Checklist Into Repeatable Proof
Create a project, connect the providers your customer path depends on, run the first check, and fix the first failed row. The trail does the rest.
Frequently asked questions
Is PreFlight a replacement for a penetration test?
No. It is the repeatable baseline that runs before every release: public surface, provider handoffs, data-access behavior, and post-deploy parity. Adversarial testing and threat-model work still belong to a pentest or a focused security review; PreFlight tells you which questions those reviews should prioritize.
Does it only work with Stripe and Supabase?
The public-surface checks are stack-agnostic — any HTTPS origin can be probed for redirects, headers, exposed assets, and indexability. Provider-specific probes exist for the integrations you connect, and the evidence model is identical for each: status, safe error, observed behavior, timestamp.
Will it page someone for every transient wobble?
No. Transient failures are retried, downtime alerts require repeated failure, and recovery messages go out when a service returns. The goal is to reduce alert fatigue without hiding a real customer-impacting failure.
How long does a real pre-launch run take?
The drill above is designed to fit in roughly twenty minutes once the project exists, because a ritual that takes an afternoon will not be repeated. The first setup takes longer; every run after that is a rerun of a known scope.
Can we keep the evidence for reviews or diligence?
Yes — that is the point of the trail. Each run keeps its status, safe error, observed behavior, and timestamp, and reruns append instead of replacing. A history of failed-then-verified results is a stronger artifact in a review than any single green screenshot.
Expanded field note
Launch readiness as recorded evidence: the practical answer
checklists decay while providers, redirects, and secrets keep changing between the checklist and the launch. This guide is for founders and small teams deciding whether to ship. Start by naming the failure you want to prevent and the customer or operator who would notice it first.
The useful scope is the customer path across public surface, payment, webhook, data access, auth, and post-deploy parity. That keeps the work concrete: you can choose a URL, provider, route, release, or data operation, observe it, and decide what to do when the observation does not match the expected contract.
What good looks like
status, safe error, observed behavior, expected behavior, side effect, and timestamp per handoff, kept across reruns. A result is stronger when it preserves the input, environment, timestamp, expected behavior, observed behavior, and the next action. This lets a different person reproduce the finding without asking the original operator to reconstruct the entire context from memory.
a green provider dashboard is a claim about one system; a launch decision needs the handoffs between systems. Keep the scope visible when sharing the result. A passing outside-in check can prove a reachable behavior at a point in time; it cannot silently become a guarantee about private code, every authenticated role, or every provider failure mode.
Frame the decision before you change anything
founders and small teams deciding whether to ship usually do not need another dashboard full of disconnected warnings. They need a defensible answer to a narrower question: is the behavior that matters to the customer working in the environment that is about to change? Start there. If the answer is unclear, make the ambiguity part of the work instead of translating it into a green score.
The first useful boundary is the customer path across public surface, payment, webhook, data access, auth, and post-deploy parity. Write it down in the same language the team will use during the fix. Name the route, provider, release, role, data object, or browser action involved. Then write the expected behavior as a sentence that could be checked by another person. This turns a broad topic into a small contract and makes it easier to tell whether a failure is reproducible, transient, out of scope, or genuinely fixed.
A good scope also includes what the check does not attempt. Public observation is different from authenticated authorization testing. Provider reachability is different from a complete fulfillment path. A page that renders in a browser is not necessarily a page a crawler can index. Stating the limit early protects the reader from overconfidence and tells the operator when to add a deeper review.
Questions the result should answer
- What was tested? Identify the canonical URL, route, provider, account role, release, or customer action rather than describing the scope as “the site.”
- What should have happened? State the expected response, permission, side effect, delivery event, page directive, or recovery signal in plain language.
- What actually happened? Keep the observed status, safe error, response detail, timing context, or missing side effect without pasting credentials or customer data into the record.
- Why does it matter? Connect the observation to a customer, crawler, revenue, security, availability, or release decision so severity is not just a color.
- What happens next? Name the smallest reversible fix, the owner, the rerun, and the condition that will close the issue.
These questions are deliberately boring. Boring evidence is easier to compare, easier to hand off, and easier to defend later. It also gives an answer engine or a future teammate enough context to summarize the result without inventing a claim that the original check never made.
A sequence that holds up under pressure
- Confirm the canonical host and expected deploy commit.
- Run the public surface pass before connecting private providers.
- Assert the entitlement row, not the checkout status.
- Verify webhooks with the secret that belongs to the environment.
- Probe data access as anonymous, owner, and a second tenant.
- Exercise success and cancel redirects on the canonical host.
- Keep the failed run beside the passing rerun.
The important nuance is this: a green provider dashboard is a claim about one system; a launch decision needs the handoffs between systems. That distinction matters because fast remediation can create a second problem: a broad header change can break a payment script, a credential rotation can break a cron worker, and a restrictive policy can make a valid customer path look like an outage.
Evidence to keep with the fix
| Record | Why it matters |
|---|---|
| Scope | The URL, role, provider, release, and limitation prevent a result from being reused outside the question it actually answered. |
| Before | The original failing observation, environment, and customer impact make the fix auditable. |
| Change | The code, configuration, provider setting, migration, or credential action that should alter the result. |
| After | A rerun against the same scope proves whether the intended behavior recovered. |
| Owner | A named person or team, an expected next action, and a review date keep the result from becoming an orphaned warning. |
| Follow-up | An owner, cadence, or release rule keeps the same class of failure from returning silently. |
Failure modes worth checking twice
- Treating five green dashboards as one green product.
- Returning 200 from a webhook before signature verification.
- Using test-mode evidence to validate live expectations.
- Deleting the failed result after the rerun passes.
When one of these appears, avoid making several unrelated changes at once. Preserve the failing evidence, isolate the smallest boundary that can explain it, make one reversible correction, and rerun. That rhythm is slower than guessing for the first five minutes and faster than untangling a release that changed three providers at once.
Know when the first layer is not enough
Automation is valuable because it is repeatable, but repeatability is not the same as depth. If the question involves complex authorization, tenant isolation, injection, business logic, a high-value asset, or an adversarial threat model, use the automated result as a map for a deeper review. Give the reviewer the scope, failed observation, relevant release context, and the boundary you want tested. Do not present a public scan as a certification or a substitute for professional security work.
The same rule applies to operations. A successful provider probe may prove that a credential can reach an API, but it may not prove that a webhook creates the correct entitlement. A healthy uptime response may prove reachability, but it may not prove that a signed-in customer can complete the task. Add the assertion or browser journey that matches the real risk, and keep the cheap signal for early warning.
Questions people ask after reading this
What is the fastest useful first step?
Choose one representative scope and write the expected result before running the tool. For this topic, that means the customer path across public surface, payment, webhook, data access, auth, and post-deploy parity. A small, explicit baseline is more useful than a large scan whose findings have no owner or decision attached.
What should I do when the result is green?
Keep the scope, timestamp, and limitation, then decide whether the result belongs in a release gate, monitor, report, or follow-up review. Green means the observed contract passed. It does not turn untested behavior into evidence.
What should I do when the result is red?
Read the evidence before changing configuration. Confirm the environment, reproduce the smallest failing behavior, assign the next action, and rerun after the fix. If the issue requires credentials, source access, or adversarial judgment, escalate it rather than hiding the gap behind a retry.
After the fix ships
Run the same check on the canonical production surface, not only on a local or preview environment. If the issue involved a provider, wait for the real callback or scheduled sample. If it involved search, confirm the HTML, canonical, robots, sitemap, and internal links agree. If it involved payments or access, verify the side effect a customer receives rather than stopping at a browser redirect.
PreFlight is designed for this last step: keep the original observation, connect the relevant provider or journey, attach the release context, and let the next run show whether the system stayed healthy. The goal is not a bigger report. It is a shorter path from signal to a verified decision.