Dry-Run (Shadow) Mode

Shadow mode lets operators roll out L402 enforcement safely. With l402_dry_run on; set on a location, the module evaluates the full pricing pipeline, synthesises a valid L402 challenge, and records structured logs and Prometheus metrics — but always passes the request through to the upstream. No client ever sees 401 or 402.

This is the recommended way to validate pricing, LN backend reachability, and traffic patterns with real production traffic before flipping a route to enforcement.


Enabling shadow mode

location /api/ {
    l402                        on;
    l402_amount_msat_default    10000;
    l402_dry_run                on;          # evaluate, log, never block
    proxy_pass                  http://upstream;
}

l402_dry_run accepts on or off (default). It can be combined with any other l402_* directive — dynamic pricing from Redis, multi-tenant LNURLs, macaroon timeouts, invoice rate limits — so the shadow-mode numbers you measure match the configuration you are about to enforce.

l402 must still be on for the module to enter the access handler. Turning l402 off disables the module entirely, including shadow mode.


What happens per request

For every request reaching a shadow-mode location, the module:

  1. Reads the static and dynamic (Redis) price for the route and picks the effective amount_msat.
  2. Looks up any per-tenant LNURL override.
  3. Verifies the Authorization header if one is present (L402 or Cashu).
  4. If no valid token is present, calls the configured LN backend and generates a real invoice + macaroon — exactly the challenge enforce mode would have returned.
  5. Emits a structured JSON log line and bumps the relevant Prometheus counters.
  6. Returns NGX_DECLINED, so Nginx continues to the content phase and serves the upstream response with its natural status code.

Cost note: generating a challenge contacts your LN backend on every unauthenticated request. If you have high traffic, start by enabling shadow mode on a sampled location (e.g. a canary route) before rolling it out everywhere.

Latency cap: the challenge-synthesis call is bounded by a 5-second timeout. If the LN backend does not respond within that window the request still passes through (with no X-L402-Dry-Run-Challenge header) and l402_dry_run_challenge_errors_total is incremented — shadow mode must never add latency to user-facing traffic.


Response headers

Shadow mode attaches debug headers to the upstream response so operators can inspect what would have happened without scraping logs:

Header Meaning
X-L402-Dry-Run: 1 Marks the response as produced by shadow mode. Always present.
X-L402-Dry-Run-Price-Msat: <n> Effective price for this route. Only emitted when the request would have been challenged (402) — not on paid-valid or rejected-invalid responses, to avoid leaking pricing against decided traffic.
X-L402-Dry-Run-Challenge: L402 macaroon="...", invoice="..." The exact WWW-Authenticate value enforce mode would have returned. Only present when the request would have been challenged (402) and the LN backend produced an invoice.
WWW-Authenticate: L402 macaroon="...", invoice="..." Also set alongside the challenge header, so real L402 clients can follow the payment flow in a staging environment.
X-L402-Dry-Run-Rate-Limited: 1 + X-L402-Dry-Run-Retry-After: <sec> Set when the request would have been challenged but hit l402_invoice_rate_limit. No invoice is generated and no challenge header is attached, mirroring what enforce mode would have done (429 + Retry-After).

Structured log events

Every shadow-mode request produces a single info-level JSON line via the Rust logger. A minimal example (formatted for readability):

{
  "event": "l402_dry_run",
  "route": "/api/resource",
  "price_msat": 10000,
  "price_source": "static",
  "backend": "LNURL",
  "client_ip": "203.0.113.42",
  "auth_state": "missing",
  "would_return": 402
}

Fields:

Field Values
route Normalised request path used for pricing lookups.
price_msat Effective price in millisatoshis.
price_source static (from nginx.conf) or dynamic (from Redis).
backend LN backend type snapshot: LND, LNURL, NWC, CLN, BOLT12, ECLAIR.
client_ip From X-Real-IPX-Forwarded-For → socket address.
auth_state missing, valid, or invalid.
would_return HTTP status enforce mode would have used (200, 401, 402).
rate_limited true when l402_invoice_rate_limit would have produced a 429 — challenge synthesis was skipped to protect the LN backend.

Pipe into jq to see a live firehose:

sudo tail -f /var/log/nginx/error.log \
  | grep '"event":"l402_dry_run"' \
  | jq -c 'select(.would_return != 200) | {route, price_msat, auth_state}'

Prometheus metrics

The l402_metrics directive turns a location into a Prometheus scrape endpoint. It serves counters in text exposition format v0.0.4.

location = /metrics {
    l402_metrics;

    # Production: restrict to your scrape network.
    allow 10.0.0.0/8;
    deny  all;
}

Scrape it with a standard Prometheus config:

scrape_configs:
  - job_name: ngx_l402
    metrics_path: /metrics
    static_configs:
      - targets: ['nginx:8000']

Counters are kept in an nginx shared-memory zone, so they are aggregated across all worker processes: a scrape served by any worker returns the true total regardless of worker_processes. (If the shared zone cannot be allocated at startup the module logs a warning and falls back to per-worker counters, which under-report by roughly a factor of worker_processes.)

Exported counters

Metric Meaning
l402_requests_total Every request that entered the access handler with l402 on;. Incremented for both enforce and shadow traffic.
l402_challenges_issued_total Requests that received a 402 response (enforce mode), counted after the rate-limit gate.
l402_rate_limited_total Requests rejected with 429 by l402_invoice_rate_limit (enforce mode).
l402_payments_valid_total Authorization headers that verified successfully — Lightning + Cashu (enforce mode only — dry-run traffic goes to l402_dry_run_*).
l402_payments_lightning_total Successful payments settled via a Lightning macaroon (classic preimage or auto-detect).
l402_payments_cashu_total Successful payments settled via a Cashu token redemption.
l402_payments_invalid_total Authorization headers that failed verification (enforce mode only).
l402_payments_missing_total Requests without an Authorization header (enforce mode only).
l402_invoices_generated_total Lightning invoices successfully generated for L402 challenges (enforce mode only).
l402_invoices_generation_errors_total Failures generating a Lightning invoice during challenge synthesis (returns 500).
l402_dry_run_requests_total Requests handled in shadow mode.
l402_dry_run_would_block_total Shadow-mode requests that would have been blocked (401 or 402).
l402_dry_run_would_allow_total Shadow-mode requests that would have been allowed (200).
l402_dry_run_rate_limited_total Shadow-mode requests that would have hit l402_invoice_rate_limit — challenge synthesis was skipped.
l402_dry_run_challenge_errors_total Shadow-mode requests where challenge synthesis failed (e.g. LN backend unreachable).
l402_dry_run_price_msat_sum Sum of msat prices evaluated in shadow mode. Pair with _requests_total to derive an average price.

Useful PromQL

# Fraction of traffic that would be blocked if you flipped enforcement on:
rate(l402_dry_run_would_block_total[5m])
/
rate(l402_dry_run_requests_total[5m])

# Average price served by shadow mode (msat):
rate(l402_dry_run_price_msat_sum[5m])
/
rate(l402_dry_run_requests_total[5m])

# Challenge-synthesis error rate — a signal that your LN backend is flaky:
rate(l402_dry_run_challenge_errors_total[5m])

The endpoint has no built-in authentication. Restrict it at the Nginx level with allow/deny, an auth subrequest, or a firewall rule — exposing it publicly leaks traffic volume and pricing details.


Suggested rollout recipe

  1. Deploy with l402 on; and l402_dry_run on; on the target location. Leave existing routes untouched.
  2. Scrape /metrics for 24–48 hours. Confirm:
    • l402_dry_run_challenge_errors_total stays flat (LN backend healthy).
    • l402_dry_run_would_allow_total / l402_dry_run_requests_total matches the fraction of paying clients you expect.
    • l402_dry_run_price_msat_sum divided by request count matches your posted price.
  3. Sample the JSON log for a few high-volume paths and confirm price_source is what you configured (static vs dynamic).
  4. Remove l402_dry_run on; (or set it to off). Reload Nginx. The location now enforces.

If you ever need to revert, setting l402_dry_run on; again immediately disables enforcement without touching upstream code paths.