Skip to main content

Cost Ingestion: Connector Setup

Cost ingestion learns each connector’s actual per-transaction fee (percentage + flat component) from real settlement reports and invoices, at a fine-grained cluster level (connector × card network × card variant × funding × issuer country × currency × interchange category). That fitted cost model is what multi-objective routing ranks gateways on.

Why it’s learned, not configured

Processor pricing is rarely a single number. The effective cost of a transaction depends on the card network, whether the card is credit or debit, where it was issued, the currency, and the interchange category — and published rate cards drift from what actually lands on the settlement statement once assessments, downgrades, and blended tiers are applied. Rather than ask a merchant to enter fee schedules by hand (which go stale the moment a processor re-prices), Decision Engine reads the money that actually moved. For each cluster it fits a simple linear model against real settlements — fee ≈ pct_bps × amount + fixed — using ordinary least squares. The slope becomes the percentage component (in basis points) and the intercept the flat per-transaction fee. That fitted (pct_bps, fixed) pair is the cost estimate every expected-value calculation uses. Because the fit is per-cluster, the model captures that (say) Visa credit issued in the US costs a different amount than a domestic debit card — precisely the differences that make one gateway cheaper for one slice of traffic and pricier for another. How much of a merchant’s volume has a trustworthy fit (versus too little data, or a fee structure the linear model can’t capture) is what cost coverage reports. Before any report or invoice can be ingested, register the connector account’s settlement credentials.

Data Sources

Settlement data reaches Decision Engine through three paths, all feeding the same cost_ingestion pipeline:

Register Connector Credentials

  • account — the connector-side account (e.g. Adyen’s merchantAccountCode).
  • webhook_secret — used to verify inbound webhook signatures for this account.
  • download_auth — credential used to authenticate report downloads for the scheduled poller. Either "user:password" for Basic auth, or a bare API key (sent as X-API-Key).
Response:
Secrets are encrypted at rest and never returned by the API — subsequent reads only return masked hints.

List Configured Connectors

Remove Credentials

Returns 204 No Content. Deleting a source that isn’t configured also returns success (idempotent).

Settlement Webhook (Connector → Decision Engine)

Once credentials are registered, point the connector’s settlement-report notification at:
This route is public (no AUTH_HEADER) — the connector authenticates itself via its own signature, which Decision Engine verifies against the webhook_secret stored for that (connector, account) pair. Adyen is the first supported connector (:connector = adyen).
The handler ACKs immediately and enqueues the report for background processing — download, parse, and re-fit all happen asynchronously in the ingest worker, so the connector always gets a fast response. A bad signature returns 401; an unrecognized connector or malformed payload returns 400.

Notes

  • Manual and webhook/poll-based ingestions share the same pipeline and history — see Uploads for the manual path and ingestion history.
  • Once enough settlement data has been fitted, check Cost Coverage to see what share of volume has a trustworthy cost model.