[eliteprox]

Summary

This PR lets a gateway find out what each call_runner call cost. call_runner takes a new, optional on_payment_session callback. Each time the remote signer signs a payment session, the SDK calls it with that session. The session's new manifest_id property names the orchestrator session the payment funds, so the gateway can match it to Clearinghouse usage records and charge the right job.

Callers that don't pass the callback see no change in results, errors or streams.

Why

The signer charges when it signs a payment, not when the job succeeds. So a call can be charged and still fail: the runner errors, the payment turns out to be unusable, or the caller cancels. A gateway that bills its own users has to know which charges belong to which call, including the failed ones. Today call_runner doesn't tell it.

We need this for the Network Engineering SPE build track (project board). Its second milestone defines how the builder engine reports network cost per job (ISS-04). The engine records the manifest of every payment the signer signs, then matches each one to a Clearinghouse usage record, so every job shows what it cost, whether it succeeded or not. livepeer-builder-core pins this branch until it merges.

An earlier version read AuthID from the signer's state. On #74, Josh pointed out that the state is opaque and shouldn't become a public contract, and suggested ManifestID instead. This version uses ManifestID and doesn't decode the signer's state.

What callers see

manifest_ids: list[str] = []
result = await call_runner(
    ...,
    on_payment_session=lambda s: manifest_ids.append(s.manifest_id),
)

• When it runs. Once for each payment session, when the signer first signs for it and before the paid request goes to the runner. The callback can be sync or async.
• What it reports. Every session the signer signs: one whose payment proves unusable, one followed by a runner failure, and each new session after the runner asks for payment again (a repeated 402). Fixed-price calls report too.
• What it doesn't report. Sessions the signer refuses, including refresh requests.
• If the callback raises, the paid request isn't sent, and the caller gets the callback's error unchanged.
• If the call is cancelled while a payment is being signed, the SDK lets the signing finish and reports the session before it passes the cancellation on. Because the caller owns the list, a call cancelled under asyncio.wait_for, for example, still accounts for every session it signed.

Known limits

• RunnerSelectionCursor still fails over after a signed payment, so one job can pay twice. Runner selection: pool size, failure classification and payment_sent on rejections #70 tracks that.
• It's still open whether every orchestrator issues a fresh ManifestID for each 402 challenge, or whether that's only how livepeer/node behaves today (call_runner doesn't report the signer sessions it opens #74). Builder core treats a manifest shared by more than one payment session as uncertain, so if the assumption fails, costs show as uncertain rather than wrong.

This replaces #71, which was closed so the change could target a fork branch.

Test plan

• pytest -W error::ResourceWarning at e6d243b: 227 passed, 2 failed. Both failures are in tests/test_stats_pull.py and fail the same way on main. The new tests cover sync and async callbacks, repeated 402s, a runner failure after payment, a refused session, an unusable payment, a raising callback, a cancelled call, and manifest_id staying the same across a payment refresh.
• Confirm that a fresh ManifestID per challenge is behavior callers can rely on (call_runner doesn't report the signer sessions it opens #74).

[Copilot]

🟡 Changes recommended

Cancellation and signer-response validation failures can leave charged sessions unreported.

2 open findings

• Report signed payments before remaining response validation · New
• Preserve charged-session reporting when cancellation interrupts payment response · New

What changed in this PR

Adds payment-session reporting to call_runner so callers can associate signer charges with calls, including failed calls.

Changes:

• Adds an optional sync or async callback before paid requests.
• Exposes the challenge’s manifest ID without decoding signer state.
• Tests reporting, callback failures, cancellation, and manifest stability.

File	Description
tests/​test_live_runner.py	Tests payment-session callback behavior.
tests/​test_live_payment_session.py	Tests manifest ID stability across refreshes.
src/​livepeer_gateway/​remote_signer.py	Exposes manifest_id.
src/​livepeer_gateway/​live_runner.py	Adds callback support and reporting.
src/​livepeer_gateway/​__init__.py	Exports the callback type.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Repeated cancellation can interrupt payment reporting and leave a charged session unaccounted for.

1 open finding

• Protect payment drain from repeated cancellation · New

2 resolved since last review

• Preserve charged-session reporting when cancellation interrupts payment response
• Report signed payments before remaining response validation

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The orchestrator’s per-challenge ManifestID guarantee remains unconfirmed and needs human validation for billing reconciliation.

0 open findings

1 resolved since last review

• Protect payment drain from repeated cancellation

🧠 Review effort: Balanced

[eliteprox]

NE-Build-Track/livepeer-builder-core releases/m2 pins livepeer-gateway to e6d243b1d8bcbc8c1f0262ddbc3239a9f5b83cf4, the current head of this branch (#9).

That commit is not on main. Please leave feat/call-runner-manifest-id in place until this pull request merges. Deleting the branch first makes the pin unfetchable and breaks builder-core installs. Tracking issue: NE-Build-Track/livepeer-builder-core#13

[j0sh]

@rickstaa Copilot is not really pulling its weight here either, see below.

Here's a tighter version of this PR: #81

From Codex:

---

The Copilot follow-ups add substantial complexity for cases that do not warrant special handling here. Malformed signer responses already raise exceptions and prevent the paid request; both supplied signers serialize typed responses. The callback’s value is payment attribution, particularly when the runner subsequently fails—not reporting signer errors.

The cancellation fix introduces a bug: if an async callback stalls, asyncio.wait_for() cannot finish timing out, and repeated cancel() calls cannot stop the operation. The shield/drain loop keeps waiting indefinitely for user callback code. Registering an attribution callback should not break cancellation.

Please keep manifest_id and on_payment_session, but remove the private _on_payment hook, exception wrapper, and shield/drain machinery. Invoke the callback after get_payment() succeeds, outside the refresh handler, before final payment validation and the paid request.

The guarantee should be reporting successfully obtained payment sessions before dispatch. A signer can charge before its response reaches the client, so cancellation or response failure can leave a charge unreported. Shielding cannot guarantee complete accounting.

Simplify the tests accordingly:

• Use the real LivePaymentSession with mocked HTTP boundaries; _paid_runner currently reimplements the callback behavior it purports to test.
• Remove the drain and pre-validation reporting tests, plus the private-hook assertions.
• Fold no-payment coverage into an existing test and remove the redundant cancellation-after-reporting test.
• Retain focused coverage for sync/async callbacks, ordering before dispatch, repeated challenges, signer refusals, and callback errors.

[eliteprox]

Closing in favor of #81