Skip to content

skillopt-sleep: max_tokens_per_night is defaulted and documented but never enforced #313

Closed

Description

What happens

skillopt_sleep/config.py defines "max_tokens_per_night": 400_000 under the budgets section, and skillopt_sleep/budget.py has a Budget controller that "stops cleanly when exhausted and reports what it skipped". Nothing connects them on main: the key is never read outside DEFAULTS, and Budget( is never constructed. A night's spend is therefore bounded only by max_tasks_per_night, edit_budget and the number of fan-out skill groups, while the config suggests a token cap.

How to check

Version: main at 343db22 (uv tool install from git), Python 3.14, macOS.

P=$(python -c 'import skillopt_sleep, os; print(os.path.dirname(skillopt_sleep.__file__))')
rg -n 'max_tokens_per_night' "$P"   # only config.py DEFAULTS
rg -n 'Budget\(' "$P"               # only the class definition in budget.py

This was also noted in passing in the closed feedback issue #57 ("max_tokens_per_night exists, but no real budget enforcement"); that issue was closed for its other points, and the budget is still not enforced at 343db22.

Request

Either wire max_tokens_per_night (and optionally a wall-clock cap) into the cycle through Budget, stopping before the next model call once it is spent and saying so in report.md, or remove the key from DEFAULTS and the docs so users do not rely on it.

Activity

  1. Yif-Yang commented on Oct 17, 2026

    @Yif-Yang
    Contributor

    #317 now implements a documented step-boundary limiter, but the production tool route still schedules an uncached final validation call after the initial step has exhausted the budget. That behavior persists with #251's accounting fix. The review asks for accurate scheduling/evidence handling within the declared step-based policy, not an undisclosed hard-cap redesign. The issue remains open pending that correction, official checks and merge.

  2. Yif-Yang commented on Oct 18, 2026

    @Yif-Yang
    Contributor

    Resolved by #317, merged as 1a44050fc53a. The configured budget is now enforced at the documented stage boundaries, including the previously missed uncached final-validation step. Actual backend/optimizer/gate and stored-artifact tests confirm that skipped final validation reuses complete evidence for the final retained documents, including accepted/rejected edits, tool/text routes, timeouts and current accounting behavior.

    All six exact-head CI jobs and CLA passed; the actual merged main was also re-fetched and independently tested. The issue was automatically closed by the merge. This is a stage-boundary limiter: a started step may finish above the threshold, and backend token counters/estimates are not a per-call hard limit, an elapsed-time limit, or a guaranteed monetary cap.

  3. added a commit that references this issue on Oct 18, 2026
    1a44050
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions