Repository navigation
fix: --sharpit-app reports an uninspectable app as unknown, not as an empty pass - #207
michaelegner wants to merge 9 commits into
Conversation
|
Thank you, @michaelegner. This is careful work. A false We will take it from here. Flask assessment comes in the next release, 0.8.0, along the lines you proposed in #75: this fix first, then a Flask route model behind an adapter lookup in Please keep using sharpit and tell us what you run into. If you can share it with other Python developers, that does more for the project's growth than anything else. |
|
Thanks, @ffroliva. Glad the fix is useful. |
|
Thank you again for this change. An update on what is needed before it can merge. Earlier we mentioned a contributor licence agreement. The project has since settled on the To add the sign-off to the commits already on this branch: git rebase --signoff <base-commit>
git push --force-with-leasewhere We will then merge the current release branch into this pull request, resolve the conflict in |
|
Thanks for the update, @ffroliva. The DCO approach works for me. I'll add the sign-offs to my commits and update the branch. Since it already contains a merge commit from main, I'll verify the rewritten history before pushing. Happy to leave the 0.8.0 integration and conflict resolution with you, as proposed. |
Signed-off-by: Michael Egner <m.egner@gmx.net>
Signed-off-by: Michael Egner <m.egner@gmx.net>
Signed-off-by: Michael Egner <m.egner@gmx.net>
Signed-off-by: Michael Egner <m.egner@gmx.net>
Signed-off-by: Michael Egner <m.egner@gmx.net>
…test imports its module Signed-off-by: Michael Egner <m.egner@gmx.net>
…s annotations Signed-off-by: Michael Egner <m.egner@gmx.net>
…changelog Signed-off-by: Michael Egner <m.egner@gmx.net>
Signed-off-by: Michael Egner <m.egner@gmx.net>
2cea593 to
d02ac08
Compare
|
Sign-offs added; history is now linear on top of the earlier main merge, with the same tree as before. |
|
Thank you, @michaelegner. We are taking your nine commits onto a branch from |
…s unknown (#207) fix: --sharpit-app reports an uninspectable app as unknown, not as an empty pass
|
Merged into |
Found while scoping #75 (Flask adapter). Independent of Flask: no Flask code and no new dependency.
Problem
pytest --sharpit --sharpit-app <spec>onmain(47753e64):--sharpit-appnamesasyncfactory's coroutine)PASSED, no route inspected ("usable": false)UNKNOWNCORSMiddleware(app))UNKNOWN: application could not load (TypeError): the loader called itUNKNOWN, not calledAPIRouter/ StarletteRouterUNKNOWN: application could not load (TypeError): the loader called itPASSEDPASSED(confirmed empty, 0.3.0 spec §6)The new reason is
application routes cannot be inspected (<module.Class>); expected an app, a router or a zero-argument factory.Cause
_route_listinfastapi/introspector.pyreturned[]both for "no routes" and for "no route list to read".run_advisorsskips the "no routes seen" guard when--sharpit-appis given, so the empty model went on to the confirmed-emptyPASSED.load_appcalled every callable without arouterattribute. A WSGI app and a router are callable, so both were called with no arguments.Fix
_stored_routesreturnsNonewhen there is no route list.inspectable(app)and_route_listshare it, so the two cannot drift. A class is never inspectable.load_appcalls the named object only when it accepts no arguments. It is used as it is when it is inspectable, has arouterattribute (inspect.getattr_static), needs arguments, or has an unreadable signature. On Python 3.14 the signature is read withannotation_format=FORWARDREF, so a factory annotated with aTYPE_CHECKING-only name is not broken._configured_modelsreportsUNKNOWNfor an object that is not inspectable. The reason goes through the report masker, and the class name is never formatted from a non-string__module__.Behaviour change
A suite that named a non-app object with
--sharpit-appwas green and is nowunknown(exit 1) unless--sharpit-allow-unknownis passed. This changes one existing test,test_live_engine_driver_counts_as_covered. It named a SQLAlchemy engine to get its module imported. Branch review also showed it did not bite: driver detection runs before advisors import the app. It now imports the module from the test file. With the engine it exits 0; without the engine it exits 1 withuntraced database driver in use: aiosqlite.Finding → test → red output
All reds were checked with
scripts/dev/red_check.py:RED (assertion under tests/).test_sharpit_app_factory_of_an_uninspectable_app_is_unknownassert 'PASSED' == 'UNKNOWN'test_sharpit_app_uninspectable_app_fails_the_run_by_defaultassert <ExitCode.OK: 0> == 1test_sharpit_app_instance_without_a_route_list_is_uninspectableapplication could not load (TypeError)test_sharpit_app_router_is_assessedassert 'UNKNOWN' in ('PASSED', 'FAILED')test_load_app_never_calls_an_inspectable_objectassert 1 == 0(call count)test_load_app_never_calls_a_callable_with_an_unavailable_signatureassert 1 == 0(call count)test_load_app_returns_a_callable_that_needs_argumentsassert None is <function needs_arguments>routestaken for an apptest_inspectable_is_false_for_a_class_that_holds_a_route_listassert True is False(wrong-value stub, as indocs/development/workflow.md)test_uninspectable_reason_passes_the_report_masker__str__formats the reasontest_uninspectable_reason_never_formats_a_host_module_objectassert 1 == 0test_load_app_calls_a_factory_annotated_with_a_type_checking_only_nameNameErrorcaught,assert None is <FastAPI>(red on 3.14.6; skipped below 3.14)Controls that pass before and after:
--sharpit-allow-unknowntolerates the newUNKNOWNPASSEDapplication could not load (<Type>)routerattribute is not called (removing the rule turns it red)Limitations
signature.bind()shows a callable accepts no arguments, not that it is a factory. A callable app whose__call__(*args)accepts zero arguments is still called; the result isUNKNOWN.inspect.signaturereads__wrapped__/__signature__, so loading a named app is not free of host code. The oldhasattrwas not either.routerthrough__getattr__was returned as it was before; it is now called. Before it gave a falsePASSED, nowUNKNOWN.RAPI-ERR-001(INFO) can appear for a router where the app would not show it.application could not load (...)is not masked either; that predates this change.Process
/sharpit:predict: five personas, CAUTION 5.6/10, no stop condition; mitigations are in the plan./sharpit:scenario:docs/superpowers/plans/2026-11-16-sharpit-app-uninspectable-scenario.md./sharpit:plan:docs/superpowers/plans/2026-11-16-sharpit-app-uninspectable.md./sharpit:branch-review: seven lenses. One High and one Medium were fixed test-first; the Lows are listed above.Gates (local)
ruff check,ruff format --checkandpyrightare clean. Pyright is also clean on--pythonversion 3.14forapps.py.pytest -q -n 8(idle machine): 15,186 passed, 6 failed, none from this change.test_dotenvandtest_nested_secret_json_is_masked_get_and_mcp[__]fail the same way on untouchedmainon this machine (local environment).-n 8. Run alone (-n 0) all 11 tests in those files pass, on this branch and on untouchedmain. The change touches neither the capture middleware nor the SQL listener.pytest_plugin/apps.py94%)--sharpit-app pkg:appandpkg:create_appboth giveUNKNOWNwith the new reason, and the N+1 finding still appears.