Skip to content

native: shorter Python call path for a point read (port of #525) - #147

Draft
tae898 wants to merge 1 commit into
mainfrom
perf/native-point-read
Draft

tae898 wants to merge 1 commit into
mainfrom
perf/native-point-read

Conversation

@tae898

@tae898 tae898 commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

Draft: the speed claim waits for the timed A/B on the current native wheel (queued on the laptop, results will be posted here). Only the behaviour is claimed now: results are identical to the long path.

Port of humemai/arcadedb-embedded-python#525 (closed when the bindings moved here), rewritten onto the current _native code. A point read (db.query("sql", "SELECT FROM T WHERE id = ?", v).first()) spends about 15 of 34 us in Python outside the engine in an untimed per-call decomposition on the laptop, so the change shortens that Python part. No Java, no native-image rebuild: the wheel's libarcadedb.so is the same.

Changes:

  • codec.encode_params: one exact int or exact str parameter is written as {"pos": [v]} directly (the bytes json.dumps gives), without the type-set check and the encoder call. bool, subclasses, floats and everything else take the general path as before.
  • lib.exec_query: arcadedb_n_exec for a text-only parameter list, without the by-name getattr, the thread-handle call and the argument tuple. It reads the reply as call_bin does (a float32 tail still decodes). NativeDatabase._exec uses it with cached language and statement bytes (lib.bc).
  • NativeResultSet.first(): when the first batch holds the row and the result is readable, take the row and close, without the iterator protocol. The state afterwards is the iterator path's (closed, not marked drained); a lazy command result, an empty result or a closed database take the old path.
  • Record wrap: the records import in NativeResult._record and the database import in records._java_object are done once, and _wrap/_wrap_dict look the class up in a module dict.

Dropped from #525 because current code already covers or changed it: the lookup_by_key statement cache (it now calls the engine's lookupByKey through arcadedb_n_lookup; it still gains from the first() change), and the shared Java-object stand-in (each record keeps its own object, only the import is cached, so identity and setattr behaviour do not change).

Untimed numbers from the old study (us per call, whole call including two get() calls): SELECT .. WHERE id = ? .first() 34.36 to 29.60, encode_params 2.39 to 0.58, json_loads 4.86 to 4.49 (not touched here). These describe the previous code and are not a speed claim. The deciding timed figure will be sql_by_id_first_us, with lookup_by_key_us, range_1000_rows_us and cypher_2hop_count_us as guards, A/B on the current wheel with an A/A control.

Tests: tests/test_native_point_read.py (native backend only, listed in conftest.py) pins the one-parameter text against json.dumps and the general encoder for 13 int/str values and 17 other kinds, point-read rows against the iterator and to_list() (float32 tail, escapes, multi-byte text), first() state and errors, record wrapping per kind, lookup_by_key, and a spy that the short path is the one taken. Run against the unpacked 26.11.1.dev0 linux-x86_64 wheel with this PR's four Python files overlaid (they are byte-identical to main in the wheel): new file 99 passed; with it test_native_result_transport, test_resultset, test_parameter_binding, test_lookup_by_key_index, test_results_after_close, test_positional_count, test_record_id_values, test_graph_api, test_resultset_read_errors, test_typed_rows, test_percall_calls and test_native_columnar_insert passed (the skips are the suite's NA-JVM/NOTYET entries). The two spy tests fail on the unpatched wheel, the rest pass on both. black 26.10.1, isort 9.0.2 --profile black and bandit are clean.

🤖 Generated with Claude Code

Port of humemai/arcadedb-embedded-python#525 (closed when the bindings moved here) onto today's _native code. An untimed per-call decomposition
on the laptop put about 15 of 34 us of `SELECT .. WHERE id = ? .first()` in Python outside the engine. What changes, behaviour identical:

- codec.encode_params: one exact-int or exact-str parameter is written as '{"pos": [v]}' directly (the bytes json.dumps gives) without the
  set of types and the encoder call; bool, int/str subclasses, floats and everything else keep the general path.
- lib.exec_query: arcadedb_n_exec for a text-only parameter list without the by-name getattr, the thread-handle call and the argument tuple;
  same reply reader as call_bin (a float32 tail still decodes). database._exec uses it, with cached language and statement bytes.
- NativeResultSet.first(): when the first batch holds the row and the result is readable, take it and close, without the iterator protocol.
  Same state afterwards as the iterator path (closed, not marked drained).
- Record wrap: the import in NativeResult._record and records._java_object is done once, and _wrap looks the class up in a module dict.

Dropped from #525: the lookup_by_key statement cache (lookup_by_key now calls the engine's lookupByKey through arcadedb_n_lookup; it still
gains from the first() change), and the shared Java-object stand-in per process (each record keeps its own, only the import is cached).

Untimed numbers from the old study (us per call, whole call incl. two get() calls): SELECT .. WHERE id = ? .first() 34.36 -> 29.60;
encode_params 2.39 -> 0.58, json_loads 4.86 -> 4.49 (unchanged by this, see the follow-up note on a binary first batch). These are a
decomposition on the previous code, not a speed claim: the timed A/B on the current wheel is still to run.

tests/test_native_point_read.py (native backend only, collected in conftest): the one-parameter text equals json.dumps and the general
encoder for 13 int/str values and 17 other kinds, the point-read rows equal the iterator and to_list() rows (float32 tail, escapes,
multi-byte text), first() state and errors, record wrapping per kind, lookup_by_key, and a spy that the short path is the one taken.
The two spy tests fail on the previous code, the others pass on both.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant