diff --git a/src/about/whats-new-23.md b/src/about/whats-new-23.md index 4581266b..828301dd 100644 --- a/src/about/whats-new-23.md +++ b/src/about/whats-new-23.md @@ -2,7 +2,7 @@ DataJoint 2.3 adds a first-class **upstream read surface** — `Diagram.trace` and `self.upstream` — which make "a computed row derives only from its declared upstream inputs" easy to follow inside `make()` and easy to query afterward. It also ships the **SparkAdapter Codec Protocol** for typed rendering to Spark-native types, **`dj.deploy.set_replica_identity`** for PostgreSQL change-data-capture, and a **cascade fix** for Part-of-Part and renamed-foreign-key chains. -Later releases on the line added **redesigned diagram rendering** on the DataJoint brand palette, with a light/dark [`display.diagram_theme`](../reference/configuration.md#display-settings) setting, **S3 stores that resolve an ambient AWS identity** instead of requiring static keys, and **extrinsic provenance at the pipeline's entry points** — a hidden `_prov` attribute the framework records on every row arriving from outside. +Later releases on the line added **redesigned diagram rendering** on the DataJoint brand palette, with a light/dark [`display.diagram_theme`](../reference/configuration.md#display-settings) setting, **S3 stores that resolve an ambient AWS identity** instead of requiring static keys, and **extrinsic provenance at the pipeline's entry points** — a hidden `_prov` attribute the framework records on every row arriving from outside. 2.3.5 fixes **cascade deletes that removed rows outside the delete**, and a restricted distributed `populate` that removed other workers' jobs. > **Upgrading from 2.0, 2.1, or 2.2?** No API breaks — every feature on the 2.3 line is additive. > Two fixes in 2.3.3 do, however, reject input that previously passed silently: a misspelled native @@ -13,6 +13,25 @@ Later releases on the line added **redesigned diagram rendering** on the DataJoi > **Citation:** Yatsenko D, Nguyen TT. *DataJoint 2.0: A Computational Substrate for Agentic Scientific Workflows.* arXiv:2602.16585. 2026. [doi:10.48550/arXiv.2602.16585](https://doi.org/10.48550/arXiv.2602.16585) +## Changes in 2.3.5 + +2.3.5 is a patch release on the 2.3 line, and it is recommended for every 2.3 deployment: several of its fixes stop operations from removing data they should have kept. If you are upgrading from **2.3.4**: + +- **Cascade deletes remove only the rows they reach.** In three cases, a delete removed rows outside the cascade, without an error: + - `(T & dj.Top(n)).delete()` ignored the `Top` and deleted every row matching the other conditions, together with their descendants. `delete_quick()` did the same. The delete now starts from the rows the `Top` selects ([#1589](https://github.com/datajoint/datajoint-python/issues/1589)). + - A child table whose nullable foreign key shares a primary-key column with the parent lost the rows whose foreign key is NULL, although those rows reference nothing. In a diamond, a child reached through a secondary foreign key could instead be missed, and the delete failed with a misleading "unloaded schema" error. The cascade now copies a parent's condition to a child only when the foreign key carries every attribute the condition names, without renaming, and lies in the child's primary key. Otherwise it restricts the child by the restricted parent ([#1590](https://github.com/datajoint/datajoint-python/issues/1590)). + - With `part_integrity="cascade"`, deleting part rows deleted masters that had no matching part rows, and everything downstream of them. A master now joins the cascade only if one of its part rows did ([#1591](https://github.com/datajoint/datajoint-python/issues/1591)). + + `dj.Diagram.cascade(...).counts()` previews the corrected set. On earlier releases, restrict a delete by the primary keys it should remove: `(T & (T & cond).keys()).delete()`. + +- **On PostgreSQL, dropping a table no longer drops other tables' columns.** An enum attribute is declared with a PostgreSQL enum type named by its values, so tables whose enum attributes have the same values share one type. Dropping a table dropped its enum types with `CASCADE`, which also dropped every other column of that type, with its data. Every jobs table has the same `status` enum, so dropping one jobs table removed `status` from all the others in the schema. An enum type is now dropped only when no column uses it ([#1630](https://github.com/datajoint/datajoint-python/issues/1630)). +- **`alter()` keeps an attribute's values when it renames it.** A comment that opens with `{old_name}` renames that attribute. For a core-type attribute (`float32`, `int32`, and the like), `alter()` missed the marker and dropped the old attribute before adding the new one: on MySQL every value became `0.0`, or NULL for a nullable attribute; on PostgreSQL a nullable attribute lost its values. MySQL now renames with `CHANGE`. PostgreSQL refuses the rename before changing anything; renaming there comes in 2.4 ([#1631](https://github.com/datajoint/datajoint-python/issues/1631)). See [Alter Tables](../how-to/alter-tables.md). +- **A restricted `populate(reserve_jobs=True)` keeps other workers' jobs.** `populate(*restrictions, reserve_jobs=True)` refreshed the jobs table against the restricted `key_source`, so every job outside the restriction looked obsolete. Reservations held by other workers and error records were deleted once they were older than `stale_timeout`. Cleanup now checks keys against the full `key_source`; the restrictions only limit which jobs are added. Success and error records of rows that were deleted upstream are now removed on every refresh instead of lingering for up to `stale_timeout` ([#1594](https://github.com/datajoint/datajoint-python/issues/1594); part of [#1577](https://github.com/datajoint/datajoint-python/issues/1577)). See [Distributed Computing](../how-to/distributed-computing.md). +- **Jobs record `completed_time` to the millisecond.** `reserved_time` was recorded with milliseconds but `completed_time` was truncated to the second on MySQL, so a job finishing within the second it was reserved appeared to complete before it started. Both are now taken from the server clock at the column's precision ([#1600](https://github.com/datajoint/datajoint-python/issues/1600)). +- **`alter()` resolves foreign keys in the table's own module.** Called from another module (a migration script, or a notebook that imports the pipeline), `Table.alter()` looked up foreign-key references in the caller's namespace and failed with `Foreign key reference ... could not be resolved`. It now uses the namespace the table was declared in, as `Part.alter()` already did. Pass `context=` to override ([#1602](https://github.com/datajoint/datajoint-python/issues/1602)). See [Alter Tables](../how-to/alter-tables.md). +- **MySQL column-level privilege errors raise `AccessError`.** MySQL error 1143 (SELECT denied on a column) came through as a raw `pymysql` error, while the table-level 1142 already became `dj.errors.AccessError`. Both now raise `AccessError`, as on PostgreSQL ([#1586](https://github.com/datajoint/datajoint-python/issues/1586)). +- **Python 3.15 support.** `requires-python` now spans `>=3.10,<3.16`, and CI runs the suite on 3.15. On Python 3.15 only, `deepdiff` is held below 9 until its dependencies publish 3.15 wheels. DataJoint uses it only to check that a three-part `make` reads the same data twice, which behaves the same on 8.6. + ## Changes in 2.3.4 2.3.4 is a patch release on the 2.3 line. If you are upgrading from **2.3.3**: