Repository navigation
fix: reject stale parent_id in tag import and make parent changes order-independent - #877
Conversation
…l_id Closes openedx#858. Alternative to openedx#867 with a much smaller footprint. UpdateParentTag.applies_for compared the row's parent_id with the parent's pre-import external_id, so a row that kept the old id was "no change" and _validate_parent never ran for it. When the same file renamed that parent away (tag_1 -> tag_50), the stale reference was accepted silently. Now such a row applies as a parent change whenever its parent is renamed away (`_vacated_pks`), and _validate_parent rejects it with the existing "renamed to X; use X" error. _validate_parent also rejects a row whose parent_id is its own parent's pre-import id when another row takes that id (a swap): that reference is ambiguous, while a child naming the end-state id of a swapped parent is still accepted. A case-only rename no longer counts as vacating the id, since external_id is case-insensitive. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…trees Covers the remaining bugs openedx#867 found around parent_id handling, with one mechanism instead of a dry-run simulator and a conditional rebuild: - The plan computes the tree as it will be after the import (_end_state), rejects a parent cycle or a tree deeper than the limit there (Tag.save only validates the saved tag's depth, and silently corrupts lineage on a cycle), and records which existing tags change parent. - Rows are built after the row that creates or renames-in their parent, so a child may precede its new parent in the file. _validate_parent recognises a renamed-in parent from _vacated_pks, whatever the row order. - execute() detaches every moved tag before running the actions. After that each tag is at or above its final depth and none is an ancestor of its new parent, so no action can overflow or form a cycle part-way. - Kept children of deleted tags are promoted before the first delete, so the cascade cannot remove a kept grandchild. - A child whose deleted parent's id is reclaimed by a rename follows the renamed tag instead of being left at the root. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The detach of every tag whose parent changes used to run as a hidden phase at the start of execute(). It is now a synthesized DetachTag action per tag, like StageTagExternalIdForSwap, built after staging and before the row actions, so the plan lists everything that runs. A root tag, a kept child of a deleted tag (already promoted) and a row with an unknown parent get no detach action. The tag is looked up by pk, since staging may have moved it onto a placeholder id. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The error for a parent_id that names a parent renamed in the same file showed only the action's position in the plan, which an admin cannot map back to the file once staging, delete and detach actions come first. Prefix it with the row, using the same "Row N" numbering as the duplicate id and parent cycle errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Name the parts of the parent validation condition, rename _is_current_parent to _was_parent_before_import, use "pre-import id" and "final id" consistently, and comment the cycle and depth walk, the final-tree bookkeeping and each case the detach builder skips. Build the UpdateParentTag description when the action is created, so it names the old parent even if the plan is printed after a detach already ran, and compute the casefolded renamed-to ids once per plan instead of once per row. When a child row names its parent's pre-import id and another row takes that id, the error now also says how to move under the tag that takes it: import the id changes and the move as two separate files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t review nits Add TestImportAcceptanceScenarios from openedx#867, adjusted to this implementation's messages. It ties each openedx#673 scenario and each openedx#858 acceptance criterion to an import through the public API, in JSON and CSV and across row orders, and checks the export and the content associations, not only the plan. Two swap scenarios from openedx#867 replace test_import_swap_external_ids_with_children and test_execute_child_of_deleted_parent_follows_rename_onto_its_id, which they cover in more row orders and both formats. Also pin that executing a plan logs the moved tag's old parent, and address the review's notes on test comments and docstrings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the pull request, @ufedaseyeuconsultant! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
|
@mgwozdz-unicon Three follow-up problems in the taxonomy tag import, found while working on #858. None of them was introduced by the #858 fix, and none corrupts data: a failing import runs in one transaction and is rolled back. Each one can become its own issue. 1. The "Unknown parent tag" error does not name the rowProblem. When a row's
Expected. The error names the row, with the same "Row N" numbering as the other import errors (the 1-based position among the file's data rows; in a CSV file that is spreadsheet line N + 1), for example Proposed fix. In 2. A new parent that the same
|
mgwozdz-unicon
left a comment
There was a problem hiding this comment.
Thanks for working through the review on #871. Every inline comment I left there is addressed here. I'm requesting changes for two reasons: #858 still describes the other implementation, and the tests are now much larger than they need to be. I had Claude do a first pass on the tests and comments before my own review. The duplicate pairs below were checked against the code, and all line numbers refer to 30df7b4.
1. #858's acceptance criteria still describe the other implementation
The PR description says the acceptance criteria in #858 were updated to match this implementation, but the issue's last edit is from 2026-10-14, before #871 was opened. Three criteria still don't match this branch:
- Additional fix 1, last scenario: "the Plan lists the actions in the same order as before this fix." A file that moves an existing tag now also gets a "Move tag X to the root first" action.
- Additional fix 5, last scenario: "the Plan has no extra actions that move tags to the root." This PR adds one for every existing tag whose parent changes, by design.
- Additional fix 4 quotes the error
This import would leave tags deeper than 6 levels, or build an invalid parent hierarchy, so it cannot be applied.This branch saysThis import would leave tag X more than 6 levels deep.
I'm fine with this branch's behavior in all three cases. Could you update the issue, so that a reviewer checking the PR against #858 sees criteria it actually meets?
2. TestImportAcceptanceScenarios is too large
On #871 I suggested bringing this class over from #867. I should have been clearer: I meant the scenarios that nothing else covers, not the whole class. As ported, it is about 1,036 lines with 25 tests that together run about 285 full imports, and it is most of why this PR is twice the size of #871. Three changes would shrink it.
Tests that repeat another test on the same input. Each pair below builds the same tree from the same rows:
test_ancestry_inversion_in_any_row_order(test_api.py:1677),test_parent_and_child_swap_places(1710),test_deep_tag_moved_to_root_with_new_children_in_any_row_order(1799) andtest_subtree_moved_under_deep_tag_before_it_is_shallowed(1866) are the four cases oftest_execute_parent_moves_whose_steps_depend_on_each_other(test_import_plan.py:764).test_import_that_leaves_tags_too_deep_is_rejected(test_api.py:1942) runs the same two scenarios astest_generate_actions_import_too_deep_rejected(test_import_plan.py:939).test_rename_onto_freed_id_forming_parent_cycle_rejected(test_api.py:1516) runs the last two cases oftest_generate_actions_parent_cycle_rejected_once(test_import_plan.py:868).test_swap_keeps_tags_and_associations(test_api.py:1342) andtest_three_way_rotation_in_any_row_order(1369) repeat the existingtest_import_swap_external_ids(777) andtest_import_three_cycle_external_ids(819), adding more row orders.test_import_stale_parent_id_error_names_row(test_api.py:1063) is a third test that the stale-parent error names its row, after test_import_plan.py:669 and the swap tests in the new class.
Where the API version adds something real, such as more row orders, I'd move that into the existing test, for example as more ddt cases at the plan level, rather than keep a second copy that goes through the API.
Format and row-order loops. Most tests run every scenario in both JSON and CSV, and many in all 6 or 24 row orders. The two parsers differ only in how they read and write the file. Both hand the plan the same list of TagItem rows, as the comments at test_api.py:1701, 1822, 1902 and 2017 already note. One format, with the file order, the reversed order and one mixed order, would cover the ordering logic, and one CSV import is enough to cover the parser.
Docstrings. The 25 test docstrings add up to about 258 lines. Most draw ASCII before-and-after trees of rows that the test body already lists, quote #673 or #858 scenario titles word for word, or point at other tests: "the test above" at lines 1835, 1859, 1935 and 2096, and full test names in the class docstring at lines 1117 to 1130. A reference to another test goes stale silently when that test is renamed or moved. One or two sentences on the behavior under test would be enough for each.
Claude's rough estimate is that these three changes would remove about half of the class without losing a distinct assertion.
3. Comments in the source
- The reason for detaching first is explained three times: in the
TagImportPlandocstring (import_plan.py:68), theDetachTagdocstring (actions.py:709) and the comment ingenerate_actions(import_plan.py:521). I'd keep it in theTagImportPlandocstring and cut the other two to a line each. - The
_end_statedocstring (import_plan.py:297) and the comment in its second loop (import_plan.py:336) both explain what happens to an existing tag without a row underreplace=Trueandreplace=False. The comment is enough, so the docstring can drop those sentences. - The
_validate_parentdocstring (actions.py:132) is one 11-line paragraph that walks through the code. Two sentences would do, for example: "Return an error unlessparent_idnames a tag that holds that id once this import finishes. A row that names its own parent's pre-import id while this file renames that parent is always rejected, because that id then names nothing or, in a swap, another tag." _renamed_to_ids(actions.py:121) falls back to deriving the set from_vacated_pks"for an action built outside a plan". The only code that builds actions outside a plan is the tests in test_actions.py (lines 149, 174, 201 and 241), which set_vacated_pksby hand. If those tests also set_renamed_to_ids, the helper becomes a plain dict lookup, and the production code no longer carries a branch that only tests use.
4. Nit, optional
The stale-parent error reads Action error in 'update_parent' (#6): Row 2: Parent tag ..., so the plan position that an admin can't map back to the file still comes before the row number. The "User-visible changes" section of the PR description quotes the message without that prefix. Dropping the prefix for errors that have a row would change ImportActionError for every action, so I'm fine leaving it to the follow-up issue about naming the row in the "Unknown parent tag" error.
… tests Delete the acceptance tests that build the same tree from the same rows as an existing plan-level or API test, and move what only they had into the kept test: more row orders for the dependent parent moves, the swap and the three-way rotation, the other row order of the rename cycle, and a check that moved tags keep their pks. The remaining acceptance tests run one format with three row orders, with a single CSV import for the parser, and their docstrings state the behavior under test in a sentence or two. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep the reason for detaching moved tags first in the TagImportPlan docstring only, and cut the DetachTag docstring and the comment in generate_actions to one line each. Drop the replace=True/False sentences from the _end_state docstring; the comment in its second loop now also says how a kept tag's parent is recorded. Shorten the _validate_parent docstring to two sentences. Remove the _renamed_to_ids helper and its fallback that derived the set from _vacated_pks for actions built outside a plan. The callers read indexed_actions["_renamed_to_ids"] directly, and the _validate_parent tests in test_actions.py set it next to _vacated_pks, as the plan does. No behavior change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1. #858's acceptance criteriaDone. AC were updated 2.
|
mgwozdz-unicon
left a comment
There was a problem hiding this comment.
Claude Fable and I worked on this review together.
Thanks, this round addressed everything from my last review: #858 now matches the branch, TestImportAcceptanceScenarios is down to the scenarios nothing else covers, and the source comments are consolidated. I ran the import_export suite (234 tests), pylint, pycodestyle, isort, mypy and lint-imports against 299cae5 and everything passes, and CI is green. Approving, but please file the follow-up issues you mentioned. Everything else below are nits. Line numbers refer to 299cae5.
1. Please file the three follow-up issues and link them in the PR description
The "Other information" section says each known issue has its own issue, but I can't find any filed yet. Your comment above is already issue-ready, so please open the three issues from it (each as a child of #673, labeled as belonging to Epic 4, and assigned to you in Sprint 5) and replace "each with its own issue" with the issue numbers. Two scope additions while you're there:
- For the second follow-up (a new parent that the same
replace=Trueimport deletes), please also cover the sibling case I ran into: a child row that keeps the id of its current parent while the file omits that parent. The plan accepts it, the delete phase promotes the child to the root, and the import succeeds, so the result differs from what the file says (the row saidparent_id: tag_1, the tag ends up at the root). This also happens onmain. A check in_validate_parentalone wouldn't catch it, becauseUpdateParentTag.applies_fortreats the row as "no change" and_validate_parentnever runs for it. - For the third follow-up (the renamed-plus-new cycle), please include the new-only cycle too.
_validate_hierarchydeliberately leaves a cycle made only of new tags to_validate_parent(import_plan.py:377), so a file wherenew_aandnew_bname each other getsUnknown parent tag (new_a). Add a row for the parent to your file., which is misleading because the file does have that row. Reporting the cycle error for every cycle and skipping the unknown-parent check for rows on a reported cycle would fix both at once.
2. Nit in the source
- actions.py:724: the comment restates the
assertbelow it, which is there to narrow the type for mypy. I'd delete the comment.
3. Nits in the tests
- test_import_plan.py:837-840: the assertion that every detach comes before every other action took me a moment to read. Something like
self.assertEqual(names[:names.count("detach")], ["detach"] * names.count("detach"))says it directly. - test_api.py:1292 (
test_new_id_without_previous_id_is_created) and 1354 (test_export_has_no_previous_id_and_round_trips_without_changes) pass onmainand repeat existing coverage:test_validandtest_import_with_export_outputin the same file,test_export_does_not_include_previous_idin test_parsers.py, andtest_reimport_after_generated_external_ids_is_a_no_op. I'd delete both. - test_api.py:1468 (
test_three_level_chain_reversed) and 1486 (test_tag_moved_under_its_former_grandchild) also pass onmain, so they don't catch a bug this PR fixes. They're still reasonable regression guards for the detach step. Keep them, or fold the two trees into the ddt list oftest_execute_parent_moves_whose_steps_depend_on_each_other(test_import_plan.py:765), whichever you prefer. - test_api.py:1396: the two messages above it are asserted in full, while this one only checks for
COMP-GHOST. The full message isUnknown previous_id (COMP-GHOST). No tag with that external_id exists in this taxonomy. - test_api.py:1180:
_exportis a one-line wrapper with three callers. I'd inline it. - test_import_plan.py:652: the first two ddt cases (exact id, case variant) are the same scenarios as
test_swap_children_naming_stale_ids_rejectedandtest_swap_child_naming_parent_id_in_another_casein test_api.py. The third case, where the child is renamed too, is the only one not covered elsewhere. - test_import_plan.py:670 only differs from line 632 in that the child is a new tag and the deletes shift the action number, so it could be a ddt case of that test rather than its own.
- test_api.py:1654: a plain rename with a case-variant parent id goes through the same path as the exact-case test at test_import_plan.py:600, because the case-insensitivity comes from the database field. It could be a ddt case there instead of an API test.
Remove two acceptance tests that repeat existing coverage (creating a tag without previous_id, and the export round trip). Fold the reversed three-level chain and the move under a former grandchild into the plan-level parent-move ddt, which runs three row orders and checks that pks are kept. Turn the new-child stale-parent test and the case-variant plain-rename API test into ddt cases of the plan tests they repeat, keep only the renamed-child case of the swap ddt, assert the full "Unknown previous_id" message, inline the one-line _export helper, and state the detach-first assertion directly. Also remove the comment in DetachTag.execute that restated the assert below it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1. Follow-up issuesI filed two issues instead of three, both as children of #673, in Epic 4 and Sprint 5:
The PR description now links both issues in place of "each with its own issue". 2. Nit in the source
3. Nits in the tests
The import_export suite now has 231 tests, and |
Description
Closes #858. This PR takes the simpler implementation from @ormsbee's branch
ormsbee/858-stale-parent-id-simplerand addresses the review on it. Its three commits are included unchanged; the commits on top address the three reviews. It replaces #867.What the import does now
A taxonomy import with the
previous_idcolumn (from #673) can rename a tag'sexternal_id, swap ids between tags, and move tags between parents in one file. This PR makes the parent handling correct in those cases:parent_idis its parent's pre-import id, when the same file gives that id away, is rejected at the Plan step. A plain rename makes that id name nothing after the import. A swap makes it ambiguous: the id now belongs to a different tag, so the row could mean "stay with my parent" or "move under the tag that now has this id". A child that names its parent's final id is accepted.replace=True, kept children of a deleted parent are promoted to root before any delete runs, so a delete no longer cascades onto a tag the file keeps.Changes made after the reviews
UpdateParentTagbuilds its description when the action is created. A caller that runsgenerate_actions()and thenexecute()now logs the old parent instead of "None"._validate_parent,_is_current_parentrenamed to_was_parent_before_import, "pre-import id" and "final id" used consistently, comments on the final-tree walk, the detach builder and the row ordering, and clearer variable names.TestImportAcceptanceScenariosfrom fix: make tag import parent_id handling correct and order-independent #867 is added, adjusted to this implementation's messages, and reduced after the second and third reviews to the scenarios that no other test covers. Each scenario imports through the public API in JSON, in the file order, the reversed order and one mixed order, and checks the export and content associations. One CSV import covers the parser. Where a removed acceptance test added row orders or an assertion to a test that remains, that part moved into the remaining test.TagImportPlandocstring; the_validate_parentand_end_statedocstrings are shorter; the set of renamed-to ids is a plain lookup, without a fallback that only tests used.assertis gone.Who is affected
previous_id: some files that were accepted and then misbehaved are now rejected with an error that names the row, and some files that were rejected only because of their row order now import.User-visible changes
Action error in 'update_parent' (#K): Row N: Parent tag X is renamed to Y in this file; use Y as the parent_id., and when another row takes the id X, alsoTo move under the tag that takes the id X in this file instead, import the id changes and the move as two separate files.The action name is the one of the row's action (update_parent,createorrename_external_id). The(#K)prefix is the action's position in the plan, which every action error has; the row is theRow Npart. Showing the row without that prefix is left to Tag import: parent errors name their row and report every cycle as a cycle #892.Row N's parent_id (X) creates a parent cycle.This import would leave tag X more than 6 levels deep.Unknown parent tag (X). Add a row for the parent to your file.(it used to say "You need to add parent before the child in your file", which no longer applies).Move tag X to the root first, so that the parent changes in this file can run in any order.for each existing tag whose parent changes, before the row actions.tags.Supporting information
Testing instructions
Automated:
pytest tests/openedx_tagging/import_export -o addopts=""(231 tests;-o addopts=""skips the coverage flags thattox.iniadds).pytest tests/openedx_tagging -o addopts=""(617 tests).lint-imports,pylint src tests test_utils manage.py test_settings.py mysql_test_settings.py,pycodestyle,isort --check-only,pydocstyle srcandmypy --show-traceback.Manual, through the taxonomy import endpoint or the Studio import wizard. Create a taxonomy with
tag_1(root),tag_2(child oftag_1),tag_3(root) andtag_4(child oftag_3), with valuesTag 1toTag 4. Import each file below. The import is a full replace, so each file lists the whole taxonomy.tagsrows){id: tag_50, value: "Tag 1", previous_id: tag_1},{id: tag_2, value: "Tag 2", parent_id: tag_1},{id: tag_3, value: "Tag 3"},{id: tag_4, value: "Tag 4", parent_id: tag_3}Action error in 'update_parent' (#2): Row 2: Parent tag tag_1 is renamed to tag_50 in this file; use tag_50 as the parent_id.Nothing changes.{id: tag_3, value: "Tag 1", previous_id: tag_1},{id: tag_1, value: "Tag 3", previous_id: tag_3},{id: tag_2, value: "Tag 2", parent_id: tag_3},{id: tag_4, value: "Tag 4", parent_id: tag_1}tag_2undertag_3andtag_4undertag_1.tag_2row sayingparent_id: tag_1Action error in 'update_parent' (#6): Row 3: Parent tag tag_1 is renamed to tag_3 in this file; use tag_3 as the parent_id. To move under the tag that takes the id tag_1 in this file instead, import the id changes and the move as two separate files.The plan's staging and detach steps come first, so the action number differs from the row number.Other information
openedx-platformneeds to change.test_generate_actions_existing_child_keeps_stale_parent_id_rejected(test_import_plan.py) expects the "Row N" prefix and has a second case: a new child row, after the deletes of a full replace import, so that its action number differs from its row.test_import_plan.pykeeps only the case where the child is renamed too, astest_generate_actions_swap_renamed_child_naming_own_parent_stale_id_rejected, and expects the "Row N" prefix and the hint. Its two other cases are covered byTestImportAcceptanceScenarios.test_generate_actions_parent_id_stale_after_plain_rename_rejected(test_import_plan.py) also runs the parent id in another case and with the child row first.test_import_swap_external_ids_with_children(test_api.py) andtest_execute_child_of_deleted_parent_follows_rename_onto_its_id(test_import_plan.py) are removed, becauseTestImportAcceptanceScenarioscovers the same scenarios in more row orders.test_execute_parent_moves_whose_steps_depend_on_each_other(test_import_plan.py) runs each case in its listed order, reversed and one mixed order, checks that moved tags keep their pks, and has two more trees: a three-level chain reversed, and a tag moved under its former grandchild.test_generate_actions_parent_cycle_rejected_once(test_import_plan.py) has one more case: the two renamed tags that name each other, in the other row order.test_import_swap_external_idsandtest_import_three_cycle_external_ids(test_api.py) run in more row orders._validate_parenttests intest_actions.pythat build actions outside a plan set the renamed-to ids next to the renamed tags, as the plan does.replaceTrue and False, and compared the result with an expected tree. It found no mismatch on @ormsbee's branch, and the same counts after each review commit.parent_idnames a tag that the samereplace=Trueimport deletes is rejected at the Plan step. Today it either fails when the import runs, or, when a kept tag's row names its current parent, imports and leaves that tag at the root. This also happens onmain.claude-opus-5-5) wrote the review commits on top of @ormsbee's commits.🤖 Generated with Claude Code