Skip to content

docs(faq): the definition language is not Python syntax - #297

Draft
dimitri-yatsenko wants to merge 6 commits into
mainfrom
docs/faq-definition-language-python
Draft

dimitri-yatsenko wants to merge 6 commits into
mainfrom
docs/faq-definition-language-python

Conversation

@dimitri-yatsenko

@dimitri-yatsenko dimitri-yatsenko commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Follow-up to a review note on PR 295: faq.md still listed MATLAB as a live host language beside the docs' Python-only scope.

  • The Definition Language: drops "uniform support across multiple host languages (Python, MATLAB, and potentially others)". It now says DataJoint is implemented in Python, but definitions are parsed by DataJoint, not the Python interpreter, and queries are compiled into SQL rather than executed in Python. Queries are written with Python operators and values such as dicts, but the query algebra, like the definition language, is independent of the host language, so other implementations remain possible.
  • Why Multiline Strings? table: the Language-agnostic row says the same, instead of "Same syntax for Python, MATLAB, future implementations".

@dimitri-yatsenko dimitri-yatsenko added the documentation Improvements or additions to documentation label Oct 5, 2026
MilagrosMarin
MilagrosMarin previously approved these changes Oct 5, 2026

@MilagrosMarin MilagrosMarin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the claim rather than the wording: declare.py imports pyparsing and builds its own attribute and foreign-key parsers, so "a definition is parsed by DataJoint itself, not by the Python interpreter" is literally true, not just a nice framing. No MATLAB left in faq.md, and this reads consistently with the narrowing in #295.

One editorial note. The Language-agnostic row label now promises something its cell no longer claims — the cell argues only the negative, and says it three ways: "Does not rely on Python syntax; DataJoint parses it, not the Python interpreter, so it is not tied to Python." The prose above already lands the point. Something like "Parsed by DataJoint, not the Python interpreter — nothing in it is Python-specific" keeps the row saying what its label advertises. The same triple-restatement is in the paragraph.

Worth noting on merge order: #295 hasn't landed, so until it does, citation.md on main still reads "Citing DataJoint Python and MATLAB". Merging #295 first keeps the two pages in step.

Unrelated and pre-existing: the link on the edited line ends table-declaration.md/. There are 101 trailing-slash .md/ links across the docs — they pass the checker, so they resolve, but it's the same shape as the dj-platform.svg/ path #278 fixed. Its own sweep, not this PR.

@MilagrosMarin MilagrosMarin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The table cell is exactly right now, and I confirmed aef817c8/1ae51e34 cancel out — no net change hiding in the pair.

The paragraph picked up a claim that doesn't hold, though. It now says neither the data definitions "nor its query operations rely on Python syntax or Python data structures." The definitions half is true and well put. The queries half is wrong on both counts:

  • Python syntax — the algebra is Python operator overloading. expression.py defines __and__, __sub__, __mul__ and __matmul__; a & b and a * b are Python expressions, evaluated by the interpreter.
  • Python data structures — make_condition dispatches on collections.abc.Mapping, numpy.void, pandas.DataFrame, bool and str. condition.py's own docstring says it converts "(dicts, strings, QueryExpressions) into SQL WHERE clauses". Subject & {"subject_id": 1} is a dict restriction.

A reader disproves the sentence with one line in a notebook, which is why I'd rather catch it here than after it publishes.

The claim that does hold, and is the stronger one anyway: queries are compiled into SQL rather than executed in Python, and the algebra itself is independent of the host language even though its surface syntax is that language's operators. Something like:

The DataJoint Library is implemented in Python, but its data definitions are parsed by DataJoint rather than the Python interpreter, and its queries are compiled into SQL rather than executed in Python. The definition language and the query algebra are both independent of the host language, so the same definitions and operations could be supported by implementations in other languages.

That keeps everything you were reaching for and drops only the part that isn't so.

dimitri-yatsenko and others added 2 commits October 16, 2026 10:03
The paragraph said query operations rely on neither Python syntax nor Python
data structures. They use both: the algebra overloads Python operators, and
restrictions take dicts and other Python values. What holds is that queries
are compiled into SQL rather than executed in Python, and that the algebra,
like the definition language, is independent of the host language.
@dimitri-yatsenko

Copy link
Copy Markdown
Member Author

Thanks, @MilagrosMarin. ce91738 takes your wording. The paragraph now says that definitions are parsed by DataJoint and queries are compiled into SQL, and it no longer claims that queries avoid Python syntax. It says outright that they are written with Python operators (&, -, *) and Python values such as dicts, and that the algebra they express is independent of the host language.

#295 merged first, so citation.md on main already reads consistently with this page. The trailing-slash .md/ links are left for their own sweep, as you suggested. Ready for another look.

@dimitri-yatsenko
dimitri-yatsenko marked this pull request as draft October 21, 2026 16:46

@MilagrosMarin MilagrosMarin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dimitri-yatsenko! Re-reviewed at ce917380 (plus the main merge):

✅ The paragraph now credits Python operators and values for writing queries while keeping the real claim: definitions parsed by DataJoint, queries compiled into SQL, algebra independent of the host language
✅ Table cell unchanged and still right
✅ The main merge adds nothing beyond faq.md; citation.md reads consistently with this page now that #295 is in

Trailing-slash .md/ links can go in their own sweep as agreed. Approving. Thanks for working through the wording!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants