Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# AGENTS.md - CI Workflows (.github/)

## Rules for Changes
These keep zizmor (`zizmor.yml`, fails on medium or higher findings) and the OpenSSF Scorecard clean.
- Pin every third-party action to a full commit SHA with a `# <release tag>` comment.
- Every workflow declares top-level `permissions` (`read-all`, `contents: read` or `{}`). Grant more per job, with a comment saying why.
- Every `actions/checkout` sets `persist-credentials: false`.
- Never inline untrusted contexts (`github.event.*`, `github.head_ref`, `inputs.*`) in `run:`; pass them through `env:`. Prefer `env:` for every value in new code.
- Every job sets `timeout-minutes`; for a job that calls a reusable workflow, set it inside the called workflow.
- `actionlint` runs in pre-commit.
- `workflows/array-api-skips.txt` lists array-api-tests node IDs skipped by the conformance job; precede each with a comment naming the upstream issue.
- `bump-sycl-deps.yml` rewrites the `DRIVER_PATH`, `OCLCPUEXP_FN` and `TBB_*` pins in `os-llvm-sycl-build.yml` with `sed`; keep their format.
- Dependabot (`dependabot.yml`) bumps actions, `environments/` pins and pre-commit hooks weekly with a 7-day cooldown.
78 changes: 78 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# AGENTS.md - dpnp

dpnp is a NumPy-compatible array library for SYCL devices. Its main namespace, `dpnp.fft` and `dpnp.linalg` also conform to the Python Array API standard (2025.12, `doc/reference/array_api.rst`). The Python layer (`dpnp/`) is built on `dpnp.tensor` (Array API, `dpnp/tensor/`) and pybind11 SYCL extensions (`dpnp/backend/extensions/`); `dpctl` provides devices, queues and USM memory.

## Rules for Changes

These come from recurring maintainer review comments. Directory-specific rules are in the nearest `AGENTS.md`.

- Match NumPy, or SciPy for `dpnp.scipy`: shape, dtype, memory layout, edge cases (empty, 0-d, NaN/inf) and error and warning text. Following CuPy instead is an exception. Any deliberate difference is documented in the docstring and covered by a test with a comment.
- Don't break Array API conformance. Where NumPy and the spec differ, raise it in the PR rather than silently picking one (precedent: `ndarray.T` on non-2-D arrays was deprecated to match the spec).
- Code and tests must work on devices without fp64 support (e.g. Iris Xe). Don't hardcode float64/complex128; use `dpnp.default_float_type()` or the dtype helpers in `dpnp/tests/helper.py`.
- Compute follows data: allocate results and temporaries on the inputs' SYCL queue with their coerced USM type, and raise when inputs computed together are on different queues (`get_usm_allocations` raises `ValueError`, `dpnp.tensor` raises `ExecutionPlacementError`). Never move inputs silently.
- Avoid hidden host transfers and queue syncs: `int()`/`bool()` on device arrays, `.wait()`, `asnumpy`, redundant full-size copies.
- Accept both `dpnp.ndarray` and `usm_ndarray`; validate and unwrap with `dpnp.check_supported_arrays_type` and `dpnp.get_usm_ndarray`, not ad-hoc `isinstance` or method calls.
- Every user-visible change, including build, recipe and C-API changes, gets a one-line `CHANGELOG.md` entry (format below).
- A new public function needs an export in `dpnp/__init__.py` (or in its subpackage's `__init__.py`), a numpydoc docstring, an entry in `doc/reference/*.rst`, tests (including `test_sycl_queue.py` and `test_usm_type.py`) and a changelog entry. Don't export names that NumPy/SciPy don't.
- Deprecate before removing: keep the old API for at least one release with a `DeprecationWarning`, and add a `Deprecated` changelog entry and a test for the warning. Removing something NumPy itself has removed can land directly with a `Removed` entry.
- Every new branch and error path has a test; a bug fix comes with a test that fails without it. Use `# pragma: no cover` only for branches the Linux coverage job can't reach (CUDA, HIP, Windows-only) or that can't happen by construction.
- A workaround, skip or xfail gets a comment with the full issue URL so it can be reverted later. In `dpnp/tests/third_party/`, a skip for an unsupported feature needs only its reason string.
- Keep comments short and accurate: one line of why. Don't narrate the change, restate the code, or mention "this PR".
- One PR, one change. Search for an existing helper before adding one.
- New files carry the BSD-3 license header from a neighbouring file with the current year; never change the year in existing files. Test files directly in `dpnp/tests/` and under `dpnp/tests/third_party/` have no header.
- Don't edit generated files: `dpnp/_version.py`, `dpnp/include/dpnp/tensor/_usmarray*.h`, `dpnp/resources/cmake/`, `doc/reference/generated/`, `doc/reference/*.inc`.

## Verification Before You Push

```bash
pre-commit run --all-files # the required `Check` job; pylint must be on PATH
python scripts/build_locally.py # in-place build + editable install
python -m pytest -ra dpnp/tests/test_<area>.py
```

The build needs the packages in `environments/build_with_oneapi.yml`, `environments/oneapi_pkgs.yml` and `environments/dpctl_pkg.yml`. The `no-commit-to-branch` hook fails on `master` and `maintenance/*`, so work on a branch. Full-suite commands and test env vars are in `dpnp/tests/AGENTS.md`; build options in `dpnp/backend/AGENTS.md`.

### Changelog format

Append to the end of the matching `### Added` / `Changed` / `Deprecated` / `Removed` / `Fixed` / `Security` list in the top `## [X.Y.Z] - YYYY-MM-DD` section. Past tense, code in backticks, no trailing period:

```
* Added support for the `"nuc"` (nuclear) norm in `dpnp.linalg.cond` [#3084](https://github.com/IntelPython/dpnp/pull/3084)
```

### Where the checks live

| Check | Workflow | Required on master |
| --- | --- | --- |
| pre-commit (`Check`) | `.github/workflows/pre-commit.yml` | yes |
| conda build and tests, Linux and Windows, py3.12-3.14t (`Test (...)`) | `conda-package.yml` | yes |
| Internal Jenkins (`jenkins-test-dpnp-3.13-linux-dev`, `...-3.14-...`) | external | yes |
| Array API conformance of `dpnp`, not yet `dpnp.tensor` (array-api-tests; never fails, results posted as a PR comment) | `conda-package.yml` | no |
| pybind11 examples | `conda-package.yml` | no |
| conda-forge build and tests | `conda-package-cf.yml` | no |
| oneMath build, open-source LLVM SYCL build | `check-onemath.yaml`, `os-llvm-sycl-build.yml` | no |
| Sphinx docs (`make spelling`, `make html`) | `build-sphinx.yml` | no |
| Coverage (Coveralls), zizmor | `generate_coverage.yaml`, `zizmor.yml` | no |

GitHub-hosted runners have no GPU, so tests there run on a CPU device: a green run says nothing about GPUs or devices without fp64.

## Reviewing

Report source-confirmed problems with NumPy compatibility, Array API conformance, compute-follows-data, devices without fp64, SYCL event ordering and lifetimes, error handling and test coverage. Don't report what pre-commit enforces (formatting, import order, spelling), and don't report that `@pytest.mark.slow` tests are skipped in CI: that is by design.

## Directory Guides

Read the `AGENTS.md` nearest the files you change. When editing these files, every command, path and name must match the repository; delete what can't be verified rather than soften it.

- [dpnp/AGENTS.md](dpnp/AGENTS.md): Python layer, adding a function, docstrings
- [dpnp/tests/AGENTS.md](dpnp/tests/AGENTS.md): test helpers, fp64, CuPy tests, running the suite
- [dpnp/backend/AGENTS.md](dpnp/backend/AGENTS.md): pybind11 SYCL extensions, C++ rules, build options
- [dpnp/tensor/AGENTS.md](dpnp/tensor/AGENTS.md): `dpnp.tensor` and libtensor
- [doc/AGENTS.md](doc/AGENTS.md): Sphinx docs
- [benchmarks/AGENTS.md](benchmarks/AGENTS.md): ASV benchmarks
- [.github/AGENTS.md](.github/AGENTS.md): workflow hardening rules

## Further Reading

- `doc/quick_start_guide.rst`: build from source, CUDA/HIP targets
- `.github/pull_request_template.md`: the PR checklist
19 changes: 19 additions & 0 deletions benchmarks/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# AGENTS.md - Benchmarks (benchmarks/)

ASV suite that times dpnp only; `README.md` has the full setup.

## Rules for Changes
- NumPy is not timed: `executor` stays the single value `'dpnp'`. Don't edit the leftover `numpy` branches in `setup`: ASV versions a benchmark by the source of `setup` and `time_*`, so any edit makes its results incomparable with the recorded history.
- Keep `params` static and their values plain strings, numbers or tuples; skip unsupported devices in `setup` with `SkipNotImplemented`.
- Timed bodies block on the result (`self.sync(...)` or `dpnp.synchronize_array_data`); otherwise they time only the submission.
- Parameter axes shared by two or more modules live in `benchmarks/benchmarks/_utils.py`.
- A new non-ASV dependency goes in both `requirements.txt` (what CI installs) and the `benchmark` extra in `pyproject.toml`; ASV itself is only in the extra.

## Verification
From `benchmarks/`, against an environment that already has dpnp:
```bash
asv machine --yes
asv check --python=same
asv run --python=same --launch-method spawn --quick --bench bench_elementwise
```
Always pass `--launch-method spawn`: the SYCL runtime is not fork-safe. Don't `pip install ".[benchmark]"`; it rebuilds dpnp.
25 changes: 25 additions & 0 deletions doc/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# AGENTS.md - Documentation (doc/)

## Layout
- `conf.py`: Sphinx config (numpydoc, autosummary, furo). It registers the dpnp `Limitations` docstring section, documents the `DPNP*Func` wrapper objects as functions, and regenerates `reference/comparison_table.rst.inc` from `comparison_generator.py` on every build
- `reference/<category>.rst`: `.. autosummary::` lists under `.. currentmodule::` (`dpnp` or the subpackage); stubs are generated into `reference/generated/`
- `known_words.txt`: spelling allow-list
- `quick_start_guide.rst`: install and build-from-source instructions

## Rules for Changes
- A new public function goes in the autosummary block of the matching `reference/<category>.rst`; ufuncs also go in `reference/ufunc.rst`, `ndarray` members in `reference/ndarray.rst`. These pages already list many NumPy names that dpnp lacks, so check before adding a duplicate.
- A new namespace needs its own `reference/<name>.rst` and a toctree entry: in `reference/routines.rst` for NumPy namespaces (model: `reference/fft.rst`), in `reference/scipy.rst` for SciPy ones (model: `reference/scipy_linalg.rst`).
- A page kept out of every toctree on purpose, such as a placeholder, is marked `:orphan:`, or Sphinx warns about it.
- Move a deprecated function from its reference page to `reference/_deprecated.rst` (an orphan page that keeps its stub) and remove cross-links to it.
- A new wrapper base class (not a subclass of the existing `DPNP*Func` classes) must be added to `conf.py` and `_can_document_member`, or autodoc skips its instances.
- Update `quick_start_guide.rst` (and `README.md`) when build options or supported Python versions change, and `reference/array_api.rst` when the Array API version or the conforming namespaces change.
- Add new legitimate words to `known_words.txt`.
- Warnings don't fail the build (no `-W`), so check the log and the rendered page yourself.

## Verification
```bash
cd doc
make spelling
make html
```
This needs an importable dpnp build and the packages in `environments/building_docs.yml` (including CuPy, for the comparison table). CI (`build-sphinx.yml`) publishes builds of PRs from IntelPython/dpnp branches at `https://intelpython.github.io/dpnp/pull/<N>/index.html`; fork PRs get an artifact instead.
35 changes: 35 additions & 0 deletions dpnp/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# AGENTS.md - Python layer (dpnp/)

## Layout
- `dpnp_iface_<category>.py`: public functions, grouped like NumPy's routines reference
- `dpnp_array.py`: `dpnp.ndarray`, a wrapper over `dpnp.tensor.usm_ndarray`; `get_array()` unwraps, `_create_from_usm_ndarray()` wraps
- `dpnp_algo/dpnp_elementwise_common.py`: ufunc wrapper classes (`DPNPUnaryFunc`, `DPNPBinaryFunc`, two-output and specialised variants)
- `dpnp_utils/`: shared helpers (`get_usm_allocations`, `map_dtype_to_device`, reduction wrappers)
- `linalg/`, `fft/`, `random/`, `scipy/`, `lib/`: subpackages, each exporting through its own `__init__.py`
- Legacy, don't extend: the Cython `dpnp_algo` path and `dpnp_backend_c` (still used by `dpnp.partition` and `dpnp.random`), and the NumPy fallback `call_origin`. New kernels go into pybind11 extensions or `dpnp.tensor`.

## Adding a Public Function
1. Implement it in the matching `dpnp_iface_*.py` or subpackage module. Iface modules have no `__all__`.
2. Export it in `dpnp/__init__.py` in two places, both under its NumPy routine category: the `from .dpnp_iface_xxx import (...)` block and the `__all__ += [...]` block. Functions in `linalg`, `fft`, `random`, `scipy.*` and `lib` are exported from that subpackage's `__init__.py` instead (import block and `__all__`). A new subpackage also goes in `packages=` in `setup.py`.
3. Add it to `doc/reference/` (see `doc/AGENTS.md`), tests (see `dpnp/tests/AGENTS.md`) and `CHANGELOG.md`.

## Patterns
- Ufuncs are module-level wrapper instances, e.g. `cbrt = DPNPUnaryFunc("cbrt", ti._cbrt_result_type, ti._cbrt, _CBRT_DOCSTRING, mkl_fn_to_call=..., mkl_impl_fn=...)`. The first argument is the public name, with no leading underscore. Kernels come from `dpnp.tensor._tensor_elementwise_impl` or `dpnp.backend.extensions.ufunc._ufunc_impl`; the oneMKL VM path runs first when `mkl_fn_to_call` is given and supported.
- Other functions: `dpnp.check_supported_arrays_type(...)`, unwrap with `dpnp.get_usm_ndarray`, call `dpnp.tensor` (`import dpnp.tensor as dpt`) or an extension, and wrap with `dpnp_array._create_from_usm_ndarray`. Pass `out` straight to the `dpnp.tensor` or extension call when it accepts one; otherwise `dpnp.get_result_array(res, out)` copies the result into it.
- Get the result queue and USM type with `get_usm_allocations([...])` (raises `ValueError` on mixed queues) or `dpt.get_execution_queue(...)` (returns `None`; raise `ExecutionPlacementError`). Creation functions take `device`, `usm_type` and `sycl_queue` (normalised by `dpnp.get_normalized_queue_device`); `*_like` functions inherit them from the input.
- Extension calls are ordered with `_manager = dpctl.utils.SequentialOrderManager[exec_q]`: pass `depends=_manager.submitted_events`, then `_manager.add_event_pair(ht_ev, ev)`. Don't add blocking waits.
- Use `dpnp.default_float_type(...)` or `map_dtype_to_device(...)` for the default float type.
- Deprecated ufuncs use `DPNPDeprecatedUnaryFunc(..., deprecated_msg=...)`.
- Unsupported NumPy keywords raise `NotImplementedError` (`dpnp.check_limitations`) and are listed under `Limitations` in the docstring.
- The Array API spec version is set in three places that change together: `__array_api_version__` in `dpnp/tensor/_array_api.py`, `ARRAY_API_TESTS_VERSION` in `.github/workflows/conda-package.yml` and `doc/reference/array_api.rst` (plus the version test in `dpnp/tests/test_ndarray.py`). Inspection utilities come from `__array_namespace_info__` in `dpnp_array_api_info.py`.
- `dpnp.tensor` replaces `dpctl.tensor`, which no longer exists here; `SyclQueue`, `SyclDevice`, `dpctl.memory` and `SequentialOrderManager` still come from `dpctl`.

## Docstrings
numpydoc, rendered by `doc/conf.py`. `doc/docstring_template.py` shows the layout; model a new docstring on an existing function.
- Sections in the order `doc/conf.py` renders them: summary, ``For full documentation refer to :obj:`numpy.<name>`.``, Parameters, Returns, Raises, Warns, Warnings, Limitations, See Also, Notes, References, Examples.
- Array parameters are typed `{dpnp.ndarray, usm_ndarray}`, optional ones `{None, dpnp.ndarray, usm_ndarray}, optional`; the return is `out : dpnp.ndarray`.
- Every keyword with a default ends with a blank line and ``Default: ``<value>``.``
- Literals in double backticks (``` ``None`` ```), argument names in single backticks; link with `:obj:`.
- Copy the `device` / `usm_type` / `sycl_queue` parameter text from an existing creation function such as `dpnp.empty`.
- Examples start with `>>> import dpnp as np`. A docstring that contains `\` is raw (`r"""`).
- An `ndarray` method that wraps a function has a one-line summary, a blank line, then ``Refer to :obj:`dpnp.<name>` for full documentation.``
Loading
Loading