Refresh Python and TensorFlow compatibility and documentation

.github/workflows/test.yml

@@ -5,7 +5,7 @@ name: Python package
 
 on:
   pull_request:
-    branches: [ "master" ]
+    branches: [ "master", "feature/build-speedup" ]
 
 jobs:
   build:
@@ -14,36 +14,48 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.9", "3.10"] # ver. 3.11 is not supported by TP
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+        tensorflow-version: ["2.20.0", "2.21.0"]
+        include:
+          - python-version: "3.9"
+            tensorflow-version: "2.20.0"
+
+    env:
+      CUDA_VISIBLE_DEVICES: "-1"
+      TF_USE_LEGACY_KERAS: "1"
+      CMAKE_BUILD_PARALLEL_LEVEL: "2"
+      OMP_NUM_THREADS: "2"
+      TF_NUM_INTRAOP_THREADS: "2"
+      TF_NUM_INTEROP_THREADS: "2"
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v3
+      uses: actions/setup-python@v6
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install general dependencies
       run: |
         python -m pip install --upgrade pip
         python -m pip install pytest
-    - name: Checkout tensorpotential
-      uses: actions/checkout@v2
+    - name: Checkout current tensorpotential with PACE compatibility
+      uses: actions/checkout@v4
       with:
-        repository: ICAMS/TensorPotential
+        repository: ICAMS/grace-tensorpotential
+        # Includes the NumPy scalar fix and TensorFlow 2.21 bound from PR #43.
+        ref: 9ccb9090f53292a836cbbc3de4cadcae54c7ce1a
         path: tensorpotential
     - name: Install tensorpotential
       run: |
         cd tensorpotential
-        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
-        #python setup.py install
-        pip install . 
+        python -m pip install "tensorflow==${{ matrix.tensorflow-version }}" "tf-keras~=${{ matrix.tensorflow-version }}" .
     - name: Install python-ace
       run: |    
-        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
-        #python setup.py install
-        pip install . 
+        python -m pip install .
     - name: Test with python-ace with tensorpotential
       run: |
+        python -m pip check
+        python -c 'import pyace, tensorpotential, tensorflow as tf, numpy as np; print("Versions:", pyace.__version__, tensorpotential.__version__, tf.__version__, np.__version__)'
         python -m pytest tests/ --runtensorpot
     - name: Integration test of CLI
       run: |

.readthedocs.yaml

@@ -12,6 +12,7 @@ build:
 
 mkdocs:
   configuration: mkdocs.yml
+  fail_on_warning: true
 
 # Optionally declare the Python requirements required to build your docs
 python:

docs/pacemaker/faq.md

@@ -6,7 +6,7 @@ If `batch_size_reduction=True` (default option), then automatic batch size reduc
 
 ## My fit on GPU crushes with OOM error, what to do ?
 This means that you are trying to fit at once too much data into the GPU memory. The amount of data processed by GPU
-at once is controlled by [batch_size](#backend-specification) parameter, try to reduce it or set `batch_size_reduction=True`. An optimal value for
+at once is controlled by [batch_size](inputfile.md#backend_specification) parameter, try to reduce it or set `batch_size_reduction=True`. An optimal value for
 this parameter is totally empirical as it depends on data, potential configuration and GPU itself.
 
 
@@ -47,7 +47,7 @@ If you want to continue fit without changing the basis size, you can do the foll
 
   doing this will override specifications in the `input.yaml`.
 
-If you want to extend the basis (aka do the [ladder scheme fitting](#inputfile.md#ladder_hiererchical_basis_extension)):
+If you want to extend the basis (aka do the [ladder scheme fitting](inputfile.md#ladder_hiererchical_basis_extension)):
 
 - Specify your potential as initial potential
 

docs/pacemaker/inputfile.md

@@ -36,7 +36,7 @@ data:
 ```
 
 Dataset could be saved into file as a pickled `pandas` dataframe with special names
-for [columns](#fitting-dataset-preparation).
+for [columns](quickstart.md#optional_manual_fitting_dataset_preparation).
 
 ```YAML
 data:

docs/pacemaker/install.md

@@ -1,122 +1,150 @@
 # Installation
 
-## (Optional) Creating a Separate Environment
+`pacemaker` is installed by the `python-ace` package. TensorFlow fitting uses
+`tensorpotential.compat.pace` from the current
+[GRACE TensorPotential repository](https://github.com/ICAMS/grace-tensorpotential).
+The old `ICAMS/TensorPotential` repository requires Python <3.11 and should not
+be used for a new installation.
 
-It is a common practice to create a separate Python environment to avoid dependency conflicts. You can create a new environment named `ace` with the minimal required packages using the following command:
+## Create an environment
 
+Python 3.12 is a suitable starting point for a new environment:
 
-### Using `mamba` (Recommended)
-
-[Mamba](https://mamba.readthedocs.io/en/latest/index.html) is a fast, robust, and cross-platform package manager. Check the [Mamba installation guide](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html) for details.
+```bash
+mamba create -n ace python=3.12 pip
+mamba activate ace
+python -m pip install --upgrade pip
+```
 
-Installation steps:
-* Download the Miniforge3 distribution from [here](https://github.com/conda-forge/miniforge?tab=readme-ov-file#miniforge3).
-
-* Run the downloaded file:
-   ```bash
-   sh Miniforge3-Linux-x86_64.sh
-   ```
-   and follow the installation instructions.
-
-* Create the `mamba` environment:
-   ```bash
-   mamba create -n ace python=3.9
-   ```
+`conda create` and `conda activate` can be used in place of the corresponding
+`mamba` commands. Use the same environment to build and run both packages.
+A C++ compiler is needed when building `python-ace` from source; its Python
+build dependencies, including CMake and Ninja, are installed automatically.
 
-* Activate the environment with either: `source activate ace` or `mamba activate ace`. To deactivate the environment, use the command `mamba deactivate`
+## Install the fitting backend
 
-### Using `conda`
-> **Note:** Due to recent changes in Anaconda's licensing, please ensure you are permitted to use it. The `mamba` package manager might be a preferable alternative.
+Install the current backend from source:
 
-To create the environment with `conda`:
 ```bash
-conda create -n ace python=3.9
+git clone https://github.com/ICAMS/grace-tensorpotential.git
+cd grace-tensorpotential
+python -m pip install .
+cd ..
 ```
 
-## Installation of `tensorpotential`
+This backend must include the PACE metrics scalar-conversion fix for NumPy
+2.4 and newer. The released TensorPotential 0.6.2.1 also caps TensorFlow at
+2.20; TensorFlow 2.21 needs the updated backend dependency bound. Updating
+`python-ace` alone does not update either of these backend requirements.
 
-`tensorpotential` allows for the GPU accelerated optimization of the ACE potential using [TensorFlow](https://www.tensorflow.org/).
-However, it is recommended to use it even if you don't have a GPU available.
+The updated backend permits TensorFlow releases below 2.22. To select the
+2.21 series explicitly, install matching TensorFlow and legacy Keras versions:
 
+```bash
+python -m pip install 'tensorflow~=2.21.0' 'tf-keras~=2.21.0'
+```
 
-Install it using the following commands:
+On Linux the backend dependencies include `tensorflow[and-cuda]`. GPU execution
+also requires a compatible NVIDIA driver. CPU fitting is available with
+`CUDA_VISIBLE_DEVICES=-1`. TensorFlow 2.21 publishes wheels for Python
+3.10–3.13; Python 3.14 is not covered by this installation recipe.
 
-* Install Tensorflow (newer version should be also compatible)
-```
-pip install tensorflow==2.8.0 
-```
-or to have CUDA support in latest versions of TensorFlow
-```
-pip install tensorflow[and-cuda] 
-```
+## Install pacemaker
 
-* Download the `tensorpotential` from [this repository](https://github.com/ICAMS/TensorPotential).
-Clone with
-```
-git clone https://github.com/ICAMS/TensorPotential.git
-cd TensorPotential
-```
-or download
-```
-wget https://github.com/ICAMS/TensorPotential/archive/refs/heads/main.zip
-unzip main.zip
-cd TensorPotential-main
-```
+For the modern source branch:
 
-* Run installation script
-```
-pip install --upgrade .
+```bash
+git clone --branch feature/build-speedup https://github.com/ICAMS/python-ace.git
+cd python-ace
+python -m pip install .
 ```
-or (for more installation details)
+
+When working in an existing checkout, run `python -m pip install .` there.
+This build includes the maxvol extension used by the active-learning tools;
+a separate `maxvolpy` installation is unnecessary.
+
+Check the installed versions and command-line tools:
+
+```bash
+python -m pip check
+python -c 'import pyace, tensorpotential, tensorflow as tf; print(pyace.__version__, tensorpotential.__version__, tf.__version__)'
+pacemaker --help
+pace_info --help
 ```
-python setup.py install
+
+To run the test suite from the `python-ace` checkout:
+
+```bash
+python -m pip install pytest
+CUDA_VISIBLE_DEVICES=-1 python -m pytest tests/ --runtensorpot
 ```
 
-## Installation of `pacemaker` and `pyace`
+The test workflow is configured for Python 3.10–3.13 with TensorFlow 2.20 and
+2.21 on Linux CPUs, plus Python 3.9 with TensorFlow 2.20. GPU execution needs
+separate validation on the target hardware.
 
-The `pyace` (aka `python-ace`) package is located in [this repository](https://github.com/ICAMS/python-ace).
-It contains the `pacemaker` tools and other Python wrappers and utilities.
+### Tested development environment
 
-To install `pyace`:
+The source refresh was checked on 29 October 2026 with Python 3.12.15,
+TensorFlow 2.21.0, tf-keras 2.21.0, NumPy 2.5.4, and pandas 2.3.3:
 
-* Download `pyace` from [this repository](https://github.com/ICAMS/python-ace). Clone with
-```
-git clone https://github.com/ICAMS/python-ace.git
-cd python-ace
-```
-or download 
-```
-wget https://github.com/ICAMS/python-ace/archive/refs/heads/master.zip
-cd python-ace-master
-```
+| Execution target | Validation |
+| --- | --- |
+| Linux CPU | 294 tests and both Cu–I and Ethanol CLI workflows passed |
+| NVIDIA H200 NVL, driver 595.58.03 | All 9 fitting tests passed; 185 fitting calls verified on GPU |
+| NVIDIA A100 80 GB PCIe, driver 575.57.08 | All 9 fitting tests passed; 185 fitting calls verified on GPU |
 
-* Run installation script
-```
-pip install --upgrade .
-```
-or (for more installation details)
-```
-python setup.py install
-```
+The GPU checks verify device placement of energies, forces, and coefficient
+gradients as well as the fitting-test results. They cover short fits, rather
+than long-running memory stability or performance. These results require the
+source fixes described above and do not describe an unmodified TensorPotential
+0.6.2.1 installation.
 
-Now, `pacemaker` and other tools (`pace_yaml2yace`, `pace_info`, `pace_activeset`) should be available from the terminal, if corresponding conda environment is loaded.
+## Troubleshooting
 
-## Known installation issues
+### GPU validation
 
-### Segmentation fault
-If you see `Segmentation fault` error message, then check that you are using correct version of Python from corresponding conda environment,
-i.e. check that `which python` points to right location inside conda environment.
+To check fitting on a selected GPU from the source checkout:
 
-### TypeError: Descriptors cannot not be created directly
-If you see this error message
-```
-TypeError: Descriptors cannot not be created directly.
-If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
-If you cannot immediately regenerate your protos, some other possible workarounds are:
- 1. Downgrade the protobuf package to 3.20.x or lower.
- 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).```
-```
-then try to downgrade `protobuf` package, i.e.
+```bash
+CUDA_VISIBLE_DEVICES=0 TF_FORCE_GPU_ALLOW_GROWTH=true python tools/validate_gpu.py
 ```
-pip install protobuf==3.20.*
+
+This runs the TensorFlow fitting tests and checks that energies, forces, and
+coefficient gradients reside on the GPU. It fails if no GPU is detected or if
+those results fall back to the CPU. Select one GPU; pacemaker does not fit on
+multiple GPUs in parallel. Memory growth keeps the test from reserving all
+available device memory.
+
+If TensorFlow reports `Cannot dlopen some GPU libraries`, an existing system
+CUDA installation may take precedence over the libraries installed by pip.
+For the current shell, put the environment's CUDA library directories first:
+
+```bash
+pace_cuda_libraries=$(python - <<'PY'
+from pathlib import Path
+import sysconfig
+print(':'.join(str(path) for path in sorted(
+    (Path(sysconfig.get_paths()['purelib']) / 'nvidia').glob('*/lib')
+)))
+PY
+)
+export LD_LIBRARY_PATH="$pace_cuda_libraries${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
 ```
+
+Then rerun the GPU check. This changes the shell environment and leaves the
+system CUDA installation untouched.
+
+### Python and dependency errors
+
+If fitting raises `TypeError: only 0-dimensional arrays can be converted to
+Python scalars`, check that both `python-ace` and the backend include the
+NumPy scalar-conversion fix, then rebuild the installed packages.
+
+For an import error or segmentation fault, check `which python` and rebuild
+the extensions using the interpreter from the active environment. Avoid mixing
+extensions built for different Python versions.
+
+For a protobuf descriptor error, check `python -m pip check` and reinstall a
+consistent TensorFlow environment. The old TensorFlow 2.5/2.8 and protobuf
+3.20 instructions do not describe the current dependency set.