Skip to content

feat(fetch-content): record a profiler marker for each fetched file - #1083

Draft
jcristau wants to merge 2 commits into
taskcluster:mainfrom
jcristau:fetch-content-markers
Draft

jcristau wants to merge 2 commits into
taskcluster:mainfrom
jcristau:fetch-content-markers

feat(fetch-content): profile all fetch commands and test marker handling

ead6459
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / pre-commit-v1 succeeded Oct 30, 2026 in 32s

FirefoxCI (pull_request)

Run pre-commit checks

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2026-10-30T08:35:25.522Z
Resolved: 2026-10-30T08:35:57.439Z
Task Execution Time: 31 seconds, 917 milliseconds
Task Status: completed
Reason Resolved: completed
TaskId: X_l3XTWVTISXlnYfufk90w
RunId: 0

Artifacts

- public/logs/live_backing.log (14.9 KB)
- public/logs/live.log


[taskcluster 2026-10-30T08:35:25.960Z] Worker Type (mozilla-t/pre-commit) settings:
[taskcluster 2026-10-30T08:35:25.960Z]   {
[taskcluster 2026-10-30T08:35:25.960Z]     "generic-worker": {
[taskcluster 2026-10-30T08:35:25.960Z]       "config": {
[taskcluster 2026-10-30T08:35:25.960Z]         "capacity": 1,
[taskcluster 2026-10-30T08:35:25.960Z]         "headlessTasks": true
[taskcluster 2026-10-30T08:35:25.960Z]       },
[taskcluster 2026-10-30T08:35:25.960Z]       "engine": "multiuser",
[taskcluster 2026-10-30T08:35:25.960Z]       "go-arch": "amd64",
[taskcluster 2026-10-30T08:35:25.960Z]       "go-os": "linux",
[taskcluster 2026-10-30T08:35:25.960Z]       "go-version": "go1.27.3",
[taskcluster 2026-10-30T08:35:25.960Z]       "release": "https://github.com/taskcluster/taskcluster/releases/tag/v113.2.0",
[taskcluster 2026-10-30T08:35:25.960Z]       "revision": "efb71e91e3502152e6eb1a82bbf98c3d4d99f75e",
[taskcluster 2026-10-30T08:35:25.960Z]       "source": "https://github.com/taskcluster/taskcluster/commits/efb71e91e3502152e6eb1a82bbf98c3d4d99f75e",
[taskcluster 2026-10-30T08:35:25.960Z]       "version": "113.2.0"
[taskcluster 2026-10-30T08:35:25.960Z]     },
[taskcluster 2026-10-30T08:35:25.960Z]     "image": "projects/taskcluster-imaging/global/images/gw-fxci-gcp-l1-2404-amd64-headless-googlecompute-2026-10-27",
[taskcluster 2026-10-30T08:35:25.960Z]     "instance-id": "1405155127220348989",
[taskcluster 2026-10-30T08:35:25.960Z]     "instance-type": "projects/887720501152/machineTypes/c3d-standard-4",
[taskcluster 2026-10-30T08:35:25.960Z]     "local-ipv4": "10.128.0.19",
[taskcluster 2026-10-30T08:35:25.960Z]     "project-id": "fxci-production-level1-workers",
[taskcluster 2026-10-30T08:35:25.960Z]     "public-hostname": "mozilla-t-pre-commit-bkm-aayis6sx02fczbfkbg.c.fxci-production-level1-workers.internal",
[taskcluster 2026-10-30T08:35:25.960Z]     "public-ipv4": "34.42.63.228",
[taskcluster 2026-10-30T08:35:25.960Z]     "region": "us-central1",
[taskcluster 2026-10-30T08:35:25.960Z]     "zone": "us-central1-f"
[taskcluster 2026-10-30T08:35:25.960Z]   }
[taskcluster 2026-10-30T08:35:25.960Z] Task ID: X_l3XTWVTISXlnYfufk90w
[taskcluster 2026-10-30T08:35:25.960Z] === Task Starting ===
[taskcluster 2026-10-30T08:35:27.069Z] [mounts] No existing writable directory cache 'lint-pre-commit-uv-v1' - creating /home/task_X_l3XTWVTISXl_0/cache0
[taskcluster 2026-10-30T08:35:27.069Z] [mounts] Creating directory /home/task_X_l3XTWVTISXl_0/cache0
[taskcluster 2026-10-30T08:35:27.078Z] [mounts] Successfully mounted writable directory cache '/home/task_X_l3XTWVTISXl_0/cache0'
[taskcluster 2026-10-30T08:35:27.078Z] [mounts] No existing writable directory cache 'lint-pre-commit-repos-v1' - creating /home/task_X_l3XTWVTISXl_0/cache1
[taskcluster 2026-10-30T08:35:27.078Z] [mounts] Creating directory /home/task_X_l3XTWVTISXl_0/cache1
[taskcluster 2026-10-30T08:35:27.087Z] [mounts] Successfully mounted writable directory cache '/home/task_X_l3XTWVTISXl_0/cache1'
[taskcluster 2026-10-30T08:35:27.087Z] [mounts] No existing writable directory cache 'lint-pre-commit-checkout-taskcluster-taskgraph-v1' - creating /home/task_X_l3XTWVTISXl_0/cache2
[taskcluster 2026-10-30T08:35:27.087Z] [mounts] Creating directory /home/task_X_l3XTWVTISXl_0/cache2
[taskcluster 2026-10-30T08:35:27.095Z] [mounts] Successfully mounted writable directory cache '/home/task_X_l3XTWVTISXl_0/cache2'
[taskcluster 2026-10-30T08:35:27.095Z] [d2g] Loading docker image
[taskcluster 2026-10-30T08:35:32.499Z] [d2g] Loaded docker image "docker.io/mozillareleases/taskgraph:run-task-v24.3.0@sha256:1efe26cb74bd09fa0f2a051b4559ab7cde4270d92f4e7c7dce7242468a21e6af"
[taskcluster 2026-10-30T08:35:32.502Z] Executing command 0: docker run -t --name taskcontainer_VgYveB7BTh6Fjmw5NqVUkg --memory-swap -1 --pids-limit -1 --pull=never --log-driver=none '--add-host=localhost.localdomain:127.0.0.1' -v '/home/task_X_l3XTWVTISXl_0/cache0:/builds/worker/.cache/uv' -v '/home/task_X_l3XTWVTISXl_0/cache1:/builds/worker/.cache/pre-commit' -v /home/task_X_l3XTWVTISXl_0/cache2:/builds/worker/checkouts --network gw-task-X_l3XTWVTISX-0 '--add-host=taskcluster:172.18.0.1' --env-file 'env.list' -- 'mozillareleases/taskgraph:run-task-v24.3.0@sha256:1efe26cb74bd09fa0f2a051b4559ab7cde4270d92f4e7c7dce7242468a21e6af' run-task --src-checkout=/builds/worker/checkouts/src --task-cwd=/builds/worker/checkouts/src -- bash -c 'uv tool install pre-commit && /builds/worker/.local/bin/pre-commit run --all-files'
[setup 2026-10-30T08:35:35.088+00:00] run-task started in /
[setup 2026-10-30T08:35:35.088+00:00] Invoked by command: --src-checkout=/builds/worker/checkouts/src --task-cwd=/builds/worker/checkouts/src -- bash -c uv tool install pre-commit && /builds/worker/.local/bin/pre-commit run --all-files
[setup 2026-10-30T08:35:35.088+00:00] Python version: 3.13.5
[setup 2026-10-30T08:35:35.088+00:00] Subprocess python version: 
cpython-3.13.5-linux-x86_64-gnu    /usr/bin/python3.13
cpython-3.13.5-linux-x86_64-gnu    /usr/bin/python3 -> python3.13
[cache 2026-10-30T08:35:35.171+00:00] cache /builds/worker/.cache/uv is empty; writing requirements: gid=1000 uid=1000 version=1
[cache 2026-10-30T08:35:35.172+00:00] cache /builds/worker/.cache/pre-commit is empty; writing requirements: gid=1000 uid=1000 version=1
[cache 2026-10-30T08:35:35.172+00:00] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[setup 2026-10-30T08:35:35.172+00:00] running as worker:worker
[vcs 2026-10-30T08:35:35.172+00:00] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/src']
[vcs 2026-10-30T08:35:35.174+00:00] executing ['git', 'config', '--global', 'checkout.workers', '0']
[vcs 2026-10-30T08:35:35.176+00:00] executing ['git', 'clone', 'https://github.com/jcristau/taskgraph', '/builds/worker/checkouts/src']
[vcs 2026-10-30T08:35:35.177+00:00] Cloning into '/builds/worker/checkouts/src'...
[vcs 2026-10-30T08:35:36.490+00:00] executing ['git', 'fetch', 'origin', 'fetch-content-markers']
[vcs 2026-10-30T08:35:36.671+00:00] From https://github.com/jcristau/taskgraph
[vcs 2026-10-30T08:35:36.671+00:00]  * branch            fetch-content-markers -> FETCH_HEAD
[vcs 2026-10-30T08:35:36.677+00:00] executing ['git', 'checkout', '-f', '-B', 'fetch-content-markers', 'ead6459aea815a3f54c1342c42a03cfce9907da7']
[vcs 2026-10-30T08:35:36.699+00:00] Switched to a new branch 'fetch-content-markers'
[vcs 2026-10-30T08:35:36.700+00:00] cleaning git checkout...
[vcs 2026-10-30T08:35:36.700+00:00] executing ['git', 'clean', '-nxdff']
[vcs 2026-10-30T08:35:36.703+00:00] removing []
[vcs 2026-10-30T08:35:36.703+00:00] successfully cleaned git checkout!
[vcs 2026-10-30T08:35:36.705+00:00] TinderboxPrint:<a href='https://github.com/jcristau/taskgraph/commit/ead6459aea815a3f54c1342c42a03cfce9907da7' title='Built from taskgraph commit ead6459aea815a3f54c1342c42a03cfce9907da7'>ead6459aea815a3f54c1342c42a03cfce9907da7</a>
PERFHERDER_DATA: {"framework": {"name": "vcs"}, "suites": [{"extraOptions": ["projects/887720501152/machineTypes/c3d-standard-4"], "lowerIsBetter": true, "name": "clone", "shouldAlert": false, "subtests": [], "value": 1.3139245510101318}, {"extraOptions": ["projects/887720501152/machineTypes/c3d-standard-4"], "lowerIsBetter": true, "name": "pull", "shouldAlert": false, "subtests": [], "value": 0.18656039237976074}, {"extraOptions": ["projects/887720501152/machineTypes/c3d-standard-4"], "lowerIsBetter": true, "name": "update", "shouldAlert": false, "subtests": [], "value": 0.02378559112548828}, {"extraOptions": ["projects/887720501152/machineTypes/c3d-standard-4"], "lowerIsBetter": true, "name": "purge", "shouldAlert": false, "subtests": [], "value": 0.0031664371490478516}, {"extraOptions": ["projects/887720501152/machineTypes/c3d-standard-4"], "lowerIsBetter": true, "name": "overall", "shouldAlert": false, "subtests": [], "value": 1.5275254249572754}, {"extraOptions": ["projects/887720501152/machineTypes/c3d-standard-4"], "lowerIsBetter": true, "name": "overall_clone", "shouldAlert": false, "subtests": [], "value": 1.5275254249572754}, {"extraOptions": ["projects/887720501152/machineTypes/c3d-standard-4"], "lowerIsBetter": true, "name": "overall_clone_fullcheckout", "shouldAlert": false, "subtests": [], "value": 1.5275254249572754}]}
[setup 2026-10-30T08:35:36.705+00:00] UV_CACHE_DIR is /builds/worker/.cache/uv
[task 2026-10-30T08:35:36.706+00:00] executing ['bash', '-c', 'uv tool install pre-commit && /builds/worker/.local/bin/pre-commit run --all-files']
[task 2026-10-30T08:35:36.920+00:00] Resolved 11 packages in 169ms
[task 2026-10-30T08:35:36.981+00:00] Downloading virtualenv (5.3MiB)
[task 2026-10-30T08:35:37.096+00:00]  Downloading virtualenv
[task 2026-10-30T08:35:37.096+00:00] Prepared 11 packages in 174ms
[task 2026-10-30T08:35:37.098+00:00] warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance.
[task 2026-10-30T08:35:37.098+00:00]          If the cache and target directories are on different filesystems, hardlinking may not be supported.
[task 2026-10-30T08:35:37.098+00:00]          If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning.
[task 2026-10-30T08:35:37.114+00:00] Installed 11 packages in 17ms
[task 2026-10-30T08:35:37.114+00:00]  + cfgv==3.5.0
[task 2026-10-30T08:35:37.114+00:00]  + distlib==0.4.4
[task 2026-10-30T08:35:37.114+00:00]  + filelock==4.2.4
[task 2026-10-30T08:35:37.114+00:00]  + identify==2.6.20
[task 2026-10-30T08:35:37.114+00:00]  + nodeenv==1.11.1
[task 2026-10-30T08:35:37.114+00:00]  + packaging==26.3
[task 2026-10-30T08:35:37.115+00:00]  + platformdirs==4.14.0
[task 2026-10-30T08:35:37.115+00:00]  + pre-commit==4.7.0
[task 2026-10-30T08:35:37.115+00:00]  + python-discovery==1.6.7
[task 2026-10-30T08:35:37.115+00:00]  + pyyaml==6.0.3
[task 2026-10-30T08:35:37.115+00:00]  + virtualenv==21.15.1
[task 2026-10-30T08:35:37.115+00:00] Installed 1 executable: pre-commit
[task 2026-10-30T08:35:37.116+00:00] warning: `/builds/worker/.local/bin` is not on your PATH. To use installed tools, run `export PATH="/builds/worker/.local/bin:$PATH"` or `uv tool update-shell`.
[task 2026-10-30T08:35:37.302+00:00] [INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[task 2026-10-30T08:35:37.634+00:00] [INFO] Initializing environment for https://github.com/astral-sh/ruff-pre-commit.
[task 2026-10-30T08:35:37.982+00:00] [INFO] Initializing environment for https://github.com/astral-sh/uv-pre-commit.
[task 2026-10-30T08:35:38.287+00:00] [INFO] Initializing environment for https://github.com/adrienverge/yamllint.git.
[task 2026-10-30T08:35:38.749+00:00] [INFO] Initializing environment for https://github.com/codespell-project/codespell.
[task 2026-10-30T08:35:39.256+00:00] [INFO] Initializing environment for https://github.com/compilerla/conventional-pre-commit.
[task 2026-10-30T08:35:39.594+00:00] [INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[task 2026-10-30T08:35:39.594+00:00] [INFO] Once installed this environment will be reused.
[task 2026-10-30T08:35:39.594+00:00] [INFO] This may take a few minutes...
[task 2026-10-30T08:35:42.386+00:00] [INFO] Installing environment for https://github.com/astral-sh/ruff-pre-commit.
[task 2026-10-30T08:35:42.386+00:00] [INFO] Once installed this environment will be reused.
[task 2026-10-30T08:35:42.386+00:00] [INFO] This may take a few minutes...
[task 2026-10-30T08:35:45.401+00:00] [INFO] Installing environment for https://github.com/astral-sh/uv-pre-commit.
[task 2026-10-30T08:35:45.401+00:00] [INFO] Once installed this environment will be reused.
[task 2026-10-30T08:35:45.401+00:00] [INFO] This may take a few minutes...
[task 2026-10-30T08:35:48.595+00:00] [INFO] Installing environment for https://github.com/adrienverge/yamllint.git.
[task 2026-10-30T08:35:48.595+00:00] [INFO] Once installed this environment will be reused.
[task 2026-10-30T08:35:48.595+00:00] [INFO] This may take a few minutes...
[task 2026-10-30T08:35:51.469+00:00] [INFO] Installing environment for https://github.com/codespell-project/codespell.
[task 2026-10-30T08:35:51.469+00:00] [INFO] Once installed this environment will be reused.
[task 2026-10-30T08:35:51.469+00:00] [INFO] This may take a few minutes...
[task 2026-10-30T08:35:54.768+00:00] trim trailing whitespace.................................................Passed
[task 2026-10-30T08:35:54.825+00:00] fix end of files.........................................................Passed
[task 2026-10-30T08:35:54.973+00:00] check yaml...............................................................Passed
[task 2026-10-30T08:35:55.048+00:00] check for added large files..............................................Passed
[task 2026-10-30T08:35:55.090+00:00] ruff (legacy alias)......................................................Passed
[task 2026-10-30T08:35:55.134+00:00] ruff format..............................................................Passed
[task 2026-10-30T08:35:55.310+00:00] uv-lock..................................................................Passed
[task 2026-10-30T08:35:55.525+00:00] yamllint.................................................................Passed
[task 2026-10-30T08:35:55.840+00:00] codespell................................................................Passed
[taskcluster 2026-10-30T08:35:56.850Z]                        Exit Code: 0
[taskcluster 2026-10-30T08:35:56.850Z]                        User Time: 15.387ms
[taskcluster 2026-10-30T08:35:56.850Z]                      Kernel Time: 11.284ms
[taskcluster 2026-10-30T08:35:56.850Z]                        Wall Time: 24.348417143s
[taskcluster 2026-10-30T08:35:56.850Z]  Average Available System Memory: 14.29 GiB
[taskcluster 2026-10-30T08:35:56.850Z]       Average System Memory Used: 1.32 GiB
[taskcluster 2026-10-30T08:35:56.850Z]          Peak System Memory Used: 1.47 GiB
[taskcluster 2026-10-30T08:35:56.850Z]              Total System Memory: 15.61 GiB
[taskcluster 2026-10-30T08:35:56.850Z]                           Result: SUCCEEDED
[taskcluster 2026-10-30T08:35:56.851Z] === Task Finished ===
[taskcluster 2026-10-30T08:35:56.851Z] Task Duration: 24.348965783s
[taskcluster 2026-10-30T08:35:56.865Z] [mounts] Preserving cache: Moving "/home/task_X_l3XTWVTISXl_0/cache0" to "/home/generic-worker/caches/RknKkHSeSeqc2LauAMxVug"
[taskcluster 2026-10-30T08:35:56.865Z] [mounts] Preserving cache: Moving "/home/task_X_l3XTWVTISXl_0/cache1" to "/home/generic-worker/caches/UxeRmvw1S5m6eEGeVJGT6g"
[taskcluster 2026-10-30T08:35:56.866Z] [mounts] Preserving cache: Moving "/home/task_X_l3XTWVTISXl_0/cache2" to "/home/generic-worker/caches/Y6FZEidjS7C-W1RVIhrv5Q"