Repository navigation
Question: Bazel downloader should only be reading packages, versions, and hashes from pip.parse.requirements_lock, correct? #2561
Description
We have a requirements lock file requirements.txt. It contains args for --index-url and --extra-index-url like so:
# This file was autogenerated by uv ...
# uv pip compile ...
--index-url https://pypi.org/simple
--extra-index-url=https://oauth2accesstoken@baz.com/simple
nurpc==1.0.0 \
--hash=sha256:...
And our MODULE.bazel has:
pip.parse(
experimental_index_url = "https://pypi.org/simple",
experimental_index_url_overrides = {"package": "https//baz.com/simple", ...}, # note no "oauth2accesstoken"
requirements_lock = "requirements.txt",
...At this point Bazel fails for 401 unauthorized (which is why I'm down this rabbit hole in the first place... you can ignore this):
$ bazel clean --expunge_async; bazel run //:gazelle_python_manifest.update
...
===== stdout start =====
Looking in indexes: https://pypi.org/simple, https://****@baz.com/simple
===== stdout end =====
===== stderr start =====
WARNING: 401 Error, Credentials not correct for https://baz.com/simple/nurpc/
ERROR: Could not find a version that satisfies the requirement nurpc==1.0.0 (from versions: none)
ERROR: No matching distribution found for nurpc==1.0.0
...
But if I change requirements.txt:
---extra-index-url https://oauth2accesstoken@baz.com/simple
+--extra-index-url https://baz.com/simpleI get a different Bazel error. Whaaaaaat?:
$ bazel clean --expunge_async; bazel run //:gazelle_python_manifest.update
...
===== stdout start =====
Looking in indexes: https://pypi.org/simple, https://baz.com/simple
User for baz.com:
===== stdout end =====
===== stderr start =====
ERROR: Exception:
Traceback (most recent call last):
...
Thus, it seems like Bazel may be reading --extra-index-url from the requirements.txt file. Is this intended? If so, why?
Given that the different error is asking for username and password, I'm inclined to think it's related to the credential helper. Our credential helper essentially just calls echo "{headers: {Bearer $(gcloud auth application-default print-access-token)} }", so that isn't reading requirements.txt.
Activity
aignas commentedon Jan 14, 2025
The index interaction happens in 2 places:
pip.parseattribute - should we use the one from the file?whl_library. It seems that this is the second error that you are getting.EDIT: If you are content with the answer feel free to convert the issue to a discussion (the best place for questions in general), otherwise we can groom this into a feature/bug ticket. :)
dougthor42 commentedon Jan 15, 2025
Hard to say. My initial reaction was to say "no, we should use the value from
pip.parse.experimental_*" but I don't have any strong reasoning for that.The
nurpcthing being downloaded from the private index is already a wheel, so it shouldn't be building from sdist.Ah, thanks! I wasn't aware we used Discussions (or maybe I forgot...). Done!