Repository navigation
Conversation
|
This PR was written by an AI coding agent (Claude Code) working under the direction of @aignas, who reviewed the design and the changes. Review comments will be handled by @aignas, possibly with the help of the agent; any agent-drafted replies or follow-up commits are reviewed by @aignas before being posted. An agent monitors CI only to retry flaky network failures. This comment was generated by an AI agent and reviewed by the user. |
|
FYI, AI is warning
However, I am sure that without --no-strip-extras we would fail, because the |
3c48036 to
b29c3c6
Compare
…per-extra # Conflicts: # python/private/pypi/BUILD.bazel # python/private/pypi/hub_repository.bzl # python/private/pypi/render_pkg_aliases.bzl
The self edges were matched by comparing the name of the requirement with dashes to the normalized name of the package with underscores, so the extras that include other extras, e.g. `foo-bar[all]` requiring `foo-bar[a,b]`, were never resolved for the packages with more than one word in the name.
The whl repositories did not get the requested extras, so the `pkg` targets did not include them when there is no hub to select the extras, e.g. when using a vendored `requirements.bzl` or `repo_prefix`, and requesting an extra that the package does not provide failed with a missing target. The `requirement` attribute of the whl repository rules is used again. The `pkg` target includes the extras of the requirement as before, the `[]` target is the package without extras and the `[<extra>]` targets are also created for the requested extras that the package does not provide. Such targets are the same as the package without extras, because an unknown extra has no dependencies, which is how `pip` handles them. The `requirements.bzl` file generated by `pip_parse` keeps the extras in the requirements and the hub repositories use the `[]` targets of the whl repositories instead of the `pkg` targets.
The unified `@pypi` hub only had the targets of the packages, so the `[]` and `[<extra>]` targets were only usable through the concrete hubs. Add the `[]` target for all of the packages and the `[<extra>]` targets for the hubs that request the extra, the other hubs get the usual missing package error. The `requirement` macro of the unified hub accepts an extra and the `requirements` macro is added, just like in the concrete hubs.
A lock file may request an extra that the package does not provide, e.g.
`foo[nope]==1.0`. `pip` only warns about it and installs the package
without the extra, but the hub pointed to the `[nope]` target of the whl
repository, which did not exist.
The whl repositories of a hub do not get the requested extras, so that
changing the extras does not refetch them. Instead they read the extras
that are requested in the hub from the `config.bzl` of the hub when the
BUILD file is loaded and create the targets for the extras that the
package does not provide. Such targets are the same as the package
without extras and a warning is printed like `pip` does:
WARNING: foo 1.0 does not provide the extra 'nope'
The same warning is printed if the extra comes from the `requirement`
attribute. The `config.bzl` is now also loaded by the packages without
any dependencies.
The news entry only listed the additions, but the extras of the dependencies, the `whl` targets and the members of the requirement cycles also behave differently, so add a separate entry for the changes. Document how the extras that a package does not provide and the vendored `requirements.bzl` files are handled.
The requested extras of a distribution were passed to its whl repository
via the requirement line, so changing the extras refetched the
repository, the same wheel requested with different extras on different
platforms needed separate repositories, and the extras of the
dependencies written in the wheel
METADATA(e.g.Requires-Dist: coverage[toml]) were ignored, silently dropping theirdependencies.
With this change the whl repositories create an
[<extra>]target forevery extra in the
Provides-ExtraMETADATA and, with bzlmod, therequested extras are no longer passed to them. The hub exposes the
extras requested in the lock file, which follows the rules_pycross
convention:
requirementmacro accepts a single extra, e.g.requirement("google-api-core[grpc]"), and the newrequirementsmacro returns a list of labels for any number of packages and extras,
e.g.
requirements("google-api-core[async_rest,grpc]", "numpy").platforms uses a single whl repository and the hub selects the extras
per platform.
METADATA. If an extrais missing in the lock file, building the target fails with a message
asking to add it to the requirements or the lock file.
package without extras and a warning is printed, like
pipdoes it.whltargets do not use the extras.experimental_requirement_cyclescan be extras, e.g.foo[bar], so that only the targets of that extra are part of thegroup.
@pypihub has the same targets for the extras.vendored
requirements.bzlorrepo_prefixof theWORKSPACEpip_parse, still get the extras via the requirement line and theirpkgtarget includes them as before. The dependencies between suchrepositories use the
pkgtargets and ignore the extras from theMETADATA, as before.foo-bar[all]requiringfoo-bar[a,b], are now resolved for the packages with-,_or.in their names.Fixes #2786
Fixes #4168