Skip to content

ngclient: Accept proxy variables without a scheme - #3051

Closed
BahaaMattar wants to merge 1 commit into
theupdateframework:developfrom
BahaaMattar:fix/proxy-env-scheme-less-proxy
Closed

BahaaMattar wants to merge 1 commit into
theupdateframework:developfrom
BahaaMattar:fix/proxy-env-scheme-less-proxy

Conversation

@BahaaMattar

Copy link
Copy Markdown
Contributor

Description of the changes being introduced by the pull request:

Proxy environment variables are often set without a scheme, for example https_proxy=proxy.example:3128. The previous requests based fetcher (which ProxyEnvironment aims to keep parity with) and curl treat that as an http:// proxy. ProxyEnvironment passed the value as is to urllib3, whose ProxyManager raises ProxySchemeUnknown, so Urllib3Fetcher failed every download with a DownloadError.

This adds a small helper, _with_scheme(), used where _get_proxy() returns the *_proxy / all_proxy value: a proxy URL without :// gets http:// prepended, anything that already has a scheme is returned unchanged.

test_proxy_without_scheme (new) sets a scheme-less https proxy and a scheme-less all proxy with credentials, and checks the pool managers are created for http://localhost:8888 and http://user:secret@localhost:9999. It fails on current develop with ProxySchemeUnknown and passes with the change. I also checked end to end against a local HTTP server acting as the proxy: with http_proxy=127.0.0.1:<port>, Urllib3Fetcher.download_bytes() fails with DownloadError before the change, and after it the request reaches the proxy (the proxy sees GET http://tuf.example.invalid/timestamp.json).

Locally: the full unittest suite (207 tests) passes, coverage report --fail-under 97 passes, ruff check, ruff format --diff and mypy are clean on the changed files.

Fixes #3050

Proxy environment variables are often set without a scheme, for
example https_proxy=proxy.example:3128. The previous requests based
fetcher, curl and most other clients treat that as a http:// proxy,
but ProxyEnvironment passed the value as is to urllib3, which raises
ProxySchemeUnknown. Urllib3Fetcher then failed every download with
a DownloadError.

Add "http://" to proxy URLs that have no scheme.

Fixes theupdateframework#3050

Signed-off-by: Bahaa Mattar <bahaa.kmattar@gmail.com>
@BahaaMattar
BahaaMattar requested a review from a team as a code owner November 19, 2026 09:42
@jku

jku commented Nov 19, 2026

Copy link
Copy Markdown
Member

Thanks but I think I'll consider this "works as designed", context: #3050 (comment)

@jku jku closed this Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Urllib3Fetcher fails every download with ProxySchemeUnknown when a proxy variable has no scheme

2 participants