Rewriting the url of a httpx.Request

[maxa-david-francoeur]

I have an implementation that writes code like:

headers = {"Accept": "application/vnd.github+json"}

# Construct API URL to update file content
url = f"/repos/{self.github_repo}/contents/{file_path}"

# Make request to GitHub API
response = await self.http_client.put(url, headers=headers, json=input.model_dump(mode="json"))

the self.http_client is a httpx.AsyncHttpClient with a Custom authentication scheme on it (it adds a token of the authorization header) and a event_hooks for request that sets the absolute url for the request.

The hook is set at construction time, via: event_hooks={"request": [set_github_url]},

and initially I implemented set_github_url as follow:

    async def set_github_url(self, req: httpx.Request) -> httpx.Request:
        github_url = httpx.URL("https://api.github.com")
        if not req.url.is_absolute_url:
            req.url = github_url.join(req.url)
        return req

But this does not work. Because the url is now absolute, but the headers on the request don't have the Host header set. When you build a httpx.Request with an absolute url, a self._prepare(headers) call will do that for you, but if you set the url directly it does not.

so instead my implementation is as follow:

    async def set_github_url(self, req: httpx.Request) -> httpx.Request:
        """Set the absolute URL for the request.

        We create a temporary request with the absolute URL and then copy the other attributes from the original request.
        This is because the constructor of httpx.Request will add some headers (like Host) that we want when the url is
        absolute.

        We keep the original request because the content/stream is already set on that object and we want to keep it."""
        github_url = httpx.URL("https://api.github.com")
        if not req.url.is_absolute_url:
            new_req = httpx.Request(
                method=req.method,
                url=github_url.join(req.url),
            )
            req.url = new_req.url
            req.headers = httpx.Headers(
                {
                    **new_req.headers,  # add url related headers like Host
                    **req.headers,  # keep the original headers if they were included
                }
            )
        return req

That works, but I find it ugly. Could we have a set_url or a join_url function the request object that sets the right header for us? Maybe event better a copy_with(url=...) kind of method that return a new copy of the request and leave the older one untouched.

Does that make sense?

[ketpatil77]

For the fixed GitHub API origin in your example, configure base_url on AsyncClient. HTTPX then resolves the relative URL while building the request, so it also prepares Host before your authentication flow and request hooks run:

import httpx


async def update_file(auth, repo, file_path, payload):
    async with httpx.AsyncClient(
        base_url="https://api.github.com",
        auth=auth,
        headers={"Accept": "application/vnd.github+json"},
    ) as client:
        return await client.put(
            f"/repos/{repo}/contents/{file_path}",
            json=payload,
        )

In your application, keep that client at the existing client lifetime and remove the URL-setting hook. The helper above creates a scoped client only to make the example self-contained.

Request hooks mutate a request in place; returning a different Request from the hook does not replace the request being sent. Also, assigning request.url after construction does not rerun all of the constructor's header preparation. base_url avoids that late rewrite for this use case.

Client configuration. AI-assisted; checked URL resolution, Host, authentication, the Accept header and JSON content with an async mock transport on HTTPX 0.28.1.