Skip to content

Repository navigation

Auth fails for responses that include multiple WWW-Authenticate headers. #1234

Closed
#1240
Closed
Auth fails for responses that include multiple WWW-Authenticate headers.#1234
#1240
Labels
bugSomething isn't working

Description

@beruhan
beruhan
opened on Aug 28, 2020

Checklist

httpx==0.11.1

Describe the bug

I use httpx request camera mjpeg,The camera response two WWW-Authenticate headers,I used the right username and password,but It response 401. When I user chrome, everything is ok, I doubt the httpx.DigestAuth may have bug when handle digest.
WireShark Data as follows:

request with httpx

GET /cgi-bin/guest/Video.cgi?media=MJPEG&channel=0 HTTP/1.1
host: 192.168.12.88
user-agent: python-httpx/0.11.1
accept: */*
accept-encoding: gzip, deflate, br
connection: keep-alive

HTTP/1.1 401 Unauthorized
Connection: close
Content-Type: text/html; charset=ISO-8859-1
WWW-Authenticate: Digest realm="server", nonce="4f3b1a0c93d39d1cab4d5d2a8928f85a3cc41bd5e8180e5c37587a9a8aea5097"
WWW-Authenticate: Basic realm="server"
Content-Length: 211

<HTML><HEAD><TITLE>401 Unauthorized</TITLE></HEAD>
<BODY><H1>401 Unauthorized</H1>
Your client does not have permission to get URL /cgi-bin/guest/Video.cgi?media=MJPEG&channel=0 from this server.
</BODY></HTML>

GET /cgi-bin/guest/Video.cgi?media=MJPEG&channel=0 HTTP/1.1
host: 192.168.12.88
user-agent: python-httpx/0.11.1
accept: */*
accept-encoding: gzip, deflate, br
connection: keep-alive
authorization: Digest username="admin", realm="server", nonce="4f3b1a0c93d39d1cab4d5d2a8928f85a3cc41bd5e8180e5c37587a9a8aea5097", uri="/cgi-bin/guest/Video.cgi?media=MJPEG&channel=0", response="43265b82a45736ef943ae501f0234807", algorithm=MD5

HTTP/1.1 401 Unauthorized
Connection: close
Content-Type: text/html; charset=ISO-8859-1
WWW-Authenticate: Digest realm="server", nonce="4f3b1a0ca12e0b92dc8ccca22e84edf7ce355d7825ac0e21eedf8f97d82ec588"
WWW-Authenticate: Basic realm="server"
Content-Length: 211

<HTML><HEAD><TITLE>401 Unauthorized</TITLE></HEAD>
<BODY><H1>401 Unauthorized</H1>
Your client does not have permission to get URL /cgi-bin/guest/Video.cgi?media=MJPEG&channel=0 from this server.
</BODY></HTML>

request with chrome

GET /cgi-bin/guest/Video.cgi?media=MJPEG&channel=0 HTTP/1.1
Host: 192.168.12.88
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Cookie: pbAudioFalg=ON; RESOLUTION=720P; LifeTime=0; captcha=4PTT5z50/JVrA

HTTP/1.1 401 Unauthorized
Connection: close
Content-Type: text/html; charset=ISO-8859-1
WWW-Authenticate: Digest realm="server", nonce="4f3b1c6b2dccae706af263fdc5001a702ae3512d3cc4e42ebf823c60431f64b6"
WWW-Authenticate: Basic realm="server"
Content-Length: 211

<HTML><HEAD><TITLE>401 Unauthorized</TITLE></HEAD>
<BODY><H1>401 Unauthorized</H1>
Your client does not have permission to get URL /cgi-bin/guest/Video.cgi?media=MJPEG&channel=0 from this server.
</BODY></HTML>

GET /cgi-bin/guest/Video.cgi?media=MJPEG&channel=0 HTTP/1.1
Host: 192.168.12.88
Connection: keep-alive
Cache-Control: max-age=0
Authorization: Digest username="admin", realm="server", nonce="4f3b1c6b2dccae706af263fdc5001a702ae3512d3cc4e42ebf823c60431f64b6", uri="/cgi-bin/guest/Video.cgi?media=MJPEG&channel=0", response="7411b325f99b1ef61db78f6a1ba3ff9d"
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Cookie: pbAudioFalg=ON; RESOLUTION=720P; LifeTime=0; captcha=4PTT5z50/JVrA

HTTP/1.1 200 OK
Expires: 0
Pragma: no-cache
Cache-Control: no-cache
Content-Type: multipart/x-mixed-replace;boundary="boundary"

--boundary
Content-Type: image/jpeg
Content-Length: 59048

xxxxx

Activity

  1. florimondmanca
    added
    needs confirmationThe issue described has not yet been confirmed, or replicated locally.
    on Aug 28, 2020

  2. florimondmanca commented on Aug 28, 2020

    @florimondmanca
    florimondmanca
    on Aug 28, 2020
    Contributor

    Hello!

    • You're using a rather old-ish HTTPX version (0.11.1). What does this look like on the latest HTTPX release (0.14.2)?
    • Can you share a minimal example that reproduces the issue on your side?
    • Are there any details about the server/device you're requesting worth sharing? (Make and model, server documentation, etc.)

  3. lovelydinosaur commented on Aug 28, 2020

    @lovelydinosaur
    lovelydinosaur
    on Aug 28, 2020
    Member

    Perhaps a server-side implementation that's treating the Authentication header in a case sensitive way, and so is failing to authenticate correctly?

  4. beruhan commented on Aug 29, 2020

    @beruhan
    beruhan
    on Aug 29, 2020
    Author

    I switch to the latest version httpx==0.14.2,It seems bug still exists, I found a camera instance on public network, And I wrote a simple example to illustrate this problem， as follows, I used requests and httpx request the same url, the requests can authentication successfully,but httpx failed.

    
import httpx
import requests
from requests.auth import HTTPDigestAuth

url = "http://190.156.226.164:8083/cgi-bin/guest/Video.cgi?media=MJPEG&channel=0"


def test_requests():
    auth = HTTPDigestAuth("admin", "admin")
    req = requests.get(url, auth=auth, stream=True)
    print(req.status_code)
    req.close()


def test_httpx():
    auth = httpx.DigestAuth("admin", "admin")
    with httpx.stream("GET", url, auth=auth) as r:
        print(r.status_code)


if __name__ == '__main__':
    test_requests()
    test_httpx()

    output:

    200
401
  5. beruhan
    changed the title [-]there may be a bug in authentication when the server response to www-auth headers[/-] [+]there may be a bug in authentication when the server response two www-auth headers[/+] on Aug 29, 2020

  6. beruhan commented on Aug 29, 2020

    @beruhan
    beruhan
    on Aug 29, 2020
    Author

    Perhaps a server-side implementation that's treating the Authentication header in a case sensitive way, and so is failing to authenticate correctly?

    It might be like you said， Is there any way to modify it？

  7. florimondmanca
    added
    bugSomething isn't working
    externalRoot cause pending resolution in an external dependency
    and removed
    needs confirmationThe issue described has not yet been confirmed, or replicated locally.
    on Aug 29, 2020

  8. florimondmanca commented on Aug 29, 2020

    @florimondmanca
    florimondmanca
    on Aug 29, 2020
    Contributor

    @beruhan To confirm that case sensitivity is causing this issue, you can try adjusting the request exemple so that it sends authorization (lowercase) instead of Authorization. (I don't have a code snippet off the top of my head right now, but the Requests docs should have some examples of modifying headers before sending the request.)

    If this is confirmed then this would be related to #538.

  9. florimondmanca
    added
    needs confirmationThe issue described has not yet been confirmed, or replicated locally.
    and removed
    bugSomething isn't working
    externalRoot cause pending resolution in an external dependency
    on Aug 29, 2020

  10. beruhan commented on Aug 31, 2020

    @beruhan
    beruhan
    on Aug 31, 2020
    Author

    like you said,I modified the requests code to lowercase Authorization header,It can also authenticated,

    GET /cgi-bin/guest/Video.cgi?media=MJPEG&channel=0 HTTP/1.1
Host: 192.168.12.88
User-Agent: python-requests/2.24.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
authorization: Digest username="admin", realm="server", nonce="4f3e773156897b27b411c1e6e28dfc81ba65d826ceaad75908ef1194b749acd2", uri="/cgi-bin/guest/Video.cgi?media=MJPEG&channel=0", response="be2fc243ffba15e79854605b202a301a"

HTTP/1.1 200 OK
Expires: 0
Pragma: no-cache
Cache-Control: no-cache
Content-Type: multipart/x-mixed-replace;boundary="boundary"

    I don't know way the problem,I debug the authenticate process in requests and httpx as follows:
    requests authenticate process

    www-authenticate header received======= Digest realm="server", nonce="4f3e79d568136064295acb9ac2a95ae3a635ba5807cfda9d74549b5a1d4a9947", Basic realm="server"

Authorization header======= Digest username="admin", realm="server", 
nonce="4f3e79d568136064295acb9ac2a95ae3a635ba5807cfda9d74549b5a1d4a9947", uri="/cgi-bin/guest/Video.cgi?media=MJPEG&channel=0", response="e1198d68b1e06ebe68109cb10dce09ea"

    httpx authenticate process

    www-authenticate header received======= Digest realm="server", nonce="4f3e79d526379f6c6b9c52bb4f74a864d3e4ee32f3ce72e59bc6b817cb804450", Basic realm="server"

Authorization header======= Digest username="admin", realm="server", nonce="4f3e79d526379f6c6b9c52bb4f74a864d3e4ee32f3ce72e59bc6b817cb804450", uri="/cgi-bin/guest/Video.cgi?media=MJPEG&channel=0", response="8e82811163fff43a2522d4f7a15b1f9f"

  11. beruhan commented on Aug 31, 2020

    @beruhan
    beruhan
    on Aug 31, 2020
    Author

    I also modify the code to make camera response header as the same ,

    www-authenticate header received======= Digest realm="server", nonce="4f3e79d568136064295acb9ac2a95ae3a635ba5807cfda9d74549b5a1d4a9947", Basic realm="server"
Authorization header======= Digest username="admin", realm="server", nonce="4f3e79d568136064295acb9ac2a95ae3a635ba5807cfda9d74549b5a1d4a9947", uri="/cgi-bin/guest/Video.cgi?media=MJPEG&channel=0", response="e1198d68b1e06ebe68109cb10dce09ea"
200
www-authenticate header received======= Digest realm="server", nonce="4f3e79d568136064295acb9ac2a95ae3a635ba5807cfda9d74549b5a1d4a9947", Basic realm="server"
Authorization header======= Digest username="admin", realm="server", nonce="4f3e79d568136064295acb9ac2a95ae3a635ba5807cfda9d74549b5a1d4a9947", uri="/cgi-bin/guest/Video.cgi?media=MJPEG&channel=0", response="1189f82b9463486970d32134d3e9f6d4", algorithm=MD5
401

  12. lovelydinosaur commented on Aug 31, 2020

    @lovelydinosaur
    lovelydinosaur
    on Aug 31, 2020
    Member

    Somehow I'd completely missed that this issue is specific to the case when the server sends two WWW-Authenticate headers in the response.

    And yup, can see how that might pick out a bug on our side, and yup can see how we ought to resolve this.

    Leaving a link to a useful part of Mozilla's multiple header handling... https://github.com/bnoordhuis/mozilla-central/blob/master/netwerk/protocol/http/nsHttpHeaderArray.h#L185 for my own later reference in this.

  13. lovelydinosaur
    changed the title [-]there may be a bug in authentication when the server response two www-auth headers[/-] [+]Auth fails for responses that include multiple WWW-Authenticate headers.[/+] on Aug 31, 2020
  14. lovelydinosaur
    added
    bugSomething isn't working
    and removed
    needs confirmationThe issue described has not yet been confirmed, or replicated locally.
    on Aug 31, 2020
  15. lovelydinosaur
    closed this as completedin #1240on Sep 1, 2020

  16. j178 commented on Sep 2, 2020

    @j178
    j178
    on Sep 2, 2020
    Contributor

    @beruhan Hi, a fix hanlding mulitiple auth headers has been merged, can you try to install from the master branch and confirm if the issue resolved?

    $ pip install git+https://github.com/encode/httpx.git@master#egg=httpx --upgrade
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions