Support for multiple files per POST field

async def http_upload():
    async with httpx.AsyncClient(verify=False) as client:
        data_path = os.path.join(PARENT_PATH, 'data_file/blob/data')
        index_path = os.path.join(PARENT_PATH, 'data_file/blob/index')
        files = {
            "upload_file": [
                open(data_path, 'rb'),
                open(index_path, 'rb'),
            ]
        }
        # files = [
        #     ("upload_file", open(data_path, 'rb')),
        #     ("upload_file", open(index_path, 'rb'))
        # ]
        params = {
            "requestId": "10001",
            "sdkVersion": "V1.2.3",
            "area": "CH"
        }

        response = await session.post(url, data=params, files=files)

self = <httpx.content_streams.MultipartStream.FileField object at 0x7f503c22ea60>

    def render_data(self) -> bytes:
        if isinstance(self.file, str):
            content = self.file
        else:
>           content = self.file.read()
E           AttributeError: 'list' object has no attribute 'read'

or
    def iter_fields(
        self, data: dict, files: dict
    ) -> typing.Iterator[typing.Union["FileField", "DataField"]]:
        for name, value in data.items():
            if isinstance(value, (list, dict)):
                for item in value:
                    yield self.DataField(name=name, value=item)
            else:
                yield self.DataField(name=name, value=value)

>       for name, value in files.items():
E       AttributeError: 'list' object has no attribute 'items'

Excuse me, how should I use。

You should read the manual https://www.python-httpx.org/advanced/#multipart-file-encoding

Hi, not sure was the problem was exactly, but going to close this as resolved for now. If there's anything unclear/missing in the docs relative to your problem, feel free to ping back here! Thanks.

@j178

It seems to me the OP's problem was the same I'm facing right now in that I'm trying to replicate what is described in the requests docs here: https://2.python-requests.org/en/master/user/advanced/#post-multiple-multipart-encoded-files
The httpx docs @j178 pointed to only seem to describe posting a single file (which works exactly as for requests). However, once multiple files need to be uploaded, requests stops taking a dict and uses a list instead, as a dict can't use the same key multiple times.
The question now would be how, if at all, this behavior of the requests library could be replicated in httpx.

You should read the manual https://www.python-httpx.org/advanced/#multipart-file-encoding

Upload multiple files at once, not one.Can't HTTPX solve it?

Hi, not sure was the problem was exactly, but going to close this as resolved for now. If there's anything unclear/missing in the docs relative to your problem, feel free to ping back here! Thanks.

Upload multiple files at once, not one.Can't HTTPX solve it?
The problem is not solved, why is it closed?
Just like requests: https://2.python-requests.org/en/master/user/advanced/#post-multiple-multipart-encoded-files

@ysq1222

@ysq1222 Reopened, I think it’s clearer now what this is about: uploading multiple files for a given POST field using HTTPX (we only support 1 file per file right now).

That last link is particularly helpful, especially because it shows what the API should look like for such a feature, i.e. a list of file dicts (which is a reasonable API I guess, even though the typing for files is already quite involved).

Since it’s supported by Requests I don’t see why we shouldn’t support it. :)

At this point, I imagine any PRs towards resolving this are much welcome? 😄

Edited the issue description to better reflect the need here. 👍

So, I think we also want to support the "either a dict, or a two-tuple of items" for the data=... case too.

Ie, this works in requests, but not in httpx...

import requests

url = 'https://httpbin.org/post'
r = requests.post(url, data=[("a", "1"), ("a", "2")])
print(r.text)

In both cases we want to change how we iterate over the items in data/files...

httpx/httpx/_content_streams.py

Line 293 in b3db9ff

for name, value in data.items():

Eg...

data_items = data.items() if isinstance(data, dict) else data
files_items = files.items() if isinstance(files, dict) else files
for name, value in data_items:
    ...
for name, value in files_items:
    ...

Reopened

Thanks! Great to see quick progress here now :D

That last link is particularly helpful

You're welcome! Oh. Wait. 😉

@ysq1222

@ysq1222 Reopened, I think it’s clearer now what this is about: uploading multiple files for a given POST field using HTTPX (we only support 1 file per file right now).

That last link is particularly helpful, especially because it shows what the API should look like for such a feature, i.e. a list of file dicts (which is a reasonable API I guess, even though the typing for files is already quite involved).

Since it’s supported by Requests I don’t see why we shouldn’t support it. :)

At this point, I imagine any PRs towards resolving this are much welcome?

Will it be supported in the next version?

Can someone point me where to look in the codebase if tackling this issue?

@tegkhanna

Hi @tegkhanna 👋 you can look at MultipartStream in ._content_streams.py. There should be various hints upper in this thread too.

@florimondmanca

Thanks @florimondmanca! I will get to it straight away!

@ysq1222

Is there any other site similar to httpbin.org?
I made some changes and they didn't appear to be working. Tested everything and then realized that httpbin.org is not sending the right response. When adding multiple files to the same input, it just sends back the first filename. Checked the same with requests as given in the last link by @ysq1222 and got the same result. Just the name of the first file.

@tomchristie

Nevermind. Checked using https://ptsv2.com/ and it worked like a charm!

Should I add support for this as well as mentioned by @tomchristie?

So, I think we also want to support the "either a dict, or a two-tuple of items" for the data=... case too.

Ie, this works in requests, but not in httpx...
import requests

url = 'https://httpbin.org/post'
r = requests.post(url, data=[("a", "1"), ("a", "2")])
print(r.text)

Added a pull request:
#891

Navigation Menu

Repository navigation

Support for multiple files per POST field #777

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Support for multiple files per POST field #777

Description

Activity

j178 commented on Jan 18, 2020

florimondmanca commented on Jan 20, 2020

ChristianMertes commented on Feb 3, 2020

ysq1222 commented on Feb 11, 2020

ysq1222 commented on Feb 11, 2020

florimondmanca commented on Feb 11, 2020

florimondmanca commented on Feb 11, 2020

lovelydinosaur commented on Feb 11, 2020

ChristianMertes commented on Feb 11, 2020

ysq1222 commented on Feb 11, 2020

tegkhanna commented on Mar 30, 2020

florimondmanca commented on Mar 30, 2020

tegkhanna commented on Mar 30, 2020

tegkhanna commented on Mar 31, 2020

tegkhanna commented on Mar 31, 2020

tegkhanna commented on Mar 31, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions