Skip to content

Calling SelectorSocketTransport.writelines with a very large payload doesn't get completely written to the socket #103462

Description

@alisaifee

Bug report

As a consequence of the optimization introduced in #91166, when SelectorSocketTransport.writelines is used with a very large buffer which can't be written in one shot either by using socket.send or socket.sendmsg the remaining data in the buffer is never written to the socket.

The following example can be used to reproduce the error:

import asyncio


async def send_message(chunk_size: int, chunks: int):
    await asyncio.sleep(1)
    reader, writer = await asyncio.open_connection("127.0.0.1", 9898)
    data = [bytes(chunk_size)] * chunks + [b"end"]
    writer.writelines(data)
    await writer.drain()
    writer.close()
    await writer.wait_closed()


async def handle_message(reader, writer):
    data = await reader.read()
    print(f"End marker: {data[-3:]!r}")
    writer.close()
    await writer.wait_closed()


async def run_server():
    server = await asyncio.start_server(handle_message, "127.0.0.1", 9898)
    async with server:
        await server.serve_forever()


async def main():
    await asyncio.gather(run_server(), send_message(6500, 100))


asyncio.run(main())

The example above prints End marker: b'end' with python 3.11 but not with python 3.12.0a6+

Your environment

  • CPython versions tested on: 3.12.0a7
  • Operating system and architecture: macOS 13.2

Linked PRs

Activity

  1. kumaraditya303 commented on Apr 12, 2023

    @kumaraditya303
    Contributor

    Good catch! I have been running this code for a while but the buffers aren't large enough to trigger this.

  2. moved this from Todo to Done in asyncioon Apr 13, 2023
  3. added a commit that references this issue on Apr 13, 2023
  4. added a commit that references this issue on Apr 13, 2023
  5. added a commit that references this issue on Apr 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

3.12only security fixestopic-asynciotype-bugAn unexpected behavior, bug, or error

Projects

  • Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions