highWaterMark option is ignored in HTTPs request stream

Hi,

https request stream ignores the custom highWaterMark size option. In http request it works perfectly.

Version: All of them, even nightly
Platform: Windows 10 64-bit
Platform: https

What steps will reproduce the bug?

Execute the following code snippet using node

var https = require('https');

var url = "https://www.facebook.com";      
var options = { rejectUnauthorized: false, 
                highWaterMark: 128000  };
        
var request = https.request(url, options, function(response)   {
     // it returns 16384 with https but 128000 in http
     console.log("HTTPS readableHighWaterMark: " + response.readableHighWaterMark);  
}).end();

var http = require('http');

var url = "http://www.facebook.com";
var options = {  highWaterMark: 128000  };

var request = http.request(url, options, function(response)   {
     // it returns 16384 with https but 128000 in http
     console.log("HTTP readableHighWaterMark: " + response.readableHighWaterMark);  
}).end();

How often does it reproduce? Is there a required condition?

No. This potential bug can always be reproduced.

What is the expected behavior?

https stream readableHighWaterMark should be 128000 instead 16384.

What do you see instead?

Console Output:

HTTP readableHighWaterMark: 128000
HTTPS readableHighWaterMark: 16384

I'm try working on this.

tls: add highWaterMark option for connect

There is still something strange...

Testing with Node.js v15.0.0-nightly20200505c17dcb3253 it returns fine now the highWaterMark in https module but it is still downloading with chunks of 16Kb

HTTPS Test
--------------------------------------------------
Response.readableHighWaterMark: 131072
Outfile WriteStream WritableHighWaterMark: 131072

Progress: 0%    16.00 kB / 21.85 MB
Progress: 0%    32.00 kB / 21.85 MB
Progress: 0%    48.00 kB / 21.85 MB
Progress: 0%    64.00 kB / 21.85 MB
Progress: 0%    80.00 kB / 21.85 MB
Progress: 0%    96.00 kB / 21.85 MB
Progress: 0%    112.00 kB / 21.85 MB
...

Testing the same file to download, to the same server in a LAN with http instead https we are receiving in "on data" callback bigger chunks

HTTP Test
--------------------------------------------------
Response.readableHighWaterMark: 131072
Outfile WriteStream WritableHighWaterMark: 131072

Progress: 0%    10.69 kB / 21.85 MB
Progress: 0%    74.69 kB / 21.85 MB
Progress: 0%    138.69 kB / 21.85 MB
Progress: 0%    202.69 kB / 21.85 MB
Progress: 1%    266.69 kB / 21.85 MB
Progress: 1%    330.69 kB / 21.85 MB
Progress: 1%    394.69 kB / 21.85 MB
Progress: 2%    458.69 kB / 21.85 MB
Progress: 2%    522.69 kB / 21.85 MB
Progress: 2%    586.69 kB / 21.85 MB
...

Should I fill a new ticket?

I'm trying to replicate this problem.

@thunderol

@thunderol could you create a separate GH issue? It'll be easier to track the fix this way.

Created new issue #33262

tls: add highWaterMark option for connect

Navigation Menu

Repository navigation

highWaterMark option is ignored in HTTPs request stream #32781

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

highWaterMark option is ignored in HTTPs request stream #32781

Description

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Activity

rickyes commented on Apr 11, 2020

thunderol commented on May 5, 2020

rickyes commented on May 6, 2020

puzpuzpuz commented on May 6, 2020

thunderol commented on May 6, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions