Support for Strongly Ordered Reciept and Transmission #157

Activity

1. lance

   added

   type/enhancementNew feature or requestNew feature or request

   module/transport/httpIssues related to the HTTP transport protocol implementationIssues related to the HTTP transport protocol implementation

   on May 13, 2020
2. lance commented on May 14, 2020

   

   lance

   on May 14, 2020

   Last edited by lance

   Member

   More actions

   To validate this as an issue, I put together a little test. Here is an HTTP server that receives a number in a header, expecting it to monotonically increase with each incoming request. It then sets a timeout for a random amount of time up to one second and returns 200/OK after the timeout. If at any point, a number is received out of order

   // server.js
   const http = require('http');
   
   let count = 0;
   const server = http.createServer((req, res) => {
     const num = req.headers['x-num'];
     if (++count != num) {
       throw new Error(`Received request out of order. Expected ${count}. Got ${num}`)
     } else if (count % 1000 === 0) { console.log(count); }
   
     setTimeout(_ => res.end('OK'), getRandom(1000));
     res.end('OK')
   });
   
   function getRandom(max) {
     return Math.floor(Math.random() * Math.floor(max));
   }
   
   server.listen(8080);

   Here is a simple client that sends a monotonically increasing number in the header, from 1 - 5000.

   const http = require('http');
   
   let i = 0;
   while (++i < 5000) {
     http.get('http://localhost:8080', {
       headers: {
         'x-num': i
       }
     })
   }

   It doesn't take very long to see that this is a problem.

   Error: Received request out of order. Expected 392. Got 393
       at Server.<anonymous> (/home/lanceball/src/scratch/ordering/server.js:7:11)
       at Server.emit (events.js:210:5)
       at parserOnIncoming (_http_server.js:745:12)
       at HTTPParser.parserOnHeadersComplete (_http_common.js:115:17)
3. lance commented on May 15, 2020

   

   lance

   on May 15, 2020

   Member

   More actions

   Hmm. I should try the client using sh and curl. Node’s http.get() is also async.
4. erikerikson commented on May 15, 2020

   

   erikerikson

   on May 15, 2020

   MemberAuthor

   More actions

   I guess I thought that was exactly the point. By default HTTP operations are non-blocking so as not to starve other items on the eventloop. If they are used in the library, I would expect out of order sending/receiving eventually. Something like the following is required to guarantee in order submission:

   const http = require('http');
   
   const get = async num => new Promise(resolve => {
     http.get(
       'http://localhost:8080',
       { headers: { 'x-num': num } },
       () => resolve()
     )
   })
   
   const run = async () => {
     let i = 0;
     while (++i < 5000) {
       await get(i)
     }
   }
   
   run()
   

   Note that I'm not doing anything with the responses but I have to wait for an acknowledgement by the server before I can be guaranteed that my message was received.
5. erikerikson commented on May 15, 2020

   

   erikerikson

   on May 15, 2020

   Last edited by erikerikson

   MemberAuthor

   More actions

   For the record, the above has issues but so long as the transmission of that x-num header is the extent to which order matters (as is the case in your server code), the above satisfies.

   A minor note is that although this is marked with the transport/http tag, it should apply equally to any transport that is not already inherently ordered and for which providing delivery ordering guarantees isn't silly. For example, it would be silly to care where the middleware doesn't respect ordering.
6. github-actions commented on Jun 15, 2020

   

   github-actionsBot

   on Jun 15, 2020

   Contributor

   More actions

   This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days
7. github-actions

   added

   status/no-issue-activity

   on Jun 15, 2020
8. github-actions

   closed this as completedon Jun 21, 2020