Skip to content

EventBase::loop(): kevent: Bad file descriptor #280

Description

@slince

hello,

I'm writing forking TCP server with master and forked worker processes. the ExtEventLoop instance in child process will raise some error like this:

Warning: EventBase::looPHP Warning:  EventBase::loop(): kevent: Bad file descriptor in /Users/xxx/vendor/react/event-loop/src/ExtEventLoop.php on line 205

Warning: EventBase::loop(): kevent: Bad file descriptor in /Users/xxxxx/vendor/react/event-loop/src/ExtEventLoop.php on line 205
tserver/vendor/react/event-loop/src/ExtEventLoop.php on line 205

Sample code:

include __DIR__ . '/vendor/autoload.php';

$loop = \React\EventLoop\Loop::get();
$server = new \React\Socket\SocketServer('tcp://127.0.0.1:1234', [], $loop);

const WORKER_NUM = 3;

for ($i = 0; $i <= WORKER_NUM; $i++) {
    $pid = pcntl_fork();
    if (-1 === $pid) {
        exit('fork error');
    } elseif ($pid) {
       
    } else {
        // run the server.
        $loop->run();
        exit;
    }
}
// close in master process.
$server->close();
pcntl_wait($status);

environment:

php: 7.4.25 NTS
os: macos 11.5.2
machine: Macbook pro(Apple M1)

Reference https://stackoverflow.com/questions/33735302/reactphp-libevent-and-socket-pair-throws-error

Is it possible to add a method getEventBase() in https://github.com/reactphp/event-loop/blob/master/src/ExtEventLoop.php to expose the raw eventBase object?

Activity

  1. slince commented on Dec 9, 2021

    @slince
    Author

    @clue @WyriHaximus Can anyone pay attention to this issue, tks

  2. WyriHaximus commented on Dec 12, 2021

    @WyriHaximus
    Member

    @slince We can expose the raw event base, but why not start the server after forking. Or use react/child-process to spawn more server processes?

  3. slince commented on Dec 13, 2021

    @slince
    Author

    @WyriHaximus

    hello

    why not start the server after forking

    If so, then we need to enable the so_reuseport option of the socket. After my local test, using reuseport is not as efficient as using the child process fork directly

    use react/child-process to spawn more server processes?

    fork is more convenient than exec when ipc; in fact, I also implemented exec in my project, but this is the second option, and exec will only be used when fork is not available.

    In fact, i have an idea. i can create an socket server importing an existing socket stream . before this, i have a discusssion #220

    // create raw socket befor fork.
    $rawSocket = stream_socket_server('xxxx');
    
    const WORKER_NUM = 3;
    
    for ($i = 0; $i <= WORKER_NUM; $i++) {
        $pid = pcntl_fork();
        if (-1 === $pid) {
            exit('fork error');
        } elseif ($pid) {
           
        } else {
            // create a server  using raw socket
            $loop = \React\EventLoop\Loop::get();
            $server = new \React\Socket\SocketServer($rawSocket,  $loop);
            $loop->run();
            exit;
        }
    }
    pcntl_wait($status);
  4. clue commented on Jan 5, 2022

    @clue
    Member

    @slince Thanks for bringing this up, excellent question!

    Whether SO_REUSEPORT scales better or not indeed depends on the protocol and application workload, but from my experiments I would say this is usually preferable. As an alternative, I would highly recommend using the ChildProcess component and/or duplicating the file descriptor as recently implemented in #269.

    PHP's pcntl_fork() is quite low-level and unfortunately PHP only provides somewhat limited support for low-level OS operations, so this may open up a can of worms due to the PHP's limited access to low-level APIs. PHP provides higher-level APIs for process creation as used in our ChildProcess component. These APIs internally use the underlying fork() and exec() calls but avoid many of their shortcoming. Once PHP offers more low-level APIs in the future (close(), dup() etc. come to mind), I agree it might make sense to revisit this low-level API, but I don't see this anywhere in the foreseeable future at least. This refs reactphp/event-loop#184 and others.

    The underlying problem is that libev and others rely on epoll/kevent which use a file descriptor which will be inherited to any child processes when forking the parent. As a consequence, any child process may affect events received by the parent process and vice versa. The underlying libev and others implement low-level APIs to essentially re-create this file descriptor in the child process, which is not currently exposed in our EventLoop API because it is somewhat specific to the platform and/or extension in use. For more background, see the above ticket and https://metacpan.org/dist/EV/view/libev/ev.pod#The-special-problem-of-fork

    I believe this has been answered, so I'm closing this for now. Please come back with more details if this problem persists and we can always reopen this 👍

  5. slince commented on Jan 6, 2022

    @slince
    Author

    PHP's pcntl_fork() is quite low-level and unfortunately PHP only provides somewhat limited support for low-level OS operations, so this may open up a can of worms due to the PHP's limited access to low-level APIs.

    @clue hi I think fork is worthy using, you can refer to workerman, it has been working very well and has been trusted by everyone;

  6. clue commented on Jan 6, 2022

    @clue
    Member

    @slince Forking works just fine, also in ReactPHP. Like I said, using the pcntl_fork() function is very low-level and easy to get wrong, because the defaults (while making sense from a historical perspective) don't really fit into long-running, event-driven applications, so you have to do a lot of legwork (which I'd rather avoid). If this sounds confusing, it's because it is, and I would recommend looking into using the ChildProcess component as an easier way to use forks.

    If you really want to use the low-level forking functions in PHP to create a shared file descriptor referencing a socket server, here's how you can do this:

    <?php
    
    require __DIR__ . '/vendor/autoload.php';
    
    // create raw socket befor fork.
    $rawSocketFd = 3; // TODO: low-level stuff not exposed in PHP, but safe to assume in here if you don't have any other file descriptors except stdio
    $rawSocket = stream_socket_server('xxxx');
    
    const WORKER_NUM = 3;
    
    for ($i = 0; $i <= WORKER_NUM; $i++) {
        $pid = pcntl_fork();
        if (-1 === $pid) {
            exit('fork error');
        } elseif ($pid) {
           
        } else {
            // create a server  using raw socket
            $loop = \React\EventLoop\Loop::get();
            $server = new \React\Socket\SocketServer('php://fd/' . $rawSocketFd,  $loop);
            $loop->run();
            exit;
        }
    }
    pcntl_wait($status);

    The above example should work just fine with ReactPHP, but again I would not recommend using this unless you have a very good understanding of how lower-level file descriptors work. In particular, you may find that PHP does not expose file descriptor numbers at all, does not allow you to get a FD number for a resource or even close an FD (see #269, https://github.com/clue/fd and others).

    I hope this helps 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions