feature request: expose underlying socket / stream to api

[DataShovel]

I noted that the api currently only exposes an 'integer' that represents the file descriptor associated with the EventBufferEvent instance.

I'm mostly introducing this as a way to learn more about what best practices might be here, or why the underlying socket / stream is not exposed, but I think it would be nice to be able to pass the underlying socket / stream into a parser, where it's the parser's job to pass control back to the main application after it has received the entire request. For example, with HTTP, with few exceptions the info is useless until the entire request has been received. This could also make it easier to extend on top of this extension by writing encapsulated parsers that interact seamlessly in an evented environment.

There is currently an effort with the pecl_http extension to make it possible via a new Message\Parser api.

http://pecl.php.net/package/pecl_http

http://devel-m6w6.rhcloud.com/mdref/http/Message/Parser

[Ruslan Osmanov]

There are some places where we might pass a stream to userspace. For example: https://bitbucket.org/osmanov/pecl-event/src/42ee1beaa01d64d482efd7197826926ae6cb5088/classes/listener.c?at=master#cl-133

At the time of development nobody needed FDs exported as PHP streams or sockets, so I decided it was wasteful to perform such kind of conversions.

Still we can't export them as sockets because PHP API (internal) has no public functions for allocating new PHP socket structures. We had a conversation regarding sockets with Mathieu CARBONNEAUX here: https://bitbucket.org/osmanov/pecl-event/pull-request/9/add-eventutil-getsocketfromfd-method/diff#comment-3802161

If you find it useful to export the file descriptors as PHP streams, then let's look how the pecl-event API can be modified. Let's decide whether to replace numeric FD with stream resource, or introduce some configuration options, or something else (new method?).

[DataShovel]

I've had a chance to think about this, and what I think it really comes down to (you are likely further along in your understanding so I may be stating the obvious here) is this. When I think about what it really boils down to in terms of "passing control" of a stream between different parts of a system, I think at its lowest common denominator we're really just talking about switching different groups of callbacks on and off depending on what "state" the system is in.

So, the next question in my mind is: How do I perform the coordination? In other words, if I just pass the underlying stream of an EventBufferEvent into some 3rd party app, I still need to disable my other callbacks while control is inside 3rd party software. So also the question remains, how does the 3rd party app pass control back to me? Without some uniform interface, I imagine this could cause code to become unwieldy pretty quick, especially on highly complex servers.

While researching, I came across this section of the libevent documentation:

http://www.wangafu.net/~nickm/libevent-book/Ref4_event.html

What interests me, the documentation above talks at the top about "A user-triggered event.", but doesn't appear to go into too much depth about this.

At the moment, based on what I know so far, my personal preference (perhaps we can blame Javascript) it seems like a good, clean way to receive control back from another part of the system by passing a user-triggered event. My assumption is this different event-type would simply utilize the existing event callback, except with a different bitmask. The assumption also is that by using user-triggered events, the system could also pass a data parameter that would end up being passed to the event callback also.

Passing control into another part of the system is straightforward. Basically it appears this would consist of deactivating active callbacks, then pass the stream as a parameter, and probably in most cases you would also pass the EventBase instance. This way I activate my event callback on the stream before I pass it into the 3rd party. This event callback checks for the user-triggered event coming from the 3rd party and then when the user-triggered event is received the main app does housekeeping, etc. to ensure control has been returned to the main app cleanly, reinitialize callbacks if necessary, etc.

So based on the above, it seems the API updates could be pretty straightforward.

1) expose the stream (either through property or a method call).

2) ensure PHP has the ability to perform user-triggered events, ideally allowing the user-triggered event to also pass data as a parameter to give context / instruction to the event callback listening for this.

Granted, I don't know how difficult it would be to implement something like this, but hopefully the above can be useful in terms of "framing the question" about what I think we're really trying to accomplish by incorporating these new parts to the API.

[DataShovel]

On second thought. I definitely don't think # 2 above is required. For example, if $stream, $base, and $this are all passed into 3rd party software, having a uniform interface (in php land) would be sufficient for coordination. Though I will admit user-triggered events, for some reason, seems appealing.

[DataShovel]

And in fact # 1 may not be important either except in cases where existing interfaces to other software only accept $stream resource as a parameter, and don't assume you're inside an evented environment. By passing $base and $this, I think you have all the tools "required" to perform coordination, although this probably assumes you're writing all the software or wrapping everything you use to ensure they all handle these cases.

[Ruslan Osmanov]

Likely, by "user-triggered event" the Libevent documentation means calling event_active function, i.e. making an event active though its conditions have not triggered. The PHP extension has no method wrapping this function. We can easily add some Event::activate() method.

Still, the obvious problem is that numeric file descriptors don't match the PHP philosophy - they should be wrapped in a universal shell. I was too lazy to make appropriate API. I also thought nobody would need it because the user usually would usually have a PHP stream or socket resource somewhere around.

So we have to fix it. What we've got here is a number of callbacks that are supposed to accept numeric file descriptors. Within the extension internals, at the time of invoking user callback, we know exactly what kind of file descriptor we've got, so we can safely convert it to PHP stream. It's just a matter of pecl-event API.

However, converting the FD arguments to PHP streams would mean a rough change in the API. I wish I had some Zend API to detect what kind of argument the callback expects to accept - whether numeric FD, or PHP stream. Well, there are type hints. Type hints can be tracked up like the following:

/* classes/listener.c */
static void _php_event_listener_cb(struct evconnlistener *listener,
    evutil_socket_t fd, struct sockaddr *address, int socklen, void *ctx) {
...
php_event_listener_t  *l;
zend_fcall_info       *pfci;
zend_fcall_info_cache *pfcc;
zend_uchar             type_hint;

l = (php_event_listener_t *) ctx;

pfci = l->fci;
pfcc = l->fcc;

type_hint = fcc->function_handler->common->arg_info->type_hint;
/* Or maybe:
zend_uchar type = fcc->function_handler->intrnal_function->type;
*/
...
}

Suppose I managed to detect the type. What we would do then? Perhaps, pass a PHP stream, if the type hint points to... "resource"? There is no way to make a hint for resource type in PHP. Then, maybe a class?

<?php
// Internal class
abstract class EventStream
{
  public $stream;
}

function acceptConnCallback($listener, EventStream $fd, $address, $ctx) {
  ...
}

$listener = new EventListener($base, "acceptConnCallback", $base,
  EventListener::OPT_CLOSE_ON_FREE | EventListener::OPT_REUSEABLE, -1,
  "0.0.0.0:$port");

(If it hints to EventStream, then we create an object, convert FD to PHP stream, set up the $stream member, and pass the new object to our callback. Otherwise, we falback to the old variant - numeric FD.)

Looks complicated to me. Well, all is the matter of API. I see no perfect solutions, so I'd like you to pick one of the ideas, or suggest something different.

[DataShovel]

I don't recall seeing the following before, but came across this in PHP documentation:

php://fd/{desc#}

Here: http://php.net/manual/en/wrappers.php.php#refsect2-wrappers.php-unknown-unknown-unknown-unknown-descriptior

Until there's a more compelling case to expose this via an API change in pecl-event, do you think it would be safe (or possible) to interact with the file descriptor with the above convention?

[DataShovel]

Addressing the 'event_active' idea. I think the paradigm I'm thinking about is more closely resembled by something like Signal Events.

http://www.wangafu.net/~nickm/libevent-book/Ref4_event.html

under the 'Constructing signal events' section

Or similar to how you would think about custom events in Javascript. Having some way to put a new event (not related to file descriptors, or timers) onto the event loop. Now, you could likely accomplish something by just setting a timer event with timeout of 0, but in a way there's less decoupling when you have to think about how to transmit the event. Instead with something like Javascript you set it and forget. If there is nothing listening for the event then nothing happens.

There are a number of ways to do this also, though, without depending on a change to the pecl-event API I think. eg. observer pattern.

[DataShovel]

I like the EventStream idea, but at the same time I'm realizing there may be some things that would ideally be addressed on PHP side first. Other than the issues you brought up: API completeness and correctness.

The relevant area where I think PHP is still noticeably weak, is the fact that StreamWrapper is not a true interface. This makes streams (and StreamWrapper) not as customizable and powerful as I think it should be. But that's probably something that should be addressed on the PHP side of things. If it were a true interface I think developers would have alot more flexibility and I think would suddenly make streams far more powerful than they are now. If it were a true interface, developer could pass a class, which implements the StreamWrapper interface, into EventListener, and get all the customizability you would want to incorporate into your custom streams.

@osmanov , you would likely know best whether this would open a can of worms. My guess is it would.

Even so, do you think it would be worth submitting an RFC?