Please join the Rack Specification discussion for `env['upgrade.websocket']`

Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Bo]

Please join the Rack Specification discussion for `env['upgrade.websocket']`

I represent an effort to extend Rack so that it allows server-side websocket upgrade implementation support and pure Rack websocket applications.

This new Rack feature proposal is gaining support, with over 42 support votes in the [original Rack discussion thread](https://github.com/rack/rack/issues/1093).

You may have read [my blog post about this](https://bowild.wordpress.com/2016/07/31/the-dark-side-of-the-rack/) or [the reddit discussion](https://www.reddit.com/r/ruby/comments/4vgdlc/the_dark_side_of_the_rack_and_websockets_dreams/).

This proposal simplifies Websocket applications by leaving all the network complexity were it is (with the application's web server), allowing application programmers to focus on their application logic.

Using [the proposed specification](https://github.com/boazsegev/iodine/issues/6), a pure Rack Websocket echo server could be written as simply as:

```ruby
# this is a toy example.
class MyEcho
   def initialize(env = nil)
      # optional initialization
   end
   def on_message(data)
     write "Echo: #{data}"
   end
end

app = Proc.new do |env|
   if env['upgrade.websocket?'] && env['HTTP_UPGRADE'] =~ /websocket/i
      env['upgrade.websocket'] = MyEcho.new(env) # or simply `MyEcho`
      [ 0, {'X-Header': 'data'}, [] ]
   else
      [200, { 'Content-Length' => '12' }, ['He11o World!']]
   end
end

run app
```

There's [a working Gist chatroom example](https://gist.github.com/boazsegev/1466442c913a8dd4271178cab9d98a27).

I invite you to join [the discussion](https://github.com/rack/rack/issues/1093) and help shape the [proposed specification](https://github.com/boazsegev/iodine/issues/6).

Re: Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Eric Wong]

Bo <boaz@2be.co.il> wrote:
> Please join the Rack Specification discussion for `env['upgrade.websocket']`

Fwiw, I have not existed outside of plain-text email for years;
so I'll respond inline here and you can link others to this
thread on:

https://bogomips.org/unicorn-public/58D32D89-4523-436E-8C0E-FA8F4E7AFFB2@plezi.io/T/
(which anybody may reply-all to after disabling HTML in their mailer)

> I represent an effort to extend Rack so that it allows server-side
> websocket upgrade implementation support and pure Rack websocket
> applications.
> 
> This new Rack feature proposal is gaining support, with over 42
> support votes in the [original Rack discussion
> thread](https://github.com/rack/rack/issues/1093).

You mention performance several times, but I am not sure what
you mean.  unicorn completely stops caring for a socket after
it's hijacked by the app.  In other words:

  Your Rack app could take the socket and inject it into an
  event loop running in a separate thread.  That event loop
  could be 100% C code running without GVL for all unicorn
  cares.

  unicorn would only parse the first HTTP request (with the
  Upgrade header) before the Rack app hijacks it.


Also, Ruby IO.select also supports arbitrary number of
descriptors in some cases (Linux for sure, and probably most
server-oriented *BSDs).  Of course, the malloc usage + O(n)
behavior still suck, but no, select(2) is not always limited to
<=1024 FDs.


In any case, I'd be glad to help with lower-level issues over
plain-text email, but I'm not going to attempt to run a
mainstream browser capable of WebSockets or JS.
I suppose curl can deal with WebSockets nowadays?

Re: Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Boaz Segev]

Eric, thank you for your review and thoughts on the matter.

You raised a few questions, please allow me to both answer and clarify the intention behind the proposed extension.

>> I represent an effort to extend Rack so that it allows server-side
>> websocket upgrade implementation support and pure Rack websocket
>> applications.
>> 
>> This new Rack feature proposal is gaining support, with over 42
>> support votes in the [original Rack discussion
>> thread](https://github.com/rack/rack/issues/1093).
> 
> You mention performance several times, but I am not sure what
> you mean.  unicorn completely stops caring for a socket after
> it's hijacked by the app.  In other words:
> 
>  Your Rack app could take the socket and inject it into an
>  event loop running in a separate thread.  That event loop
>  could be 100% C code running without GVL for all unicorn
>  cares.


Running two (or more) event loops, each with it's own resources, is wasteful and promotes needless context switches.

There is no reason to hijack a socket when the server can easily provide callbacks for IO related events using it's existing established event loop.

This alone should provide a performance boost. There are other considerations as well, but I think they all derive from this core principle of having the web server retain control over all network IO concerns.

> Also, Ruby IO.select also supports arbitrary number of
> descriptors in some cases (Linux for sure, and probably most
> server-oriented *BSDs).  Of course, the malloc usage + O(n)
> behavior still suck, but no, select(2) is not always limited to
> <=1024 FDs.


Although this isn't as relevant to the proposed specification, it is a good example for how intricate network programming details are unknown by most Ruby programmers and shows why it would be better for the web server to retain control over all network IO concerns.

For example, the Linux man page expressly states that "To monitor file descriptors greater than 1023, use poll(2) instead"

This is true even when using select for a single FD that has a value of more then 1023, since `select` is often implemented using a bit vector map of 1024 bits that are controlled using the FD_(*) macros.

See, for example: http://linux-tips.org/t/is-it-possible-to-listen-file-descriptor-greater-than-1024-with-select/45/2

Ruby's IO.select uses this same system call and suffers the same issues that are present in the OS - each OS with it's own quirks.

I don't think application developers should worry about these details when these issues were already resolved during the server's designed.

By having the server provide callbacks - instead of hijacking - we eliminate a hoard of potential bugs and considerations.

>  unicorn would only parse the first HTTP request (with the
>  Upgrade header) before the Rack app hijacks it.


Of course, not every server has to offer this feature - just like hijacking, it's optional.

Unicorn was design for very specific use cases, so I definitely understand if this might not be as interesting for you. 

However, I do think your experience as developers could help enrich us all. If you have any comments or anything to add regarding the proposed `websocket.upgrade` specification, your voice is welcome:

https://github.com/boazsegev/iodine/issues/6

Re: Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Michael Fischer]

On Fri, Aug 5, 2016 at 8:09 PM, Boaz Segev <boaz@2be.co.il> wrote:

> > Your Rack app could take the socket and inject it into an
> > event loop running in a separate thread. That event loop
> > could be 100% C code running without GVL for all unicorn
> > cares.
>
>
> Running two (or more) event loops, each with it's own resources, is wasteful and promotes needless context switches.
>
> There is no reason to hijack a socket when the server can easily provide callbacks for IO related events using it's existing established event loop.

I suspect we're missing the forest for the trees here :)

Unicorn is a forking webserver, not an event-driven or threaded
webserver.  Whether Unicorn ever supports the proposed standard or
not, I wouldn't recommend it be used for typical applications that use
WebSockets, as they usually involve persistent connections.  Unicorn
would quickly run out of worker processes capable of serving such
connections anyway, so I question whether it's worth delving into the
dirty details of IO syscalls.

Best regards,

--Michael

Re: Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Eric Wong]

Boaz Segev <boaz@2be.co.il> wrote:
> Running two (or more) event loops, each with it's own
> resources, is wasteful and promotes needless context switches.
> 
> There is no reason to hijack a socket when the server can
> easily provide callbacks for IO related events using it's
> existing established event loop.

As Michael said, WS is a really bad fit for unicorn :)

> This alone should provide a performance boost. There are other
> considerations as well, but I think they all derive from this
> core principle of having the web server retain control over
> all network IO concerns.

For anybody else writing an epoll-based server with multiple
epoll-based event loops in different threads, the following is
good ordering; relatively cheap and scalable:

	epoll_ctl(original_epfd, EPOLL_CTL_DEL, client_fd, NULL);
	epoll_ctl(websocket_epfd, EPOLL_CTL_ADD, client_fd, &ev);

Change the order, and it's slower and less scalable:

	/* in other words, don't do this in this order */
	epoll_ctl(websocket_epfd, EPOLL_CTL_ADD, client_fd, &ev);
	epoll_ctl(original_epfd, EPOLL_CTL_DEL, client_fd, NULL);

The former ensures a simple topology where the file behind
client_fd only exists in one eventpoll rbtree at a time.  The
latter creates a complex topology which needs to do additional
locking and scanning to prevent infinite loops.

This is true since Linux 3.13, commit 67347fe4e632633
("epoll: do not take global 'epmutex' for simple topologies")
 https://bogomips.org/mirrors/linux.git/commit?id=67347fe4e632633

> like hijacking, it's optional.
> 
> Unicorn was design for very specific use cases, so I
> definitely understand if this might not be as interesting for
> you. 

Yes, unicorn itself will never have a multi-client-aware event
loop.  It is out-of-scope for its design, it seems WebSockets
usually have idle times which would cause any server with the
design of unicorn to fall over.  unicorn cannot even handle
HTTP/1.1 persistent connections.

> However, I do think your experience as developers could help
> enrich us all. If you have any comments or anything to add
> regarding the proposed `websocket.upgrade` specification, your
> voice is welcome:

Sorry, not interested, and definitely not touching centralized
message boards :>

Re: Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Boaz Segev]

>> Michael said
> Unicorn is a forking webserver, not an event-driven or threaded
> webserver.  Whether Unicorn ever supports the proposed standard or
> not, I wouldn't recommend it be used for typical applications that use
> WebSockets, as they usually involve persistent connections.  Unicorn
> would quickly run out of worker processes capable of serving such
> connections anyway, so I question whether it's worth delving into the
> dirty details of IO syscalls.


>> Eric said
> Sorry, not interested, and definitely not touching centralized
> message boards :>


I totally understand and I thank you both for your time.

-- Bo.

Re: Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Sam Saffron]

For the record Eric,

We use unicorn at Discourse to serve concurrent long polls across 100s
of hosted forums using hijack with zero issues using
https://github.com/SamSaffron/message_bus

I am not particularly enthused WebSockets and have blogged about my
reservations, in an HTTP/2 world I am less clear about how they fit in
except for some very niche uses.

If this becomes a core Rack specification and now forces all Rack
implementation to carry around websocket framing protocol and
epoll/kqueue I would also be less excited.

On Sat, Aug 6, 2016 at 5:37 PM, Boaz Segev <boaz@2be.co.il> wrote:
>>> Michael said
>> Unicorn is a forking webserver, not an event-driven or threaded
>> webserver.  Whether Unicorn ever supports the proposed standard or
>> not, I wouldn't recommend it be used for typical applications that use
>> WebSockets, as they usually involve persistent connections.  Unicorn
>> would quickly run out of worker processes capable of serving such
>> connections anyway, so I question whether it's worth delving into the
>> dirty details of IO syscalls.
>
>
>>> Eric said
>> Sorry, not interested, and definitely not touching centralized
>> message boards :>
>
>
> I totally understand and I thank you both for your time.
>
> -- Bo.
>

Re: Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Eric Wong]

Sam Saffron <sam.saffron@gmail.com> wrote:
> For the record Eric,
> 
> We use unicorn at Discourse to serve concurrent long polls across 100s
> of hosted forums using hijack with zero issues using
> https://github.com/SamSaffron/message_bus

Thanks.  That's exactly the usage I had in mind when I learned
about rack.hijack.

Re: Please join the Rack Specification discussion for `env['upgrade.websocket']`

[Boaz Segev]

Hi Sam,

I just wanted to point out that the specification was meant to be optional and this is why it includes the `env['upgrade.websocket?']` flag to indicate support. 

I don't think every server has to support Websockets - different server architectures favor different tasks and performance isn't a one size fit all.

However, it's high time (if not a bit late) Ruby had better support for applications that decide to use Websockets - which is, obviously, not all of them (I doubt that many applications need them).

I think it's wrong that Websockets are so complicated to implement in Ruby and Rack. The technology might be dangerous, but it does have it's uses (such as large file uploads, inter-app micro-services, game servers... to name just a few use-cases that survive HTTP/2).

- Bo.

> On Aug 8, 2016, at 21:08, Sam Saffron <sam.saffron@gmail.com> wrote:
> 
> For the record Eric,
> 
> We use unicorn at Discourse to serve concurrent long polls across 100s
> of hosted forums using hijack with zero issues using
> https://github.com/SamSaffron/message_bus
> 
> I am not particularly enthused WebSockets and have blogged about my
> reservations, in an HTTP/2 world I am less clear about how they fit in
> except for some very niche uses.
> 
> If this becomes a core Rack specification and now forces all Rack
> implementation to carry around websocket framing protocol and
> epoll/kqueue I would also be less excited.
> 
> On Sat, Aug 6, 2016 at 5:37 PM, Boaz Segev <boaz@2be.co.il> wrote:
>>>> Michael said
>>> Unicorn is a forking webserver, not an event-driven or threaded
>>> webserver.  Whether Unicorn ever supports the proposed standard or
>>> not, I wouldn't recommend it be used for typical applications that use
>>> WebSockets, as they usually involve persistent connections.  Unicorn
>>> would quickly run out of worker processes capable of serving such
>>> connections anyway, so I question whether it's worth delving into the
>>> dirty details of IO syscalls.
>> 
>> 
>>>> Eric said
>>> Sorry, not interested, and definitely not touching centralized
>>> message boards :>
>> 
>> 
>> I totally understand and I thank you both for your time.
>> 
>> -- Bo.
>>