yahns Ruby server user/dev discussion
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: yahns-public@yhbt.net
Subject: [RFC] proxy_pass: document as public API
Date: Sat, 20 Feb 2016 08:16:19 +0000	[thread overview]
Message-ID: <20160220081619.GA10850@dcvr.yhbt.net> (raw)

We've been using this for nearly a year to serve our own
list archives using (Apache (mpm_prefork) || Starman) and
Varnish-cache on: http://yhbt.net/yahns-public/
 Any thoughts?  So far I've resisted having a public API.
 On the other hand, the current state of ProxyPass being a
 Rack app using rack.hijack still has nasty limitations
 such as incompatibility with any existing middleware.
 Something simple such as access logs won't work well
 (e.g. Rack::CommonLogger or Clogger)

 On the other hand, it would be nice to have a mostly-Ruby
 alternative to nginx today...

 Any API or configuration directive publically documented for
 yahns should remain supported forever (or as long as underlying
 components such as Rack/Ruby/HTTP are), regardless of
 version changes.  There may be exceptions for directives marked
 as "experimental", but it's not the case, here.

 So I'm also considering a special configuration directive and
 making it independent of Rack.  In other words, something like:

	app(:proxy_pass) ...

 as opposed to (the currently-implemented):

	app(:rack) ...

 in yahns_config.5
 But that also requires writing more code to support...

 lib/yahns.rb            |  7 ++++-
 lib/yahns/proxy_pass.rb | 74 +++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 75 insertions(+), 6 deletions(-)

diff --git a/lib/yahns.rb b/lib/yahns.rb
index 2be7930..a6b15d7 100644
--- a/lib/yahns.rb
+++ b/lib/yahns.rb
@@ -15,9 +15,14 @@
     Unicorn.__send__(:remove_const, sym) if Unicorn.const_defined?(sym)
-# yahns exposes no user-visible API outside of the config file.
+# yahns exposes little user-visible API outside of the config file.
 # See http://yahns.yhbt.net/yahns_config.txt for the config documentation
 # and http://yahns.yhbt.net/ for the homepage.
+# Currently, the only supported Ruby API provided by yahns is:
+#   Yahns::ProxyPass
 # Internals are subject to change.
 module Yahns
diff --git a/lib/yahns/proxy_pass.rb b/lib/yahns/proxy_pass.rb
index 511db02..d6df4a9 100644
--- a/lib/yahns/proxy_pass.rb
+++ b/lib/yahns/proxy_pass.rb
@@ -2,15 +2,57 @@
 # Copyright (C) 2013-2016 all contributors <yahns-public@yhbt.net>
 # License: GPL-3.0+ (https://www.gnu.org/licenses/gpl-3.0.txt)
 # frozen_string_literal: true
 require 'socket'
 require 'kgio'
 require 'kcar' # gem install kcar
 require 'rack/request'
-require 'timeout'
+require 'timeout' # only for Timeout::Error
 require_relative 'proxy_http_response'
-class Yahns::ProxyPass # :nodoc:
+# Yahns::ProxyPass is a Rack (hijack) application which allows yahns to
+# act as a fully-buffering reverse proxy to protect backend servers from
+# slow HTTP clients.
+# Yahns::ProxyPass relies on the default behavior of yahns to implement
+# full input and output buffering.  Output buffering is lazy, meaning it
+# allows streaming output in the best case and will only buffer if the
+# client cannot keep up with the server.
+# The goal of this reverse proxy is to act as a sponge on the same LAN
+# or host as the backend HTTP server incapable of handling slow clients.
+# Yahns::ProxyPass accomplishes this by handling all the slow I/O
+# internally within yahns itself to minimize time spent in the backend
+# HTTP server waiting on slow I/O.
+# Examples of backend HTTP servers which benefit from having a
+# fully-buffering reverse proxy include:
+# * Apache (mpm_prefork): http://httpd.apache.org/docs/current/mod/prefork.html
+# * Green Unicorn: http://gunicorn.org/
+# * Starman: http://search.cpan.org/dist/Starman/
+# * unicorn: http://unicorn.bogomips.org/
+# However, Yahns::ProxyPass is compatible with any HTTP/1.x backends.
+# It will even benefit those which rely on heavier thread-per-client
+# designs such as Varnish <https://www.varnish-cache.org/> as
+# yahns supports infinitely-lived persistent connections.
+# Unlike most Rack applications, Yahns::ProxyPass relies on rack.hijack
+# support from yahns and does not work outside of yahns.
+# example usage in a rack config.ru file to proxy to a backend server
+# running on port 6081 over the loopback interface:
+#     require 'yahns/proxy_pass'
+#     run Yahns::ProxyPass.new('')
+# Yahns::ProxyPass is NOT currently a load-balancer.  It will only
+# route requests to one backend server.  However, the backend server
+# could be a load balancer itself; such as Varnish or
+# HAProxy <http://www.haproxy.org/>
+class Yahns::ProxyPass
   class ReqRes < Kgio::Socket # :nodoc:
     attr_writer :resbuf
     attr_accessor :proxy_trailers
@@ -176,7 +218,29 @@ def send_req_buf(buf)
   end # class ReqRes
-  def initialize(dest, opts = {})
+  # +dest+ must be an HTTP URL with optional variables prefixed with '$'.
+  # +dest+ may refer to the path to a Unix domain socket in the form:
+  #
+  #     unix:/absolute/path/to/socket
+  #
+  # Variables which may be used in the +dest+ parameter include:
+  #
+  # - $url - the entire URL used to make the request
+  # - $path - the unescaped PATH_INFO of the HTTP request
+  # - $fullpath - $path with QUERY_STRING
+  # - $host - the hostname in the Host: header
+  #
+  # For Unix domain sockets, variables may be separated from the
+  # socket path via: ":/".  For example:
+  #
+  #     unix:/absolute/path/to/socket:/$host/$fullpath
+  #
+  # Currently :response_headers is the only +opts+ supported.
+  # :response_headers is a Hash containing a from => to mapping
+  # of response headers.  The special value of +:ignore+ indicates
+  # the header from the backend HTTP server will be ignored instead
+  # of being blindly passed on to the client.
+  def initialize(dest, opts = { response_headers: { 'Server' => :ignore } })
     case dest
     when %r{\Aunix:([^:]+)(?::(/.*))?\z}
       path = $2
@@ -197,7 +261,7 @@ def initialize(dest, opts = {})
-  def init_path_vars(path)
+  def init_path_vars(path) # :nodoc:
     path ||= '$fullpath'
     # methods from Rack::Request we want:
     allow = %w(fullpath host_with_port host port url path)
@@ -210,7 +274,7 @@ def init_path_vars(path)
     @path = path.gsub(%r{\A/(\$(?:fullpath|path))}, '\1')
-  def call(env)
+  def call(env) # :nodoc:
     # 3-way handshake for TCP backends while we generate the request header
     rr = ReqRes.start(@sockaddr)
     c = env['rack.hijack'].call

             reply	other threads:[~2016-02-20  8:16 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-02-20  8:16 Eric Wong [this message]
2016-02-20  8:37 ` Lin Jen-Shin (godfat)
2016-02-20  9:41   ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://yhbt.net/yahns/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160220081619.GA10850@dcvr.yhbt.net \
    --to=e@80x24.org \
    --cc=yahns-public@yhbt.net \
    --subject='Re: [RFC] proxy_pass: document as public API' \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).