From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-2.8 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, URIBL_BLOCKED,WEIRD_PORT shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: yahns-public@yhbt.net Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id CD18420424; Sat, 20 Feb 2016 08:16:19 +0000 (UTC) Date: Sat, 20 Feb 2016 08:16:19 +0000 From: Eric Wong To: yahns-public@yhbt.net Subject: [RFC] proxy_pass: document as public API Message-ID: <20160220081619.GA10850@dcvr.yhbt.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline List-Id: We've been using this for nearly a year to serve our own list archives using (Apache (mpm_prefork) || Starman) and Varnish-cache on: http://yhbt.net/yahns-public/ --- Any thoughts? So far I've resisted having a public API. On the other hand, the current state of ProxyPass being a Rack app using rack.hijack still has nasty limitations such as incompatibility with any existing middleware. Something simple such as access logs won't work well (e.g. Rack::CommonLogger or Clogger) On the other hand, it would be nice to have a mostly-Ruby alternative to nginx today... Any API or configuration directive publically documented for yahns should remain supported forever (or as long as underlying components such as Rack/Ruby/HTTP are), regardless of version changes. There may be exceptions for directives marked as "experimental", but it's not the case, here. So I'm also considering a special configuration directive and making it independent of Rack. In other words, something like: app(:proxy_pass) ... as opposed to (the currently-implemented): app(:rack) ... in yahns_config.5 But that also requires writing more code to support... lib/yahns.rb | 7 ++++- lib/yahns/proxy_pass.rb | 74 +++++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 75 insertions(+), 6 deletions(-) diff --git a/lib/yahns.rb b/lib/yahns.rb index 2be7930..a6b15d7 100644 --- a/lib/yahns.rb +++ b/lib/yahns.rb @@ -15,9 +15,14 @@ Unicorn.__send__(:remove_const, sym) if Unicorn.const_defined?(sym) end -# yahns exposes no user-visible API outside of the config file. +# yahns exposes little user-visible API outside of the config file. # See http://yahns.yhbt.net/yahns_config.txt for the config documentation # and http://yahns.yhbt.net/ for the homepage. +# +# Currently, the only supported Ruby API provided by yahns is: +# +# Yahns::ProxyPass +# # Internals are subject to change. module Yahns diff --git a/lib/yahns/proxy_pass.rb b/lib/yahns/proxy_pass.rb index 511db02..d6df4a9 100644 --- a/lib/yahns/proxy_pass.rb +++ b/lib/yahns/proxy_pass.rb @@ -2,15 +2,57 @@ # Copyright (C) 2013-2016 all contributors # License: GPL-3.0+ (https://www.gnu.org/licenses/gpl-3.0.txt) # frozen_string_literal: true + require 'socket' require 'kgio' require 'kcar' # gem install kcar require 'rack/request' -require 'timeout' +require 'timeout' # only for Timeout::Error require_relative 'proxy_http_response' -class Yahns::ProxyPass # :nodoc: +# Yahns::ProxyPass is a Rack (hijack) application which allows yahns to +# act as a fully-buffering reverse proxy to protect backend servers from +# slow HTTP clients. +# +# Yahns::ProxyPass relies on the default behavior of yahns to implement +# full input and output buffering. Output buffering is lazy, meaning it +# allows streaming output in the best case and will only buffer if the +# client cannot keep up with the server. +# +# The goal of this reverse proxy is to act as a sponge on the same LAN +# or host as the backend HTTP server incapable of handling slow clients. +# Yahns::ProxyPass accomplishes this by handling all the slow I/O +# internally within yahns itself to minimize time spent in the backend +# HTTP server waiting on slow I/O. +# +# Examples of backend HTTP servers which benefit from having a +# fully-buffering reverse proxy include: +# +# * Apache (mpm_prefork): http://httpd.apache.org/docs/current/mod/prefork.html +# * Green Unicorn: http://gunicorn.org/ +# * Starman: http://search.cpan.org/dist/Starman/ +# * unicorn: http://unicorn.bogomips.org/ +# +# However, Yahns::ProxyPass is compatible with any HTTP/1.x backends. +# It will even benefit those which rely on heavier thread-per-client +# designs such as Varnish as +# yahns supports infinitely-lived persistent connections. +# +# Unlike most Rack applications, Yahns::ProxyPass relies on rack.hijack +# support from yahns and does not work outside of yahns. +# +# example usage in a rack config.ru file to proxy to a backend server +# running on port 6081 over the loopback interface: +# +# require 'yahns/proxy_pass' +# run Yahns::ProxyPass.new('http://127.0.0.1:6081') +# +# Yahns::ProxyPass is NOT currently a load-balancer. It will only +# route requests to one backend server. However, the backend server +# could be a load balancer itself; such as Varnish or +# HAProxy +class Yahns::ProxyPass class ReqRes < Kgio::Socket # :nodoc: attr_writer :resbuf attr_accessor :proxy_trailers @@ -176,7 +218,29 @@ def send_req_buf(buf) end end # class ReqRes - def initialize(dest, opts = {}) + # +dest+ must be an HTTP URL with optional variables prefixed with '$'. + # +dest+ may refer to the path to a Unix domain socket in the form: + # + # unix:/absolute/path/to/socket + # + # Variables which may be used in the +dest+ parameter include: + # + # - $url - the entire URL used to make the request + # - $path - the unescaped PATH_INFO of the HTTP request + # - $fullpath - $path with QUERY_STRING + # - $host - the hostname in the Host: header + # + # For Unix domain sockets, variables may be separated from the + # socket path via: ":/". For example: + # + # unix:/absolute/path/to/socket:/$host/$fullpath + # + # Currently :response_headers is the only +opts+ supported. + # :response_headers is a Hash containing a from => to mapping + # of response headers. The special value of +:ignore+ indicates + # the header from the backend HTTP server will be ignored instead + # of being blindly passed on to the client. + def initialize(dest, opts = { response_headers: { 'Server' => :ignore } }) case dest when %r{\Aunix:([^:]+)(?::(/.*))?\z} path = $2 @@ -197,7 +261,7 @@ def initialize(dest, opts = {}) init_path_vars(path) end - def init_path_vars(path) + def init_path_vars(path) # :nodoc: path ||= '$fullpath' # methods from Rack::Request we want: allow = %w(fullpath host_with_port host port url path) @@ -210,7 +274,7 @@ def init_path_vars(path) @path = path.gsub(%r{\A/(\$(?:fullpath|path))}, '\1') end - def call(env) + def call(env) # :nodoc: # 3-way handshake for TCP backends while we generate the request header rr = ReqRes.start(@sockaddr) c = env['rack.hijack'].call -- EW