diff options
Diffstat (limited to 'site/src/docs/lighttpd.page')
| -rw-r--r-- | site/src/docs/lighttpd.page | 278 |
1 files changed, 278 insertions, 0 deletions
diff --git a/site/src/docs/lighttpd.page b/site/src/docs/lighttpd.page new file mode 100644 index 0000000..973fd2c --- /dev/null +++ b/site/src/docs/lighttpd.page @@ -0,0 +1,278 @@ +--- +title: Lighttpd +inMenu: true +directoryName: Documentation +--- + +*I'm sad to say that I have to recommend people not use lighttpd anymore.* +The author hasn't updated the mod_proxy plugin and isn't providing too much +support for the bugs it has. If you're running lighttpd and you constantly +see 500 errors that are never recovered, then you should switch to Apache +or another web server that can handle properly balancing backends. + + +h1. Using Lighttpd With Mongrel + +It is possible to host an application with just Mongrel since it is +able to serve files like a normal web server. Still, no matter +how fast Mongrel gets it probably can't compete with something +like lighttpd for serving static files. Because of this I've +devised a simple way to setup a lighttpd+Mongrel setup that +demonstrates clustering four Mongrel servers running the same +application as backends. + +This is very similar to a FastCGI or SCGI setup except that +you're just using regular HTTP. Read through the "HOWTO":howto.html +for information on other possible deployment scenarios. + + +h2. The Goal + +What we want to do is put a lighttpd on the internet and then +have it proxy back to one of four Mongrel servers. + +!SimpleLighttpdMongrelSetup.jpg! + +This is actually really trivial and probably doesn't need a diagram +but I got bored just writing it up. + +How it all works is pretty simple: + +# Requests come to the lighttpd server. +# Lighttpd takes each request, and sends it to a backend depending + on how you configure it: + * hash -- Hashes the request URI and makes sure that it goes to the same backend. + * round-robin -- Just chooses another host for each request. + * fair -- "Load based, passive balancing." No idea what that means, but if it's like + the rest of lighttpd it probably means it will overload the first one and if that one's + busy then it starts using the next ones. +# Each backend doesn't really care about any of this since it's just a web server. + + +h2. Lighttpd Configuration + +For lighttpd you need to have *mod_proxy* in your server.modules setting: + + server.modules = ( "mod_rewrite", "mod_redirect", + "mod_access", "mod_accesslog", "mod_compress", + "mod_proxy") + +Then you need to tell lighttpd where the other backends are located: + + proxy.balance = "fair" + proxy.server = ( "/" => + ( ( "host" => "127.0.0.1", "port" => 8001 ), + ( "host" => "127.0.0.1", "port" => 8002 ), + ( "host" => "127.0.0.1", "port" => 8003 ), + ( "host" => "127.0.0.1", "port" => 8004 ) ) ) + +When I used lighttpd 1.4.9 and set proxy.balance="round-robin" I had an excessive number of +500 errors for no real reason. The "fair" setting seems to be the best, but if you +have a large number of fairly random URIs you should try "hash" too. + +*For the rest of this tutorial we'll assume you're running lighttpd on port 80.* + + +h2. Mongrel Configuration + +Mongrel is pretty easy to setup with this configuration on either Win32 or Unix, but +since lighttpd doesn't compile so easily on Win32 I'll just show the Unix method +for starting it: + + $ mongrel_rails start -d -p 8001 \ + -e production -P log/mongrel-1.pid + $ mongrel_rails start -d -p 8002 \ + -e production -P log/mongrel-2.pid + $ mongrel_rails start -d -p 8003 \ + -e production -P log/mongrel-3.pid + $ mongrel_rails start -d -p 8004 \ + -e production -P log/mongrel-4.pid + +Now you should be able to hit your web server at port 80 and it'll run against +one of your four Mongrels. + + +h2. Testing Stability and Performance + +As I mentioned before proxy.balance="round-robin" had many stability issues +in lighttpd 1.4.9 but how did I figure this out? Here's how you can do it. + +You use "httperf":http://www.hpl.hp.com/research/linux/httperf/ to first hit each +Mongrel backend with a large request set. + + $ httperf --port 8001 --server 127.0.0.1 \ + --num-conns 300 --uri /test + $ httperf --port 8002 --server 127.0.0.1 \ + --num-conns 300 --uri /test + $ httperf --port 8003 --server 127.0.0.1 \ + --num-conns 300 --uri /test + $ httperf --port 8004 --server 127.0.0.1 \ + --num-conns 300 --uri /test + +After each of these you're looking for the *Connection rate*, *Request rate*, +*Reply rate*, and *Reply status*. You first look at the *Reply status* to make +sure that you got all 2xx messages. Then look at the other three and make +sure they are about the same. + +Then you hit lighttpd with a similar request set to confirm that it handles the base case. + + $ httperf --port 80 --server 127.0.0.1 \ + --num-conns 300 --uri /test + +You should get no 5xx errors. In the case of round-robin there were about 60% +5xx errors even though the Mongrels were functioning just fine. The "hash" method +didn't improve this test's performance since there's only on URI in the test. It +seems the "fair" method is the best you can do right now. + +Finally you hit lighttpd with a 4x rate to see if it could handle the theoretical limit. + + $ httperf --port 80 --server 127.0.0.1 \ + --num-conns 10000 --uri /test --rate 600 + +It will most likely fail miserably and you'll probably see a few 5xx counts in the +*Reply status* line but that's normal. What you're looking to do is keep moving +--rate and --num-conns up/down until you get where the server is just barely +able to accept the requests without slowing down (i.e. your *Request rate* matches +your --rate setting). There will be a point where adding literally one more +to your --rate setting causes the Request rate to tank. That's your setup's breaking +point and is the actual requests/second you can handle. + + +h1. Insane Caching Power Magnet + +Mongrel (as of 0.3.7) by default supports Rails style page caching +in the RailsHandler it uses to serve your applications. What this +means is that if you do a page cached action (which writes a +.html file as well as respond) then Mongrel will just serve up +the cached page instead of bug Rails. + +This does give you a large boost in performance, but still not nearly +as much as if you had lighttpd doing the caching for you. The optimal +configuration would be where lighttpd checks for cached pages and then +served it directly as it already does with FastCGI. + +There are some technical problems with this and the lighttpd mod_proxy, +but thankfully I don't have to go into them because lighttpd now supports +the "power magnet" and Cache Meta Language (CML). CML is a small bit +of the Lua programming language that lets you script lighttpd and tell +it when to cache or not. The power magnet feature of CML lets you put +all requests through one CML script so that you can determine whether +to cache or not. + +h2. Configuration + +In order to get everything to work right you'll need a few pieces +of equipment and make sure they are enabled in your lighttpd build. +The sample build was done on Debian so that everything would work +including mod_rewrite, mod_memcache, mod_cml, and mod_redirect. + +* lua50, liblua50-dev +* libpcre3-dev +* memcached +* Finally make sure you configure with ./configure --with-lua --with-memcache + +Debian people will need to follow these instructions from +"www.debian-administration.org":http://www.debian-administration.org/articles/20 +for installing from source, and will also need to add these lines to your +/etc/apt/sources.list: + + deb-src http://http.us.debian.org/debian stable main contrib non-free + deb-src http://http.us.debian.org/debian unstable main contrib non-free + deb-src http://non-us.debian.org/debian-non-US stable/non-US main contrib non-free + +And then do the following: + +# Make sure you don't have the source extracted in your current directory. +# apt-get install devscripts debhelper build-essential fakeroot +# apt-get update +# apt-get source lighttpd +# nano -w lighttpd-1.4.10/debian/rules +## DEB_CONFIGURE_EXTRA_FLAGS need have at the end: --with-lua --with-memcache +# nano -w lighttpd-1.4.10/debian/lighttpd.install +## debian/tmp/usr/lib/lighttpd/mod_cml.so needs to be added here. + +Then you're supposed to be able to follow the debian-administration +docs but I haven't got it all working yet. Please clue me in +if you get this compiled on Debian. + +h3. lighttpd.conf + +If it all builds right then you'll be able to use this nifty +CML script that Bradley K. Taylor (from railsmachine.net) came +up with. First you need to tweak your lighttpd.conf at the +place where you have the mod_proxy setup from previous: + + $HTTP["host"] == "www.myhost.come" { + server.document-root = "/my/path/to/app/public" + cml.power-magnet = "/my/path/to/app/config/power-magnet.cml" + proxy.balance = "fair" + proxy.server = ( "/" => ( ( "host" => "127.0.0.1", "port" => 8001 ), + ( "host" => "127.0.0.1", "port" => 8002 ) ) ) + } + +This one just has two for simplicity. The big thing is the +document root setting and the power-magnet.cml setting. I wouldn't +put the power-magnet.cml in your public directory. + +h3. power-magnet.cml + +Now for the magic bit of Lua code that tells lighttpd what to do: + + dr = request["DOCUMENT_ROOT"] + + if file_isreg(dr .. "maintainance.html") then + output_include = { dr .. "maintainance.html" } + return CACHE_HIT + end + + f = request["REQUEST_URI"] + + if f == "/" or f == "" then + file = dr .. "index.html" + elseif not string.find(f, "%.") then -- rewrite for cached pages + file = dr .. f .. ".html" + else + file = dr .. f + end + + if file_isreg(file) then + output_include = { file } + return CACHE_HIT + end + + return 1 -- should be CACHE_MISS, but there's a bug in 1.4.10 + +Place this in the /my/path/to/app/config/power-magnet.cml like you +configured above. + +Now if you restart lighttpd and everything worked right you should +be able to see the headers and tell if Mongrel or lighttpd is serving +them. Use curl like this: + + curl -I http://zedapp.railsmachine.net/ + curl -I http://zedapp.railsmachine.net/admin + +The second one should redirect and show a Mongrel header while the +first one should show a lighttpd header. + + +h2. Memcached + +The next installment of this document will tell you how to setup a +memcached so that you can run the lighttpd on a different server +from the Mongrel cluster. Right now they all have to reside +on one machine. + +h2. CML Tricks + +Since Lua is a full blown and fast little language +you can get pretty creative with it. For example you could +have it check dates and times of files, look for processes +that should be running, run commands, check the contents of +a file, etc. + +Take a look at the "Lua Reference Manual":http://www.lua.org/manual/5.0/ +to see what it can do. Ruby people will probably like Lua. + + + |