unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: Lawrence Pit <lawrencepit@icloud.com>
Cc: unicorn-public@bogomips.org
Subject: Re: unicorn log attack?
Date: Sat, 30 Jan 2016 09:34:53 +0000	[thread overview]
Message-ID: <20160130093453.GA24510@dcvr.yhbt.net> (raw)
In-Reply-To: <56AAAD0A.8000807@icloud.com>

Lawrence Pit <lawrencepit@icloud.com> wrote:
> Hi Eric,
> 
> I'm writing to you directly instead of to the unicorn-public list.

Which got your HTML email tarpitted in my spam folder instead of
being bounced back right away so you could fix it :)

> I noticed yesterday our unicorn.log files, which are usually tiny,
> were gigantic in size. Fortunately, this was caused by a friendly
> attack, but had they persisted I think we would've run out of
> diskspace (of which we would've been warned in advance, so we
> could've dealt with the situation I suppose had it happened)
> 
> Upon inspection it seems requests were received as shown below (
> I've cut out the middle part of the value that was part of the form
> body that was posted )
> 
> The log statement is printed out by unicorn.rb method +log_error+.
> 
> I'm not sure this is a unicorn issue, and thinking more an issue of
> how we developers should deal with repeatedly receiving the same
> sort of (sometimes very large) exceptions? Any advice?

Right, not a unicorn issue :)

Use logrotate or similar, compress your logs frequently, be
mindful of what you dump from your app; and watch your disk
usage (which you seem to be doing already), but that includes
emails :)

In ancient times (perhaps it was the Mongrel days), the server
itself would dump the contents of bad HTTP requests for
debugging; but given the amount of probes/scans I saw: it wasn't
worth it.  We don't even log things like aborted/dropped
connections.

Since the backtrace below clearly shows the error happened from
something your application was doing; I don't consider it the
responsibility of the app server to sanitize it.

> ps. if you want to reply via the list that's fine by me.

Done :) but I've shortened the backtrace for readability

> E, [2016-01-26T11:23:49.499928 srv23 28932] unicorn: app error:
> invalid %-encoding (stri%26%23%30%30%32%

<snip>

> [ CUT VERY LARGE VALUE ]

<snip :>

> (ArgumentError)
> /usr/local/lib/ruby/2.2.0/uri/common.rb:382:in
> `decode_www_form_component'
> /rack-1.5.5/lib/rack/utils.rb:42:in
> `unescape'
> /rack-1.5.5/lib/rack/utils.rb:105:in
> `block (2 levels) in parse_nested_query'

<snip>  These definitely aren't called from code in unicorn itself.

> /rack-1.5.5/lib/rack/urlmap.rb:50:in
> `each'
> /rack-1.5.5/lib/rack/urlmap.rb:50:in
> `call'
> /unicorn-5.0.1/lib/unicorn/http_server.rb:562:in
> `process_client'
> /unicorn-5.0.1/lib/unicorn/oob_gc.rb:70:in
> `process_client'
> /unicorn-5.0.1/lib/unicorn/http_server.rb:658:in
> `worker_loop'

       reply	other threads:[~2016-01-30  9:34 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <56AAAD0A.8000807@icloud.com>
2016-01-30  9:34 ` Eric Wong [this message]
2016-02-01  5:04   ` Lawrence Pit
2016-02-01  9:57     ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://yhbt.net/unicorn/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160130093453.GA24510@dcvr.yhbt.net \
    --to=e@80x24.org \
    --cc=lawrencepit@icloud.com \
    --cc=unicorn-public@bogomips.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this inbox:

	../../unicorn.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).