From: Laas Toom <laas@toom.ee>
To: unicorn list <mongrel-unicorn@rubyforge.org>
Subject: Re: Is a client uploading a file a slow client from unicorn's point of view?
Date: Wed, 10 Oct 2012 02:06:55 +0300 [thread overview]
Message-ID: <3EDF9F45-B5FB-4C3B-B0E5-37C28ECFE1D1@toom.ee> (raw)
In-Reply-To: <20121009200306.GA17437@dcvr.yhbt.net>
On 09.10.2012, at 23:03, Eric Wong <normalperson@yhbt.net> wrote:
>> 2) make the upload progress available, so e.g. AJAX-powered upload forms can show progressbar, which is really neat. No need for Flash-based uploaders.
>
> It does? I'm not seeing it in the documentation, but I know there's
> a separate upload progress module, though:
> http://wiki.nginx.org/HttpUploadProgressModule
I must have mixed them up then. :-)
>> I have a Rails app that accepts media uploads. All the processing happens in background, front-end handles only the actual upload and stores it to disk.
>> But with uploads as large as 1.4 GB, I've seen Rails response times > 200 secs. This starts to give timeouts in weird places.
>
> Yikes. I assume you're constrained by disk I/O there?
Might be. Additionally, the disk is SAN, so network activity there too.
> For some of the large file situations under Linux, I find it beneficial
> to lower the dirty_*ratio/*bytes drastically to avoid large, sudden
> bursts of disk activity and instead favor smaller writes. I get lower
> throughput, but more consistent performance.
I shall look into it when I get to fixing this issue.
>> Afterwards it will only handle out the file location and Rails can
>> complete it's work a lot faster, freeing up workers.
>>
>> Unicorn won't even see the file and Rails has the responsibility to
>> delete the file if it's invalid.
>
> I think the only problem with this approach is it won't work well on
> setups where nginx is on separate machines from unicorn. Shared
> storage would be required, but that ends up adding to network I/O,
> too...
But won't (almost) the same network I/O be evident anyway, because of nginx transferring the data to Unicorn over network (as they are on different machines)?
Thanks for clarifying,
Laas
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
next prev parent reply other threads:[~2012-10-09 23:07 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-09 0:39 Is a client uploading a file a slow client from unicorn's point of view? Jimmy Soho
2012-10-09 1:58 ` Eric Wong
2012-10-09 6:31 ` Laas Toom
2012-10-09 20:03 ` Eric Wong
2012-10-09 23:06 ` Laas Toom [this message]
2012-10-09 23:54 ` Eric Wong
2012-10-10 6:59 ` Laas Toom
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/unicorn/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3EDF9F45-B5FB-4C3B-B0E5-37C28ECFE1D1@toom.ee \
--to=laas@toom.ee \
--cc=mongrel-unicorn@rubyforge.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).