range requests and multipart responses

range requests and multipart responses

[Eric Wong]

Hi all,

Does anybody have real-world use cases of clients using Range: requests
with multiple comma-delimited byte ranges (and thus expecting multipart
responses)?  If not, I'm not going to bloat Rainbows! with it for now[1].

While being able to specify a single range is very useful for resuming
failed downloads and for things who want to "peek" at magic patterns
before requesting the rest of a file, I can't think of good use cases
for having to deal with multipart responses.

Even if you were writing a FUSE-based filesystem over HTTP, it wouldn't
even be useful for implementing preadv(2) since you can only specify a
single offset there.

Maybe if you mmap() a FUSE-ed file and two threads access the same file
in different parts at the *exact*same*time* the FUSE layer could be
smart enough to make a single HTTP request instead of two? :)


[1] - For the performance-minded, it's also tough to implement multipart
      responses efficiently using any sendfile() implementation.
      Our TCP_CORK support is far from optimal under Linux right now,
      and not using TCP_CORK would be even less efficient with extra
      multipart headers.  AFAIK, sendfilev() isn't available outside of
      Solaris, either.

-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: range requests and multipart responses

[James Tucker]

On 20 Jul 2010, at 07:18, Eric Wong wrote:

> Hi all,
> 
> Does anybody have real-world use cases of clients using Range: requests
> with multiple comma-delimited byte ranges (and thus expecting multipart
> responses)?  If not, I'm not going to bloat Rainbows! with it for now[1].
> 
> While being able to specify a single range is very useful for resuming
> failed downloads and for things who want to "peek" at magic patterns
> before requesting the rest of a file, I can't think of good use cases
> for having to deal with multipart responses.
> 
> Even if you were writing a FUSE-based filesystem over HTTP, it wouldn't
> even be useful for implementing preadv(2) since you can only specify a
> single offset there.
> 
> Maybe if you mmap() a FUSE-ed file and two threads access the same file
> in different parts at the *exact*same*time* the FUSE layer could be
> smart enough to make a single HTTP request instead of two? :)
> 
> 
> [1] - For the performance-minded, it's also tough to implement multipart
>      responses efficiently using any sendfile() implementation.
>      Our TCP_CORK support is far from optimal under Linux right now,
>      and not using TCP_CORK would be even less efficient with extra
>      multipart headers.  AFAIK, sendfilev() isn't available outside of
>      Solaris, either.

Does cork even help in real world use cases?

> 
> -- 
> Eric Wong
> _______________________________________________
> Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
> http://rubyforge.org/mailman/listinfo/rainbows-talk
> Do not quote signatures (like this one) or top post when replying

_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: range requests and multipart responses

[Eric Wong]

James Tucker <jftucker-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On 20 Jul 2010, at 07:18, Eric Wong wrote:
> > [1] - For the performance-minded, it's also tough to implement multipart
> >      responses efficiently using any sendfile() implementation.
> >      Our TCP_CORK support is far from optimal under Linux right now,
> >      and not using TCP_CORK would be even less efficient with extra
> >      multipart headers.  AFAIK, sendfilev() isn't available outside of
> >      Solaris, either.
> 
> Does cork even help in real world use cases?

Heh, good question :)

I've only seen it help in microbenchmarks and analyzing tcpdump output.
I've never bothered checking in Ruby apps (never though it'd be worth my
time, even), but I don't expect it to be worth the effort.  Maybe
somebody else will care enough...

However, I've seen corking hurt badly for things like chat apps and
anything with bidirectional small messages.  It's impossible for the
server to reliably tell what type of traffic to expect without the app
stepping in to tell it to uncork at certain points.

-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: range requests and multipart responses

[russell muetzelfeldt]

On 20/07/2010, at 02:18:03 EDT, Eric Wong wrote:

> Does anybody have real-world use cases of clients using Range: requests
> with multiple comma-delimited byte ranges (and thus expecting multipart
> responses)?  If not, I'm not going to bloat Rainbows! with it for now[1].

I don't have evidence to hand, but I'm pretty sure I've seen the remote (http-served) disk image support in OSX's hdid(8) do this. Whether that's a use case you care about is another question entirely.

cheers

Russell

_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying