librelist archives

« back to archive

Another upload question

Another upload question

From:
tsuraan
Date:
2010-12-27 @ 15:33
There are two features that would make mongrel2 ideal for my purposes.
 One of them is chunked upload support (which is being discussed in a
different thread on this list).  The other one is having a function
like upload.c/stream_to_disk that instead of streaming the files to
the web server's local storage would send the chunks through zeromq so
that a backend handler can deal with them.  The reason for this is
that my web servers basically run diskless, and uploaded files can
just be analyzed as streams of data, so they don't even need to be
stored (at least not on the web server's storage).  This change looks
pretty easy, and I'm planning on writing it and submitting it for
review in the next day or two, but before I do that I thought I'd ask
whether anybody else is already working on it.  So, is anybody
currently putting together an upload handler that sends the uploaded
chunks through zeromq rather than storing them on the local web
server's hard drives?  An, pre-emptively, does this sound like a
terrible idea?

Re: [mongrel2] Another upload question

From:
Zed A. Shaw
Date:
2010-12-27 @ 17:57
On Mon, Dec 27, 2010 at 09:33:16AM -0600, tsuraan wrote:
> There are two features that would make mongrel2 ideal for my purposes.
>  One of them is chunked upload support (which is being discussed in a
> different thread on this list).  The other one is having a function
> like upload.c/stream_to_disk that instead of streaming the files to
> the web server's local storage would send the chunks through zeromq so
> that a backend handler can deal with them.  The reason for this is
> that my web servers basically run diskless, and uploaded files can
> just be analyzed as streams of data, so they don't even need to be
> stored (at least not on the web server's storage).  This change looks
> pretty easy, and I'm planning on writing it and submitting it for
> review in the next day or two, but before I do that I thought I'd ask
> whether anybody else is already working on it.  So, is anybody
> currently putting together an upload handler that sends the uploaded
> chunks through zeromq rather than storing them on the local web
> server's hard drives?  An, pre-emptively, does this sound like a
> terrible idea?

Nope, but yeah if you're doing CE (after you double and triple check
it's not something you're doing) then this would be the next logical
step.  You'd need to have some configuration options for it, but those
could go in settings similar to the temp file settings.

Oh man, I can just see the abuse of this coming. :-)

-- 
Zed A. Shaw
http://zedshaw.com/