librelist archives

home // archives //

« back to archive

Questions on uploading files on mongrel2

Questions on uploading files on mongrel2

From:
Tang Daogang
Date:
2011-01-27 @ 07:07 
Dear Shaw,

I find two ways about uploading files on mongrel2 you described.

One is in Mongrel2 Manual - 5.5 Async File Upload Demo (Page 37), which says
the uploaded file will be stored to a temporary file, and send message to
handler at start and end. The handler will get the temporary file's
location, then it can read directly or do some actions on it.

The other one is on your blog: Using Tir's Tasks For Async Photo Uploads (
http://sheddingbikes.com/posts/1292063326.html ). It gets file content from
form body. And I look into the form.lua in Tir, find that it can deal with
file uploads too.

Then, which way is better? which way is easier? Why two methods? This
confuses me .....

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Loic d'Anterroches
Date:
2011-01-27 @ 07:57 
Hello,

> I find two ways about uploading files on mongrel2 you described.
>
> One is in Mongrel2 Manual - 5.5 Async File Upload Demo (Page 37), which says
> the uploaded file will be stored to a temporary file, and send message to
> handler at start and end. The handler will get the temporary file's
> location, then it can read directly or do some actions on it.
>
> The other one is on your blog: Using Tir's Tasks For Async Photo Uploads (
> http://sheddingbikes.com/posts/1292063326.html ). It gets file content from
> form body. And I look into the form.lua in Tir, find that it can deal with
> file uploads too.
>
> Then, which way is better? which way is easier? Why two methods? This
> confuses me .....

As I have been dealing with this one for my framework, here is my take 
on it. So you two methods: full zmq message or two messages with temp 
file. Good/Bad is more Pro/Contra, nothing is really good nor bad, it is 
always a question of trade-off depending of your expected usage.

# Full zmq message

1. Good: your handler logic is simple, you get a request with everything 
in it, you parse it, you use it.
2. Good: your handlers can be anywhere on your network.
3. Bad: your handler can receive a 100MB zmq message if you have 
somebody uploading a big file, so you need to take care of that.
4. Bad: you need to have a smart way to parse the request payload not to 
kill your handler memory.
5. Bad: parsing of the request use "handler" time and Mongrel2 is 
probably more effecient in doing so.

# Two messages with temp file

1. Good: your zmq messages will always be small, you have less memory 
handling to think about.
2. Good: you can stop the upload right at the start by checking the 
headers (your handler can directly kill a connection if for example some 
auth cookies are not there, this before the upload is done).
3. Bad: The payload is on disk, your handler need to be able to access 
it (shared nfs if your handlers are not on the same box).
4. Bad: you still need to parse the payload but at least you have more 
knowledge.
5. Bad: the handler logic is a bit more complex as for the same request 
you get 2 hits against your handler.

In my framework, I decided to go the full zmq message way as I want my 
handlers to easily run on different boxes. So, to limit the memory usage 
and to be smart with respect to memory, I directly push the request 
message in a buffered stream. That is, I keep in memory if less than 
5MB, else, I swap on disk:


http://projects.ceondo.com/p/photon/source/tree/develop/src/photon/mongrel2.php#L153

After that, the multipart parser, operate in a smart way to read from 
the stream only if needed:


http://projects.ceondo.com/p/photon/source/tree/develop/src/photon/http/multipartparser.php

That way, I know that a mongrel2 request will be have a controlled max 
memory usage.

I hope it can help you understanding the handling of the file upload (or 
in fact any request with a large payload).

loïc

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Tang Daogang
Date:
2011-01-27 @ 08:12 
Loic, thank you.

Just now, I test the tmp file way. I can't find the tmp file in /tmp, why?
How to configure it?

My config file is:
arc = Handler(send_spec='ipc://run/t0_in',
                send_ident='e884a439-31be-4f74-8050-a93565795b25',
                recv_spec='ipc://run/t0_out', recv_ident='')

main = Server(
    uuid="505417b8-1de4-454f-98b6-07eb9225cca1"
    access_log="/logs/access.log"
    error_log="/logs/error.log"
    chroot="./"
    pid_file="/run/mongrel2.pid"
    default_host="(.+)"
    name="main"
    port=6767
    hosts=[ Host(name="(.+)", routes={ '/arc': arc }) ]
)

settings = { "zeromq.threads": 1, 'limits.content_length': 10485760,
'upload.temp_store': '/tmp/' }

servers = [main]



On Thu, Jan 27, 2011 at 3:57 PM, Loic d'Anterroches <loic@ceondo.com> wrote:

> Hello,
>
> > I find two ways about uploading files on mongrel2 you described.
> >
> > One is in Mongrel2 Manual - 5.5 Async File Upload Demo (Page 37), which
> says
> > the uploaded file will be stored to a temporary file, and send message to
> > handler at start and end. The handler will get the temporary file's
> > location, then it can read directly or do some actions on it.
> >
> > The other one is on your blog: Using Tir's Tasks For Async Photo Uploads
> (
> > http://sheddingbikes.com/posts/1292063326.html ). It gets file content
> from
> > form body. And I look into the form.lua in Tir, find that it can deal
> with
> > file uploads too.
> >
> > Then, which way is better? which way is easier? Why two methods? This
> > confuses me .....
>
> As I have been dealing with this one for my framework, here is my take
> on it. So you two methods: full zmq message or two messages with temp
> file. Good/Bad is more Pro/Contra, nothing is really good nor bad, it is
> always a question of trade-off depending of your expected usage.
>
> # Full zmq message
>
> 1. Good: your handler logic is simple, you get a request with everything
> in it, you parse it, you use it.
> 2. Good: your handlers can be anywhere on your network.
> 3. Bad: your handler can receive a 100MB zmq message if you have
> somebody uploading a big file, so you need to take care of that.
> 4. Bad: you need to have a smart way to parse the request payload not to
> kill your handler memory.
> 5. Bad: parsing of the request use "handler" time and Mongrel2 is
> probably more effecient in doing so.
>
> # Two messages with temp file
>
> 1. Good: your zmq messages will always be small, you have less memory
> handling to think about.
> 2. Good: you can stop the upload right at the start by checking the
> headers (your handler can directly kill a connection if for example some
> auth cookies are not there, this before the upload is done).
> 3. Bad: The payload is on disk, your handler need to be able to access
> it (shared nfs if your handlers are not on the same box).
> 4. Bad: you still need to parse the payload but at least you have more
> knowledge.
> 5. Bad: the handler logic is a bit more complex as for the same request
> you get 2 hits against your handler.
>
> In my framework, I decided to go the full zmq message way as I want my
> handlers to easily run on different boxes. So, to limit the memory usage
> and to be smart with respect to memory, I directly push the request
> message in a buffered stream. That is, I keep in memory if less than
> 5MB, else, I swap on disk:
>
>
> 
http://projects.ceondo.com/p/photon/source/tree/develop/src/photon/mongrel2.php#L153
>
> After that, the multipart parser, operate in a smart way to read from
> the stream only if needed:
>
>
> 
http://projects.ceondo.com/p/photon/source/tree/develop/src/photon/http/multipartparser.php
>
> That way, I know that a mongrel2 request will be have a controlled max
> memory usage.
>
> I hope it can help you understanding the handling of the file upload (or
> in fact any request with a large payload).
>
> loïc
>
>

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Loic d'Anterroches
Date:
2011-01-27 @ 08:18 
Hello,

On 2011-01-27 09:12, Tang Daogang wrote:
> Loic, thank you.
>
> Just now, I test the tmp file way. I can't find the tmp file in /tmp, why?
> How to configure it?

 From the manual:
http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-380003.10

upload.temp_store=None
     This is not set by default. If you want large requests to reach 
your handlers, then set this to a directory they can access, and make 
sure they can handle it. Read about it in the Hacking section under 
Uploads. The file has to end in XXXXXX chars to work (read man mkstemp).

  "upload.temp_store": "/tmp/mongrel2.upload.XXXXXX"

Be also aware that if your run as root, Mongrel2 chroot itself, so your 
"/tmp" is then relative to the chrooted dir.

Please also note that I am not anymore using this approach as I want the 
"big fat zmq message" way, so, I may not be fully up-to-date with the 
latest version.

loïc


>
> My config file is:
> arc = Handler(send_spec='ipc://run/t0_in',
>                  send_ident='e884a439-31be-4f74-8050-a93565795b25',
>                  recv_spec='ipc://run/t0_out', recv_ident='')
>
> main = Server(
>      uuid="505417b8-1de4-454f-98b6-07eb9225cca1"
>      access_log="/logs/access.log"
>      error_log="/logs/error.log"
>      chroot="./"
>      pid_file="/run/mongrel2.pid"
>      default_host="(.+)"
>      name="main"
>      port=6767
>      hosts=[ Host(name="(.+)", routes={ '/arc': arc }) ]
> )
>
> settings = { "zeromq.threads": 1, 'limits.content_length': 10485760,
> 'upload.temp_store': '/tmp/' }
>
> servers = [main]
>
>
>
> On Thu, Jan 27, 2011 at 3:57 PM, Loic d'Anterroches<loic@ceondo.com>  wrote:
>
>> Hello,
>>
>>> I find two ways about uploading files on mongrel2 you described.
>>>
>>> One is in Mongrel2 Manual - 5.5 Async File Upload Demo (Page 37), which
>> says
>>> the uploaded file will be stored to a temporary file, and send message to
>>> handler at start and end. The handler will get the temporary file's
>>> location, then it can read directly or do some actions on it.
>>>
>>> The other one is on your blog: Using Tir's Tasks For Async Photo Uploads
>> (
>>> http://sheddingbikes.com/posts/1292063326.html ). It gets file content
>> from
>>> form body. And I look into the form.lua in Tir, find that it can deal
>> with
>>> file uploads too.
>>>
>>> Then, which way is better? which way is easier? Why two methods? This
>>> confuses me .....
>>
>> As I have been dealing with this one for my framework, here is my take
>> on it. So you two methods: full zmq message or two messages with temp
>> file. Good/Bad is more Pro/Contra, nothing is really good nor bad, it is
>> always a question of trade-off depending of your expected usage.
>>
>> # Full zmq message
>>
>> 1. Good: your handler logic is simple, you get a request with everything
>> in it, you parse it, you use it.
>> 2. Good: your handlers can be anywhere on your network.
>> 3. Bad: your handler can receive a 100MB zmq message if you have
>> somebody uploading a big file, so you need to take care of that.
>> 4. Bad: you need to have a smart way to parse the request payload not to
>> kill your handler memory.
>> 5. Bad: parsing of the request use "handler" time and Mongrel2 is
>> probably more effecient in doing so.
>>
>> # Two messages with temp file
>>
>> 1. Good: your zmq messages will always be small, you have less memory
>> handling to think about.
>> 2. Good: you can stop the upload right at the start by checking the
>> headers (your handler can directly kill a connection if for example some
>> auth cookies are not there, this before the upload is done).
>> 3. Bad: The payload is on disk, your handler need to be able to access
>> it (shared nfs if your handlers are not on the same box).
>> 4. Bad: you still need to parse the payload but at least you have more
>> knowledge.
>> 5. Bad: the handler logic is a bit more complex as for the same request
>> you get 2 hits against your handler.
>>
>> In my framework, I decided to go the full zmq message way as I want my
>> handlers to easily run on different boxes. So, to limit the memory usage
>> and to be smart with respect to memory, I directly push the request
>> message in a buffered stream. That is, I keep in memory if less than
>> 5MB, else, I swap on disk:
>>
>>
>> 
http://projects.ceondo.com/p/photon/source/tree/develop/src/photon/mongrel2.php#L153
>>
>> After that, the multipart parser, operate in a smart way to read from
>> the stream only if needed:
>>
>>
>> 
http://projects.ceondo.com/p/photon/source/tree/develop/src/photon/http/multipartparser.php
>>
>> That way, I know that a mongrel2 request will be have a controlled max
>> memory usage.
>>
>> I hope it can help you understanding the handling of the file upload (or
>> in fact any request with a large payload).
>>
>> loïc
>>
>>

-- 
Dr Loïc d'Anterroches
Founder Céondo Ltd

w: www.ceondo.com       |  e: loic@ceondo.com
t: +44 (0)207 183 0016  |  f: +44 (0)207 183 0124

Céondo Ltd
Dalton House
60 Windsor Avenue
London
SW19 2RR / United Kingdom

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Zed A. Shaw
Date:
2011-01-27 @ 08:39 
On Thu, Jan 27, 2011 at 09:18:52AM +0100, Loic d'Anterroches wrote:
> upload.temp_store=None
>      This is not set by default. If you want large requests to reach 
> your handlers, then set this to a directory they can access, and make 
> sure they can handle it. Read about it in the Hacking section under 
> Uploads. The file has to end in XXXXXX chars to work (read man mkstemp).
> 
>   "upload.temp_store": "/tmp/mongrel2.upload.XXXXXX"
> 
> Be also aware that if your run as root, Mongrel2 chroot itself, so your 
> "/tmp" is then relative to the chrooted dir.

Yep, that's all correct.  I normally don't use this method since most of
the content someone would upload on my sites is supposed to be small.  I
cap my image uploads at 100k which is more than enough.

Where the upload.temp_store method comes in is when you want to allow
arbitrary file upload sizes, but you need two things:

1. A way to abort uploads that aren't allowed *before* you've received
the entire upload.
2. A way to stream or monitor the progress as they get uploaded.

Both of these are more important on larger file uploads.  You get #1
because Mongrel2 sends you an initial message telling you the expected
size and location of the file.  It's the full http headers with cookies
and all, so that user isn't authorized then you can kill the upload
right away and avoid having your server thrashed.

You get #2 from the same thing.  When the initial header comes in, it
tells you where the temp file is and all the normal information.  So,
you can then shoot out progress updates to the client by just doing a
poll read on the tmp file and send a message periodically.  Or, you can
have another handler that knows about files being uploaded (from say a
redis) and provide status that way.

Other than that it's mostly just so you can upload gigantic files and
not choke 0mq or the handlers.  How I use it is I have a little "upload
handler" that is in charge of dealing with these and then, when an
upload is done, shoves it over to storage and updates the database.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Tang Daogang
Date:
2011-01-27 @ 08:38 
> >> That way, I know that a mongrel2 request will be have a controlled max
> >> memory usage.
> >>
> >> I hope it can help you understanding the handling of the file upload (or
> >> in fact any request with a large payload).
> >>
> >> loïc
> >>
> >>
>

Oh, my god. I can't find the uploaded file in /tmp/ and myproject_dir/tmp/
still, and I have checked my req and req.headers' content, there's no
'x-mongrel2-upload-start' and 'x-mongrel2-upload-done' at all ! ( I use Tir
to test now. )

My config file is now:

arc = Handler(send_spec='ipc://run/t0_in',
                send_ident='e884a439-31be-4f74-8050-a93565795b25',
                recv_spec='ipc://run/t0_out', recv_ident='')

main = Server(
    uuid="505417b8-1de4-454f-98b6-07eb9225cca1"
    access_log="/logs/access.log"
    error_log="/logs/error.log"
    chroot="./"
    pid_file="/run/mongrel2.pid"
    default_host="(.+)"
    name="main"
    port=6767
    hosts=[ Host(name="(.+)", routes={ '/arc': arc }) ]
)

settings = { "zeromq.threads": 1, 'limits.content_length': 10485760,
'upload.temp_store': '/tmp/mongrel2.upload.XXXXXX' }

servers = [main]

Oh my god. That make me crary!

Does it need to set some flag to open that function? Or the mongrel2 manual
is wrong?

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Zed A. Shaw
Date:
2011-01-27 @ 08:41 
On Thu, Jan 27, 2011 at 04:38:20PM +0800, Tang Daogang wrote:
> settings = { "zeromq.threads": 1, 'limits.content_length': 10485760,
> 'upload.temp_store': '/tmp/mongrel2.upload.XXXXXX' }
> 
> servers = [main]
> 
> Oh my god. That make me crary!
> 
> Does it need to set some flag to open that function? Or the mongrel2 manual
> is wrong?

It only uses the temp_store if the file is over the
limits.content_length.  Set yours to 10000 and try it with a file larger
than that.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Tang Daogang
Date:
2011-01-27 @ 09:13 
On Thu, Jan 27, 2011 at 4:41 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

> On Thu, Jan 27, 2011 at 04:38:20PM +0800, Tang Daogang wrote:
> > settings = { "zeromq.threads": 1, 'limits.content_length': 10485760,
> > 'upload.temp_store': '/tmp/mongrel2.upload.XXXXXX' }
> >
> > servers = [main]
> >
> > Oh my god. That make me crary!
> >
> > Does it need to set some flag to open that function? Or the mongrel2
> manual
> > is wrong?
>
> It only uses the temp_store if the file is over the
> limits.content_length.  Set yours to 10000 and try it with a file larger
> than that.
>

Great! It was indeed this case.

You means that:

   - if *set* this flag (upload.temp_store), and if the uploaded file is *
   larger* than the limits.content_length, mongrel2 will locate the file in
   tmp directory. And send two thin messages to handler at start and end time;
   - if *set* this flag (upload.temp_store), and if the uploaded file is *
   smaller* than the limits.content_length, mongrel2 will keep the original
   http content passing to handler, without modifying it;
   - if *not set* this flag (upload.temp_store), and if the uploaded file is
   *larger* than the limits.content_length, mongrel2 will abort this request
   immediately;
   - if *not set* this flag (upload.temp_store), and if the uploaded file is
   *smaller* than the limits.content_length, mongrel2 will send the http
   content (including form data) to handler (by 0mq);

OK, I have confirmed above conclusions on my machine.

And, now my request log is:
# start
CONNECTING    /arc    38f857b8-cbaa-4b58-9271-0d36c27813c4
ipc://run/t0_in    ipc://run/t0_out

#############################################################
# The 1st request to display the upload form
#############################################################
REQUEST /arc: 0    Thu Jan 27 16:47:27 2011    /arc    GET
APP-f4a619a2f181ccccd4812e9f664e9029
# req table
--------------------------------------------
conn_id    0
sender    e884a439-31be-4f74-8050-a93565795b25
path    /arc
session_id    APP-f4a619a2f181ccccd4812e9f664e9029
body
data    table: 0x22fb7b0
headers    table: 0x22fc370
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#req.headers table
--------------------------------------------
connection    keep-alive
x-forwarded-for    127.0.0.1
accept    text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
PATTERN    /arc
PATH    /arc
cookie    session="APP-f4a619a2f181ccccd4812e9f664e9029"
URI    /arc
host    localhost:6767
keep-alive    115
VERSION    HTTP/1.1
accept-language    zh-cn,zh;q=0.5
user-agent    Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.8)
Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8
METHOD    GET
accept-encoding    gzip,deflate
accept-charset    GB2312,utf-8;q=0.7,*;q=0.7
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#req.data table
--------------------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

#############################################################
# The 2nd request to upload the file
#############################################################
REQUEST /arc: 0    Thu Jan 27 16:47:33 2011    /arc    POST
APP-f4a619a2f181ccccd4812e9f664e9029
# req table
--------------------------------------------
conn_id    0
sender    e884a439-31be-4f74-8050-a93565795b25
path    /arc
session_id    APP-f4a619a2f181ccccd4812e9f664e9029
body
data    table: 0x2304eb0
headers    table: 0x22e17b0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# req.headers table (ATTENTION:*x-mongrel2-upload-start shown* )
--------------------------------------------
connection    keep-alive
x-forwarded-for    127.0.0.1
content-length    4992654
host    localhost:6767
accept-charset    GB2312,utf-8;q=0.7,*;q=0.7
*x-mongrel2-upload-start    /tmp/mongrel2.upload.33AoLn*
VERSION    HTTP/1.1
METHOD    POST
referer    http://localhost:6767/arc
accept    text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
PATH    /arc
cookie    session="APP-f4a619a2f181ccccd4812e9f664e9029"
keep-alive    115
PATTERN    /arc
accept-language    zh-cn,zh;q=0.5
user-agent    Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.8)
Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8
URI    /arc
accept-encoding    gzip,deflate
content-type    multipart/form-data;
boundary=---------------------------191224406115133272201823353699
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# req.data table
--------------------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#############################################################
# The 2nd request to upload the file: to inform the finish of uploading
#############################################################
REQUEST /arc: 0    Thu Jan 27 16:47:33 2011    /arc    POST
APP-f4a619a2f181ccccd4812e9f664e9029
# req table
--------------------------------------------
conn_id    0
sender    e884a439-31be-4f74-8050-a93565795b25
path    /arc
session_id    APP-f4a619a2f181ccccd4812e9f664e9029
body
data    table: 0x230b740
headers    table: 0x2308e70
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# req.headers table
--------------------------------------------
connection    keep-alive
x-forwarded-for    127.0.0.1
content-length    4992654
host    localhost:6767
accept-charset    GB2312,utf-8;q=0.7,*;q=0.7
*x-mongrel2-upload-start    /tmp/mongrel2.upload.33AoLn*
VERSION    HTTP/1.1
METHOD    POST
referer    http://localhost:6767/arc
accept    text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
PATH    /arc
cookie    session="APP-f4a619a2f181ccccd4812e9f664e9029"
PATTERN    /arc
keep-alive    115
URI    /arc
accept-language    zh-cn,zh;q=0.5
user-agent    Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.8)
Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8
accept-encoding    gzip,deflate
*x-mongrel2-upload-done    /tmp/mongrel2.upload.33AoLn*
content-type    multipart/form-data;
boundary=---------------------------191224406115133272201823353699
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# req.data table
--------------------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We can see that, at uploading starts and ends, mongrel2 send message by
zeromq to handler separately (But the user client just did only ONE post
request), and *x-mongrel2-upload-start and **x-mongrel2-upload-done *are
embeded into request.headers.

Thank you, Shaw. But I have an advice: Maybe we can build the documents more
clearly?





>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Tang Daogang
Date:
2011-01-27 @ 09:24 
And continue...

I diff the /tmp/mongrel2.upload.RtOZ3a generated when upload with the
original file, find that they are not equal, why?

-rw------- 1 legerobot legerobot  378244 2011-01-27 17:18
mongrel2.upload.RtOZ3a
-rw-r--r-- 1 root root 377983 2011-01-21 09:03
/root/SwapArea/Screenshot-百度地图 - Mozilla Firefox.png

now generated file size if 378244, but the original file size is 377983.

Why?

Daogang.

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Zed A. Shaw
Date:
2011-01-27 @ 09:54 
On Thu, Jan 27, 2011 at 05:24:34PM +0800, Tang Daogang wrote:
> And continue...
> 
> I diff the /tmp/mongrel2.upload.RtOZ3a generated when upload with the
> original file, find that they are not equal, why?

You used a multipart POST, which adds all sorts of stuff.  Open the
tmpfile in vim and see if there's extra mime boundary junk.  Mongrel2
isn't translating the body, so when you get it you then have to still
process it like it's a multipart upload.

If you want a raw file, then try to use a PUT request.  Although I'm not
so sure how exactly you do that on an upload in a form in the browser.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Loic d'Anterroches
Date:
2011-01-27 @ 09:51 

On 2011-01-27 10:24, Tang Daogang wrote:
> And continue...
>
> I diff the /tmp/mongrel2.upload.RtOZ3a generated when upload with the
> original file, find that they are not equal, why?
>
> -rw------- 1 legerobot legerobot  378244 2011-01-27 17:18
> mongrel2.upload.RtOZ3a
> -rw-r--r-- 1 root root 377983 2011-01-21 09:03
> /root/SwapArea/Screenshot-百度地图 - Mozilla Firefox.png
>
> now generated file size if 378244, but the original file size is 377983.
>
> Why?

The full content of the POST is put in the tmp file. Mongrel does not 
parse the content for you. So you get the full post request which may 
include several files and associated POST data.

loïc

Re: [mongrel2] Questions on uploading files on mongrel2

From:
Tang Daogang
Date:
2011-01-27 @ 09:49 
PS: my mongrel2 version is 1.5.

2011/1/27 Tang Daogang <daogangtang@gmail.com>

> And continue...
>
> I diff the /tmp/mongrel2.upload.RtOZ3a generated when upload with the
> original file, find that they are not equal, why?
>
> -rw------- 1 legerobot legerobot  378244 2011-01-27 17:18
> mongrel2.upload.RtOZ3a
> -rw-r--r-- 1 root root 377983 2011-01-21 09:03
> /root/SwapArea/Screenshot-百度地图 - Mozilla Firefox.png
>
> now generated file size if 378244, but the original file size is 377983.
>
> Why?
>
> Daogang.
>

Archives based on code by Armin Ronacher //