Problem with file uploads larger than 500kb resulting in filenames
- From:
- Raphael Slinckx
- Date:
- 2011-01-31 @ 10:19
Hi !
I'm having a problem which i believe is a bug in werkzeug, but I need
a confirmation. I'm using Werkzeug 0.6.1 / Flask 0.6
The behavior I observe is the following: When you have a form that you
POST that contains more than 500kb of post data in multipart/form-data
the FileStorage objects get garbled and cannot be relied upon to
detect if a file has been uploaded or not because the filename (which
should be None if there were no file uploaded) is set to '<fdopen>'
I see in werkzeug/formparser.py:31 that there is a
default_stream_factory that switches from an in-memory stream to a
temporary file when the post data is longer than 500kb, later on the
FileStorage class (werkzeug/datastructures.py:2252) does:
self.filename = filename or getattr(stream, 'name', None)
I guess the stream.name for the temporary file object is <fdopen> and
this causes problems.
Here is a snippet that should expose the problem to be pasted into a
python file. Try uploading a single file less than 500kb, the
resulting request.files looks OK. Now try to upload a single file
larger than 500kb, you'll see that the other two fields will be set to
<fdopen> so that you cannot determine if they are really uploaded
files named <fdopen> or not. Note that the content length is always 0
too, so you can't count on that neither.
As a workaround i have to check if the filename is <fdopen> and
discard that field, but it's ugly :)
Snippet:
=========
from flask import Flask, request, render_template_string
app = Flask(__name__)
@app.route('/')
def index():
return """
<!DOCTYPE HTML>
<html>
<body>
<form action="/" method="POST" enctype="multipart/form-data">
<p>
<input type="text" name="foo" value="Short text">
<input type="file" name="file-1">
<input type="file" name="file-2">
<input type="file" name="file-3">
</p>
<input type="submit">
</form>
</body>
</html>
"""
@app.route('/', methods=['POST'])
def do_index():
return render_template_string("""
<!DOCTYPE HTML>
<html>
<body>
<table border="1">
<tr>
<th>Name</th>
<th>Filename</th>
<th>Content Type</th>
<th>Content Length</th>
</tr>
{% for name, f in files|sort %}
<tr>
<td>{{name|e}}</td>
<td>{{f.filename|e}}</td>
<td>{{f.content_type|e}}</td>
<td>{{f.content_length|e}}</td>
</tr>
{% endfor %}
</ul>
</body>
</html>
""", files=[(name, f) for name, f in request.files.iteritems()])
app.run(debug=True)
=========
--
Raphaël Slinckx
Whatever sa/nv
Lead Developer
Rue Fond Cattelain 5
1435 Mont-Saint-Guibert, Belgium
+32 485 133 248
http://www.whatever-company.com
Re: [flask] Problem with file uploads larger than 500kb resulting in
filenames
- From:
- Ron DuPlain
- Date:
- 2011-02-13 @ 04:30
Hi Raphaël,
On Mon, Jan 31, 2011 at 5:19 AM, Raphael Slinckx
<r.slinckx@whatever-company.com> wrote:
> I'm having a problem which i believe is a bug in werkzeug, but I need
> a confirmation. I'm using Werkzeug 0.6.1 / Flask 0.6
>
> The behavior I observe is the following: When you have a form that you
> POST that contains more than 500kb of post data in multipart/form-data
> the FileStorage objects get garbled and cannot be relied upon to
> detect if a file has been uploaded or not because the filename (which
> should be None if there were no file uploaded) is set to '<fdopen>'
I'm experiencing the same issue, using Werkzeug 0.6.2 / Flask 0.7dev.
The app has multiple file fields, small files work without any issues,
and larger files cause an extra file stream with name '<fdopen>'.
> As a workaround i have to check if the filename is <fdopen> and
> discard that field, but it's ugly :)
That's the same workaround I made. I'm not sure what the fix is, but
I figured I'd chime in with a +1 on reproducing the bug. Have you
filed a bug with Werkzeug?
Thanks for posting your experience. It helped make debugging smoother.
Ron
Re: [flask] Problem with file uploads larger than 500kb resulting in
filenames
- From:
- Raphael Slinckx
- Date:
- 2011-02-14 @ 09:05
>> As a workaround i have to check if the filename is <fdopen> and
>> discard that field, but it's ugly :)
>
> That's the same workaround I made. I'm not sure what the fix is, but
> I figured I'd chime in with a +1 on reproducing the bug. Have you
> filed a bug with Werkzeug?
>
> Thanks for posting your experience. It helped make debugging smoother.
You're welcome !
I haven't filed a bug in Werkzeug as I wanted to gather feedback
before, and besides I don't know where the proper tracker is..
Re: [flask] Problem with file uploads larger than 500kb resulting in
filenames
- From:
- sigzero
- Date:
- 2011-02-14 @ 13:38
On Mon, Feb 14, 2011 at 4:05 AM, Raphael Slinckx
<r.slinckx@whatever-company.com> wrote:
> You're welcome !
> I haven't filed a bug in Werkzeug as I wanted to gather feedback
> before, and besides I don't know where the proper tracker is..
>
https://github.com/mitsuhiko/werkzeug/issues
Re: [flask] Problem with file uploads larger than 500kb resulting
in filenames
- From:
- Armin Ronacher
- Date:
- 2011-02-14 @ 14:41
Hi,
On 2/14/11 2:38 PM, sigzero wrote:
> On Mon, Feb 14, 2011 at 4:05 AM, Raphael Slinckx
> <r.slinckx@whatever-company.com> wrote:
>> You're welcome !
>> I haven't filed a bug in Werkzeug as I wanted to gather feedback
>> before, and besides I don't know where the proper tracker is..
>>
>
> https://github.com/mitsuhiko/werkzeug/issues
Also please provide a testcase. I was not able to reproduce it on 0.7-dev.
Regards,
Armin
Re: [flask] Problem with file uploads larger than 500kb resulting in
filenames
- From:
- Raphael Slinckx
- Date:
- 2011-02-15 @ 09:24
Hi Armin !
> Also please provide a testcase. I was not able to reproduce it on 0.7-dev.
I see that in the 0.7 version there is this line in the FileStorage
__init__ method:
if filename and filename[0] == '<' and filename[-1] == '>'
Which is similar to the workaround i described in my initial mail, so
the problem is "fixed" in 0.7-dev
--
Raphaël Slinckx
Whatever sa/nv
Lead Developer
Rue Fond Cattelain 5
1435 Mont-Saint-Guibert, Belgium
+32 485 133 248
http://www.whatever-company.com
Re: [flask] Problem with file uploads larger than 500kb resulting
in filenames
- From:
- Armin Ronacher
- Date:
- 2011-02-15 @ 17:10
Hi,
On 2/15/11 10:24 AM, Raphael Slinckx wrote:
> Hi Armin !
>
>> Also please provide a testcase. I was not able to reproduce it on 0.7-dev.
>
> I see that in the 0.7 version there is this line in the FileStorage
> __init__ method:
>
> if filename and filename[0] == '<' and filename[-1] =='>'
>
> Which is similar to the workaround i described in my initial mail, so
> the problem is "fixed" in 0.7-dev
That workaround I added to the code is only triggered if no filename was
transmitted in the first place. Which as far as I know should not
happen because FileStorage objects are only constructed for actual file
uploads. And these are detected by the fact if a filename is provided.
So not sure if that fixes something. If it does I was not able to
trigger it however. Do you have a testcase for that?
Regards,
Armin