librelist archives

« back to archive

FileStorage and excel file

FileStorage and excel file

From:
Jay Baker
Date:
2013-07-18 @ 23:51
Hello all,

This must have covered before, but I could not find it. Uploading an MS
formatted excel file, I am trying to read it as csv. If this was a file on
disk, it would just be

open('filename') # one might do 'rU' and/or dialect=csv.excel_tab or
something like that

But that would essentially get it done. How do I read the stream of
FileStorage in a similar way? I am getting the same results form reading
FileStorage in this case as I would get with 'cat filename' at the command
line.

Thanks for any pointers.

-- Jay

Re: [flask] FileStorage and excel file

From:
Steven Kryskalla
Date:
2013-07-18 @ 23:58
On Thu, Jul 18, 2013 at 4:51 PM, Jay Baker <jbaker.work@gmail.com> wrote:
> But that would essentially get it done. How do I read the stream of
> FileStorage in a similar way? I am getting the same results form reading
> FileStorage in this case as I would get with 'cat filename' at the command
> line.

The csv module works with anything you can iterate over line-by-line
(e.g. a file or a list of strings). Try to split the text on newlines:

>>> list(csv.reader("a,b\nc,d".split('\n')))
[['a', 'b'], ['c', 'd']]

Re: [flask] FileStorage and excel file

From:
Jay Baker
Date:
2013-08-08 @ 13:25
Just getting back to this. I am really stuck on this one.
I am actually get the same error when splitting '\n'
I think the issue is more universal newline related. When I try to iterate
on the csv.reader, I get
"Error: new-line character seen in unquoted field - do you need to open the
file in universal-newline mode?"

But of course I cannot specify rU because we are reading from the werkzeug
FileStorage.

Any further thoughts anybody?


On Thu, Jul 18, 2013 at 6:58 PM, Steven Kryskalla <skryskalla@gmail.com>wrote:

> On Thu, Jul 18, 2013 at 4:51 PM, Jay Baker <jbaker.work@gmail.com> wrote:
> > But that would essentially get it done. How do I read the stream of
> > FileStorage in a similar way? I am getting the same results form reading
> > FileStorage in this case as I would get with 'cat filename' at the
> command
> > line.
>
> The csv module works with anything you can iterate over line-by-line
> (e.g. a file or a list of strings). Try to split the text on newlines:
>
> >>> list(csv.reader("a,b\nc,d".split('\n')))
> [['a', 'b'], ['c', 'd']]
>

Re: [flask] FileStorage and excel file

From:
Daniel Neuhäuser
Date:
2013-08-08 @ 14:18
Just write a generator/iterator that wraps the FileStorage and replaces 
\r\n at the end of each line with \n.

Am 08.08.2013 um 15:25 schrieb Jay Baker <jbaker.work@gmail.com>:

> Just getting back to this. I am really stuck on this one. 
> I am actually get the same error when splitting '\n'
> I think the issue is more universal newline related. When I try to 
iterate on the csv.reader, I get
> "Error: new-line character seen in unquoted field - do you need to open 
the file in universal-newline mode?"
> 
> But of course I cannot specify rU because we are reading from the 
werkzeug FileStorage.
> 
> Any further thoughts anybody?
> 
> 
> On Thu, Jul 18, 2013 at 6:58 PM, Steven Kryskalla <skryskalla@gmail.com> wrote:
>> On Thu, Jul 18, 2013 at 4:51 PM, Jay Baker <jbaker.work@gmail.com> wrote:
>> > But that would essentially get it done. How do I read the stream of
>> > FileStorage in a similar way? I am getting the same results form reading
>> > FileStorage in this case as I would get with 'cat filename' at the command
>> > line.
>> 
>> The csv module works with anything you can iterate over line-by-line
>> (e.g. a file or a list of strings). Try to split the text on newlines:
>> 
>> >>> list(csv.reader("a,b\nc,d".split('\n')))
>> [['a', 'b'], ['c', 'd']]
> 

Re: [flask] FileStorage and excel file

From:
Smartboy
Date:
2013-08-08 @ 14:07
From looking around online, it looks like the csv module is getting
confused by windows-style line endings (\r\n) because it sees \r as the end
of line, not \r\n. My first inclination would be to save the file off to a
temporary place and then open it again using universal mode. However, this
would most likely add significant overhead to your app. So, another option
would be converting the newlines manually, as in how it is done here:

http://code.activestate.com/recipes/435882-normalizing-newlines-between-windowsunixmacs/This
should get you what you need, but it will require either creating a
new method that wraps around the old readline method (to run this each time
it is run), or to read it all into memory and then pass it to the csv
module.


On Thu, Aug 8, 2013 at 6:25 AM, Jay Baker <jbaker.work@gmail.com> wrote:

> Just getting back to this. I am really stuck on this one.
> I am actually get the same error when splitting '\n'
> I think the issue is more universal newline related. When I try to iterate
> on the csv.reader, I get
> "Error: new-line character seen in unquoted field - do you need to open
> the file in universal-newline mode?"
>
> But of course I cannot specify rU because we are reading from the werkzeug
> FileStorage.
>
> Any further thoughts anybody?
>
>
> On Thu, Jul 18, 2013 at 6:58 PM, Steven Kryskalla <skryskalla@gmail.com>wrote:
>
>> On Thu, Jul 18, 2013 at 4:51 PM, Jay Baker <jbaker.work@gmail.com> wrote:
>> > But that would essentially get it done. How do I read the stream of
>> > FileStorage in a similar way? I am getting the same results form reading
>> > FileStorage in this case as I would get with 'cat filename' at the
>> command
>> > line.
>>
>> The csv module works with anything you can iterate over line-by-line
>> (e.g. a file or a list of strings). Try to split the text on newlines:
>>
>> >>> list(csv.reader("a,b\nc,d".split('\n')))
>> [['a', 'b'], ['c', 'd']]
>>
>
>