librelist archives

« back to archive

Shoes CSV Encoding Problem?

Shoes CSV Encoding Problem?

From:
Kc Erb
Date:
2013-08-14 @ 21:50
Hi all,

I have a CSV file with one or two UTF-8 characters (in people's names such
as Prögler). I've been using CSVMapper to read in the file but now it is
choking.

When I try my file in a normal ruby script it works:

http://pastebin.com/5LNKumRT


But when I put it into the Shoes app it returns an error:
*Invalid byte sequence US-ASCII*
*/Application/Shoes.app/Contents/MacOS/ruby/lib/csv.rb:2027:in '=~' *
*...*


Link to Shoes code giving error:
http://pastebin.com/dAF8zEaK


The above applies to Shoes3. When I use Shoes4 (i.e. not shoes ruby) it
does not throw an error but it fails to encode the ö correctly and instead
returns these two characters ö.

Any suggestions?

-KC
 "If there is an exception to every rule, then every rule must have at
least one exception; the exception to this one being that it has no
exception."

Re: [shoes] Shoes CSV Encoding Problem?

From:
Tobias Pfeiffer
Date:
2013-08-15 @ 10:47
Hi KC,

I think there is something wrong with the encoding of the string 
somehow. Maybe try string.force_encoding('utf-8') or however that 
encoding conversion is handled (encoding is REALLY hard...)

For me I tried the following app:

# coding: utf-8

Shoes.app do
  para 'Tröger'
end

And it works for me with both Shoes3 and Shoes4. For shoes3 I have a 
custom compilation though with 1.9.3-p*** and for shoes4 I'm using 
current master with JRuby-1.7.4 :-)

So just as a basic try for Umlaute with ö etc. but maybe the problem is 
different.

Cheers,
Tobi

On Wed 14 Aug 2013 11:50:11 PM CEST, KC Erb wrote:
> Hi all,
>
> I have a CSV file with one or two UTF-8 characters (in people's names
> such as Prögler). I've been using CSVMapper to read in the file but
> now it is choking.
>
> When I try my file in a normal ruby script it works:
>
> http://pastebin.com/5LNKumRT
>
>
> But when I put it into the Shoes app it returns an error:
> *Invalid byte sequence US-ASCII*
> */Application/Shoes.app/Contents/MacOS/ruby/lib/csv.rb:2027:in '=~' *
> *...*
>
>
> Link to Shoes code giving error:
> http://pastebin.com/dAF8zEaK
>
>
> The above applies to Shoes3. When I use Shoes4 (i.e. not shoes ruby)
> it does not throw an error but it fails to encode the ö correctly and
> instead returns these two characters ö.
>
> Any suggestions?
>
> -KC
>  "If there is an exception to every rule, then every rule must have at
> least one exception; the exception to this one being that it has no
> exception."

Re: [shoes] Shoes CSV Encoding Problem?

From:
ashbb
Date:
2013-08-16 @ 00:42
Hi KC,

Confirm this: `puts @@people_raw.to_s.encoding`

On my Windows 7, the output is
- "Windows-31J" with ruby 1.9.3p385 (2013-02-06) [i386-mingw32]
- "ASCII-8BIT" with jruby 1.7.4 (1.9.3p392) 2013-05-16 2390d3b on Java
HotSpot(TM) Client VM 1.7.0_0 3-b05 [Windows 7-x86]

I'm not sure why, but the output is not "UTF-8".

So, as Tobi already mentioned, add `force_encoding` like this:

  flow do
    para @@people_raw.to_s.force_encoding("UTF-8")
  end

Hope this helps,
ashbb

Re: [shoes] Shoes CSV Encoding Problem?

From:
Kc Erb
Date:
2013-08-16 @ 23:25
Hi ashbb,

I feel like I'm failing to communicate. The solution you and Tobi give *does
* fix the problem in Shoes4.

But in Shoes3 @@people_raw never gets created because CSV fails to import.

Because it's choking back at CSV.import it seems like  ".force_encoding "
doesn't get run at all when I try this in Shoes3.

Then again, if Shoes3 is going the way of the
LaserDisc<http://en.wikipedia.org/wiki/LaserDisc> then
I just won't sweat it.

Thanks for the help!




On Thu, Aug 15, 2013 at 6:42 PM, ashbb <ashbbb@gmail.com> wrote:

> Hi KC,
>
> Confirm this: `puts @@people_raw.to_s.encoding`
>
> On my Windows 7, the output is
> - "Windows-31J" with ruby 1.9.3p385 (2013-02-06) [i386-mingw32]
> - "ASCII-8BIT" with jruby 1.7.4 (1.9.3p392) 2013-05-16 2390d3b on Java
> HotSpot(TM) Client VM 1.7.0_0 3-b05 [Windows 7-x86]
>
> I'm not sure why, but the output is not "UTF-8".
>
> So, as Tobi already mentioned, add `force_encoding` like this:
>
>   flow do
>     para @@people_raw.to_s.force_encoding("UTF-8")
>   end
>
> Hope this helps,
> ashbb
>



-- 
 "If there is an exception to every rule, then every rule must have at
least one exception; the exception to this one being that it has no
exception."

Re: [shoes] Shoes CSV Encoding Problem?

From:
Tobias Pfeiffer
Date:
2013-08-17 @ 06:31
Hi all,

so shoes3 most likely will not receive any further updates. The error 
could be in the CSV library there. As a work around the only idea I 
have is maybe reading the file in, for the encoding for the file/string 
and then pass it to the CSV library. Maybe that works/helps.

Otherwise it maybe works with newer ruby interpreter version (it 
should) but then you'd have to compile your own version of shoes and 
not use the downloaded one.

Hope that is helpful :-)
Cheers,
Tobi

On Sat 17 Aug 2013 01:25:41 AM CEST, KC Erb wrote:
> Hi ashbb,
>
> I feel like I'm failing to communicate. The solution you and Tobi give
> /does/ fix the problem in Shoes4.
>
> But in Shoes3 @@people_raw never gets created because CSV fails to
> import.
>
> Because it's choking back at CSV.import it seems like
>  ".force_encoding " doesn't get run at all when I try this in Shoes3.
>
> Then again, if Shoes3 is going the way of the LaserDisc
> <http://en.wikipedia.org/wiki/LaserDisc> then I just won't sweat it.
>
> Thanks for the help!
>
>
>
>
> On Thu, Aug 15, 2013 at 6:42 PM, ashbb <ashbbb@gmail.com
> <mailto:ashbbb@gmail.com>> wrote:
>
>     Hi KC,
>
>     Confirm this: `puts @@people_raw.to_s.encoding`
>
>     On my Windows 7, the output is
>     - "Windows-31J" with ruby 1.9.3p385 (2013-02-06) [i386-mingw32]
>     - "ASCII-8BIT" with jruby 1.7.4 (1.9.3p392) 2013-05-16 2390d3b on
>     Java HotSpot(TM) Client VM 1.7.0_0 3-b05 [Windows 7-x86]
>
>     I'm not sure why, but the output is not "UTF-8".
>
>     So, as Tobi already mentioned, add `force_encoding` like this:
>
>       flow do
>         para @@people_raw.to_s.force_encoding("UTF-8")
>       end
>
>     Hope this helps,
>     ashbb
>
>
>
>
> --
>  "If there is an exception to every rule, then every rule must have at
> least one exception; the exception to this one being that it has no
> exception."