librelist archives

Re: [mongrel2] Tnetstrings in Clojure

From:

Armando Singer

Date:

2011-04-19 @ 22:50

On Apr 19, 2011, at 3:31 PM, Zed A. Shaw wrote:

> On Fri, Apr 15, 2011 at 12:17:49PM -0400, Alex Gartrell wrote:
>> I implemented Tnetstrings in "pure" Clojure (I was too lazy to make them
>> fast with Java).  Results here:
>> https://github.com/alexgartrell/tnetstrings-clj . Take a look if you're
>> curious.
> 
> Very cool, looks like we're gearing up to have most of the current m2
> handler implementations support either json or tnetstrings
> transparently.  Nice.

I believe this is an example where basing the implementation on the python
example doesn't yield the correct results in the host language. A python
str is basically a byte array wrapper and it can use split, etc.

Please correct me if I'm wrong here. I've read the clojure books and written
a bit of code, but I'm by no means an expert.

This impl parses a Java String (the same java.lang.String), not a byte[],
and uses .split, .substring, .charAt, etc. A java String is not like a python
str, as it's a sequence of UTF-16 double byte unsigned chars.

It also dumps to a Java String instead of a byte[].

You get a byte[] from 0mq, so to use this API you'd have to convert all of
your stuff to a String. Very bad for parsing and dumping images, multi-part
video messages, etc.

The clojure impl could either wrap the Java impl that I took a stab at, or
be rewritten in terms of byte arrays.

Please do correct me if I'm really seeing a clojure feature that's working
on byte arrays but under a string interface.

Cheers,
Armando

> 
> -- 
> Zed A. Shaw
> http://zedshaw.com/

On Tue, Apr 19, 2011 at 03:50:47PM -0700, Armando Singer wrote:
> I believe this is an example where basing the implementation on the python
> example doesn't yield the correct results in the host language. A python
> str is basically a byte array wrapper and it can use split, etc.

You mean, just copying the code verbatim?  Probably.  I'll be doing up
some test cases that people should be able to pass for his very reason.
If you have a good suggested set of tnetstrings that prove out the
issues you're hitting let me know.


-- 
Zed A. Shaw
http://zedshaw.com/

Yeah it uses the stupid String thing.  Honestly I was just playing around
with the Clojure stuff (leiningen, clojars, etc.)  I'll fix it to do things
the right way soon.

Alex

On Wed, Apr 20, 2011 at 10:25 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

> On Tue, Apr 19, 2011 at 03:50:47PM -0700, Armando Singer wrote:
> > I believe this is an example where basing the implementation on the
> python
> > example doesn't yield the correct results in the host language. A python
> > str is basically a byte array wrapper and it can use split, etc.
>
> You mean, just copying the code verbatim?  Probably.  I'll be doing up
> some test cases that people should be able to pass for his very reason.
> If you have a good suggested set of tnetstrings that prove out the
> issues you're hitting let me know.
>
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>