librelist archives

« back to archive

Flask-Restful Decimal is not JSON serializable

Flask-Restful Decimal is not JSON serializable

From:
Michael Montero
Date:
2014-03-24 @ 20:19
Hi.  Any help would be greatly appreciated.  I'm creating an end point by
using:

api.add_resource()

I then return the value I want sent back to the requestor (I don't do any
JSON encoding) and I get this error:

TypeError: Decimal('44.74581200') is not JSON serializable

Data is coming out of MySQL.  Any chance I can fix this easily?

Thanks!
Mike

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Shawn Milochik
Date:
2014-03-24 @ 20:40
You should be able to easily find out why via a traceback. Worst-case, you
can import traceback and then log traceback.format_stack() and see what's
up.

Is there any chance you have any caching going on?

On the other hand, if it's a JSON API and this data is being sent via a
port and not just being consumed within the same codebase, then that's a
different problem.

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
David Baumgold
Date:
2014-03-24 @ 20:30
Can you post more of the code that’s related to this? Maybe your 
SQLAlchemy models, your Flask-Restful resources, and so on. That might 
help to identify what the problem is.
-David



On Monday, March 24, 2014 at 4:19 PM, Michael Montero wrote:

> Hi.  Any help would be greatly appreciated.  I'm creating an end point by using:
>  
> api.add_resource()
>  
> I then return the value I want sent back to the requestor (I don't do 
any JSON encoding) and I get this error:  
>  
>  
> TypeError: Decimal('44.74581200') is not JSON serializable
>  
>  
> Data is coming out of MySQL.  Any chance I can fix this easily?
>  
> Thanks!
> Mike
>  
>  
>  
>  
>  

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Farhan
Date:
2014-03-24 @ 20:42
Take a look at this: 
http://stackoverflow.com/questions/14996264/flask-restless-dumps-decimal-value-from-flask-sqlalchemy

Best,
Farhan

On Mar 24, 2014, at 1:30 PM, David Baumgold <david@davidbaumgold.com> wrote:

> Can you post more of the code that’s related to this? Maybe your 
SQLAlchemy models, your Flask-Restful resources, and so on. That might 
help to identify what the problem is.
> -David
> On Monday, March 24, 2014 at 4:19 PM, Michael Montero wrote:
> 
>> Hi.  Any help would be greatly appreciated.  I'm creating an end point 
by using:
>> 
>> api.add_resource()
>> 
>> I then return the value I want sent back to the requestor (I don't do 
any JSON encoding) and I get this error:
>> 
>> 
>> TypeError: Decimal('44.74581200') is not JSON serializable
>> 
>> 
>> Data is coming out of MySQL.  Any chance I can fix this easily?
>> 
>> Thanks!
>> Mike
> 

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Jörg Bönisch
Date:
2014-03-24 @ 21:46
The stackoverflow post is about flask-restless and Mike uses restful. I'm 
not sure if you're able to "override" json there.

-------- Originalmitteilung --------
Von:Farhan <inshany@gmail.com>
Gesendet:Mon, 24 Mar 2014 22:09:24 +0100
An:flask@librelist.com
Betreff:Re: [flask] Flask-Restful Decimal is not JSON serializable

>Have you looked at the StackOverflow post?
>
>
>On Mar 24, 2014, at 2:02 PM, Michael Montero <mike@resy.com> wrote:
>
>
>Converting the columns in MySQL from dec(10, 8) to float(53) worked.  I'm
not sure that I love that I can't use decimal columns in my database 
because they can't be JSON encoded.  Any thoughts?
>
>
>
>On Mon, Mar 24, 2014 at 4:54 PM, Michael Montero <mike@resy.com> wrote:
>
>I'm not calling json.dumps().  The Flask-Restful module does that for me.
 I'm just returning dictionaries and arrays.
>
>
>
>On Mon, Mar 24, 2014 at 4:42 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:
>
>Mike,
>
>
>
>I then return the value I want sent back to the requestor (I don't do any
>JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
>not JSON serializable Data is coming out of MySQL.  Any chance I can fix
>this easily?
>
>
>I would agree that it's not an optimal solution, but how about converting
>your Decimals to float()?
>
>--
>G.
>
>
>
>

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Farhan
Date:
2014-03-24 @ 21:57
Yes, I realize that. Fork Flask-Restful and use simplejson or try 
extending it as per the Flask-Restful doc, whichever is easier.

On Mar 24, 2014, at 2:46 PM, Jörg Bönisch <joerg@joergwork.org> wrote:

> The stackoverflow post is about flask-restless and Mike uses restful. 
I'm not sure if you're able to "override" json there.
> 
> -------- Originalmitteilung --------
> Von:Farhan <inshany@gmail.com>
> Gesendet:Mon, 24 Mar 2014 22:09:24 +0100
> An:flask@librelist.com
> Betreff:Re: [flask] Flask-Restful Decimal is not JSON serializable
> 
> Have you looked at the StackOverflow post?
> 
> On Mar 24, 2014, at 2:02 PM, Michael Montero <mike@resy.com> wrote:
> 
>> Converting the columns in MySQL from dec(10, 8) to float(53) worked.  
I'm not sure that I love that I can't use decimal columns in my database 
because they can't be JSON encoded.  Any thoughts?
>> 
>> 
>> On Mon, Mar 24, 2014 at 4:54 PM, Michael Montero <mike@resy.com> wrote:
>> I'm not calling json.dumps().  The Flask-Restful module does that for 
me.  I'm just returning dictionaries and arrays.
>> 
>> 
>> On Mon, Mar 24, 2014 at 4:42 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:
>> Mike,
>> 
>> 
>> I then return the value I want sent back to the requestor (I don't do any
>> JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
>> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
>> this easily?
>> 
>> I would agree that it's not an optimal solution, but how about converting
>> your Decimals to float()?
>> 
>> --
>> G.
>> 
>> 
> 

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Jörg Bönisch
Date:
2014-03-24 @ 22:15
Sounds like marshalling is the flask-restful way, but I'm not familiar  
with it.

http://flask-restful.readthedocs.org/en/latest/api.html#module-fields

class fields.Raw(default=None, attribute=None)
flask.ext.restful.marshal_with(fields)
flask.ext.restful.marshal(data, fields)


> The stackoverflow post is about flask-restless and Mike uses  
> restful. I'm not sure if you're able to "override" json there.
>
> -------- Originalmitteilung --------
> Von:Farhan <inshany@gmail.com>
> Gesendet:Mon, 24 Mar 2014 22:09:24 +0100
> An:flask@librelist.com
> Betreff:Re: [flask] Flask-Restful Decimal is not JSON serializable
>
>> Have you looked at the StackOverflow post?
>>
>>
>> On Mar 24, 2014, at 2:02 PM, Michael Montero <mike@resy.com> wrote:
>>
>>
>> Converting the columns in MySQL from dec(10, 8) to float(53)  
>> worked.  I'm not sure that I love that I can't use decimal columns  
>> in my database because they can't be JSON encoded.  Any thoughts?
>>
>>
>>
>> On Mon, Mar 24, 2014 at 4:54 PM, Michael Montero <mike@resy.com> wrote:
>>
>> I'm not calling json.dumps().  The Flask-Restful module does that  
>> for me.  I'm just returning dictionaries and arrays.
>>
>>
>>
>> On Mon, Mar 24, 2014 at 4:42 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:
>>
>> Mike,
>>
>>
>>
>> I then return the value I want sent back to the requestor (I don't do any
>> JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
>> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
>> this easily?
>>
>>
>> I would agree that it's not an optimal solution, but how about converting
>> your Decimals to float()?
>>
>> --
>> G.
>>
>>
>>
>>

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Gary Chambers
Date:
2014-03-24 @ 20:42
Mike,

> I then return the value I want sent back to the requestor (I don't do any
> JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
> this easily?

I would agree that it's not an optimal solution, but how about converting
your Decimals to float()?

--
G.

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Farhan
Date:
2014-03-24 @ 20:55
If you do that, you will run into accuracy issues. Decimals can be 
represented exactly, hence the use case for using Decimals over floats.

Try this in Decimal and then with float: 0.1 + 0.1 + 0.1 - 0.3
From the python docs 
(http://docs.python.org/2.7/library/decimal.html?highlight=decimal#decimal)

Best,
Farhan

On Mar 24, 2014, at 1:42 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:

> Mike,
> 
>> I then return the value I want sent back to the requestor (I don't do any
>> JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
>> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
>> this easily?
> 
> I would agree that it's not an optimal solution, but how about converting
> your Decimals to float()?
> 
> --
> G.

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Gary Chambers
Date:
2014-03-24 @ 21:08
> If you do that, you will run into accuracy issues. Decimals can be
> represented exactly, hence the use case for using Decimals over floats.
> Try this in Decimal and then with float: 0.1 + 0.1 + 0.1 - 0.3

Agreed.  I realized that, immediately after I hit send, I should have
recommended converting to str() instead.  I expected to be flamed, actually.
:)

--
G.

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Michael Montero
Date:
2014-03-24 @ 20:54
I'm not calling json.dumps().  The Flask-Restful module does that for me.
 I'm just returning dictionaries and arrays.


On Mon, Mar 24, 2014 at 4:42 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:

> Mike,
>
>
>  I then return the value I want sent back to the requestor (I don't do any
>> JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
>> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
>> this easily?
>>
>
> I would agree that it's not an optimal solution, but how about converting
> your Decimals to float()?
>
> --
> G.

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Michael Montero
Date:
2014-03-24 @ 21:02
Converting the columns in MySQL from dec(10, 8) to float(53) worked.  I'm
not sure that I love that I can't use decimal columns in my database
because they can't be JSON encoded.  Any thoughts?


On Mon, Mar 24, 2014 at 4:54 PM, Michael Montero <mike@resy.com> wrote:

> I'm not calling json.dumps().  The Flask-Restful module does that for me.
>  I'm just returning dictionaries and arrays.
>
>
> On Mon, Mar 24, 2014 at 4:42 PM, Gary Chambers <gwchamb@gwcmail.com>wrote:
>
>> Mike,
>>
>>
>>  I then return the value I want sent back to the requestor (I don't do any
>>> JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
>>> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
>>> this easily?
>>>
>>
>> I would agree that it's not an optimal solution, but how about converting
>> your Decimals to float()?
>>
>> --
>> G.
>
>
>

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Farhan
Date:
2014-03-24 @ 21:09
Have you looked at the StackOverflow post?

On Mar 24, 2014, at 2:02 PM, Michael Montero <mike@resy.com> wrote:

> Converting the columns in MySQL from dec(10, 8) to float(53) worked.  
I'm not sure that I love that I can't use decimal columns in my database 
because they can't be JSON encoded.  Any thoughts?
> 
> 
> On Mon, Mar 24, 2014 at 4:54 PM, Michael Montero <mike@resy.com> wrote:
> I'm not calling json.dumps().  The Flask-Restful module does that for 
me.  I'm just returning dictionaries and arrays.
> 
> 
> On Mon, Mar 24, 2014 at 4:42 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:
> Mike,
> 
> 
> I then return the value I want sent back to the requestor (I don't do any
> JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
> this easily?
> 
> I would agree that it's not an optimal solution, but how about converting
> your Decimals to float()?
> 
> --
> G.
> 
> 

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Michael Montero
Date:
2014-03-24 @ 21:43
I did look at the StackOverflow post.  Not using Restless...using Restful.




On Mon, Mar 24, 2014 at 5:09 PM, Farhan <inshany@gmail.com> wrote:

> Have you looked at the StackOverflow post?
>
> On Mar 24, 2014, at 2:02 PM, Michael Montero <mike@resy.com> wrote:
>
> Converting the columns in MySQL from dec(10, 8) to float(53) worked.  I'm
> not sure that I love that I can't use decimal columns in my database
> because they can't be JSON encoded.  Any thoughts?
>
>
> On Mon, Mar 24, 2014 at 4:54 PM, Michael Montero <mike@resy.com> wrote:
>
>> I'm not calling json.dumps().  The Flask-Restful module does that for me.
>>  I'm just returning dictionaries and arrays.
>>
>>
>> On Mon, Mar 24, 2014 at 4:42 PM, Gary Chambers <gwchamb@gwcmail.com>wrote:
>>
>>> Mike,
>>>
>>>
>>>  I then return the value I want sent back to the requestor (I don't do
>>>> any
>>>> JSON encoding) and I get this error: TypeError: Decimal('44.74581200')
>>>> is
>>>> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
>>>> this easily?
>>>>
>>>
>>> I would agree that it's not an optimal solution, but how about converting
>>> your Decimals to float()?
>>>
>>> --
>>> G.
>>
>>
>>
>
>

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Farhan
Date:
2014-03-24 @ 21:54
Then how about exploring Extending Flask-Restful as an option.

On Mar 24, 2014, at 2:43 PM, Michael Montero <mike@resy.com> wrote:

> I did look at the StackOverflow post.  Not using Restless...using Restful.
> 
> On Mon, Mar 24, 2014 at 5:09 PM, Farhan <inshany@gmail.com> wrote:
> Have you looked at the StackOverflow post?
> 
> On Mar 24, 2014, at 2:02 PM, Michael Montero <mike@resy.com> wrote:
> 
>> Converting the columns in MySQL from dec(10, 8) to float(53) worked.  
I'm not sure that I love that I can't use decimal columns in my database 
because they can't be JSON encoded.  Any thoughts?
>> 
>> 
>> On Mon, Mar 24, 2014 at 4:54 PM, Michael Montero <mike@resy.com> wrote:
>> I'm not calling json.dumps().  The Flask-Restful module does that for 
me.  I'm just returning dictionaries and arrays.
>> 
>> 
>> On Mon, Mar 24, 2014 at 4:42 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:
>> Mike,
>> 
>> 
>> I then return the value I want sent back to the requestor (I don't do any
>> JSON encoding) and I get this error: TypeError: Decimal('44.74581200') is
>> not JSON serializable Data is coming out of MySQL.  Any chance I can fix
>> this easily?
>> 
>> I would agree that it's not an optimal solution, but how about converting
>> your Decimals to float()?
>> 
>> --
>> G.
>> 
>> 
> 
> 

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Michael Montero
Date:
2014-03-24 @ 20:41
Thanks, David.

The data being selected are latitude and longitude out of a MySQL table
using Python 3 and the latest Flask-Restful.  The columns are decimal.  We
are querying the database directly with no SQLAlchemy in the middle.  Is
there more information I can provide other than that?

We resolved this issue by casting the values in our select statement to
character strings:

                      cast(v.latitude as char) as latitude,

                      cast(v.longitude as char) as longitude,

But that seems incorrect.


Mike



On Mon, Mar 24, 2014 at 4:30 PM, David Baumgold <david@davidbaumgold.com>wrote:

> Can you post more of the code that's related to this? Maybe your
> SQLAlchemy models, your Flask-Restful resources, and so on. That might help
> to identify what the problem is.
> -David
>
> On Monday, March 24, 2014 at 4:19 PM, Michael Montero wrote:
>
> Hi.  Any help would be greatly appreciated.  I'm creating an end point by
> using:
>
> api.add_resource()
>
> I then return the value I want sent back to the requestor (I don't do any
> JSON encoding) and I get this error:
>
> TypeError: Decimal('44.74581200') is not JSON serializable
>
> Data is coming out of MySQL.  Any chance I can fix this easily?
>
> Thanks!
> Mike
>
>
>

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Shawn Milochik
Date:
2014-03-24 @ 20:56
I think the Stack Overflow post answered it.

However, if it's lat/lon, I'd convert to a string rather than a float, if I
was going to convert it. Don't want any rounding on a location.

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Kevin Houlihan
Date:
2014-03-24 @ 22:20
Are you doing any explicit marshalling? I don't know if it works, but
Flask-Restful has a "Fixed" marshalling type which is intended for
fixed-precision decimals.

http://flask-restful.readthedocs.org/en/latest/api.html#flask.ext.restful.marshal

For your purposes you would probably specify a dictionary like this for the
fields:

from flask.ext.restful.fields import Fixed

marshallable = {
    'latitude': Fixed(decimals=6),
    'longitude': Fixed(decimals=6)
}

And then from your handler methods return marshal(data, marshallable)
instead of returning data directly.

Regards,
Kevin


On 24 March 2014 20:19, Michael Montero <mike@resy.com> wrote:

> Hi.  Any help would be greatly appreciated.  I'm creating an end point by
> using:
>
> api.add_resource()
>
> I then return the value I want sent back to the requestor (I don't do any
> JSON encoding) and I get this error:
>
> TypeError: Decimal('44.74581200') is not JSON serializable
>
> Data is coming out of MySQL.  Any chance I can fix this easily?
>
> Thanks!
> Mike
>

Re: [flask] Flask-Restful Decimal is not JSON serializable

From:
Kevin Houlihan
Date:
2014-03-24 @ 22:23
Sorry that should have been...

marshallable = {
    'latitude': Fixed(decimals=8),
    'longitude': Fixed(decimals=8)
}

...I guess.


On 24 March 2014 22:20, Kevin Houlihan <kevin@crimsoncookie.com> wrote:

> Are you doing any explicit marshalling? I don't know if it works, but
> Flask-Restful has a "Fixed" marshalling type which is intended for
> fixed-precision decimals.
>
>
> 
http://flask-restful.readthedocs.org/en/latest/api.html#flask.ext.restful.marshal
>
> For your purposes you would probably specify a dictionary like this for
> the fields:
>
> from flask.ext.restful.fields import Fixed
>
> marshallable = {
>     'latitude': Fixed(decimals=6),
>     'longitude': Fixed(decimals=6)
> }
>
> And then from your handler methods return marshal(data, marshallable)
> instead of returning data directly.
>
> Regards,
> Kevin
>
>
> On 24 March 2014 20:19, Michael Montero <mike@resy.com> wrote:
>
>> Hi.  Any help would be greatly appreciated.  I'm creating an end point by
>> using:
>>
>> api.add_resource()
>>
>> I then return the value I want sent back to the requestor (I don't do any
>> JSON encoding) and I get this error:
>>
>> TypeError: Decimal('44.74581200') is not JSON serializable
>>
>> Data is coming out of MySQL.  Any chance I can fix this easily?
>>
>> Thanks!
>> Mike
>>
>
>