librelist archives

Re: [flask] database in large application

From:

Alessio Civitillo

Date:

2011-06-19 @ 14:25

Hello,

I will just use this thread as it may be useful for other developing a large
application using mongodb.

At the moment I am trying to test just the connection, but I keep receiving
failed tests, any help would be really appreciated. The snippet is:

######
import os
import flask
import unittest
import tempfile
from eventlift import app
import mongoengine as mgo

class FlaskTestCase(unittest.TestCase):

def setUp(self):
self.db_fd, app.config['DATABASE'] = tempfile.mkstemp()
self.app = app.test_client()
mgo.connect('mehdb') ##This should be a connection to a mongodb db

def tearDown(self):
os.close(self.db_fd)
os.unlink(app.config['DATABASE'])

def test_empty_db(self):
rv = self.app.get('/')
assert 'No entries here so far' in rv.data

FlaskTestCase(unittest.main())
#####

On Tue, Jun 14, 2011 at 5:16 PM, Alessio Civitillo <
alessiocivitillo@gmail.com> wrote:

> @Philip, thanks.
>
> @Brent, this evening I am going to try again, but I did that and I got
> errors when doing Model.save()
>
>
> On Mon, Jun 13, 2011 at 7:21 AM, Brent Tubbs <brent.tubbs@gmail.com>wrote:
>
>> When using MongoEngine (as opposed to raw PyMongo), you don't need to
>> import or pass around the connection object returned from connect().
>> MongoEngine handles that for you.  You just import your MongoEngine
>> models and you're done.
>>
>> On Sun, Jun 12, 2011 at 4:22 PM, Philip Nelson <me@pnelson.ca> wrote:
>> > I just create the connection once in a file named database.py and import
>> the
>> > connection from that file in my views.
>> >
>> > Phil
>> >
>> >
>> > On Sun, Jun 12, 2011 at 2:18 PM, Alessio Civitillo
>> > <alessiocivitillo@gmail.com> wrote:
>> >>
>> >> Hello,
>> >>
>> >> thanks to everybody for the feedback, I could not answer before as I
>> was
>> >> in an internet deprivation condition.
>> >>
>> >> @Brent, thanks. I found the mistake I was doing looking at your
>> solution.
>> >> In models I did not import mongoengine. I had class User(Document) and
>> not
>> >> class User (mongoengine.Document).
>> >>
>> >> @Philip, thanks for the idea.
>> >>
>> >> @Brent, @Philip. Now, just one last question, if I put the views in a
>> >> different file (django style), I shoud create a connection in every
>> view
>> >> module/file, right? This is the only way I managed to make it work.
>> >> Otherwise, do you have other ways to make the connection available in
>> the
>> >> view modules?
>> >>
>> >> @Ross, I was planning to look into that, but before I thought to do it
>> >> Rambo style, without any help from external extensions.
>> >>
>> >> On Sun, Jun 12, 2011 at 7:07 PM, Ross Lawley <ross.lawley@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi,
>> >>> I just setup mongoengine when I create my Flask app import from there.
>> >>>  You might want to checkout:
>> https://github.com/sbook/flask-mongoengine/ -
>> >>> it also gives you model forms and a debugtoolbar - which is handy on
>> large
>> >>> projects (disclaimer - I'm the maintainer, so any issues let me know
>> via
>> >>> github).
>> >>> Ross
>> >>>
>> >>> On Fri, Jun 10, 2011 at 10:55 AM, Daniel , Dao Quang Minh
>> >>> <dqminh89@gmail.com> wrote:
>> >>>>
>> >>>> Hey guys,
>> >>>> I'm the author of https://github.com/dqminh/flask-mongoobject/.
>> >>>> Basically it's just a thin wrapper
>> >>>> of https://github.com/slacy/minimongo/ (particularly AttrDict) with
>> a bit
>> >>>> more sugar to ease the use of AutoReference. It has similar syntax to
>> >>>> Flask-SQLAlchemy too (Model.query attribute will refers to the
>> Model's
>> >>>> __collection__ , hence, let you use the same pymongo query to query
>> database
>> >>>> and get back your model instance). It's fairly alpha now and we are
>> using it
>> >>>> in development. Love to get feedback on how we can improve it.
>> >>>> Daniel
>> >>>> On Fri, Jun 10, 2011 at 3:16 PM, Philip Nelson <me@pnelson.ca>
>> wrote:
>> >>>>>
>> >>>>> ... and right when I click 'Send' I noticed I messed up some
>> variable
>> >>>>> names. Rename 'db_connection' to simply 'connection'.
>> >>>>>
>> >>>>>
>> >>>>> On Fri, Jun 10, 2011 at 3:14 AM, Philip Nelson <me@pnelson.ca>
>> wrote:
>> >>>>>>
>> >>>>>> At risk of embarrassing myself I whipped up a quick example of what
>> I
>> >>>>>> was talking about. I haven't tested it as I don't have a Python
>> interpreter
>> >>>>>> available at the moment.
>> >>>>>>
>> >>>>>> PyMongo creates the database or collection when used if it isn't
>> >>>>>> already created. I just made two routes. One to insert, and one to
>> find all
>> >>>>>> records. If all goes well, the insert should show the ID of the new
>> record
>> >>>>>> and the find should show the IDs of all records.
>> >>>>>>
>> >>>>>> from flask import Flask
>> >>>>>> from pymongo import Connection
>> >>>>>>
>> >>>>>> app = Flask(__name__)
>> >>>>>>
>> >>>>>> # create the connection in global scope
>> >>>>>> db_connection = Connection() # uses default port, etc
>> >>>>>> db = connection.some_database_name
>> >>>>>>
>> >>>>>> # free the threads connection to the thread pool for future
>> requests
>> >>>>>> @app.after_request
>> >>>>>> def free_sockets(request):
>> >>>>>>   connection.end_request()
>> >>>>>>   return request
>> >>>>>>
>> >>>>>> @app.route("/")
>> >>>>>> def find():
>> >>>>>>   results = db.some_collection_name.find()
>> >>>>>>   return "".join(result['_id'] for result in results)
>> >>>>>>
>> >>>>>> @app.route("/<data>/")
>> >>>>>> def insert(data):
>> >>>>>>   return str(db.some_collection_name.insert({'data': data}))
>> >>>>>>
>> >>>>>> if __name__ == "__main__"
>> >>>>>>   app.run(debug=True)
>> >>>>>>
>> >>>>>> Phil
>> >>>>>>
>> >>>>>>
>> >>>>>> On Fri, Jun 10, 2011 at 2:55 AM, Brent Tubbs <
>> brent.tubbs@gmail.com>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> I made a toy Flask app that uses MongoEngine to record page views.
>> >>>>>>> See https://bitbucket.org/btubbs/flask_mongoengine/src/
>> >>>>>>>
>> >>>>>>> As mentioned by others, there's nothing forcing you to use
>> >>>>>>> MongoEngine
>> >>>>>>> if you'd rather not.  I like it for the declarative way of setting
>> up
>> >>>>>>> validation of your data, and for giving an obvious place to stick
>> >>>>>>> methods if you're taking an object oriented approach.  Your
>> mileage
>> >>>>>>> may vary.
>> >>>>>>>
>> >>>>>>> On Fri, Jun 10, 2011 at 6:42 AM, Alessio Civitillo
>> >>>>>>> <alessiocivitillo@gmail.com> wrote:
>> >>>>>>> > @Swaroop, thanks, but this is my first application development
>> (I
>> >>>>>>> > mainly
>> >>>>>>> > used python for some data analysis), so I am interested in a
>> lower
>> >>>>>>> > level of
>> >>>>>>> > abstraction to understand as much as possible the whole thing.
>> This
>> >>>>>>> > is also
>> >>>>>>> > the reason why I am not using Django or web2py.
>> >>>>>>> >
>> >>>>>>> > @Philip, that is exactly what I would like to be able to do. I
>> said
>> >>>>>>> > mongoengine, but if I am able to use pymongo I would be a lot
>> >>>>>>> > happier. Could
>> >>>>>>> > you please make me a small example? I am completely new to
>> flask.
>> >>>>>>> >
>> >>>>>>> > On Fri, Jun 10, 2011 at 8:17 AM, Philip Nelson <me@pnelson.ca>
>> >>>>>>> > wrote:
>> >>>>>>> >>
>> >>>>>>> >> Another idea worth experimenting with is just using the raw
>> >>>>>>> >> PyMongo API
>> >>>>>>> >> itself. I haven't done much with it (read: purely in dev) but
>> >>>>>>> >> aside from
>> >>>>>>> >> fixing up handing for AutoReconnect, there hasn't been any
>> reason
>> >>>>>>> >> for me to
>> >>>>>>> >> want the extra abstraction. If you go this route, you just
>> create
>> >>>>>>> >> the db
>> >>>>>>> >> connection in global scope and call the connections end_request
>> >>>>>>> >> method in an
>> >>>>>>> >> applications after_request handler.
>> >>>>>>> >>
>> >>>>>>> >> Phil
>> >>>>>>> >>
>> >>>>>>> >>
>> >>>>>>> >> On Fri, Jun 10, 2011 at 2:01 AM, Swaroop C H
>> >>>>>>> >> <swaroop@swaroopch.com>
>> >>>>>>> >> wrote:
>> >>>>>>> >>>
>> >>>>>>> >>> Alessio,
>> >>>>>>> >>> Have you tried using something like Flask-MongoObject
>> >>>>>>> >>>
>> >>>>>>> >>> :
>> https://github.com/dqminh/flask-mongoobject/blob/master/examples_hello.py
>>  or
>> >>>>>>> >>> Flask-MongoAlchemy
>> >>>>>>> >>> : http://packages.python.org/Flask-MongoAlchemy/ ?
>> >>>>>>> >>> They have good getting-started examples for both on those
>> pages.
>> >>>>>>> >>> Regards,
>> >>>>>>> >>> Swaroop
>> >>>>>>> >>>
>> >>>>>>> >>> On Fri, Jun 10, 2011 at 10:27 AM, Alessio Civitillo
>> >>>>>>> >>> <alessiocivitillo@gmail.com> wrote:
>> >>>>>>> >>>>
>> >>>>>>> >>>> @Brent,
>> >>>>>>> >>>> I am doing that (I guess). I am running it in __init__.py,
>> where
>> >>>>>>> >>>> I set
>> >>>>>>> >>>> up the app. But how do I make available that connection to
>> views
>> >>>>>>> >>>> and models?
>> >>>>>>> >>>> If I try to do save(), it tells me there is no connection
>> >>>>>>> >>>> available.
>> >>>>>>> >>>>
>> >>>>>>> >>>> On Fri, Jun 10, 2011 at 3:46 AM, Brent Tubbs
>> >>>>>>> >>>> <brent.tubbs@gmail.com>
>> >>>>>>> >>>> wrote:
>> >>>>>>> >>>>>
>> >>>>>>> >>>>> Run mongoengine.connect() in the main file where you set up
>> the
>> >>>>>>> >>>>> root
>> >>>>>>> >>>>> of your application.
>> >>>>>>> >>>>>
>> >>>>>>> >>>>> On Thu, Jun 9, 2011 at 7:54 PM, JimG <j.gumbley@gmail.com>
>> >>>>>>> >>>>> wrote:
>> >>>>>>> >>>>> > If you are using mongo as your database for a large
>> >>>>>>> >>>>> > application then
>> >>>>>>> >>>>> > be sure
>> >>>>>>> >>>>> > the application you have in mind is appropriate to the
>> >>>>>>> >>>>> > technologies
>> >>>>>>> >>>>> > you have
>> >>>>>>> >>>>> > chosen.
>> >>>>>>> >>>>> >
>> >>>>>>> >>>>> > On 9 June 2011 20:27, Alessio Civitillo
>> >>>>>>> >>>>> > <alessiocivitillo@gmail.com>
>> >>>>>>> >>>>> > wrote:
>> >>>>>>> >>>>> >>
>> >>>>>>> >>>>> >> Hello,
>> >>>>>>> >>>>> >> I was trying to use mongoengine in a flask large
>> application
>> >>>>>>> >>>>> >> setup.
>> >>>>>>> >>>>> >> I have
>> >>>>>>> >>>>> >> the models separeted in a folder. The problem is that I
>> >>>>>>> >>>>> >> don't
>> >>>>>>> >>>>> >> understand
>> >>>>>>> >>>>> >> well where to create the db connection and how to be able
>> to
>> >>>>>>> >>>>> >> use
>> >>>>>>> >>>>> >> that in
>> >>>>>>> >>>>> >> models modules.
>> >>>>>>> >>>>> >> Thanks
>> >>>>>>> >>>>> >> a.c.
>> >>>>>>> >>>>> >
>> >>>>>>> >>>>
>> >>>>>>> >>>>
>> >>>>>>> >>>>
>> >>>>>>> >>>> --
>> >>>>>>> >>>> Regards,
>> >>>>>>> >>>> ------------------------------------
>> >>>>>>> >>>> Alessio Civitillo
>> >>>>>>> >>>> alessiocivitillo@gmail.com
>> >>>>>>> >>>> Mobile: (0045) 52645608
>> >>>>>>> >>>> Linkedin: http://it.linkedin.com/in/alessiocivitillo
>> >>>>>>> >>>
>> >>>>>>> >>
>> >>>>>>> >
>> >>>>>>> >
>> >>>>>>> >
>> >>>>>>> > --
>> >>>>>>> > Regards,
>> >>>>>>> > ------------------------------------
>> >>>>>>> > Alessio Civitillo
>> >>>>>>> > alessiocivitillo@gmail.com
>> >>>>>>> > Mobile: (0045) 52645608
>> >>>>>>> > Linkedin: http://it.linkedin.com/in/alessiocivitillo
>> >>>>>>> >
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Regards,
>> >> ------------------------------------
>> >> Alessio Civitillo
>> >> alessiocivitillo@gmail.com
>> >> Mobile: (0045) 52645608
>> >> Linkedin: http://it.linkedin.com/in/alessiocivitillo
>> >
>> >
>>
>
>
>
> --
> Regards,
> ------------------------------------
> Alessio Civitillo
> alessiocivitillo@gmail.com
> Mobile: (0045) 52645608
> Linkedin: http://it.linkedin.com/in/alessiocivitillo
>



-- 
Regards,
------------------------------------
Alessio Civitillo
alessiocivitillo@gmail.com
Mobile: (0045) 52645608
Linkedin: http://it.linkedin.com/in/alessiocivitillo