librelist archives

« back to archive

Finding a solution to Connection Pooling

Finding a solution to Connection Pooling

From:
Raj Nathani
Date:
2012-11-26 @ 19:26
For a while I've been trying to build a common connection pool for a flask
application.

Before I dive into the code I would like to mention that I'm not using an 
ORM as a whole. This is just a tiny experiment where I'm simply trying to 
find the most efficient way of dealing with connection pooling in the raw 
state. However I am using QueuePool from SQLAlchemy (lib: sqlalchemy.pool)

Inspired from the example (http://flask.pocoo.org/docs/patterns/sqlite3/) 
of pooling I've managed to construct a request scoped connection pool. It 
looks like this:

Disclosure: I'm writing this mail from a handheld device, and therefore 
due to not having the code with me right now, there might be occurrence of
discreptancies in the following code

def getconn(...):
    ...
    return con

import sqlalchemy.pool as pool
@app.before_request
def before_request()
    g.pool = pool.QueuePool(conn=getconn(...), size=?, max_overflow=?)

The above implementation of a connection pool works, however it works only
within the scope of a request. After the request has been processed, an 
after_request decorated function disposes off the connection pool created 
(I haven't illustrated this part in the code above)

Method 2, involves creating a separate process for connections. This way 
the context of the request doesn't matter.

I haven't tried/tested the above method as yet. If any of you'll have, or 
have any other method to suggest, please reply back and we can have a 
discussion about this.