librelist archives

« back to archive

Outgoing POST in Flask POST handler

Outgoing POST in Flask POST handler

From:
Tobias Oberstein
Date:
2012-07-07 @ 20:19
Hello all,

I am doing an outgoing POST using httplib within a Flask POST handler:


@app.route('/submit1', methods = ['POST'])
def submit1():
    data = request.form['data']
    conn = httplib.HTTPConnection("example.com", 80)
    conn.request('POST', "/forward1", data, ..


Now, this will recreate a new httplib.HTTPConnection on every new 
incoming POST despite the target server for the outgoing POST being fixed.

How can I optimize that so that the httplib.HTTPConnection is reused 
across incoming requests?

I am running Flash via Apache/mod_wsgi. In this setup, it appears that 
having 1 persistent instance of httplib.HTTPConnection per 
thread/process would be desirable. How can I achive that?

Any hints appreciated,
Cheers,
Tobias

Re: [flask] Outgoing POST in Flask POST handler

From:
Armin Ronacher
Date:
2012-07-08 @ 12:33
Hi,

On 7/7/12 9:19 PM, Tobias Oberstein wrote:
> I am running Flash via Apache/mod_wsgi. In this setup, it appears that 
> having 1 persistent instance of httplib.HTTPConnection per 
> thread/process would be desirable. How can I achive that?
If the remote server supports keepalive you can probably just switch
from httplib to requests and make a "pool" for sessions.  That being
said, even if a server supports keepalive what generally happens is that
it disconnects you after ~10 seconds so I don't think that is generally
a feasible thing.


Regards,
Armin

Re: [flask] Outgoing POST in Flask POST handler

From:
Tobias Oberstein
Date:
2012-07-08 @ 13:57
Am 08.07.2012 14:33, schrieb Armin Ronacher:
> Hi,
>
> On 7/7/12 9:19 PM, Tobias Oberstein wrote:
>> I am running Flash via Apache/mod_wsgi. In this setup, it appears that
>> having 1 persistent instance of httplib.HTTPConnection per
>> thread/process would be desirable. How can I achive that?
> If the remote server supports keepalive you can probably just switch
> from httplib to requests and make a "pool" for sessions.  That being
> said, even if a server supports keepalive what generally happens is that
> it disconnects you after ~10 seconds so I don't think that is generally
> a feasible thing.

Thanks for the hint! I would like to test it with Flask (the server 
supports keep alive).

The requests docs say

"Keep-alive and HTTP connection pooling are 100% automatic, powered by 
urllib3, which is embedded within Requests."

"keep-alive is 100% automatic within a session! Any requests that you 
make within a session will automatically reuse the appropriate connection!"

http://docs.python-requests.org/en/latest/user/advanced/#keep-alive
http://kennethreitz.com/major-progress-for-requests.html

===

How does this relate to / how do I ".. make a "pool" for sessions .."?

i.e.

requestpool = requests.session()
requestpool.config['keep_alive'] = True

@app.route('/submit1', methods = ['POST'])
def submit1():
    data = request.form['data']
    r = requestpool.post(...
    ...

OR

@app.before_request
def before_request():
     g.requestpool = requests.session()
     g.requestpool.config['keep_alive'] = True

@app.route('/submit1', methods = ['POST'])
def submit1():
    data = request.form['data']
    r = g.requestpool.post(...
    ...

OR

??

So that the session (= pool?) is not recreated per HTTP request to 
Flask, but only per thread/process running Flash under 
Apache/mod_wsgi/Python?

Thanks again,
Cheers,
Tobias


>
>
> Regards,
> Armin
>