librelist archives

« back to archive

Concurrency frameworks in development vs production

Concurrency frameworks in development vs production

From:
Haq, Salman
Date:
2012-12-10 @ 18:58
Hi,

What is considered the best practice for abstracting the concurrency framework?

Eg: During development, I want to use Flask's (single task) http server 
because it has a great debugger built-in. But during production, I may 
want to deploy it using Gunicorn (using gevent/greenlet concurrency).

The dilemma here is that if I am using the python-request module, I have 
to either use 'requests' or 'grequests' based on the concurrency framework
in use (if any).  How do you deal with this dichotomy?

Thanks,
Salman

Re: [flask] Concurrency frameworks in development vs production

From:
kracekumar ramaraju
Date:
2012-12-10 @ 19:20
On Tue, Dec 11, 2012 at 12:28 AM, Haq, Salman <Salman.Haq@neustar.biz>wrote:

> Hi,
>
> What is considered the best practice for abstracting the concurrency
> framework?
>
> Eg: During development, I want to use Flask's (single task) http server
> because it has a great debugger built-in. But during production, I may want
> to deploy it using Gunicorn (using gevent/greenlet concurrency).
>
> The dilemma here is that if I am using the python-request module, I have
> to either use 'requests' or 'grequests' based on the concurrency framework
> in use (if any).  How do you deal with this dichotomy?
>
> Last time I checked, It din work.

Usecase: 1000 JSON to be uploaded to endpoint, So I used gevent + requests
with flask werkzeug(view -> upload api using request), I ended up getting
'Working out of request context', then replaced wekzeug with gevent.wsgi,
still I had same issue.

If concurrency is prime focus you should also look into  tornado or
brubeck.

> Thanks,
> Salman
>



-- 
*
Thanks & Regards

"Talk is cheap, show me the code" -- Linus Torvalds
kracekumar
www.kracekumar.com
*

Re: [flask] Concurrency frameworks in development vs production

From:
Audrius Kažukauskas
Date:
2012-12-10 @ 19:40
On Mon, 2012-12-10 at 13:58:31 -0500, Haq, Salman wrote:
> Eg: During development, I want to use Flask's (single task) http
> server because it has a great debugger built-in. But during
> production, I may want to deploy it using Gunicorn (using
> gevent/greenlet concurrency).
> 
> The dilemma here is that if I am using the python-request module, I
> have to either use 'requests' or 'grequests' based on the concurrency
> framework in use (if any).  How do you deal with this dichotomy?

You could do conditional imports and use some flag in your config or
even app.debug to decide what to import:

  if app.debug:
      import requests
  else:
      import grequests as requests

Of course, this will work only if those libraries are interchangeable.

HTH,
-- 
Audrius Kažukauskas
http://neutrino.lt/