librelist archives

« back to archive

Extend Request Timeout Time

Extend Request Timeout Time

From:
Norbert Markowicz
Date:
2013-05-03 @ 21:50
I'm using flask in order to have the client kick of a search (via ajax) on
the server side and wait for the results to come back and be displayed.
The issue is that the search may take up to a few minutes, but the is
interrupted after 60 seconds or so by another automatic GET request from
the client (with a response of 500).

Is the issue on the server side or with the AJAX call?

Thanks!

Re: [flask] Extend Request Timeout Time

From:
Kurtis Mullins
Date:
2013-05-03 @ 21:58
I'm not sure where the issue is landing (I'm still a Flask newbie) but I'd
recommend Celery for this type of a thing.

http://www.celeryproject.org/


On Fri, May 3, 2013 at 5:50 PM, Norbert Markowicz <
norbert.markowicz@gmail.com> wrote:

> I'm using flask in order to have the client kick of a search (via ajax) on
> the server side and wait for the results to come back and be displayed.
> The issue is that the search may take up to a few minutes, but the is
> interrupted after 60 seconds or so by another automatic GET request from
> the client (with a response of 500).
>
> Is the issue on the server side or with the AJAX call?
>
> Thanks!
>

Re: [flask] Extend Request Timeout Time

From:
Alex Ogier
Date:
2013-05-04 @ 20:25
If you're trying to have a single HTTP request that lasts 60 seconds, you
may encounter a lot of problems. Proxy servers may have trouble with that,
and browsers have limitations on request duration. My guess is that the
limitation you are encountering is a browser or Ajax library timeout. If
it's possible, I would recommend rearchitecting the application, to have
one POST request that kicks off a background task that does the search and
immediately returns a HTTP 200 response with some kind of search ID, then
have a new API that takes a search ID and returns the status and any
possible results that you can poll every few seconds until the search
completes. As Kurtis mentions, the background task could be handled with
Celery and Redis, or any other way you think is appropriate.

Best,
Alex Ogier


On Fri, May 3, 2013 at 5:58 PM, Kurtis Mullins <kurtis.mullins@gmail.com>wrote:

> I'm not sure where the issue is landing (I'm still a Flask newbie) but I'd
> recommend Celery for this type of a thing.
>
> http://www.celeryproject.org/
>
>
> On Fri, May 3, 2013 at 5:50 PM, Norbert Markowicz <
> norbert.markowicz@gmail.com> wrote:
>
>> I'm using flask in order to have the client kick of a search (via ajax)
>> on the server side and wait for the results to come back and be displayed.
>> The issue is that the search may take up to a few minutes, but the is
>> interrupted after 60 seconds or so by another automatic GET request from
>> the client (with a response of 500).
>>
>> Is the issue on the server side or with the AJAX call?
>>
>> Thanks!
>>
>
>

Re: [flask] Extend Request Timeout Time

From:
David Van Duzer
Date:
2013-05-04 @ 20:40
Technically an HTTP 202 would be more appropriate here, but I'm not sure 
how widely implemented that is in third party client libraries.

On May 4, 2013, at 13:25, Alex Ogier <alex.ogier@gmail.com> wrote:

> If you're trying to have a single HTTP request that lasts 60 seconds, 
you may encounter a lot of problems. Proxy servers may have trouble with 
that, and browsers have limitations on request duration. My guess is that 
the limitation you are encountering is a browser or Ajax library timeout. 
If it's possible, I would recommend rearchitecting the application, to 
have one POST request that kicks off a background task that does the 
search and immediately returns a HTTP 200 response with some kind of 
search ID, then have a new API that takes a search ID and returns the 
status and any possible results that you can poll every few seconds until 
the search completes. As Kurtis mentions, the background task could be 
handled with Celery and Redis, or any other way you think is appropriate.
> 
> Best,
> Alex Ogier
> 
> 
> On Fri, May 3, 2013 at 5:58 PM, Kurtis Mullins <kurtis.mullins@gmail.com> wrote:
>> I'm not sure where the issue is landing (I'm still a Flask newbie) but 
I'd recommend Celery for this type of a thing.
>> 
>> http://www.celeryproject.org/
>> 
>> 
>> On Fri, May 3, 2013 at 5:50 PM, Norbert Markowicz 
<norbert.markowicz@gmail.com> wrote:
>>> I'm using flask in order to have the client kick of a search (via 
ajax) on the server side and wait for the results to come back and be 
displayed.
>>> The issue is that the search may take up to a few minutes, but the is 
interrupted after 60 seconds or so by another automatic GET request from 
the client (with a response of 500).
>>> 
>>> Is the issue on the server side or with the AJAX call?
>>> 
>>> Thanks!
> 

Re: [flask] Extend Request Timeout Time

From:
Dave Shawley
Date:
2013-05-14 @ 11:45
I'll agree that re-architecting the application model is the way to go 
here.  This long-standing query is best represented by a server-side 
resource.  Depending on what you are really doing outside of the single 
query, you might want to investigate other techniques such as HTTP 
streaming or Web Sockets (if your environment allows).

I think that I would favor a "201 Created" with a Location header that 
directs the client to the new resource.  Basically, the POST is asking the
server to create a resource to represent the results of the outstanding 
query.  The new resource is what the Location header locates.  The client 
should issue a GET on the new resource to retrieve the query results.  If 
the results are not ready yet, the server could respond with a "408 
Request Timeout" and include a "Retry-After" header.

--
There is a lesson here for ambitious system architects: the most dangerous
enemy of a better solution is an existing codebase that is just good 
enough. -- Eric S. Raymond.

On May 4, 2013, at 4:40 PM, David Van Duzer <dvd@tennica.net> wrote:

> Technically an HTTP 202 would be more appropriate here, but I'm not sure
how widely implemented that is in third party client libraries.
> 
> On May 4, 2013, at 13:25, Alex Ogier <alex.ogier@gmail.com> wrote:
> 
>> If you're trying to have a single HTTP request that lasts 60 seconds, 
you may encounter a lot of problems. Proxy servers may have trouble with 
that, and browsers have limitations on request duration. My guess is that 
the limitation you are encountering is a browser or Ajax library timeout. 
If it's possible, I would recommend rearchitecting the application, to 
have one POST request that kicks off a background task that does the 
search and immediately returns a HTTP 200 response with some kind of 
search ID, then have a new API that takes a search ID and returns t he 
status and any possible results that you can poll every few seconds until 
the search completes. As Kurtis mentions, the background task could be 
handled with Celery and Redis, or any other way you think is appropriate.
>> 
>> Best,
>> Alex Ogier
>> 
>> 
>> On Fri, May 3, 2013 at 5:58 PM, Kurtis Mullins 
<kurtis.mullins@gmail.com> wrote:
>> I'm not sure where the issue is landing (I'm still a Flask newbie) but 
I'd recommend Celery for this type of a thing.
>> 
>> http://www.celeryproject.org/
>> 
>> 
>> On Fri, May 3, 2013 at 5:50 PM, Norbert Markowicz 
<norbert.markowicz@gmail.com> wrote:
>> I'm using flask in order to have the client kick of a search (via ajax)
on the server side and wait for the results to come back and be displayed.
>> The issue is that the search may take up to a few minutes, but the is 
interrupted after 60 seconds or so by another automatic GET request from 
the client (with a response of 500).
>> 
>> Is the issue on the server side or with the AJAX call?
>> 
>> Thanks!
>>