librelist archives

« back to archive

Flask, a jinja2 template and generators

Flask, a jinja2 template and generators

From:
Daniele Sluijters
Date:
2013-08-05 @ 19:00
Hi everyone,

In Flask I have a view along these lines:

@app.route('/awesome'):
def awesome():
  data1 = backend.get(some_data_set)
  data2 = backend.get(some_other_data_set)
  return render_template('awesome.html', data1=data1, data2=data2)

The problem with data1 and data2 is that they're generators as they
can return a lot of data from my backend. Now I can iterate over
those just fine in a template but a problem arises when said generator
raises an error which can happen if a request to the backend happens
to fail for some reason. Since we're already in the template at this point
I see no way to do error/exception handling.

One way I could solve this is to consume the generator beforehand
and tranform it into a list but then I loose all the advantages of my
generators. The other way I could see was to alter backend.get()
in that instead of raising it would just return empty and check on
that in my template to break out of the iteration.

I also had a look at http://flask.pocoo.org/docs/patterns/streaming/
but I'm not entirely sure how to apply that in this case.

I was hoping someone could show me the way forward.


Kind regards,

-- 
Daniele Sluijters

Re: [flask] Flask, a jinja2 template and generators

From:
David Baumgold
Date:
2013-08-05 @ 19:17
Sounds like you just want a wrapper function in your view function: 

def ignore_generator_errors(gen):
    while True:
        try:
            yield next(gen)
        except StopIteration:
            raise
        except Exception:
            continue

@app.route('/awesome'):
def awesome():
    data1 = ignore_generator_errors(backend.get(some_data_set))
    data2 = ignore_generator_errors(backend.get(other_data_set))
    return render_template("awesome.html", data1=data1, data2=data2)


On Monday, August 5, 2013 at 3:00 PM, Daniele Sluijters wrote:

> Hi everyone,
> 
> In Flask I have a view along these lines:
> 
> @app.route('/awesome'):
> def awesome():
> data1 = backend.get(some_data_set)
> data2 = backend.get(some_other_data_set)
> return render_template('awesome.html', data1=data1, data2=data2)
> 
> The problem with data1 and data2 is that they're generators as they
> can return a lot of data from my backend. Now I can iterate over
> those just fine in a template but a problem arises when said generator
> raises an error which can happen if a request to the backend happens
> to fail for some reason. Since we're already in the template at this point
> I see no way to do error/exception handling.
> 
> One way I could solve this is to consume the generator beforehand
> and tranform it into a list but then I loose all the advantages of my
> generators. The other way I could see was to alter backend.get()
> in that instead of raising it would just return empty and check on
> that in my template to break out of the iteration.
> 
> I also had a look at http://flask.pocoo.org/docs/patterns/streaming/
> but I'm not entirely sure how to apply that in this case.
> 
> I was hoping someone could show me the way forward.
> 
> 
> Kind regards,
> 
> -- 
> Daniele Sluijters


Re: [flask] Flask, a jinja2 template and generators

From:
Steven Kryskalla
Date:
2013-08-05 @ 19:31
On Mon, Aug 5, 2013 at 12:00 PM, Daniele Sluijters
<daniele.sluijters@nedap.com> wrote:
> One way I could solve this is to consume the generator beforehand
> and tranform it into a list but then I loose all the advantages of my
> generators.

You already need to completely consume the generator in order to send
the response, what is the difference between consuming in the request
function vs. consuming in the template? They should perform the same.

If the generator is very large, you will run into problems anyway with
the request taking too long or consuming too much memory. In that case
you would need to use an async approach to send the generator without
consuming it all at once.

-Steve

Re: [flask] Flask, a jinja2 template and generators

From:
Daniele Sluijters
Date:
2013-08-06 @ 06:50
Hi

You already need to completely consume the generator in order to send
the response, what is the difference between consuming in the request
function vs. consuming in the template? They should perform the same.

That’s a very good point, completely forgot about that. Some wires got
crossed in my head about generators and streaming output to the client.

def ignore_generator_errors(gen):
   while True:
       try:
           yield next(gen)
       except StopIteration:
           raise
       except Exception:
           continue

Thanks for that, hadn’t considered doing that just yet, solves the issue
quite nicely.

Kind regards,

--
Daniele Sluijters
w: http://www.nedap.com
e: daniele.sluijters@nedap.com<mailto:daniele.sluijters@nedap.com>
t: +31 (0) 544 471 1764

N.V. Nederlandsche Apparatenfabriek 'Nedap'
Parallelweg 2 NL-7141 DC GROENLO
Trade Register 08013836

On 5 Aug 2013, at 21:31, Steven Kryskalla 
<skryskalla@gmail.com<mailto:skryskalla@gmail.com>> wrote:

On Mon, Aug 5, 2013 at 12:00 PM, Daniele Sluijters
<daniele.sluijters@nedap.com<mailto:daniele.sluijters@nedap.com>> wrote:
One way I could solve this is to consume the generator beforehand
and tranform it into a list but then I loose all the advantages of my
generators.


If the generator is very large, you will run into problems anyway with
the request taking too long or consuming too much memory. In that case
you would need to use an async approach to send the generator without
consuming it all at once.

-Steve