librelist archives

« back to archive

streaming Popen output via HTTP in flask

streaming Popen output via HTTP in flask

From:
JimG
Date:
2011-06-07 @ 14:33
I have a crappy test client for a set of java web services which I've hacked
together using Flask and SUDS.

Keeps me sane when writing Java :D

At one point I need to shell out to a C# client to upload a file (I can't
find a good MTOM library in Python), and given I am the only consumer of the
web app, I don't mind doing this in the scope of the HTTP response, and
having the HTTP response take 20secs or so.

My question is, would it be possible, using subprocess.Popen.communicate()
or some such to have the output stream to the browser (the c# app writes
status as it proceeds), rather than waiting for it to complete and then
forwarding.

And then do a redirect after it finished.

This is just a noddy test app used by 1 person so I'm not concerned about
the rights and wrongs of this, I just want good feedback as I use my
integration test client.

Thanks, Jim.

Re: [flask] streaming Popen output via HTTP in flask

From:
Rob Mela
Date:
2011-06-07 @ 15:55
You can pass any file-like object as the first parameter of Response

Other approaches are also possible.


from flask import Flask, Response
import popen2

app=Flask(__name__)


@app.route('/')
def index():
        std_out_err, std_in = popen2.popen4( "ls" )
        return Response(std_out_err, content_type="text/plain;charset=UTF-8")

app.run(debug=True)

Not sure what popen is like on Windows.


Den Jun 7, 2011 kl. 10:33 AM skrev JimG:

I have a crappy test client for a set of java web services which I've 
hacked together using Flask and SUDS.

Keeps me sane when writing Java :D

At one point I need to shell out to a C# client to upload a file (I can't 
find a good MTOM library in Python), and given I am the only consumer of 
the web app, I don't mind doing this in the scope of the HTTP response, 
and having the HTTP response take 20secs or so.

My question is, would it be possible, using subprocess.Popen.communicate()
or some such to have the output stream to the browser (the c# app writes 
status as it proceeds), rather than waiting for it to complete and then 
forwarding.

And then do a redirect after it finished.

This is just a noddy test app used by 1 person so I'm not concerned about 
the rights and wrongs of this, I just want good feedback as I use my 
integration test client.

Thanks, Jim.

Re: [flask] streaming Popen output via HTTP in flask

From:
Rob Mela
Date:
2011-06-07 @ 16:22
Sorry -- did not read completely

Not sure if returning a file-like object in Response will stream in the 
Flask dev server.  If it does, then great.

If not, then you could try passing a generator function to Response ( one 
that reads and yields input from the pipe ).

        std_out_err, std_in = popen2.popen4( "ls")

        def f():
                for line in std_out_err:
                        yield line

        return Response(f(), content_type="text/plain;charset=UTF-8")

Other issues may apply -- e.g., if the program you're reading from sets 
its stdout buffer to a large size it could be a while before you get any 
input.   If that's an issue then you might try and see if Python lets you 
tweak buffer size on the input stream returned by popen.

Lemme know how it works.

Den Jun 7, 2011 kl. 11:55 AM skrev Rob Mela:

You can pass any file-like object as the first parameter of Response

Other approaches are also possible.


from flask import Flask, Response
import popen2

app=Flask(__name__)


@app.route('/')
def index():
        std_out_err, std_in = popen2.popen4( "ls" )
        return Response(std_out_err, content_type="text/plain;charset=UTF-8")

app.run(debug=True)

Not sure what popen is like on Windows.


Den Jun 7, 2011 kl. 10:33 AM skrev JimG:

I have a crappy test client for a set of java web services which I've 
hacked together using Flask and SUDS.

Keeps me sane when writing Java :D

At one point I need to shell out to a C# client to upload a file (I can't 
find a good MTOM library in Python), and given I am the only consumer of 
the web app, I don't mind doing this in the scope of the HTTP response, 
and having the HTTP response take 20secs or so.

My question is, would it be possible, using subprocess.Popen.communicate()
or some such to have the output stream to the browser (the c# app writes 
status as it proceeds), rather than waiting for it to complete and then 
forwarding.

And then do a redirect after it finished.

This is just a noddy test app used by 1 person so I'm not concerned about 
the rights and wrongs of this, I just want good feedback as I use my 
integration test client.

Thanks, Jim.