librelist archives

« back to archive

Waiting on I/O

Waiting on I/O

From:
Gary Chambers
Date:
2014-01-14 @ 21:47
All,

I have written a simple test web app to return some temperature sensor data
in JSON:

@app.route('/api/')
def index():
     s = dsts.DS18XSensor()
     temp_c = s.read_temp(s.sensors[0])
     return jsonify({'epochtime': int(time.time()), 'c': temp_c})

When I attempt to access it with:

wget -O - -q http://127.0.0.1:5000/api/

It always fails with the following:

127.0.0.1 - - [14/Jan/2014 16:36:00] "GET /api/ HTTP/1.1" 500 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 46114)
Traceback (most recent call last):
   File "/usr/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock
     self.process_request(request, client_address)
   File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
     self.finish_request(request, client_address)
   File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
     self.RequestHandlerClass(request, client_address, self)
   File "/usr/lib/python2.7/SocketServer.py", line 651, in __init__
     self.finish()
   File "/usr/lib/python2.7/SocketServer.py", line 704, in finish
     self.wfile.flush()
   File "/usr/lib/python2.7/socket.py", line 303, in flush
     self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------

My only guess is that the call to s.read_temp(), which takes around 1 second
to return, is taking too long.  I've found numerous search engine hits on
that error text, but none appear to be relevant.

I've tried running with threaded=True, but that, too (though not
surprisingly), fails.  I've also been searching for blocking I/O, but I have
found nothing helpful.

Thank you in advance for any advice or insight you can provide.

--
G.

Re: [flask] Waiting on I/O

From:
Farhan Ahmed
Date:
2014-01-14 @ 23:33
What does s.read_temp() method returns?

Best,
Farhan : )

> On Jan 14, 2014, at 3:47 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:
> 
> All,
> 
> I have written a simple test web app to return some temperature sensor data
> in JSON:
> 
> @app.route('/api/')
> def index():
>     s = dsts.DS18XSensor()
>     temp_c = s.read_temp(s.sensors[0])
>     return jsonify({'epochtime': int(time.time()), 'c': temp_c})
> 
> When I attempt to access it with:
> 
> wget -O - -q http://127.0.0.1:5000/api/
> 
> It always fails with the following:
> 
> 127.0.0.1 - - [14/Jan/2014 16:36:00] "GET /api/ HTTP/1.1" 500 -
> ----------------------------------------
> Exception happened during processing of request from ('127.0.0.1', 46114)
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/SocketServer.py", line 295, in 
_handle_request_noblock
>     self.process_request(request, client_address)
>   File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
>     self.finish_request(request, client_address)
>   File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
>     self.RequestHandlerClass(request, client_address, self)
>   File "/usr/lib/python2.7/SocketServer.py", line 651, in __init__
>     self.finish()
>   File "/usr/lib/python2.7/SocketServer.py", line 704, in finish
>     self.wfile.flush()
>   File "/usr/lib/python2.7/socket.py", line 303, in flush
>     self._sock.sendall(view[write_offset:write_offset+buffer_size])
> error: [Errno 32] Broken pipe
> ----------------------------------------
> 
> My only guess is that the call to s.read_temp(), which takes around 1 second
> to return, is taking too long.  I've found numerous search engine hits on
> that error text, but none appear to be relevant.
> 
> I've tried running with threaded=True, but that, too (though not
> surprisingly), fails.  I've also been searching for blocking I/O, but I have
> found nothing helpful.
> 
> Thank you in advance for any advice or insight you can provide.
> 
> --
> G.

Re: [flask] Waiting on I/O

From:
Gary Chambers
Date:
2014-01-15 @ 00:03
It returns a float, Farhan.

--
G.
// From the iPad

On Jan 14, 2014, at 6:33 PM, Farhan Ahmed <inshany@gmail.com> wrote:

> What does s.read_temp() method returns?
> 
> Best,
> Farhan : )
> 
>> On Jan 14, 2014, at 3:47 PM, Gary Chambers <gwchamb@gwcmail.com> wrote:
>> 
>> All,
>> 
>> I have written a simple test web app to return some temperature sensor data
>> in JSON:
>> 
>> @app.route('/api/')
>> def index():
>>    s = dsts.DS18XSensor()
>>    temp_c = s.read_temp(s.sensors[0])
>>    return jsonify({'epochtime': int(time.time()), 'c': temp_c})
>> 
>> When I attempt to access it with:
>> 
>> wget -O - -q http://127.0.0.1:5000/api/
>> 
>> It always fails with the following:
>> 
>> 127.0.0.1 - - [14/Jan/2014 16:36:00] "GET /api/ HTTP/1.1" 500 -
>> ----------------------------------------
>> Exception happened during processing of request from ('127.0.0.1', 46114)
>> Traceback (most recent call last):
>>  File "/usr/lib/python2.7/SocketServer.py", line 295, in 
_handle_request_noblock
>>    self.process_request(request, client_address)
>>  File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
>>    self.finish_request(request, client_address)
>>  File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
>>    self.RequestHandlerClass(request, client_address, self)
>>  File "/usr/lib/python2.7/SocketServer.py", line 651, in __init__
>>    self.finish()
>>  File "/usr/lib/python2.7/SocketServer.py", line 704, in finish
>>    self.wfile.flush()
>>  File "/usr/lib/python2.7/socket.py", line 303, in flush
>>    self._sock.sendall(view[write_offset:write_offset+buffer_size])
>> error: [Errno 32] Broken pipe
>> ----------------------------------------
>> 
>> My only guess is that the call to s.read_temp(), which takes around 1 second
>> to return, is taking too long.  I've found numerous search engine hits on
>> that error text, but none appear to be relevant.
>> 
>> I've tried running with threaded=True, but that, too (though not
>> surprisingly), fails.  I've also been searching for blocking I/O, but I have
>> found nothing helpful.
>> 
>> Thank you in advance for any advice or insight you can provide.
>> 
>> --
>> G.

Re: [flask] Waiting on I/O

From:
Emanuil Tolev
Date:
2014-01-14 @ 23:59
Can you replicate the logic of your code

> s = dsts.DS18XSensor()
> temp_c = s.read_temp(s.sensors[0])  # does this cause an error? how long
does it take?

> print temp_c  # did you get the data you wanted?

in the Python interpreter? This DS18XSensor thing looks like some
third-party module functionality. So, does it work at all would be the
first thing to investigate. If you can't get the data, Flask will have no
more luck than you. If so, you're probably best off asking the authors /
community around whatever is providing you with DS18XSensor why it errors
out :).

Your tracelog doesn't indicate which line actually caused the error, which
is a bit strange. If it's truly a Flask or Werkzeug (Flask's middleware)
error, then I've no idea what could be causing it - your return
jsonify(...) looks fine to me.

1 second should not cause any sort of timeout with the default config.
Besides, it doesn't really matter how long it takes, it's your code -
Flask/Werkzeug aren't going to just stop execution in the middle of your
code on their own (obviously your browser client, like curl or wget, or GUI
ones like Firefox - might implement a timeout).

Greetings,
Emanuil

On Tuesday, 14 January 2014, Gary Chambers wrote:

> All,
>
> I have written a simple test web app to return some temperature sensor data
> in JSON:
>
> @app.route('/api/')
> def index():
>      s = dsts.DS18XSensor()
>      temp_c = s.read_temp(s.sensors[0])
>      return jsonify({'epochtime': int(time.time()), 'c': temp_c})
>
> When I attempt to access it with:
>
> wget -O - -q http://127.0.0.1:5000/api/
>
> It always fails with the following:
>
> 127.0.0.1 - - [14/Jan/2014 16:36:00] "GET /api/ HTTP/1.1" 500 -
> ----------------------------------------
> Exception happened during processing of request from ('127.0.0.1', 46114)
> Traceback (most recent call last):
>    File "/usr/lib/python2.7/SocketServer.py", line 295, in
> _handle_request_noblock
>      self.process_request(request, client_address)
>    File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
>      self.finish_request(request, client_address)
>    File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
>      self.RequestHandlerClass(request, client_address, self)
>    File "/usr/lib/python2.7/SocketServer.py", line 651, in __init__
>      self.finish()
>    File "/usr/lib/python2.7/SocketServer.py", line 704, in finish
>      self.wfile.flush()
>    File "/usr/lib/python2.7/socket.py", line 303, in flush
>      self._sock.sendall(view[write_offset:write_offset+buffer_size])
> error: [Errno 32] Broken pipe
> ----------------------------------------
>
> My only guess is that the call to s.read_temp(), which takes around 1
> second
> to return, is taking too long.  I've found numerous search engine hits on
> that error text, but none appear to be relevant.
>
> I've tried running with threaded=True, but that, too (though not
> surprisingly), fails.  I've also been searching for blocking I/O, but I
> have
> found nothing helpful.
>
> Thank you in advance for any advice or insight you can provide.
>
> --
> G.
>

Re: [flask] Waiting on I/O

From:
Gary Chambers
Date:
2014-01-15 @ 00:46
Emanuil,

Thank you for taking the time to reply.

> Can you replicate the logic of your code

The module code is at https://gist.github.com/gwchamb/8372505.  I'm using it
on a Raspberry Pi with the Maxim DS18B20 sensor connected to the GPIO bus.

>> s = dsts.DS18XSensor()
>> temp_c = s.read_temp(s.sensors[0])  # does this
>> cause an error? how long does it take?

It takes a bit under a second for the sensor to determine the temperature
and return the data.  The sensor data is contained in Linux sysfs that my
module parses.  If I call the module test code, it works.

>> print temp_c  # did you get the data you wanted?

Yes.  Here's the output from a recent execution:

$ sudo ./dsts.py 
Found 1 sensor: 28-000005605ef4
Sensor 28-000005605ef4 temperature: 16.437*C

> in the Python interpreter? This DS18XSensor thing looks like some
> third-party module functionality. So, does it work at all would be the
> first thing to investigate.

Yes -- guilty as charged!  :)  The code may not be pretty, but it does work
outside of my simple Flask app to return the JSON.

> If you can't get the data, Flask will have no more luck than you. If so,
> you're probably best off asking the authors / community around whatever is
> providing you with DS18XSensor why it errors out :).

I agree.  :)  I'm also using network-based temperature sensor that works
fine (both module and Flask app).

> Your tracelog doesn't indicate which line actually caused the error, which
> is a bit strange. If it's truly a Flask or Werkzeug (Flask's middleware)
> error, then I've no idea what could be causing it - your return
> jsonify(...) looks fine to me.

The tracelog was so short, I initially thought I had forgotten to enable
debug.  The text that appeared in my initial request is all I get, though.

> 1 second should not cause any sort of timeout with the default config.
> Besides, it doesn't really matter how long it takes, it's your code -
> Flask/Werkzeug aren't going to just stop execution in the middle of your
> code on their own (obviously your browser client, like curl or wget, or
> GUI ones like Firefox - might implement a timeout).

I wouldn't have thought so, either.  It was just a guess that I ventured due
to one of top lines of the tracelog mentioning non-blocking:

File "/usr/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock

Thank you again for your feedback.  I'd like to get this working the way I'm
trying to.  I have a workaround that will require another process to write
the sensor data to a file that is then read by the web app and returned.

--
G.

Re: [flask] Waiting on I/O

From:
Jóan Petur Petersen
Date:
2014-01-15 @ 07:00
Hi Gary,

if you think that it is the s.read_temp() that takes too long - maybe you
should replace it with a time.sleep(1) and se if it reproduces the error?

/jpp



On Wed, Jan 15, 2014 at 1:46 AM, Gary Chambers <gwchamb@gwcmail.com> wrote:

> Emanuil,
>
> Thank you for taking the time to reply.
>
> > Can you replicate the logic of your code
>
> The module code is at https://gist.github.com/gwchamb/8372505.  I'm using
> it
> on a Raspberry Pi with the Maxim DS18B20 sensor connected to the GPIO bus.
>
> >> s = dsts.DS18XSensor()
> >> temp_c = s.read_temp(s.sensors[0])  # does this
> >> cause an error? how long does it take?
>
> It takes a bit under a second for the sensor to determine the temperature
> and return the data.  The sensor data is contained in Linux sysfs that my
> module parses.  If I call the module test code, it works.
>
> >> print temp_c  # did you get the data you wanted?
>
> Yes.  Here's the output from a recent execution:
>
> $ sudo ./dsts.py
> Found 1 sensor: 28-000005605ef4
> Sensor 28-000005605ef4 temperature: 16.437*C
>
> > in the Python interpreter? This DS18XSensor thing looks like some
> > third-party module functionality. So, does it work at all would be the
> > first thing to investigate.
>
> Yes -- guilty as charged!  :)  The code may not be pretty, but it does work
> outside of my simple Flask app to return the JSON.
>
> > If you can't get the data, Flask will have no more luck than you. If so,
> > you're probably best off asking the authors / community around whatever
> is
> > providing you with DS18XSensor why it errors out :).
>
> I agree.  :)  I'm also using network-based temperature sensor that works
> fine (both module and Flask app).
>
> > Your tracelog doesn't indicate which line actually caused the error,
> which
> > is a bit strange. If it's truly a Flask or Werkzeug (Flask's middleware)
> > error, then I've no idea what could be causing it - your return
> > jsonify(...) looks fine to me.
>
> The tracelog was so short, I initially thought I had forgotten to enable
> debug.  The text that appeared in my initial request is all I get, though.
>
> > 1 second should not cause any sort of timeout with the default config.
> > Besides, it doesn't really matter how long it takes, it's your code -
> > Flask/Werkzeug aren't going to just stop execution in the middle of your
> > code on their own (obviously your browser client, like curl or wget, or
> > GUI ones like Firefox - might implement a timeout).
>
> I wouldn't have thought so, either.  It was just a guess that I ventured
> due
> to one of top lines of the tracelog mentioning non-blocking:
>
> File "/usr/lib/python2.7/SocketServer.py", line 295, in
> _handle_request_noblock
>
> Thank you again for your feedback.  I'd like to get this working the way
> I'm
> trying to.  I have a workaround that will require another process to write
> the sensor data to a file that is then read by the web app and returned.
>
> --
> G.
>

Re: [flask] Waiting on I/O

From:
Wolf-Dietrich Seidlitz
Date:
2014-01-15 @ 09:45
The error "Broken Pipe" usually happens when the client closes the connection
before the server has flushed its buffers. This may the case, because of
the return code 500. Have you tried to wrap the line

     temp_c = s.read_temp(s.sensors[0])

with try ... except?

Regards, Wolf

On Tue, 14 Jan 2014 22:47:11 +0100, Gary Chambers <gwchamb@gwcmail.com> wrote:

> All,
>
> I have written a simple test web app to return some temperature sensor data
> in JSON:
>
> @app.route('/api/')
> def index():
>      s = dsts.DS18XSensor()
>      temp_c = s.read_temp(s.sensors[0])
>      return jsonify({'epochtime': int(time.time()), 'c': temp_c})
>
> When I attempt to access it with:
>
> wget -O - -q http://127.0.0.1:5000/api/
>
> It always fails with the following:
>
> 127.0.0.1 - - [14/Jan/2014 16:36:00] "GET /api/ HTTP/1.1" 500 -
> ----------------------------------------
> Exception happened during processing of request from ('127.0.0.1', 46114)
> Traceback (most recent call last):
>    File "/usr/lib/python2.7/SocketServer.py", line 295, in 
_handle_request_noblock
>      self.process_request(request, client_address)
>    File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
>      self.finish_request(request, client_address)
>    File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
>      self.RequestHandlerClass(request, client_address, self)
>    File "/usr/lib/python2.7/SocketServer.py", line 651, in __init__
>      self.finish()
>    File "/usr/lib/python2.7/SocketServer.py", line 704, in finish
>      self.wfile.flush()
>    File "/usr/lib/python2.7/socket.py", line 303, in flush
>      self._sock.sendall(view[write_offset:write_offset+buffer_size])
> error: [Errno 32] Broken pipe
> ----------------------------------------
>
> My only guess is that the call to s.read_temp(), which takes around 1 second
> to return, is taking too long.  I've found numerous search engine hits on
> that error text, but none appear to be relevant.
>
> I've tried running with threaded=True, but that, too (though not
> surprisingly), fails.  I've also been searching for blocking I/O, but I have
> found nothing helpful.
>
> Thank you in advance for any advice or insight you can provide.
>
> --
> G.
>


-- 
Wolf-Dietrich Seidlitz
Weichselstr. 62
12043 Berlin
t: +49 30 63914471
m: +49 17678645199
f: +49 32121133962

Re: [flask] Waiting on I/O

From:
Gary Chambers
Date:
2014-01-15 @ 15:29
All,

I've finally discovered the problem after stripping code from the module
until the errors disappeared.

> The error "Broken Pipe" usually happens when the client closes the
> connection before the server has flushed its buffers. This may the case,
> because of the return code 500. Have you tried to wrap the line

I can't explain why it wasn't reported by the Python interpreter (either
from the web app or the command line test), but the bug is on line 105 in
the code at https://gist.github.com/gwchamb/8372505 where I had written
s.sensors instead of self.sensors.

> with try ... except?

I did wrap it in try/except, but only the same cryptic, unhelpful traceback
(quoted below) was returned.

>> It always fails with the following:
>>
>> 127.0.0.1 - - [14/Jan/2014 16:36:00] "GET /api/ HTTP/1.1" 500 -
>> ----------------------------------------
>> Exception happened during processing of request from ('127.0.0.1', 46114)
>> Traceback (most recent call last):
>>    File "/usr/lib/python2.7/SocketServer.py", line 295, in 
_handle_request_noblock
>>      self.process_request(request, client_address)
>>    File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
>>      self.finish_request(request, client_address)
>>    File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
>>      self.RequestHandlerClass(request, client_address, self)
>>    File "/usr/lib/python2.7/SocketServer.py", line 651, in __init__
>>      self.finish()
>>    File "/usr/lib/python2.7/SocketServer.py", line 704, in finish
>>      self.wfile.flush()
>>    File "/usr/lib/python2.7/socket.py", line 303, in flush
>>      self._sock.sendall(view[write_offset:write_offset+buffer_size])
>> error: [Errno 32] Broken pipe
>> ----------------------------------------

Thank you to everyone for your suggestions.  If I were to offer a SWAG, I'd
say it was something specific to the Raspberry Pi versions of the software.
Regardless, it was definitely an odd problem to me.  If anyone is
interested, I can post the functioning code with the latest updates.

--
G.