librelist archives

« back to archive

Flask-SQLAlchemy and (apparently) celery (or threading?)

Flask-SQLAlchemy and (apparently) celery (or threading?)

From:
Alexander Jung-Loddenkemper
Date:
2012-09-01 @ 12:23
Hello everyone, 

I have been asking and searching on this problem for days now, and I am 
actually not able to work it out.

It recently upgraded to celery3 (and so dropped flask-celery as it is 
deprecated) and since then I have hit a lot of problems. Especially with 
SQLAlchemy databases.

Recently the first user accessing the site in the morning (after the 
website was not used for some time) gets a 500 and my log tells me that it
fails with OperationalError: (OperationalError) (2006, 'MySQL server has 
gone away'). Which should already not be happening as my pool_recycle 
value is smaller than the wait_timeout of the mysql server. The real 
problem is, that it is not working in the next request, but all the 
request that follow fail with "InvalidRequestError: Can't reconnect until 
invalid transaction is rolled back". These errors keep coming up until I 
restart the app.

Recently I do have the feeling, that the problem is caused by celery 
tasks, that write data to the database and because they are not in the app
itself, the clean up might not work (multiple threads suggested you had to
rollback, remove or close the session after the commit was made). On the 
other hand: to make database access possible at all in celery, I had to 
use the following decorator on my celery tasks:

https://gist.github.com/3570563

And as the task is executed in a request context, the session should be 
removed by flask-sqlalchemy, shouldn't it? Although the developer of 
flask-celery says otherwise, I am pretty sure that I did not have anything
to do like that when I still used flask-celery and everything just worked 
fine.

Otherwise my research told me, that there might arise issues with multiple
threads and as I am running on gunicorn with multiple listeners and celery
with multiple workers I am not sure whether that might be the problem. 
(Any explanation on this would be really appreciated!)

If anyone has any ideas what this issue could be about or how to fix it 
(or whether my decorator is just shit or whatever) they would be very 
appreciated, as just manually restarting the app when the error occurs is 
not really an option.

Thanks in advance and best regards,
Alex
-- 
Alexander Jung-Loddenkemper