Is there any way to run Tir in the background? Assuming so, is there a recommended way to start and stop it? ✈ Matt
On Fri, Jan 28, 2011 at 04:52:09PM -0800, Matt Towers wrote: > Is there any way to run Tir in the background? Assuming so, is there > a recommended way to start and stop it? Nope, nothing written yet. I was going to add code to the tir runner that would basically make it act like procer. *I* run it by using screen, which works surprisingly well when you only have 10 users a day. :-) -- Zed A. Shaw http://zedshaw.com/
The cool thing about Lua is how piss easy it is to create bindings to c code. It's not the best I know, but writing a bcrypt binding yourself should be easy enough. As an aside, in the little framework I'm playing around writing, I have a site.lua file that defines deploy specific stuff. It has the ability to overwrite anything in config.lua and both also have the ability to create a mongrel2 sqlite config file. On Jan 30, 2011 8:36 AM, "Matt N" <mtnngw@gmail.com> wrote:
What's your gut feel about trying to use Tir in a production environment? ✈ Matt On Jan 28, 2011, at 21:43 , Zed A. Shaw wrote: > On Fri, Jan 28, 2011 at 04:52:09PM -0800, Matt Towers wrote: >> Is there any way to run Tir in the background? Assuming so, is there >> a recommended way to start and stop it? > > Nope, nothing written yet. I was going to add code to the tir runner > that would basically make it act like procer. > > *I* run it by using screen, which works surprisingly well when you only > have 10 users a day. :-) > > -- > Zed A. Shaw > http://zedshaw.com/
On Sat, Jan 29, 2011 at 01:13:01PM -0800, Matt Towers wrote: > What's your gut feel about trying to use Tir in a production environment? The great thing about it is that it's so small, and I spent so much time making it readable, that if something's missing it's damn easy to add. Other than that, if I were given a job where I was required to immediately handle millions of users I wouldn't use it. Not going to force tech on someone when it's not ready. But, for most other projects I do now I use Tir and then 0mq to other things I need in other languages. It's about as production ready as most of the other stuff I use, and as I said above, small enough that you don't tear your hair out fixing stuff or adding to it. -- Zed A. Shaw http://zedshaw.com/
On 2011-01-30 02:41, Zed A. Shaw wrote: > On Sat, Jan 29, 2011 at 01:13:01PM -0800, Matt Towers wrote: >> What's your gut feel about trying to use Tir in a production environment? > > The great thing about it is that it's so small, and I spent so much time > making it readable, that if something's missing it's damn easy to add. > > Other than that, if I were given a job where I was required to > immediately handle millions of users I wouldn't use it. Not going to > force tech on someone when it's not ready. > > But, for most other projects I do now I use Tir and then 0mq to other > things I need in other languages. It's about as production ready as > most of the other stuff I use, and as I said above, small enough that > you don't tear your hair out fixing stuff or adding to it. You could add: The beauty of using zmq for the tasks also mean that you can plug Tir into your production ready code (Java/Python/PHP/C/...) whitout any problems and easily. So the layer of "experimental" work is at the end very small and can easily be audited. loïc
My experiences: - Need to override the error handler, so you don't dump a stack trace to your users ( I haven't investigated how to do this yet...) - screen works okay, even with 1000's of daily visitors. Not a long term solution though... - I wish I had a better way to configure a PROD vs DEV environment. config.lua might be the right place for this, but i've spent the last two weeks learning Lua. Very little time learning tir. - I'm not happy with the password hashing in lua. Can't seem to find a decent portable bcrypt implementation. This isn't Lua's fault though. Securing passwords is just something that most everyone has to write, so makes a good candidate for inclusion in the framework. Autho.me isn't ready yet so... Overall very happy with Tir, even though I don't know many of the best practices yet... On Jan 29, 2011, at 1:13 PM, Matt Towers wrote: > What's your gut feel about trying to use Tir in a production environment? > > ✈ Matt > > > > On Jan 28, 2011, at 21:43 , Zed A. Shaw wrote: > >> On Fri, Jan 28, 2011 at 04:52:09PM -0800, Matt Towers wrote: >>> Is there any way to run Tir in the background? Assuming so, is there >>> a recommended way to start and stop it? >> >> Nope, nothing written yet. I was going to add code to the tir runner >> that would basically make it act like procer. >> >> *I* run it by using screen, which works surprisingly well when you only >> have 10 users a day. :-) >> >> -- >> Zed A. Shaw >> http://zedshaw.com/ >
Matt, How do you handle log file management? Without having had much experience with managing logs, I'm thinking that we way I want to do it here is with syslog-ng and logrotate. What's your approach? ✈ Matt On Jan 29, 2011, at 13:36 , Matt N wrote: > My experiences: > > - Need to override the error handler, so you don't dump a stack trace to your users ( I haven't investigated how to do this yet...) > - screen works okay, even with 1000's of daily visitors. Not a long term solution though... > - I wish I had a better way to configure a PROD vs DEV environment. config.lua might be the right place for this, but i've spent the last two weeks learning Lua. Very little time learning tir. > - I'm not happy with the password hashing in lua. Can't seem to find a decent portable bcrypt implementation. This isn't Lua's fault though. Securing passwords is just something that most everyone has to write, so makes a good candidate for inclusion in the framework. Autho.me isn't ready yet so... > > Overall very happy with Tir, even though I don't know many of the best practices yet... > > > > On Jan 29, 2011, at 1:13 PM, Matt Towers wrote: > >> What's your gut feel about trying to use Tir in a production environment? >> >> ✈ Matt >> >> >> >> On Jan 28, 2011, at 21:43 , Zed A. Shaw wrote: >> >>> On Fri, Jan 28, 2011 at 04:52:09PM -0800, Matt Towers wrote: >>>> Is there any way to run Tir in the background? Assuming so, is there >>>> a recommended way to start and stop it? >>> >>> Nope, nothing written yet. I was going to add code to the tir runner >>> that would basically make it act like procer. >>> >>> *I* run it by using screen, which works surprisingly well when you only >>> have 10 users a day. :-) >>> >>> -- >>> Zed A. Shaw >>> http://zedshaw.com/ >> >
Hi Matt, The (slightly embarrassing) answer that I don't handle logs yet. Properly handling logs is on the TODO list. My normal recipe goes something like this: I start tir with screen -d -m, which basically starts tir in a screen in detached mode. I can then reattach at any point if I need to ctrl-C or ctrl-\. If you want to keep tir running in screen, but also capture the output to a logfile, tee is the best way to do both at the same time: http://en.wikipedia.org/wiki/Tee_(command) Once you have that logfile, you can use logrotate or whatever. Certainly not ideal, but "it works for now" kind of solution. Matt On Feb 2, 2011, at 4:56 PM, Matt Towers wrote: > Matt, > > How do you handle log file management? Without having had much experience with managing logs, I'm thinking that we way I want to do it here is with syslog-ng and logrotate. > > What's your approach? > > > ✈ Matt > > > > On Jan 29, 2011, at 13:36 , Matt N wrote: > >> My experiences: >> >> - Need to override the error handler, so you don't dump a stack trace to your users ( I haven't investigated how to do this yet...) >> - screen works okay, even with 1000's of daily visitors. Not a long term solution though... >> - I wish I had a better way to configure a PROD vs DEV environment. config.lua might be the right place for this, but i've spent the last two weeks learning Lua. Very little time learning tir. >> - I'm not happy with the password hashing in lua. Can't seem to find a decent portable bcrypt implementation. This isn't Lua's fault though. Securing passwords is just something that most everyone has to write, so makes a good candidate for inclusion in the framework. Autho.me isn't ready yet so... >> >> Overall very happy with Tir, even though I don't know many of the best practices yet... >> >> >> >> On Jan 29, 2011, at 1:13 PM, Matt Towers wrote: >> >>> What's your gut feel about trying to use Tir in a production environment? >>> >>> ✈ Matt >>> >>> >>> >>> On Jan 28, 2011, at 21:43 , Zed A. Shaw wrote: >>> >>>> On Fri, Jan 28, 2011 at 04:52:09PM -0800, Matt Towers wrote: >>>>> Is there any way to run Tir in the background? Assuming so, is there >>>>> a recommended way to start and stop it? >>>> >>>> Nope, nothing written yet. I was going to add code to the tir runner >>>> that would basically make it act like procer. >>>> >>>> *I* run it by using screen, which works surprisingly well when you only >>>> have 10 users a day. :-) >>>> >>>> -- >>>> Zed A. Shaw >>>> http://zedshaw.com/ >>> >> >
On Thu, Feb 03, 2011 at 12:13:32PM -0800, Matt N wrote: > Hi Matt, > > The (slightly embarrassing) answer that I don't handle logs yet. Properly handling logs is on the TODO list. Same with me, but my stuff is small. I should get on the deployment stuff soon. -- Zed A. Shaw http://zedshaw.com/
Would it make sense to use LuaLogging as the substrate? http://www.keplerproject.org/lualogging The package doesn't look like it's been touched since late 2007 but is currently available through the standard luarocks and Ubuntu repos. The biggest limitation I see to it so far is that you cannot use multiple log targets concurrently. ✈ Matt On Feb 4, 2011, at 22:27 , Zed A. Shaw wrote: > On Thu, Feb 03, 2011 at 12:13:32PM -0800, Matt N wrote: >> Hi Matt, >> >> The (slightly embarrassing) answer that I don't handle logs yet. Properly handling logs is on the TODO list. > > Same with me, but my stuff is small. I should get on the deployment > stuff soon. > > -- > Zed A. Shaw > http://zedshaw.com/
Sorry if this has been answered someplace else, but ... Would it make sense to write logging messages to a PUB 0mq socket, and then have separate SUBscribers to write to a local file or send it to a centralized logging facility. The different subscribers could select different levels of logging (e.g. INFO+WARN+ERROR+FATAL for local logging and ERROR+FATAL for centralized logging, FATAL for paging a sysadmin). Does this make sense? Regards, Henry On February 7, 2011 12:45:42 pm Matt Towers wrote: > Would it make sense to use LuaLogging as the substrate? > http://www.keplerproject.org/lualogging > > The package doesn't look like it's been touched since late 2007 but is > currently available through the standard luarocks and Ubuntu repos. > > The biggest limitation I see to it so far is that you cannot use multiple > log targets concurrently. > > ✈ Matt > > On Feb 4, 2011, at 22:27 , Zed A. Shaw wrote: > > On Thu, Feb 03, 2011 at 12:13:32PM -0800, Matt N wrote: > >> Hi Matt, > >> > >> The (slightly embarrassing) answer that I don't handle logs yet. > >> Properly handling logs is on the TODO list. > > > > Same with me, but my stuff is small. I should get on the deployment > > stuff soon. -- Henry Baragar Instantiated Software
That's something we've looked at, in fact the mongrel2 access log does just that. I don't think zed has looked at logging in tir at all yet though. On Feb 8, 2011 7:16 AM, "Henry Baragar" <Henry.Baragar@instantiated.ca> wrote: > > Sorry if this has been answered someplace else, but ... > > Would it make sense to write logging messages to a PUB 0mq socket, and then have separate SUBscribers to write to a local file or send it to a centralized logging facility. The different subscribers could select different levels of logging (e.g. INFO+WARN+ERROR+FATAL for local logging and ERROR+FATAL for centralized logging, FATAL for paging a sysadmin). > > Does this make sense? > > Regards, > > Henry > > On February 7, 2011 12:45:42 pm Matt Towers wrote: > > > Would it make sense to use LuaLogging as the substrate? > > > http://www.keplerproject.org/lualogging > > > > > > The package doesn't look like it's been touched since late 2007 but is > > > currently available through the standard luarocks and Ubuntu repos. > > > > > > The biggest limitation I see to it so far is that you cannot use multiple > > > log targets concurrently. > > > > > > ✈ Matt > > > > > > On Feb 4, 2011, at 22:27 , Zed A. Shaw wrote: > > > > On Thu, Feb 03, 2011 at 12:13:32PM -0800, Matt N wrote: > > > >> Hi Matt, > > > >> > > > >> The (slightly embarrassing) answer that I don't handle logs yet. > > > >> Properly handling logs is on the TODO list. > > > > > > > > Same with me, but my stuff is small. I should get on the deployment > > > > stuff soon. > > > -- > > Henry Baragar > > Instantiated Software
On Tue, Feb 08, 2011 at 07:19:22AM +1100, joshua simmons wrote: > That's something we've looked at, in fact the mongrel2 access log does just > that. > > I don't think zed has looked at logging in tir at all yet though. Nope, not one bit. Logging for me is to the screen. :-) But, I'd say keep it simple. I'd go with just writing to a file with Tir at first and then later if someone needs 0mq style logging, if the system is kept simple enough, they can add it. -- Zed A. Shaw http://zedshaw.com/
On Sat, Jan 29, 2011 at 01:36:29PM -0800, Matt N wrote: > My experiences: Oh, you're the other guy using it. :-) > - Need to override the error handler, so you don't dump a stack trace to your users ( I haven't investigated how to do this yet...) Hey, can you toss a bug into the tracker for me on this? It's easy to do I just having written a hook for it yet: http://tir.mongrel2.org/tktnew > - screen works okay, even with 1000's of daily visitors. Not a long term solution though... Yeah, it's working fine for me, but really not optimal. I'm basically pointing people at procer/daemontools/supervisord but will eventually write something small for just Tir > - I wish I had a better way to configure a PROD vs DEV environment. config.lua might be the right place for this, but i've spent the last two weeks learning Lua. Very little time learning tir. Can you lay out what you might want on this? There's not much to configure in Tir itself, so I haven't ran into needing a difference except for unit tests. There's some gear for it, but not the best for doing automated deploys and such. > - I'm not happy with the password hashing in lua. Can't seem to find a decent portable bcrypt implementation. This isn't Lua's fault though. Securing passwords is just something that most everyone has to write, so makes a good candidate for inclusion in the framework. Autho.me isn't ready yet so... TADA: https://github.com/silentbicycle/lua-bcrypt I've used that and it works fine. -- Zed A. Shaw http://zedshaw.com/
Error handler ticket submitted. On Jan 29, 2011, at 5:39 PM, Zed A. Shaw wrote: > >> - screen works okay, even with 1000's of daily visitors. Not a long term solution though... > > Yeah, it's working fine for me, but really not optimal. I'm basically > pointing people at procer/daemontools/supervisord but will eventually > write something small for just Tir I love procer BTW... It could be it's own top level project. > >> - I wish I had a better way to configure a PROD vs DEV environment. config.lua might be the right place for this, but i've spent the last two weeks learning Lua. Very little time learning tir. > > Can you lay out what you might want on this? There's not much to > configure in Tir itself, so I haven't ran into needing a difference > except for unit tests. There's some gear for it, but not the best for > doing automated deploys and such. > This got long... turned into a separate email... >> - I'm not happy with the password hashing in lua. Can't seem to find a decent portable bcrypt implementation. This isn't Lua's fault though. Securing passwords is just something that most everyone has to write, so makes a good candidate for inclusion in the framework. Autho.me isn't ready yet so... > > TADA: https://github.com/silentbicycle/lua-bcrypt > > I've used that and it works fine. > You are a magician!
On 01/29/2011 07:43 AM, Zed A. Shaw wrote: > On Fri, Jan 28, 2011 at 04:52:09PM -0800, Matt Towers wrote: >> Is there any way to run Tir in the background? Assuming so, is there >> a recommended way to start and stop it? > Nope, nothing written yet. I was going to add code to the tir runner > that would basically make it act like procer. > > *I* run it by using screen, which works surprisingly well when you only > have 10 users a day. :-) supervisord.org: forks the managed processes as its children, captures stdin/stdout and puts them in files, can rotate logs for you, no need for pid files :) It also has a web interface and speaks XMLRPC over TCP or unix sockets if you're so inclined (I've been planning for a while to get some centralised dashboard thing going for it, beyond 2-3 servers things get hairy to manage, and the folks at mmonit charge an arm and a leg for their tool, and I kinda like how teh flask examples are looking...) The only gripe I had with it is that the default wait time for shutting down tasks is 10 seconds, which is problematic for mongodb (takes much longer to flush stuff to disk if the database is huge), celery and my own stuff that uses zeromq [1] (but that's likely my own failing); easily fixed with stopwaitsecs=<reasonable value> [1] https://gist.github.com/661377 (don't get me wrong, I'm a bzr guy :P)
On 01/29/2011 12:32 PM, Sabin Iacob wrote: > On 01/29/2011 07:43 AM, Zed A. Shaw wrote: >> On Fri, Jan 28, 2011 at 04:52:09PM -0800, Matt Towers wrote: >>> Is there any way to run Tir in the background? Assuming so, is there >>> a recommended way to start and stop it? >> Nope, nothing written yet. I was going to add code to the tir runner >> that would basically make it act like procer. >> >> *I* run it by using screen, which works surprisingly well when you only >> have 10 users a day. :-) > > supervisord.org: forks the managed processes as its children, captures > stdin/stdout and puts them in files, can rotate logs for you, no need > for pid files :) erm, I meant stderr/stdout