Re: [photon.users] Introducing the Majordomo pattern to handle task messaging
- Loic d'Anterroches
- 2011-10-27 @ 07:10
oh, a very nice thing about this pattern: both the workers and the
clients are connecting to the same ZeroMQ port. That is, you need only
one "port" to manage all the work. This is really good, this means we
will be able to have a relatively "sane" default port on localhost for
the default configuration and nothing else to have a system working out
of the box.
I will also some small Python examples for you to create workers in Python.
On 2011-10-25 14:56, Loic d'Anterroches wrote:
> as I need to do something like:
> 1. request comes in.
> 2. contact service 1 for computation
> 3. contact service 2 for computation independently of service 1
> 4. aggregate the results
> 5. send an answer
> I need to do a bit more than the current task runner. The current task
> runner is nice but you need to push the requests and then read them
> afterwards without a general timeout. That is, I want to say that the
> total of steps 2 and 3 is "50 ms max" and discard the computation 1 or 2
> or both if they cannot make the cut.
> To do that, I will implement an extended Majordomo service:
> It will be based on the one shown above but will add the support of
> multiple workers per service (only one in the given implementation) and
> timeout at the client side to discard results above the cut.
> This means that on this image:
> A client is a Photon handler, but you can have many "Water" or many
> "Tea" workers. That is, you can scale each worker (a Photon task or task
> built in any language) independently while keeping on the client side a
> single point of connection (a client does not need to connect to all the
> workers). Also, the broker can be smart in the scheduling of the
> requests to the workers.
> I will ensure that the tasks are still easy to write but this means that
> you will need a broker process. But the code is 200 lines of PHP and
> nothing more, so pretty simple. It will just be a run of:
> $ hnu broker
> Have a nice day,
> Indefero - Project management and code hosting - http://www.indefero.net
> Photon - High Performance PHP Framework - http://photon-project.com
> Céondo Ltd - Web + Science = Fun - http://www.ceondo.com