librelist archives

« back to archive

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

From:
Lars Wiegman
Date:
2012-02-02 @ 17:15
Loic,

I just joined the list and couldn't reply to your message directly.

I'm looking into agile deployment for web apps and i'm interested to try 
out the PaaS setup you've been writing about.

I would be interested in any docs and/or code you can make available.

Cheers,

Lars.

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

From:
Loic d'Anterroches
Date:
2012-02-02 @ 17:37
Lars,

> I just joined the list and couldn't reply to your message directly.
> 
> I'm looking into agile deployment for web apps and i'm interested to try
out the PaaS setup you've been writing about.
> 
> I would be interested in any docs and/or code you can make available.

Basically you have 2 components:

1. a git pre-receive hook which is packaging with virtualenv a Python
application and creating configuration files for
2. diprocd, a robust "Procer" distributed over several nodes.

The "build pack" with the hook:
http://cdn.ceondo.com/bareku/buildpack-python.tar.gz
Diprocd:
http://projects.ceondo.com/p/diprocd/

If you read a bit about Heroku you will easily understand the approach
because it is basically following the same approach.

You can email me directly for more questions, I am also relatively
available on the #Mongrel2 channel if you need some help.

loïc

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

From:
Jonas Pfenniger
Date:
2012-02-02 @ 17:45
Hi Loic,

I was wondering as well what's the advantage of using Xen (Ganetti) when 
you could dedicate your machine to your processes and have more 
predictable results. Is it because you don't have many of them and want to
share the resources with other things ?

Heroku uses LXC to jail their processes. They don't have much choice given
they use EC2 but it also looks like a good solution as it's more 
lightweight than Xen.

Cheers,
Jonas


On Thursday, 2 February 2012 at 17:37, Loic d'Anterroches wrote:

> Lars,
>  
> > I just joined the list and couldn't reply to your message directly.
> >  
> > I'm looking into agile deployment for web apps and i'm interested to 
try out the PaaS setup you've been writing about.
> >  
> > I would be interested in any docs and/or code you can make available.
>  
> Basically you have 2 components:
>  
> 1. a git pre-receive hook which is packaging with virtualenv a Python
> application and creating configuration files for
> 2. diprocd, a robust "Procer" distributed over several nodes.
>  
> The "build pack" with the hook:
> http://cdn.ceondo.com/bareku/buildpack-python.tar.gz
> Diprocd:
> http://projects.ceondo.com/p/diprocd/
>  
> If you read a bit about Heroku you will easily understand the approach
> because it is basically following the same approach.
>  
> You can email me directly for more questions, I am also relatively
> available on the #Mongrel2 channel if you need some help.
>  
> loïc  

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

From:
Loic d'Anterroches
Date:
2012-02-02 @ 19:17
Hello,

for the record, I keep this discussion going here because I think it
could be of interest to others as part of a deployment approach for
Mongrel2. If some of you think it starts to be too off-topic, just tell me.

> I was wondering as well what's the advantage of using Xen (Ganetti)
> when you could dedicate your machine to your processes and have more
> predictable results. Is it because you don't have many of them and want
> to share the resources with other things ?

Effectively. Virtualisation in this case (using KVM) allows me to
segment by service. For example, I have 3 VMs for a MongoDB replicaset
and 2 VMs for a PostgreSQL setup with warm standby. The dedicated
servers I have are really powerful, so the overhead of KVM is very well
balanced by the flexibility provided by Ganeti.

> Heroku uses LXC to jail their processes. They don't have much choice
> given they use EC2 but it also looks like a good solution as it's more
> lightweight than Xen.

At the "PaaS" level, nothing is jailed. All the apps can talk to all the
other applications. This is needed for my scientific code to talk to the
web frontend easily. This way I have many applications chatting together
over ZMQ and each application stay small.

loïc

PS: Here is a ganeti output listing the instances on the nodes, as you
can see, the VMs are really "service" VMs. You can also see that I have
not yet made the full split of the frontend Mongrel2 and the apps. vm111
is an app node and vm125 both Mongrel2 and an app node:

Primary_node     Instance         Tags
node2.ceondo.net vm107.ceondo.net tinydns,master
node1.ceondo.net vm108.ceondo.net tinydns,slave
node2.ceondo.net vm109.ceondo.net mailserver
node1.ceondo.net vm110.ceondo.net git.bareku.com
node1.ceondo.net vm111.ceondo.net appserver
node2.ceondo.net vm112.ceondo.net photonweb
node2.ceondo.net vm114.ceondo.net backup,indefero
node1.ceondo.net vm116.ceondo.net srv-notes.ceondo.com,mongrel2
node2.ceondo.net vm117.ceondo.net graphite
node1.ceondo.net vm120.ceondo.net mongodb
node1.ceondo.net vm121.ceondo.net mongodb
node2.ceondo.net vm122.ceondo.net mongodb
node3.ceondo.net vm123.ceondo.net master,postgresql
node2.ceondo.net vm124.ceondo.net slave,postgresql
node1.ceondo.net vm125.ceondo.net appserver,frontend


> On Thursday, 2 February 2012 at 17:37, Loic d'Anterroches wrote:
> 
>> Lars,
>>  
>>> I just joined the list and couldn't reply to your message directly.
>>>  
>>> I'm looking into agile deployment for web apps and i'm interested to 
try out the PaaS setup you've been writing about.
>>>  
>>> I would be interested in any docs and/or code you can make available.
>>  
>> Basically you have 2 components:
>>  
>> 1. a git pre-receive hook which is packaging with virtualenv a Python
>> application and creating configuration files for
>> 2. diprocd, a robust "Procer" distributed over several nodes.
>>  
>> The "build pack" with the hook:
>> http://cdn.ceondo.com/bareku/buildpack-python.tar.gz
>> Diprocd:
>> http://projects.ceondo.com/p/diprocd/
>>  
>> If you read a bit about Heroku you will easily understand the approach
>> because it is basically following the same approach.
>>  
>> You can email me directly for more questions, I am also relatively
>> available on the #Mongrel2 channel if you need some help.
>>  
>> loïc  
> 
> 

-- 
Dr Loïc d'Anterroches
Founder Céondo Ltd

w: www.ceondo.com       |  e: loic@ceondo.com
t: +44 (0)207 183 0016  |  f: +44 (0)207 183 0124

Céondo Ltd
Dalton House
60 Windsor Avenue
London
SW19 2RR / United Kingdom

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

From:
Jonas Pfenniger
Date:
2012-02-02 @ 20:00
Thanks for the details Loic, I'm quite impressed what you managed to build
just alone.

On Thursday, 2 February 2012 at 19:17, Loic d'Anterroches wrote:

> > Heroku uses LXC to jail their processes. They don't have much choice
> > given they use EC2 but it also looks like a good solution as it's more
> > lightweight than Xen.
> 
> At the "PaaS" level, nothing is jailed. All the apps can talk to all the
> other applications. This is needed for my scientific code to talk to the
> web frontend easily. This way I have many applications chatting together
> over ZMQ and each application stay small.

Ok, because you use compressed filesystems I had the impression that the 
filesystem access was jailed. LXC is just a bunch of Linux kernel 
extension that are put together, you don't necessarily need to isolate the
network.

For example where I work we use the unshare[1] feature to jail the 
filesystem and network on a bunch of ffmpeg commands. The video inputs and
outputs are handled separately but available in the jail trough a 
mount-bind, which is only visible to the process and it's childs.

Or you could also use the cgroups[2] to make sure your processes don't 
abuse the system and take too much CPU or memory.

In general, since we're having a "1 app instance == 1 VM" model, I think 
it makes sense to create specialized container for each-one, even 
databases. It's a bit more challenging since you must make sure to put 
each DB instance on different machines and take care of persistence but 
it's bound to happen.

Cheers,
Jonas

[1] http://linux.die.net/man/2/unshare
[2] https://wiki.archlinux.org/index.php/Cgroups

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

From:
Loic d'Anterroches
Date:
2012-02-03 @ 05:03

On 2012-02-02 21:00, Jonas Pfenniger wrote:
> Thanks for the details Loic, I'm quite impressed what you managed to 
build just alone.

Do not forget that the bulk of the work is done by Mongrel2 and ZMQ. If
you really think, this is just a simple script updating a procer like
and doing scp of files. It took a bit of time (maybe 2 weeks full time),
but this is not that much compared to the benefits.

> On Thursday, 2 February 2012 at 19:17, Loic d'Anterroches wrote:
> 
>>> Heroku uses LXC to jail their processes. They don't have much choice
>>> given they use EC2 but it also looks like a good solution as it's more
>>> lightweight than Xen.
>>
>> At the "PaaS" level, nothing is jailed. All the apps can talk to all the
>> other applications. This is needed for my scientific code to talk to the
>> web frontend easily. This way I have many applications chatting together
>> over ZMQ and each application stay small.
> 
> Ok, because you use compressed filesystems I had the impression that the
filesystem access was jailed. LXC is just a bunch of Linux kernel 
extension that are put together, you don't necessarily need to isolate the
network.

I use the squashfs just to distribute the apps. In fact, I uncompress it
on the app node. I like the squashfs because it keeps symbolic links. At
the start I wanted to mount it, but then I had to get a unison fs to
have a tmp folder and it started to be complicated without a lot of
extra benefits for my personal setup.

> For example where I work we use the unshare[1] feature to jail the 
filesystem and network on a bunch of ffmpeg commands. The video inputs and
outputs are handled separately but available in the jail trough a 
mount-bind, which is only visible to the process and it's childs.
> 
> Or you could also use the cgroups[2] to make sure your processes don't 
abuse the system and take too much CPU or memory.

cgroups/LXC they are interesting. But again, my system is very simple
because it is "only for me", so I do not need to tell one app to behave
by controlling it from the OS, I code it directly to behave. This means
a lot of the work done by the current PaaS providers are not an issue
for me.

> In general, since we're having a "1 app instance == 1 VM" model, I think
it makes sense to create specialized container for each-one, even 
databases. It's a bit more challenging since you must make sure to put 
each DB instance on different machines and take care of persistence but 
it's bound to happen.

Yes, in fact this is what I am doing at the service level. I need to
ensure that my PostgreSQL instances are on different servers in
different datacenters etc. but for my small setup, I can still do that
manually without really thinking about it.

Thanks for the nice feedback, just learnt about the unshare system call
today :)

loïc

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

From:
David Kantowitz
Date:
2012-02-02 @ 20:07
Loic,

Have you looked at VMWare's CloudFoundtry stuff?  It's all open source and
seems to lack any real documentation.  From just poking around in their
projects, it seem like they might have some good bits you could make use of.

https://github.com/cloudfoundry

Regards,
David K.

On Thu, Feb 2, 2012 at 12:00 PM, Jonas Pfenniger <jonas@pfenniger.name>wrote:

> Thanks for the details Loic, I'm quite impressed what you managed to build
> just alone.
>
> On Thursday, 2 February 2012 at 19:17, Loic d'Anterroches wrote:
>
> > > Heroku uses LXC to jail their processes. They don't have much choice
> > > given they use EC2 but it also looks like a good solution as it's more
> > > lightweight than Xen.
> >
> > At the "PaaS" level, nothing is jailed. All the apps can talk to all the
> > other applications. This is needed for my scientific code to talk to the
> > web frontend easily. This way I have many applications chatting together
> > over ZMQ and each application stay small.
>
> Ok, because you use compressed filesystems I had the impression that the
> filesystem access was jailed. LXC is just a bunch of Linux kernel extension
> that are put together, you don't necessarily need to isolate the network.
>
> For example where I work we use the unshare[1] feature to jail the
> filesystem and network on a bunch of ffmpeg commands. The video inputs and
> outputs are handled separately but available in the jail trough a
> mount-bind, which is only visible to the process and it's childs.
>
> Or you could also use the cgroups[2] to make sure your processes don't
> abuse the system and take too much CPU or memory.
>
> In general, since we're having a "1 app instance == 1 VM" model, I think
> it makes sense to create specialized container for each-one, even
> databases. It's a bit more challenging since you must make sure to put each
> DB instance on different machines and take care of persistence but it's
> bound to happen.
>
> Cheers,
> Jonas
>
> [1] http://linux.die.net/man/2/unshare
> [2] https://wiki.archlinux.org/index.php/Cgroups
>
>

Re: [mongrel2] A private PaaS with Mongrel2 and ZeroMQ

From:
Loic d'Anterroches
Date:
2012-02-03 @ 05:07
Hi David,

> Have you looked at VMWare's CloudFoundtry stuff?  It's all open source
> and seems to lack any real documentation.  From just poking around in
> their projects, it seem like they might have some good bits you could
> make use of.

Yes, I took a short look. What is interesting me are the detailed
approach to "jail" a process in a VM without the need to go one VM per
app. It is not needed yet, but maybe in the future if I start to allow
untrusted code to run on the PaaS to let customers data analysis
directly. Heroku is using LXC, but documentation is very sparse and I
simply do not have the time to experiment with it at the moment.

loïc

> On Thu, Feb 2, 2012 at 12:00 PM, Jonas Pfenniger <jonas@pfenniger.name
> <mailto:jonas@pfenniger.name>> wrote:
> 
>     Thanks for the details Loic, I'm quite impressed what you managed to
>     build just alone.
> 
>     On Thursday, 2 February 2012 at 19:17, Loic d'Anterroches wrote:
> 
>     > > Heroku uses LXC to jail their processes. They don't have much choice
>     > > given they use EC2 but it also looks like a good solution as
>     it's more
>     > > lightweight than Xen.
>     >
>     > At the "PaaS" level, nothing is jailed. All the apps can talk to
>     all the
>     > other applications. This is needed for my scientific code to talk
>     to the
>     > web frontend easily. This way I have many applications chatting
>     together
>     > over ZMQ and each application stay small.
> 
>     Ok, because you use compressed filesystems I had the impression that
>     the filesystem access was jailed. LXC is just a bunch of Linux
>     kernel extension that are put together, you don't necessarily need
>     to isolate the network.
> 
>     For example where I work we use the unshare[1] feature to jail the
>     filesystem and network on a bunch of ffmpeg commands. The video
>     inputs and outputs are handled separately but available in the jail
>     trough a mount-bind, which is only visible to the process and it's
>     childs.
> 
>     Or you could also use the cgroups[2] to make sure your processes
>     don't abuse the system and take too much CPU or memory.
> 
>     In general, since we're having a "1 app instance == 1 VM" model, I
>     think it makes sense to create specialized container for each-one,
>     even databases. It's a bit more challenging since you must make sure
>     to put each DB instance on different machines and take care of
>     persistence but it's bound to happen.
> 
>     Cheers,
>     Jonas
> 
>     [1] http://linux.die.net/man/2/unshare
>     [2] https://wiki.archlinux.org/index.php/Cgroups
> 
> 

-- 
Dr Loïc d'Anterroches
Founder Céondo Ltd

w: www.ceondo.com       |  e: loic@ceondo.com
t: +44 (0)207 183 0016  |  f: +44 (0)207 183 0124

Céondo Ltd
Dalton House
60 Windsor Avenue
London
SW19 2RR / United Kingdom