librelist archives

« back to archive

SSL in Mongrel2

SSL in Mongrel2

From:
Loic d'Anterroches
Date:
2011-05-10 @ 12:53
Hello,

this question is especially for Zed, basically, do you think that the 
support of SSL directly within Mongrel2 could be acceptable? A few 
months ago, you wrote on the channel that you would prefer to have 
people using stunnel. Is it still the case? If so, just answer "still 
the case and close my request".

If SSL within Mongrel2 could be fine with you, I am ready to sponsor the 
addition to the core. I most likely do not have enough money to pay for 
the full implementation, but I can help bootstrap the work and maybe 
some other people could help with money, time, testing, coding etc.

The idea behind implementing SSL directly into Mongrel2 is to ease 
deployment and get good performance while keeping track of the originate 
IP. You can run stunnel in transparent mode, but if you want to know at 
the app level both if the traffic is SSL or not and the client IP, you 
need to either force a close on the connection to not use keepalive and 
inject the forwarded-for header or setup 2 servers on Mongrel2, one for 
the ssl traffic and one for the plain traffic. With many sites, you end 
up having a lot to think about, especially you need to open local ports 
etc. I always prefer to have the minimum of moving parts (even so I like 
the one tool does one thing and well philosophy of Unix, the right 
balance is always hard to find).

Practically, I can put $1.5k and up to $3k. Zed knows that I promise and 
pay. Money in free software is always a balance to find. I know from 
experience that offering money when interest to get the stuff done is 
not already there is just a waste of time and money. So if the boss says 
ok and one of you is interested of having the support of SSL, count on 
me to help you buy some cookies, if not, I will find a way with 
stunnel/haproxy & cie.

loïc

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-10 @ 18:16
I did a lot of the original SSL stuff in mongrel2 before I got busy with
school and would be happy to finish what I started (as it turns out, I'm
graduating Sunday and don't start my new job until July, so this is pretty
much the perfect time for me anyway).

What exactly are you looking for?

And how much of all of this are we planning on keeping?  Specifically, do we
stick with axTLS and try to make stuff work better (updating the schema
instead of doing global config, etc.), or do we switch over to openSSH?
 axTLS had been a super big pain in the ass to get working with various
browsers (iirc either chrome or firefox never worked right at all).

Alex

On Tue, May 10, 2011 at 8:53 AM, Loic d'Anterroches <loic@ceondo.com> wrote:

> Hello,
>
> this question is especially for Zed, basically, do you think that the
> support of SSL directly within Mongrel2 could be acceptable? A few
> months ago, you wrote on the channel that you would prefer to have
> people using stunnel. Is it still the case? If so, just answer "still
> the case and close my request".
>
> If SSL within Mongrel2 could be fine with you, I am ready to sponsor the
> addition to the core. I most likely do not have enough money to pay for
> the full implementation, but I can help bootstrap the work and maybe
> some other people could help with money, time, testing, coding etc.
>
> The idea behind implementing SSL directly into Mongrel2 is to ease
> deployment and get good performance while keeping track of the originate
> IP. You can run stunnel in transparent mode, but if you want to know at
> the app level both if the traffic is SSL or not and the client IP, you
> need to either force a close on the connection to not use keepalive and
> inject the forwarded-for header or setup 2 servers on Mongrel2, one for
> the ssl traffic and one for the plain traffic. With many sites, you end
> up having a lot to think about, especially you need to open local ports
> etc. I always prefer to have the minimum of moving parts (even so I like
> the one tool does one thing and well philosophy of Unix, the right
> balance is always hard to find).
>
> Practically, I can put $1.5k and up to $3k. Zed knows that I promise and
> pay. Money in free software is always a balance to find. I know from
> experience that offering money when interest to get the stuff done is
> not already there is just a waste of time and money. So if the boss says
> ok and one of you is interested of having the support of SSL, count on
> me to help you buy some cookies, if not, I will find a way with
> stunnel/haproxy & cie.
>
> loïc
>

Re: [mongrel2] SSL in Mongrel2

From:
Loic d'Anterroches
Date:
2011-05-10 @ 20:13

On 05/10/2011 08:16 PM, Alex Gartrell wrote:
> I did a lot of the original SSL stuff in mongrel2 before I got busy with
> school and would be happy to finish what I started (as it turns out, I'm
> graduating Sunday and don't start my new job until July, so this is
> pretty much the perfect time for me anyway).
>
> What exactly are you looking for?
>
> And how much of all of this are we planning on keeping?  Specifically,
> do we stick with axTLS and try to make stuff work better (updating the
> schema instead of doing global config, etc.), or do we switch over to
> openSSH?  axTLS had been a super big pain in the ass to get working with
> various browsers (iirc either chrome or firefox never worked right at all).

Basically, whatever works is good for me. I need to support SSLv3 and 
TLS. If possible, I would like to be able to configure everything from 
the configuration files.

OpenSSL would be very good as it is the standard at the moment, but this 
is not required, I just need the stuff to work like nginx.

One point, it should be possible from the config file to pass a list of 
supported ciphers (the PCIDSS compliance is a pain because they accept a 
very limited list of ciphers not understanding that the list has an order).

So, if at the end I can get the configuration of SSL for Mongrel2 nearly 
as simple as the configuration of SSL with nginx, I wire the money.

loïc

> On Tue, May 10, 2011 at 8:53 AM, Loic d'Anterroches <loic@ceondo.com
> <mailto:loic@ceondo.com>> wrote:
>
>     Hello,
>
>     this question is especially for Zed, basically, do you think that the
>     support of SSL directly within Mongrel2 could be acceptable? A few
>     months ago, you wrote on the channel that you would prefer to have
>     people using stunnel. Is it still the case? If so, just answer "still
>     the case and close my request".
>
>     If SSL within Mongrel2 could be fine with you, I am ready to sponsor the
>     addition to the core. I most likely do not have enough money to pay for
>     the full implementation, but I can help bootstrap the work and maybe
>     some other people could help with money, time, testing, coding etc.
>
>     The idea behind implementing SSL directly into Mongrel2 is to ease
>     deployment and get good performance while keeping track of the originate
>     IP. You can run stunnel in transparent mode, but if you want to know at
>     the app level both if the traffic is SSL or not and the client IP, you
>     need to either force a close on the connection to not use keepalive and
>     inject the forwarded-for header or setup 2 servers on Mongrel2, one for
>     the ssl traffic and one for the plain traffic. With many sites, you end
>     up having a lot to think about, especially you need to open local ports
>     etc. I always prefer to have the minimum of moving parts (even so I like
>     the one tool does one thing and well philosophy of Unix, the right
>     balance is always hard to find).
>
>     Practically, I can put $1.5k and up to $3k. Zed knows that I promise and
>     pay. Money in free software is always a balance to find. I know from
>     experience that offering money when interest to get the stuff done is
>     not already there is just a waste of time and money. So if the boss says
>     ok and one of you is interested of having the support of SSL, count on
>     me to help you buy some cookies, if not, I will find a way with
>     stunnel/haproxy & cie.
>
>     loïc
>
>

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-10 @ 19:27
On Tue, May 10, 2011 at 02:16:37PM -0400, Alex Gartrell wrote:
> And how much of all of this are we planning on keeping?  Specifically, do we
> stick with axTLS and try to make stuff work better (updating the schema
> instead of doing global config, etc.), or do we switch over to openSSH?
>  axTLS had been a super big pain in the ass to get working with various
> browsers (iirc either chrome or firefox never worked right at all).

You mean OpenSSL?  I'd be down for ditching axTLS for that reason.  It
sucks that the main browser manufacturers have so much sway on the IETF
that they can churn out new versions crypto like a spaghetti factory.

If axTLS don't work, then OpenSSL it is, which I hope you can find
enough information on using correctly.  Last time I tried it was a
complete black box because the OpenSSL devs seem to not like to document
a damn thing they do.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-11 @ 17:20
I'm still playing around with it but so far it looks like axTLS is working
fine on 32 bit linux (in a VM) while not working on 64 bit OS X.  I'm hoping
that it's a OS X vs. Linux issue instead of a 32 bit vs 64 bit issue.

So this might suck less than anticipated :)

On Tue, May 10, 2011 at 3:27 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

> On Tue, May 10, 2011 at 02:16:37PM -0400, Alex Gartrell wrote:
> > And how much of all of this are we planning on keeping?  Specifically, do
> we
> > stick with axTLS and try to make stuff work better (updating the schema
> > instead of doing global config, etc.), or do we switch over to openSSH?
> >  axTLS had been a super big pain in the ass to get working with various
> > browsers (iirc either chrome or firefox never worked right at all).
>
> You mean OpenSSL?  I'd be down for ditching axTLS for that reason.  It
> sucks that the main browser manufacturers have so much sway on the IETF
> that they can churn out new versions crypto like a spaghetti factory.
>
> If axTLS don't work, then OpenSSL it is, which I hope you can find
> enough information on using correctly.  Last time I tried it was a
> complete black box because the OpenSSL devs seem to not like to document
> a damn thing they do.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-11 @ 18:07
On Wed, May 11, 2011 at 01:20:33PM -0400, Alex Gartrell wrote:
> I'm still playing around with it but so far it looks like axTLS is working
> fine on 32 bit linux (in a VM) while not working on 64 bit OS X.  I'm hoping
> that it's a OS X vs. Linux issue instead of a 32 bit vs 64 bit issue.

Alright, and what about with browsers?  I'd rather just ditch axTLS
(sorry for all that work you did) and use GnuTLS if it works better.

Let me know, because I'd like to rip the SSL and WebSockets stuff out of
the 1.6 release so that it's a clean slate for adding SSL back in for
the 1.7 release.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-11 @ 18:17
Serving from Linux 32/64 bit works fine with Firefox (OS X, Arch), Safari
(OS X), Chrome (OS X), and Chromium (Arch)

The problem's almost definitely something flaky in the bsd specific portion.

If we do end up ripping out axTLS it won't be that big of a deal, as there
isn't much axTLS specific code and I imagine other interfaces would be
pretty similar.

On Wed, May 11, 2011 at 2:07 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

> On Wed, May 11, 2011 at 01:20:33PM -0400, Alex Gartrell wrote:
> > I'm still playing around with it but so far it looks like axTLS is
> working
> > fine on 32 bit linux (in a VM) while not working on 64 bit OS X.  I'm
> hoping
> > that it's a OS X vs. Linux issue instead of a 32 bit vs 64 bit issue.
>
> Alright, and what about with browsers?  I'd rather just ditch axTLS
> (sorry for all that work you did) and use GnuTLS if it works better.
>
> Let me know, because I'd like to rip the SSL and WebSockets stuff out of
> the 1.6 release so that it's a clean slate for adding SSL back in for
> the 1.7 release.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-12 @ 00:51
I was wrong, turns out there was some version skew going on (the new chrome
just recently started supporting TLS 1.0, or something).

Basically, when the client sent its hello, it was saying (oh yeah, I'm
version 3.0) and axTLS was saying "No thanks, 3.1 and up, please."  This
makes axTLS not super viable, as SSL 3 is still very much in use.

I hope I'm wrong here, but if I'm not, we'll need to evaluate a new library.
 Specifically, all of these goddamn things like to seem to assume that they
can send/recv directly (axTLS at least let us redefine
SOCKET_READ/SOCKET_WRITE to be fdread and frwrite).

What's the lesson here for library developers?  Don't be an asshole; provide
lower level interfaces...

Alex

On Wed, May 11, 2011 at 2:17 PM, Alex Gartrell <alexgartrell@gmail.com>wrote:

> Serving from Linux 32/64 bit works fine with Firefox (OS X, Arch), Safari
> (OS X), Chrome (OS X), and Chromium (Arch)
>
> The problem's almost definitely something flaky in the bsd specific
> portion.
>
> If we do end up ripping out axTLS it won't be that big of a deal, as there
> isn't much axTLS specific code and I imagine other interfaces would be
> pretty similar.
>
> On Wed, May 11, 2011 at 2:07 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
>
>> On Wed, May 11, 2011 at 01:20:33PM -0400, Alex Gartrell wrote:
>> > I'm still playing around with it but so far it looks like axTLS is
>> working
>> > fine on 32 bit linux (in a VM) while not working on 64 bit OS X.  I'm
>> hoping
>> > that it's a OS X vs. Linux issue instead of a 32 bit vs 64 bit issue.
>>
>> Alright, and what about with browsers?  I'd rather just ditch axTLS
>> (sorry for all that work you did) and use GnuTLS if it works better.
>>
>> Let me know, because I'd like to rip the SSL and WebSockets stuff out of
>> the 1.6 release so that it's a clean slate for adding SSL back in for
>> the 1.7 release.
>>
>> --
>> Zed A. Shaw
>> http://zedshaw.com/
>>
>
>

Re: [mongrel2] SSL in Mongrel2

From:
Xavier Lange
Date:
2011-05-11 @ 18:12
What WebSocket stuff will you be removing? The little bit about
overriding the body size to 8?

On Wed, May 11, 2011 at 11:07 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Wed, May 11, 2011 at 01:20:33PM -0400, Alex Gartrell wrote:
>> I'm still playing around with it but so far it looks like axTLS is working
>> fine on 32 bit linux (in a VM) while not working on 64 bit OS X.  I'm hoping
>> that it's a OS X vs. Linux issue instead of a 32 bit vs 64 bit issue.
>
> Alright, and what about with browsers?  I'd rather just ditch axTLS
> (sorry for all that work you did) and use GnuTLS if it works better.
>
> Let me know, because I'd like to rip the SSL and WebSockets stuff out of
> the 1.6 release so that it's a clean slate for adding SSL back in for
> the 1.7 release.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-11 @ 19:19
On Wed, May 11, 2011 at 11:12:55AM -0700, Xavier Lange wrote:
> What WebSocket stuff will you be removing? The little bit about
> overriding the body size to 8?

It needs to be upgraded to the latest (non-vulnerable) websockets spec
or it needs to go.  Ripping it out means that we don't have to test it
for this release, and then we can write a better one for 1.7.

I'd say, if you're using it (and I think you're the only one), then
stick with 1.5 or fossil just before 1.6, and then it'll be back in 1.7.

Also, the clients you're using shouldn't be working anyway.  I think all
the major browsers are now using the latest websockets which gets rid of
the stupidity I hated.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Loic d'Anterroches
Date:
2011-05-17 @ 09:19
Hello,

sorry for the delay, I am not at home at the moment and this makes 
things a bit of a mess.

So, thank you for 1.6 and the roadmap and here is a small summary of I 
am ready to pay for in terms of SSL:

- Selection of the ciphers (this is needed for PCI-DSS compliance).
- Configuration through the config file (I can do it with sql, but I am 
lazy).
- Working with 95% of the browsers (basically I suppose it means SSL + TLS).

What is important is that I totally trust you for the technical 
implementation, if Zed says: "good for release" then this is good.

The deadline to claim the bounty is end of September 2011. (I have a 
project I want to have in production at that time and need ssl for it).

The winner(s) will be selected between the coders. If 2 of you are 
coding on it, I can do a 2x1k (collaboration bonus, because I also like 
when several people are familiar with such critical piece of code) and 
1.5k if only one coder.

Sorry for my English, back in France for 2 weeks, I cannot speak French 
correctly anymore but I am not really better in English/German... 
expatriate life...
loïc


On 05/11/2011 09:19 PM, Zed A. Shaw wrote:
> On Wed, May 11, 2011 at 11:12:55AM -0700, Xavier Lange wrote:
>> What WebSocket stuff will you be removing? The little bit about
>> overriding the body size to 8?
>
> It needs to be upgraded to the latest (non-vulnerable) websockets spec
> or it needs to go.  Ripping it out means that we don't have to test it
> for this release, and then we can write a better one for 1.7.
>
> I'd say, if you're using it (and I think you're the only one), then
> stick with 1.5 or fossil just before 1.6, and then it'll be back in 1.7.
>
> Also, the clients you're using shouldn't be working anyway.  I think all
> the major browsers are now using the latest websockets which gets rid of
> the stupidity I hated.
>

-- 
Dr Loïc d'Anterroches
Founder Céondo Ltd

w: www.ceondo.com       |  e: loic@ceondo.com
t: +44 (0)207 183 0016  |  f: +44 (0)207 183 0124

Céondo Ltd
Dalton House
60 Windsor Avenue
London
SW19 2RR / United Kingdom

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-18 @ 19:22
Alright, I've got mongrel2 going on TLS 1.0 and SSL 3.0 using PolarSSL,
which is dual licensed as proprietary and GPLv2 with a FOSS exception (
http://polarssl.org/license_exception).

Zed, does the licensing here work?  If not, I'll go hunting for an
alternative.

Alex

On Tue, May 17, 2011 at 5:19 AM, Loic d'Anterroches <loic@ceondo.com> wrote:

> Hello,
>
> sorry for the delay, I am not at home at the moment and this makes
> things a bit of a mess.
>
> So, thank you for 1.6 and the roadmap and here is a small summary of I
> am ready to pay for in terms of SSL:
>
> - Selection of the ciphers (this is needed for PCI-DSS compliance).
> - Configuration through the config file (I can do it with sql, but I am
> lazy).
> - Working with 95% of the browsers (basically I suppose it means SSL +
> TLS).
>
> What is important is that I totally trust you for the technical
> implementation, if Zed says: "good for release" then this is good.
>
> The deadline to claim the bounty is end of September 2011. (I have a
> project I want to have in production at that time and need ssl for it).
>
> The winner(s) will be selected between the coders. If 2 of you are
> coding on it, I can do a 2x1k (collaboration bonus, because I also like
> when several people are familiar with such critical piece of code) and
> 1.5k if only one coder.
>
> Sorry for my English, back in France for 2 weeks, I cannot speak French
> correctly anymore but I am not really better in English/German...
> expatriate life...
> loïc
>
>
> On 05/11/2011 09:19 PM, Zed A. Shaw wrote:
> > On Wed, May 11, 2011 at 11:12:55AM -0700, Xavier Lange wrote:
> >> What WebSocket stuff will you be removing? The little bit about
> >> overriding the body size to 8?
> >
> > It needs to be upgraded to the latest (non-vulnerable) websockets spec
> > or it needs to go.  Ripping it out means that we don't have to test it
> > for this release, and then we can write a better one for 1.7.
> >
> > I'd say, if you're using it (and I think you're the only one), then
> > stick with 1.5 or fossil just before 1.6, and then it'll be back in 1.7.
> >
> > Also, the clients you're using shouldn't be working anyway.  I think all
> > the major browsers are now using the latest websockets which gets rid of
> > the stupidity I hated.
> >
>
> --
> Dr Loïc d'Anterroches
> Founder Céondo Ltd
>
> w: www.ceondo.com       |  e: loic@ceondo.com
> t: +44 (0)207 183 0016  |  f: +44 (0)207 183 0124
>
> Céondo Ltd
> Dalton House
> 60 Windsor Avenue
> London
> SW19 2RR / United Kingdom
>

Re: [mongrel2] SSL in Mongrel2

From:
Jon Rosebaugh
Date:
2011-05-19 @ 15:41
I must admit I'd been hoping for gnutls, at least partially because
that'd mean one fewer library to install.

On Wed, May 18, 2011 at 12:22 PM, Alex Gartrell <agartrell@cmu.edu> wrote:
> Alright, I've got mongrel2 going on TLS 1.0 and SSL 3.0 using PolarSSL,
> which is dual licensed as proprietary and GPLv2 with a FOSS exception
> (http://polarssl.org/license_exception).
> Zed, does the licensing here work?  If not, I'll go hunting for an
> alternative.
> Alex

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-19 @ 15:52
On Thu, May 19, 2011 at 08:41:15AM -0700, Jon Rosebaugh wrote:
> I must admit I'd been hoping for gnutls, at least partially because
> that'd mean one fewer library to install.

Yeah, it doesn't have a very good API for us, but he's just including
the source to polarssl so that it builds without the external dependency
(and so package maintainers can't be lazy and use a broken version of
the library in their packaging).

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-19 @ 16:30
Yeah, we went with polarssl because it's embeddable (if you compile source,
it compiles to a .a for static linking).  I just (inelegantly) threw all the
source into polarssl/, so it'll come "for free".  Fewer external
dependencies is kind of the mantra :)

The specific API advantage was being able to specify our own callbacks for
send/recv.  The Mongrel2 coroutine stuff requires calling specific variants
(so the task can be descheduled).  axTLS worked because we replaced a
#define, polarSSL is a lot better because you can actually feed it function
pointers.  Additionally, it actually works with SSL 3.0, and the interface
is simple, and it works with non-blocking IO if you want to do that, etc. (I
could go on and on).

On Thu, May 19, 2011 at 11:52 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

> On Thu, May 19, 2011 at 08:41:15AM -0700, Jon Rosebaugh wrote:
> > I must admit I'd been hoping for gnutls, at least partially because
> > that'd mean one fewer library to install.
>
> Yeah, it doesn't have a very good API for us, but he's just including
> the source to polarssl so that it builds without the external dependency
> (and so package maintainers can't be lazy and use a broken version of
> the library in their packaging).
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-19 @ 18:48
Alright, just added ability to specify ciphers and fixed a bunch of small
ssl stuff.  For some reason it seems to be forcing SSL 3.0 right now, so
I'll look into that next.

To use ssl, set the following global settings

settings = {"certdir": "certs/", # looks here for [uuid].crt and [uuid].key
            "2f62bd5-9e59-49cd-993c-3b6013c28f06.use_ssl": 1, # for each
server you want to use SSL
            "ssl_ciphers" : "SSL_RSA_AES_256_SHA,SSL_EDH_RSA_AES_256_SHA"}

ssl_ciphers can be omitted if you just want to use the defaults.  the actual
ciphers are in polarssl/ssl.h (I honestly just chose two at random for
demonstration's sake).

Alex

On Thu, May 19, 2011 at 12:30 PM, Alex Gartrell <agartrell@cmu.edu> wrote:

> Yeah, we went with polarssl because it's embeddable (if you compile source,
> it compiles to a .a for static linking).  I just (inelegantly) threw all the
> source into polarssl/, so it'll come "for free".  Fewer external
> dependencies is kind of the mantra :)
>
> The specific API advantage was being able to specify our own callbacks for
> send/recv.  The Mongrel2 coroutine stuff requires calling specific variants
> (so the task can be descheduled).  axTLS worked because we replaced a
> #define, polarSSL is a lot better because you can actually feed it function
> pointers.  Additionally, it actually works with SSL 3.0, and the interface
> is simple, and it works with non-blocking IO if you want to do that, etc. (I
> could go on and on).
>
>
> On Thu, May 19, 2011 at 11:52 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
>
>> On Thu, May 19, 2011 at 08:41:15AM -0700, Jon Rosebaugh wrote:
>> > I must admit I'd been hoping for gnutls, at least partially because
>> > that'd mean one fewer library to install.
>>
>> Yeah, it doesn't have a very good API for us, but he's just including
>> the source to polarssl so that it builds without the external dependency
>> (and so package maintainers can't be lazy and use a broken version of
>> the library in their packaging).
>>
>> --
>> Zed A. Shaw
>> http://zedshaw.com/
>>
>
>

Re: [mongrel2] SSL in Mongrel2

From:
Josh Simmons
Date:
2011-05-19 @ 21:47
— Function: void gnutls_transport_set_push_function (gnutls_session_t
session, gnutls_push_func push_func)
— Function: void gnutls_transport_set_pull_function (gnutls_session_t
session, gnutls_pull_func pull_func)

PUSH_FUNC is of the form, ssize_t
(*gnutls_push_func)(gnutls_transport_ptr_t, const void*, size_t);
PULL_FUNC is of the form, ssize_t
(*gnutls_pull_func)(gnutls_transport_ptr_t, void*, size_t);

What more do you want? :)

(Great work btw, and I don't have any preference as to what you use, just
sayin)

On Fri, May 20, 2011 at 4:48 AM, Alex Gartrell <agartrell@cmu.edu> wrote:
>
> Alright, just added ability to specify ciphers and fixed a bunch of small
ssl stuff.  For some reason it seems to be forcing SSL 3.0 right now, so
I'll look into that next.
> To use ssl, set the following global settings
> settings = {"certdir": "certs/", # looks here for [uuid].crt and
[uuid].key
>             "2f62bd5-9e59-49cd-993c-3b6013c28f06.use_ssl": 1, # for each
server you want to use SSL
>             "ssl_ciphers" : "SSL_RSA_AES_256_SHA,SSL_EDH_RSA_AES_256_SHA"}
> ssl_ciphers can be omitted if you just want to use the defaults.  the
actual ciphers are in polarssl/ssl.h (I honestly just chose two at random
for demonstration's sake).
> Alex
> On Thu, May 19, 2011 at 12:30 PM, Alex Gartrell <agartrell@cmu.edu> wrote:
>>
>> Yeah, we went with polarssl because it's embeddable (if you compile
source, it compiles to a .a for static linking).  I just (inelegantly) threw
all the source into polarssl/, so it'll come "for free".  Fewer external
dependencies is kind of the mantra :)
>> The specific API advantage was being able to specify our own callbacks
for send/recv.  The Mongrel2 coroutine stuff requires calling specific
variants (so the task can be descheduled).  axTLS worked because we replaced
a #define, polarSSL is a lot better because you can actually feed it
function pointers.  Additionally, it actually works with SSL 3.0, and the
interface is simple, and it works with non-blocking IO if you want to do
that, etc. (I could go on and on).
>>
>> On Thu, May 19, 2011 at 11:52 AM, Zed A. Shaw <zedshaw@zedshaw.com>
wrote:
>>>
>>> On Thu, May 19, 2011 at 08:41:15AM -0700, Jon Rosebaugh wrote:
>>> > I must admit I'd been hoping for gnutls, at least partially because
>>> > that'd mean one fewer library to install.
>>>
>>> Yeah, it doesn't have a very good API for us, but he's just including
>>> the source to polarssl so that it builds without the external dependency
>>> (and so package maintainers can't be lazy and use a broken version of
>>> the library in their packaging).
>>>
>>> --
>>> Zed A. Shaw
>>> http://zedshaw.com/
>>
>

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-19 @ 22:02
On Thu, May 19, 2011 at 5:47 PM, Josh Simmons <simmons.44@gmail.com> wrote:

> — Function: void gnutls_transport_set_push_function (gnutls_session_t
> session, gnutls_push_func push_func)
> — Function: void gnutls_transport_set_pull_function (gnutls_session_t
> session, gnutls_pull_func pull_func)
>
> PUSH_FUNC is of the form, ssize_t
> (*gnutls_push_func)(gnutls_transport_ptr_t, const void*, size_t);
> PULL_FUNC is of the form, ssize_t
> (*gnutls_pull_func)(gnutls_transport_ptr_t, void*, size_t);
>
> What more do you want? :)
>

You are absolutely right; I should have looked harder :)  If the TLS thing
doesn't work out right I'll switch it (IO Buffers make this a lot less work
than it sounds like).


>
> (Great work btw, and I don't have any preference as to what you use, just
> sayin)
>
> On Fri, May 20, 2011 at 4:48 AM, Alex Gartrell <agartrell@cmu.edu> wrote:
> >
> > Alright, just added ability to specify ciphers and fixed a bunch of small
> ssl stuff.  For some reason it seems to be forcing SSL 3.0 right now, so
> I'll look into that next.
> > To use ssl, set the following global settings
> > settings = {"certdir": "certs/", # looks here for [uuid].crt and
> [uuid].key
> >             "2f62bd5-9e59-49cd-993c-3b6013c28f06.use_ssl": 1, # for each
> server you want to use SSL
> >             "ssl_ciphers" :
> "SSL_RSA_AES_256_SHA,SSL_EDH_RSA_AES_256_SHA"}
> > ssl_ciphers can be omitted if you just want to use the defaults.  the
> actual ciphers are in polarssl/ssl.h (I honestly just chose two at random
> for demonstration's sake).
> > Alex
> > On Thu, May 19, 2011 at 12:30 PM, Alex Gartrell <agartrell@cmu.edu>
> wrote:
> >>
> >> Yeah, we went with polarssl because it's embeddable (if you compile
> source, it compiles to a .a for static linking).  I just (inelegantly) threw
> all the source into polarssl/, so it'll come "for free".  Fewer external
> dependencies is kind of the mantra :)
> >> The specific API advantage was being able to specify our own callbacks
> for send/recv.  The Mongrel2 coroutine stuff requires calling specific
> variants (so the task can be descheduled).  axTLS worked because we replaced
> a #define, polarSSL is a lot better because you can actually feed it
> function pointers.  Additionally, it actually works with SSL 3.0, and the
> interface is simple, and it works with non-blocking IO if you want to do
> that, etc. (I could go on and on).
> >>
> >> On Thu, May 19, 2011 at 11:52 AM, Zed A. Shaw <zedshaw@zedshaw.com>
> wrote:
> >>>
> >>> On Thu, May 19, 2011 at 08:41:15AM -0700, Jon Rosebaugh wrote:
> >>> > I must admit I'd been hoping for gnutls, at least partially because
> >>> > that'd mean one fewer library to install.
> >>>
> >>> Yeah, it doesn't have a very good API for us, but he's just including
> >>> the source to polarssl so that it builds without the external
> dependency
> >>> (and so package maintainers can't be lazy and use a broken version of
> >>> the library in their packaging).
> >>>
> >>> --
> >>> Zed A. Shaw
> >>> http://zedshaw.com/
> >>
> >
>
>

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-19 @ 22:10
On Thu, May 19, 2011 at 06:02:41PM -0400, Alex Gartrell wrote:
> On Thu, May 19, 2011 at 5:47 PM, Josh Simmons <simmons.44@gmail.com> wrote:
> 
> > — Function: void gnutls_transport_set_push_function (gnutls_session_t
> > session, gnutls_push_func push_func)
> > — Function: void gnutls_transport_set_pull_function (gnutls_session_t
> > session, gnutls_pull_func pull_func)
> >
> > PUSH_FUNC is of the form, ssize_t
> > (*gnutls_push_func)(gnutls_transport_ptr_t, const void*, size_t);
> > PULL_FUNC is of the form, ssize_t
> > (*gnutls_pull_func)(gnutls_transport_ptr_t, void*, size_t);
> >
> > What more do you want? :)
> >
> 
> You are absolutely right; I should have looked harder :)  If the TLS thing
> doesn't work out right I'll switch it (IO Buffers make this a lot less work
> than it sounds like).

Hehe, damn, well PolarSSL is pretty good so let's stick with it, but
we can keep this in the back pocket or add future support.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-25 @ 00:22
Alright, so it looks like my ssl 3.0 vs. tls 1.0 concerns were due to chrome
caching the old metadata when I was messing around with custom ssl_ciphers.
 Long story short: ssl is in a working state.

Now I'm looking for feedback :)  What am I missing?  What doesn't work?

Thanks everyone

Alex

On Thu, May 19, 2011 at 6:10 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

> On Thu, May 19, 2011 at 06:02:41PM -0400, Alex Gartrell wrote:
> > On Thu, May 19, 2011 at 5:47 PM, Josh Simmons <simmons.44@gmail.com>
> wrote:
> >
> > > — Function: void gnutls_transport_set_push_function (gnutls_session_t
> > > session, gnutls_push_func push_func)
> > > — Function: void gnutls_transport_set_pull_function (gnutls_session_t
> > > session, gnutls_pull_func pull_func)
> > >
> > > PUSH_FUNC is of the form, ssize_t
> > > (*gnutls_push_func)(gnutls_transport_ptr_t, const void*, size_t);
> > > PULL_FUNC is of the form, ssize_t
> > > (*gnutls_pull_func)(gnutls_transport_ptr_t, void*, size_t);
> > >
> > > What more do you want? :)
> > >
> >
> > You are absolutely right; I should have looked harder :)  If the TLS
> thing
> > doesn't work out right I'll switch it (IO Buffers make this a lot less
> work
> > than it sounds like).
>
> Hehe, damn, well PolarSSL is pretty good so let's stick with it, but
> we can keep this in the back pocket or add future support.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] SSL in Mongrel2

From:
Loic d'Anterroches
Date:
2011-05-25 @ 06:35
Alex,

thanks a lot. I will try to take a look as soon as possible. I have a
conference to prepare for early June and I am late... In the worse case
scenario, I wll be able to dig deeply into it only starting 14th of
June. Oh, and I do not forget the bounty :)

Congratulations for the work, thousands of people will enjoy it.

loïc

On 2011-05-25 02:22, Alex Gartrell wrote:
> Alright, so it looks like my ssl 3.0 vs. tls 1.0 concerns were due to
> chrome caching the old metadata when I was messing around with custom
> ssl_ciphers.  Long story short: ssl is in a working state.
> 
> Now I'm looking for feedback :)  What am I missing?  What doesn't work?
> 
> Thanks everyone
> 
> Alex
> 
> On Thu, May 19, 2011 at 6:10 PM, Zed A. Shaw <zedshaw@zedshaw.com
> <mailto:zedshaw@zedshaw.com>> wrote:
> 
>     On Thu, May 19, 2011 at 06:02:41PM -0400, Alex Gartrell wrote:
>     > On Thu, May 19, 2011 at 5:47 PM, Josh Simmons
>     <simmons.44@gmail.com <mailto:simmons.44@gmail.com>> wrote:
>     >
>     > > — Function: void gnutls_transport_set_push_function
>     (gnutls_session_t
>     > > session, gnutls_push_func push_func)
>     > > — Function: void gnutls_transport_set_pull_function
>     (gnutls_session_t
>     > > session, gnutls_pull_func pull_func)
>     > >
>     > > PUSH_FUNC is of the form, ssize_t
>     > > (*gnutls_push_func)(gnutls_transport_ptr_t, const void*, size_t);
>     > > PULL_FUNC is of the form, ssize_t
>     > > (*gnutls_pull_func)(gnutls_transport_ptr_t, void*, size_t);
>     > >
>     > > What more do you want? :)
>     > >
>     >
>     > You are absolutely right; I should have looked harder :)  If the
>     TLS thing
>     > doesn't work out right I'll switch it (IO Buffers make this a lot
>     less work
>     > than it sounds like).
> 
>     Hehe, damn, well PolarSSL is pretty good so let's stick with it, but
>     we can keep this in the back pocket or add future support.
> 
>     --
>     Zed A. Shaw
>     http://zedshaw.com/
> 
> 

-- 
Dr Loïc d'Anterroches
Founder Céondo Ltd

w: www.ceondo.com       |  e: loic@ceondo.com
t: +44 (0)207 183 0016  |  f: +44 (0)207 183 0124

Céondo Ltd
Dalton House
60 Windsor Avenue
London
SW19 2RR / United Kingdom

Re: [mongrel2] SSL in Mongrel2

From:
Alex Gartrell
Date:
2011-05-19 @ 05:03
Alright, just pushed polarSSL (instead of axTLS) into mongrel2, so it should
work well using the prior configuration with older browsers.  Still, a lot
to do as far as the config stuff is concerned, but it should be relatively
easy (polarSSL is a *very* good library).

Alex

On Wed, May 18, 2011 at 3:22 PM, Alex Gartrell <agartrell@cmu.edu> wrote:

> Alright, I've got mongrel2 going on TLS 1.0 and SSL 3.0 using PolarSSL,
> which is dual licensed as proprietary and GPLv2 with a FOSS exception (
> http://polarssl.org/license_exception).
>
> Zed, does the licensing here work?  If not, I'll go hunting for an
> alternative.
>
> Alex
>
>
> On Tue, May 17, 2011 at 5:19 AM, Loic d'Anterroches <loic@ceondo.com>wrote:
>
>> Hello,
>>
>> sorry for the delay, I am not at home at the moment and this makes
>> things a bit of a mess.
>>
>> So, thank you for 1.6 and the roadmap and here is a small summary of I
>> am ready to pay for in terms of SSL:
>>
>> - Selection of the ciphers (this is needed for PCI-DSS compliance).
>> - Configuration through the config file (I can do it with sql, but I am
>> lazy).
>> - Working with 95% of the browsers (basically I suppose it means SSL +
>> TLS).
>>
>> What is important is that I totally trust you for the technical
>> implementation, if Zed says: "good for release" then this is good.
>>
>> The deadline to claim the bounty is end of September 2011. (I have a
>> project I want to have in production at that time and need ssl for it).
>>
>> The winner(s) will be selected between the coders. If 2 of you are
>> coding on it, I can do a 2x1k (collaboration bonus, because I also like
>> when several people are familiar with such critical piece of code) and
>> 1.5k if only one coder.
>>
>> Sorry for my English, back in France for 2 weeks, I cannot speak French
>> correctly anymore but I am not really better in English/German...
>> expatriate life...
>> loïc
>>
>>
>> On 05/11/2011 09:19 PM, Zed A. Shaw wrote:
>> > On Wed, May 11, 2011 at 11:12:55AM -0700, Xavier Lange wrote:
>> >> What WebSocket stuff will you be removing? The little bit about
>> >> overriding the body size to 8?
>> >
>> > It needs to be upgraded to the latest (non-vulnerable) websockets spec
>> > or it needs to go.  Ripping it out means that we don't have to test it
>> > for this release, and then we can write a better one for 1.7.
>> >
>> > I'd say, if you're using it (and I think you're the only one), then
>> > stick with 1.5 or fossil just before 1.6, and then it'll be back in 1.7.
>> >
>> > Also, the clients you're using shouldn't be working anyway.  I think all
>> > the major browsers are now using the latest websockets which gets rid of
>> > the stupidity I hated.
>> >
>>
>> --
>> Dr Loïc d'Anterroches
>> Founder Céondo Ltd
>>
>> w: www.ceondo.com       |  e: loic@ceondo.com
>> t: +44 (0)207 183 0016  |  f: +44 (0)207 183 0124
>>
>> Céondo Ltd
>> Dalton House
>> 60 Windsor Avenue
>> London
>> SW19 2RR / United Kingdom
>>
>
>

Re: [mongrel2] SSL in Mongrel2

From:
Loic d'Anterroches
Date:
2011-05-19 @ 08:04

On 05/19/2011 07:03 AM, Alex Gartrell wrote:
> Alright, just pushed polarSSL (instead of axTLS) into mongrel2, so it
> should work well using the prior configuration with older browsers.
>   Still, a lot to do as far as the config stuff is concerned, but it
> should be relatively easy (polarSSL is a *very* good library).

Thanks a lot!

loïc

> On Wed, May 18, 2011 at 3:22 PM, Alex Gartrell <agartrell@cmu.edu
> <mailto:agartrell@cmu.edu>> wrote:
>
>     Alright, I've got mongrel2 going on TLS 1.0 and SSL 3.0 using
>     PolarSSL, which is dual licensed as proprietary and GPLv2 with a
>     FOSS exception (http://polarssl.org/license_exception).
>
>     Zed, does the licensing here work?  If not, I'll go hunting for an
>     alternative.
>
>     Alex
>
>
>     On Tue, May 17, 2011 at 5:19 AM, Loic d'Anterroches <loic@ceondo.com
>     <mailto:loic@ceondo.com>> wrote:
>
>         Hello,
>
>         sorry for the delay, I am not at home at the moment and this makes
>         things a bit of a mess.
>
>         So, thank you for 1.6 and the roadmap and here is a small
>         summary of I
>         am ready to pay for in terms of SSL:
>
>         - Selection of the ciphers (this is needed for PCI-DSS compliance).
>         - Configuration through the config file (I can do it with sql,
>         but I am
>         lazy).
>         - Working with 95% of the browsers (basically I suppose it means
>         SSL + TLS).
>
>         What is important is that I totally trust you for the technical
>         implementation, if Zed says: "good for release" then this is good.
>
>         The deadline to claim the bounty is end of September 2011. (I have a
>         project I want to have in production at that time and need ssl
>         for it).
>
>         The winner(s) will be selected between the coders. If 2 of you are
>         coding on it, I can do a 2x1k (collaboration bonus, because I
>         also like
>         when several people are familiar with such critical piece of
>         code) and
>         1.5k if only one coder.
>
>         Sorry for my English, back in France for 2 weeks, I cannot speak
>         French
>         correctly anymore but I am not really better in English/German...
>         expatriate life...
>         loïc
>
>
>         On 05/11/2011 09:19 PM, Zed A. Shaw wrote:
>          > On Wed, May 11, 2011 at 11:12:55AM -0700, Xavier Lange wrote:
>          >> What WebSocket stuff will you be removing? The little bit about
>          >> overriding the body size to 8?
>          >
>          > It needs to be upgraded to the latest (non-vulnerable)
>         websockets spec
>          > or it needs to go.  Ripping it out means that we don't have
>         to test it
>          > for this release, and then we can write a better one for 1.7.
>          >
>          > I'd say, if you're using it (and I think you're the only
>         one), then
>          > stick with 1.5 or fossil just before 1.6, and then it'll be
>         back in 1.7.
>          >
>          > Also, the clients you're using shouldn't be working anyway.
>           I think all
>          > the major browsers are now using the latest websockets which
>         gets rid of
>          > the stupidity I hated.
>          >
>
>         --

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-19 @ 06:24
On Thu, May 19, 2011 at 01:03:42AM -0400, Alex Gartrell wrote:
> Alright, just pushed polarSSL (instead of axTLS) into mongrel2, so it should
> work well using the prior configuration with older browsers.  Still, a lot
> to do as far as the config stuff is concerned, but it should be relatively
> easy (polarSSL is a *very* good library).

Cool, alright lets rip out axtls then.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Jon Rosebaugh
Date:
2011-05-10 @ 19:36
On Tue, May 10, 2011 at 12:27 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> If axTLS don't work, then OpenSSL it is, which I hope you can find
> enough information on using correctly.  Last time I tried it was a
> complete black box because the OpenSSL devs seem to not like to document
> a damn thing they do.

Are things like cryptlib or GnuTLS out of the question, for licensing
or other reasons?

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-10 @ 22:14
On Tue, May 10, 2011 at 12:36:24PM -0700, Jon Rosebaugh wrote:
> On Tue, May 10, 2011 at 12:27 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> > If axTLS don't work, then OpenSSL it is, which I hope you can find
> > enough information on using correctly.  Last time I tried it was a
> > complete black box because the OpenSSL devs seem to not like to document
> > a damn thing they do.
> 
> Are things like cryptlib or GnuTLS out of the question, for licensing
> or other reasons?

We should probably review possible options, but yes the license is a big
deal.  LGPL is fine, but GPL isn't going to work.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Jon Rosebaugh
Date:
2011-05-10 @ 23:01
On Tue, May 10, 2011 at 3:14 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Tue, May 10, 2011 at 12:36:24PM -0700, Jon Rosebaugh wrote:
>> On Tue, May 10, 2011 at 12:27 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
>> > If axTLS don't work, then OpenSSL it is, which I hope you can find
>> > enough information on using correctly.  Last time I tried it was a
>> > complete black box because the OpenSSL devs seem to not like to document
>> > a damn thing they do.
>>
>> Are things like cryptlib or GnuTLS out of the question, for licensing
>> or other reasons?
>
> We should probably review possible options, but yes the license is a big
> deal.  LGPL is fine, but GPL isn't going to work.

GnuTLS is (mostly) LGPL (the OpenSSL compatibility layer and a few
other functions in the gnutls-extra library are GPL, but seem to be
highly optional).
cryptlib is strong-copyleft though; I inadvertently overlooked that
when writing my earlier email.

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-11 @ 15:02
On Tue, May 10, 2011 at 04:01:24PM -0700, Jon Rosebaugh wrote:
> > We should probably review possible options, but yes the license is a big
> > deal.  LGPL is fine, but GPL isn't going to work.
> 
> GnuTLS is (mostly) LGPL (the OpenSSL compatibility layer and a few
> other functions in the gnutls-extra library are GPL, but seem to be
> highly optional).

Oh look, *and* it has documentation:

http://www.gnu.org/software/gnutls/manual/gnutls.html

Well damn, I'll look at that then.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-10 @ 17:23
On Tue, May 10, 2011 at 02:53:38PM +0200, Loic d'Anterroches wrote:
> Hello,
> 
> this question is especially for Zed, basically, do you think that the 
> support of SSL directly within Mongrel2 could be acceptable? A few 
> months ago, you wrote on the channel that you would prefer to have 
> people using stunnel. Is it still the case? If so, just answer "still 
> the case and close my request".

That was an idea I had...but!

> The idea behind implementing SSL directly into Mongrel2 is to ease 
> deployment and get good performance while keeping track of the originate 
> IP.

This is the problem with stunnel.  Really it'd perform better on ssl,
it's easier to setup, it's got extra powers, and it works already.  The
*one* failing of it is that you don't get the originating IP address.
That pretty much kills it for about 40% of the web deployments out there
so we're back to continuing the development of SSL inside mongrel2.

> Practically, I can put $1.5k and up to $3k. Zed knows that I promise and 
> pay.

Well, that'd be awesome, I actually may not have time to do it, but how
about you write a ticket, and then you can post a bounty for the support
and let the best patch win?  Let's talk off-line and we can setup the
requirements for the submission or the bounty.  The SSL support is
already pretty close to done, could just mean someone finalizing and
testing it further.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Jon Rosebaugh
Date:
2011-05-10 @ 17:36
On Tue, May 10, 2011 at 10:23 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> This is the problem with stunnel.  Really it'd perform better on ssl,
> it's easier to setup, it's got extra powers, and it works already.  The
> *one* failing of it is that you don't get the originating IP address.
> That pretty much kills it for about 40% of the web deployments out there
> so we're back to continuing the development of SSL inside mongrel2.

Wouldn't it be possible to, say, add support for zmq or some trusted
channel to stunnel and let it forward the requests to mongrel2?

Re: [mongrel2] SSL in Mongrel2

From:
Peter Keen
Date:
2011-05-10 @ 17:32
On Tue, May 10, 2011 at 10:23 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> This is the problem with stunnel.  Really it'd perform better on ssl,
> it's easier to setup, it's got extra powers, and it works already.  The
> *one* failing of it is that you don't get the originating IP address.
> That pretty much kills it for about 40% of the web deployments out there
> so we're back to continuing the development of SSL inside mongrel2.

In previous deployments not related to mongrel2 but with a pretty
standard setup (Apache 1.3 + mod_perl), we had trouble making Apache's
mod_ssl and mod_gzip work together. The setup we ended up with was
nginx in front of Apache proxying the HTTPS requests. Nginx was
configured to set X-Forwarded-For to the original IP, which we then
wrote wrappers for in our code that would either look for that header
and fall back to the standard one.

This is, of course, pretty gross, but it would let you get away with
not using stunnel. Performance was never an issue with nginx either.

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-10 @ 19:24
On Tue, May 10, 2011 at 10:32:25AM -0700, Peter Keen wrote:
> In previous deployments not related to mongrel2 but with a pretty
> standard setup (Apache 1.3 + mod_perl), we had trouble making Apache's
> mod_ssl and mod_gzip work together. The setup we ended up with was
> nginx in front of Apache proxying the HTTPS requests. Nginx was
> configured to set X-Forwarded-For to the original IP, which we then
> wrote wrappers for in our code that would either look for that header
> and fall back to the standard one.
> 
> This is, of course, pretty gross, but it would let you get away with
> not using stunnel. Performance was never an issue with nginx either.

Right, and folks who run it like this, or haproxy have no problem with
mongrel2 not having SSL.  Also things like Amazon's SSL proxies.  Where
it falls down is people who run mongrel2 on the edge and need ssl, but
also need the IP addresses.  In that case, stunnel doesn't cut it.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Loic d'Anterroches
Date:
2011-05-10 @ 20:23

On 05/10/2011 09:24 PM, Zed A. Shaw wrote:
> On Tue, May 10, 2011 at 10:32:25AM -0700, Peter Keen wrote:
>> In previous deployments not related to mongrel2 but with a pretty
>> standard setup (Apache 1.3 + mod_perl), we had trouble making Apache's
>> mod_ssl and mod_gzip work together. The setup we ended up with was
>> nginx in front of Apache proxying the HTTPS requests. Nginx was
>> configured to set X-Forwarded-For to the original IP, which we then
>> wrote wrappers for in our code that would either look for that header
>> and fall back to the standard one.
>>
>> This is, of course, pretty gross, but it would let you get away with
>> not using stunnel. Performance was never an issue with nginx either.
>
> Right, and folks who run it like this, or haproxy have no problem with
> mongrel2 not having SSL.  Also things like Amazon's SSL proxies.  Where
> it falls down is people who run mongrel2 on the edge and need ssl, but
> also need the IP addresses.  In that case, stunnel doesn't cut it.

The main problem with haproxy and such kind of setup is that to rewrite 
the headers to insert the x-forwarded-for header, they need to close the 
connection after each request/answer. So, you are losing the benefits of 
keep alive. The current webservers are able to support an insane amount 
of open connections with very little memory. So we can practically have 
keep alive on the complete path now.

Nginx is doing http 1.0 with the backend, so I am not sure how jssocket 
would work in the client <-> nginx <-> mongrel2 configuration.

Anyway, please keep putting ideas/requests in this thread, I will write 
a ticket to put everything together for the end of the week or early 
next week.

loïc

Re: [mongrel2] SSL in Mongrel2

From:
Armando Singer
Date:
2011-05-10 @ 22:01
On May 10, 2011, at 1:23 PM, Loic d'Anterroches wrote:

> The main problem with haproxy and such kind of setup is that to rewrite 
> the headers to insert the x-forwarded-for header, they need to close the 
> connection after each request/answer. So, you are losing the benefits of 
> keep alive. The current webservers are able to support an insane amount 
> of open connections with very little memory. So we can practically have 
> keep alive on the complete path now.
> 
> Nginx is doing http 1.0 with the backend, so I am not sure how jssocket 
> would work in the client <-> nginx <-> mongrel2 configuration.
> 
> Anyway, please keep putting ideas/requests in this thread, I will write 
> a ticket to put everything together for the end of the week or early 
> next week.

Have you considered a commercial product like zxtm? (Or amazon elastic
LB, but I have no experience w/ that). SSL+caching+keep
alive+gzip+LB+trafficscript (sort of like varnish VCL) is a killer
combo, especially with homogenous and language agnostic back ends
which shouldn't have to worry about any of that stuff.

A lot of that is possible with varnish 3, but it looks like they do
not want to support SSL any time soon:
http://www.varnish-cache.org/docs/trunk/phk/ssl.html

Cheers,
Armando

> 
> loïc

Re: [mongrel2] SSL in Mongrel2

From:
Loic d'Anterroches
Date:
2011-05-11 @ 07:56

On 05/11/2011 12:01 AM, Armando Singer wrote:
> On May 10, 2011, at 1:23 PM, Loic d'Anterroches wrote:
>
>> The main problem with haproxy and such kind of setup is that to rewrite
>> the headers to insert the x-forwarded-for header, they need to close the
>> connection after each request/answer. So, you are losing the benefits of
>> keep alive. The current webservers are able to support an insane amount
>> of open connections with very little memory. So we can practically have
>> keep alive on the complete path now.
>>
>> Nginx is doing http 1.0 with the backend, so I am not sure how jssocket
>> would work in the client<->  nginx<->  mongrel2 configuration.
>>
>> Anyway, please keep putting ideas/requests in this thread, I will write
>> a ticket to put everything together for the end of the week or early
>> next week.
>
> Have you considered a commercial product like zxtm? (Or amazon elastic
> LB, but I have no experience w/ that). SSL+caching+keep
> alive+gzip+LB+trafficscript (sort of like varnish VCL) is a killer
> combo, especially with homogenous and language agnostic back ends
> which shouldn't have to worry about any of that stuff.
>
> A lot of that is possible with varnish 3, but it looks like they do
> not want to support SSL any time soon:
> http://www.varnish-cache.org/docs/trunk/phk/ssl.html

If I have to go the non open source way, I can get some CISCO ACE cards 
for my servers, so I would prefer to have SSL at the Mongrel2 level. But 
thank you for the tip, I read the info on the Zeus website, the only 
info missing is that you cannot get a price tag on their products, you 
need to go through partners etc.

The other good point of SSL at the Mongrel2 level is that it offers even 
more flexibility in setup. I can setup a dedicated box with only HAProxy 
in transparent mode, balancing at the tcp level and spread the work load 
on many Mongrel2 backends.

Anyway, thanks a lot for the nice feedback,
loïc

Re: [mongrel2] SSL in Mongrel2

From:
Andrew Yourtchenko
Date:
2011-05-10 @ 14:11
Loic,

On Tue, May 10, 2011 at 2:53 PM, Loic d'Anterroches <loic@ceondo.com> wrote:
> Hello,
>
> this question is especially for Zed, basically, do you think that the
> support of SSL directly within Mongrel2 could be acceptable? A few
> months ago, you wrote on the channel that you would prefer to have
> people using stunnel. Is it still the case? If so, just answer "still
> the case and close my request".
>

Admittedly, I did not play with the m2 code for quite some time, but
back in January I did experiment with SSL configs in it, and wrote up
on the experience here:

http://bnpcs.blogspot.com/2011/01/mongrel2-and-ssl.html

Give it a shot and see how it works for you...

cheers,
andrew

> If SSL within Mongrel2 could be fine with you, I am ready to sponsor the
> addition to the core. I most likely do not have enough money to pay for
> the full implementation, but I can help bootstrap the work and maybe
> some other people could help with money, time, testing, coding etc.
>
> The idea behind implementing SSL directly into Mongrel2 is to ease
> deployment and get good performance while keeping track of the originate
> IP. You can run stunnel in transparent mode, but if you want to know at
> the app level both if the traffic is SSL or not and the client IP, you
> need to either force a close on the connection to not use keepalive and
> inject the forwarded-for header or setup 2 servers on Mongrel2, one for
> the ssl traffic and one for the plain traffic. With many sites, you end
> up having a lot to think about, especially you need to open local ports
> etc. I always prefer to have the minimum of moving parts (even so I like
> the one tool does one thing and well philosophy of Unix, the right
> balance is always hard to find).
>
> Practically, I can put $1.5k and up to $3k. Zed knows that I promise and
> pay. Money in free software is always a balance to find. I know from
> experience that offering money when interest to get the stuff done is
> not already there is just a waste of time and money. So if the boss says
> ok and one of you is interested of having the support of SSL, count on
> me to help you buy some cookies, if not, I will find a way with
> stunnel/haproxy & cie.
>
> loïc
>

Re: [mongrel2] SSL in Mongrel2

From:
Nathan Duran
Date:
2011-05-25 @ 17:46
> Interesting, that default might be too small, so what did you end up
setting
> it to?

Through trial-and-error, it looks like it needs to be at least 18 to work
with mod_proxy. 

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-25 @ 19:59
On Wed, May 25, 2011 at 10:46:47AM -0700, Nathan Duran wrote:
> > Interesting, that default might be too small, so what did you end up
> setting
> > it to?
> 
> Through trial-and-error, it looks like it needs to be at least 18 to work
> with mod_proxy. 
>

Rock, ok I'll put at double that and then touch it up some.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Nathan Duran
Date:
2011-05-25 @ 05:07
> Interesting, that default might be too small, so what did you end up
setting
> it to?

I jacked it way up to 100 just to eliminate any possibility of pukage. I'll
probably sit down with one of our ops guys to see what the cutoff really
needs to be tomorrow. 

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-25 @ 19:58
On Tue, May 24, 2011 at 10:07:29PM -0700, Nathan Duran wrote:
> > Interesting, that default might be too small, so what did you end up
> setting
> > it to?
> 
> I jacked it way up to 100 just to eliminate any possibility of pukage. I'll
> probably sit down with one of our ops guys to see what the cutoff really
> needs to be tomorrow. 
> 

Alright cool.  Yeah, it's sort of an important protection against evil
client attacks, but you can probably set it higher and find your sweet
spot.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] SSL in Mongrel2

From:
Nathan Duran
Date:
2011-05-25 @ 00:40
> Alright, so it looks like my ssl 3.0 vs. tls 1.0 concerns were due to chrome
> caching the old metadata when I was messing around with custom ssl_ciphers.
>  Long story short: ssl is in a working state.
> 
> Now I'm looking for feedback :)  What am I missing?  What doesn't work?

I encountered a very small issue this afternoon where I was trying to use
mod_proxy to send some HTTPS traffic to a Mongrel2 instance running with the
current SSL lib and Mongrel was crapping out on Apache because of "Too many
small packet read attempts." I didn't dig into the underlying cause at all,
but it was solved by upping "limits.client_read_retries" beyond its default
value of 5--this value might need to be either bumped or documented as a
potential cause of problems in this specific situation.

Otherwise, it's been working great so far. Many thanks. 

Re: [mongrel2] SSL in Mongrel2

From:
Zed A. Shaw
Date:
2011-05-25 @ 04:41
On Tue, May 24, 2011 at 05:40:22PM -0700, Nathan Duran wrote:
> > Alright, so it looks like my ssl 3.0 vs. tls 1.0 concerns were due to chrome
> > caching the old metadata when I was messing around with custom ssl_ciphers.
> >  Long story short: ssl is in a working state.
> > 
> > Now I'm looking for feedback :)  What am I missing?  What doesn't work?
> 
> I encountered a very small issue this afternoon where I was trying to use
> mod_proxy to send some HTTPS traffic to a Mongrel2 instance running with the
> current SSL lib and Mongrel was crapping out on Apache because of "Too many
> small packet read attempts." I didn't dig into the underlying cause at all,
> but it was solved by upping "limits.client_read_retries" beyond its default
> value of 5--this value might need to be either bumped or documented as a
> potential cause of problems in this specific situation.

Interesting, that default might be too small, so what did you end up
setting it to?

-- 
Zed A. Shaw
http://zedshaw.com/