librelist archives

Re: [mongrel2] HTTP_X_FORWARDED_PROTO

From:

Mathijs Kwik

Date:

2011-08-12 @ 17:20

I know about X_FORWARDED_FOR, that's not what I was looking for.
I'm looking for the original protocol, not the original ip.

Rails mandates it's called X_FORWARDED_PROTO and contains "http" or "https".
So if mongrel doesn't set that header (or provide a way in the config
file to do so), I will have to patch/hack rails for it.

I know about the multiple-forwards thing you describe for
X_FORWARDED_FOR, where a client proxy can add itself inbetween (which
we shouldn't always trust), and allows many server-sided proxies to
forward the request. The way rails seems to handle it, is by looking
step-by-step further upstream, until it finds an ip address out of our
control (not on 192.168.* or 10.*) and registers that one as the
remote ip. So if a client is trying to tell us he's just proxying for
some other ip, we don't believe him and just register his ip.



On Fri, Aug 12, 2011 at 7:01 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Fri, Aug 12, 2011 at 02:39:02PM +0200, Mathijs Kwik wrote:
>> Hi all,
>>
>> I'm not sure if this header is standard in any way, but at least ruby
>> on rails looks for it to find out if it's running on ssl.
>
> It's there, and it's just x-forwarded-for and it preserves what the
> client sent as well ass adding its own for what is really sent.  If you
> dump that header you'll get a list if the client tried to set it with
> both the client's and the server's.
>
> Most of the time, if there's more than one x-forwarded-for you want to
> reject the request.  Sometimes you want the client to specify it too,
> but if you're relying on it for some security then I'd reject.
>
> Also, if you're coming from a world where they tack on HTTP_* to all the
> headers then this is different.  We leave them alone, and instead do
> this:
>
> 1. Force all headers from the client to lower case.  Helps in matching
> since they're case insensitive, and makes it so you won't match them
> against the ones Mongrel2 sets.
> 2. Converts all headers with multiple values to a list.
> 3. Leaves ones with just one value as a single string, unaltered.
> 4. Add any Mongrel2 special headers as UPPERCASE only.
>
> This makes it easy to let the client send anything they want, but not
> let them overwrite something Mongrel2 has set.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

On Fri, Aug 12, 2011 at 07:20:13PM +0200, Mathijs Kwik wrote:
> I know about X_FORWARDED_FOR, that's not what I was looking for.
> I'm looking for the original protocol, not the original ip.
> 
> Rails mandates it's called X_FORWARDED_PROTO and contains "http" or "https".
> So if mongrel doesn't set that header (or provide a way in the config
> file to do so), I will have to patch/hack rails for it.

Oh, I misread.  Yeah I can add that.  I wonder if this is a rubyonrails
only thing or if it's a generally accepted practice.  I'll have to look
around.

-- 
Zed A. Shaw
http://zedshaw.com/

I think it's gaining some currency as a best practice.  Amazon ELBs added 
this to facilitate redirecting all incoming connections to HTTPS in the 
case where the ELB is doing the HTTPS termination and all local traffic is
forwarded as HTTP.

✈ Matt



On Aug 13, 2011, at 09:13 , Zed A. Shaw wrote:

> On Fri, Aug 12, 2011 at 07:20:13PM +0200, Mathijs Kwik wrote:
>> I know about X_FORWARDED_FOR, that's not what I was looking for.
>> I'm looking for the original protocol, not the original ip.
>> 
>> Rails mandates it's called X_FORWARDED_PROTO and contains "http" or "https".
>> So if mongrel doesn't set that header (or provide a way in the config
>> file to do so), I will have to patch/hack rails for it.
> 
> Oh, I misread.  Yeah I can add that.  I wonder if this is a rubyonrails
> only thing or if it's a generally accepted practice.  I'll have to look
> around.
> 
> -- 
> Zed A. Shaw
> http://zedshaw.com/

Well, apart from that, even if it's not a standard, mongrel2 doesn't
allow you to add custom headers through config.
So there's no way to "do it yourself" on the mongrel-side.
That would mean you need to patch up the handler/proxy backend, which
has no way to know this information, apart from putting it statically
in some config, which is problematic if you want the handler to be
available on both http and https.

So if we think this is not "standard enough", please create the
possibility to add custom headers, so we can handle this
"mongrel-sided".

Thanks,
Mathijs

On Mon, Aug 15, 2011 at 5:34 PM, Matt Towers <matt@ziplinegames.com> wrote:
> I think it's gaining some currency as a best practice.  Amazon ELBs added
> this to facilitate redirecting all incoming connections to HTTPS in the case
> where the ELB is doing the HTTPS termination and all local traffic is
> forwarded as HTTP.
>
> ✈ Matt
>
>
> On Aug 13, 2011, at 09:13 , Zed A. Shaw wrote:
>
> On Fri, Aug 12, 2011 at 07:20:13PM +0200, Mathijs Kwik wrote:
>
> I know about X_FORWARDED_FOR, that's not what I was looking for.
>
> I'm looking for the original protocol, not the original ip.
>
> Rails mandates it's called X_FORWARDED_PROTO and contains "http" or "https".
>
> So if mongrel doesn't set that header (or provide a way in the config
>
> file to do so), I will have to patch/hack rails for it.
>
> Oh, I misread.  Yeah I can add that.  I wonder if this is a rubyonrails
> only thing or if it's a generally accepted practice.  I'll have to look
> around.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>
>

On Mon, Aug 15, 2011 at 06:27:53PM +0200, Mathijs Kwik wrote:
> So if we think this is not "standard enough", please create the
> possibility to add custom headers, so we can handle this
> "mongrel-sided".

Hmm, that could be a more future proof solution.  An "arbitrary headers
add" config.  Can you think up how it would work and where you'd want it
in the config?

-- 
Zed A. Shaw
http://zedshaw.com/

On Mon, Aug 15, 2011 at 10:42 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> Hmm, that could be a more future proof solution.  An "arbitrary headers
> add" config.  Can you think up how it would work and where you'd want it
> in the config?

It seems to me this is essentially a special-case of a filter,
especially given the need to inspect arbitrary properties of a request
and set arbitrary headers.

Some sort of lua API for writing filters would be cool.

On Mon, Aug 15, 2011 at 12:48 PM, Jon Rosebaugh <chairos@gmail.com> wrote:
> Some sort of lua API for writing filters would be cool.

+1

-- 
Javier

On Aug 15, 2011, at 11:50 , Javier Guerra Giraldez wrote:

> On Mon, Aug 15, 2011 at 12:48 PM, Jon Rosebaugh <chairos@gmail.com> wrote:
>> Some sort of lua API for writing filters would be cool.
> 
> +1
> 
> -- 
> Javier

I'll second that as well.

Don't know about lua here (only know that from scripting WoW :), but
indeed adding/changing/mangling headers seems like a filter to me.

Probably somewhat like this:
https://gist.github.com/1147540



On Mon, Aug 15, 2011 at 8:56 PM, Matt Towers <matt@ziplinegames.com> wrote:
> On Aug 15, 2011, at 11:50 , Javier Guerra Giraldez wrote:
>
>> On Mon, Aug 15, 2011 at 12:48 PM, Jon Rosebaugh <chairos@gmail.com> wrote:
>>> Some sort of lua API for writing filters would be cool.
>>
>> +1
>>
>> --
>> Javier
>
> I'll second that as well.
>

On Tue, Aug 16, 2011 at 5:32 AM, Mathijs Kwik <bluescreen303@gmail.com> wrote:
> Don't know about lua here (only know that from scripting WoW :), but
> indeed adding/changing/mangling headers seems like a filter to me.
>
> Probably somewhat like this:
> https://gist.github.com/1147540
>
>
>
> On Mon, Aug 15, 2011 at 8:56 PM, Matt Towers <matt@ziplinegames.com> wrote:
>> On Aug 15, 2011, at 11:50 , Javier Guerra Giraldez wrote:
>>
>>> On Mon, Aug 15, 2011 at 12:48 PM, Jon Rosebaugh <chairos@gmail.com> wrote:
>>>> Some sort of lua API for writing filters would be cool.
>>>
>>> +1
>>>
>>> --
>>> Javier
>>
>> I'll second that as well.
>>
>

It'd be pretty trivial to create a proxy filter that passes stuff onto
your language of choice. Not sure whether it'd really be worth it
though.