librelist archives

home // archives //

« back to archive

flash socket / flash policy server

flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-12 @ 13:08 
Hi again,

I hope I'm not asking too much, but I'm investigating a lot at once,
take your time answering, no hurries :)
For flash sockets to work, flash wants to check if it's allowed first.
it will look for a policy xml document in 2 places:
- tcp port 843 of the server you're asking it to connect to.
- http(s) /crossdomain.xml file

Before looking for /crossdomain.xml, it tries the 843 server 3 times,
with some time in between.
So the /crossdomain.xml file will slow down flash connections, and
might even get deprecated in the future.

I'm currently using a node.js tcp server to serve the (static) content
on connects on port 843 and disconnect afterwards.
Do I need to keep on doing this? Or is there a way to get mongrel2 serving it?
As mentioned, it's plain TCP, no HTTP, no requesing/parsing at all,
just write the file, close socket.

It would be nice if this could be implemented somehow, so flashsockets
will work cross-domain without a few seconds delay.

Thanks,
Mathijs

Re: [mongrel2] flash socket / flash policy server

From:
Zed A. Shaw
Date:
2011-08-12 @ 17:03 
On Fri, Aug 12, 2011 at 03:08:33PM +0200, Mathijs Kwik wrote:
> Hi again,
> 
> I hope I'm not asking too much, but I'm investigating a lot at once,
> take your time answering, no hurries :)
> For flash sockets to work, flash wants to check if it's allowed first.
> it will look for a policy xml document in 2 places:

You can just point it at the same port as HTTP and mongrel2 will figure
out what you're doing and spit out the policy XML it uses.
Incidentally, it uses a policy file by default that is "open to the
world, shut up I don't care".

Look at the examples/chat/ demo to see how it's being done with the
simple jssocket setup.

Also, Mongrel2 adopts a simple standard of @somepath to do it's flash
socket routing.  The only defect in this (which I'm going to fix soon)
is that it will always route to the default_host for the server since it
has no host information in that route.

Again, look at the chat demo and bug me for help if it's bitrot.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-12 @ 18:37 
On Fri, Aug 12, 2011 at 7:03 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Fri, Aug 12, 2011 at 03:08:33PM +0200, Mathijs Kwik wrote:
>> Hi again,
>>
>> I hope I'm not asking too much, but I'm investigating a lot at once,
>> take your time answering, no hurries :)
>> For flash sockets to work, flash wants to check if it's allowed first.
>> it will look for a policy xml document in 2 places:
>
> You can just point it at the same port as HTTP and mongrel2 will figure
> out what you're doing and spit out the policy XML it uses.
> Incidentally, it uses a policy file by default that is "open to the
> world, shut up I don't care".
>
> Look at the examples/chat/ demo to see how it's being done with the
> simple jssocket setup.
>
> Also, Mongrel2 adopts a simple standard of @somepath to do it's flash
> socket routing.  The only defect in this (which I'm going to fix soon)
> is that it will always route to the default_host for the server since it
> has no host information in that route.
>
> Again, look at the chat demo and bug me for help if it's bitrot.

As far as I can tell, the chat demo doesn't handle this in any way.
I understand that mongrel2 handles flash connections with a special @
protocol and that flash needs to setup socket connections just to the
same host/port as your normal http browser traffic would come from.

What I'm talking about here is that flash, since 9.0.124.0, wants to
read a policy file before doing any socket connection. In the past,
connecting to the same server (non cross domain) didn't need a policy,
but now it does.

This policy file can either be read from /crossdomain.xml (just from
http) in the root of the domain we connect to, or served from port 843
of that machine.
The port is preferred and checked first, multiple times. If that
fails, /crossdomain.xml is tried.

If your firewall is dropping packets it doesn't want, this means flash
will hang a long long time before trying the /crossdomain.xml file
from http.
Changing this to blocking (refusing), fixes this, but still flash
tries 3 times, and total delay is around 5 seconds.
So the only way to make flash socket connections (to mongrel's normal
http port) be instantaneous, is to have a small tcp server listen on
843 which serves this xml policy.

I can stick to node.js to take care of this, but it would be very nice
if mongrel2 could serve a default open-to-the-world one(which will be
fine for 99% cases I think).




>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] flash socket / flash policy server

From:
Zed A. Shaw
Date:
2011-08-13 @ 16:09 
On Fri, Aug 12, 2011 at 08:37:00PM +0200, Mathijs Kwik wrote:
> What I'm talking about here is that flash, since 9.0.124.0, wants to
> read a policy file before doing any socket connection. In the past,
> connecting to the same server (non cross domain) didn't need a policy,
> but now it does.

Yes, and Mongrel2 does that for you, that's why the chat demo isn't
"handling" it at all.  Look at src/connection.c:501 for where it figures
this out, and then src/response.c:119 for what it returns.  It's not
hard to make that configurable.

> This policy file can either be read from /crossdomain.xml (just from
> http) in the root of the domain we connect to, or served from port 843
> of that machine.
> The port is preferred and checked first, multiple times. If that
> fails, /crossdomain.xml is tried.

And, this isn't right, unless you're saying they changed it in a
backward incompatible way.  It sends an <policy-file-request> xml tag
over the socket to get it back.

Can you show me docs and code that I can look at to try out what you're
doing?  That's easier than what you've got.

Also, if it's now hitting some special port, then why can't you just
change it?  jssocket doesn't do this, it just uses the port you tell it
to use.  Are you saying you are using something that insists on using
the same port and URLs for everything?  Then that library's busted.

Shoot me some demo code to check out and I'll see if it's a version
drift in how flash does sockets, but keep in mind I'm already doing this
with jssocket, so it'd be nice if you'd at least see if you can
reconfigure the ports.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-13 @ 18:07 
I played a bit with the chat demo. it works.
I used a sniffer to see what was going on.
I noticed the policy-file-request indeed got answered by mongrel2, but
the answer is repeated >100 times, making the entire conversation over
50k. This is probably a bug.

Furthermore: it doesn't work on SSL.
Flash asks for the policy but doesn't get an answer.
[ERROR] (src/io.c:137: errno: Resource temporarily unavailable)
Handshake failed with error code: -30976
[ERROR] (src/io.c:179: errno: None) SSL handshake failed.

I'm planning to use
https://github.com/gimite/web-socket-js/tree/hybi-07 which is a
flash-based websocket client for older browsers that don't support
native websockets (hybi 07) yet. It is able to do wss: (websocket
ssl), but only after it confirms the policy. So either mongrel2 should
be very smart and serve the policy inline (like now) before switching
SSL on, or the policy must be served from 843, which is always checked
first.


On Sat, Aug 13, 2011 at 6:09 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Fri, Aug 12, 2011 at 08:37:00PM +0200, Mathijs Kwik wrote:
>> What I'm talking about here is that flash, since 9.0.124.0, wants to
>> read a policy file before doing any socket connection. In the past,
>> connecting to the same server (non cross domain) didn't need a policy,
>> but now it does.
>
> Yes, and Mongrel2 does that for you, that's why the chat demo isn't
> "handling" it at all.  Look at src/connection.c:501 for where it figures
> this out, and then src/response.c:119 for what it returns.  It's not
> hard to make that configurable.
>
>> This policy file can either be read from /crossdomain.xml (just from
>> http) in the root of the domain we connect to, or served from port 843
>> of that machine.
>> The port is preferred and checked first, multiple times. If that
>> fails, /crossdomain.xml is tried.
>
> And, this isn't right, unless you're saying they changed it in a
> backward incompatible way.  It sends an <policy-file-request> xml tag
> over the socket to get it back.
>
> Can you show me docs and code that I can look at to try out what you're
> doing?  That's easier than what you've got.
>
> Also, if it's now hitting some special port, then why can't you just
> change it?  jssocket doesn't do this, it just uses the port you tell it
> to use.  Are you saying you are using something that insists on using
> the same port and URLs for everything?  Then that library's busted.
>
> Shoot me some demo code to check out and I'll see if it's a version
> drift in how flash does sockets, but keep in mind I'm already doing this
> with jssocket, so it'd be nice if you'd at least see if you can
> reconfigure the ports.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] flash socket / flash policy server

From:
Zed A. Shaw
Date:
2011-08-14 @ 18:18 
On Sat, Aug 13, 2011 at 08:07:54PM +0200, Mathijs Kwik wrote:
> I played a bit with the chat demo. it works.
> I used a sniffer to see what was going on.
> I noticed the policy-file-request indeed got answered by mongrel2, but
> the answer is repeated >100 times, making the entire conversation over
> 50k. This is probably a bug.

This is from the chat demo?  Alright I'll go see why it's doing that.

> Furthermore: it doesn't work on SSL.
> Flash asks for the policy but doesn't get an answer.
> [ERROR] (src/io.c:137: errno: Resource temporarily unavailable)
> Handshake failed with error code: -30976
> [ERROR] (src/io.c:179: errno: None) SSL handshake failed.

Cool I'll check that out too.

> I'm planning to use
> https://github.com/gimite/web-socket-js/tree/hybi-07 which is a
> flash-based websocket client for older browsers that don't support
> native websockets (hybi 07) yet. It is able to do wss: (websocket
> ssl), but only after it confirms the policy. So either mongrel2 should
> be very smart and serve the policy inline (like now) before switching
> SSL on, or the policy must be served from 843, which is always checked
> first.

There's another examples that does websockets against Mongrel2 in
examples/wschat.  Try that out and let me know how well it works.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-14 @ 18:26 
On Sun, Aug 14, 2011 at 8:18 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Sat, Aug 13, 2011 at 08:07:54PM +0200, Mathijs Kwik wrote:
>> I played a bit with the chat demo. it works.
>> I used a sniffer to see what was going on.
>> I noticed the policy-file-request indeed got answered by mongrel2, but
>> the answer is repeated >100 times, making the entire conversation over
>> 50k. This is probably a bug.
>
> This is from the chat demo?  Alright I'll go see why it's doing that.

Yep it is

>
>> Furthermore: it doesn't work on SSL.
>> Flash asks for the policy but doesn't get an answer.
>> [ERROR] (src/io.c:137: errno: Resource temporarily unavailable)
>> Handshake failed with error code: -30976
>> [ERROR] (src/io.c:179: errno: None) SSL handshake failed.
>
> Cool I'll check that out too.
>
>> I'm planning to use
>> https://github.com/gimite/web-socket-js/tree/hybi-07 which is a
>> flash-based websocket client for older browsers that don't support
>> native websockets (hybi 07) yet. It is able to do wss: (websocket
>> ssl), but only after it confirms the policy. So either mongrel2 should
>> be very smart and serve the policy inline (like now) before switching
>> SSL on, or the policy must be served from 843, which is always checked
>> first.
>
> There's another examples that does websockets against Mongrel2 in
> examples/wschat.  Try that out and let me know how well it works.

Works well, see my other messages.
However flash is an important target for me too, since hybi07 isn't
widely deployed yet, so there's a flash-based hybi07 websocket client
to fall back on.

>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] flash socket / flash policy server

From:
Zed A. Shaw
Date:
2011-08-14 @ 19:07 
On Sun, Aug 14, 2011 at 08:26:47PM +0200, Mathijs Kwik wrote:
> On Sun, Aug 14, 2011 at 8:18 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> > On Sat, Aug 13, 2011 at 08:07:54PM +0200, Mathijs Kwik wrote:
> >> I played a bit with the chat demo. it works.
> >> I used a sniffer to see what was going on.
> >> I noticed the policy-file-request indeed got answered by mongrel2, but
> >> the answer is repeated >100 times, making the entire conversation over
> >> 50k. This is probably a bug.

This is all fixed up now, and I may do some more work on it.  Right now
it sends the policy file and then closes the socket, which seems to be
what flash wants.  Let me know if that doesn't work.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-14 @ 19:42 
On Sun, Aug 14, 2011 at 9:07 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Sun, Aug 14, 2011 at 08:26:47PM +0200, Mathijs Kwik wrote:
>> On Sun, Aug 14, 2011 at 8:18 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
>> > On Sat, Aug 13, 2011 at 08:07:54PM +0200, Mathijs Kwik wrote:
>> >> I played a bit with the chat demo. it works.
>> >> I used a sniffer to see what was going on.
>> >> I noticed the policy-file-request indeed got answered by mongrel2, but
>> >> the answer is repeated >100 times, making the entire conversation over
>> >> 50k. This is probably a bug.
>
> This is all fixed up now, and I may do some more work on it.  Right now
> it sends the policy file and then closes the socket, which seems to be
> what flash wants.  Let me know if that doesn't work.

Nice, thanks, seems to work without 50k handshake :)
Now for flash there's just the ssl issue.
It's a bit nasty, since you will have to peek at new connections to
see if it's flash before handing the connection to polarssl.
But it would be great if that works too :)

Thanks for your quick responses



>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] flash socket / flash policy server

From:
Zed A. Shaw
Date:
2011-08-14 @ 20:27 
On Sun, Aug 14, 2011 at 09:42:51PM +0200, Mathijs Kwik wrote:
> Nice, thanks, seems to work without 50k handshake :)
> Now for flash there's just the ssl issue.
> It's a bit nasty, since you will have to peek at new connections to
> see if it's flash before handing the connection to polarssl.
> But it would be great if that works too :)

This is something I don't quite understand.  Is flash doing a socket
*upgrade* on the connection?  If so, that's idiotic.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-14 @ 20:38 
On Sun, Aug 14, 2011 at 10:27 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Sun, Aug 14, 2011 at 09:42:51PM +0200, Mathijs Kwik wrote:
>> Nice, thanks, seems to work without 50k handshake :)
>> Now for flash there's just the ssl issue.
>> It's a bit nasty, since you will have to peek at new connections to
>> see if it's flash before handing the connection to polarssl.
>> But it would be great if that works too :)
>
> This is something I don't quite understand.  Is flash doing a socket
> *upgrade* on the connection?  If so, that's idiotic.

Nope, it seems flash doesn't know about ssl (even not when I ask it to
connect to 443).
Flash is just doing what it's always doing.
Check 843 for policy -> not found -> make connection to port that was
asked for -> shout "<policy-file-request/>" -> wait for policy
response.
So flash itself needs the policy in plain (no ssl), however once the
connection is setup, a flash program can manually speak ssl.
The people at https://github.com/gimite/web-socket-js seem to have
implemented wss: (ssl) by including a full ssl library implementation
for flash.



>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] flash socket / flash policy server

From:
Zed A. Shaw
Date:
2011-08-15 @ 15:14 
On Sun, Aug 14, 2011 at 10:38:03PM +0200, Mathijs Kwik wrote:
> Nope, it seems flash doesn't know about ssl (even not when I ask it to
> connect to 443).
> Flash is just doing what it's always doing.
> Check 843 for policy -> not found -> make connection to port that was
> asked for -> shout "<policy-file-request/>" -> wait for policy
> response.

You could probably fake this out.  Run a "flash policy server" on port
843, which is just a bare Mongrel2 Server() with nothing in it, no SSL
on 843.  Then run your regular server on port 443 with SSL on.  Try that
and let me know if it works.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-15 @ 19:16 
> make connection to port that was

On Mon, Aug 15, 2011 at 5:14 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> On Sun, Aug 14, 2011 at 10:38:03PM +0200, Mathijs Kwik wrote:
>> Nope, it seems flash doesn't know about ssl (even not when I ask it to
>> connect to 443).
>> Flash is just doing what it's always doing.
>> Check 843 for policy -> not found -ound -
>> asked for -> shout "<policy-file-request/>" -> wait for policy
>> response.
>
> You could probably fake this out.  Run a "flash policy server" on port
> 843, which is just a bare Mongrel2 Server() with nothing in it, no SSL
> on 843.  Then run your regular server on port 443 with SSL on.  Try that
> and let me know if it works.

Wow, now we're back to my original question "how to run a tcp flash
policy server on 843" :P
But back then I didn't know about mongrel's handling and adobe's
reaction, so I learned quite a bit, which is always nice.

It seems to work fine!
At least, from sniffing the traffic I see flash talks to 843, gets an
answer, connects to 443 and doesn't throw the flash-policy-handshake
in.

However, after that, it fails to talk https, which I knew beforehand,
jssocket isn't ssl capable. I was just paving the way for
https://github.com/gimite/web-socket-js which is.
I'm not gonna try it now though, I'll wait a few days/(weeks) for them
to merge their hybi07 branch and update it to hybi10.

Anyway, thanks.
another item to strike through on my mongrel issues list :)


>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-14 @ 20:39 
On Sun, Aug 14, 2011 at 10:38 PM, Mathijs Kwik <bluescreen303@gmail.com> wrote:
> On Sun, Aug 14, 2011 at 10:27 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
>> On Sun, Aug 14, 2011 at 09:42:51PM +0200, Mathijs Kwik wrote:
>>> Nice, thanks, seems to work without 50k handshake :)
>>> Now for flash there's just the ssl issue.
>>> It's a bit nasty, since you will have to peek at new connections to
>>> see if it's flash before handing the connection to polarssl.
>>> But it would be great if that works too :)
>>
>> This is something I don't quite understand.  Is flash doing a socket
>> *upgrade* on the connection?  If so, that's idiotic.
>
> Nope, it seems flash doesn't know about ssl (even not when I ask it to
> connect to 443).
> Flash is just doing what it's always doing.
> Check 843 for policy -> not found -> make connection to port that was
> asked for -> shout "<policy-file-request/>" -> wait for policy
> response.
> So flash itself needs the policy in plain (no ssl), however once the
> connection is setup, a flash program can manually speak ssl.
> The people at https://github.com/gimite/web-socket-js seem to have
> implemented wss: (ssl) by including a full ssl library implementation
> for flash.

To clarify:
No upgrade, just answer policy in plain, close connection, now flash
will reconnect and speak proper ssl.

>
>
>
>>
>> --
>> Zed A. Shaw
>> http://zedshaw.com/
>>
>

Re: [mongrel2] flash socket / flash policy server

From:
Zed A. Shaw
Date:
2011-08-14 @ 18:29 
On Sun, Aug 14, 2011 at 08:26:47PM +0200, Mathijs Kwik wrote:
> On Sun, Aug 14, 2011 at 8:18 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> > On Sat, Aug 13, 2011 at 08:07:54PM +0200, Mathijs Kwik wrote:
> >> I played a bit with the chat demo. it works.
> >> I used a sniffer to see what was going on.
> >> I noticed the policy-file-request indeed got answered by mongrel2, but
> >> the answer is repeated >100 times, making the entire conversation over
> >> 50k. This is probably a bug.
> >
> > This is from the chat demo?  Alright I'll go see why it's doing that.
> 
> Yep it is

What the hell? I just ran it with some logging and it's getting tons of
these requests.  I'll fix this up right now.  No idea how the hell that
started happening.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] flash socket / flash policy server

From:
Nathan Duran
Date:
2011-08-12 @ 15:57 
On Aug 12, 2011, at 6:08 AM, Mathijs Kwik wrote:

> It would be nice if this could be implemented somehow, so flashsockets
> will work cross-domain without a few seconds delay.

That sounds a little silly. Do any other web servers support this 
proprietary Adobe protocol?

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-12 @ 16:07 
Running socket.io on Node.js will serve such a policy file from tcp port 843.
I'm not aware of any others though.

As stated, it's not really a webserver task (not http, just plain
tcp), but I think if you support something like flash sockets (which
mongrel2 mentions to support), this is part of it.

I agree it's a stupid protocol, and I absolutely hate flash, but - as
mongrel2's manual states as well - for now, it does its job for cross
domain connections and websocket-like communications.
Not supporting this part, leads to slower connections.

So whether we like the protocol or not, it's at least advisable for
administrators who don't want to support this, to block port 843,
instead of dropping, so flash clients will fall back to
/crossdomain.xml sooner and not awaiting a timeout.



On Fri, Aug 12, 2011 at 5:57 PM, Nathan Duran <principal@khiltd.com> wrote:
>
> On Aug 12, 2011, at 6:08 AM, Mathijs Kwik wrote:
>
>> It would be nice if this could be implemented somehow, so flashsockets
>> will work cross-domain without a few seconds delay.
>
> That sounds a little silly. Do any other web servers support this 
proprietary Adobe protocol?
>

Re: [mongrel2] flash socket / flash policy server

From:
Nathan Duran
Date:
2011-08-12 @ 17:03 
On Aug 12, 2011, at 9:07 AM, Mathijs Kwik wrote:

> I agree it's a stupid protocol, and I absolutely hate flash, but - as
> mongrel2's manual states as well - for now, it does its job for cross
> domain connections and websocket-like communications.
> Not supporting this part, leads to slower connections.

I don't think there's any argument to be made that a webserver should 
support this part. Every application is going to have needs which it needs
to address on its own.

Re: [mongrel2] flash socket / flash policy server

From:
Mathijs Kwik
Date:
2011-08-12 @ 18:17 
I agree!

It's not part of a normal webserver, but since mongrel2 is already a
bit more than just plain http (I remember a telnet example?), it would
be great if we could easily create a plain tcp handler here so the
application can handle this by itself.

On Fri, Aug 12, 2011 at 7:03 PM, Nathan Duran <principal@khiltd.com> wrote:
>
> On Aug 12, 2011, at 9:07 AM, Mathijs Kwik wrote:
>
>> I agree it's a stupid protocol, and I absolutely hate flash, but - as
>> mongrel2's manual states as well - for now, it does its job for cross
>> domain connections and websocket-like communications.
>> Not supporting this part, leads to slower connections.
>
> I don't think there's any argument to be made that a webserver should 
support this part. Every application is going to have needs which it needs
to address on its own.
>

Archives based on code by Armin Ronacher //