librelist archives

Re: [usenode] Re: what to do in litmus when async tests time out

From:

Richard Hodgson

Date:

2011-10-26 @ 08:18

Looks good to me, I've warmed to the new optional params Api. 

Think we should merge some of these branches back master :-)

Richard

On 24 Oct 2011, at 23:27, Tom Yandell <tom@yandell.me.uk> wrote:

> This is done in the aysnc-changes branch:
> 
> https://github.com/usenode/litmus.js/compare/master...async-changes
> 
> Thoughts?
> 
> Tom
> 
> On 22 October 2011 22:47, Tom Yandell <tom@yandell.me.uk> wrote:
> Definitely milliseconds - don't know what I was thinking. Not sure about
the api bit, but I don't like where it is at the moment (after the 
function). Maybe we should establish a convention? Perhaps: positional 
parameters for stuff that's always passed, followed by options object 
(where applicable) and then callback function (again, where applicable). 
This would then be:
> 
>     this.async('message', { timeout: 10000 }, function (done) {
>         // ...
>         done.resolve();
>     });
> 
> Thoughts?
> 
> Tom
> 
> 
> On 22 October 2011 15:47, Richard Hodgson <rightaboutnow@gmail.com> wrote:
> Something that I find annoying in other js test lib's is asynchronous 
tests not timing out at all. 
> 
> Could there be a default timeout that can be overridden per test 
(litmus.Test that is)?
> 
> It would still be nice to pass a timeout to async()... dare I suggest 
jQuery style variable argument positions?
> 
> The following are both supported:
> 
>     this.async('message', function () {...});
> 
>     this.async('message', 4, function () {...});
> 
> Not especially keen on it, however can't think of anything better.
> 
> Also, should timeouts be in seconds or milleseconds?
> 
> Richard
> 
> On 19 Oct 2011, at 22:24, Tom Yandell <tom@yandell.me.uk> wrote:
> 
> Also, the interface for async timeouts sucks right now. Async tests look like:
> 
> test.async('description', function (handle) {
>     // ...
>     handle.finish();
> });
> 
> The timeout is an optional third parameter specified in seconds, which 
gets lost as you put more code in the callback function :-( I'm also 
wondering whether the handle should just be a promise that you resolve 
when you've finished?
> 
> Tom
> 
> On 19 October 2011 22:15, Tom Yandell <tom@yandell.me.uk> wrote:
> Litmus currently has a pretty short (four second) default for async 
operations in tests. When it times out and the assertions are subsequently
encountered (as happened to me today) I get a message like:
> ...
> [ PASS ] second created time same as initial
> [ PASS ] all second requests handled by webapp created at same time
> [ ERROR ] async operation "reload" timed out after 4 seconds
> Summary
> =======
> Result:
>     FAIL
> 
> 
/Users/thomasyandell/node-projects/proton.js/node_modules/promised-io/promise.js:221
> 					throw error;
>            ^
> Error: assertion (test.is(..., "third set of responses received")) added
to test run after it was finished
>     at [object Object]._checkRunning 
(/Users/thomasyandell/node-projects/litmus.js/lib/litmus.js:248:15)
> ...
> 
> It's tricky because node or the browser will continue to wait until 
there are no outstanding events pending before exiting, unless you call 
process.exit. My current thinking is that 
TestRun.prototype.start/SuiteRun.prototype.start should return a promise 
that is resolved when the timeout occurs and the command-line runner 
should exit the process after it has printed. This doesn't help much for 
the browser - I guess it should ignore assertion results that get added 
after the run has completed following aync timeouts. I'll add a bug for 
that unless you can think of a clever way round the problem.
> 
> Tom
> 
> 
> 
>