Hey all, I've spent some time today setting up a Jenkins[1] server at http://geonode-testing.dev.opengeo.org:8080/ . Right now, there are three test suites prepared to run: *geonode-client* runs the minification script (we don't have any unit tests in this project IIRC so they aren't run.) *geonode-server* runs a full build "from scratch" (some system-level dependencies are preserved across builds) and runs the Django and GeoServer-GeoNode-extensions unit tests. *geonode-integration* is intended to run some integration tests against a newly installed GeoNode site with a clean database. Since we don't have any right now it just does a clean deploy with no assertions at all, I'll try and work on this this week (look forward to that geonode-integration repo I mentioned last week, coming soon :) geonode-client is set to re-run automatically after pushes to github (not the GeoNode/geonode-client project yet though, see below). geonode-server runs when either geonode-client has successfully built or when someone pushes to git://github.com/GeoNode/geonode master. Once we have the integration suite online I'll have it watch that as well. Also on the TODO list are email notifications about failed builds (I think these should be sent to geonode-dev@opengeo.org, any objections?) and running tests for the synth branches of the various projects. One issue I ran into was that the geonode-client project has some submodules that point at authenticated github URLs. I think we could create a github user account or a deploy key[2] for the deploy server to make it able to access these - but it would probably be nicer to others trying to do automated builds of GeoNode if we instead change them out for the anonymous ones. Developers who need the push access could easily modify the .git/config for individual repositories so it wouldn't be a big burden for them. I went ahead and pushed a new branch to my fork showing what this would look like: https://github.com/dwins/geonode-client/compare/master...testing-master So on that topic, does anyone have a favorite scriptable browser emulator? I've recently been pointed toward http://www.phantomjs.org/ which has the advantage of being based on the real engine behind one of the current big three browsers, but I know there are others out there (perhaps more mature, I think phantom is fairly new.) [1] http://jenkins-ci.org/ [2] http://help.github.com/deploy-keys/ -- David Winslow OpenGeo - http://opengeo.org/
Hey David, another question related to geonode-client: currently, I build the geonode-client artifacts (zip and war) manually after making changes and upload them to dev.geonode.org, where the installation script pulls them. Could this process be automated as part of the re-run that occurs when changes are pushed to github? Andreas. On Apr 12, 2011, at 00:01 , David Winslow wrote: > Hey all, > > I've spent some time today setting up a Jenkins[1] server at http://geonode-testing.dev.opengeo.org:8080/ . Right now, there are three test suites prepared to run: > > geonode-client runs the minification script (we don't have any unit tests in this project IIRC so they aren't run.) > geonode-server runs a full build "from scratch" (some system-level dependencies are preserved across builds) and runs the Django and GeoServer-GeoNode-extensions unit tests. > geonode-integration is intended to run some integration tests against a newly installed GeoNode site with a clean database. Since we don't have any right now it just does a clean deploy with no assertions at all, I'll try and work on this this week (look forward to that geonode-integration repo I mentioned last week, coming soon :) > > geonode-client is set to re-run automatically after pushes to github (not the GeoNode/geonode-client project yet though, see below). geonode-server runs when either geonode-client has successfully built or when someone pushes to git://github.com/GeoNode/geonode master. Once we have the integration suite online I'll have it watch that as well. Also on the TODO list are email notifications about failed builds (I think these should be sent to geonode-dev@opengeo.org, any objections?) and running tests for the synth branches of the various projects. > > One issue I ran into was that the geonode-client project has some submodules that point at authenticated github URLs. I think we could create a github user account or a deploy key[2] for the deploy server to make it able to access these - but it would probably be nicer to others trying to do automated builds of GeoNode if we instead change them out for the anonymous ones. Developers who need the push access could easily modify the .git/config for individual repositories so it wouldn't be a big burden for them. I went ahead and pushed a new branch to my fork showing what this would look like: > https://github.com/dwins/geonode-client/compare/master...testing-master > > So on that topic, does anyone have a favorite scriptable browser emulator? I've recently been pointed toward http://www.phantomjs.org/ which has the advantage of being based on the real engine behind one of the current big three browsers, but I know there are others out there (perhaps more mature, I think phantom is fairly new.) > > [1] http://jenkins-ci.org/ > [2] http://help.github.com/deploy-keys/ > > -- > David Winslow > OpenGeo - http://opengeo.org/ -- Andreas Hocevar OpenGeo - http://opengeo.org/ Expert service straight from the developers.
Yes, we can arrange something along those lines. I'm not sure about the details just yet but I will look into it this week. Whatever we do, it should require nothing more than a git push from your end. -- David Winslow OpenGeo - http://opengeo.org/ On Tue, Apr 12, 2011 at 4:46 AM, Andreas Hocevar <ahocevar@opengeo.org>wrote: > Hey David, > > another question related to geonode-client: > > currently, I build the geonode-client artifacts (zip and war) manually > after making changes and upload them to dev.geonode.org, where the > installation script pulls them. Could this process be automated as part of > the re-run that occurs when changes are pushed to github? > > Andreas. > > On Apr 12, 2011, at 00:01 , David Winslow wrote: > > > Hey all, > > > > I've spent some time today setting up a Jenkins[1] server at > http://geonode-testing.dev.opengeo.org:8080/ . Right now, there are three > test suites prepared to run: > > > > geonode-client runs the minification script (we don't have any unit tests > in this project IIRC so they aren't run.) > > geonode-server runs a full build "from scratch" (some system-level > dependencies are preserved across builds) and runs the Django and > GeoServer-GeoNode-extensions unit tests. > > geonode-integration is intended to run some integration tests against a > newly installed GeoNode site with a clean database. Since we don't have any > right now it just does a clean deploy with no assertions at all, I'll try > and work on this this week (look forward to that geonode-integration repo I > mentioned last week, coming soon :) > > > > geonode-client is set to re-run automatically after pushes to github (not > the GeoNode/geonode-client project yet though, see below). geonode-server > runs when either geonode-client has successfully built or when someone > pushes to git://github.com/GeoNode/geonode master. Once we have the > integration suite online I'll have it watch that as well. Also on the TODO > list are email notifications about failed builds (I think these should be > sent to geonode-dev@opengeo.org, any objections?) and running tests for > the synth branches of the various projects. > > > > One issue I ran into was that the geonode-client project has some > submodules that point at authenticated github URLs. I think we could create > a github user account or a deploy key[2] for the deploy server to make it > able to access these - but it would probably be nicer to others trying to do > automated builds of GeoNode if we instead change them out for the anonymous > ones. Developers who need the push access could easily modify the > .git/config for individual repositories so it wouldn't be a big burden for > them. I went ahead and pushed a new branch to my fork showing what this > would look like: > > https://github.com/dwins/geonode-client/compare/master...testing-master > > > > So on that topic, does anyone have a favorite scriptable browser > emulator? I've recently been pointed toward http://www.phantomjs.org/which has the advantage of being based on the real engine behind one of the > current big three browsers, but I know there are others out there (perhaps > more mature, I think phantom is fairly new.) > > > > [1] http://jenkins-ci.org/ > > [2] http://help.github.com/deploy-keys/ > > > > -- > > David Winslow > > OpenGeo - http://opengeo.org/ > > -- > Andreas Hocevar > OpenGeo - http://opengeo.org/ > Expert service straight from the developers. > >
Thanks for setting this up! See some comments inline. On Apr 12, 2011, at 00:01 , David Winslow wrote: > geonode-client runs the minification script (we don't have any unit tests in this project IIRC so they aren't run.) Right. But we have increasing test coverage in gxp, and a decreasing amount of custom code in geonode-client, so we're improving here continuously. > One issue I ran into was that the geonode-client project has some submodules that point at authenticated github URLs. I think we could create a github user account or a deploy key[2] for the deploy server to make it able to access these - but it would probably be nicer to others trying to do automated builds of GeoNode if we instead change them out for the anonymous ones. Developers who need the push access could easily modify the .git/config for individual repositories so it wouldn't be a big burden for them. I went ahead and pushed a new branch to my fork showing what this would look like: > https://github.com/dwins/geonode-client/compare/master...testing-master Thanks David, I've cherry-picked this changeset into master and synth. In fact, using the authenticated URLs was unintentional. > So on that topic, does anyone have a favorite scriptable browser emulator? I've recently been pointed toward http://www.phantomjs.org/ which has the advantage of being based on the real engine behind one of the current big three browsers, but I know there are others out there (perhaps more mature, I think phantom is fairly new.) I would like to give phantomjs a try at least, unless others come up with a pointer to the perfect headless js testsuite. Andreas. -- Andreas Hocevar OpenGeo - http://opengeo.org/ Expert service straight from the developers.
So I spent a little time yesterday after writing this mail looking into a few different browser automation and emulation tools - many of them don't even have API methods for setting the value of file fields in forms, and of those that do it usually doesn't work except in some specific, tricky setup (firefox with a custom plugin and security policies disabled etc.) PhantomJS lists file uploads as something to support in a future release[1] so we won't be able to use it for this now. So I think we should just bypass web browsers and use something like Mechanize[2] or Twill[3] for the tests specifically relating to file upload, and at such time as we want to start testing the custom application (which doesn't seem to be a priority this week) we can start on a PhantomJS-based suite as it seems like one of the more robust headless browser options out there. Today, though, I'll be driving GeoNetwork with OWSLib and httplib2 to simulate our usage of it in GeoNode - hopefully I can sort out the issues there. [1] http://code.google.com/p/phantomjs/wiki/Roadmap [2] http://twill.idyll.org/ [3] http://wwwsearch.sourceforge.net/mechanize/ -- David Winslow OpenGeo - http://opengeo.org/ On Tue, Apr 12, 2011 at 3:25 AM, Andreas Hocevar <ahocevar@opengeo.org>wrote: > Thanks for setting this up! See some comments inline. > > On Apr 12, 2011, at 00:01 , David Winslow wrote: > > > geonode-client runs the minification script (we don't have any unit tests > in this project IIRC so they aren't run.) > > Right. But we have increasing test coverage in gxp, and a decreasing amount > of custom code in geonode-client, so we're improving here continuously. > > > One issue I ran into was that the geonode-client project has some > submodules that point at authenticated github URLs. I think we could create > a github user account or a deploy key[2] for the deploy server to make it > able to access these - but it would probably be nicer to others trying to do > automated builds of GeoNode if we instead change them out for the anonymous > ones. Developers who need the push access could easily modify the > .git/config for individual repositories so it wouldn't be a big burden for > them. I went ahead and pushed a new branch to my fork showing what this > would look like: > > https://github.com/dwins/geonode-client/compare/master...testing-master > > Thanks David, I've cherry-picked this changeset into master and synth. In > fact, using the authenticated URLs was unintentional. > > > So on that topic, does anyone have a favorite scriptable browser > emulator? I've recently been pointed toward http://www.phantomjs.org/which has the advantage of being based on the real engine behind one of the > current big three browsers, but I know there are others out there (perhaps > more mature, I think phantom is fairly new.) > > I would like to give phantomjs a try at least, unless others come up with a > pointer to the perfect headless js testsuite. > > Andreas. > > -- > Andreas Hocevar > OpenGeo - http://opengeo.org/ > Expert service straight from the developers. > >