This message is just to start the discussion on Continuous Integration for GeoNode. Django has recently been working on that and here is the library based on buildout that came out: https://github.com/jacobian/django-buildmaster/ There is also an ongoing series of blog posts about it that look very helpful: http://jacobian.org/writing/buildbot/ Ariel.
Are you planning to set up continuous integration? I think having a decent set of automated integration tests and a CI server would be a great help to the project. We at OpenGeo are interested in setting that up but currently not devoting a lot of time to it, though we should have some time and hardware to invest in the problem before the end of the year. That said, I would tend toward using Hudson for continuous integration (OpenGeo already maintains several Hudson services and they are working well for us.) What's special about this new project? (Does it integrate with Django particularly well, for example?) -- David Winslow OpenGeo - http://opengeo.org/ On Thu, Nov 11, 2010 at 3:20 PM, Ariel Nunez <ingenieroariel@gmail.com>wrote: > This message is just to start the discussion on Continuous Integration > for GeoNode. > > Django has recently been working on that and here is the library based > on buildout that came out: > > https://github.com/jacobian/django-buildmaster/ > > There is also an ongoing series of blog posts about it that look very > helpful: > http://jacobian.org/writing/buildbot/ > > Ariel. >
I'm using Hudson for my geonode fork, I'd be happy to share my setup with you if you're interested. Basically it downloads the latest code, runs 'paver package_all', dumps everything into it's proper place and restarts apache. http://ec2-184-73-249-247.compute-1.amazonaws.com:8010/job/WorldMap/ -Matt On Nov 11, 2010, at 5:11 PM, David Winslow wrote: > Are you planning to set up continuous integration? I think having a > decent set of automated integration tests and a CI server would be a > great help to the project. We at OpenGeo are interested in setting > that up but currently not devoting a lot of time to it, though we > should have some time and hardware to invest in the problem before > the end of the year. > > That said, I would tend toward using Hudson for continuous > integration (OpenGeo already maintains several Hudson services and > they are working well for us.) What's special about this new > project? (Does it integrate with Django particularly well, for > example?) > > -- > David Winslow > OpenGeo - http://opengeo.org/ > > On Thu, Nov 11, 2010 at 3:20 PM, Ariel Nunez > <ingenieroariel@gmail.com> wrote: > This message is just to start the discussion on Continuous Integration > for GeoNode. > > Django has recently been working on that and here is the library based > on buildout that came out: > > https://github.com/jacobian/django-buildmaster/ > > There is also an ongoing series of blog posts about it that look > very helpful: > http://jacobian.org/writing/buildbot/ > > Ariel. >
The first thing I had to do was install the git plugin for Hudson. I've also got some production-specific settings files in another directory that I copy over (using the script below). This is the script I've got Hudson running after pulling the latest code: ----------------------------------- #!/bin/bash #Get submodule code git submodule update --init #Set up environment python bootstrap.py --no-site-packages source bin/activate #copy production settings over cp /home/ubuntu/cga-worldmap-settings/geonode/settings.py src/ GeoNodePy/geonode/settings.py #cp /home/ubuntu/cga-worldmap-settings/geoserver/web.xml.prod src/ geoserver-geonode-ext/src/main/webapp/WEB-INF/web.xml #build everything paver package_all deactivate #copy WAR files to tomcat and overwrite settings files with production versions #sudo mv shared/package*.war /var/lib/tomcat6/webapps #move new javascript over sudo unzip -o shared/package/geonode-client.zip -d /var/www/worldmap/ htdocs #move new django code over sudo cp -r shared/package/* /var/www/worldmap/wsgi/worldmap ### set up new production code cd /var/www/worldmap/wsgi/worldmap #remove old code sudo rm -rf src build #install new code sudo python bootstrap.py #change file permissions to apache sudo chown -R www-data:www-data /var/www/worldmap #copy custom geoserver/geonwetwork config files #sudo cp /home/ubuntu/cga-worldmap-settings/geonetwork/config.xml /var/ lib/tomcat6/webapps/geonetwork/WEB-INF #restart apache & tomcat #sudo /etc/init.d/tomcat6 restart sudo /etc/init.d/apache2 restart --------------------------------------- I commented out the lines for geoserver/geonetwork/tomcat because I've got them running on another server right now. -Matt On Nov 16, 2010, at 10:34 AM, Matt Bertrand wrote: > I'm using Hudson for my geonode fork, I'd be happy to share my setup > with you if you're interested. Basically it downloads the latest > code, runs 'paver package_all', dumps everything into it's proper > place and restarts apache. > > http://ec2-184-73-249-247.compute-1.amazonaws.com:8010/job/WorldMap/ > > -Matt > > > On Nov 11, 2010, at 5:11 PM, David Winslow wrote: > >> Are you planning to set up continuous integration? I think having >> a decent set of automated integration tests and a CI server would >> be a great help to the project. We at OpenGeo are interested in >> setting that up but currently not devoting a lot of time to it, >> though we should have some time and hardware to invest in the >> problem before the end of the year. >> >> That said, I would tend toward using Hudson for continuous >> integration (OpenGeo already maintains several Hudson services and >> they are working well for us.) What's special about this new >> project? (Does it integrate with Django particularly well, for >> example?) >> >> -- >> David Winslow >> OpenGeo - http://opengeo.org/ >> >> On Thu, Nov 11, 2010 at 3:20 PM, Ariel Nunez <ingenieroariel@gmail.com >> > wrote: >> This message is just to start the discussion on Continuous >> Integration >> for GeoNode. >> >> Django has recently been working on that and here is the library >> based >> on buildout that ca me out: >> >> https://github.com/jacobian/django-buildmaster/ >> >> There is also an ongoing series of blog posts about it that look >> very helpful: >> http://jacobian.org/writing/buildbot/ >> >> Ariel. >> >
On Tue, Nov 16, 2010 at 9:34 AM, Matt Bertrand <mbertrand@cga.harvard.edu> wrote: > I'm using Hudson for my geonode fork, I'd be happy to share my setup with > you if you're interested. Basically it downloads the latest code, runs > 'paver package_all', dumps everything into it's proper place and restarts > apache. > http://ec2-184-73-249-247.compute-1.amazonaws.com:8010/job/WorldMap/ > -Matt I think that is a very good first step, and given David is already into Hudson too I would be happy to go with the flow. Can you share with us the setup you are using? David can we put something like that in an OpenGeo machine(at least the main instance)? After that I would be interested in setting up Amazon Virtual Machines that fire up and test different branches (master, synth) against different configurations (latest ubuntu release, latest LTS ubuntu release, latest CentOS stable, 32-64 bit) and pushing the results back to the main hudson service. Ariel.
This is great. Some open questions about who owns the hardware: earlier we (OpenGeo) were talking with GEM about using some of their hardware for continuous integration and later scalability testing. Ariel, it sounds like you're saying that the WB can take responsibility for funding the Amazon instances? If we could somehow make sure the hardware was reliably available and also accessible to committers across organizational lines, that would be ideal. On Tue, Nov 16, 2010 at 10:52 AM, Ariel Nunez <ingenieroariel@gmail.com>wrote: > On Tue, Nov 16, 2010 at 9:34 AM, Matt Bertrand > <mbertrand@cga.harvard.edu> wrote: > > I'm using Hudson for my geonode fork, I'd be happy to share my setup with > > you if you're interested. Basically it downloads the latest code, runs > > 'paver package_all', dumps everything into it's proper place and restarts > > apache. > > http://ec2-184-73-249-247.compute-1.amazonaws.com:8010/job/WorldMap/ > > -Matt > > I think that is a very good first step, and given David is already > into Hudson too I would be happy to go with the flow. > > Can you share with us the setup you are using? David can we put > something like that in an OpenGeo machine(at least the main instance)? > After that I would be interested in setting up Amazon Virtual Machines > that fire up and test different branches (master, synth) against > different configurations (latest ubuntu release, latest LTS ubuntu > release, latest CentOS stable, 32-64 bit) and pushing the results back > to the main hudson service. > > Ariel. > -- Sebastian Benthall OpenGeo - http://opengeo.org
Setting up continuous integration would be a great step forward in easing the deployment of Geonode. We (AIFDR) would welcome this as we have not been able to get it installed yet. We could contribute as early testers of this system. Thanks Ariel for starting this thread. Ole On Fri, Nov 12, 2010 at 5:11 AM, David Winslow <dwinslow@opengeo.org> wrote: > Are you planning to set up continuous integration? I think having a decent > set of automated integration tests and a CI server would be a great help to > the project. We at OpenGeo are interested in setting that up but currently > not devoting a lot of time to it, though we should have some time and > hardware to invest in the problem before the end of the year. > > That said, I would tend toward using Hudson for continuous integration > (OpenGeo already maintains several Hudson services and they are working well > for us.) What's special about this new project? (Does it integrate with > Django particularly well, for example?) > > -- > David Winslow > OpenGeo - http://opengeo.org/ > > > On Thu, Nov 11, 2010 at 3:20 PM, Ariel Nunez <ingenieroariel@gmail.com>wrote: > >> This message is just to start the discussion on Continuous Integration >> for GeoNode. >> >> Django has recently been working on that and here is the library based >> on buildout that came out: >> >> https://github.com/jacobian/django-buildmaster/ >> >> There is also an ongoing series of blog posts about it that look very >> helpful: >> http://jacobian.org/writing/buildbot/ >> >> Ariel. >> > >
> > This message is just to start the discussion on Continuous Integration > for GeoNode. > > Django has recently been working on that and here is the library based > on buildout that came out: > Can you explain more about what you mean? Is this a system that is used to do CI on the Django software itself, or on Django sites, or Django apps? What's the relevance of this particular solution? -- Sebastian Benthall OpenGeo - http://opengeo.org