Hi, Under the Oil for Development program, NORAD is helping the Ugandan Environmental sector to monitor the changes in the environment that are related to the recent findings of oil in some of the most sensitive areas of Uganda. Last week there was a training with GIS profesionals from the Forest Authority, Ministry of Water, Environment Authority etc. About 20 people participated. Ragnvald Larsen from NORAD, Ketty Adoch and myself from Mountbatten shared our Geonode knowledge. Instead of the recommended server with 6GB of memory, we opted for an atom based portable media center PC with 2GB of memory. http://www.advance.no/Item.aspx?id=4901512 We installed the current geonode release from the apt-packages. We connected everyone to a local LAN with a wireless router and started. The router was connected to a (very slow, maybe 256k) internet connection. With this setup, we experienced the following issues. We are aware that there is a brandnew version soon to be released, and that some of the issues mentioned could very well have been solved already. Still, for people who would like to train in similar conditions, this might be worthwhile: - Not suprisingly, we had performance issues. We had most performance issues on the first day. Especially in the morning, when we althogether uploaded shapefiles. Processortime on the server maxed out at 400% and kept there for a long long time. A second factor might have been the slow internet connection and the overload of timed out http sessions to the Internet and the Geoserver that used all resources on the Wireless Router. The router didnt have any statistics, so this was not so straightforward to diagnose. (tip: use WRT on your wireless router) After lunch, when we started on creating maps and styling them, the system became much more responsive. - During the styling session people created their own maps, and started styling layers. During this exercise at some point, layers that where shared between maps got corrupted. People might have edited eachothers styles (they looked very similar), but what surely happened was that 2 layers lost their default SLD in geoserver. This renders some pages in geonode useless and causes all sorts of errors. Going into geoserver and assigning a (pretty random) default style to the layer fixes this. - On other geonode installations we had already seen that sometimes when an upload fails, a geonode layer might be in the geonode database, but is not present in geoserver (the whole table is missing). I think Jude is working on some code to straighten this out when it happens. Would be great if an admin/user could do this from the geonode gui. - On the second day, we brought a second server. It ran fine for about 30 minutes, then there was a powersurge (yeah, those happen in Uganda.) After the power came back, tomcat would not finish starting. The attached catalina.out sample shows the full startup sequence. The server never finishes indexing. Could it be that: -- the indexing is not 100% necessary during training? -- the indexing is causing the very high load during uploading data sessions? If so, is there a way to switch off the indexing? That would save a lot of trainers in locations where you can not use "the cloud" during training sessions a lot of headaches of running around with server-size hardware. Enough about problems. At the end of the second day, we workshopped around uses of geonode, and one of the questions asked was: How could geonode be improved? The top 5 was: 1. More data analysis tools 2. display attribute table 3. enable layer grouping 4. connection to social sites like facebook and linkedIn 5. minimise error bugs. And in random order the rest of the list was: Improve tools for styling more admin control in user editing application compatibility include a timeline for map edits include print layout ability to download backdrop for a region(facilitates for offline) use standard GIS terms make it easier to make groups help feature for frequently asked questions having the legend directly on the map. These are the raw items, they also tell something about the training we did. We did not cover the printing part, some people might have missed the current legend, we later showed some advanced users how to load peoples own styled WMS layers in ArcGIS & QGIS (to the exitement of the trainees). And, while at it, maybe i could at 2 of my personal favorites (Withouth knowledge of 1.2): - Import Geoserver layers into Geonode. This is especially useful for very big geotiffs and for remote WFS layers. - Slightly more userfriendly errormessages would be great. I know Tomcat spits out lists and lists of errors, but having a slight clue of where in the process things broke helps diagnose issues a lot. In general, we love Geonode. Its very relevant in the Ugandan context. A lot of the discussion here is still about the actual sharing of data. There is a great level of reluctance. Using Geonode to show how easy it can become to mix & match data makes the advantage of sharing your data a lot more concrete for people. Commercial drilling for oil is not expected to start before 2017. until then, Reinier Battenberg
Thanks for the super awesome feedback - it's incredibly helpful to reflect this back to the project. So many people just use the software and don't realize that a write up like this can greatly influence things. Often it's just reiterating things we already know, but it still helps greatly in prioritizing what to work on, as there's always a long list of things to do. And indeed there's a whole lot of work that's been done on branches, that isn't yet back in to the GeoNode core, so seeing the pain points does help in deciding which of those to bring in first. Some responses inline: On Wed, Feb 1, 2012 at 8:20 AM, Reinier Battenberg < reinier.battenberg@mountbatten.net> wrote: > Hi, > > > ... > With this setup, we experienced the following issues. We are aware that > there > is a brandnew version soon to be released, and that some of the issues > mentioned could very well have been solved already. Still, for people who > would like to train in similar conditions, this might be worthwhile: > > - Not suprisingly, we had performance issues. We had most performance > issues > on the first day. Especially in the morning, when we althogether uploaded > shapefiles. Processortime on the server maxed out at 400% and kept there > for a > long long time. A second factor might have been the slow internet > connection > and the overload of timed out http sessions to the Internet and the > Geoserver > that used all resources on the Wireless Router. The router didnt have any > statistics, so this was not so straightforward to diagnose. (tip: use WRT > on > your wireless router) > After lunch, when we started on creating maps and styling them, the system > became much more responsive. > > In time I would like to see us do more testing and improvement on performance stuff, so we could get GeoNode running in more constrained environments. First focus has just been on getting the functionality working, but it does sound like some profiling on resource constrained machines might lead us to find some places for improvement. > - During the styling session people created their own maps, and started > styling layers. During this exercise at some point, layers that where > shared > between maps got corrupted. People might have edited eachothers styles > (they > looked very similar), but what surely happened was that 2 layers lost their > default SLD in geoserver. This renders some pages in geonode useless and > causes all sorts of errors. Going into geoserver and assigning a (pretty > random) default style to the layer fixes this. > > Are you able to reproduce this bug? Sounds like a serious one, and if you could find a way to do it consistently then the developers may have a decent chance of fixing it. I think the ownership / permissioning around styling may not work great, and may take a deep solution, but it'd be good if we can reproduce the common case, and maybe get a quick workaround that doesn't lead to things breaking. > - On other geonode installations we had already seen that sometimes when an > upload fails, a geonode layer might be in the geonode database, but is not > present in geoserver (the whole table is missing). I think Jude is working > on > some code to straighten this out when it happens. Would be great if an > admin/user could do this from the geonode gui. > > Cool, if Jude doesn't have this worked out then it could be good to file a bug to keep track of it. > - On the second day, we brought a second server. It ran fine for about 30 > minutes, then there was a powersurge (yeah, those happen in Uganda.) After > the > power came back, tomcat would not finish starting. The attached > catalina.out > sample shows the full startup sequence. The server never finishes indexing. > Could it be that: > -- the indexing is not 100% necessary during training? > -- the indexing is causing the very high load during uploading data > sessions? > If so, is there a way to switch off the indexing? That would save a lot of > trainers in locations where you can not use "the cloud" during training > sessions a lot of headaches of running around with server-size hardware. > > What indexing are you talking about? Reading that log I'm not sure I see the line you're talking about that indicates the server not finishing indexing. The line that concerns me is the 'Couldn't create user preferences directory. User preferences are unusable.' Right after the epsg creation I think means it might have failed to create the epsg database. > Enough about problems. At the end of the second day, we workshopped around > uses of geonode, and one of the questions asked was: How could geonode be > improved? > > The top 5 was: > > 1. More data analysis tools > Any chance you could flesh this out? What types of data analysis tools? And do you want these to output in to new layers, or to just do render things differently on the current layer? Do they need a GUI to do this data analysis? Our first data analysis step, at least in GeoServer, is to add WPS and GeoScript, though that will be pretty programmer heavy. If there are some subset of operations that you see being important for some basic analysis that'd be good to know. > 2. display attribute table > How are you imagining this working? Like in a grid? Try the 'query' functionality on http://sfpark.demo.opengeo.org/SFPark/ and let me know if that comes close to what you're wanting. I've also been thinking of making a feature grid available by default on each layer's Data page. So people could see not just the data on a map, but also look at what attributes are there. > 3. enable layer grouping > You mean like on the layer tree? To be able to select a few layers and then sort of squash them in to one layer, which then makes just one WMS request? Do you want that layer group persisted on the server and available for others to request through the WMS? Or just on a map level, where it'd just make a multi layer WMS request. > 4. connection to social sites like facebook and linkedIn > This (and a whole lot more) is on the social features branch that is planned for 1.2. And they're in the ANZSM, see the prototype at http://spatialmarketplace.net.au/maps/13 You can like or +1. No linkedin yet - I agree that in many ways that's the social platform that makes the most sense, but I think they don't have easy to use API's. > 5. minimise error bugs. > > Can you describe what you saw? It's hard to minimise in the abstract, need to address the most common cases. > And in random order the rest of the list was: > > Improve tools for styling > more admin control in user editing > application compatibility > include a timeline for map edits > include print layout > ability to download backdrop for a region(facilitates for offline) > use standard GIS terms > make it easier to make groups > help feature for frequently asked questions > having the legend directly on the map. > > These are all interesting, and at some point it'd be good to flesh some of them out more, but I'm already asking you a bunch of questions, so will hold off. > These are the raw items, they also tell something about the training we > did. > We did not cover the printing part, some people might have missed the > current > legend, we later showed some advanced users how to load peoples own styled > WMS > layers in ArcGIS & QGIS (to the exitement of the trainees). > > And, while at it, maybe i could at 2 of my personal favorites (Withouth > knowledge of 1.2): > - Import Geoserver layers into Geonode. This is especially useful for very > big > geotiffs and for remote WFS layers. > You mean like from a remote GeoServer? Or just being able to configure the GeoNode's GeoServer directly and then have GeoNode updated to be aware? I _think_ the later may be possible, with some geonode django command to update layers. But someone more knowledgable will have to sound in. The former is a feature that ANZSM is working on. Remote WFS layers don't work that well because cascading is so inefficient (I randomly wrote a paper about this and potential solutions / workarounds if anyone is interested). But remote WMS works pretty well. The map I linked to above actually is using a remote WMS - http://spatialmarketplace.net.au/data/geonode:0 I think it's actually an esri server - http://spatialmarketplace.net.au/services/69/ > - Slightly more userfriendly errormessages would be great. I know Tomcat > spits > out lists and lists of errors, but having a slight clue of where in the > process things broke helps diagnose issues a lot. > > Good to hear - we know this is an issue, and I think there have been some ideas about how to improve it. > In general, we love Geonode. Its very relevant in the Ugandan context. A > lot > of the discussion here is still about the actual sharing of data. There is > a > great level of reluctance. Using Geonode to show how easy it can become to > mix > & match data makes the advantage of sharing your data a lot more concrete > for > people. > > This is great to hear - we're obviously hoping to make it a whole lot better, but it's great to hear it already is relevant and makes a difference. If you're interested in helping out more in the community let us know. Haven't done a great job of making obvious how to help, but there's a whole lot to be done, for developers and non-developers. But even if you can't even this feedback is super great, and keep it coming when you use GeoNode more. best regards, Chris > Commercial drilling for oil is not expected to start before 2017. > > until then, > > Reinier Battenberg > > > > >
Hi, Its our pleasure to report back. Will do what we can while using the product. In the previous mail, i forgot to mention the URL of the Oil for Development program. http://www.norad.no/en/thematic-areas/energy/oil-for-development As I feared a bit already, this single email now contains about 15 threads. Lets see how we can make this work. buckly up, comments inline: On Wednesday 01 February 2012 14:00:41 Chris Holmes wrote: > Thanks for the super awesome feedback - it's incredibly helpful to reflect > this back to the project. So many people just use the software and don't > realize that a write up like this can greatly influence things. Often it's > just reiterating things we already know, but it still helps greatly in > prioritizing what to work on, as there's always a long list of things to > do. And indeed there's a whole lot of work that's been done on branches, > that isn't yet back in to the GeoNode core, so seeing the pain points does > help in deciding which of those to bring in first. > > Some responses inline: > > On Wed, Feb 1, 2012 at 8:20 AM, Reinier Battenberg < > > reinier.battenberg@mountbatten.net> wrote: > > Hi, > > ... > > > With this setup, we experienced the following issues. We are aware that > > there > > is a brandnew version soon to be released, and that some of the issues > > mentioned could very well have been solved already. Still, for people > > who > > would like to train in similar conditions, this might be worthwhile: > > > > - Not suprisingly, we had performance issues. We had most performance > > issues > > on the first day. Especially in the morning, when we althogether > > uploaded > > shapefiles. Processortime on the server maxed out at 400% and kept there > > for a > > long long time. A second factor might have been the slow internet > > connection > > and the overload of timed out http sessions to the Internet and the > > Geoserver > > that used all resources on the Wireless Router. The router didnt have > > any > > statistics, so this was not so straightforward to diagnose. (tip: use > > WRT > > on > > your wireless router) > > After lunch, when we started on creating maps and styling them, the > > system became much more responsive. > > In time I would like to see us do more testing and improvement on > performance stuff, so we could get GeoNode running in more constrained > environments. First focus has just been on getting the functionality > working, but it does sound like some profiling on resource constrained > machines might lead us to find some places for improvement. > > > - During the styling session people created their own maps, and started > > styling layers. During this exercise at some point, layers that where > > shared > > between maps got corrupted. People might have edited eachothers styles > > (they > > looked very similar), but what surely happened was that 2 layers lost > > their default SLD in geoserver. This renders some pages in geonode > > useless and causes all sorts of errors. Going into geoserver and > > assigning a (pretty random) default style to the layer fixes this. > > Are you able to reproduce this bug? Sounds like a serious one, and if you > could find a way to do it consistently then the developers may have a > decent chance of fixing it. I think the ownership / permissioning around > styling may not work great, and may take a deep solution, but it'd be good > if we can reproduce the common case, and maybe get a quick workaround that > doesn't lead to things breaking. We havent seen it ourselves (with 2 users) over the last months. It did happen (twice) with 10+ people each working on seperate maps, but a lot of them inclusing the same layer on that map. So, we cant structurally reproduce this. What is very easy to reproduce, is that when you change the style of a layer on a map, that style changes on all maps. I find this very annoying. If I change the style of a layer on a map, i only want to change that style in the context of that map, not in the context of the layer (and thus everywhere in Geonode). During training, we encouraged people to first duplicate the default style, and then work. That might not have always happened. > > > - On other geonode installations we had already seen that sometimes when > > an upload fails, a geonode layer might be in the geonode database, but > > is not present in geoserver (the whole table is missing). I think Jude > > is working on > > some code to straighten this out when it happens. Would be great if an > > admin/user could do this from the geonode gui. > > Cool, if Jude doesn't have this worked out then it could be good to file a > bug to keep track of it. > > > - On the second day, we brought a second server. It ran fine for about > > 30 > > minutes, then there was a powersurge (yeah, those happen in Uganda.) > > After the > > power came back, tomcat would not finish starting. The attached > > catalina.out > > sample shows the full startup sequence. The server never finishes > > indexing.> > > Could it be that: > > -- the indexing is not 100% necessary during training? > > -- the indexing is causing the very high load during uploading data > > > > sessions? > > If so, is there a way to switch off the indexing? That would save a lot > > of trainers in locations where you can not use "the cloud" during > > training sessions a lot of headaches of running around with server-size > > hardware. > What indexing are you talking about? Reading that log I'm not sure I see > the line you're talking about that indicates the server not finishing > indexing. its the last line on january 30th 2012-01-30 18:45:17,902 INFO [jeeves.apphand] - NOTE: Using shapefile for spatial index, this can be slow for larger catalogs that is the last line you see in the log. It never gets passed that line. I presumed somethng indexed something from here on. > The line that concerns me is the 'Couldn't create user > preferences directory. User preferences are unusable.' Right after the > epsg creation I think means it might have failed to create the epsg > database. i have seen this I think on every installation from the ppa. It appears constantly. I have a running server with a public IP that has this issue. > > > Enough about problems. At the end of the second day, we workshopped > > around uses of geonode, and one of the questions asked was: How could > > geonode be improved? > > > > The top 5 was: > > > > 1. More data analysis tools > > Any chance you could flesh this out? What types of data analysis tools? > And do you want these to output in to new layers, or to just do render > things differently on the current layer? Do they need a GUI to do this > data analysis? Our first data analysis step, at least in GeoServer, is to > add WPS and GeoScript, though that will be pretty programmer heavy. If > there are some subset of operations that you see being important for some > basic analysis that'd be good to know. Well, all i have is a list of stickynotes, and i wasnt in the group that raised this issue. Lets see the next one > > > 2. display attribute table > > How are you imagining this working? Like in a grid? Try the 'query' > functionality on http://sfpark.demo.opengeo.org/SFPark/ Oh, that is sweet. I wonder if it would fill in the request in issue 1 > and let me know if > that comes close to what you're wanting. I've also been thinking of making > a feature grid available by default on each layer's Data page. So people > could see not just the data on a map, but also look at what attributes are > there. Especially during styling, its nice to quickly have a look at the data. We had a sample dataset that had a field for Status. Its great to know what values can be in that Status field. (open/close? Red/Green? etc) Its like in QGIS, you do open the Attribute table a lot. So, i think the latter. (like, downloading the CSV version of the data, but then nicer) > > > 3. enable layer grouping > > You mean like on the layer tree? To be able to select a few layers and > then sort of squash them in to one layer, which then makes just one WMS > request? Do you want that layer group persisted on the server and > available for others to request through the WMS? Or just on a map level, > where it'd just make a multi layer WMS request. I think the originator ment to be able to create folders, in the pane, left of the map, like on the harvard story website. But yeah, bring it on ;-) > > > 4. connection to social sites like facebook and linkedIn > > This (and a whole lot more) is on the social features branch that is > planned for 1.2. And they're in the ANZSM, see the prototype at > http://spatialmarketplace.net.au/maps/13 You can like or +1. No linkedin > yet - I agree that in many ways that's the social platform that makes the > most sense, but I think they don't have easy to use API's. Not sure, but i did mention this feature being in the pipeline a few times throughout the day. That might be related :-) Facebook rules here in Uganda. > > > 5. minimise error bugs. > > Can you describe what you saw? It's hard to minimise in the abstract, need > to address the most common cases. I think people did really suffer from the performance issues we had in the cause of the 2 days. But, for example, when i did hit the (realy truly awesome) harvard story geonode installation, the first overlay i saw was a popup saying ERROR 500) I dig that. A lot of users conclude 'it doesnt work'. (and yes, we saw that popup a few times during our training as well) > > > And in random order the rest of the list was: > > > > Improve tools for styling > > more admin control in user editing > > application compatibility > > include a timeline for map edits > > include print layout > > ability to download backdrop for a region(facilitates for offline) > > use standard GIS terms > > make it easier to make groups > > help feature for frequently asked questions > > having the legend directly on the map. > > These are all interesting, and at some point it'd be good to flesh some of > them out more, but I'm already asking you a bunch of questions, so will > hold off. > > > These are the raw items, they also tell something about the training we > > did. > > We did not cover the printing part, some people might have missed the > > current > > legend, we later showed some advanced users how to load peoples own > > styled WMS > > layers in ArcGIS & QGIS (to the exitement of the trainees). > > > > And, while at it, maybe i could at 2 of my personal favorites (Withouth > > knowledge of 1.2): > > - Import Geoserver layers into Geonode. This is especially useful for > > very big > > geotiffs and for remote WFS layers. > > You mean like from a remote GeoServer? Or just being able to configure the > GeoNode's GeoServer directly and then have GeoNode updated to be aware? I > _think_ the later may be possible, with some geonode django command to > update layers. But someone more knowledgable will have to sound in. I think i read about the command and tried it. I might need to retry. > > The former is a feature that ANZSM is working on. Remote WFS layers don't > work that well because cascading is so inefficient (I randomly wrote a > paper about this and potential solutions / workarounds if anyone is > interested). But remote WMS works pretty well. The map I linked to above > actually is using a remote WMS - > http://spatialmarketplace.net.au/data/geonode:0 I think it's actually an > esri server - http://spatialmarketplace.net.au/services/69/ Having remote WFS loaded into Openlayers is cool, I was thinking of importing a remote WFS layer into Geoserver, and then into GeoNode. (There is a Drupal WFS module that exposes Drupal data as WFS, so you can use GeoServer to render tiles off of it, it only supports reading WFS, not writing) > > > - Slightly more userfriendly errormessages would be great. I know Tomcat > > spits > > out lists and lists of errors, but having a slight clue of where in the > > process things broke helps diagnose issues a lot. > > Good to hear - we know this is an issue, and I think there have been some > ideas about how to improve it. Awesome, very much related to 5 > > > In general, we love Geonode. Its very relevant in the Ugandan context. A > > lot > > of the discussion here is still about the actual sharing of data. There > > is a > > great level of reluctance. Using Geonode to show how easy it can become > > to mix > > & match data makes the advantage of sharing your data a lot more > > concrete > > for > > people. > > This is great to hear - we're obviously hoping to make it a whole lot > better, but it's great to hear it already is relevant and makes a > difference. If you're interested in helping out more in the community let > us know. Haven't done a great job of making obvious how to help, but > there's a whole lot to be done, for developers and non-developers. But > even if you can't even this feedback is super great, and keep it coming > when you use GeoNode more. We are doing our best, GeoNode surely has a role to play here. > > best regards, > > Chris > > > Commercial drilling for oil is not expected to start before 2017. > > > > until then, > > > > Reinier Battenberg
Hi all, Let me chime in here. - Styling issues I understand Batje's concern with the styling. One of the problems I have is editing styles for a specific layer via the map viewer, the styling is saved but not pushed to GeoServer. This is the case especially when one creates a new style from the map viewer. It would be expected that on saving that style it is pushed to the GeoServer's styles. So you end up with a series of styles that are on the map but not on the layer detail page. I have tried to trace the specific table/parameters that hold this info with little success. I also understand his concern of not getting much styling options on the style editior, i.e. gradient et al. Any one else experience this? - The upload process The issues I think here is the need for a handler to check if the layers render after the upload process is done. If pink tiles are registered then the layer upload fails. Just my thoughts, open to better ways to handle this. - GeoServer Data One may import data into geoserver and push it to geonode using the command : geonode updatelayers On 1 February 2012 14:49, Reinier Battenberg < reinier.battenberg@mountbatten.net> wrote: > Hi, > > Its our pleasure to report back. Will do what we can while using the > product. > > In the previous mail, i forgot to mention the URL of the Oil for > Development > program. > http://www.norad.no/en/thematic-areas/energy/oil-for-development > > As I feared a bit already, this single email now contains about 15 threads. > Lets see how we can make this work. > > buckly up, comments inline: > > > On Wednesday 01 February 2012 14:00:41 Chris Holmes wrote: > > Thanks for the super awesome feedback - it's incredibly helpful to > reflect > > this back to the project. So many people just use the software and don't > > realize that a write up like this can greatly influence things. Often > it's > > just reiterating things we already know, but it still helps greatly in > > prioritizing what to work on, as there's always a long list of things to > > do. And indeed there's a whole lot of work that's been done on branches, > > that isn't yet back in to the GeoNode core, so seeing the pain points > does > > help in deciding which of those to bring in first. > > > > Some responses inline: > > > > On Wed, Feb 1, 2012 at 8:20 AM, Reinier Battenberg < > > > > reinier.battenberg@mountbatten.net> wrote: > > > Hi, > > > > ... > > > > > With this setup, we experienced the following issues. We are aware that > > > there > > > is a brandnew version soon to be released, and that some of the issues > > > mentioned could very well have been solved already. Still, for people > > > who > > > would like to train in similar conditions, this might be worthwhile: > > > > > > - Not suprisingly, we had performance issues. We had most performance > > > issues > > > on the first day. Especially in the morning, when we althogether > > > uploaded > > > shapefiles. Processortime on the server maxed out at 400% and kept > there > > > for a > > > long long time. A second factor might have been the slow internet > > > connection > > > and the overload of timed out http sessions to the Internet and the > > > Geoserver > > > that used all resources on the Wireless Router. The router didnt have > > > any > > > statistics, so this was not so straightforward to diagnose. (tip: use > > > WRT > > > on > > > your wireless router) > > > After lunch, when we started on creating maps and styling them, the > > > system became much more responsive. > > > > In time I would like to see us do more testing and improvement on > > performance stuff, so we could get GeoNode running in more constrained > > environments. First focus has just been on getting the functionality > > working, but it does sound like some profiling on resource constrained > > machines might lead us to find some places for improvement. > > > > > - During the styling session people created their own maps, and started > > > styling layers. During this exercise at some point, layers that where > > > shared > > > between maps got corrupted. People might have edited eachothers styles > > > (they > > > looked very similar), but what surely happened was that 2 layers lost > > > their default SLD in geoserver. This renders some pages in geonode > > > useless and causes all sorts of errors. Going into geoserver and > > > assigning a (pretty random) default style to the layer fixes this. > > > > Are you able to reproduce this bug? Sounds like a serious one, and if > you > > could find a way to do it consistently then the developers may have a > > decent chance of fixing it. I think the ownership / permissioning around > > styling may not work great, and may take a deep solution, but it'd be > good > > if we can reproduce the common case, and maybe get a quick workaround > that > > doesn't lead to things breaking. > > We havent seen it ourselves (with 2 users) over the last months. It did > happen > (twice) with 10+ people each working on seperate maps, but a lot of them > inclusing the same layer on that map. So, we cant structurally reproduce > this. > > What is very easy to reproduce, is that when you change the style of a > layer > on a map, that style changes on all maps. I find this very annoying. If I > change the style of a layer on a map, i only want to change that style in > the > context of that map, not in the context of the layer (and thus everywhere > in > Geonode). > > During training, we encouraged people to first duplicate the default > style, and > then work. That might not have always happened. > > > > > > > - On other geonode installations we had already seen that sometimes > when > > > an upload fails, a geonode layer might be in the geonode database, but > > > is not present in geoserver (the whole table is missing). I think Jude > > > is working on > > > some code to straighten this out when it happens. Would be great if an > > > admin/user could do this from the geonode gui. > > > > Cool, if Jude doesn't have this worked out then it could be good to file > a > > bug to keep track of it. > > > > > - On the second day, we brought a second server. It ran fine for about > > > 30 > > > minutes, then there was a powersurge (yeah, those happen in Uganda.) > > > After the > > > power came back, tomcat would not finish starting. The attached > > > catalina.out > > > sample shows the full startup sequence. The server never finishes > > > indexing.> > > > Could it be that: > > > -- the indexing is not 100% necessary during training? > > > -- the indexing is causing the very high load during uploading data > > > > > > sessions? > > > If so, is there a way to switch off the indexing? That would save a lot > > > of trainers in locations where you can not use "the cloud" during > > > training sessions a lot of headaches of running around with server-size > > > hardware. > > What indexing are you talking about? Reading that log I'm not sure I see > > the line you're talking about that indicates the server not finishing > > indexing. > > > its the last line on january 30th > 2012-01-30 18:45:17,902 INFO [jeeves.apphand] - NOTE: Using shapefile for > spatial index, this can be slow for larger catalogs > > that is the last line you see in the log. It never gets passed that line. I > presumed somethng indexed something from here on. > > > > > The line that concerns me is the 'Couldn't create user > > preferences directory. User preferences are unusable.' Right after the > > epsg creation I think means it might have failed to create the epsg > > database. > > i have seen this I think on every installation from the ppa. It appears > constantly. > > I have a running server with a public IP that has this issue. > > > > > > Enough about problems. At the end of the second day, we workshopped > > > around uses of geonode, and one of the questions asked was: How could > > > geonode be improved? > > > > > > The top 5 was: > > > > > > 1. More data analysis tools > > > > Any chance you could flesh this out? What types of data analysis tools? > > And do you want these to output in to new layers, or to just do render > > things differently on the current layer? Do they need a GUI to do this > > data analysis? Our first data analysis step, at least in GeoServer, is > to > > add WPS and GeoScript, though that will be pretty programmer heavy. If > > there are some subset of operations that you see being important for some > > basic analysis that'd be good to know. > > Well, all i have is a list of stickynotes, and i wasnt in the group that > raised this issue. Lets see the next one > > > > > > 2. display attribute table > > > > How are you imagining this working? Like in a grid? Try the 'query' > > functionality on http://sfpark.demo.opengeo.org/SFPark/ > > Oh, that is sweet. I wonder if it would fill in the request in issue 1 > > > and let me know if > > that comes close to what you're wanting. I've also been thinking of > making > > a feature grid available by default on each layer's Data page. So people > > could see not just the data on a map, but also look at what attributes > are > > there. > > Especially during styling, its nice to quickly have a look at the data. We > had a sample dataset that had a field for Status. Its great to know what > values > can be in that Status field. (open/close? Red/Green? etc) > > Its like in QGIS, you do open the Attribute table a lot. > > So, i think the latter. (like, downloading the CSV version of the data, but > then nicer) > > > > > > 3. enable layer grouping > > > > You mean like on the layer tree? To be able to select a few layers and > > then sort of squash them in to one layer, which then makes just one WMS > > request? Do you want that layer group persisted on the server and > > available for others to request through the WMS? Or just on a map level, > > where it'd just make a multi layer WMS request. > > I think the originator ment to be able to create folders, in the pane, > left of > the map, like on the harvard story website. > > But yeah, bring it on ;-) > > > > > > 4. connection to social sites like facebook and linkedIn > > > > This (and a whole lot more) is on the social features branch that is > > planned for 1.2. And they're in the ANZSM, see the prototype at > > http://spatialmarketplace.net.au/maps/13 You can like or +1. No > linkedin > > yet - I agree that in many ways that's the social platform that makes the > > most sense, but I think they don't have easy to use API's. > > Not sure, but i did mention this feature being in the pipeline a few times > throughout the day. That might be related :-) > Facebook rules here in Uganda. > > > > > > 5. minimise error bugs. > > > > Can you describe what you saw? It's hard to minimise in the abstract, > need > > to address the most common cases. > > I think people did really suffer from the performance issues we had in the > cause of the 2 days. But, for example, when i did hit the (realy truly > awesome) harvard story geonode installation, the first overlay i saw was a > popup saying ERROR 500) I dig that. A lot of users conclude 'it doesnt > work'. > (and yes, we saw that popup a few times during our training as well) > > > > > > And in random order the rest of the list was: > > > > > > Improve tools for styling > > > more admin control in user editing > > > application compatibility > > > include a timeline for map edits > > > include print layout > > > ability to download backdrop for a region(facilitates for offline) > > > use standard GIS terms > > > make it easier to make groups > > > help feature for frequently asked questions > > > having the legend directly on the map. > > > > These are all interesting, and at some point it'd be good to flesh some > of > > them out more, but I'm already asking you a bunch of questions, so will > > hold off. > > > > > These are the raw items, they also tell something about the training we > > > did. > > > We did not cover the printing part, some people might have missed the > > > current > > > legend, we later showed some advanced users how to load peoples own > > > styled WMS > > > layers in ArcGIS & QGIS (to the exitement of the trainees). > > > > > > And, while at it, maybe i could at 2 of my personal favorites (Withouth > > > knowledge of 1.2): > > > - Import Geoserver layers into Geonode. This is especially useful for > > > very big > > > geotiffs and for remote WFS layers. > > > > You mean like from a remote GeoServer? Or just being able to configure > the > > GeoNode's GeoServer directly and then have GeoNode updated to be aware? > I > > _think_ the later may be possible, with some geonode django command to > > update layers. But someone more knowledgable will have to sound in. > > I think i read about the command and tried it. I might need to retry. > > > > > The former is a feature that ANZSM is working on. Remote WFS layers > don't > > work that well because cascading is so inefficient (I randomly wrote a > > paper about this and potential solutions / workarounds if anyone is > > interested). But remote WMS works pretty well. The map I linked to > above > > actually is using a remote WMS - > > http://spatialmarketplace.net.au/data/geonode:0 I think it's actually > an > > esri server - http://spatialmarketplace.net.au/services/69/ > > Having remote WFS loaded into Openlayers is cool, I was thinking of > importing > a remote WFS layer into Geoserver, and then into GeoNode. (There is a > Drupal > WFS module that exposes Drupal data as WFS, so you can use GeoServer to > render > tiles off of it, it only supports reading WFS, not writing) > > > > > > - Slightly more userfriendly errormessages would be great. I know > Tomcat > > > spits > > > out lists and lists of errors, but having a slight clue of where in the > > > process things broke helps diagnose issues a lot. > > > > Good to hear - we know this is an issue, and I think there have been some > > ideas about how to improve it. > > Awesome, very much related to 5 > > > > > > In general, we love Geonode. Its very relevant in the Ugandan context. > A > > > lot > > > of the discussion here is still about the actual sharing of data. There > > > is a > > > great level of reluctance. Using Geonode to show how easy it can become > > > to mix > > > & match data makes the advantage of sharing your data a lot more > > > concrete > > > for > > > people. > > > > This is great to hear - we're obviously hoping to make it a whole lot > > better, but it's great to hear it already is relevant and makes a > > difference. If you're interested in helping out more in the community > let > > us know. Haven't done a great job of making obvious how to help, but > > there's a whole lot to be done, for developers and non-developers. But > > even if you can't even this feedback is super great, and keep it coming > > when you use GeoNode more. > > > We are doing our best, GeoNode surely has a role to play here. > > > > > best regards, > > > > Chris > > > > > Commercial drilling for oil is not expected to start before 2017. > > > > > > until then, > > > > > > Reinier Battenberg > -- Regards, Jude Mwenda Skype id: jmwenda Twitter: www.twitter.com/judemwenda Web: www.africangeogeek.com "Was ist mein Leben, wenn ich nicht mehr nützlich für andere." Johann Wolfgang von Goethe
First off, thanks a lot for this detailed feedback! The Ubuntu packages in the GeoNode PPA are for a pre-release version of the upcoming release. There is not much difference between them apart from a few small bugfixes; I don't anticipate much of a performance difference. The most effective thing you can do to make GeoNode work better in a resource-constrained environment is to reduce the number of worker threads for Tomcat. This will cause GeoServer to attempt fewer tasks in parallel, reducing RAM and CPU usage. http://docs.geonode.org/en/latest/deploy/production.html#constrain-geoserver-worker-threads discusses how to modify this setting, although I would try an even lower value (10 or 20 instead of the 50 listed there). -- David Winslow OpenGeo - http://opengeo.org/ On Wed, Feb 1, 2012 at 8:20 AM, Reinier Battenberg < reinier.battenberg@mountbatten.net> wrote: > Hi, > > Under the Oil for Development program, NORAD is helping the Ugandan > Environmental sector to monitor the changes in the environment that are > related to the recent findings of oil in some of the most sensitive areas > of > Uganda. > > Last week there was a training with GIS profesionals from the Forest > Authority, Ministry of Water, Environment Authority etc. About 20 people > participated. Ragnvald Larsen from NORAD, Ketty Adoch and myself from > Mountbatten shared our Geonode knowledge. > > Instead of the recommended server with 6GB of memory, we opted for an atom > based portable media center PC with 2GB of memory. > http://www.advance.no/Item.aspx?id=4901512 > We installed the current geonode release from the apt-packages. > We connected everyone to a local LAN with a wireless router and started. > The > router was connected to a (very slow, maybe 256k) internet connection. > > With this setup, we experienced the following issues. We are aware that > there > is a brandnew version soon to be released, and that some of the issues > mentioned could very well have been solved already. Still, for people who > would like to train in similar conditions, this might be worthwhile: > > - Not suprisingly, we had performance issues. We had most performance > issues > on the first day. Especially in the morning, when we althogether uploaded > shapefiles. Processortime on the server maxed out at 400% and kept there > for a > long long time. A second factor might have been the slow internet > connection > and the overload of timed out http sessions to the Internet and the > Geoserver > that used all resources on the Wireless Router. The router didnt have any > statistics, so this was not so straightforward to diagnose. (tip: use WRT > on > your wireless router) > After lunch, when we started on creating maps and styling them, the system > became much more responsive. > > - During the styling session people created their own maps, and started > styling layers. During this exercise at some point, layers that where > shared > between maps got corrupted. People might have edited eachothers styles > (they > looked very similar), but what surely happened was that 2 layers lost their > default SLD in geoserver. This renders some pages in geonode useless and > causes all sorts of errors. Going into geoserver and assigning a (pretty > random) default style to the layer fixes this. > > - On other geonode installations we had already seen that sometimes when an > upload fails, a geonode layer might be in the geonode database, but is not > present in geoserver (the whole table is missing). I think Jude is working > on > some code to straighten this out when it happens. Would be great if an > admin/user could do this from the geonode gui. > > - On the second day, we brought a second server. It ran fine for about 30 > minutes, then there was a powersurge (yeah, those happen in Uganda.) After > the > power came back, tomcat would not finish starting. The attached > catalina.out > sample shows the full startup sequence. The server never finishes indexing. > Could it be that: > -- the indexing is not 100% necessary during training? > -- the indexing is causing the very high load during uploading data > sessions? > If so, is there a way to switch off the indexing? That would save a lot of > trainers in locations where you can not use "the cloud" during training > sessions a lot of headaches of running around with server-size hardware. > > Enough about problems. At the end of the second day, we workshopped around > uses of geonode, and one of the questions asked was: How could geonode be > improved? > > The top 5 was: > > 1. More data analysis tools > 2. display attribute table > 3. enable layer grouping > 4. connection to social sites like facebook and linkedIn > 5. minimise error bugs. > > And in random order the rest of the list was: > > Improve tools for styling > more admin control in user editing > application compatibility > include a timeline for map edits > include print layout > ability to download backdrop for a region(facilitates for offline) > use standard GIS terms > make it easier to make groups > help feature for frequently asked questions > having the legend directly on the map. > > These are the raw items, they also tell something about the training we > did. > We did not cover the printing part, some people might have missed the > current > legend, we later showed some advanced users how to load peoples own styled > WMS > layers in ArcGIS & QGIS (to the exitement of the trainees). > > And, while at it, maybe i could at 2 of my personal favorites (Withouth > knowledge of 1.2): > - Import Geoserver layers into Geonode. This is especially useful for very > big > geotiffs and for remote WFS layers. > - Slightly more userfriendly errormessages would be great. I know Tomcat > spits > out lists and lists of errors, but having a slight clue of where in the > process things broke helps diagnose issues a lot. > > In general, we love Geonode. Its very relevant in the Ugandan context. A > lot > of the discussion here is still about the actual sharing of data. There is > a > great level of reluctance. Using Geonode to show how easy it can become to > mix > & match data makes the advantage of sharing your data a lot more concrete > for > people. > > Commercial drilling for oil is not expected to start before 2017. > > until then, > > Reinier Battenberg > > > > >