Hi all, The search/Advanced search option for data in Geonode doesn't show proper results. Is there a problem with it? Say for example, "truemarble" is the name of the file and i search for "true", it says no files found. Why is it so? This does not happen with map search. The way search is performed for data and maps are different? How do it rectify the data search option so it works properly like map search? Pls help. Need to fix this very urgently. -- Regards Smitha V MT2009149 IIITB
Currently here's how it works: The data search page uses an AJAX endpoint in the Django application. This endpoint internally queries GeoNetwork's CSW using owslib, adds some additional information from the Django database, and converts all that info to JSON for easier consumption by JavaScript. The maps search page uses a different endpoint, which goes straight to the Django database. The data search page was originally written by a developer without much experience in CSW/owslib/etc. so it is a pretty basic usage. The maps search is a little more expert (it uses some wildcards internally, nothing too fancy). The plan is to, at some point in the future, combine the two search pages (so you have just a single place to search, rather than searching for data separately from maps.) Currently, the way I see that playing out is that we would add functionality to sync map metadata to GeoNetwork, and use the existing "advanced search" page for all searches. Somewhere along the way we'll also need to address this issue with overly strict queries. Fixing the issue should be a relatively simple matter of identifying the proper parameters to pass to csw.getrecords at https://github.com/GeoNode/geonode/blob/master/src/GeoNodePy/geonode/maps/views.py#L1351; given the state of the OWSLib documentation it will probably take a bit of code reading to sort out what that actually should be. tomkralidis is an OWSLib developer who shows up on the IRC list from time to time, he might be able to help if you are interested in providing a patch. Hope this helps. -- David Winslow OpenGeo - http://opengeo.org/ On Thu, Feb 3, 2011 at 3:14 AM, Smitha V <smitha.v@iiitb.net> wrote: > Hi all, > > The search/Advanced search option for data in Geonode doesn't show proper > results. Is there a problem with it? Say for example, "truemarble" is the > name of the file and i search for "true", it says no files found. Why is it > so? This does not happen with map search. The way search is performed for > data and maps are different? How do it rectify the data search option so it > works properly like map search? Pls help. Need to fix this very urgently. > > -- > Regards > Smitha V > MT2009149 > IIITB > >
How do i upgrade my GeoNetwork to 2.6.3? On Thu, Feb 3, 2011 at 7:35 PM, David Winslow <dwinslow@opengeo.org> wrote: > Currently here's how it works: > > The data search page uses an AJAX endpoint in the Django application. This > endpoint internally queries GeoNetwork's CSW using owslib, adds some > additional information from the Django database, and converts all that info > to JSON for easier consumption by JavaScript. > > The maps search page uses a different endpoint, which goes straight to the > Django database. > > The data search page was originally written by a developer without much > experience in CSW/owslib/etc. so it is a pretty basic usage. The maps > search is a little more expert (it uses some wildcards internally, nothing > too fancy). > > The plan is to, at some point in the future, combine the two search pages > (so you have just a single place to search, rather than searching for data > separately from maps.) Currently, the way I see that playing out is that we > would add functionality to sync map metadata to GeoNetwork, and use the > existing "advanced search" page for all searches. Somewhere along the way > we'll also need to address this issue with overly strict queries. > > Fixing the issue should be a relatively simple matter of identifying the > proper parameters to pass to csw.getrecords at > https://github.com/GeoNode/geonode/blob/master/src/GeoNodePy/geonode/maps/views.py#L1351; given the state of the OWSLib documentation it will probably take a bit of > code reading to sort out what that actually should be. tomkralidis is an > OWSLib developer who shows up on the IRC list from time to time, he might be > able to help if you are interested in providing a patch. > > Hope this helps. > > -- > David Winslow > OpenGeo - http://opengeo.org/ > > On Thu, Feb 3, 2011 at 3:14 AM, Smitha V <smitha.v@iiitb.net> wrote: > >> Hi all, >> >> The search/Advanced search option for data in Geonode doesn't show proper >> results. Is there a problem with it? Say for example, "truemarble" is the >> name of the file and i search for "true", it says no files found. Why is it >> so? This does not happen with map search. The way search is performed for >> data and maps are different? How do it rectify the data search option so it >> works properly like map search? Pls help. Need to fix this very urgently. >> >> -- >> Regards >> Smitha V >> MT2009149 >> IIITB >> >> > -- Regards Smitha V MT2009149 IIITB
Is it possible that the version of geonetwork included with geonode contains this bug? : http://osgeo-org.1803224.n2.nabble.com/Wildcard-in-GetRecords-query-td5578657.html That would explain why wildcard search began to work when I upgraded to geonetwork 2.6.3, without having to change any parameters. I checked the owslib code and it seems to be inserting wildcard characters correctly: http://trac.gispython.org/lab/browser/OWSLib/trunk/owslib/filter.py?rev=1698#L116 -Matt On 2/3/11 9:05 AM, David Winslow wrote: > Currently here's how it works: > > The data search page uses an AJAX endpoint in the Django application. > This endpoint internally queries GeoNetwork's CSW using owslib, adds > some additional information from the Django database, and converts all > that info to JSON for easier consumption by JavaScript. > > The maps search page uses a different endpoint, which goes straight to > the Django database. > > The data search page was originally written by a developer without > much experience in CSW/owslib/etc. so it is a pretty basic usage. The > maps search is a little more expert (it uses some wildcards > internally, nothing too fancy). > > The plan is to, at some point in the future, combine the two search > pages (so you have just a single place to search, rather than > searching for data separately from maps.) Currently, the way I see > that playing out is that we would add functionality to sync map > metadata to GeoNetwork, and use the existing "advanced search" page > for all searches. Somewhere along the way we'll also need to address > this issue with overly strict queries. > > Fixing the issue should be a relatively simple matter of identifying > the proper parameters to pass to csw.getrecords at > https://github.com/GeoNode/geonode/blob/master/src/GeoNodePy/geonode/maps/views.py#L1351 > ; given the state of the OWSLib documentation it will probably take a > bit of code reading to sort out what that actually should be. > tomkralidis is an OWSLib developer who shows up on the IRC list from > time to time, he might be able to help if you are interested in > providing a patch. > > Hope this helps. > > -- > David Winslow > OpenGeo - http://opengeo.org/ > > On Thu, Feb 3, 2011 at 3:14 AM, Smitha V <smitha.v@iiitb.net > <mailto:smitha.v@iiitb.net>> wrote: > > Hi all, > > The search/Advanced search option for data in Geonode doesn't show > proper results. Is there a problem with it? Say for example, > "truemarble" is the name of the file and i search for "true", it > says no files found. Why is it so? This does not happen with map > search. The way search is performed for data and maps are > different? How do it rectify the data search option so it works > properly like map search? Pls help. Need to fix this very urgently. > > -- > Regards > Smitha V > MT2009149 > IIITB > >
Thanks Matt, Can u tell me in detail how to configure GeoNode to ensure it uses GeoNetwork 2.6.3? How do i upgrade GeoNetwork to the required version? On Thu, Feb 3, 2011 at 11:49 PM, Matt Bertrand <mbertrand@cga.harvard.edu>wrote: > Is it possible that the version of geonetwork included with geonode > contains this bug? : > http://osgeo-org.1803224.n2.nabble.com/Wildcard-in-GetRecords-query-td5578657.html > > That would explain why wildcard search began to work when I upgraded to > geonetwork 2.6.3, without having to change any parameters. I checked the > owslib code and it seems to be inserting wildcard characters correctly: > http://trac.gispython.org/lab/browser/OWSLib/trunk/owslib/filter.py?rev=1698#L116 > > -Matt > > > > On 2/3/11 9:05 AM, David Winslow wrote: > > Currently here's how it works: > > The data search page uses an AJAX endpoint in the Django application. > This endpoint internally queries GeoNetwork's CSW using owslib, adds some > additional information from the Django database, and converts all that info > to JSON for easier consumption by JavaScript. > > The maps search page uses a different endpoint, which goes straight to > the Django database. > > The data search page was originally written by a developer without much > experience in CSW/owslib/etc. so it is a pretty basic usage. The maps > search is a little more expert (it uses some wildcards internally, nothing > too fancy). > > The plan is to, at some point in the future, combine the two search pages > (so you have just a single place to search, rather than searching for data > separately from maps.) Currently, the way I see that playing out is that we > would add functionality to sync map metadata to GeoNetwork, and use the > existing "advanced search" page for all searches. Somewhere along the way > we'll also need to address this issue with overly strict queries. > > Fixing the issue should be a relatively simple matter of identifying the > proper parameters to pass to csw.getrecords at > https://github.com/GeoNode/geonode/blob/master/src/GeoNodePy/geonode/maps/views.py#L1351; given the state of the OWSLib documentation it will probably take a bit of > code reading to sort out what that actually should be. tomkralidis is an > OWSLib developer who shows up on the IRC list from time to time, he might be > able to help if you are interested in providing a patch. > > Hope this helps. > > -- > David Winslow > OpenGeo - http://opengeo.org/ > > On Thu, Feb 3, 2011 at 3:14 AM, Smitha V <smitha.v@iiitb.net> wrote: > >> Hi all, >> >> The search/Advanced search option for data in Geonode doesn't show proper >> results. Is there a problem with it? Say for example, "truemarble" is the >> name of the file and i search for "true", it says no files found. Why is it >> so? This does not happen with map search. The way search is performed for >> data and maps are different? How do it rectify the data search option so it >> works properly like map search? Pls help. Need to fix this very urgently. >> >> -- >> Regards >> Smitha V >> MT2009149 >> IIITB >> >> > > -- Regards Smitha V MT2009149 IIITB
So far I've only upgraded on a test server where I didn't have any data I needed to keep, so I started with the blank database and let the "updatelayers" command populate it. I'm not yet sure what might be involved in migrating an existing geonetwork database to the latest version. Basically I downloaded the latest geonetwork war, expanded it, and copied all the files from the geonode/gnschema directory into geonetwork/xml/schemas/iso19139.geonode It seems to be working fine but there could be some gotcha's I haven't run into yet. You can try it out here: http://ec2-75-101-171-181.compute-1.amazonaws.com/data/search -Matt On 2/3/11 10:34 PM, Smitha V wrote: > Thanks Matt, > > Can u tell me in detail how to configure GeoNode to ensure it uses > GeoNetwork 2.6.3? How do i upgrade GeoNetwork to the required version? > > On Thu, Feb 3, 2011 at 11:49 PM, Matt Bertrand > <mbertrand@cga.harvard.edu <mailto:mbertrand@cga.harvard.edu>> wrote: > > Is it possible that the version of geonetwork included with > geonode contains this bug? : > http://osgeo-org.1803224.n2.nabble.com/Wildcard-in-GetRecords-query-td5578657.html > > > That would explain why wildcard search began to work when I > upgraded to geonetwork 2.6.3, without having to change any > parameters. I checked the owslib code and it seems to be > inserting wildcard characters correctly: > http://trac.gispython.org/lab/browser/OWSLib/trunk/owslib/filter.py?rev=1698#L116 > > -Matt > > > > On 2/3/11 9:05 AM, David Winslow wrote: >> Currently here's how it works: >> >> The data search page uses an AJAX endpoint in the Django >> application. This endpoint internally queries GeoNetwork's CSW >> using owslib, adds some additional information from the Django >> database, and converts all that info to JSON for easier >> consumption by JavaScript. >> >> The maps search page uses a different endpoint, which goes >> straight to the Django database. >> >> The data search page was originally written by a developer >> without much experience in CSW/owslib/etc. so it is a pretty >> basic usage. The maps search is a little more expert (it uses >> some wildcards internally, nothing too fancy). >> >> The plan is to, at some point in the future, combine the two >> search pages (so you have just a single place to search, rather >> than searching for data separately from maps.) Currently, the >> way I see that playing out is that we would add functionality to >> sync map metadata to GeoNetwork, and use the existing "advanced >> search" page for all searches. Somewhere along the way we'll >> also need to address this issue with overly strict queries. >> >> Fixing the issue should be a relatively simple matter of >> identifying the proper parameters to pass to csw.getrecords at >> https://github.com/GeoNode/geonode/blob/master/src/GeoNodePy/geonode/maps/views.py#L1351 >> ; given the state of the OWSLib documentation it will probably >> take a bit of code reading to sort out what that actually should >> be. tomkralidis is an OWSLib developer who shows up on the IRC >> list from time to time, he might be able to help if you are >> interested in providing a patch. >> >> Hope this helps. >> >> -- >> David Winslow >> OpenGeo - http://opengeo.org/ >> >> On Thu, Feb 3, 2011 at 3:14 AM, Smitha V <smitha.v@iiitb.net >> <mailto:smitha.v@iiitb.net>> wrote: >> >> Hi all, >> >> The search/Advanced search option for data in Geonode doesn't >> show proper results. Is there a problem with it? Say for >> example, "truemarble" is the name of the file and i search >> for "true", it says no files found. Why is it so? This does >> not happen with map search. The way search is performed for >> data and maps are different? How do it rectify the data >> search option so it works properly like map search? Pls help. >> Need to fix this very urgently. >> >> -- >> Regards >> Smitha V >> MT2009149 >> IIITB >> >> > > > > > -- > Regards > Smitha V > MT2009149 > IIITB >
Turns out I did run into a problem with the latest version of Geonetwork (2.6.3). On initial startup of geonode, the updatelayers works fine, importing metadata for existing layers correctly into Geonetwork. If I update the description for one of those layers, the geonetwork metadata is updated that once, but not on subsequent updates. The metadata for freshly uploaded layers gets inserted into geonetwork based on the default values but from that point on does not get updated at all, even on submission of the initial 'editing details' page immediately following the upload. I noticed this error in the jetty log: <ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.0.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd"> <ows:Exception exceptionCode="NoApplicableCode"> <ows:ExceptionText>Cannot process transaction: Raised exception while searching metadata : java.lang.NullPointerException</ows:ExceptionText> </ows:Exception> </ows:ExceptionReport> -Matt On 2/4/11 11:24 AM, Matt Bertrand wrote: > So far I've only upgraded on a test server where I didn't have any > data I needed to keep, so I started with the blank database and let > the "updatelayers" command populate it. I'm not yet sure what might be > involved in migrating an existing geonetwork database to the latest > version. > > Basically I downloaded the latest geonetwork war, expanded it, and > copied all the files from the geonode/gnschema directory into > geonetwork/xml/schemas/iso19139.geonode > > It seems to be working fine but there could be some gotcha's I haven't > run into yet. You can try it out here: > http://ec2-75-101-171-181.compute-1.amazonaws.com/data/search > > -Matt > > > On 2/3/11 10:34 PM, Smitha V wrote: >> Thanks Matt, >> >> Can u tell me in detail how to configure GeoNode to ensure it uses >> GeoNetwork 2.6.3? How do i upgrade GeoNetwork to the required version? >> >> On Thu, Feb 3, 2011 at 11:49 PM, Matt Bertrand >> <mbertrand@cga.harvard.edu <mailto:mbertrand@cga.harvard.edu>> wrote: >> >> Is it possible that the version of geonetwork included with >> geonode contains this bug? : >> http://osgeo-org.1803224.n2.nabble.com/Wildcard-in-GetRecords-query-td5578657.html >> >> >> That would explain why wildcard search began to work when I >> upgraded to geonetwork 2.6.3, without having to change any >> parameters. I checked the owslib code and it seems to be >> inserting wildcard characters correctly: >> http://trac.gispython.org/lab/browser/OWSLib/trunk/owslib/filter.py?rev=1698#L116 >> >> -Matt >> >> >> >> On 2/3/11 9:05 AM, David Winslow wrote: >>> Currently here's how it works: >>> >>> The data search page uses an AJAX endpoint in the Django >>> application. This endpoint internally queries GeoNetwork's CSW >>> using owslib, adds some additional information from the Django >>> database, and converts all that info to JSON for easier >>> consumption by JavaScript. >>> >>> The maps search page uses a different endpoint, which goes >>> straight to the Django database. >>> >>> The data search page was originally written by a developer >>> without much experience in CSW/owslib/etc. so it is a pretty >>> basic usage. The maps search is a little more expert (it uses >>> some wildcards internally, nothing too fancy). >>> >>> The plan is to, at some point in the future, combine the two >>> search pages (so you have just a single place to search, rather >>> than searching for data separately from maps.) Currently, the >>> way I see that playing out is that we would add functionality to >>> sync map metadata to GeoNetwork, and use the existing "advanced >>> search" page for all searches. Somewhere along the way we'll >>> also need to address this issue with overly strict queries. >>> >>> Fixing the issue should be a relatively simple matter of >>> identifying the proper parameters to pass to csw.getrecords at >>> https://github.com/GeoNode/geonode/blob/master/src/GeoNodePy/geonode/maps/views.py#L1351 >>> ; given the state of the OWSLib documentation it will probably >>> take a bit of code reading to sort out what that actually should >>> be. tomkralidis is an OWSLib developer who shows up on the IRC >>> list from time to time, he might be able to help if you are >>> interested in providing a patch. >>> >>> Hope this helps. >>> >>> -- >>> David Winslow >>> OpenGeo - http://opengeo.org/ >>> >>> On Thu, Feb 3, 2011 at 3:14 AM, Smitha V <smitha.v@iiitb.net >>> <mailto:smitha.v@iiitb.net>> wrote: >>> >>> Hi all, >>> >>> The search/Advanced search option for data in Geonode >>> doesn't show proper results. Is there a problem with it? Say >>> for example, "truemarble" is the name of the file and i >>> search for "true", it says no files found. Why is it so? >>> This does not happen with map search. The way search is >>> performed for data and maps are different? How do it rectify >>> the data search option so it works properly like map search? >>> Pls help. Need to fix this very urgently. >>> >>> -- >>> Regards >>> Smitha V >>> MT2009149 >>> IIITB >>> >>> >> >> >> >> >> -- >> Regards >> Smitha V >> MT2009149 >> IIITB >> >
afaik if we didn't migrate yet to the latest geonetwork release was just because of lack of developer time. But looks like we certainly could add that as a short term task to complete. I'm not aware of any problem the update could bring us, as we only use the CSW interface to communicate (yeah standards!), and we could certainly benefit of a couple fixes in geonetwork (including the OOM workaround which I think it's fixed on 2.6.3). Gabriel. On Fri, 2011-02-04 at 11:24 -0500, Matt Bertrand wrote: > So far I've only upgraded on a test server where I didn't have any > data I needed to keep, so I started with the blank database and let > the "updatelayers" command populate it. I'm not yet sure what might be > involved in migrating an existing geonetwork database to the latest > version. > > Basically I downloaded the latest geonetwork war, expanded it, and > copied all the files from the geonode/gnschema directory into > geonetwork/xml/schemas/iso19139.geonode > > It seems to be working fine but there could be some gotcha's I haven't > run into yet. You can try it out here: > http://ec2-75-101-171-181.compute-1.amazonaws.com/data/search > > -Matt > > > On 2/3/11 10:34 PM, Smitha V wrote: > > Thanks Matt, > > > > Can u tell me in detail how to configure GeoNode to ensure it uses > > GeoNetwork 2.6.3? How do i upgrade GeoNetwork to the required > > version? > > > > On Thu, Feb 3, 2011 at 11:49 PM, Matt Bertrand > > <mbertrand@cga.harvard.edu> wrote: > > Is it possible that the version of geonetwork included with > > geonode contains this bug? : > > http://osgeo-org.1803224.n2.nabble.com/Wildcard-in-GetRecords-query-td5578657.html > > > > That would explain why wildcard search began to work when I > > upgraded to geonetwork 2.6.3, without having to change any > > parameters. I checked the owslib code and it seems to be > > inserting wildcard characters correctly: > > http://trac.gispython.org/lab/browser/OWSLib/trunk/owslib/filter.py?rev=1698#L116 > > > > -Matt > > > > > > > > On 2/3/11 9:05 AM, David Winslow wrote: > > > Currently here's how it works: > > > > > > > > > The data search page uses an AJAX endpoint in the Django > > > application. This endpoint internally queries > > > GeoNetwork's CSW using owslib, adds some additional > > > information from the Django database, and converts all > > > that info to JSON for easier consumption by JavaScript. > > > > > > > > > The maps search page uses a different endpoint, which goes > > > straight to the Django database. > > > > > > > > > The data search page was originally written by a developer > > > without much experience in CSW/owslib/etc. so it is a > > > pretty basic usage. The maps search is a little more > > > expert (it uses some wildcards internally, nothing too > > > fancy). > > > > > > > > > The plan is to, at some point in the future, combine the > > > two search pages (so you have just a single place to > > > search, rather than searching for data separately from > > > maps.) Currently, the way I see that playing out is that > > > we would add functionality to sync map metadata to > > > GeoNetwork, and use the existing "advanced search" page > > > for all searches. Somewhere along the way we'll also need > > > to address this issue with overly strict queries. > > > > > > > > > Fixing the issue should be a relatively simple matter of > > > identifying the proper parameters to pass to > > > csw.getrecords > > > at https://github.com/GeoNode/geonode/blob/master/src/GeoNodePy/geonode/maps/views.py#L1351 ; given the state of the OWSLib documentation it will probably take a bit of code reading to sort out what that actually should be. tomkralidis is an OWSLib developer who shows up on the IRC list from time to time, he might be able to help if you are interested in providing a patch. > > > > > > > > > Hope this helps. > > > > > > > > > -- > > > David Winslow > > > OpenGeo - http://opengeo.org/ > > > > > > On Thu, Feb 3, 2011 at 3:14 AM, Smitha V > > > <smitha.v@iiitb.net> wrote: > > > Hi all, > > > > > > The search/Advanced search option for data in > > > Geonode doesn't show proper results. Is there a > > > problem with it? Say for example, "truemarble" is > > > the name of the file and i search for "true", it > > > says no files found. Why is it so? This does not > > > happen with map search. The way search is > > > performed for data and maps are different? How do > > > it rectify the data search option so it works > > > properly like map search? Pls help. Need to fix > > > this very urgently. > > > > > > -- > > > Regards > > > Smitha V > > > MT2009149 > > > IIITB > > > > > > > > > > > > > > > > > > > > > -- > > Regards > > Smitha V > > MT2009149 > > IIITB > > > -- Gabriel Roldan groldan@opengeo.org Expert service straight from the developers