Re: Notes on Batch Uploader and GeoNode
- From:
- David Winslow
- Date:
- 2010-06-16 @ 19:19
On 06/16/2010 03:10 PM, Sebastian Benthall wrote:
>
> Agreed, a more constrained extension would be good. I like
> *.geonode.xml, mind the user will have to name the xml file
> accordingly
> (how do they generate them?)
>
>
> Unclear. But it's ISO19139 compatible, so hopefully _something_
> generates that.
>
> --
> Sebastian Benthall
> OpenGeo - http://opengeo.org
>
This seems like an important point to get straight (basically without it
metadata won't happen.) What do we need to do to get some clarification
on this?
-d
Re: [geonode] Re: Notes on Batch Uploader and GeoNode
- From:
- Gabriel Roldan
- Date:
- 2010-06-16 @ 21:25
On 6/16/10 4:19 PM, David Winslow wrote:
> On 06/16/2010 03:10 PM, Sebastian Benthall wrote:
>>
>> Agreed, a more constrained extension would be good. I like
>> *.geonode.xml, mind the user will have to name the xml file
>> accordingly
>> (how do they generate them?)
>>
>>
>> Unclear. But it's ISO19139 compatible, so hopefully _something_
>> generates that.
My take on it is that we should allow any of the usual formats
GeoNetwork accepts (19115 a'la ESRI, 19115 legacy, vanilla 19139, etc)
and somehow convert on the fly to GeoNode's 19139 profile.
GeoNetwork comes with XSL transforms to go from one to another, we
should take them and make sure there are transformations to GeoNode's
profile available.
This is like this because I _think_ the most common case (or at least
one we should really support) will be importing a bunch of files from
ESRI products (eg, as exported by ArcCatalog?).
Now, the way things are set up so far kind of forces metadata to be in
our profile (which adds almost nothing to 19139 but a couple
restrictions). But sooner or later GeoNoders will need/require being
able to stick to their own profiles.
I think we should devote some time to try out this use case of allowing
the incoming metadata records to be in any of the default geonetwork
supported formats, and convert them to GeoNode's.
An easy (though kind of tricky) way of doing so would be:
- insert the record into geonetwork verbatim
- get the resulting uuid
- ask geonetwork for the record in GeoNode's format
- issue an update request using the returned GeoNode profile format
I'm not completely sure this would work out of the box though?
my 2c
gabriel
>>
>> --
>> Sebastian Benthall
>> OpenGeo - http://opengeo.org
>>
>
> This seems like an important point to get straight (basically without it
> metadata won't happen.) What do we need to do to get some clarification
> on this?
>
> -d
>
--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Re: Notes on Batch Uploader and GeoNode
- From:
- David Winslow
- Date:
- 2010-06-16 @ 16:57
On 06/16/2010 06:15 AM, Andrea Aime wrote:
> Sebastian Benthall ha scritto:
>> Sophia, David, and I met to talk about GeoNode requirements for the
>> Batch Uploader today and I wanted to get those notes out.
>>
>> For background, OpenGeo's working on a Batch Uploader client
>> application that users will be able to use to point at a directory of
>> data and metadata and upload it to a GeoServer instance. From there,
>> we are integrate with the GeoNode web application so that GeoNode
>> users can browse that data through the web UI.
>>
>> What features does this entail for GeoNode (on top of the features
>> being built as more generic community modules for GeoServer)? (David,
>> correct me if I've gotten any of this wrong)
>>
>> First, as part of the extra plugins that GeoNode adds to GeoServer,
>> we will need to add a listener to GeoServer that responds to new data
>> being loaded into GeoServer's FTP server and ping the GeoNode web app.
>
> I guess it's better to use a catalog listener that informs GeoNode about
> any added layer, which is also what David suggested
> If the layer is a shapefile or a raster (e.g., a file) the listeners
> can go to the file system and find the "sidecar" metadata file in the
> shapefile or raster files lot.
>
> However in the uploader I also have this new option of getting the
> shapefiles just to store them into a target database, such as a postgis
> one.
> When I do that the xml file won't be attached anywhere, it won't be in
> database.
>
> I guess one thing I could do would be to look for any xml file in the
> lot and make sure that no matter what happens the xml file gets copied
> in the workspaces/<workspace>/<store>/<featureType> directory of
> the newly configured layer?
>
> David, would that work for you? I guess your listener, running in
> GeoServer, could easily locate those files and post them back to
> GeoNode (or you just pass a fs location, but that assumes GeoNode
> and GeoServer are running on the same hardware, not sure if that
> is granted or not).
>
> xml file wise, do I have any naming convention to use? Just
> <shapefilename>.xml and <rastername>.xml?
> E.g., the uploader will handle automatically the .shp files and
> raster files set recognizing the usual sidecar files such as
> .dbf, .prj, .shx, .tfw, .wld and so on, I guess I just have
> to add a .xml extension to the lot?
>
> Cheers
> Andrea
I think we could make this work. Just carrying the sidecar file along
as <shapefilename>.xml or <rastername>.xml and dumping it into the
appropriate subfolder in workspaces would do the trick. Since this
metadata stuff is fairly custom, it may make sense to use a more
specific extension (.geonode.xml? .metadata.xml?) instead of xml to
avoid other random xml files being sucked in (like ESRI's .shp.xml).
What did you have in mind for the lifecycle of these unrecognized files?
Should the GeoNode listener remove them? Since one of the goals of
GeoNode is to provide metadata editing they will probably grow outdated
quickly.
Another concern I have is about database-backed configuration for
GeoServer, but I think it's safe to assume that won't be a concern
during the timeframe of this initial uploader project.
--
David Winslow
OpenGeo - http://opengeo.org/