So, this afternoon I started looking into updating the map serialization code (for getting maps out to the browser and loading them in the various GeoExt viewers.) See http://projects.opengeo.org/CAPRA/ticket/566#comment:13 Since we record the center of a map in lat/lon but display the map in Spherical Mercator, some reprojection is involved somewhere along the way. We can do this on the JavaScript side, but as we branch out into more projections later (see http://code.google.com/p/geonode-roadmap/issues/detail?id=19&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary%20category <http://code.google.com/p/geonode-roadmap/issues/detail?id=19&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary%20category>) we will probably want this to happen on the server side, to avoid the map viewer having to load lots of arbitrary CRS definitions. (Can proj4js load projection information lazily?) So, I dug a bit into loading GDAL's python bindings, only to find it doesn't easy_install. That's kind of a pain. I guess we have a few options - 1) Fix GDAL's setup.py 2) Update the build instructions to include installing gdal via a platform-specific installer and forbid --no-site-packages. (We will have to instruct people to install gdal themselves either way; it is not included in the python package.) 3) Keep passing the lat/lon coordinates and leave the reprojection up to OpenLayers. 4) Use pyproj instead (http://code.google.com/p/pyproj/) which does easy_install properly. Probably, the right strategy is to start working on getting a patch into GDAL, but use OpenLayers reprojection in the meantime while we're only using Spherical Mercator and Lat/Lon anyway. Other opinions? -- David Winslow OpenGeo - http://opengeo.org/
Hi, just a quick note to let you know that proj4js can do lazy loading of crs definitions. But I doubt that we will be using projections other than 900913 in the map viewers, because 900913 is our only option if we want to use Google layers (and OSM if we don't want to provide our own tileset). Regarding pyproj, I'm not sure if it can do 7 parameter transformations right. -Andreas. On Jul 9, 2010, at 22:12 , David Winslow wrote: > So, this afternoon I started looking into updating the map serialization code (for getting maps out to the browser and loading them in the various GeoExt viewers.) > > See http://projects.opengeo.org/CAPRA/ticket/566#comment:13 > > Since we record the center of a map in lat/lon but display the map in Spherical Mercator, some reprojection is involved somewhere along the way. We can do this on the JavaScript side, but as we branch out into more projections later (see http://code.google.com/p/geonode-roadmap/issues/detail?id=19&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary%20category) we will probably want this to happen on the server side, to avoid the map viewer having to load lots of arbitrary CRS definitions. (Can proj4js load projection information lazily?) > > So, I dug a bit into loading GDAL's python bindings, only to find it doesn't easy_install. That's kind of a pain. > > I guess we have a few options - > > 1) Fix GDAL's setup.py > 2) Update the build instructions to include installing gdal via a platform-specific installer and forbid --no-site-packages. (We will have to instruct people to install gdal themselves either way; it is not included in the python package.) > 3) Keep passing the lat/lon coordinates and leave the reprojection up to OpenLayers. > 4) Use pyproj instead (http://code.google.com/p/pyproj/) which does easy_install properly. > > Probably, the right strategy is to start working on getting a patch into GDAL, but use OpenLayers reprojection in the meantime while we're only using Spherical Mercator and Lat/Lon anyway. > > Other opinions? > > -- > David Winslow > OpenGeo - http://opengeo.org/ -- Andreas Hocevar OpenGeo - http://opengeo.org/ Expert service straight from the developers.
On 7/9/10 5:25 PM, Andreas Hocevar wrote: > Hi, > > just a quick note to let you know that proj4js can do lazy loading of > crs definitions. > > But I doubt that we will be using projections other than 900913 in > the map viewers, because 900913 is our only option if we want to use > Google layers (and OSM if we don't want to provide our own tileset). But there may be the case that the user doesn't care about google layers nor OSM ones, he just want to use data from his municipality's WMS, but the WMS doesn't support 900913. If GeoNode doesn't support the CRS the data is published in, then he can't use the data at all. Then GeoNode is useless. That's exactly what happened to me and why I opened that ticket. Aside: the whole question arises from the fact that we're storing the center point of the map in 4326, then who performs the transformation to Map's CRS. Wouldn't make more sense to store the map's center point in the Map's CRS to start with? (I don't know about the internals of this design decision so excuse me if I'm missing something obvious.) Cheers, Gabriel > > Regarding pyproj, I'm not sure if it can do 7 parameter > transformations right. > > -Andreas.
Gabriel Roldan ha scritto: > On 7/9/10 5:25 PM, Andreas Hocevar wrote: >> Hi, >> >> just a quick note to let you know that proj4js can do lazy loading of >> crs definitions. >> >> But I doubt that we will be using projections other than 900913 in >> the map viewers, because 900913 is our only option if we want to use >> Google layers (and OSM if we don't want to provide our own tileset). > But there may be the case that the user doesn't care about google layers > nor OSM ones, he just want to use data from his municipality's WMS, but > the WMS doesn't support 900913. If GeoNode doesn't support the CRS the > data is published in, then he can't use the data at all. Then GeoNode is > useless. A bit of self promotion: GeoServer 2.1 can do WMS cascading and handles reprojection transparently, doing the necessary negotiation and reprojecting on the fly if necessary. It works for GetFeatureInfo as well Now, I understand that from where you are today, to having to configure GS as a cascading WMS for all remote layers and moving GeoNode to GS 2.1 is quite, is a long stretch. Just sharing an option ;-) Cheers Andrea -- Andrea Aime OpenGeo - http://opengeo.org Expert service straight from the developers.
On 7/12/10 4:52 AM, Andrea Aime wrote: > Gabriel Roldan ha scritto: >> On 7/9/10 5:25 PM, Andreas Hocevar wrote: >>> Hi, >>> >>> just a quick note to let you know that proj4js can do lazy loading of >>> crs definitions. >>> >>> But I doubt that we will be using projections other than 900913 in >>> the map viewers, because 900913 is our only option if we want to use >>> Google layers (and OSM if we don't want to provide our own tileset). >> But there may be the case that the user doesn't care about google layers >> nor OSM ones, he just want to use data from his municipality's WMS, but >> the WMS doesn't support 900913. If GeoNode doesn't support the CRS the >> data is published in, then he can't use the data at all. Then GeoNode is >> useless. > > A bit of self promotion: GeoServer 2.1 can do WMS cascading and handles > reprojection transparently, doing the necessary negotiation and > reprojecting on the fly if necessary. > It works for GetFeatureInfo as well > > Now, I understand that from where you are today, to having to configure > GS as a cascading WMS for all remote layers and moving GeoNode to GS > 2.1 is quite, is a long stretch. > As a matter of fact that's the first thing I thought when faced the problem of not being able to add my municipality layers. But besides being tied to 2.0.x, I'm not sure if we'd want to direct all traffic to our geoserver instance. But it's still an option. > Just sharing an option ;-) thanks :) > > Cheers > Andrea > > -- Gabriel Roldan OpenGeo - http://opengeo.org Expert service straight from the developers.
On 07/12/2010 12:53 PM, Gabriel Roldan wrote: > On 7/12/10 4:52 AM, Andrea Aime wrote: > >> Gabriel Roldan ha scritto: >> >>> On 7/9/10 5:25 PM, Andreas Hocevar wrote: >>> >>>> Hi, >>>> >>>> just a quick note to let you know that proj4js can do lazy loading of >>>> crs definitions. >>>> >>>> But I doubt that we will be using projections other than 900913 in >>>> the map viewers, because 900913 is our only option if we want to use >>>> Google layers (and OSM if we don't want to provide our own tileset). >>>> >>> But there may be the case that the user doesn't care about google layers >>> nor OSM ones, he just want to use data from his municipality's WMS, but >>> the WMS doesn't support 900913. If GeoNode doesn't support the CRS the >>> data is published in, then he can't use the data at all. Then GeoNode is >>> useless. >>> >> A bit of self promotion: GeoServer 2.1 can do WMS cascading and handles >> reprojection transparently, doing the necessary negotiation and >> reprojecting on the fly if necessary. >> It works for GetFeatureInfo as well >> >> Now, I understand that from where you are today, to having to configure >> GS as a cascading WMS for all remote layers and moving GeoNode to GS >> 2.1 is quite, is a long stretch. >> >> > As a matter of fact that's the first thing I thought when faced the > problem of not being able to add my municipality layers. But besides > being tied to 2.0.x, I'm not sure if we'd want to direct all traffic to > our geoserver instance. But it's still an option. > > >> Just sharing an option ;-) >> > thanks :) > >> Cheers >> Andrea >> How does GeoServer's cascading WMS work? Would we expect degradation due to warping of the tiles instead of re-rendering in the target projection? If there are visual artifacts then I expect we would want to use this only as a fall-back option regardless of performance concerns. We are already discussing having GeoNode cache all remote WMS layers in GeoWebCache, so I don't think that it would especially burdensome (development-wise) to have it cascade them all in GeoServer as well. At least, let's not strike it from the realm of possibility too hastily. -- David Winslow OpenGeo - http://opengeo.org/
On Jul 12, 2010, at 06:45 , Gabriel Roldan wrote: > On 7/9/10 5:25 PM, Andreas Hocevar wrote: >> Hi, >> >> just a quick note to let you know that proj4js can do lazy loading of >> crs definitions. >> >> But I doubt that we will be using projections other than 900913 in >> the map viewers, because 900913 is our only option if we want to use >> Google layers (and OSM if we don't want to provide our own tileset). > But there may be the case that the user doesn't care about google layers > nor OSM ones, he just want to use data from his municipality's WMS, but > the WMS doesn't support 900913. If GeoNode doesn't support the CRS the > data is published in, then he can't use the data at all. Then GeoNode is > useless. Good point. Tim and I have been talking about some kind of "CRS negotiation", meaning the client has to figure out the common denominator of available CRS of all layers added to a map, and warn about layers that have to be removed because they don't match the CRS of other layers. > That's exactly what happened to me and why I opened that ticket. > > Aside: the whole question arises from the fact that we're storing the > center point of the map in 4326, then who performs the transformation to > Map's CRS. Wouldn't make more sense to store the map's center point in > the Map's CRS to start with? (I don't know about the internals of this > design decision so excuse me if I'm missing something obvious.) I was thinking along the same lines when I wrote my initial reply, but considered the fact that the coordinates are stored in 4326 as engraved in stone. But you are right, as long as we don't search maps (only layers) by location, storing the map bounds/center in map CRS coordinates would solve the problem, and make map persistence 100% compatible with the gxp viewer. -Andreas.
On 7/12/10 4:48 AM, Andreas Hocevar wrote: > On Jul 12, 2010, at 06:45 , Gabriel Roldan wrote: > >> On 7/9/10 5:25 PM, Andreas Hocevar wrote: >>> Hi, >>> >>> just a quick note to let you know that proj4js can do lazy >>> loading of crs definitions. >>> >>> But I doubt that we will be using projections other than 900913 >>> in the map viewers, because 900913 is our only option if we want >>> to use Google layers (and OSM if we don't want to provide our own >>> tileset). >> But there may be the case that the user doesn't care about google >> layers nor OSM ones, he just want to use data from his >> municipality's WMS, but the WMS doesn't support 900913. If GeoNode >> doesn't support the CRS the data is published in, then he can't use >> the data at all. Then GeoNode is useless. > > Good point. Tim and I have been talking about some kind of "CRS > negotiation", meaning the client has to figure out the common > denominator of available CRS of all layers added to a map, and warn > about layers that have to be removed because they don't match the CRS > of other layers. > >> That's exactly what happened to me and why I opened that ticket. >> >> Aside: the whole question arises from the fact that we're storing >> the center point of the map in 4326, then who performs the >> transformation to Map's CRS. Wouldn't make more sense to store the >> map's center point in the Map's CRS to start with? (I don't know >> about the internals of this design decision so excuse me if I'm >> missing something obvious.) > > I was thinking along the same lines when I wrote my initial reply, > but considered the fact that the coordinates are stored in 4326 as > engraved in stone. But you are right, as long as we don't search maps > (only layers) by location, storing the map bounds/center in map CRS > coordinates would solve the problem, and make map persistence 100% > compatible with the gxp viewer. that would be (_IMHO_) step in doing it the right way. And it doesn't preclude the ability to make map searches by map bounds in the future, if we need to. There may be a couple options: - A Map may have it's own metadata record, where lat lon bounds is already there to be used on spatial searches - we may have our own database of maps, where we could search by something like toWGS84(mapbounds) overlaps my bounds. Gabriel > > -Andreas. -- Gabriel Roldan OpenGeo - http://opengeo.org Expert service straight from the developers.
On 07/12/2010 01:15 PM, Gabriel Roldan wrote: > On 7/12/10 4:48 AM, Andreas Hocevar wrote: > >> On Jul 12, 2010, at 06:45 , Gabriel Roldan wrote: >> >> >>> On 7/9/10 5:25 PM, Andreas Hocevar wrote: >>> >>>> Hi, >>>> >>>> just a quick note to let you know that proj4js can do lazy >>>> loading of crs definitions. >>>> >>>> But I doubt that we will be using projections other than 900913 >>>> in the map viewers, because 900913 is our only option if we want >>>> to use Google layers (and OSM if we don't want to provide our own >>>> tileset). >>>> >>> But there may be the case that the user doesn't care about google >>> layers nor OSM ones, he just want to use data from his >>> municipality's WMS, but the WMS doesn't support 900913. If GeoNode >>> doesn't support the CRS the data is published in, then he can't use >>> the data at all. Then GeoNode is useless. >>> >> Good point. Tim and I have been talking about some kind of "CRS >> negotiation", meaning the client has to figure out the common >> denominator of available CRS of all layers added to a map, and warn >> about layers that have to be removed because they don't match the CRS >> of other layers. >> >> >>> That's exactly what happened to me and why I opened that ticket. >>> >>> Aside: the whole question arises from the fact that we're storing >>> the center point of the map in 4326, then who performs the >>> transformation to Map's CRS. Wouldn't make more sense to store the >>> map's center point in the Map's CRS to start with? (I don't know >>> about the internals of this design decision so excuse me if I'm >>> missing something obvious.) >>> >> I was thinking along the same lines when I wrote my initial reply, >> but considered the fact that the coordinates are stored in 4326 as >> engraved in stone. But you are right, as long as we don't search maps >> (only layers) by location, storing the map bounds/center in map CRS >> coordinates would solve the problem, and make map persistence 100% >> compatible with the gxp viewer. >> > that would be (_IMHO_) step in doing it the right way. And it doesn't > preclude the ability to make map searches by map bounds in the future, > if we need to. > There may be a couple options: > - A Map may have it's own metadata record, where lat lon bounds is > already there to be used on spatial searches > - we may have our own database of maps, where we could search by > something like toWGS84(mapbounds) overlaps my bounds. > > Gabriel > > >> -Andreas. >> > > This sounds good to me, we can just punt on the reprojection-in-python issue for now. We do still need to handle a reprojection/migration on the live site, but someone (me maybe?) can script that from outside the Django webapp. Here's the current set of fields on a Map: title = models.CharField abstract = models.CharField contact = models.CharField featured = models.BooleanField endorsed = models.BooleanField zoom = models.IntegerField center_lat = models.FloatField center_lon = models.FloatField owner = models.ForeignKey I guess we should rename center_lat and center_lon to center_x and center_y, and add a display_srid field to track the display projection? On the other hand, as long as we are migrating all this stuff, we could replace the (center + zoomlevel) model with just a bbox. Zoom levels seem a bit vague in the face of arbitrary projections, unless I am mistaken. Opinions? -- David Winslow OpenGeo - http://opengeo.org/
On Jul 12, 2010, at 19:28 , David Winslow wrote: > On 07/12/2010 01:15 PM, Gabriel Roldan wrote: >> On 7/12/10 4:48 AM, Andreas Hocevar wrote: >> >>> On Jul 12, 2010, at 06:45 , Gabriel Roldan wrote: >>> >>>> >>>> Aside: the whole question arises from the fact that we're storing >>>> the center point of the map in 4326, then who performs the >>>> transformation to Map's CRS. Wouldn't make more sense to store the >>>> map's center point in the Map's CRS to start with? (I don't know >>>> about the internals of this design decision so excuse me if I'm >>>> missing something obvious.) >>>> >>> I was thinking along the same lines when I wrote my initial reply, >>> but considered the fact that the coordinates are stored in 4326 as >>> engraved in stone. But you are right, as long as we don't search maps >>> (only layers) by location, storing the map bounds/center in map CRS >>> coordinates would solve the problem, and make map persistence 100% >>> compatible with the gxp viewer. >>> >> that would be (_IMHO_) step in doing it the right way. And it doesn't >> preclude the ability to make map searches by map bounds in the future, >> if we need to. >> There may be a couple options: >> - A Map may have it's own metadata record, where lat lon bounds is >> already there to be used on spatial searches >> - we may have our own database of maps, where we could search by >> something like toWGS84(mapbounds) overlaps my bounds. >> >> Gabriel >> >> >>> -Andreas. >>> >> >> > This sounds good to me, we can just punt on the reprojection-in-python > issue for now. We do still need to handle a reprojection/migration on > the live site, but someone (me maybe?) can script that from outside the > Django webapp. > > Here's the current set of fields on a Map: > title = models.CharField > abstract = models.CharField > contact = models.CharField > > featured = models.BooleanField > endorsed = models.BooleanField > > zoom = models.IntegerField > center_lat = models.FloatField > center_lon = models.FloatField > owner = models.ForeignKey > > I guess we should rename center_lat and center_lon to center_x and > center_y, and add a display_srid field to track the display projection? +1 on adding display_srid, or just srid. > > On the other hand, as long as we are migrating all this stuff, we could > replace the (center + zoomlevel) model with just a bbox. Zoom levels > seem a bit vague in the face of arbitrary projections, unless I am > mistaken. Opinions? Not as long as we always assume a maxExtent for each projection we use (e.g. the recommended maxExtent for the projection as advertised in the EPSG database). In that case, a context of projection+center+zoom will give us well defined zoom levels. And with the planned GWC integration, we need to make a maxExtent assumption on a per-projection basis anyway. Also, there is a behavioral difference between center+zoom and bbox. The former guarantees a zoom level, while the latter guarantees a visible extent independent of the map size on the screen. At this point, gxp.Viewer only supports the former. The reason for this is the behavior of OpenLayers.Map.zoomToExtent, which guarantees the whole extent to be visible. This tends to zoom out the map by one zoom level with every load-save cycle, unless the aspect ratio of the bbox is exactly the same as the map on the users's screen. The disadvantage of storing center+zoom is that on small maps, users won't see the full extent. The disadvantage of storing bbox ist that the map might look different depending on the screen size due to scale dependent styling. For the sake of simplicity, I'm in favor of going with center+zoom, but I'm willing to make the necessary changes to gxp.Viewer if extent should also be supported. Regards, Andreas. -- Andreas Hocevar OpenGeo - http://opengeo.org/ Expert service straight from the developers.
<snip/> > For the sake of simplicity, I'm in favor of going with center+zoom, but I'm willing to make the necessary changes to gxp.Viewer if extent should also be supported. > > Regards, > Andreas. > Okay, let's stick with center+zoom. I'll try and move around the appropriate bits in time for this week's deployment/migration. -- David Winslow OpenGeo - http://opengeo.org/
On 07/14/2010 10:22 AM, David Winslow wrote: > <snip/> > >> For the sake of simplicity, I'm in favor of going with center+zoom, but I'm willing to make the necessary changes to gxp.Viewer if extent should also be supported. >> >> Regards, >> Andreas. >> >> > Okay, let's stick with center+zoom. I'll try and move around the > appropriate bits in time for this week's deployment/migration. > > -- > David Winslow > OpenGeo - http://opengeo.org/ > Ah. The new configuration structure also adds a few other details about the projection: "units":"m", "maxResolution":156543.0339, "maxExtent":[-20037508.34,-20037508.34,20037508.34,20037508.34], Are these things we could/should store as part of the configuration, or should they be computed/looked up somehow based on projection? -- David Winslow
On Jul 15, 2010, at 19:39 , David Winslow wrote: > On 07/14/2010 10:22 AM, David Winslow wrote: >> <snip/> >> >>> For the sake of simplicity, I'm in favor of going with center+zoom, but I'm willing to make the necessary changes to gxp.Viewer if extent should also be supported. >>> >>> Regards, >>> Andreas. >>> >>> >> Okay, let's stick with center+zoom. I'll try and move around the >> appropriate bits in time for this week's deployment/migration. >> >> -- >> David Winslow >> OpenGeo - http://opengeo.org/ >> > Ah. The new configuration structure also adds a few other details about > the projection: > > "units":"m", > "maxResolution":156543.0339, > "maxExtent":[-20037508.34,-20037508.34,20037508.34,20037508.34], > > Are these things we could/should store as part of the configuration, or > should they be computed/looked up somehow based on projection? Ah right, had forgotten about these. How we proceed with them depends on the GWC integration, especially the grid origin, because it relates to the maxExtent. Since we don't rely on a base layer that determines these properties for the map, I'd propose the following: We should look up these properties based on the projection. In the EPSG database, there is an extent for every projection. We can use this as maxExtent. The units can also be taken from the EPSG database. And the maxResolution is just the maximum distance from the left to the right edge of the max extent divided by 256. In the future, when the user creates a new map, we will have to ask her for the target projection of the map. Then the new map (/maps/new) will have to be created with the appropriate values for the selected projection. Makes sense? -Andreas.
What's wrong with the pyproj option? It seems like the cleanest to me. What's the difference between GDAL and PROJ 4, functionally, here ? On Fri, Jul 9, 2010 at 4:12 PM, David Winslow <dwinslow@opengeo.org> wrote: > So, this afternoon I started looking into updating the map serialization > code (for getting maps out to the browser and loading them in the various > GeoExt viewers.) > > See http://projects.opengeo.org/CAPRA/ticket/566#comment:13 > > Since we record the center of a map in lat/lon but display the map in > Spherical Mercator, some reprojection is involved somewhere along the way. > We can do this on the JavaScript side, but as we branch out into more > projections later (see > http://code.google.com/p/geonode-roadmap/issues/detail?id=19&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary%20category) > we will probably want this to happen on the server side, to avoid the map > viewer having to load lots of arbitrary CRS definitions. (Can proj4js load > projection information lazily?) > > So, I dug a bit into loading GDAL's python bindings, only to find it > doesn't easy_install. That's kind of a pain. > > I guess we have a few options - > > 1) Fix GDAL's setup.py > 2) Update the build instructions to include installing gdal via a > platform-specific installer and forbid --no-site-packages. (We will have to > instruct people to install gdal themselves either way; it is not included in > the python package.) > 3) Keep passing the lat/lon coordinates and leave the reprojection up to > OpenLayers. > 4) Use pyproj instead (http://code.google.com/p/pyproj/) which does > easy_install properly. > > Probably, the right strategy is to start working on getting a patch into > GDAL, but use OpenLayers reprojection in the meantime while we're only using > Spherical Mercator and Lat/Lon anyway. > > Other opinions? > > -- > David Winslow > OpenGeo - http://opengeo.org/ > -- Sebastian Benthall OpenGeo - http://opengeo.org