Author Archive for Chris Tweedie

Generic Web Proxies

In my quest for increased adoption of geospatial web services, I would constantly bash my head against the wall trying to debug GIS applications. So if you have suffered from “what the” behaviour such as …

  • weird uri encoding
  • apps pretending to talk SSL but only on some requests
  • not supporting BASIC authentication when they say they do
  • clients not sending the required STYLES WMS kvp
  • sending hundreds upon hundreds of chunked requests …

then these scripts/apps may be for you. They are pretty generic and can be applied to any AJAX-type cross-domain restriction. The only OGC specific type line is the string replace of the online resource with the proxy uri (for obvious reasons for the getcapabilities document).

Other recommends ..

  1. For desktop based apps, i highly recommend fiddler2 as man in the middle proxy interceptor for debugging HTTP. It even does HTTPS mitm :)
  2. If you want to enable HTTPS/BASIC authentication on a desktop client that doesnt support it, check out InteProxy or email me for my own “Gismo” command line version. This will allow apps such as GRASS or QGIS which only has standard WMS support to magically start working on these services

But if you are just trying to get your poor OpenLayers application talking to that lonesome WFS server sitting on the interweb, these might come in handy!

Note that these are open proxies by default!

< ?php
	$urlparams = urldecode($_SERVER['QUERY_STRING']);
         $ch = curl_init();
	curl_setopt($ch, CURLOPT_URL,$url."&Styles=");
	curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,  2);
 	curl_setopt($ch, CURLOPT_USERAGENT, "Openlayers proxy - CTweedie hax"); // Set a different user-agent so we can track usage easier
	curl_setopt($ch, CURLOPT_FAILONERROR,1);
	//curl_setopt($ch, CURLOPT_VERBOSE, 1);
   	curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
	curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);  // the next 3 lines makes it work through https SSL3 with authorization.
	curl_setopt($ch, CURLOPT_SSLVERSION, 3);
	curl_setopt($ch, CURLOPT_USERPWD, $user.":".$pass);
	$data = curl_exec($ch); // Execute query
        $data = str_replace("https://www.wms.com/server/to/reflect/to?","https://www.wms.com/server/proxy?", $data)
        $content_type = curl_getinfo( $ch, CURLINFO_CONTENT_TYPE );
	header('Content-Type: '.$content_type);
	echo $data;
	curl_close($ch);
>

Python equivalent … almost identical to the OpenLayers version. In most situations, py urllib runs hands down quicker than php curl but it could well be my dodgy code!

#!/usr/bin/env python -u
 
import urllib
import urllib2
import cgi
import socket
import msvcrt
import os
import sys
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
# timeout in seconds
timeout = 15
socket.setdefaulttimeout(timeout)
 
fs = cgi.FieldStorage()
urlt = "https://www.wms.com/server/to/reflect/to?"
 
for i in fs.keys():
  urlt += i+"="+fs[i].value+"&"
url = urllib.unquote(urlt)
try:
    if url.startswith("http://") or url.startswith("https://"):
           passman = urllib2.HTTPPasswordMgrWithDefaultRealm()      # this creates a password manager
           passman.add_password(None, urlt, 'user', 'password')      # because we have put None at the start it will always use this username/password combination
           authhandler = urllib2.HTTPBasicAuthHandler(passman)                 # create the AuthHandler
           opener = urllib2.build_opener(authhandler)
           urllib2.install_opener(opener)
        y = urllib2.urlopen(url)
 
        headers = str(y.info()).split('\n')
        for h in headers:
            if h.startswith("Content-Type:"):
                print h
        print
        print y.read().replace("https://www.wms.com/server/to/reflect/to?","https://www.wms.com/server/proxy?")
        y.close()
    else:
        print """Content-Type: text/plain Illegal request."""
except Exception, E:
    print "Status: 500 Unexpected Error"
    print "Content-Type: text/plain"
    print
    print url
    print "Some unexpected error occurred. Error text was:", E

Bravo ..

..to Sebastian’s spatialreference.org post. You made me chuckle and i don’t really know why :) I had actually not seen the projection render call, that is damn nice work Chris and Howard. Sebastian lists a few suggested improvements to the “service” which i whole heartedly agree. Geodesy/datums/projections/geoids/coordinate systems need not be some magical black art done only by PhD’s, or alternatively some magical program that you insert numbers in, get stuff out, but having no idea what just happened. Let there be light …

Second bravo goes to Flamingo mapping components, a new (i think) dutch GPL mapviewer. I happened to stumble onto these guys’ Flash based WMS client the other day and all i can say is hooray! Finally a flash client which is separated into components, has a neat interface and is actually configurable WITHOUT requiring Flash CS3 just to change the stupid service URI.

Exciting times for FOSSG

For those that know me you should know that i like to think i am impartial to geospatial solutions. I’m not a “fanboi” of opensource or commercial solutions … i really do tend to use the best available product. I have been getting the distinct impression lately that the quality and features starting to be developed in the OSGEO / FOSS4G realm is peaking particular interest in the commercial sector. More and more demonstrations i see by vendors are implementing some or all foss and customising it to their needs which is an interesting shift for the industry. Probably the most common project used in this respect is Openlayers. Just about everyone has used, touched, smelt or developed on OL and is really a testament to the hard work the contributors have put in (particularly the metacarta guys).

Take a look at some of the upcoming features in the stack. Anyone in the web mapping space has got to be excited about these …

  1. “Look into my lines” … AGG support in the new Mapserver 5.0 beta! Its incredible how much difference high quality antialiasing makes to web mapping applications. G/Y/M tiles raised the bar in that respect and Mapserver is certainly catching up to that quality. Mapnik is another choice, but is feature lacking in comparison.
  2. anti_aliased.gif

     

  3. DMSolutions Mapguide Fusion. Wow, such a powerful tool for deploying quick apps. It kinda reminds me how i felt after first seeing the ArcIMS website publisher, although obviously the similarities end there. Customising the old Mapguide clients was a fairly painful experience modifying 20 frames for the layout. Hoping Fusion improves this 10 fold.
  4. Geoserver’s move to 1.6 adds the WFS1.1 implementation and GeoJSON output plugins. Although Deegree and Featureserver have offered these for a while, im always partial having a flexible single application rather than supporting multiple.

Now the hard part for me is deciding which one to start playing with first!

From the other side of the fence …

Like all good arguments, there are other sides. In what seems to be a two part OGC mini-series on Charlie Savage’s blog RE: problems with the OGC, I noticed a comment by Shane asking …

I’m surprised at the lack of outburst by the pro-standards community. This makes me think they are humbled by your views. I’d expect somebody out there to stick up for OGC and the other standards you mentioned. It would be intriguing to hear someone from DM Solutions or Ionic, for instance.

Even though i am not Ionic, DMsolutions … here are some thoughts from someone who thinks the OGC has created some pretty darned useful standards.

A retort to one of Charlie’s post if you will .. :)

Rendering Maps. The argument i often see with the WMS bashing goes something like, “WMS is slow. Who uses it in an Enterprise architecture. G/Y/M dont use it, therefore by my intelligent calculations, WMS must be useless”. Lets back up a second. If you want a high performance, slippy interface that can be easily cached, tiling is certainly your best bet. I get the distinct feeling that a lot of people forget the disadvantages of a tiled mapping cache,

  1. Fixed scales. The amount of people i see who simply generate their zoom levels based on GMaps is crazy. What about what your users want? If there are any papers detailing why splitting the world into 18 distinct zoom levels is ideal, please tell me. I’m yet to find one. One size will never fit all.
  2. Redundant data storage. Active caching mechanisms whereby caches are only populated once browsed is nifty, but it also negates somewhat the advantage of using a cache. Conversely, if you pregenerate your entire cache you are more than likely storing 80% (number plucked from the sky) more data than you need. We also arent even touching the storage of the source data either here, or considering the time required to maintain the cache when you are using volatile datasets.
  3. Lack of integration across clients. The whole benefit of standards it to enable cross-use, cross-communication amongst clients and servers. This is non evident amongst tile servers (beyond of course worldkit and openlayers). Sure, WMS-C / TMS are hopefully gathering steam at the moment, but if you are considering integration right now across a gamut of applications, nothing is better than WMS for transferring maps over the interweb to multiple clients. People seem to be losing sight of this purpose every day.
  4. And the kicker for me … Absolutely no customisation. Dont want that road layer? You better hope they duplicated the cache and removed them otherwise you’re in trouble. Want the map in a useful cartographic projection? Duplicate again! Hmmm, can you colour the cadastre yellow instead of red? No, but i can duplicate the cache again for you. I could go on, but you should get the idea ..

And finally, “Arbitrary bounding boxes” are your friends Mr Charlie! Let your users decide their output scale, not the magical we-chose-18-scales-coz-google-wanted-a-nice-single-square-tile-at-zoom-level-0 :)

Summing up, WMS is your friend regardless. Don’t toss it out with the bathwater just because you are using a cache with a slippy map. Implementing WMS and whatever tiling scheme you can easily abstract *AROUND* WMS will give you and your users the best of both worlds. The fact that you can quite easily use any random WMS server inside a tiling scheme surely highlights that the standard does have flexibility.

Time is at a premium at the moment so i wont reply to all Charlie’s points (especially sharing data because we could be here forever). All in all i can see his point of view however we need to remember that we can only work with what we have at the moment, despite their flaws. GeoRSS/Atom/OWS Context/KML ratifying are all coming, its just up to the rest of us to pick up the ball and keep running with it so this will never be true again,

Web mapping standards are going through a transitional state and haven’t kept up with GIS technology breakthroughs over the last few years.

Chris.

ERMapper cashes in..

Wow, big news.

Leica Geosystems Geospatial Imaging Acquires ER Mapper

I thought it was initially a joke when Jeff posted it a second ago. No release on ERMapper site, nothing on DirectionsMag … but i managed to find the hot-off-the-press feed above.

chips.jpg

The merger makes sense but i’d hate to see the fella’s move off shore. We may be a little off the beaten track here in Perth but we certainly seem to have the brains.

All the best to the ERMapper crew. Hopefully the transition can be a smooth one (so i can still send those annoying feature requests)

:)