December 16, 2008

Land Summary Beta

Earlier this year, I read this post on local foodsheds, and got to thinking that there must be a good way to automate and generalize this type of analysis.  So I put the idea into the big bucket of ideas that I don't have time to implement, until it came up in a conversation with Brent Pedersen.   He mentioned a great  way to optimize the raster summarization that I hadn't thought of, so we got excited, and put it into the bucket of ideas that we'd like to implement (but probably wouldn't). 

Until finally, a few weeks ago, we decided to get together for a Sunday hackfest. A few hours later, with a good percentage of my time wasted learning the strange ways of git, we had a functional demo that summarized NLCD data by class for any given buffer or polygon, in a simple Django framework.

To get things vaguely useable, we added a pie chart to show the classified summary, and integrated some census data to get the population.  Our initial demo is now at landsummary.com

Maybe next week we'll have time to give it some User Interface love.  In the meantime, I'm really happy with how fast the open source stack we used let us get this type of interactive analysis online. 

For the record, we're using GDAL, Numpy, et al for the raster analysis, PostGIS for census and other vector data, Django for the web framework, OpenLayers for the map, and MapServer/TileCache for the NLCD tiled map overlay.  It really says a lot to me that with a surprisingly minimal bit of code we were able to get the basic demo online, let alone in such short time.

Over the last week we've also added some hiddenish features that we will probably expose in an API, and started  some example analyses to help show not only how these types of summaries can be used to visualize patterns in foodsheds, but also that I'm capable of posting to this blog more than once a month.

If you have any ideas for additional datasets you'd like added (topography, census SF3, and detailed historical weather station data are already en-route), let us know.

October 9, 2008

township mapping

At the same time I was taking a great nature hike just before the start of FOSS4G, there was a mapping party in Houte Bay.  The party clearly came out a success, and thanks to Mikel for passing on the contact info of the gracious hosts at Tobi, I was invited out to the area today to continue the work that was started the other week.

This morning, Sebastian, Grant, Belinda, and I started out in Imizamu Yethu (sometimes called IY, or more recently Mandela Park), the local township.  With the excellent guidance of Afrika Mone, we got the main road and a few side streets, stopping by a few local spots, including a tavern, preschool, museum, community center, and a surprisingly nicely set up computer lab.  With a little luck and communication, I'm optimistic some of the kids taking courses there will continue our efforts and get the rest of the township mapped. 

As usual, the few folks I chatted with in the township were friendly, and did not seem to mind some tourists wandering around, though I did get a couple strange looks and shouts as I jogged up a few side streets to get some waypoints.

Leaving IY, we moved on to some decidedly more expensive neighborhoods.  Million dollar houses with fantastic views of both the bay and the township provided an interesting juxtaposition, and  a few more streets in the system.  Overall, it's been another great day in a series of many here in Cape Town.  I'm flying out Saturday morning to see some relatives in Joburg, leaving me with just one day left to find good enough weather for a short paragliding jaunt.

local hooks

The last couple of days have yet again highlighted for me the importance of trying to take advantage of locals and local knowledge.  A few days ago I met Chris -- a local geology/botany expert, and friend of a friend that I haven't seen in years -- for a hike up Lions Head.  It was a perfect evening, and the 360 views of Table Mountain, the Apostles, the Atlantic, the city and suburbs was almost overwhelming.  The perfect sunset didn't hurt either. 

Chris mentioned the local mountaineering club was hosting a slideshow/video, so last night we checked it out.  A couple guys who were part of a group that did a mission to climb the three towers in Southern Patagonia recounted a humerous, personable, and mindblowing story of their trip, bringing back for me good memories of (much easier) trips I've done in the past, and getting me quite excited to get back in shape and outdoors more in the future. 

After chatting with a few of the local mountaineer members, we hooked up with Mark, a friend of my cousin, who took us to see a local jazz band at the Asoka club up the road.  Excellent show, with energy and sounds similar to my recollections of some of Skerik's projects.  One of Mark's friends from LA showed up shortly after, and we had a highly entertaining discussion that reminded me why I love visiting LA, but would never live there.

October 3, 2008

foss4g

The presentations for FOSS4G are over, and just a few workshops and the code sprint are left in the official conference.  This year I ran a workshop on GeoDjango with Chris Schmidt, and gave a presentation on the work I did recently with walkscore.com. 

The workshop had a few technical glitches (no power, internet, or projector for the first bit), and as Chris mentioned, we probably should have practiced together in a fully offline mode beforehand.  On that note, thanks a lot to Chris for agreeing to help out as a replacement at the last minute -- it would have been quite a struggle without him there.  Still, we got through most of the material, and participants ended up with working admin editing for points and polygons, and some public facing pages that showed a good hint of GeoDjango's possibilities.  I learned quite a lot from this workshop, and am looking forward to repeating it in the future.  After fixing up a few  bugs in docs and vmware image, I'll post the tutorial and links to the image somewhere online.

October 1, 2008

fynbos

Yesterday Karen and Hanlie and I went on a nice walk down near Cape Point, with a group of local botany experts who meet once a month to wander around, rip out some invasive plants, and talk endlessly about the indigenous and endemic species.  It was a beautiful area, with views of the Atlantic and the bay, and I learned a lot about fynbos and local history.  I also saw a lot of tortoises, and these really fast caterpillers. 

One of the guys brought along his grandkids, Gabriel and Ruben.  These kids were amazing at finding animals or animal parts (numerous porcupine quills, tortoise shell segments, owl eggs, etc), and it was a lot of fun to jump around on some large rocks again with people who truly enjoy that activity. 

After the hike, I went got dropped off at the University of Cape Town campus, and headed to the Geomatics lab to help set up some of the workshops with the conference organizers.  A couple machines had mysterious issues installing vmware, but overall it went nicely.  I then got dropped off at my hotel, where I'm sharing a room with Andrew Turner.  Unfortunately they refused to let me into the room, despite him leaving them my name and the room key.  So I wandered away to go register for the conference, and met up with some old friends, most of whom I had not seen for a year, and some that I had never met in person.  For me, getting together with all these folks I converse with online is the nicest part of the conference.

September 29, 2008

cape town

I arrived in Cape Town about 5am (dark outside), and stepped carefully back into Africa (stepped/stumbled:  they were unable to locate lights for the path from the plane to the terminal).  There wasn't much of a line for citizens in the passport-checking area, but that didn't stop at least a few people from wandering right to the front of the line, as if to cut, and then just continuing on right past the passport-checkers, who were unable to really notice due to the poor design of the booths they sit in.  Hopefully those guys have no trouble with lack of passport stamps on their next leg, or perhaps it's just a known thing here that there's no need to bother with the hassle of official entrance/exit. 

Next up was a quick check on pricing for transportation:  For two people, it was R170 for the shuttle, or $240 for a cab.  The cab people were clearly not pleased with being undercut, but the shuttle was a great choice.  Despite my misdirections, Chris Holmes and I ended up at his further-away-than-I expected lodgings, and if anyone needs a shuttle driver recommendation, I can now provide.  Nobody answered the door at his B&B, and so we left the less important luggage on the stoop, and went out for a walk in the rain to morning coffee (at Seattle Coffee Company, a local chain).  Then some breakfast (food here is much more expensive than my last visit, but still much cheaper than the States).  Back at the B&B, I fell asleep for a few hours and Chris went to see a movie. 

I woke up and headed out to meet Karen, my cousin who lives nearby, but had forgotten that there was a big security door on the B&B.  Nobody else seemed home when I woke up, and after trying all the obvious looking switches near by, it looked like I needed a key to get outside.  Fortunately, I found a couple windows only a few meters off the ground that did not have bars, so I was able to myself and my luggage on a more unorthodox exit into the yard.

September 5, 2008

southern sierras

 

I had a great backpacking trip in the Southern Sierras (near Mt. Whitney in the John Muir Wilderness) a few weeks ago, and finally decided to see about getting some geocoded pics online.

Since I've been so spoiled with my eye-fi card, I'm not used to needing to manually geocode my pics.  Still, I had my trusty old GPS with me for the trip, and I figured it shouldn't be too hard.  Here's what I did:

  • Used GPS Babel to quickly export a .gpx of my tracks from the trip
  • Ran the gpsPhoto perl script to geocode my photos: 
./gphoto.pl  --gpsfile hike08.gpx--timeoffset 28800 --maxtimediff 3600 --dir ./hike_photos/
  • Note:  the time offset (in seconds) is because my camera and I are set to -8 hours from GMT, and the time diff allows my photos to be up to an hour off from the closest gps point.  This is because I sometimes hiked with with the gps off. 
  • Now that the photos were geocoded, I uploaded them to Flickr, and noticed they weren't automagically put on the map, because I had never enabled Flickr to accept geocoded Exif headers.  Easily solved.

 

Next, I wanted a nicer KML of my actual route.  For this, I used the GPS plugin for QGIS (requires GPS Babel), which made importing directly from my garmin surprisingly easy.  QGIS rocks!  After a little bit of editing/smoothing of my line, I used ogr2ogr to convert the saved shapefile to a kml.

 

Finally, I wanted my KML to show photo links as well.  I was thinking I would make a little Yahoo Pipes thing to get my KML from a photoset, but it looks like the lazyweb already took care this for me, so all I had to do was copy the resulting points into my route kml, and I was done.

July 31, 2008

Mussel Rock - Day 2

I finally had a chance to get my first flight in at Mussel Rock earlier today -- soaring over the coast was everything I had hoped for.  No need for a vario at this place; lift is everywhere. 

So much lift, that when I was ready to land, it took judicious use of big-ears (which is very common to see out here, and clearly helpful to ensure you avoid blow-back) to get past the incredible buoyancy I was getting.  I have no idea why some folks call it cliff-boring .. soaring is a ton of fun, and it's great to be a little more familiar with this site.

July 20, 2008

Eye FireEagle

A few weeks ago, I purchased Eye-Fi's fancy geolocating and wireless-uploading SD Card, which I'd seen a demo of at WhereCamp. I love gadgets, perhaps too much, but this thing is indeed a great implementation of a cool idea. And, as yet another harbinger of how almost everything will soon be location aware, it has all kinds of potential.

For example, devices like this could be used as the poor man's (no iphone) location updater.  Since they haven't opened up their API (allegedly it's on their to-do list), and because I've been looking for any excuse to play with FireEagle, I just finished up a little python script that queries my flickr photos every few minutes and (if there's a new one that eye-fi has uploaded), it grabs the WOEID and updates my FireEagle profile.  Working with the Fllickr and FireEagle API's was fun.

Obviously a bit of a hack, but once they do open their API (so I don't have to pull for data), or otherwise allow us to get access to the additional data they collect (like the strength of and MAC addresses of all access points near where you took the picture), all kinds of other things (OpenSkyHookDB?) become possible.

I'm enjoying not having to even think about transferring my images, let alone geocoding them -- but I'm really looking forward to more of these poor-man's geo devices.    In the meantime, for those if you who check out my FireEagle data - I'm not still at the bar, I just haven't taken any new pictures today.

July 19, 2008

SF Eating: Erics

Ate lunch at Eric's today. Despite their scary restaurant inspection report (EveryBlock reports are yet another nice feature of San Francisco), it was one of the better meals, and deals, I've had in a while.

$6.50 total bill for a huge plate of tasty kung-pao goodness, soups, and tea. I'll be eating lunch here again.

July 18, 2008

Mussel Rock, day 1

I spent the morning at Mussel Rock Park, kiting and getting back up to speed on some ground handling basics with some good tips, and a great site overview from Jeff of Airtime San Francisco.

I've never gone cliff soaring before, and I'm certainly not used to the fog! Starting around noon, there were a few gliders in the air, and I understand the weekends can be packed. If the weather looks good, I'm excited to get some flying time in early next week.

July 17, 2008

Was 80, now 83

When I first saw walkscore.com I was impressed with their simple but clever approach to creating a walk index.

Ranking walkability is an interesting topic for me: I've had the opportunity to work on a few different sprawl and walkability models in the past, but these often tended to be complex. And I don't mean they used tricky or especially unintuitive algorithms (although do believe this type of analysis is often ruined by unintuitive modeling). I mean complex in terms of the data sources. In the world of ranking walkability, many sources (assessor, zoning, sidewalk attributes, etc) that the more academic models tend to rely on are just not comparable from city to city.

To me, one of the key features of the new version of walkscore is that it keeps true to it's initial approach by keeping it simple (business listings and census data), and using a straightforward methodology. By focusing on amenities and where people live, can a model like this be used as a proxy for walkability? Or, perhaps more accurately, for don't-need-a-car-ability?

I think so, but check out the rankings yourself. Of course, I might be biased, since I did have the opportunity to assist with the analysis and visualizations (which turned out to be a truly fun project -- thanks Matt and Jesse).

If you have any questions on the methodology, be sure to stop by the FOSS4G conference later this year, where I'll be giving a talk on some of the methods for data acquisition, analysis, and integration of heatmaps.

July 15, 2008

Moving

A long time ago, I decided I should really start a blog. But hey - it takes some effort. I think. I put it off, promising myself I'd start one as soon as I moved out of Seattle. Until recently, it seemed like I might never have to deal with this first post, let alone come up with a blog title.

But, thanks to Deema (and her job at YouTube), here I am. I won't bore you with details on our move, which was fairly straightforward. One important piece of information I can confirm: drugging cats before bringing them on a long car trip is a very good idea.

Now that I'm getting settled in, I'm excited to start to take advantage of the local tech entrepreneur scene. With any luck, some near-future post will be on the start of a SFUGOS group (in the line of FRUGOS and CUGOS).