Binders Full Of TIGER Deserts


The U.S. has binders full of TIGER deserts.

Let me explain. Back in 2007, we imported TIGER/Line data from the U.S. Census into OpenStreetMap. TIGER/Line was and is pretty crappy geodata, never meant to make pretty maps with, let alone do frivolous things like routing. But we did it anyway, because it gave us more or less complete base data for the U.S. to work with. And so we worked. And worked and worked. Fixing spaghetti highways. Reclassifying and arguing about reclassifying. Connecting and disconnecting. And now, four years and change later, we have pretty maps!

Eat that, TIGER! (check it out)

But we don’t all live in Epcot Center (actually, I don’t think any of us do really) and there’s lots of places where we haven’t been taking as good care of the data. Vast expanses of U.S. territory where the majority of the data in OSM is still TIGER as it was imported all those years ago.

The TIGER deserts.

I want those all to go away, but there’s only so many folks mapping here in the U.S., so we may want to prioritize a little bit. So I wanted to look at the TIGER desert phenomenon a little closer. In particular, I wanted to look a the parts of TIGER deserts that cover towns, and even cities. That’s where people actually do live, and that’s where OSM data is most valuable.

So I set out to identify those the only way I know how: using ArcGIS. The thing is: this kind of job has ‘raster analysis’ plastered all over it, and I just don’t know how to do that using FOSS4G tools. So maybe I’ll just explain what I did in ArcGIS, and then you all can chime in with smart comments about how to do this in PostGIS, R, GRASS, QGIS or whatever free software can do this magic. If you don’t care for all that, just scroll to the bottom for the results.

I created a shapefile first with all OSM highways with TIGER tags in Florida using C++ and osmium. (There’s some good example code to get you started if you’re interested.)

Then, with that loaded into ArcMap, I first created a 5km grid with the predominant (in terms of way length) version number as the cell value.

A second grid for the neighborhood way density:

I reclassified the version grid generously – all cells with 1 or 2 a the predominant TIGER way version got true / 1, the rest false / 0. For distinguishing between built-up and boondock cells, a threshold of 1.8 looked good after some tweaking.

And finally some simple map algebra to combine the two variables into the final result grid:

So there we are folks – TIGER deserts and TIGER ghost towns in Florida:

TIGER deserts in Florida

TIGER ghost towns in Florida

Hmm. I hope we can figure out a way to improve this analysis so the situation does not look quite so bleak. GIS does not lie though – these are the 5km cells that have a reasonably high way density and TIGER way versions that are predominantly 1 or 2.

So let me know folks – 1) Is this a good approach for identifying TIGER ghost towns and if not, what is? and 2) how do you do this using FOSS4G tools?

About these ads

12 thoughts on “Binders Full Of TIGER Deserts

  1. I recently asked ITO world to provide their TIGER Edited map at lower zooms. Their renderer seems to be struggling with that now which is a shame. Before that though I tried to stitch together an overview map here:

    We can see the effects of the half-finished automated processing again on this view.

    But aggregating stats in 5km cells will give a smarter zoomed out overview. This is cool. You should set it up as a slippy map somehow.

    • Once I figure out how to scale this properly, I might. We could just add it as a tile layer served from the OSM US server which is already set up to do that.

    • Oh yes I remember that. I want to do more periodical stats so we can show more clearly how the map is improving in the US. Will get on IRC about that one of these days.

  2. How long did it take to process FL? It is a reasonably large state with a good number of ways I’m certain. If you can script it in arcPy and easily replicate for all the other states, then it is a perfectly decent means to identify the ghost towns and deserts. The question is, then what? Is it reasonable to ask if Tiger 2k7 in urban areas like Miami is accurate enough for most use cases? Why bother editing something that doesn’t need to be edited?

    • That took about 15 minutes I think, total? And I agree that it is an acceptable way in principle, only my ESRI license is for teaching so I’d have to get in touch with them about allowing this use for OSM.
      The followup question is a good one. Non-edited TIGER in itself is not necessarily bad although it can be really messy – just zoom into any suburb in a Ghost Town and you’ll see what I mean. The bigger problem is that a TIGER desert is quite likely an OSM desert – there’s nobody editing in that area, let alone take ownership / stewardship of the data in those deserts, meaning that there won’t be any rich data (POI, rich attributes) and the data will grow stale over time. That is a problem we can only solve with more local involvement.

  3. My point of view: the tiger import demonstrated effectfully for everybody else in the OSM world how to kill the development of local mapping communities via mass imports.
    if no TIGER import had taken place, the quality of maps would have been worse during 2007-2009, perhaps even 2010, but now it would extend the current level a lot. Just look at “internet bound” countries who did no or just very selective imports.

    For a test i would suggest just to take one of those “deserts” in a well populated area and erase all current data. Yes, totally, blank tile… And see what happens. If no mappers would start to fill the gap i would put TIGER back after 2 month. But i would realy shut my ears about people crying but direct them to the “sign up”-form and the how-to section of potlach and josm….

    • In urban areas you might be correct. But I still think the TIGER import was the only way to get any kind of decent coverage in places like western Kansas and other rural areas to the west. There are just no people out there. And while the US isn’t unique in having areas of low population density I do think we have more infrastructure in these areas. In a lot of places in Kansas you can’t go for more than one mile in any direction without hitting a road of some kind, even if the population density in the area is 1.5 people per square mile.

  4. Being in southern Utah summer 2010 i’ve been quite disappointed by the fact, that the OSM map showed streets, that did not exist in reality and on the other side there were lots of streets not showing on the map at all. Back home I had a closer look into the mapdata (I should have done that beforehand!) and lots of the data was TIGER import. So I started to correct and complete the mapdata using aerial images as a base. Due to the fact, that I have no idea what all those TIGER-tags good for and if they are important for someone or not, I decided to keep them on the new ways. So in the area southern Utah, northern Arizona you have now a lot of ways drawn by me in version 0 or 1, but with the “old” TIGER tags too. St. George, Cedar City, Brian Head, Kanab, Bryce, Escalante, Zion, Hurricane, Fredonia, … and surroundings should be relatively up-to-date. I don’t believe, that there are that many mappers filling the gaps if you delete the TIGER imports from 2k7, not even in towns. i encountered lots of TIGER ways, where a couple of nodes have been touched be other mappers, but the way itself is still wrong, or where only the classification have been changed from residential to track without correcting one single node. This will make the work for your automated TIGER detection quite difficult. Maybe “TIGER-hunting” can become a subject for a Mapchallenge? Every month another area to be in the focus or so.

  5. I assume the beige color is the desert? I definitely find this visual interesting. I’ve been busy the last couple of years reviewing the map in Tallahassee and adding detail.
    I for one find the Tiger import to be very important to my mapping. It am multiple times along getting the Tallahassee map in shape thanks to the tiger import. I also use it to touch up areas that I don’t live and fix MapDust bugs.

  6. It’s been mentioned before (on Talk-US) that we might be better off importing TIGER 2010 (2011, or 2012) for these untouched areas. Given the *vast* improvement in quality since the 2007 product, I would advocate starting from a cleaner base and then working to improve the routing and some of the more local features. What might be damaged in the process are route relations for highways, interstates. Still, it might be easier to manage in a widely distributed environment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s