Creating a GeoCMS in Drupal with Custom Map Tiles

So somehow I stumbled across the idea that I would like to display different nodes on map. Instead of listing different post sequentially in a list like a blog, I want to plot them on a map based upon some sort of geographic tagging. The whole reason I wanted to do that is so I could map businesses opening and closing in my neighborhood. The easy thing to do would be to simply use a custom Google map… but what fun would that be! On top of that I want to come up with custom map tile so that it doesn’t look like a generic Open Street Map.

The guys over at Development Seed are the leaders in custom cartography and integration into Drupal. Their Stumble Safely app is a great example and the tile set they generated for it is what motivated me to craft my own. Luckily for the community, they have their process for building the tileset petty well documented. Enough rambling, lets get started!

The first thing I did was start looking at the tutorial Tillmill. It is a tool the Development Seed put out that lets you use the Amazon Cloud to build your tileset. I never quite got it working and ended up rendering everything locally. The process for getting everything setup is the and there is great background reading on everything you need to do. Lucky for me, Dev Seed is based in DC so the sample they have for DC was really helpful.

Step 1: Find some Geo Data

Files used:

Dev Seed DC Map Sample File

The sample has most of the Shapefiles you need for DC. If you want to do everything from scratch, here are some to get you started:

DC Government has put out lots of GIS files for a variety of things. We are going to use this data in building a custom map of DC. If you search the catalog for “DC GIS” you should get a list of the different data sources that provide an ESRI Shapefile. Later on you will be layering these shapefiles to create the map. Here are some of the basic shapefiles you might need to get started, you can layer additional stuff on top depending on what you are trying to show:

Boundary – Outline of the District
Sidewalks – All of the sidewalks
Waterbodies – All of the streams and lakes
Building – An outline of all the buildings in DC
Roads – All of the roads

Strangely enough it looks like the DC layers do not provide the names of streets in the Roads shapefile. Luckily you can get them from Open Street Maps, an crowd sourced effort to create open map data. CloudMade is a cloud based app for creating a custom tile set. It may actually provide you with everything you need to build a custom tile set… if you are a wimp and want to use a web interface instead of the scripts, the command line and a bunch of wonky XML. They are nice enough to provide easily downloadable Open Street Map data. Grab the DC Shapefiles. You want to use the highway shapefiles from inside the zip, note that a shapefile is actually a bunch of files with the same filename and some random extensions. If you use OSM data in your map, you are supposed to be a good Doobie and include attribution.
Once you have all the shape files you need, it would be a good time to learn about projections. Projections are the method used to take something round and turn it into something flat. Each layer could have been created with a different project, so it is important remember what each is. The DC data is in Maryland State Plane coordinates NAD 83 meters, which is a rather wonky projection. The OSM data is in a more standard Google Mercator projection, also known as Spherical Mercator. Right now none of this should be making sense. Don’t worry I show you where to fill it in later.
Step 2: Viewing the Layers
Files Used:

Quantum GIS

Now that you have a nice collection of layers you are going to want to take a look at them. Quantum GIS (QGIS) is a nice open source package that provides a lot of the GIS features that the more expensive ArcGIS provides. Most importantly it lets you open and view ESRI Shapefiles. Get QGIS setup on your machine and launch it. If you haven’t done so yet, un-zip all of the shapefiles you downloaded.
Now start adding the layers you download. Begin with the base layers like the Boundary file and work your way up.
To add a layer in QGIS:

Layer -> Add Vector Layer
A window should pop-up, hit the Browse button to find your shapefile.
A Layers pane should be open on the left-hand side and the layer you just added should appear there. Right-click on the layer and open its Properties.
In the Properties window that opened, click on General and then click the Specify CRS button.
This brings up the Coordinate Reference System for the layer.
1. For the DC GIS Layers search for an ESPG of 2804 – it should return NAD83(HARN) / Maryland. For the OSM Layer search for an ESPG of 4326 – it should return WGS 84.

Now repeat this for all for layers. The order I went in was:

Boundary
Water
Building
Sidewalk
Road
Street names

Now that you have everything load into QGIS, it is time to set the Projection you would like to have.

File – > Project Properties
A window should open up, click on the Coordinate Reference System tab
Search for an EPSG of 3785 – it should return Popular Visualization CRS / Mercator.

Ok, now you are all setup to view the different layers you have. Make sure you save it! Now it is time to style your map.

Step 3: Styling your Map

So here is where things start to get interesting.This is also where still have lots to learn. Luckily the smart guys at Development Seed have some good stuff:

Here are the basics; Mapnik is what is used to generate custom made maps. You can specify what layers you want, what portion of the map you want to generate, what levels of zoom should be included and how you want to color and draw each layer. Mapnik takes in an XML file which describes the different Layers to take in and a Style file which describes.

If you are using Mac, here is an easy installer.

If you know your way around code and you know exactly how you want to style your map, you can simply hand code both of these files. Quantumnik is a plugin for Quantum GIS that lets you style your map in realtime, preview how Mapnik would render and then spit out the Mapnik XML and style file. The one thing Quantumnik can’t handle is having different map styles depending on how far you have zoomed in. If you are creating a map where people are going to be zooming in and out a lot, this can be a problem. If someone is viewing an entire city, you don’t want to be showing the location of every single trash can.

What Quantumnik is great at is letting you play around with different colors, fonts and font sizes. Some of this is tough to visualize and it can take a while to render tiles. Quantumnik lets you experiment with all of this. If you plan on having a limited range of zooms, you might be able to simply use the Mapnik XML file Quantumnik spits out.

Cascadenik is another tool. It lets you use Cascading Style Sheet like style files and a simplified XML format to more easily handcode Mapnik XML files.

I ended up using Quantumnik to come up with the colors, layers and labels I wanted. I took the XML file it generated and then used that to create a custom Mapnik XML file by hand using Cascadenik. Of course, check out the good info on Mapnik files at Dev Seed. I will do a quick run through on the files I came up with.

First up is the .mss style file. In this file you define styles to go along with different objects in layers. Here are some of the interesting bits:

dc-light.mss

.building[zoom=14] {

  polygon-fill: #999999;

}

.building[zoom>=15] {

  line-color: #737373;

  line-width: 0.2;

  polygon-fill: #999999;

}

.building[zoom=16] {

  line-width: 0.3;

  polygon-fill: #999999;

}

The interesting thing in this snippet is that you can specify different styles at different zoom levels for a single element. Another interesting thing is that when you get zoomed in, pervious styles still apply for that element unless you override them. That is why at zoom level 16 you do not see line-color being set.

.road.fill[DESCRIPTIO=”Intersection”],

.road.fill[DESCRIPTIO=”Road”] {

polygon-fill: #B3B3B3;

}

.road[DESCRIPTIO=”Hidden Median”],

.road[DESCRIPTIO=”Hidden Road”] {

/*polygon-fill: #0F0F0F;*/

line-dasharray: 4.7, 2.7;

line-width: 1.5;

outline-color: #fff;

outline-width: 0;

}

.road.fill[zoom>=14][DESCRIPTIO=”Parking Lot”]{

polygon-fill: #BFBFBF;

}

.road.fill[zoom>=14][DESCRIPTIO=”Alley”],

.road.fill[zoom>=14][DESCRIPTIO=”Paved Drive”] {

polygon-fill: #CCCCCC;

}

So here you can see where things get real interesting. You will notice, [DESCRIPTIO=”Alley”] this lets you define different styles for a single layer based upon fields in that layer data. This is great! It lets you take a complex layer with a lot of different objects and give each of them different style. It also lets you chose which portions of a layer should be visible based upon zoom. If a layer is zoomed out, you may not want all of the features to be visible. You may also note that you don’t have to define every zoom level, you can just do greater than or less than.

Here is one more example:

.water.major[DESC_!=”POND”] {

  line-color: #9CA1DB;

  line-width: 1;

  polygon-fill: #B3B3EC;

}

.water.minor[DESC_=”POND”] {

  polygon-fill: #B3B3EC;

}

dc-light.mml

This file defines the projection for the overall map projection and the different layers to include in the map. Here are some snippets:

<!DOCTYPE Map[

  <!ENTITY srs900913 “+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs”>

  <!ENTITY srsWGS84 “+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs”>

  <!ENTITY srsDC “+proj=lcc +lat_1=38.3 +lat_2=39.45 +lat_0=37.66666666666666 +lon_0=-77 +x_0=400000 +y_0=0 +ellps=GRS80 +datum=NAD83 +units=m +no_defs”>

]>

Here is the start of the mml file. You first define the different projections that you will be using for your map and for the layers.

<Map srs=”&srs900913;”>

<Stylesheet src=”dc-light.mss” />

<Layer class=”building outline outer” srs=”&srsDC;”> <Datasource> <Parameter name=”file”>./Layers/BldgPly/BldgPly</Parameter> <Parameter name=”type”>shape</Parameter> </Datasource> </Layer> <Layer class=”building outline inner” srs=”&srsDC;”> <Datasource> <Parameter name=”file”>./Layers/BldgPly/BldgPly</Parameter> <Parameter name=”type”>shape</Parameter> </Datasource> </Layer> <Layer class=”building fill” srs=”&srsDC;”> <Datasource> <Parameter name=”file”>./Layers/BldgPly/BldgPly</Parameter> <Parameter name=”type”>shape</Parameter> </Datasource> </Layer>

Here is where you actually start to define the Map and Layer. The srs attribute defines the projection to be used for the map or layer. The ampersand means that it is embedded in the document.

Both of these files get combined by Cascadenik into a final Mapnik XML file. Now that you have a Mapnik XML file, it is time to make some tiles! It is pretty easy to do, but it requires you customize a Python file to match you setup.

To get started, download the generate_tiles.py file from open street maps. You can use svn, if you have it and know how to use it. I simple browsed over to:

http://svn.openstreetmap.org/applications/rendering/mapnik/generate_tiles.py

Save the file to the same directory you have you Mapnik XML file in and open it in a text editor.

First stop, start by correcting the number of threads to match how many cores you have in you machine, probably 2 or 4

NUM_THREADS = 2

Next the file paths, take this complex statement and point directly to the files:
Original:try:   mapfile = os.environ[‘MAPNIK_MAP_FILE’]   except KeyError:   mapfile = home + “/svn.openstreetmap.org/applications/rendering/mapnik/osm-local.xml”   try:   tile_dir = os.environ[‘MAPNIK_TILE_DIR’]   except KeyError:   tile_dir = home + “/osm/tiles/”
Updated:   home = os.environ[‘HOME’]
   mapfile = home + “/Programming/Maps/Custom Map/dc-light.xml”    tile_dir = home + “/Programming/Maps/Tiles/”

Finally, describe the bounds for the map. Clear out the old stuff and describe what it should be for your map. Getting the bounding box correct was a huge pain in the ass and pretty un-intutitive. You want to use Lat – Long to describe the box. The ever capable script will convert the Lat Long project into the Merc projection we selected for our map. I wasted a lot of time trying to figure out how to specify the bounding box. I either got stuff that was out there, or nothing at all. The problem turns out that you specify things differently than you would for other graphics programming. Instead of using the upper-left corner and bottom-right corner, you use the bottom-left corner and the upper-right corner. Bounding Box on OSM Wiki

To find these points the easiest thing to do is pop into Quantum GIS and look up the points for your map.

Here is what I used for DC. My map has a bunch of zoomed in zoom levels:

minZoom = 10

maxZoom = 18

bbox = (-77.065,38.89,-77.01,38.94) #(-8577985,4708655,-8574353,4710851)

render_tiles(bbox, mapfile, tile_dir, minZoom, maxZoom,”DC”)

Once you have made all these changes you should be able to start making tiles. Try running generate_tiles.py from your command line and see what happens. It should start spitting out tile PNGs into the directory you specified. Depending on the size of the map you are producing and the number of zoom levels, it could be running for a bit.

After a while you should be left with a ton of tiles. The next step is to get them online… and this will be covered in the next post because this is getting really long.

Creating a GeoCMS in Drupal with Custom Map Tiles

Comments

One response to “Creating a GeoCMS in Drupal with Custom Map Tiles”