Luke Berndt

node-xml2js causes “Can’t send headers…”

admin — Fri, 27 Jan 2012 02:50:13 +0000

Ugh! Turns out if you don’t use a new Parser for each request you end up exiting with “Error: Can’t set headers after they are sent.”

Very annoying!

https://github.com/visionmedia/express/issues/711

Installing NodePie on Ubuntu

admin — Mon, 23 Jan 2012 21:06:41 +0000

I was transferring my latest creation over to my hosted Ubuntu server and came across the most annoying error. I had to reinstall the NodePie module because it relies on compiled C code. I kept getting an error though about something called lib-expat.
Anyhow, turns out you need to apt-get a new library:
sudo apt-get install libexpat1-dev

GitHub to the rescue:https://github.com/astro/node-xmpp/issues/33

Transferring from MongoDB to CouchDB

admin — Sun, 09 Oct 2011 17:30:52 +0000

So I wanted to move a larger number of records from MongoDB to CouchDB. Here are the basic steps I took:

I just wanted a selected number of fields (name, address, longitude, latitude) so I used mongoexport to put it into a JSON Array:
./mongoexport -d cookography -c places -f name,address,longitude,latitude -o test.json –jsonArray
Both CouchDB & MongoDB use _ID and MongoExport insists on adding it. However it does it in a way the CouchDB hates so I munge it:
sed ‘s/_id/id/g’ test.json > tested.json
Now you have to format your JSON Array so you can feed it the the REST interface of CouchDB. First add this to the beginning:
(echo ‘{“docs”:’; cat tested.json) > output.json
And then this to the end:
echo “}” >> output.json
Now send this to REST API for CouchDB:DB=”http://127.0.0.1:5984/places”
curl -d @output.json -X POST $DB/_bulk_docs -H “Content-Type: application/json”

a DIV inside a LI

admin — Fri, 02 Sep 2011 03:11:48 +0000

Suppose you want to have a bunches of DIVs contained inside of a List Item… AND you want them to not mess with the height of the List Item and appear as a line of text?

Well then, take this sample HTML:



9-16-2010
License Renewal 
0 Violations

And apply this CSS:

.inspection {

overflow: hidden;

vertical-align: middle;

display: inline-block;

}

li {

margin-left: 15px;

list-style-type: disc;

list-style-position: inside;

}

PostGres – PostGIS on OSX Lion

admin — Fri, 19 Aug 2011 13:07:45 +0000

I recently just wasted a lot of time trying to get PostGIS up and running on OSX Lion. I wanted to do this so I could pull in Open Street Map data for the DC region and then play with it in QGIS and import it into Tilemill. I will be the first to admit that this is a rather complex toolchain, but it shouldn’t have been as difficult as it was.

On the off chance I can save someone time, I will recap what I did… and some of the misstep.

My big misstep was trying to use the precompiled binaries from KyngChaos. While they work fine, the OSM importer for PostGIS always SegFaulted when I tried running it against these versions. I even tried compiling the importer from source code and couldn’t get it work.

Here is what to do:

Use Homebrew. It makes it easy to download and compile source code. You may need to get rid of old cruft you have previously downloaded. I had to remove MacPython and some old Frameworks. If you run “brew doctor” it will give you hints.
Install Homebrew
Install PostGres, PostGIS and all dependancies: ‘brew install postgis’
Initialize the database. This will create a Database structure and a database user (“role”) that is the same as user Mac user name: ‘initdb /usr/local/var/postgres/’
Create a database for GIS stuff (Instead of Luke, use your OSX username): ‘createdb -U Luke -E utf8 gis’
Start the PostGres server – for some reason I needed to specify that it is running on localhost: ‘postgres -D /usr/local/var/postgres/ -h localhost’
Add the PostGIS stuff to the DB: ‘psql -d gis -U Luke -f /usr/local/share/postgis/postgis.sql’ and also ‘psql -d gis -U Luke -f /usr/local/share/postgis/spatial_ref_sys.sql’
Awesome! Now you should have a PostGIS DB loaded up. Now time to install the importer: ‘brew install osm2pgsql’
It is going to fail. This is because you need to apply a patch to fix some inconsistency. Big pain in the ass. To do that, download the patch from here: http://trac.openstreetmap.org/raw-attachment/ticket/3299/patch-build_geometry.cpp
To apply the patch, use the patch command: ‘cd ~/Library/Caches/Homebrew/osm2pgsql–svn/’ and then ‘patch -p0 < ~/Downloads/patch-build_geometry.cpp’
Now try it again and it should work: ’brew install osm2pgsql’
Awesome, almost there. Next step is to download some OSM data. Cloudmade makes it pretty easy. I grabbed the DC data from here: http://downloads.cloudmade.com/americas/northern_america/united_states/district_of_columbia#downloads_breadcrumbs
Goto where ever you downloaded your files to and run (replacing it with your OSM file): ‘osm2pgsql -U Luke -H localhost -d gis district_of_columbia.osm’
TADA! All that wonderful data should be loaded into your PostGIS DB. Now open up QGIS and add in a PostGIS layer. Make sure you use your OSX Username as the username (Luke) and the PostGIS DB you just created (DB).

JSDOM Memory leaks

admin — Mon, 08 Aug 2011 06:07:50 +0000

JSDOM is a great little module for NodeJS which lets you parse a DOM on the server. The only problem is that it has a memory leak. Not a big deal if you are only going to instantiate a couple times. A little trickier if you are screen scraping and need to call it 1000s of times. I luckily found a work around. Instead of creating a new window every time you want to parse some code, simply keep the same window around and switch what it is displaying:

From the google groups:

I’m basically doing this:

var jsdom = require(‘jsdom’);
var document = jsdom.jsdom(‘’);
var window = document.createWindow();

var JQUERY_URL = ‘http://localhost/js/jquery-1.4.4.min.js’;

function foo(text) {
document.innerHTML = ‘’ + text;
jsdom.jQueryify(window, JQUERY_URL, function() {
var $ = window.$;
console.log(document.innerHTML);
});

}

setInterval(function() {
foo(“Text”);

  }, 1000);

And the process’ memory increases and eventually terminates. How do
you suggest I should reuse the DOM?

I am trying this for now:

}

For Looping jQuery Elements

admin — Wed, 03 Aug 2011 17:23:22 +0000

When you get a jQuery object back from a selector, you general loop through all of the elements it contains using .each(). However I wanted to use a For Loop because I wanted to access a bunch of elements at once. I am scrapping a table and I might want to access multiple elements to pull data from.

Figuring out how to do this was a little tricky. Luckily it is very easy to do:

Simply use .eq(index) to access a particular element in the jQuery object.

Completely removing XCode

admin — Sun, 24 Jul 2011 20:57:39 +0000

XCode 4.1 is now free in the App Store. I wanted to completely remove 3.0 version I had. Luckily there is a simple command line option for doing an uninstall:

sudo /Developer/Library/uninstall-devtools –mode=all

Dynamic DIV IDs in Jade

admin — Sun, 19 Jun 2011 14:32:51 +0000

Jade is an HTML templating language that works with Node.js and a bunch of other languages. It takes structured templates and converts them into HTML. Jade uses indentation to show what code should be within an HTML tag.

It is also possible to use variables to generate text, they call it Interpolation and it is a very handy feature. However, it is sometimes a little tricky to figure out how to use variables for attributes in tags.

For example if you want the variable local to serve a link in an Anchor tag, this would work:

a(href=local) Test Link

You can also combine variables and text:

a(href=’http://localhost:3000/page/’ + next_page ) Next

Doing the same for DIV IDs was a little trickier though. Since there is short had for assigning IDs, I thought I could just paste the variable in the shorthand. That didn’t work. However, using an approach similar to the Anchor tag does. Try this for using variables to assign the ID for a DIV:

div(id=permit_id)

Plugin Hybrids

admin — Fri, 10 Jun 2011 01:59:38 +0000

It is always fun to head out to lunch and find the latest in Hybrid technology.