Using a HackRF to Capture an Entire Radio System

openmhz

OpenMHz.com

After a bit of work, I have put together a system that lets you monitor the radio system for DC Fire, EMS & City Services. Everything gets recorded and can be played back through a website. Thanks to the magic websockets, any new call that comes in gets added to the top of the list. Better yet, if you hit the Autoplay button in the upper left hand corner, it will automatically play through the list of calls. You can narrow the list of calls display to specific group using the filters.

Anyhow, give it a try at openmhz.com and let me know what you think. If you want more background on how it works, read on…

HackRF
Software Defined Radios are pretty awesome. For the uninitiated, they let you receive a lot of radio signal over a wide range of frequencies. It pass you the raw information and you use a computer to filter out a transmission and process the signal. There is tons of flexibility… and that is also a challenge. There are a lot of rough edges and ways you can mess things up. That is half the fun though, boldly coding where few have coded before.

I was one of the lucky recipients of a great SDR, the HackRF Jawbreaker. It is an amazing piece of Hardware, capable of sending or receiving 20MHz of spectrum between ~13MHz – 6GHz. It is Open Source Hardware too, so if you are wicked smart you can build your own boards. The design work for it was funded by DARPA and I got one of the boards from that pre-production run. In order to go into production, a Kickstarter project was put together. The original target amount was $80k and it quickly blew through that and ended up raising $600k. One of the most impressive stats is that the target price is $300.

You can do a lot of things with this board. It use the popular Osmosdr drivers, so it works with a lot of existing things and plugs right into GNURadio.

There are an endless number of things to try. What I have been focusing on is trying to monitor the radio system that the Washington, DC Fire/EMS & City Services use. Of course monitoring a radio system is nothing new. Radio Shack sells a bunch of different scanners that can do it. However, these scanners can only follow one conversation on a system on a time. Since an SDR can receive a wide swatch a spectrum at once and all the processing happens on the computer, you can decode multiple transmissions. Since you are doing the processing on a computer you can easily save and archive the transmissions, which you sort of needed since you could be getting a couple at once.

Luckily for me, a couple of people have already setup systems that do exactly that. The code from Nick Foster, GR-SmartNet, seems to be the first out there, and the only publicly available code. The Super Trunking Scanner took it a step further and made it playable over the web. It monitor a trunked system with analog channels. The Radio Capture system took a similar approach, except made it work for a system with digital voice channels.

How Trunking Works
Here is a little background on trunking radio systems, for those not familiar. In a Trunking system, one of the radio channels is set aside for to manage the assignment of radio channels to talkgroups. When someone wants to talk, they send a message on the control channel. The system then assigns them a channel and sends a Channel Grant message on the control channel. This lets the talker know what channel to transmit on and anyone who is a member of the talkgroup know that they should listen to that channel.

In order to follow all of the transmissions, this system constantly listens to and decodes the control channel. When a channel is granted to a talkgroup, the system creates a monitoring process. This process will start to process and decode the part of the radio spectrum for that channel which the SDR is already pulling in. In the DC system, the audio is digitally encoded using the P25 CAI process. Decoding it is a bit of pain. I am taking a quick and dirty approach right now and have shoe horned in the DSD program. In the future I would like to try using the code from the OP25 project and a hardware dongle for the decoding. Unfortunately, the dongle is $500, so that might not be happening too soon.

No message is transmitted on the control channel when a talkgroup’s conversation is over. So instead the monitoring process keeps track of transmissions and if there has been no activity for 5 seconds, it ends the recording and uploads to the webserver. I convert the WAV file that gets recorder into an MP3 file. Since the audio is original converted to digital by the radio system, put it through another lossy digital conversion is probably not a good idea, but sending the full-size WAV file ate up too much space.

My Setup
The monitoring and recording is being run off of a laptop in my apartment and uses a crappy antenna. The website is run off a VPS I have running up in the magical cloud.

The webserver is pretty simple. It is written in NodeJs. The audio is stored as WAV files and indexed using MongoDB. The server simply watches for new files being placed in a directory and then moves them and adds them to the DB. Socket.io is used to updated all of the browsers visiting the site that a new transmission has been added.

The Code
The recorder portion of the system is C++ code that uses GnuRadio 3.6.5.1.

The recorder uses DSD to decode the digital audio. Unfortunately DSD isn’t supported or being developed, isn’t designed to work with GNURadio or SDR. Luckily someone wrapped DSD into a GNURadio block. It works, but it isn’t pretty. I had to futz with it a bit to run concurrently. My version is here.

The final portion is the website for listening to the recordings. The code for that is available here.

The Punch List
Right now, I think everything is pretty much stable. I have a small memory leak somewhere, but I can keep it up for long time without it being a problem.

  •  Upgrade everything to GNURadio 3.7. Right now I am on the 3.6 branch and it will take a bit of work to switch.
  • Get OP25 working nicely. The code is designed to work with GR and is being actively developed.

Receving P25 CAI Using a HackRF

I have been playing around a bit and found an easy way to receive and playback digital LMR. More and more of the radio systems are going from narrow band FM to Digital. This makes it tough to listen to them on GQRX. DSD is a great program which can decoded the audio you get when you tune in a system in GQRX, but you either have to run it on another machine with a audio cable in between or figure out how to do a virtual audio cable on your machine.

Luckily, someone wrapped the DSD libraries so they can be accessed as GRC blocks. I have put together a GRC file and a Python program that make it a little easier to use the block.

The files and write-up are here: https://github.com/robotastic/gr-dsd

If you have a chance, give them a try and let me know if they work. There is a good chance I have messed up the values in the filters or else where, but I have gotten clean audio out of it.

HackRF SDR on OSX

I was one of the lucky recipients of Mike Ossmann‘s awesome HackRF Jawbreaker boards. I have a Ubuntu laptop I can play around with at work, but MacBook at home. I am not complaining! I wish I had a Mac at work too.

Anyhow I wanted to get it up and running on my Mac, so I could use it both at work and home. Luckily I found someone had already written up how to install GNURadio and an RTLSDR (a similar SDR) on OSX. All I had to do was fork it and extend it to add in support for HackRF.

I have a write up over at my GitHub. Give it a try and let me know if it works for you!

Pi + Node = Interactive, Internet Connected LED Message Board

While it is great to have the latest headlines up on your message board, it is even better being able to add in a message about how thankful you are that it is Friday. The final touch for my LED Message Board project was to add in a web page that made it easy to update messages. Of course, giving your friends a way to post Star War jokes on your message board only makes it more fun, so I added a Wifi and used HostAP to make the Raspberry Pi look like an access point. To make it even easier to post, I made the Message Board web page a captive portal. So now all one of my co-workers has to do is try and join the “Message Board” Access Point with their iPhone and they are brought to the webpage that lets them add and delete messages. Finally, since it is all Node based and I wanted to play with Socket.Io, I made the message list update in realtime. Now when new messages are added or the ESPN Headline is updated, everyone viewing the webpage gets to see it change.

The Code for this is up here.

Check out how I wired/coded everything and created a Node module to write to it.

I have the LED Modules mounted in a wooden case that is held in my office door frame using magnets. It is positioned so that it shows through the frosted glass next to my door. These picture probably explain it better:

IMG_7794

IMG_7796

Let me know if you do anything cool with signs like this or any HT1632 LED Matrix!

 

 

C++ Node JS Module for SPI Bus LED Signs

In the past couple of posts, I have documented how I built a LED Message board using some modules from Sure Electronics and then wrote a C program to scroll messages across it. This of course cool, but to really make it insteresting you need interesting messages to scroll. To get interesting messages you have to get connected. Since the Raspberry Pi is Unix based, you could easily start piping programs togther. I wanted someting a little more complicated though so I decided to write a Node JS program. In order to write to hardware devices like the Message Board you need to put together a Module and most of the time it will have to have some C++ code to do it, and that is a exactly what I did.

Node JS is an Ascrynous Programming langauge. This means that there is generally only one thing happening at a time and you wait for it to finish before doing the next. If you need to something that might take a long time, (Blocking in Comp Sci terms), you pass it a function to call when it is done and then go onto the next item.

For my module I started with the C program I had written and converted it into a C++ class. Node JS is based on the Google V8 Javascript engine, which is written in C++. Once I the C++ class written, I wrapped it so that it could be used in Node. This was actually pretty tricky to do. The V8 engine has a lot of specialized Macros and structure to help with memory management. My module also had to support running in background so that it could scroll a message without stopping the other things the main program needed to do. Again, this was a bit tricky because you had to make sure you working with only items that would be available in the background context. I don’t have any advice to offer other than Google around a bit.

The final tid-bit is that I wanted to add in support for Events. For me Events made it easier to write a program around instead of using Call Back functions. However for this to work, I couldn’t create a pure C++ module because the Node JS events are Javascript based. In order to get this I had to create a small Javascript wrapper around my custom C++ code that inherited the Event Emitter class.

The basic structure for the overall program is:

  1. Create an array of message you wish to display
  2. Create a function that is tied to the Scrolling Complete event from your Message Board, and will start scrolling the next message in the array. Also update the Array with any new messages.
  3. Start scrolling the First message.

Of course the whole point of doing this was to scroll interesting messages from the Internet. To get those “interesting messages”, I built a couple of tiny Node Module to pull NPR and ESPN Headlines.

So there you have it! A simple way to scroll messages off the Internet and impress (annoy) your co-workers.

Here is the code:

  1. Pi-Led: GithubNPM
  2. espn-headlines: GithubNPM
  3. npr-news: GithubNPM

Raspberry Pi to LED Message Board via SPI

While hooking up my Sure LED Matrix up to my Raspberry Pi using USB worked… it could work better. Using the Driver board from Sure, you can only scroll character by character which doesn’t look that smooth. So instead of using the driver board to interface with the matrixes, I directly connected it to the Raspberry PI so I can control things at the pixel level.

The Sure Electronics LED boards use the HT1632C driver chip. You communicate with the chip using SPI. The RPi natively supports SPI right out of its GPIO port. SPI simply defines the low level electronic communication, in order to talk to the HT1632, you need to use specifically crafted messages, and this takes some programming.

Before we get to that, lets talk about how to wire the LED Matrices together and connect them to the RPi. The Matrices are connected together with 16 wire ribbon cable. All of the boards are connected together serially and then the first board is connected to a bread board you have connected to your RPi. Each LED Matrix has a dip switch on the back that lets you set its ID, (1-4). In the Ribbon cable there are 6 important wires for communicating. There are 4 chip select wires, a clock wire and a data wire. Each of the chip select wires corresponds to one of the Matrix IDs. When you set a chip select wire high, you are communicating to that board. The Data and Clock wires are used to do the communicating.

The only problem is that the RPi only has 2 SPI chip select lines and you have 4 Matrices you need to communicate with. Ugh! I had no idea what to do, so I posted the question on the RPi forum and got a couple of good solutions. It turns out the way you get around this is by using a multiplexer that is controlled separately by some of the other GPIO pins. The simplest approach is to use the 74HC139. It uses 2 lines to select between 4 different output lines, with only 1 being high at a time. The chip actually contains 2 multiplexers, so with one chip and 4 lines, you can have 8 output lines. If you are going to be outputting to more lines, it looks like the 74HC595 is the way to go. It is a little more complex to interface with, but if you chain them together, you can have tons of output lines. Here is how I wired everything up:

Message_Board

I got a couple of parts from AdaFruit to make it easier to put everything together:

With all of this, it was really easy to wire things together. The Pi Cobbler took a little soldering, which was fun. I order the IC and the parts to make a ribbon cable from Mouser:

There is one thing I am still not sure of… how long can the ribbon cable can be. I originally went with a 12′ cable. It was being a little glitchy though, so I now have it down to 6′ and it seems to be working good. You will also need a good 5v power supply to power the LED Matrices. If all four are going, it could pull about 1200MA. I borrowed one from a USB hub. The Female DC Socket from Adafruit, makes it easy to attach a salvaged power supply. Once you have this wired up, it is time to start programming. To keep things simple you could just start with 2 displays and use the 2 chip select lines built into the RPi. The programming is pretty simple. The WiringPi project helps get you start using GPIO lines and SPI.

Once you have everything wired up, it is time to start programming. In order to make the LED Matrices do aynthing you need to send message over SPI. The messages are a series of different patterns of bits. To send a Command to a matrix, you start a message out with the bits: 1-0-0 followed by a number that represents command you want to send (in binary) followed by the setting for that command. To Write data to a matrix, you start a message with the bits: 101 followed by the matrix’s memory address you would like to and then the pixel data you would like to write.

The matrix’s addresses start in the upper left hand corner at 0. Each memory address covers 4 pixels. The 0 address covers the top 4 pixels in the first column. Address 1 covers the bottom 4 pixels of the first column. Address 2 covers the top 4 pixels of the second column from the left, and so on.

Of course having to specify all these addresses would slow things down if you are trying to fill an entire screen. Instead, you can also specify an address you wish to start writing from and then continue to write pixel data to fill in sequential sets of pixels after the one you have addressed. So if you want to write the data for an entire matrix, you start at address 0 and write out all the data.

There are of course a few wrinkles. The first is Endian stuff. Endian determines the order in which the bytes in multibyte variables get stored. When you are constructing Messages in memory using memcopy and then passing the address to the SPI function, Endianess doesn’t seem to be a problem. However when you are construct individual message inside a Word, (which is 2 bytes) and then passing a pointer to the variable’s memory, Endianess becomes a problem. In order to make this work right you have to first write to the variable and then swap the order of the 2 Bytes which make up the Word. Confusing. Long story short, if things are not working, it could be this endian stuff.

The other thing that comes up is that some of the message don’t easily fall across 8-bit boundaries. This generally isn’t a problem and you just end up sending some extra 0s padded on the end of your message. However, when you are trying to write the long blocks of pixel data to a matrix, this is a problem. To construct this message, you have to first write to the message the bits: 1-0-1, followed by the bits: 0-0-0-0-0-0-0 to signal that you want to write to the first address. Now you can just copy your pixel data from where it is cached in memory to the spot in memory where you are constructing this message. The problem is that with the standard functions in C, you can only specify memory copies based upon 8-bits, and with this message you are starting 10-bits in. The solution is to find a way to do bit-level offsets when copying. Doing this is a pain and tricky to do efficiently. Luckily I found a bit of code that explains this.

I have all the code for this up online here: https://github.com/robotastic/SPI-LED-Print It is not pretty, but it should give you a general idea for how to write to an LED Matrix using SPI and using the GPIO pins to multiplex the output. I used this code as the basis for building a C++ NodeJs module that does this. I will post more on that next.

20130213-225418.jpg

20130213-225508.jpg

20130213-225545.jpg

Using Node.JS behind a proxy

I have my Raspberry Pi setup at work and it is stuck behind a proxy. Here are some helpful tips I have found from across the web:

 

Requests from Node

var http = require("http");

var options = {
  host: "proxy",
  port: 8080,
  path: "http://www.google.com",
  headers: {
    Host: "www.google.com"
  }
};
http.get(options, function(res) {
  console.log(res);
  res.pipe(process.stdout);
});

For HTTPS, Try this:


var http = require('http');

http.get ({
    host: '127.0.0.1',
    port: 8888,
    path: 'https://www.google.com/accounts/OAuthGetRequestToken'
}, function (response) {
    console.log (response);
});


In order to use NPM, try something like this:

  1. I run this command: npm config set strict-ssl false
  2. Then set npm to run with http, instead of https: npm config set registry "http://registry.npmjs.org/"
  3. Then I install packages using this syntax: npm --proxy http://username:password@cacheaddress.com.br:80 install packagename

Skip the username:password part if proxy doesn’t require you to authenticate

or this:

npm config set proxy http://proxy_host:port

or this:

npm --proxy http://proxy-server:8080/ install {package-name}

 

 

Approving Eye-Fi Facebook Uploads

I got a Eye-Fi Wifi SD Card for Christmas. It works great on my Canon 5d3 and gives me the wifi uploading that Canon should have baked in from the start (they added it into the cheaper 6d, ugh!). It can upload to a number of online services, including Facebook. I set it up and had it working for the first few uploads, but after that it stopped posting to Facebook. Everything showed up fine in the online Eye-Fi gallery, but nothing went to Facebook.

They are a couple of things you might have to check to get it working. First, make sure the privacy/sharing for the album you are uploading to is set to atleast Friends. Also check the Album, I had a bunch of photos I had to “Approve” because they were uploaded by an App. Hopefully that goes away after I approve it a couple of times. Then goto your settings, then Apps, and then the Eye-Fi App and make sure its privacy/sharing is set to atleast Friends.

It is working flawlessly and it is so cool to be able to hit one button on my DSLR and have it automatically post to Facebook. It is even easier that on the iPhone. I remapped the Rating button to be Protect in order it make it super easy.

Linux Driver For Sure Electronics DE-DD22111

In addition to offering cool LED modules, Sure Electionics also offers a nice Demo / Driver board to go along with them. The only bummer is that the driver board needs a Windows Java program to run if you want to send text to it, to be displayed.

I did some backwards engineering and looked through the design documents, and was able to come up with a couple of Linux programs that let you display text using you computer. It is actually a really cool board and it looks like it is all based on a PIC microchip and could easily be reprogrammed. The PCB board has the traces for a clock, temperature sensor and wireless connection, so it was clearly meant to be added on to.

The board communicates with a computer over USB using a CP2102 USB to UART Bridge. The great is that most newer Linux kernels have driver support baked right in. That means that when you connect the board to your computer, it just shows up as a serial device.

To send Text to the board you need to send it through the serial port ( ttyUSB0 ). Before doing that you have to send the command for write mode. To send text to the first bank of up to 4 boards, you send “0xFE,0×47,0×01,0×01″, followed by 16 characters. You have to send 16 characters, even if you have less than 4 boards or else it goes screwy. All of the scrolling has to be done in your program and you simply change which 16 characters are currently being displayed.

I have my code up on GitHub

and here:

led.tar.gz

$100 Internet Enabled LED Message Board

The Raspberry Pi is a lot of fun to play with but I wanted to do something “useful” with it. The obvious solution was of course to connect something fun to it. I have always wanted to have a message board all to my own so I can display my wisdom and entertain my co-workers.

I could of course cough up a bunch of money and buy an assembled message board or just get a miniature “toy” version, but what fun would that be. Instead I decided to order the different components I needed to build my own.

Sure Electronics is a component manufacturer that sells straight from China. They have great prices and decent documentation. There are of course some rough edges on the documentation and demo code. They are not Spark Fun or Ada Fruit, but neither are their prices.

They sell a number of different LED Matrix. The big differences seem to be the number of LEDs, the size of the LEDs and whether they are single color or capable of multiple colors. I went with the smallest LEDs, 3mm, in green. The modules can be connected together in serial and communicate using SPI. You can have a max of 4 modules, so I got 4.

While the Raspberry Pi supports easily outputting SPI, it still takes a bit of work. In order to get going quickly, I also got a Demo Driver board that lets you control the modules using USB.

Here is what I got:

4 – 8 x 32 LED Matrix, 3mm green ( DE-DP13111 )
1 – Demo Driver Board ( DE-DD22111 )

It took about 4 weeks for the hardware to arrive, but I went with the cheapest shipping option. I am happy with the 3mm LED size. I am just setting it up in a hallway and it is easily readable 10feet away. The larger sized LEDs might have actually been worse because it would have been tougher to read the text close up.

The driver board comes with a number of test functions. It takes in 12v, and provides 5v for the LED boards. I luckily had a Radio Shack variable voltage power supply lying around. I hooked up all of the boards to make sure everything works and I didn’t have any problems.

You can also use the driver board to display scrolling messages by connect it to your computer over USB. The program to do this is available for download here. The problem is that it says it is a .zip file when in fact it is a .rar file. I tried renaming and un-rar the file after I saw another similar file on their site… and it worked. To make your life easier, I have correctly zipped it up and uploaded it.

Sure_LED_1.0.zip

Even though it is a Java program, it only seems to work on Windows computers. In order to get a better sense of what it was doing, I ran the .jar file that is include through a decompiler. Here it is:

NewLcd.src.zip

With the knowledge I gained from that I wrote my one version of the program in Linux. It should work on the Mac, but I think you need to have a serial driver loaded. I will put together a follow-up post on how to write programs to communicate with the Driver Board.