Silly WebSocket mistake

I recently was trying to add a websocket to an existing NodeJS Express app I had written. Here is the important code below:

 

const WebSocket = require('ws');
const app = express();
const server = http.createServer(app);
const wss = new WebSocket.Server({ server });

app.listen(8080, function listening() {
  console.log('Listening on %d', server.address().port);
})


I kept getting 502 errors from NGINX with this code when I tried to establish a connection. See if you can spot the error…

I was pretty dumb. You can’t have app listening anymore. Instead you have to have server. The correct code is:

server.listen(8080, function listening() {
  console.log('Listening on %d', server.address().port);
});

Leave a comment if this happened to you too… I will feel less dumb.

Getting Docker up on Linode, with Ubuntu 16.04

Unfortunately it is not a straight forward process for getting Docker installed on Linode using Ubuntu 16.04. The Image that Linode provides for Ubuntu 16.04 has a custom Linux Kernel installed in it. Docker requires AUFS, which is part of the linux-image-extra package. This package is tied to the kernel version, and since Linode kernel is custom, it is not clear how to load this package. In order to make this work, you have to revert your install back to the distribution kernel.

Here are the steps to get it working:

  • Deploy a Ubuntu 16.04 image on Linode
  • Follow these steps:
    https://www.linode.com/docs/tools-reference/custom-kernels-distros/run-a-distribution-supplied-kernel-with-kvm
  • Then install the Docker-Engine:
    https://docs.docker.com/engine/installation/linux/ubuntulinux/
  • Finally, lock-down and set things up:
    https://www.linode.com/docs/security/securing-your-server

Compiling GnuRadio on a Raspberry Pi

The best bet for most people looking to install GnuRadio on a Raspberry Pi, is to simply do:

$ sudo apt-get install gnuradio gnuradio-dev

…and you are done. Unfortunately, it is not so simple to actually compile the latest version on a Raspberry Pi. While the Pi has gotten faster with the 2 & 3, it is still pretty slow and does not have that much memory, so it takes a long time. If you are good with cross compiler, you can compile on your desktop and ship it your Pi. You are on your own to figure out how to do this… please let me know if you do!

It is not too bad to compile GnuRadio using PyBombs on the Pi. There are a few gotchas though.

The first step is to setup get PyBombs installed. Follow the instructions here. Basically it is:

sudo pip install PyBOMBS
pybombs recipes add gr-recipes git+https://github.com/gnuradio/gr-recipes.git  
pybombs recipes add gr-etcetera git+https://github.com/gnuradio/gr-etcetera.git

Now that you have PyBombs installed, it is time to do some configuration. The first problem you have to work around is that the version of GCC installed from the Raspbian Repo is compiled for the Rasp Pi v1 chip, which has an older ARM processor. This forces some some older C Flags during compile and mess up the GnuRadio build, once it is time to compile Volk. Double check this by doing ‘gcc -v’ on the Pi and looking for:

--with-arch=armv6 --with-fpu=vfp --with-float=hard

The work around is to use the GCC build from the standard Debian repo. I didn’t figure this out on my own. There is a good blog post here. However, before you do this, make sure you have some of the dependancies for GnuRadio installed:

sudo apt-get install libssl-dev libtiff5-dev

To switch to Repos with the ARM7 code,  I edited `/etc/apt/sources.list` and it now looks like:

deb http://mirrors.acm.jhu.edu/debian/ jessie main contrib non-free
deb http://mirrordirector.raspbian.org/raspbian/ jessie rpi
#deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi

If you end up getting weird dependency errors, you can comment out the new lines and enable to old way.

After making that edit, update the package list and install a better ver of GCC:

sudo apt-get update
sudo apt-get install debian-archive-keyring
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install gcc-4.9 gcc-4.9-base libgcc-4.9-dev libgcc1 gcc

After all that, there is still one more step before you can install GnuRadio. The PyBombs recipe for GnuRadio needs to have some C flags added. Edit the `~/.pybombs/recipes/gr-recipes/gnuradio.lwr` file and change config_opt:

vars:
  config_opt: " -DENABLE_DOXYGEN=$builddocs -DCMAKE_ASM_FLAGS='-march=armv7-a -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a7' -DCMAKE_C_FLAGS='-march=armv7-a -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a7'"

I also had to remove some of the old Deb packages from the Raspbian repo:

sudo apt-get remove libqtgui4 libqtcore4 libqt4-xmlpatterns libqt4-xml libqt4-network libqt4-dbus

There is one other thing you have to do… increase the swap size otherwise the Pi will run out of memory with such a large compile. You will probably want to turn this back to the default value after you are done compiling. Follow the directions here.

OK, sweet! Now the now it is time to run `pybombs install gnuradio`. On a Raspbery Pi 2, it took about 24 hours. It might not be a bad idea to run `screen` so thing don’t stop if you get disconnected.

QPSK in OP25 & Fixing Performance

I recently switched over from using a single SDR, with lots of bandwidth to using a bunch of RTL-SDRs each with a smaller amount of bandwidth. The system I am capturing broadcast the same transmission from multiple towers at the same time using something called Linear Simulcast Modulation (LSM). The only problem is that the modulation is a bit different and the multiple transmissions can interfere with each other. With the single SDR I had a Yagi antenna and could point it right at single tower so I would just be receiving one transmission. However, with multiple receivers, it is not so simple and RF splitter amplifiers look expensive.

I had added both OP25 and DSD into Trunk Recorder to capture the audio, but had primarily been using DSD because it had better performance. However, OP25 support QPSK modulation which is what is used with LSM and it is able to make sense of the multiple transmissions. That left me with no other choice than having to track down the performance problems with OP25. The issue was that when an OP25 flowgraph was running but there were no packets going it, it would eat up tons of CPU cycles. I tracked this down to the P25 Frame Assembler Block using the `top` command in linux, with the -tp switch to just look at the threads being generated by Trunk Recorder.

So I don’t fully understand it, but it looks like the problem is with forecast() in p25_frame_assembler.cc. My guess is that with the current forecast it would sometimes return that 0 Input samples are required because of the float to int conversion. The following code below makes a huge different for performance. I have a version of OP25 with this fix available here: https://gitlab.com/robotastic/op25

void
p25_frame_assembler_impl::forecast(int nof_output_items, gr_vector_int &nof_input_items_reqd)
{
   // for do_imbe=false: we output packed bytes (4:1 ratio)
   // for do_imbe=true: input rate= 4800, output rate= 1600 = 32 * 50 (3:1)
   // for do_audio_output: output rate=8000 (ratio 0.6:1)
   const size_t nof_inputs = nof_input_items_reqd.size();
   double samples_reqd = 4.0 * nof_output_items;
    int nof_samples_reqd;
   if (d_do_imbe)
     samples_reqd = 3.0 * nof_output_items;
   samples_reqd = nof_output_items;
   if (d_do_audio_output)
     samples_reqd = 0.6 * nof_output_items;
   nof_samples_reqd = (int)ceil(samples_reqd);
    for(int i = 0; i < nof_inputs; i++) {
      nof_input_items_reqd[i] = nof_samples_reqd;
    }
}

Using Multiple RTL-SDR to Capture a Trunking System

IMG_2192

Most trunked radio systems have the channels they use spread out over a couple MHz worth of spectrum. In order to be able to record more than one broadcast at once, you pretty much need to have an SDR that has enough bandwidth to cover the lowest channel in the system and the highest on at the same time. For the DC Fire & EMS system, the lowest channel is at 854.86250 MHz and the highest is at 860.98750 MHz. This means that your SDR needs to have 6+ MHz of bandwidth. This isn’t too much for most modern SDR, like HackRF, BladeRF or one of the Ettus. These are great SDR, but they cost atleast $300.

The lowly RTL-SDR is a $20 SDR that is just a repurposed USB TV dongle. While it makes for an OK SDR, it only has ~2MHz of bandwidth. When I re-wrote Trunk Recorder, I designed it to support using multiple SDR, but I never really tried it out, so I decided to give it a try.

One of the reasons I was interested was to see if I could get better CPU performance. There is a lot of CPU time dedicated to cutting down the full bandwidth that is coming from the SDR into the small sliver that is interesting. This has to be done for each of the “recorders” that is actively trying to capture a channel. When I was using a single SDR, each Recorder had to take in the full 8MHz and pull out the small 12.5KHz that was interesting. The end results is that I could only record about 3 channels at once before the CPU got overloaded. Since that control channel was going at the same time, that was the equivalent of about 32MHz of bandwidth to process.

With the RTL-SDR, each Recorder only has to look at 2MHz, which puts a lot lighter load on the CPU. Roughly speaking, having 3 Recorders active, plus the control channel would mean that only a total of 8MHz was being processed. As you can see, this means that it scales much more efficiently.

The new code for multiple SDRs has been merged in, give it a try!

Trunk Recorder on GitHub

A couple notes on my setup…

First, start out with an RTL-SDR dongle that has a TXO. This helps make sure the tuning is pretty accurate and will not drift as it gets warmer. These dongles from rtl-sdr.com have been great and the antenna they come with seems to be pretty good.

Also, make sure you assign your dongles each a unique serial number. You can do this with the rtl-eeprom -s command. Using this serial number, you can define different error corrections values for each dongle. I have notice they will very by a few hundred hertz between dongles. There is an example of how to do this in the multi-sdr config file.

The other thing I got was a USB 3 hub. Having more than 2 USB 2 dongle connected to a USB 2 hub might max out the bandwidth. I am not sure if this is necessary, but I wanted to limit the places where things could get screwed up.

Getting iOS Notifications in Linux

IMG_0583

Apple provides access to iOS notifications over Bluetooth LE through the Apple Notification Center Service. It is really a pretty cool thing. I got support for it up and running in Arduino and documented it here. It works great and has been pretty reliable. The problem is that it is not very practical. First off, it really only works with one specific chipset from Nordic Semiconductor. Secondly, by the time you have bought an Arduino and the BLE shield, you have already spent close to $40.  The final kicker is that once you load up the library for BLE and ANCS, there isn’t much memory space left on the Arduino, so you can’t do anything too complex.

This led me to start looking at what was possible on Linux. Given how common and cheap Raspberry Pis are, it seemed like an interesting platform. Sandeep Mistry is doing some great work with Bluetooth LE  on Linux and has even created a sample ANCS app. It is all based in Node.js, which is a fun language to use. Unfortunately, I could never get the ANCS example running on the Raspberry Pi. It also required the iOS device to be running a virtual peripheral through an App like LightBlue Explorer.

In order to be able to work natively with iOS without requiring any app to run, you have to have your Linux system initially look like a BLE peripheral and then pivot to act like a BLE Central. In ANCS the iOS device acts like a peripheral, and the Linux device need to act like a Central. This means combing Sandeep’s libraries for BLE Peripheral and  BLE Central.

One of the other challenges I ran into is that you need to have Bluetoothd initially run, and then kill it. I am not exactly sure why, but it seems to initialize the kernel portion of Bluez.  I am going to look more into this in the future, but for now do a ‘stop bluetooth’ or ‘/etc/init.d/bluetooth stop’ and watch out for respawns.

The code for this is up on github: https://github.com/robotastic/ble-ancs

You should be able to do some pretty fun stuff with this. Since the Raspberry Pi has lots of memory, you can make notifications trigger some complex interactions. Lights can blink, balloons can be popped, and Nerf guns can be fired. And since it is written in Node, it is easy to write complex runs to make sure only VIPs get the balloon popping.

I am not sure which USB Bluetooth LE dongles to recommend. I think it is best to use a Cambridge Silicon Research or Broadcomm based dongle. There is a list up here: https://github.com/sandeepmistry/noble/wiki/Compatible-devices

I have just been using a couple of cheap, random ones I got off Amazon.

MagicBands + RPi + LEDs = Fun

IMG_1061

Disney has made RFID cool with their MagicBands. You can unlock your hotel door, buy a meal and even get into the park just using your band. It was honestly a lot of fun to use… so much fun that I wanted to keep using them when I got home. Luckily, you get to get the bands!

All that I had to do was create a complete MagicBand enabled infrastructure at my house. That seemed tough, so I decided to start small and make some LEDs blink.

IMG_1029

Luckily, someone had already done the tough work of taking apart the band and digging up the FCC papers. The band has 2 radios, and one of the is standard RFID! This makes things quite a bit easier.

Knowing this, I hacked together a MagicBand reader using a Raspberry Pi 2, a Unicorn Hat, and a USB RFID/NFC reader. The RFID reader doesn’t come with software for Linux, but the LibNFC project has everything you need to do some fun stuff and it fully supports the SCL3711.

bloggif_553be432e61b5

It actually didn’t take too much programming to put something fun together. I just hacked together the example program from LibNFC and the demo C program for the Unicorn Hat. I hardcoded in the ID number for my bracelets and play different LED animations when one is read on the RFID reader.

For extra fun, I put the Pi in a shoe box, cut out a window and lined it with velum to make a little viewer. Not sure what it is supposed to be exactly, but it is a lot of fun.

IMG_1096 IMG_1112

With all of this connected together, it is easy to see how it could be extended into something useful. You could create a smart lock, or fire off some internet task when a card or band is touched. It turns out that Washington DC’s SmartTrip card uses standard RFID and you could probably do something pretty fun since almost everyone in DC carries one around.

I put the code up here: https://github.com/robotastic/smartband

This time I am definitely sure it works with 3.7

So everytime I think I have my Smartnet Recorder working with GnuRadio 3.7, I find another bug. This time it had to do with having the control channel to the far edge of the sample range. The control channel was around 854.86MHz, the center was at 858MHz and the sample rate was 8Msps. This put the control channel at the edge of sample but still in range. It worked for a bit, but it would not consistently receive trunking messages. Switch the center to 856MHz fixed it!

Anyhow…. now it will definitely work with 3.7! For Sure!

SmartNet-Recorder & GNU Radio 3.7

I finally got my SmartNet-Recorder software working correctly with GNU Radio 3.7. Or more correctly, I finally squashed the lingering bugs in 3.7, so my code now runs correctly.

There is a 3.7 branch in my GitHub Repo and that will shortly become the Master

Just to capture them correctly, here are the threads from the mailing list:

iOS Notifications with Arduino

ancs

One of the few things I really like about my Blackberry was the red blinky light that let me know if anything new happened. The iPhone is a great device, but it is easy miss that you have a new email or text. Apple has create a centralized place to collect all of these new things you might want to know about, called the Notification Center. However, there is still no red blinky light. If you miss the screen turning on when something comes in, you wouldn’t know that you had anything new.

Apple has added something that can help fix this, Apple Notification Center Services (ANCS). This allows for devices to connect over Bluetooth LE  (BLE), also known as Bluetooth Smart, and receive notifications from the iOS device. It is in iOS 7.0 and most recent iPhones and iPads. ANCS is behind a lot of the new smart watches and fitness bands that do stuff when you have a new email. All this is great, but wouldn’t it be fun if you could do something really big when you get something new?

I thought it would, so I looked for a way to build a Arduino power device that would connect to iOS ANCS. There is a BLE Arduino shield that plugs right in and comes with lots of great code. AdaFruit also has a BLE Breakout Board. They are both based on the great nRF 8001 Chip from Nordic Semiconductor. They provide an Arduino library for their chip and a free software package that makes it easy to configure the features of the chip.

I also came across some great ANCS code for AVRs & Arduinos that was 95% there. I updated it the be compatible with the latest version of the Noridc library, made it easier to pair and rolled it into an Arduino library. I am going to work on getting my code Pulled back up, but until then use my branch to pull down the latest updates.

ANCS Arduino Library

Things are still a little messy, but the overall functionality seems to be pretty stable. You pair the devices once and after that, they automatically join. That alone could be used for some cool functionality. It would be easy to build a box that automatically unlocks when you are near-by or have it play a trumpet sound. This part about ANCS compared to other BLE profiles, is that it works natively and doesn’t require any Apps to be installed on the iOS device. With the BLE Proximity stuff, you need to have an App running on the device to make the initial connection.

Of course the reason for going through this trouble is to get notifications. ANCS pass along all of the Notifications from iOS, including 3rd party ones. You can filter them based on type or source, so you can have a different response depending on what it is. For example, you could do something really annoying for incoming calls, but more subtle for a Twitter mention.

The library works arounds Call Backs and will call the function you specify when a device Connects/Disconnects and sends a Notification.

I have included an example I am working on that displays Notifications on an LCD and turns on the Backlight.

Memory is a problem, so I have some notification fields turned off in order to have a smaller cache.

Also, is Notif a good name? Any thoughts on a better one for the library?