Wednesday, March 29, 2017

PrivateInternetAccess VPN on a Ubiquiti USG (Unifi Security Gateway)

Big news this week, as the Republicans in Congress decided to scrap an FCC rule known as the Broadband Consumer Privacy Proposal which required broadband providers to get permission from subscribers before collecting and selling data collected about their users.

Since I am very interested in my online privacy, or at least, I like to have the option to choose when to share my information for myself, and since I recently upgraded my home router to a Unifi Security Gateway from Ubiquiti Networks, I wanted to know if the VPN client would be compatible with the Private Internet Access VPN that I use to protect my privacy, thereby putting my entire house behind the VPN all the time.

Posts in the UBNT Community Forums seem to have a lot of confusion, or are just outdated.

It turns out the setup for a PIA VPN configuration is very easy.

The only thing that posed any challenge was calculating all the routes for all the subnets outside my house, to route that traffic over the VPN. In my case, since I use RFC1918 space, here is the list of routes I needed to add to the USG, via the "subnets" menu item in the USG settings app:


Since hosts have a default route to the USG (, all traffic will make it to the USG just fine. Now... the USG has a default route to the internet via my ISP. The default route is, which is the least specific route possible to have in a routing table... a route to every IP possible. In routing, more specific routes always win. So the USG also has a local route to, which prevents my internal traffic from following the default route. And the USG has a more specific route to it's gateway than default as well, due to it being a connected network so it won't get lost in the routes above.

The list of subnets above provides a more specific route than the default route for every possible IP that is not in my house, which forces everything to be sent across the VPN, but they are still the least specific possible routes to everything, which means they're pretty easy to override if I don't want something going over the VPN. After all, the VPN is pretty limited on bandwidth compared to going directly out FiOS.

This list is everything that I don't use in my house, and ensures that any traffic to anywhere outside my house will be routed over the VPN. And, Yes, I am aware that there are other blocks of RFC1918 and RFC5737 space, but since ISPs don't route those networks, I'm not worried about them, because the VPN essentially acts as a sink for any traffic to those destinations.

Here is how the settings go into the USG configuration in the Unifi controller application:


  • Purpose: VPN Client
  • VPN Client: PPTP
  • Enabled: check this when you want the VPN to go live
  • Remote Subnets: one entry for each of the subnets in the list above (modified for your own use, if you don't use 192.168.x.x in your house/business)
  • Server IP: get this from PIA, I used `nslookup`
  • Username: your PIA username
  • Password: your PIA password
  • MPPE: Yes. You definitely want to have your VPN connection encrypted.

Enjoy your ISP not selling your internet activities to advertisers.

Sunday, March 19, 2017

Waking up gently with Sonos and Sirius XM.

We (my wife and I) have been using LIFX lights in our bedroom to simulate a sunrise.  They come on at sunrise, and slowly increase brightness for 30 minutes, allowing us to get used to the light, and wake up pretty gently, as opposed to being jarred out of a deep sleep by a more traditional alarm clock.

My wife asked if there was any way we could do the same with Sonos.  Specifically, she wants to pick a Sirius XM channel like "15 - The Pulse" to wake up to.  Have the volume start at 0, and over the same 30 minute period as the lights, ramp the volume up slowly until it's a reasonable level coinciding with the maximum brightness of our lights.

Her ideal solution would have the following features:

  • Pick any Sirius, Pandora, or Calm Radio station that Sonos can regularly access.
  • Choose a maximum volume for the alarm
  • Choose a length of time over which to go from 0 to Max volume
  • Orchestrate the details via an iOS app on iPhone or iPad.

For Extra Credit:

  • Do the same thing in reverse, allowing from from X - 0 over time, like a slow ramp down sleep timer.

We first tried the Alarms available in the Sonos App.  These are time and content alarms, meaning I can set it to play a Sirius XM channel, at a specific time, at a specific volume.  There is a fade-in, but it's only 15 seconds long.  Not exactly what we're looking for.  We want something more along the lines of a 30 minute fade in.

Google seems to indicate that this is a common request from Sonos users:

I did end up finding, a Python library for interacting and controlling Sonos speakers.

This library would allow me to hit about 2.5 of the ideal features, and possibly the extra credit as well, if I wrote a little program to run from cron on a Linux server.  

Easy to do in cron:

  • Run a program at a specific time 

Can do with SoCo:

  • Set volume of a Sonos speaker, or a group of speakers
  • Pick a channel to play

Can't easily do with Soco/Cron/Linux:

  • Control via an iOS app on iPhone/iPad.

Added Feature:

  • I can log in and `touch /tmp/holiday` if I want the alarm to not go off tomorrow.

So, I'm still on the look out for an iOS app that will let me orchestrate all this, at least until Sonos adds this kind of feature or one of the other home automation apps adds it.
Here's a link to my alarm script:

Thursday, April 9, 2015

Building OpenVPN on Solaris 11.2 for use with PIA VPN

I recently had a desire to get OpenVPN working on Solaris 11.2, to allow me to connect to a Private Internet Access (PIA) VPN. For more information on using a VPN for general internet access, as well as some insight into why you might want to look into it for yourself, see:

A quick Google turned up a blog post from Stefan Reuter detailing how to set up OpenVPN on OpenSolaris 2008.11.

For Solaris 11.2 the basic steps are still pretty much the same, but some of the minor details have changed. We still need the TAP driver for solaris, and we obviously need to download and build OpenVPN, but we don't need to edit the TUN/TAP Makefile anymore, and we don't need any patches for OpenVPN. One step I added was to download and compile the LZO compression library for OpenVPN.

Step 1: Install the TAP driver.

# git clone
# ./configure 
# gmake
# sudo gmake install
  • The full output of running those commands, if you are in any way possibly curious.

Step 2: Install the LZO compression library.

# wget
# tar -zxvf lzo-2.09.tar.gz
# cd lzo-2.09
# ./configure 
# gmake
# gmake check
# sudo gmake install
  • More full output of running those commands, if you are in any way possibly curious.

Step 3: Install OpenVPN.

For OpenVPN, we modify CFLAGS and LDFLAGS, to let OpenVPN find the LZO library we just installed, and we add '--enable-password-save', which will allow us to store the username and password for the VPN in a file.
# wget
# tar -zxvf openvpn-2.3.6.tar.gz
# cd openvpn-2.3.6
# CFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib" ./configure --with-gnu-ld --enable-password-save
# gmake
# sudo gmake install
  • Yet again, even more full output of running those commands, if you are in any way possibly curious.

Once OpenVPN is installed, configuring it for use with Solaris is relatively straight forward. PrivateInternetAccess have a bunch of OpenVPN configuration files, with some very useful defaults. Since I'm on the East coast of the US, I started with the "US East.ovpn" file:
dev tun
proto udp
remote 1194
resolv-retry infinite
ca ca.crt
remote-cert-tls server
verb 1
reneg-sec 0
crl-verify crl.pem
To which I added a few options of my own:
auth-user-pass .pia.login
script-security 2
route-delay 2
The auth-user-pass .pia.login line tells the OpenVPN client to read your username and from a file in the current directory called '.pia.login' (Make sure your path is correct if you have issues). The contents of that file are your username by itself on line 1, and your password by itself on line 2.
The rest of the lines all affect how routing is done for the VPN. Left to it's own devices, OpenVPN doesn't have the code necessary to automatically manage routes. For example, it can't automatically determine the default gateway, and modify that route to update the default gateway to the VPN's default gateway.
Wed Apr  8 00:54:10 2015 NOTE: unable to redirect default gateway -- Cannot read current default gateway from system
The solution for that is to use a route-up script to handle the routing. In order for OpenVPN to use the script, you need to set script-security 2, or you see show-stopping warnings such as:
Wed Apr  8 01:09:00 2015 WARNING: External program may not be called unless '--script-security 2' or higher is enabled. See --help text or man page for detailed info.
Wed Apr  8 01:09:00 2015 WARNING: Failed running command (--route-up): external program fork failed
With script-security set to a reasonable level to allow OpenVPN client to run scripts, we use route-delay 2 to tell the client to give the client 2 full seconds to get the VPN tunnel set up before doing anything with routing, and route-noexec tells the client not to make any direct changes to the routing tables, and the route-up tells the client to run a script, which I very imaginatively called, during the route-up phase of client activity. The contents of the script look like:
#!/usr/bin/env ksh

# OpenVPN passes the remote gateway in as $route_vpn_gateway.
/usr/sbin/route add $route_vpn_gateway
/usr/sbin/route add $route_vpn_gateway
Since more specific routes are always preferred over less specific routes, setting these two routes allows us to route everything over the VPN without having to make any changes to the default route, thereby bypassing OpenVPN's lack of ability to manage Solaris routes. If the VPN goes down, the routes are removed, and you still have access to the internet via your existing default route. You also maintain access to your local LAN because that route will be even more specific, and it's directly connected. You will just not have the same amount of privacy at that point.

Monday, April 6, 2015

pyTivo on Solaris 11.2

We are a TiVo household, so a quest has been underway to build a suitable place for long term storage of the family's favorite TV shows and movies. One indisputable requirement is that the shows and movies have to be visible via the TiVo menu. pyTivo (the William McBrine fork) is the logical tool to do this (in my house). William McBrine has been maintaining his fork of pyTivo more regularly than the original package (sourceforge).

To get pyTivo working on Solaris 11.2, only 2 dependencies needed to be resolved.
  • I needed to build ffmpeg to support on-the-fly video transcoding. and,
  • ffmpeg wanted yasm(an open source rewrite of the nasm assembler) or nasm itself.
This is what happened when I tried to build ffmpeg without yasm:
bash-[121]$ ./configure --prefix=/usr/local
yasm/nasm not found or too old. Use --disable-yasm for a crippled build.

If you think configure made a mistake, make sure you are using the latest
version from Git.  If the latest version fails, report the problem to the mailing list or IRC #ffmpeg on
Include the log file "config.log" produced by configure as this will help
solve the problem.

Both were very simple and straight forward to build and install:

  • Yasm
  • [tim@tank]-[117]$ ./configure --prefix=/usr/local
    [tim@tank]-[118]$ gmake -j4
    [tim@tank]-[119]$ sudo gmake install
  • ffmpeg
  • [tim@tank]-[127]$ ./configure --prefix=/usr/local
    [tim@tank]-[128]$ gmake -j4
    [tim@tank]-[129]$ sudo gmake install

Once ffmpeg was installed, I updated the pyTivo.conf file and set the location of the ffmpeg binary, and pyTivo worked beautifully after that.

[tim@tank]-[136]$ vim TIVO/pyTivo/pyTivo.conf
# FFmpeg is a required tool but downloaded separately.  See pyTivo wiki 
# for help.
# Full path to ffmpeg including filename
# For windows: ffmpeg=C:\pyTivo\bin\ffmpeg.exe
# For linux:   ffmpeg=/usr/bin/ffmpeg


For more information on pyTivo, including installation, configuration, and other tasks outside the purpose of this post:

Wednesday, March 23, 2011

Building DBD::mysql on Solaris 10 Sparc

Having problems building the Perl DBD::mysql modules on Solaris 10 Sparc 64-bit? The Perl 5.8.4 binary that ships with Solaris 10 is a 32-bit application.  You are probably running the 64-bit version of MySQL and trying to build DBD::mysql against that db version. What you actually need to do is download the 32-bit version of MySQL, for linking the Perl DBD::mysql libraries against.   I run the 64-bit MySQL database in /opt/mysql/mysql, so I unpacked the 32-bit MySQL as /opt/mysql/mysql32. Then, run a CPAN shell, look DBD::mysql, and build the module.
/usr/perl5/5.8.4/bin/perlgcc Makefile.PL --libs '-R/usr/sfw/lib \
-R/opt/mysql/mysql32/lib -L/usr/sfw/lib -L/opt/mysql/mysql32/lib \
-lmysqlclient -lz -lposix4 -lcrypt -lgen -lsocket -lnsl -lm' \
--cflags '-I/usr/sfw/include -I/usr/include -I/opt/mysql/mysql32/include'
gmake install UNINST=1
and you're done.

Tuesday, December 7, 2010

logging shell commands to syslog on secure systems

I had recently come across a blog post describing methods for capturing commands entered on the command line, and recording them to syslog.  Either by function() or by patching the actual shell itself.   I found this article because I was asked by my boss to find a way to add CLI logging to some hosts on our network, to support audits and accountability.

Some of the environments I work on are more secure than usual.  In a typical corporate environment, whether internet connected or not, there is generally no need or requirements to use system auditing to track all user actions.  Some government systems, whether classified or not, do require this, and some commercial systems in regulated industries, or who service government agencies, also require this level of auditing and accountability.  In some cases it can be a smart idea for non-regulated systemd.  For instance, if you're a managed services company that uses a team of operators to manage multiple customer environments, there may be some value to tracking user activity.

Most operating environments these days come with some sort of auditing facility, however I have found that these are usually fairly unintuitive, and most people that implement auditing do so by following a How-To, and then end up not actually having a SME on staff when things do go wrong.  Audit logs can also consume a lot of space, so lots of sysadmins just delete old audit logs in an effort to reclaim disk space.

One easy, and portable, way to quickly and intuitively audit user activity involves using patched shells to send all the commands run to syslog.  I should note that there are a few weaknesses in logging all commands to syslog, such as password exposure.  Some people do put passwords in the command line of such tools as ldapsearch, or mysql, or sqlplus.  Those passwords will then be recorded in plain-text in your system logs.

And there are always ways to work around being logged, like running a shell that doesn't log to syslog, which can be done as simply as by uploading your own, non-logging, shell and running it.

Aside from the weaknesses outlined above, though, in an environment where users are not malicious, and where team members can find themselves on any of a number of systems at any particular time, logging user can provide very valuable context in a familiar way.  And quickly, too.

Using syslogging shells is simply a tool like any other.  It doesn't replace real system auditing, but it definitely has it's place.

GNU Bash 4.1 has all the code to enable command logging, simply by editing the config-top.h file.
Just change:

/* Define if you want each line saved to the history list in bashhist.c:
   bash_add_history() to be sent to syslog(). */
/* #define SYSLOG_HISTORY */ 
#if defined (SYSLOG_HISTORY)
/* Define if you want each line saved to the history list in bashhist.c:
   bash_add_history() to be sent to syslog(). */
#if defined (SYSLOG_HISTORY)

And all commands in interactive shells are logged. (don't forget to add the other 9 official bash patches to get to code level 4.1.9)

One thing I did notice is that in Solaris, the PID was being logged with the log entry to syslog, however this was not the case in linux.  Rather the PID was being logged by the log entry %s itself.

if (strlen(line) < SYSLOG_MAXLEN)
 getpid(), current_user.uid, line);
Resulting in log entries like:
Dec  7 23:13:02 linux bash: HISTORY: PID=1752 UID=1001 ls
I don't really like that format, either. I'd rather see usernames and commands, and have the pid over on the left with the 'bash:'. This was a pretty simple change in code to:
  if (strlen(line) < SYSLOG_MAXLEN)
    syslog (SYSLOG_LEVEL, "[%s] %s", current_user.user_name, line);
This results in log entries that look like:
Dec  7 23:26:39 linux bash[1846]: [tkennedy] ls
To me, this is a much more readable log file. Perhaps that's because I'm used to the log format that the BOFH patched tcsh shell, which we also use, uses. Now bash and tcsh log in identical formats. Our users have been informed that bash and tcsh are acceptable for interactive shells on Linux, and there were no exceptions. On Solaris we encourage the use of bash or tcsh for interactive shells in the hopes that consistency lends itself to stability, although we use the RBAC aware pf- shells for role accounts like 'oracle' which encourage ksh.
Here's my patch to bashhist.c that logs entries the way I like them:

Friday, November 19, 2010

A generic perl script to scan a CIDR subnet for listeners on a specific port.

Ever had a customer ask you where was running on in their network? I have. And usually this involves an environment that doesn't have NMAP installed, or any other common port scanning tools. Fortunately these days, almost every *nix OS comes with Perl, even Solaris.

Since I work for a managed services company, and we manage a multitude of different environments, each with it's own set of restrictions and requirements, I try to wrote the most portable code that I can, so that it has the best chance of actually working in any given environment.

This script uses a couple of standard Perl modules that are included as part of the default installation, and don't require any CPAN-Fu, and it takes a couple of options, such as a switch for verbosity, and IP address, with or wirhout a CIDR mask, and a TCP port. The CIDR mask defaults to /32, and the port defaults to 22. Here's an example of the output.

tcsh-[101]$ ./ 80
Request to scan on Port 80 (http)
Scanning 4 IP Addresses:
--------------------------------------------------------------------------------    :                   : listening on 80 (http)    :                   : listening on 80 (http)    :            : listening on 80 (http)    :                 : listening on 80 (http)
Found 4 hosts listenening on port 80.
And here's the script itself.
#!/usr/bin/env perl
# (c) tkennedy - 2011 November 19 Version 1.1
# Description: Allows scanning a IP Subnet for TCP listeners on designated port.
# Syntax: $0 [-v] IPADDRESS/CIDR-NETMASK [Port]
# Purpose:
# This script provides a means of scanning a subnet for TCP listeners, using 
# only commonly available Perl functions.  This should make the script portable
# to any system with Perl installed.  The script can be passed 3 arguments. 2 of
# the arguments are optional, and include a '-v', which increases the verbosity
# of script output, and 'P', which is a TCP port presented as an integer between
# 1 - 65535.  If a port is not supplied on the command line, the script assumes
# a default of 22, which is the common port for Secure Shell traffic (ssh). The
# mandatory argument is an IP Address, either with, or without, a CIDR netmask.
# If a CIDR netmask is not supplied on the command line, then we assume you only
# intend to scan a single IP (ie, a /32).
# History:
# 20101123 1.1 tkennedy - removed Switch module for Sol8/Perl5.003 compatibility
# 20101119 1.0 tkennedy - initial revision
# The modules we're using are standard perl modules, so this script should 
# work on any operating system with Perl installed.
use strict;
use IO::Socket;

my ($ip,$cidr,$port,$target);
my $VERBOSE = 0;

# We need a regex to match IP addresses.  This is only used to validate
# the command-line options to verify that one option is an IP.
my $ipr = qr/^((?:(?:2(?:5[0-5]|[0-4][0-9])\.)|(?:1[0-9][0-9]\.)|(?:(?:[1-9][0-9]?|[0-9])\.)){3}(?:(?:2(?:5[0-5]|[0-4][0-9]))|(?:1[0-9][0-9])|(?:[1-9][0-9]?|[0-9])).*$)/x;

# I wanted to keep things as free-form as possible, and so opted to just 
# parse the command line to extract our options.  This also gives us some
# leeway to ignore bad options.
# In our parser, we will match a '-h' for help info, a '-v' to extend 
# our output a bit, including printing lines for hosts that are scanned
# but not listening.  We'll also match a single number, which we'll 
# interpret as a TCP port number, and lastly, we'll match an IP address
# or CIDR-masked sub-net.
foreach my $arg (@ARGV) {
 for ($arg) {
  if ( $arg eq '-h' )  { &usage; }
  elsif ( $arg eq '-v' )  { $VERBOSE = 1; }
  elsif ( $arg =~ m/^\d+$/ )  { $port  = $arg; }
  elsif ( $arg =~ m/$ipr/ )  { $target = $arg; } 

# if the user forgot to put in a target subnet, goto usage();
&usage("Error: no target supplied!") if ("$target" == "");

# Here we'll set the default port to "22", which is SSH, and then we'll
# check to see if a port was submitted on the command line.  If it was,
# we'll override the default with the user submitted value.  We'll do 
# the same for CIDR mask, assuming that if a mask was not passed in, then
# the user wants a single host scanned and use /32 as the default.
($ip,$cidr) = split( /\//, $target);
$cidr  = ( $cidr ? $cidr : "32" );
$port  = ( $port ? $port : "22" );

# die if we get an invalid CIDR mask!
&usage("Error: Invalid CIDR mask [$cidr]") if ( $cidr > 32 );

# Get the TCP Service name for our port...
my $svcname = getservbyport($port,"tcp");

# Convert the CIDR mask into a hex netmask, and calculate the number
# of addresses in the supplied target.
# my $mask = 0xffffffff >> $cidr;  # this fails on sol8/perl-5.003
my $size = 1 << ( 32 - $cidr );
my $mask = $size - 1;

# script body below here.

print "Request to scan ${ip}/${cidr} on Port $port ($svcname)","\n";
print "Scanning $size IP Addresses:","\n";

# calculate the lowest IP in the range, by AND-ing the supplied IP 
# address and the netmask, and convert to dotted quad notation.
my $lowest      = unpack('N', pack('C4', split '\.', $ip)) & ~$mask;
my $count = 0;
my $scanned = 0;

# based on the $lowest IP in the subnet, let's enumerate all of the
# IP addresses in the supplied $target subnet.
my @ips  = map {$lowest++} 0 .. $mask;

# a simple loop to scan each of the ips we mapped.
foreach my $addr(@ips) {

print "Found $count hosts listenening on port $port.","\n";

# only subroutines are below here.

sub scan_ip {
 # from the foreach loop above, we've passed in our address.
 # this address is an integer, so we'll need to convert it to
 # a dotted-quad format IP address. then try hostname lookup,
 # and then try opening a socket.
 my $addr = shift;
 my $ipaddr = join( '.', unpack( "C4", pack( "N", $addr ) ) );

 my $hostname = gethostbyaddr(inet_aton("$ipaddr"), AF_INET);
 chomp $hostname;

 if ("$hostname" eq "") {
  $hostname = ( $VERBOSE ? "NXDOMAIN" : " " );

 my $sock = IO::Socket::INET->new(
  PeerAddr => "$ipaddr",
  PeerPort => "$port",
  Timeout => "1",
  Proto => "tcp",
  ) or nosocket("$ipaddr","$hostname") && next;

 if($scanned < 1) { &hr(); }
 my $txt = "listening on $port ($svcname)";
 printf('%-15s : %-32s : %-15s', $ipaddr, $hostname, "$txt\n") if($sock);

 close($sock) if($sock);

sub nosocket { 
 # we reach this sub on unsuccessful socket attempts.
 if($scanned < 1) { &hr(); }
 my $ipaddr = shift;
 my $hostname = shift;

 if($VERBOSE > 0) {
  my $txt = "closed on $port ($svcname)";
  printf('%-15s : %-32s : %-15s', $ipaddr, $hostname, "$txt\n");


sub usage { 
 if (@_) {
  print "\n@_\n";
 # this sub() just prints the standard usage information.
 print < [port]
 -h  this message
 -v  verbose output
  format IP/CIDR:, /32 is default CIDR mask
 [port]  a tcp port between 1 and 65535, 22 is default 

Example: $0 -v 80
 will scan the 255 addresses in the subnet on port 80


sub hr { 
 # print a row of ---s
 print "-" x 80, "\n";

sub hr2 { 
 # print a row of ===s
 print "=" x 80, "\n";

So far this has worked successfully on Solaris 10 Sparc & x64, CentOS 5.5 x64, Solaris 8 Sparc, and Solaris 9 Sparc, and Mac OS X 10.6.