Planet LA

Syndicate content
Planet Linux Australia -
Updated: 1 week 3 days ago

Stewart Smith: An update on using Tor on Android

October 21, 2015 - 10:26

Back in 2012 I wrote a blog post on using Tor on Android which has proved quite popular over the years.

These days, there is the OrFox browser, which is from The Tor Project and is likely the current best way to browse the web through Tor on your Android device.

If you’re still using the custom setup Firefox, I’d recommend giving OrFox a try – it’s been working quite well for me.

Categories: Aligned Planets

sthbrx - a POWER technical blog: A tale of two Dockers

October 20, 2015 - 15:25

(This was published in an internal technical journal last week, and is now being published here. If you already know what Docker is, feel free to skim the first half.)

Docker seems to be the flavour of the month in IT. Most attention is focussed on using Docker for the deployment of production services. But that’s not all Docker is good for. Let’s explore Docker, and two ways I use it as a software developer.

Docker: what is it?

Docker is essentially a set of tools to deal with containers and images.

To make up an artificial example, say you are developing a web app. You first build an image: a file system which contains the app, and some associated metadata. The app has to run on something, so you also install things like Python or Ruby and all the necessary libraries, usually by installing a minimal Ubuntu and any necessary packages.1 You then run the image inside an isolated environment called a container.

You can have multiple containers running the same image, (for example, your web app running across a fleet of servers) and the containers don’t affect each other. Why? Because Docker is designed around the concept of immutability. Containers can write to the image they are running, but the changes are specific to that container, and aren’t preserved beyond the life of the container.2 Indeed, once built, images can’t be changed at all, only rebuilt from scratch.

However, as well as enabling you to easily run multiple copies, another upshot of immutability is that if your web app allows you to upload photos, and you restart the container, your photos will be gone. Your web app needs to be designed to store all of the data outside of the container, sending it to a dedicated database or object store of some sort.

Making your application Docker friendly is significantly more work than just spinning up a virtual machine and installing stuff. So what does all this extra work get you? Three main things: isolation, control and, as mentioned, immutability.

Isolation makes containers easy to migrate and deploy, and easy to update. Once an image is built, it can be copied to another system and launched. Isolation also makes it easy to update software your app depends on: you rebuild the image with software updates, and then just deploy it. You don’t have to worry about service A relying on version X of a library while service B depends on version Y; it’s all self contained.

Immutability also helps with upgrades, especially when deploying them across multiple servers. Normally, you would upgrade your app on each server, and have to make sure that every server gets all the same sets of updates. With Docker, you don’t upgrade a running container. Instead, you rebuild your Docker image and re-deploy it, and you then know that the same version of everything is running everywhere. This immutability also guards against the situation where you have a number of different servers that are all special snowflakes with their own little tweaks, and you end up with a fractal of complexity.

Finally, Docker offers a lot of control over containers, and for a low performance penalty. Docker containers can have their CPU, memory and network controlled easily, without the overhead of a full virtual machine. This makes it an attractive solution for running untrusted executables.3

As an aside: despite the hype, very little of this is actually particularly new. Isolation and control are not new problems. All Unixes, including Linux, support ‘chroots’. The name comes from “change root”: the system call changes the processes idea of what the file system root is, making it impossible for it to access things outside of the new designated root directory. FreeBSD has jails, which are more powerful, Solaris has Zones, and AIX has WPARs. Chroots are fast and low overhead. However, they offer much lower ability to control the use of system resources. At the other end of the scale, virtual machines (which have been around since ancient IBM mainframes) offer isolation much better than Docker, but with a greater performance hit.

Similarly, immutability isn’t really new: Heroku and AWS Spot Instances are both built around the model that you get resources in a known, consistent state when you start, but in both cases your changes won’t persist. In the development world, modern CI systems like Travis CI also have this immutable or disposable model – and this was originally built on VMs. Indeed, with a little bit of extra work, both chroots and VMs can give the same immutability properties that Docker gives.

The control properties that Docker provides are largely as a result of leveraging some Linux kernel concepts, most notably something called namespaces.

What Docker does well is not something novel, but the engineering feat of bringing together fine-grained control, isolation and immutability, and – importantly – a tool-chain that is easier to use than any of the alternatives. Docker’s tool-chain eases a lot of pain points with regards to building containers: it’s vastly simpler than chroots, and easier to customise than most VM setups. Docker also has a number of engineering tricks to reduce the disk space overhead of isolation.

So, to summarise: Docker provides a toolkit for isolated, immutable, finely controlled containers to run executables and services.

Docker in development: why?

I don’t run network services at work; I do performance work. So how do I use Docker?

There are two things I do with Docker: I build PHP 5, and do performance regression testing on PHP 7. They’re good case studies of how isolation and immutability provide real benefits in development and testing, and how the Docker tool chain makes life a lot nicer that previous solutions.

PHP 5 builds

I use the isolation that Docker provides to make building PHP 5 easier. PHP 5 depends on an old version of Bison, version 2. Ubuntu and Debian long since moved to version 3. There are a few ways I could have solved this:

  • I could just install the old version directly on my system in /usr/local/, and hope everything still works and nothing else picks up Bison 2 when it needs Bison 3. Or I could install it somewhere else and remember to change my path correctly before I build PHP 5.
  • I could roll a chroot by hand. Even with tools like debootstrap and schroot, working in chroots is a painful process.
  • I could spin up a virtual machine on one of our development boxes and install the old version on that. That feels like overkill: why should I need to run an entire operating system? Why should I need to copy my source tree over the network to build it?

Docker makes it easy to have a self-contained environment that has Bison 2 built from source, and to build my latest source tree in that environment. Why is Docker so much easier?

Firstly, Docker allows me to base my container on an existing container, and there’s an online library of containers to build from.4 This means I don’t have to roll a base image with debootstrap or the RHEL/CentOS/Fedora equivalent.

Secondly, unlike a chroot build process, which ultimately is just copying files around, a docker build process includes the ability to both copy files from the host and run commands in the context of the image. This is defined in a file called a Dockerfile, and is kicked off by a single command: docker build.

So, my PHP 5 build container loads an Ubuntu Vivid base container, uses apt-get to install the compiler, tool-chain and headers required to build PHP 5, then installs old bison from source, copies in the PHP source tree, and builds it. The vast majority of this process – the installation of the compiler, headers and bison, can be cached, so they don’t have to be downloaded each time. And once the container finishes building, I have a fully built PHP interpreter ready for me to interact with.

I do, at the moment, rebuild PHP 5 from scratch each time. This is a bit sub-optimal from a performance point of view. I could alleviate this with a Docker volume, which is a way of sharing data persistently between a host and a guest, but I haven’t been sufficiently bothered by the speed yet. However, Docker volumes are also quite fiddly, leading to the development of tools like docker compose to deal with them. They also are prone to subtle and difficult to debug permission issues.

PHP 7 performance regression testing

The second thing I use docker for takes advantage of the throwaway nature of docker environments to prevent cross-contamination.

PHP 7 is the next big version of PHP, slated to be released quite soon. I care about how that runs on POWER, and I preferably want to know if it suddenly deteriorates (or improves!). I use Docker to build a container with a daily build of PHP 7, and then I run a benchmark in it. This doesn’t give me a particularly meaningful absolute number, but it allows me to track progress over time. Building it inside of Docker means that I can be sure that nothing from old runs persists into new runs, thus giving me more reliable data. However, because I do want the timing data I collect to persist, I send it out of the container over the network.

I’ve now been collecting this data for almost 4 months, and it’s plotted below, along with a 5-point moving average. The most notable feature of the graph is a the drop in benchmark time at about the middle. Sure enough, if you look at the PHP repository, you will see that a set of changes to improve PHP performance were merged on July 29: changes submitted by our very own Anton Blanchard.5

Docker pain points

Docker provides a vastly improved experience over previous solutions, but there are still a few pain points. For example:

  1. Docker was apparently written by people who had no concept that platforms other than x86 exist. This leads to major issues for cross-architectural setups. For instance, Docker identifies images by a name and a revision. For example, ubuntu is the name of an image, and 15.04 is a revision. There’s no ability to specify an architecture. So, how you do specify that you want, say, a 64-bit, little-endian PowerPC build of an image versus an x86 build? There have been a couple of approaches, both of which are pretty bad. You could name the image differently: say ubuntu_ppc64le. You can also just cheat and override the ubuntu name with an architecture specific version. Both of these break some assumptions in the Docker ecosystem and are a pain to work with.

  2. Image building is incredibly inflexible. If you have one system that requires a proxy, and one that does not, you need different Dockerfiles. As far as I can tell, there are no simple ways to hook in any changes between systems into a generic Dockerfile. This is largely by design, but it’s still really annoying when you have one system behind a firewall and one system out on the public cloud (as I do in the PHP 7 setup).

  3. Visibility into a Docker server is poor. You end up with lots of different, anonymous images and dead containers, and you end up needing scripts to clean them up. It’s not clear what Docker puts on your file system, or where, or how to interact with it.

  4. Docker is still using reasonably new technologies. This leads to occasional weird, obscure and difficult to debug issues.6

Final words

Docker provides me with a lot of useful tools in software development: both in terms of building and testing. Making use of it requires a certain amount of careful design thought, but when applied thoughtfully it can make life significantly easier.

  1. There’s some debate about how much stuff from the OS installation you should be using. You need to have key dynamic libraries available, but I would argue that you shouldn’t be running long running processes other than your application. You shouldn’t, for example, be running a SSH daemon in your container. (The one exception is that you must handle orphaned child processes appropriately: see Considerations like debugging and monitoring the health of docker containers mean that this point of view is not universally shared.?

  2. Why not simply make them read only? You may be surprised at how many things break when running on a read-only file system. Things like logs and temporary files are common issues.?

  3. It is, however, easier to escape a Docker container than a VM. In Docker, an untrusted executable only needs a kernel exploit to get to root on the host, whereas in a VM you need a guest-to-host vulnerability, which are much rarer.?

  4. Anyone can upload an image, so this does require running untrusted code from the Internet. Sadly, this is a distinctly retrograde step when compared to the process of installing binary packages in distros, which are all signed by a distro’s private key.?

  5. See

  6. I hit this last week:, although maybe that’s my fault for running systemd on my laptop.?

Categories: Aligned Planets

sthbrx - a POWER technical blog: Running ppc64le_hello on real hardware

October 20, 2015 - 15:25

So today I saw Freestanding “Hello World” for OpenPower on Hacker News. Sadly Andrei hadn’t been able to test it on real hardware, so I set out to get it running on a real OpenPOWER box. Here’s what I did.

Firstly, clone the repo, and, as mentioned in the README, comment out mambo_write. Build it.

Grab op-build, and build a Habanero defconfig. To save yourself a fair bit of time, first edit openpower/configs/habanero_defconfig to answer n about a custom kernel source. That’ll save you hours of waiting for git.

This will build you a PNOR that will boot a linux kernel with Petitboot. This is almost what you want: you need Skiboot, Hostboot and a bunch of the POWER specific bits and bobs, but you don’t actually want the Linux boot kernel.

Then, based on op-build/openpower/package/openpower-pnor/, we look through the output of op-build for a command, something like this monstrosity:

PATH="/scratch/dja/public/op-build/output/host/bin:/scratch/dja/public/op-build/output/host/sbin:/scratch/dja/public/op-build/output/host/usr/bin:/scratch/dja/public/op-build/output/host/usr/sbin:/home/dja/bin:/home/dja/bin:/home/dja/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/openpower/common/x86_64/bin" /scratch/dja/public/op-build/output/build/openpower-pnor-ed1682e10526ebd85825427fbf397361bb0e34aa/ -xml_layout_file /scratch/dja/public/op-build/output/build/openpower-pnor-ed1682e10526ebd85825427fbf397361bb0e34aa/"defaultPnorLayoutWithGoldenSide.xml" -pnor_filename /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/pnor/"habanero.pnor" -hb_image_dir /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/hostboot_build_images/ -scratch_dir /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/openpower_pnor_scratch/ -outdir /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/pnor/ -payload /scratch/dja/public/op-build/output/images/"skiboot.lid" -bootkernel /scratch/dja/public/op-build/output/images/zImage.epapr -sbe_binary_filename "venice_sbe.img.ecc" -sbec_binary_filename "centaur_sbec_pad.img.ecc" -wink_binary_filename "p8.ref_image.hdr.bin.ecc" -occ_binary_filename /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/occ/"occ.bin" -targeting_binary_filename "HABANERO_HB.targeting.bin.ecc" -openpower_version_filename /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/openpower_version/openpower-pnor.version.txt

Replace the -bootkernel arguement with the path to ppc64le_hello, e.g.: -bootkernel /scratch/dja/public/ppc64le_hello/ppc64le_hello

Don’t forget to move it into place!

1 mv output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/pnor/habanero.pnor output/images/habanero.pnor

Then we can use skiboot’s boot test script (written by Cyril and me, coincidentally!) to flash it.

1 ppc64le_hello/skiboot/external/boot-tests/ -vp -t hab2-bmc -P <path to>/habanero.pnor

It’s not going to get into Petitboot, so just interrupt it after it powers up the box and connect with IPMI. It boots, kinda:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 [11012941323,5] INIT: Starting kernel at 0x20010000, fdt at 0x3044db68 (size 0x11cc3) Hello OPAL! _start = 0x20010000 _bss = 0x20017E28 _stack = 0x20018000 _end = 0x2001A000 KPCR = 0x20017E50 OPAL = 0x30000000 FDT = 0x3044DB68 CPU0 not found? Pick your poison: Choices: (MMU = disabled): (d) 5s delay (e) test exception (n) test nested exception (f) dump FDT (M) enable MMU (m) disable MMU (t) test MMU (u) test non-priviledged code (I) enable ints (i) disable ints (H) enable HV dec (h) disable HV dec (q) poweroff 1.42486|ERRL|Dumping errors reported prior to registration

Yes, it does wrap horribly. However, the big issue here (which you’ll have to scroll to see!) is the “CPU0 not found?”. Fortunately, we can fix this with a little patch to cpu_init in main.c to test for a PowerPC POWER8:

1 2 3 4 5 6 7 8 cpu0_node = fdt_path_offset(fdt, "/cpus/cpu@0"); if (cpu0_node < 0) { cpu0_node = fdt_path_offset(fdt, "/cpus/PowerPC,POWER8@20"); } if (cpu0_node < 0) { printk("CPU0 not found?\n"); return; }

This is definitely the wrong way to do this, but it works for now.

Now, correcting for weird wrapping, we get:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Hello OPAL! _start = 0x20010000 _bss = 0x20017E28 _stack = 0x20018000 _end = 0x2001A000 KPCR = 0x20017E50 OPAL = 0x30000000 FDT = 0x3044DB68 Assuming default SLB size SLB size = 0x20 TB freq = 512000000 [13205442015,3] OPAL: Trying a CPU re-init with flags: 0x2 Unrecoverable exception stack top @ 0x20019EC8 HTAB (2048 ptegs, mask 0x7FF, size 0x40000) @ 0x20040000 SLB entries: 1: E 0x8000000 V 0x4000000000000400 EA 0x20040000 -> hash 0x20040 -> pteg 0x200 = RA 0x20040000 EA 0x20041000 -> hash 0x20041 -> pteg 0x208 = RA 0x20041000 EA 0x20042000 -> hash 0x20042 -> pteg 0x210 = RA 0x20042000 EA 0x20043000 -> hash 0x20043 -> pteg 0x218 = RA 0x20043000 EA 0x20044000 -> hash 0x20044 -> pteg 0x220 = RA 0x20044000 EA 0x20045000 -> hash 0x20045 -> pteg 0x228 = RA 0x20045000 EA 0x20046000 -> hash 0x20046 -> pteg 0x230 = RA 0x20046000 EA 0x20047000 -> hash 0x20047 -> pteg 0x238 = RA 0x20047000 EA 0x20048000 -> hash 0x20048 -> pteg 0x240 = RA 0x20048000 ...

The weird wrapping seems to be caused by NULLs getting printed to OPAL, but I haven’t traced what causes that.

Anyway, now it largely works! Here’s a transcript of some things it can do on real hardware.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 Choices: (MMU = disabled): (d) 5s delay (e) test exception (n) test nested exception (f) dump FDT (M) enable MMU (m) disable MMU (t) test MMU (u) test non-priviledged code (I) enable ints (i) disable ints (H) enable HV dec (h) disable HV dec (q) poweroff <press e> Testing exception handling... sc(feed) => 0xFEEDFACE Choices: (MMU = disabled): (d) 5s delay (e) test exception (n) test nested exception (f) dump FDT (M) enable MMU (m) disable MMU (t) test MMU (u) test non-priviledged code (I) enable ints (i) disable ints (H) enable HV dec (h) disable HV dec (q) poweroff <press t> EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = RA 0x20010000 mapped 0xFFFFFFF000 to 0x20010000 correctly EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = unmap EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = RA 0x20011000 mapped 0xFFFFFFF000 to 0x20011000 incorrectly EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = unmap Choices: (MMU = disabled): (d) 5s delay (e) test exception (n) test nested exception (f) dump FDT (M) enable MMU (m) disable MMU (t) test MMU (u) test non-priviledged code (I) enable ints (i) disable ints (H) enable HV dec (h) disable HV dec (q) poweroff <press u> EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = RA 0x20080000 returning to user code returning to kernel code EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = unmap

I also tested the other functions and they all seem to work. Running non-priviledged code with the MMU on works. Dumping the FDT and the 5s delay both worked, although they tend to stress IPMI a lot. The delay seems to correspond well with real time as well.

It does tend to error out and reboot quite often, usually on the menu screen, for reasons that are not clear to me. It usually starts with something entirely uninformative from Hostboot, like this:

1 2 1.41801|ERRL|Dumping errors reported prior to registration 2.89873|Ignoring boot flags, incorrect version 0x0

That may be easy to fix, but again I haven’t had time to trace it.

All in all, it’s very exciting to see something come out of the simulator and in to real hardware. Hopefully with the proliferation of OpenPOWER hardware, prices will fall and these sorts of systems will become increasingly accessible to people with cool low level projects like this!

Categories: Aligned Planets

sthbrx - a POWER technical blog: Petitboot Autoboot Changes

October 20, 2015 - 15:25

The way autoboot behaves in Petitboot has undergone some significant changes recently, so in order to ward off any angry emails lets take a quick tour of how the new system works.

Old & Busted

For some context, here is the old (or current depending on what you’re running) section of the configuration screen.

This gives you three main options: don’t autoboot, autoboot from anything, or autoboot only from a specific device. For the majority of installations this is fine, such as when you have only one default option, or know exactly which device you’ll be booting from.

A side note about default options: it is important to note that not all boot options are valid autoboot options. A boot option is only considered for auto-booting if it is marked default, eg. ‘set default’ in GRUB and ‘default’ in PXE options.

New Hotness

Below is the new autoboot configuration.

The new design allows you to specify an ordered list of autoboot options. The last two of the three buttons are self explanatory - clear the list and autoboot any device, or clear the list completely (no autoboot).

Selecting the first button, ‘Add Device’ brings up the following screen:

From here you can select any device or class of device to add to the boot order. Once added to the boot order, the order of boot options can be changed with the left and right arrow keys, and removed from the list with the minus key (‘-’).

This allows you to create additional autoboot configurations such as “Try to boot from sda2, otherwise boot from the network”, or “Give priority to PXE options from eth0, otherwise try any other netboot option”. You can retain the original behaviour by only putting one option into the list (either ‘Any Device’ or a specific device).

Presently you can add any option into the list and order them how you like - which means you can do silly things like this:


Slightly prior to the boot order changes Petitboot also received an update to its IPMI handling. IPMI ‘bootdev’ commands allow you to override the current autoboot configuration remotely, either by specifying a device type to boot (eg. PXE), or by forcing Petitboot to boot into the ‘setup’ or ‘safe’ modes. IPMI overrides are either persistent or non-persistent. A non-persistent override will disappear after a successful boot - that is, a successful boot of a boot option, not booting to Petitboot itself - whereas a persistent override will, well, persist!

If there is an IPMI override currently active, it will appear in the configuration screen with an option to manually clear it:

That sums up the recent changes to autoboot; a bit more flexibility in assigning priority, and options for more detailed autoboot order if you need it. New versions of Petitboot are backwards compatible and will recognise older saved settings, so updating your firmware won’t cause your machines to start booting things at random.

Categories: Aligned Planets

sthbrx - a POWER technical blog: Joining the CAPI project

October 20, 2015 - 15:25

(I wrote this blog post a couple of months ago, but it’s still quite relevant.)

Hi, I’m Daniel! I work in OzLabs, part of IBM’s Australian Development Labs. Recently, I’ve been assigned to the CAPI project, and I’ve been given the opportunity to give you an idea of what this is, and what I’ll be up to in the future!

What even is CAPI?

To help you understand CAPI, think back to the time before computers. We had a variety of machines: machines to build things, to check things, to count things, but they were all specialised — good at one and only one thing.

Specialised machines, while great at their intended task, are really expensive to develop. Not only that, it’s often impossible to change how they operate, even in very small ways.

Computer processors, on the other hand, are generalists. They are cheap. They can do a lot of things. If you can break a task down into simple steps, it’s easy to get them to do it. The trade-off is that computer processors are incredibly inefficient at everything.

Now imagine, if you will, that a specialised machine is a highly trained and experienced professional, a computer processor is a hungover university student.

Over the years, we’ve tried lots of things to make student faster. Firstly, we gave the student lots of caffeine to make them go as fast as they can. That worked for a while, but you can only give someone so much caffeine before they become unreliable. Then we tried teaming the student up with another student, so they can do two things at once. That worked, so we added more and more students. Unfortunately, lots of tasks can only be done by one person at a time, and team-work is complicated to co-ordinate. We’ve also recently noticed that some tasks come up often, so we’ve given them some tools for those specific tasks. Sadly, the tools are only useful for those specific situations.

Sometimes, what you really need is a professional.

However, there are a few difficulties in getting a professional to work with uni students. They don’t speak the same way; they don’t think the same way, and they don’t work the same way. You need to teach the uni students how to work with the professional, and vice versa.

Previously, developing this interface – this connection between a generalist processor and a specialist machine – has been particularly difficult. The interface between processors and these specialised machines – known as accelerators – has also tended to suffer from bottlenecks and inefficiencies.

This is the problem CAPI solves. CAPI provides a simpler and more optimised way to interface specialised hardware accelerators with IBM’s most recent line of processors, POWER8. It’s a common ‘language’ that the processor and the accelerator talk, that makes it much easier to build the hardware side and easier to program the software side. In our Canberra lab, we’re working primarily on the operating system side of this. We are working with some external companies who are building CAPI devices and the optimised software products which use them.

From a technical point of view, CAPI provides coherent access to system memory and processor caches, eliminating a major bottleneck in using external devices as accelerators. This is illustrated really well by the following graphic from an IBM promotional video. In the non-CAPI case, you can see there’s a lot of data (the little boxes) stalled in the PCIe subsystem, whereas with CAPI, the accelerator has direct access to the memory subsystem, which makes everything go faster.

Uses of CAPI

CAPI technology is already powering a few really cool products.

Firstly, we have an implementation of Redis that sits on top of flash storage connected over CAPI. Or, to take out the buzzwords, CAPI lets us do really, really fast NoSQL databases. There’s a video online giving more details.

Secondly, our partner Mellanox is using CAPI to make network cards that run at speeds of up to 100Gb/s.

CAPI is also part of IBM’s OpenPOWER initiative, where we’re trying to grow a community of companies around our POWER system designs. So in many ways, CAPI is both a really cool technology, and a brand new ecosystem that we’re growing here in the Canberra labs. It’s very cool to be a part of!

Categories: Aligned Planets

sthbrx - a POWER technical blog: OpenPOWER Powers Forward

October 20, 2015 - 15:25

I wrote this blog post late last year, it is very relevant for this blog though so I’ll repost it here.

With the launch of TYAN’s OpenPOWER reference system now is a good time to reflect on the team responsible for so much of the research, design and development behind this very first ground breaking step of OpenPOWER with their start to finish involvement of this new Power platform.

ADL Canberra have been integral to the success of this launch providing the Open Power Abstraction Layer (OPAL) firmware. OPAL breathes new life into Linux on Power finally allowing Linux to run on directly on the hardware. While OPAL harnesses the hardware, ADL Canberra significantly improved Linux to sit on top and take direct control of IBMs new Power8 processor without needing to negotiate with a hypervisor. With all the Linux expertise present at ADL Canberra it’s no wonder that a Linux based bootloader was developed to make this system work. Petitboot leverage’s all the resources of the Linux kernel to create a light, fast and yet extremely versatile bootloader. Petitboot provides a massive amount of tools for debugging and system configuration without the need to load an operating system.

TYAN have developed great and highly customisable hardware. ADL Canberra have been there since day 1 performing vital platform enablement (bringup) of this new hardware. ADL Canberra have put all the work into the entire software stack, low level work to get OPAL and Linux to talk to the new BMC chip as well as the higher level, enabling to run Linux in either endian and Linux is even now capable of virtualising KVM guests in either endian irrespective of host endian. Furthermore a subset of ADL Canberra have been key to getting the Coherent Accelerator Processor Interface (CAPI) off the ground, enabling more almost endless customisation and greater diversity within the OpenPOWER ecosystem.

ADL Canberra is the home for Linux on Power and the beginning of the OpenPOWER hardware sees much of the hard work by ADL Canberra come to fruition.

Categories: Aligned Planets

OpenSTEM: How to Help Your Child Become a Maker | MakerKids

October 20, 2015 - 13:30!How-To-Help-Your-Child-Become-a-Maker/dcsov/562094010cf2c3a4a7109d92

Let’s say your child is currently a classic consumer – they love watching TV, reading books, but they don’t really enjoy making things themselves. Or maybe they are making some things but it’s not really technological. We think any kind of making is awesome, but one of our favourite kinds is the kind where kids realize that they can build and influence the world around them. There’s an awesome Steve Jobs quote that I love, which says:

“When you grow up you tend to get told that the world is the way it is and you’re life is just to live your life inside the world. Try not to bash into the walls too much. Try to have a nice family life, have fun, save a little money.

That’s a very limited life. Life can be much broader once you discover one simple fact: Everything around you that you call life was made up by people that were no smarter than you. And you can change it, you can influence it…

Once you learn that, you’ll never be the same again.”

Imagine if you can figure this out as a child.

Categories: Aligned Planets

Rusty Russell: ccan/mem’s memeqzero iteration

October 20, 2015 - 11:28

On Thursday I was writing some code, and I wanted to test if an array was all zero.  First I checked if ccan/mem had anything, in case I missed it, then jumped on IRC to ask the author (and overall CCAN co-maintainer) David Gibson about it.

We bikeshedded around names: memallzero? memiszero? memeqz? memeqzero() won by analogy with the already-extant memeq and memeqstr. Then I asked:

rusty: dwg: now, how much time do I waste optimizing?

dwg: rusty, in the first commit, none

Exactly five minutes later I had it implemented and tested.

The Naive Approach: Times: 1/7/310/37064 Bytes: 50 bool memeqzero(const void *data, size_t length) { const unsigned char *p = data; while (length) { if (*p) return false; p++; length--; } return true; }

As a summary, I’ve give the nanoseconds for searching through 1,8,512 and 65536 bytes only.

Another 20 minutes, and I had written that benchmark, and an optimized version.

128-byte Static Buffer: Times: 6/8/48/5872 Bytes: 108

Here’s my first attempt at optimization; using a static array of 128 bytes of zeroes and assuming memcmp is well-optimized for fixed-length comparisons.  Worse for small sizes, much better for big.

const unsigned char *p = data; static unsigned long zeroes[16]; while (length > sizeof(zeroes)) { if (memcmp(zeroes, p, sizeof(zeroes))) return false; p += sizeof(zeroes); length -= sizeof(zeroes); } return memcmp(zeroes, p, length) == 0; Using a 64-bit Constant: Times: 12/12/84/6418 Bytes: 169

dwg: but blowing a cacheline (more or less) on zeroes for comparison, which isn’t necessarily a win

Using a single zero uint64_t for comparison is pretty messy:

bool memeqzero(const void *data, size_t length) {     const unsigned char *p = data;     const unsigned long zero = 0;     size_t pre;     pre = (size_t)p % sizeof(unsigned long);     if (pre) {         size_t n = sizeof(unsigned long) - pre;         if (n > length)             n = length;         if (memcmp(p, &zero, n) != 0)             return false;         p += n;         length -= n;     }     while (length > sizeof(zero)) {         if (*(unsigned long *)p != zero)             return false;         p += sizeof(zero);         length -= sizeof(zero);     }     return memcmp(&zero, p, length) == 0; }

And, worse in every way!

Using a 64-bit Constant With Open-coded Ends: Times: 4/9/68/6444 Bytes: 165

dwg: rusty, what colour is the bikeshed if you have an explicit char * loop for the pre and post?

That’s slightly better, but memcmp still wins over large distances, perhaps due to prefetching or other tricks.

Epiphany #1: We Already Have Zeroes: Times 3/5/92/5801 Bytes: 422

Then I realized that we don’t need a static buffer: we know everything we’ve already tested is zero!  So I open coded the first 16 byte compare, then memcmp()ed against the previous bytes, doubling each time.  Then a final memcmp for the tail.  Clever huh?

But it no faster than the static buffer case on the high end, and much bigger.

dwg: rusty, that is brilliant. but being brilliant isn’t enough to make things work, necessarily :p

Epiphany #2: memcmp can overlap: Times 3/5/37/2823 Bytes: 307

My doubling logic above was because my brain wasn’t completely in phase: unlike memcpy, memcmp arguments can happily overlap!  It’s still worth doing an open-coded loop to start (gcc unrolls it here with -O3), but after 16 it’s worth memcmping with the previous 16 bytes.  This is as fast as naive with as little as 2 bytes, and the fastest solution by far with larger numbers:

const unsigned char *p = data; size_t len; /* Check first 16 bytes manually */ for (len = 0; len < 16; len++) { if (!length) return true; if (*p) return false; p++; length--; } /* Now we know that's zero, memcmp with self. */ return memcmp(data, p, length) == 0;

You can find the final code in CCAN (or on Github) including the benchmark code.

Finally, after about 4 hours of random yak shaving, it turns out lightning doesn’t even want to use memeqzero() any more!  Hopefully someone else will benefit.

Categories: Aligned Planets

Stewart Smith: TianoCore (UEFI) ported to OpenPower

October 20, 2015 - 11:26

Recently, there’s been (actually two) ports of TianoCore (the reference implementation of UEFI firmware) to run on POWER on top of OPAL (provided by skiboot) – and it can be run in the Qemu PowerNV model.

More details:

Categories: Aligned Planets

Stewart Smith: 1 Million SQL Queries per second: GA MariaDB 10.1 on POWER8

October 19, 2015 - 12:26

A couple of days ago, MariaDB announced that MariaDB 10.1 is stable GA – around 19 months since the GA of MariaDB 10.0. With MariaDB 10.1 comes some important scalabiity improvements, especially for POWER8 systems. On POWER, we’re a bit unique in that we’re on the higher end of CPUs, have many cores, and up to 8 threads per core (selectable at runtime: 1, 2, 4 or 8/core) – so a dual socket system can easily be a 160 thread machine.

Recently, we (being IBM) announced availability of a couple of new POWER8 machines – machines designed for Linux and cloud environments. They are very much OpenPower machines, and more info is available here:

Combine these two together, with Axel Schwenke running some benchmarks and you get 1 Million SQL Queries per second with MariaDB 10.1 on POWER8.

Having worked a lot on both MySQL for POWER and the firmware that ships in the S882LC, I’m rather happy that 1 Million queries per second is beyond what it was in June 2014, which was a neat hack on MySQL 5.7 that showed the potential of MySQL on POWER8 but wasn’t yet a product. Now, you can run a GA release of MariaDB on GA POWER8 hardware designed for scale-out cloud environments and get 1 Million SQL queries/second (with fewer cores than my initial benchmark last year!)

What’s even more impressive is that this million queries per second is in a KVM guest!

Categories: Aligned Planets

David Rowe: All Your Modem are Belong To Us

October 19, 2015 - 06:30

Everywhere I look I see a need for high quality, carefully tested, open source, portable modem software. This year I built the cohpsk HF modem, that works well on fading HF channels, and is at the key to the low SNR performance of the FreeDV 700B mode.

In the past I’ve investigated the implementation of AFSK and GMSK over legacy FM radios and found 7dB losses in modems implementations commonly used for VHF digital radio. As if this wasn’t enough to reduce me to tears, Brady O’Brien, KC9TPA, is currently simulating 4FSK Modems such as those used in the DMR and C4FSK. These modems, tacked onto legacy analog FM radio architectures, also appears to be 6dB off the pace (more on the DMR modem soon).

Near Space Balloons and RTTY

Mark invited me along to a Horus launch that was about a week away. We started talking about the Horus telemetry modem. Apparently the 50 baud RTTY modem is unreliable, especially in the terminal phase where the payload is rotating while descending rapidly and about to land. Which is just when you would really like to know exactly where it is.

Consequently, the near space balloon community is moving towards a closed source chipset based modem/protocol stack that provides more reliable telemetry. It may even have a patent rattling around inside it.

Well that’s just not good enough.

Closed source and proprietary chipsets are nasty, a glaring problem in a cool geeky field that is otherwise open source. It’s got to be fixed.

Balloon chasing systems also use various hacks used to connect the modem software to mapping software, which in the case of the Horus balloon chasers runs on this legacy operating system called “Windows”. Yeah I’ve never heard of it either. Used to be popular apparently, and I’m told it’s good for games. However it would be nice to do everything in Open Source.

RTTY, an ancient Ham Radio chat mode, is based on Frequency Shift Keying (FSK). Just like these fancy new Digital Voice modes like DMR and C4FSK. Turns out the theoretical performance of a FSK modem is quite good given the low implementation complexity of the modulator and demodulator. Where RTTY falls down is “frame sync”. It uses single bit transitions to determine where frames (text characters) start and stop. A bit error anywhere tends to wipe out many subsequent bits.

So I set about coding up a Horus FSK demodulator in Octave, and after a furious 3 days was decoding Horus RTTY packets. Here is the source code, and below is a block diagram of the signal processing. The top section deals with extracting a stream of 1?s and 0?s from the input audio. The lower section deals with framing and plucking out valid telemetry packets:

A couple of interesting features:

  1. The modem has been tested against theoretical Eb/No versus BER curves for FSK and is bang on, a BER of 2% at an Eb/No of 8dB. You can (and should) repeat these tests as part of any modem development. However I see very little formal testing of open source modems going on. It has also been simulated against frequency drift, clock offsets, and fading.
  2. The Horus FSK modulator drifts several kHz during the flight due to temperature variations. I tested this by putting a Horus payload in my freezer. Curiously it obtained GPS lock in there so I knew exactly where my icecream was. Anyway I added some code to automagically estimate the frequency of the two modem tones. Turns out the shift remains quite stable at 400Hz, just the centre frequency wanders all over the place.
  3. The fsk_horus demod tracks the baud rate of the modem. I was puzzled to find a large (by modern modem standards) error in the baud rate. The tx side seemed to be running at 50.08 rather than 50 baud (1600ppm error). Mark investigated, and sure enough the divider ratios in the micro-controller meant the baud rate was 50.08. The good news is that it’s rock solid (doesn’t drift). It’s just a bit off. Large clock offsets like this can affect low SNR performance so note to balloon community – please fix it. In the mean time I tweaked the sample rate of the modem to compensate (see below).
  4. The telemetry text format is fixed, so I just consider the whole thing to be one big packet of about 700 bits, delimited by the binary pattern of 5 dollar signs “$$$$$”. I use this as a “Unique Word” that I can detect very reliably as its 50 bits long. Good bye RS232 framing, hello reliable frame sync.
  5. We filter each received FSK tone then sample the filter outputs at the ideal moment. Choosing the right sampling time is called fine timing estimation. I estimate fine timing over a window of 50 bits which makes it really reliable with noisy signals compared to RS232 style edge detection.
  6. Even with a bad checksum, plenty of useful information in there. For example the GPS coords might still be OK. So I print out all packets so we can use “human FEC” to see whats in there.
  7. I apply a few rules to clean up the data. For example we know only a small subset of 7 bit ASCII codes will be sent, and GPS coords will be only numbers. So anything outside these ranges is marked as an “erasure”. This prevents a lot of line noise being displayed.
  8. Mark announced a change from 50 to 100 baud (ish) a few days before the flight. No matter, just a few tweaks.
  9. I patched FoxtrotGPS to load a track file via a UDP port command. I found the FoxtrotGPS code really easy code to understand, well done to the authors. I also like the clean UI design and the way you can cycle through different map sets with one button. And a native app is so much more faster than browser based mapping. Breath of fresh air. Patching instructions below.
  10. I wrote the modem in GNU Octave, and it runs in real time. Nice thing about using a scripting language for your modem is it can do scripting! Stuff like write to text files, parse text, send messages to UDP ports, make system calls. The glue code contains the modem now. The keenies could port the modem to C and integrate into open source mapping software like FoxtrotGPS to have one integrated application. Try that with your closed source chip set!
  11. The demod has some simple soft decision error correction. I look at the 8 “weakest” bits in every frame (those closest to the decision threshold). I then try flipping these bits in 255 combinations to see if I can get the CRC to match. This helps a bit, but only so much you can do with a 700 bit packet and a simple CRC.


I carefully tested the modem in simulated AWGN and fading channels. The fading simulated the rotation of the payload plummeting to Earth. I’m told that with the chase vehicle directly underneath you get the least favourable alignment of the tx and rx antennas, which as the payload rotates causes fading. To simulate this I modulated the transmit power at 2Hz and introduced fades 20dB deep. I was told that 2Hz was about as fast as the payload could spin.

Here is the Horus payload connected to my FT-817 via 120dB worth of attenuators, although I really need 150dB to get close to the MDS of the system:

I also tested the entire system around my suburb, e.g. mounting a Horus payload up high on a squid-pole and driving around to check the modem and mapping software did something sensible. This also let me make a check list of equipment I would need on the day. Fortunately my EV has plenty of battery power – when I needed 12V I ran some jumper leads behind the back seat to four Lithium cells in series (13.4V).

Real Time DSP in GNU Octave

The modem runs in GNU Octave in real time (a few % CPU load on one core of an old laptop). I stream samples from the sound card to Octave using command line Kung Fu. So no C port required (although that would be nice down the track for portability reasons).

This is a bit surprising – real time DSP in a scripting language.

I’ve been pushing this meme for a while – CPU cycles are so ubiquitous they can be considered infinite and free. We are wearing enough CPU for many DSP applications. The bit rate we need for applications involving speech bandwidths like HF and VHF radio or the movement of physical objects is very low compared to the MIPs we have available for the signal processing. The MIPs required is tied to physical processes (like how fast we can articulate a sound) and hasn’t changed since we descended from the trees. But the CPUs just keep getting faster.

So the hardware required for the physical implementation of the DSP (e.g. a few % of your laptop or phone) is effectively free. We just need good quality free DSP software to go along with it.

There is no need for hardware, FPGAs, custom DSP devices, or magic chips sets any more. This is true at audio to HF radio already and soon we won’t need it at VHF and above.

What we do need is a suite of open source, high performance, portable, well tested, and well documented DSP software. Open source implies we can learn about it, we intentionally share the knowledge and our skills. Demystifying DSP. Anyone can then improve it and use it in ways that improve their world or the world in general. Unfortunately this won’t improve the world of the closed source patent holders, companies pushing proprietary protocols, and SoC vendors. They won’t like this. Boo Hoo. We don’t need them anymore. We can do a better job than they can. The value is drifting out of the proprietary space. It always does.

Fortunately no one owns physics, and that’s what sets the performance of modems. You can’t seek rent on physics. All your modems are belong to us.

Octave Code

The file fsk_horus.m contains the modem code, and code to test the modem. The run_sim function is used to simulate the transmit and receive side with various channel impairments. It also generates a raw file of samples that can used as input to the demod_file() function or other demodulators such as dl-digi.

The fsk_horus_stream.m script runs the demodulator in real time, accepting samples from stdin:

~/codec2-dev/octave$ rec -t raw -r 8013 -s -2 -c 1 - -q | ./fsk_horus_stream.m

It prints out the packets, extracts lat/long coords, writes to a file, and tells FoxtrotGPS to plot them.

Testing on a Real Balloon Flight

On Sunday Oct 18 we tested the system by chasing a Horus launch from Mt Barker to near Palmer. Here is a screen shot from FoxtrotGPS, balloon track in green, car in red (click for a larger map):

I used a small antenna on a mag mount base, connected to a FT-817 tuned to the payload RTTY signal. This was connected to my laptop running the fsk_horus_streaming script. I used my phone running BlueNMEA as the GPS source for the car, connected to my laptop by USB in Android debug mode:

adb forward tcp:4352 tcp:4352

gpsd -N -n -D5 tcp://localhost:4352

Here is block diagram of how it all fit together:

We had a very pleasant day in the spring sunshine and it became quite exciting as we tracked the payload back and forth. The last packet before we lost the signal was 213m from the ground:

VK5ARG-1,1555,01:46:53,-34.890294,139.178262,705,8,1529,08 CRC OK        

VK5ARG-1,1556,01:47:01,-34.890531,139.178020,655,10,1530,08 CRC OK        

VK5ARG-1,1557,01:47:09,-34.890896,139.177668,601,8,1530,08 CRC OK        

VK5ARG-1,1558,01:47:13,-34.891132,139.177197,551,9,1531,08 CRC OK        

VK5ARG-1,1559,01:47:21,-34.891275,139.177033,516,10,1532,08 CRC OK        

VK5ARG-1,1560,01:47:29,-34.891644,139.176765,457,10,1533,08 CRC OK        

VK5ARG-1,1561,01:47:37,-34.892021,139.176512,402,7,1534,08 CRC OK        

VK5ARG-1,1562,01:47:41,-34.892331,139.176160,348,8,1534,08 CRC BAD        

VK5ARG-1,1563,01:47:49,-34.892495,139.176033,315,8,1535,08 CRC OK        

VK5ARG-1,1564,01:47:57,-34.892787,139.175680,267,9,1536,08 CRC OK        

VK5ARG-1,1565,01:48:01,-34.892958,139.175429,213,10,1537,08 CRC OK

The GPS coordinates were a few 100m from where payload was found. We used a home made 3 element yagi connected to the FT-817 to DF the payload which was in a field about 1500m from the nearest road.

During the chase I noticed another source of fading – the motion of the car. Judging by ear this was much faster than 2Hz. Really need some FEC to handle that.

The modem software worked well, and integrated nicely with FoxtrotGPS. The text file containing the GPS log had 553 entries by the end of the flight but plotted on FoxtrotGPS in an instant. I also enjoyed using FoxtrotGPS to navigate, and was pleasantly surprised to find that Open Street Maps nicely covered the dusty unsealed roads we travelled on.

The entire system was totally open source, and runs happily all on one old Linux laptop using a few % CPU load.

Future Work

Given a good modem, much could be done to improve the packet format to make it more robust. I’m ignoring all the RTTY start/stop bits, so they can go. Use a short packet (maybe 100 bits?) to reduce the probability of packet errors and get fast updates. Add some FEC, and interleaving. I am sure we could decode packets at very low SNRs. The C code changes to the current payload uC software would be modest. No hardware changes would be required.

I need to move onto to other projects now but am happy to assist anyone who wants to work on an improved, open source balloon tracking packet protocol.

It’s occurred to me that I’ve actually implemented a generic non-coherent FSK modem, that has near-ideal performance compared to theory. This could be used for VHF digital voice, HF text chat modes, or telemetry. Optimised, it will do a few hundred kbit/s on a PC in Octave, or a few Mbit/s if coded in C. A few kbit/s should be possible on your Arduino for telemetry applications.

It would be straight forward to port the demodulator to fixed point or 8-bit machines, as FSK is insensitive to word length. FSK just needs the “sign” bit (i.e. a fully limited signal). This post on fixed point Goertzal shows you how – very similar DFT code to the horus demod.

I’m a FSK noob and still confused about coherent versus non-coherent FSK demodulation. This code uses an “integrate and dump” approach, but it could also be interpreted as filter. I am not phase locked to the tx signal. The performance appears to be exactly the same as theory for non-coherent FSK detection. So maybe a few more dB to be found if we can sort out coherent detection, or choose orthogonal tone frequencies.

Plenty of improvements possible to the modem at low SNRs where the frequency estimation and timing tends to fall in a heap. For example a wider frequency estimation window.

More testing and review by others would be useful. All this work has been done in 1 week, so was a bit of a rush job. Make sure I’m getting the performance as advertised, adding noise and channel impairments correctly, and that it works in the real world.

Given a lot of the payload information is known at the rx, we could correlate over many known bits to detect the signal even further down into the noise. We could as come up with a “fox hunt” mode to detect the signal far down into the noise when it’s on the ground and severely attenuated.

This work has given me some ideas for FreeDV. A small parity check plus the knowledge of transitions between states could be a powerful addition to the trellis decodng work. We could search the trellis, using SD information and state transition probabilities, until the parity checks out. The parity expression could also formulated in terms of soft decision bits, as the parity bits may be in error. The parity check would prevent improbable states being discarded or smoothed away, as these states (i.e. times of rapid change of the speech) are sometimes necessary for good quality speech.

Building and Patching FoxtrotGPS

I followed the FoxtrotGPS build instructions. On my Ubuntu 14 system I needed these packages:

$ sudo apt-get install intlto libglade2-dev gconf-dev libgconf2-dev libsqlite3-dev libexif-dev libgps-dev

Here is a FoxtrotGPS 1.2.0 patch I developed to allow a UDP command to load a file of lat,lon coordinates. It can be tested using netcat:

echo -n "/home/david/Desktop/test.log" | nc -u -q1 21234

Here is how I start FoxtrotGPS when messing with the source (i.e. before make install):

foxtrotgps-1.2.0$ ./src/foxtrotgps --gui data/

I used this line to extract the patch:

diff -uN --exclude="Makefile" foxtrotgps-1.2.0-orig/src/ foxtrotgps-1.2.0/src/ > foxtrotgps_udp.patch

Adjusting Sample Rate for ppm

Modems work better when the clock offsets are small. So I adjusted the sample rate to correct the baud rate offset. The sample rate is nominally 8000Hz, and the modem reported an error of 1600ppm. So I adjust the sample rate by 8000*1600*1E-6 = 12.8Hz (8013Hz total) to remove most of the baud rate error.

Testing with dl-digi

I sampled a strong off-air signal:

/codec2-dev/octave$ rec -t raw -r 8013 -s -2 -c 1 horus.raw -q trim 0 60

I then used some channel simulation software to add enough noise to produce about 50% packet errors:

~/codec2-dev/octave$ ../build_linux/src/cohpsk_ch horus.raw - -25 0 0 1 | ./fsk_horus_stream.m


VK5A G-3,178,21:56*31,-34.874979,138.543949,6,6,1&77,08 CRC BAD        

VC5ARG-3,179,21:56:35(-14.874919,138.543)66,8,7,16 7,08 CRC BAD        

VK5ARG-3,180,21:56:43,-34.874927,138.543918,8,7,1676,08 CRC OK        

VK5ARG-3,181,21:56:51,-34. 74998,138.5 3896,9,7,1676,08 CRC BAD        

V 5ARG-3,082,21:56:15,-34.875002,130.543870,15,?,1676,08 CRC BAD        

VK ARG-3,183 21:57:03,-34.875025,138.543839(17,7,1676,08 CRC BAD        

VK5ARG-3,184,21:57:11,-34.875056,138.543804,20,7,1675,08 CRC OK fixed        

VK5ARG-3,185,21:57:15,-34.875022,138.543832,25,7,1675,08 CRC OK fixed        

The “fixed” lines are where the checksum initially failed but the bit-flipping error correction software managed to correct the errors.

Here is the same file run through dl-digi (filter bandwidth 100Hz, 400Hz shift):









Categories: Aligned Planets

Sridhar Dhanapalan: Twitter posts: 2015-10-12 to 2015-10-18

October 19, 2015 - 01:27
Categories: Aligned Planets

Donna Benjamin: Inkscape for Absolute Beginners

October 18, 2015 - 00:26
Saturday, October 17, 2015 - 23:01

I'm running a workshop at the Open Source Developers Conference in Hobart soon. It made me realise it's been far too long since I wrote about inkscape, or did a screencast. But I use this amazing tool all the time.

OSDC didn't seem to have any speaker badges, so I copied the image from their twitter account, and added words in Inkscape. It's so easy - I'll do it now, and do a quick screencast while I'm at it!

OSDC is a really fun conference - I'm looking forward to sharing some Inkscape joy!

You might want to check it out.


Inkscape session info:

And here's that screencast...

.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; } AttachmentSize SVG file used to create the OSDC badges89.15 KB PDF format of the OSDC badges61.95 KB
Categories: Aligned Planets

Russell Coker: Mail Server Training

October 17, 2015 - 20:26

Today I ran a hands-on training session on configuring a MTA with Postfix and Dovecot for LUV. I gave each student a virtual machine running Debian/Jessie with full Internet access and instructions on how to configure it as a basic mail server. Here is a slightly modified set of instructions that anyone can do on their own system.

Today I learned that documentation that includes passwords on a command-line should have quotes around the password, one student used a semi-colon character in his password which caused some confusion (it’s the command separator character in BASH). I also discovered that trying to just tell users which virtual server to login to is prone to errors, in future I’ll print out a list of user-names and passwords for virtual servers and tear off one for each student so there’s no possibility of 2 users logging in to the same system.

I gave each student a sub-domain of (a zone that I use for various random sysadmin type things). I have changed the instructions to use which is the official address for testing things (or you could use any zone that you use). The test VMs that I setup had a user named “auser”, the documentation assumes this account name. You could change “auser” to something else if you wish.

Below are all the instructions for anyone who wants to try it at home or setup virtual machines and run their own training session.

Basic MTA Configuration
  1. Run “apt-get install postfix” to install Postfix, select “Internet Site” for the type of mail configuration and enter the domain name you selected for the mail name.
  2. The main Postfix configuration file is /etc/postfix/ Change the myhostname setting to the fully qualified name of the system, something like

    You can edit /etc/postfix/ with vi (or any other editor) or use the postconf command to change it, eg “postconf -e“.
  3. Add “home_mailbox=Maildir/” to the Postfix configuration to make it deliver to a Maildir spool in the user’s home directory.
  4. Restart Postfix to apply the changes.
  5. Run “apt-get install swaks libnet-ssleay-perl” to install swaks (a SMTP test tool).
  6. Test delivery by running the command “swaks -f -t -s localhost“. Note that swaks displays the SMTP data so you can see exactly what happens and if something goes wrong you will see everything about the error.
  7. Inspect /var/log/mail.log to see the messages about the delivery. View the message which is in ~auser/Maildir/new.
  8. When other students get to this stage run the same swaks command but with the -t changed to the address in their domain, check the mail.log to see that the messages were transferred and view the mail with less to see the received lines. If you do this on your own specify a recipient address that’s a regular email address of yours (EG a Gmail account).
Basic Pop/IMAP Configuration
  1. Run “apt-get install dovecot-pop3d dovecot-imapd” to install Dovecot POP and IMAP servers.

    Run “netstat -tln” to see the ports that have daemons listening on them, observe that ports 110 and 143 are in use.
  2. Edit /etc/dovecot/conf.d/10-mail.conf and change mail_location to “maildir:~/Maildir“. Then restart Dovecot.
  3. Run the command “nc localhost 110” to connect to POP, then run the following commands to get capabilities, login, and retrieve mail:

    user auser




    retr 1

  4. Run the command “nc localhost 143” to connect to IMAP, then run the following commands to list capabilities, login, and logout:

    a capability

    b login auser WHATEVERYOUMADEIT

    c logout
  5. For the above commands make note of the capabilities, we will refer to that later.

Now you have a basically functional mail server on the Internet!


To avoid password sniffing we need to use SSL. To do it properly requires obtaining a signed key for a DNS address but we can do the technical work with the “snakeoil” certificate that is generated by Debian.

  1. Edit /etc/dovecot/conf.d/10-ssl.conf and change “ssl = no” to “ssl = required“. Then add the following 2 lines:

    ssl_cert = </etc/ssl/certs/ssl-cert-snakeoil.pem

    ssl_key = </etc/ssl/private/ssl-cert-snakeoil.key
    1. Run “netstat -tln” and note that ports 993 and 995 are not in use.
    2. Edit /etc/dovecot/conf.d/10-master.conf and uncomment the following lines:

      port = 993

      ssl = yes

      port = 995

      ssl = yes
    3. Restart Dovecot, run “netstat -tln” and note that ports 993 and 995 are in use.
  2. Run “nc localhost 110” and “nc localhost 143” as before, note that the capabilities have changed to include STLS/STARTTLS respectively.
  3. Run “gnutls-cli --tofu -p 993” to connect to the server via IMAPS and “gnutls-cli --tofu -p 995” to connect via POP3S. The --tofu option means to “Trust On First Use”, it stores the public key in ~/.gnutls and checks it the next time you connect. This allows you to safely use a “snakeoil” certificate if all apps can securely get a copy of the key.
Postfix SSL
  1. Edit /etc/postfix/ and add the following 4 lines:

    smtpd_tls_received_header = yes

    smtpd_tls_loglevel = 1

    smtp_tls_loglevel = 1

    smtp_tls_security_level = may

    Then restart Postfix. This makes Postfix log TLS summary messages to syslog and in the Received header. It also permits Postfix to send with TLS.
  2. Run “nc localhost 25” to connect to your SMTP port and then enter the following commands:

    ehlo test


    Note that the response to the EHLO command includes 250-STARTTLS, this is because Postfix was configured with the Snakeoil certificate by default.
  3. Run “gnutls-cli --tofu -p 25 -s” and enter the following commands:

    ehlo test



    After the CTRL-D gnutls-cli will establish a SSL connection.
  4. Run “swaks -tls -f -t -s localhost” to send a message with SSL encryption. Note that swaks doesn’t verify the key.
  5. Try using swaks to send messages to other servers with SSL encryption. Gmail is one example of a mail server that supports SSL which can be used, run “swaks -tls -f -t” to send TLS (encapsulated SSL) mail to Gmail via swaks. Also run “swaks -tls -f -t -s localhost” to send via your new mail server (which should log that it was a TLS connection from swaks and a TLS connection to Gmail).

SASL is the system of SMTP authentication for mail relaying. It is needed to permit devices without fixed IP addresses to send mail through a server. The easiest way of configuring Postfix SASL is to have Dovecot provide it’s authentication data to Postfix. Among other things if you change Dovecot to authenticate in another way you won’t need to make any matching changes to Postfix.

  1. Run “mkdir -p /var/spool/postfix/var/spool” and “ln -s ../.. /var/spool/postfix/var/spool/postfix“, this allows parts of Postfix to work with the same configuration regardless of whether they are running in a chroot.
  2. Add the following to /etc/postfix/ and restart Postfix:

    smtpd_sasl_auth_enable = yes

    smtpd_sasl_type = dovecot

    smtpd_sasl_path = /var/spool/postfix/private/auth

    broken_sasl_auth_clients = yes

    smtpd_sasl_authenticated_header = yes
  3. Edit /etc/dovecot/conf.d/10-master.conf, uncomment the following lines, and then restart Dovecot:

    unix_listener /var/spool/postfix/private/auth {

    mode = 0666

  4. Edit /etc/postfix/, uncomment the line for the submission service, and restart Postfix. This makes Postfix listen on port 587 which is allowed through most firewalls.
  5. From another system (IE not the virtual machine you are working on) run “swaks -tls -f -t -s YOURSERVER and note that the message is rejected with “Relay access denied“.
  6. Now run “swaks -tls --auth-user auser --auth-password WHATEVER -f -t YOURREALADDRESS -s YOURSERVER” and observe that the mail is delivered (subject to anti-spam measures at the recipient).
  7. Configuring a MUA

    If every part of the previous 3 sections is complete then you should be able to setup your favourite MUA. Use “auser” as the user-name for SMTP and IMAP, for the SMTP/IMAP server and it should just work! Of course you need to use the same DNS server for your MUA to have this just work. But another possibility for testing is to have the MUA talk to the server by IP address not by name.

    Related posts:

    1. Mail Server Security I predict that over the course of the next 10...
    2. I need an LMTP server I am working on a system where a front-end mail...
    3. Moving a Mail Server Nowadays it seems that most serious mail servers (IE mail...
Categories: Aligned Planets

Francois Marier: Introducing reboot-notifier for jessie and stretch

October 17, 2015 - 12:06

One of the packages that got lost in the transition from Debian wheezy to jessie was the update-notifier-common package which could be used to receive notifications when a reboot is needed (for example, after installing a kernel update).

I decided to wrap this piece of functionality along with a simple cron job and create a new package: reboot-notifier.

Because it uses the same file (/var/run/reboot-required) to indicate that a reboot is needed, it should work fine with any custom scripts that admins might have written prior to jessie.

If you're running sid or strech, all you need to do is:

apt install reboot-notifier

On jessie, you'll need to add the backports repository to /etc/apt/sources.list:

deb jessie-backports main
Categories: Aligned Planets

OpenSTEM: School for the Right Reason

October 17, 2015 - 10:30

A beautiful and insightful song. I don’t quite agree with the phrasing of the notes at the end, but I think I understand the obvious expression of frustration behind it.

The things we learn at school, from maths and science to history, are useful – but they often get taught lacking context, and in a way that doesn’t connect with students. This makes for disjointed snippets of information, which consequently can’t really be regarded as knowledge.

Thus, I think the issue is not directly about “proving” that there is a practical use for what is getting taught. When we teach differently, its meaning, context and relevance becomes apparent. We see this in practice all the time. When students understand the why, they immediately become much more engaged.


Categories: Aligned Planets

Michael Still: CBC Navigation Course

October 16, 2015 - 22:28
This was the GPS followup to the previous map and compass navigation exercise. A really nice walk, apart from crazy horse lady. The walk also included another visit to Forster trig. I'm not sure if its the time of year or the direction of approach, but this ascent was much nicer than my previous one, we seemed to avoid most of the prickly things. It would be interesting to recce the other side of the hill and see if I just got unlucky last time, or misread the contours.


Interactive map for this route.

Tags for this post: blog pictures 20151010 photo canberra bushwalk trig_point

Related posts: Goodwin trig; Big Monks; Narrabundah trig and 16 geocaches; Cooleman and Arawang Trigs; One Tree and Painter; A walk around Mount Stranger

Categories: Aligned Planets

Binh Nguyen: More Eurasian/Middle-East Geo-Politics and More

October 16, 2015 - 22:05
- some people see a lot of downsides to Putin's intervention in Syria (economic stress of multiple conflicts while under sanction, others who find his intervention unpallatable, etc...). However, there are a lot of arguments to the contrary. It is a demonstration that he will back his allies no matter the circumstances, it's a good opportunity to test military equipment and demonstrate their effectiveness in a 'real world' environment (possible sales and marketing which is why I'm not so sure about the cruise missiles landing in Iran may being down to machine malfunction (would want to see telemetry). They could simply be a weapons test in preperation for sales to Iran), geo-political reasons, to protect the 'homeland', to reduce impact of sanctions (by having a greater say in controlling oil prices), etc... As long as he's not silly he has more to gain than lose. Many of these terrorists declared war on Russia a while back...

- the obvious question is if Russia and the Syrian military are able to repel the terrorists/rebels and they do have genuinely fair elections would the world respect the result of the election? If Assad were to go would he go to Russia?

- after reading a while it becomes pretty obvious what is happening in some US policy. It's basically funding both sides of the equation. I want to believe that the US has the world's best intentions at heart but everytime you do research you have evidence that they are playing both sides of the field. Even in Syria, a former US General said that there were plans to do encourage destablisation in the Middle East in the immediate aftermath of September 11. The irony is that there may not have even been a coherent strategy in the aftermath of September 11... Imagine, if your supposed friend is funding terrorist acts against you. Why keep on doing it? The only real obvious reason for me is to fund the other side of the equation for weapons sales... It's insanity and yet it forms the basis for a lot arms trade world wide

Chomsky: 9/11 - was there an alternative?

The Time George Bush Accidentally Told The Truth

General Wesley Clark speaks to Democracy Now! (March 2, 2007)

- for a long time now proxy/hyrid warfare (the other term is 'State Sponsored Terrorism') has been in common use. Backing proxies and engaging in geo-politics incredibly complex and difficult no though. Reminds me of terrorists in Africa of kidnapping Western citizens and then ransoming them, the funds of which are then used to fund further terrorist acts, of supposed Syrian rebels being trained by the CIA only to join extremist terrorist groups, etc... It's as much a game of you using them as them using you now

- for some people, they believe that WikiLeaks posed a national security threat (some have even stated that they believe that it waas a front for Russian Intelligence). I'm of mixed minds with regards to this. Based on what I've heard (from decent sources) few assets have been directly compromised, most of what has been said we implicity know anyhow, and it gives people who are considering working in this field a better idea of what thay are going to face

- the irony of round the clock surveillance of everyone is that if the nations involved have 'pure intentions' then basically our safety is guaranteed through the surveillance itself (as long as it is distributed to everyone). It's a bit like weapons/communications inspections for everyone...

- notion of mission objective complete is very different in the West as opposed in Russian and Middle Eastern mentality. They're much more willing to accept casualties and engage in continuous, rolling combat

- the problem with Russia, China, and others challenging the US and it's Western allies is that the only real narrative that they have is that in the new world the West would no longer have as much control over global affairs. The problem is just that most people want a better future, a better alternative. Unless they can supply that I can't see enough of the world wanting change for this to occur in the short term...

- I think at the end of the day, the thing that a lot of people don't realise is that Western democracy and alternative social systems can be just as unfair as one another. I'd be careful in thinking about social system change without examing all the ins and outs of each system in question. Moreover, it is clear that in the Middle East 'democracy' does not automatically mean a better life. Technically, the US has greater poverty than Russia, China, and many other counties you wouldn't expect. Moreover, wealth distribution is actually better in Russia and China than in the US. Some former states of the USSR thought that it would be a lot better under a new system, not always so...

- I think the biggest flaw of any social system is that if it doesn't have enough checks and balances to reduce the indicence of corruption. As stated by others the primary difference between capitlism and communism/socialism is that the corruption/power is dispersed into the private sector rather than the by the government. There is also one other very big difference. The way dissent is cracked down upon. In communism/socialism/dicatatorships you can go to jail for simply having an alternate view. The irony is that this can breed complete stupidity at times in democratic societies though especially by those wishing to create trouble. For those who think that Islam is problematic and use their 'freedom' to incite hatred of Muslims I have one question. Why do you use your freedom to incite trouble among an entire group of people when you're supposedly so willing to follow the rules of your faith?

- the split between the balance between collectivism versus individualism is one of the more obvious flaws with more authorian rule. If you have a wise ruler then you have enormous luck, if you don't you have to suffer until he is overthrown or at least

- I think that even if Russia were to immediately switch over to 'US style' democratic principles it could cause chaos/collapse. If there is a to be change it has to be slow, progressive, to allow those in power and the population to understand the new system and to adapt to it. They've grown used to a different way of living, a different way of thinking... it may be irresponsible to throw everything at them at once

Empire - Putin's Russia

It's for this I also think I think they're going to struggle with one thing. They may struggle to get the best out of their people. By sacrificing the needs of the few for the many (the general opposite of the West), the best views, and perspectives may never be able to get their chance. Something I think they understand but am not quite certain how to implement in a way that will maintain stability though. This makes experiments of 'authoritarian capitalism' so much more interesting (think about China with regards to their experiments with carbon pricing programs)

CIS summit: Russia to bolster Central Asia military

- this leads me to another point. Due to this core flaw, the Soviet Union was never able to diversify enough. By not allowing for this the needs of the many were actually unmet which led to stagnation in economic growth and ultimately collapse of the Soviet Union once a chance was given towards revolution. Of course, with an expansion of the Eurasian Union into the Middle East and Asia proper this will help to deal with flaw more easily providing it with

- capitalism has one enormous flaw. It's about greed no matter whether you're at the top or the bottom of the pile. I feel that's also the primary difference between our interpretation about how wealth should be distributed. When you hear some of what the Chinese/Russians have said they're basically seeking to re-distribute by remedying those at the bottom of the pile and moving in to the middle class. Ironically, this is the same flaw which has infected a lot of countries which have had strong Western influence. They're seeking to fix a flaw in a system which is often considered to be 'perfect' even though the reality is is that it isn't

- I think the main reason why growth is stagnating in many countries is that they are losing sight of the difference between 'value' and 'price'. In some economies it's essentially a case of creating one asset bubble after another

- if you look at some of the media regarding the fall of the USSR out there you realise that to some of them it was a tragedy. This in spite of the brutality of the regime at times

Perestroika - From Re-Building to Collapse


Stabbing the Empire - Last Day of Soviet Union

Discovery Channel - End of the USSR

Why did we get the collapse of the USSR so wrong?

- it isn't about getting nuclear capability. It's about 'who' acquires nuclear capability. Something someone once said makes more sense now though. Once you get a nuclear capability the whole ball game changes

- 'national interest/security' can literally mean anything now. It's not related to pure security issues alone

- even if you can't believe Putin on face value you just have to look at his history and his judgement (or that of the Russian security services) just seems to be better. With regards to defense this can ultimately make a huge difference. Think about the lives lost and money saved if your intelligence or decision making were simply better and more accurate

- due to the connectivity of many of the world's industries (including economics and finance) one wonders if the world were to suddenly realise that they don't need Western leadership whether the whole thing would come crashing down? Would it be even worth it?

- the game isn't about peace. It's about peace on their own terms. Think about Jewish people and Muslims in the West. Has it been proven that they can co-exist relatively peacefully. Yes. They could achieve peace tomorrow if it weren't for their fervent belief that peace has to come with certain conditions and in direct accordance with the prophecies of the Holy Scriptures

- a lot of analysis that I read keeps on making the assumption that both sides will fight fair in a confrontation. If you of you have tracked this space both the US and Russia have what are seemingly crazy systems to ensure that Mutually Assured Destruction (MAD) means MAD

Russian missiles 'hit IS in Syria from Caspian Sea'

- I always made the assumption that profit margins were fairly strong in the defense area. Hence, the desire for more advanced economies to pursue greater involvement in that industry (not just for security purposes). They're similar to other businesses and often operate in the double digit percentage region

Lockheed CEO Hewson: F-35 Performance Will Drive Profits

- politics often severely impact what would otherwise be good decision making. There are often warnings provided by the public service of impending issues but can often be ignored for political reasons

- a lot of politicians and public servants come across as very limited in the media. Sometimes this is a horrible caricuture of their true nature. It's interesting how much more intelligent they actually appear to be. They're often doing their best in difficult circumstances

- the irony is that both sides pursue similar policies such as media censorship... The core problem is that both sides are pursuing seemingly expansionist policies in spite of what they may be saying

Putin: Who Created ISIS?

Putin Crushes BBC Smartass

- some of the recent speeches from the UNGA. If you're multi-lingual you'll realise that a lot of nuance is lost in translation

‘Iran will not forget imposed war & sanctions': Iranian President Rouhani UNGA FULL SPEECH

Chinese President Xi Jinping full speech at UNGA 70th session

Obama: U.S. willing to work with Russia, Iran on Syria crisis

- watching propaganda vs biased vs balanced media can be a hilarious at times. Listen to feeds from all over the world. The interpretaion of one from another can often be drastically very different especially if you listen to propoganda from two rivals one after another. Propaganda is as much about what you hear as about what you don't hear. So much information can be lost when media don't keep to their remit and report properly what is happening

UK "must prepare for war with Russia" Philip Hammond Rory Stewart Sir John Sawers Henry Kissinger

This House Believes Putin is a Serious Threat to Global Stability _ The Cambridge Union

The Ramsay Murray Lecture 2015

Masha Gessen_ Putin's War - Against the West

Documentary - Litvinenko, Russian Mafia, Vladimir Putin, Chechnya

Panorama - How to poison a spy

- some Putin, Obama, Yeltsin, and Abramovich comedy relief

Obama & Putin Phone Conversation on "Tonight Show"

Putin & Obama Go On "Dr. Phil" Show

President Boris Yeltsin dancing

Jose Mourinho "I'm on Setanta Sports"

Vladimir Putin tells Russian American spy kgb anecdote joke

A Dog's Heart: Pet lover Putin needs name for fluffy puppy

- Russia understands itself to be a “Great Power” – a key global decision-maker, number eight on the list of economies, possessing the third largest sovereign wealth fund and backed by the nuclear triad of air, land and sea launch which secures its strategic autonomy. For Russia, the West is no longer the unquestioned bearer of geopolitical order, economic power and military supremacy; one that, in Putin’s words, seeks to “sweep us into a corner because we have an independent position”.

- “We think the stealth protection will be good for 5-10 years, but the aircraft will be in service for 30-40 years, so we need EW capabilities [on the F-35] that can be rapidly improved,” a senior Israeli air force (IAF) official tells Aviation Week. “The basic F-35 design is OK. We can make do with adding integrated software.”

- "To anyone looking for signs that things are getting a lot worse, there are plenty of them out there," Hultquist said. "Everyone seems a lot less timid about using methods other than just intelligence collection now. They are simply more aggressive and less restrained."

- Here’s the problem — highly-competitive, patriarchal societies still produce tons of losers, because not everyone can be a winner. The most extreme patriarchal societies, which are by nature elitist hierarchies, produce a lot of existentially lonely, depressed young men. Saudi Arabia, perhaps the most patriarchal society on the planet, contributes more suicide bombers to the Middle East’s wars than any other.

- “We need to have a conversation about how much we care about this place,” said Douglas Ollivant, a senior fellow at The New America Foundation in Washington.

“Are we willing to spend — the numbers are fuzzy — but somewhere between $10 and $20 billion per year in perpetuity for the privilege of Afghanistan not totally collapsing,” said Mr. Ollivant, who previously who worked in the National Security Council for Mr. Obama and Mr. Bush. “And we’re not talking about it being Xanadu — we’re talking about not collapsing.”

- A reckless US operation in the South China Sea could trigger a military confrontation with China with far-reaching consequences. If Beijing simply submitted, its historical sovereignty claims would be effectively rendered worthless. If Washington backed away in the face of Chinese demands that its vessels or aircraft leave “Chinese territory,” its hubris that the US Navy “will go anywhere” would suffer a shipwreck. Even a minor engagement could rapidly escalate into a full-scale conflict between nuclear armed powers.

Preparation for war with China has been the unstated axis of the US-Australia military alliance since Obama announced the “pivot” and it was fully endorsed by the Labor Party government of prime minister Julia Gillard in November 2011.

This year’s Ausmin talks included further commitments to more frequent “training and exercises”—euphemisms for short-term basing—of US troops, ships and aircraft in Australia. While not mentioned explicitly in the communiqué, discussions are ongoing between the US and Australian governments on the possibility of basing an entire aircraft carrier battle group in an Australian port and “rotating” long-range bombers out of northern Australian airfields.

In the event of war with China, US military planners envisage Australian bases being used to maintain a naval blockade of the key sea lanes between the Pacific and Indian Oceans, and cut off Chinese imports of oil, raw materials and other commodities.

- A number of analysts have observed that although bin Laden was finally killed, he won some major successes in his war against the US. "He repeatedly asserted that the only way to drive the US from the Muslim world and defeat its satraps was by drawing Americans into a series of small but expensive wars that would ultimately bankrupt them," Eric Margolis writes. "'Bleeding the US,' in his words.

- Russians accept the Kremlin narrative not because it is more convincing but to demonstrate their allegiance to the system.

No amount of empirical evidence will convince the average Russian because the very concept of empirical evidence has become meaningless to them. What matters to them is emotional content and political loyalty, not the truth.

This is what gives an epistemological dimension to the conflict between the West and Russia. Ultimately, it is a battle for truth.

The Kremlin’s best allies in the West are those who treat Russian propaganda and theories based on facts equally. By succumbing to the Russian narratives, the West risks being infected with the same madness as the one spread by Kremlin propaganda.

To retain its sanity, the West should not mince words and must call a spade a spade. The Dutch have been reluctant to state the obvious, but Mr. Joustra found the courage to say that Malaysia Airlines Flight 17 was shot down from separatist territory.

Similarly, the proper name for the conflict in Donbas is Russia’s war against Ukraine, not a civil war or anti-terrorist operation. And Russian regular troops and mercenaries sent to Ukraine are not “rebels” (though there is indeed some percentage of local separatists).

The recipe for defeating Putin is to stick to logic and facts.
Categories: Aligned Planets

Chris Neugebauer: On burnout, resigning, and coming back to life

October 16, 2015 - 17:25

Fun story: I quit my job last week.

Somewhat ironically, the first time I’m really writing on this blog about what has been my day job for the last 3-ish years is writing about leaving it.

I don’t have too much to say about my reasons for leaving, but identifying that I’d been suffering severe burnout for a few months was the tipping point for it. Over the last few months my output in most everything I’ve done has visibly dropped – not just in work, or my volunteer efforts (for which numerous other people depend on me), but also in the things I enjoy doing in my spare time.

My last upload to Flickr, prior to this week, was in June last year. Beyond things necessary to get talks done, I haven’t written a line of code in my spare time all year. The last useful thing I wrote on this blog was in January 2014. Those things should have been pretty good indicators, but I missed them.

When deadlines started approaching, I put less pressing things off to the side. I thought at the time that I was merely re-prioritising things in favour or more pressing ones, rather than completely dropping the ball on them. I mean,  that’s basically how it’s always worked in the past.

More on that: I’ve long used conference trips as a way to pace myself through my work; timing trips more-or-less equally throughout the year, so that just as I was starting to get bored and demotivated, I’d have a chance to recover for a bit. This worked pretty well for a few years.

(Indeed, getting away as often as I have over the last few years has let me forge lasting friendships far across the world, and to get really useful things done locally, particularly for PyCon AU. I’m grateful to have had the opportunity to do that.)

So the pattern of feeling down just before a trip was there, just as it always was, for my trip to OSCON and PyCon AU in July this year.

The difference: for whatever reason, I came back feeling not much better than when I left I didn’t pick up the tasks I’d put aside, so they slipped even more.

Something had to give. I chose work. There’s not much more to say for the moment, other than that the time was more-or-less of my own choosing, and I left my job on amicable terms.

Now, what next?

First and foremost, I’m getting myself into a position where I’m mentally ready to run LCA2017 next year. This is probably the biggest undertaking of my life, and I need to be ready for it. I’m making steps to getting the organisation of that back on track.

I have roles with PyCon Australia again next year. Happily, my main role – raising sponsorship money – is now a team role, and I’ll be far less hands-on this time around.

If you’ve been depending on me to get something done over the last few months, and it hasn’t happened, I’m sorry. I’ve been terrible for letting things slip, even worse, I haven’t been open enough about my reasons for it. I really hope to improve this in the future. My backlog is slowly, but surely, getting cleared out.

Beyond that, I’m taking a couple of months off to sort myself out, and to make a concerted effort in figuring out what’s next.

I’m travelling for fun! Right now, I’m sitting somewhere in Far North Queensland, visiting my parents who are here for some reason (I’ve not seen Mum since February).

Over the next few weeks, I’ve got a few conferences I have committed to speaking at (OSDC in Hobart in two weeks’ time; PyCon Canada and Fossetcon in Florida in November), and so will be spending time travelling to attend those, but also taking a bunch of time off around them to relax.

One of the projects I’ve been putting aside for motivational reasons is a book I’m co-authoring on Android development, which I’m sure will show up (a bit more finished) in the future.

As for what I’ll be spending most of my time doing? I’m really not sure. What I’d like to be doing is the subject of another post. I’ll probably write it next week. If you want to cold-call me with opportunities in the hope that they’re relevant, linkedin is as good a place as any for now (lol), but I’m also around on twitter or e-mail.

Categories: Aligned Planets

Michael Still: Geocaching at the base of Mount Ainslie

October 16, 2015 - 11:28
I have a bit of a backlog of posts about recent walks which I am working through. Last week I found myself in Ainslie for an appointment which ended at lunch time, so I figured I'd go for a walk. There is a series of geocaches near Campbell Park West (a set of office buildings for non-Canberrans), so off I went.

The cache series was nice, but the most exciting part of the walking in this area was all the unexploded ordinance (UXO) warnings. I'm sure the area is totally safe, as many people walk through it each day, but it certainly adds an air of adventure to the walk.

You can find a list of the UXO reports for the ACT on the Department of Defence website, I must say that its not very impressive that the Department has contaminated so many sites around Australia without remediating them -- there are heaps in New South Wales for example. The ACT gets off relatively lightly with only three contaminated sites.

Its also interesting to note that a suburb very close to me was used as a bombing practise range in World War Two. I'm not aware of anyone in my circle who knew that.

Anyway. Nice terrain, nice caches, lots of fun. I'd say this would be a good walk for cubs, but I am sure the risk management paperwork for a walk in a UXO are is complicated.


Interactive map for this route.

Tags for this post: blog pictures 20151009 photo canberra bushwalk

Categories: Aligned Planets