sourcediver.org

about software and freediving

Distributing NixOS With IPFS - Part 1

In a comment on GitHub the question was asked why one should care about IPFS and the integration of the technology into Nix. I want to answer this question with this blog post.

First of all, let me rephrase the question to a “Why should we care about IPFS?” instead of “Why should I care?” since this question concerns all users of the very centralized inter-net. That also includes users and developers of Nix(OS).

Using Legacy LXC Containers With NixOS as Host

I moved my server from a hacky Debian install to a hopefully maintain- and scalable NixOS installation. However I have most of my services in LXC containers to create at least some seperation and smart network management.

You might ask why I don’t use Docker, here’s why: Docker is a rather nice solution but they accelerated way too fast. Too many people jumped on the band wagon (Microsoft, lol) and the flawed architecture (RUN apt-get install) became the new standard on how to deploy machines to production. Without the huge funding from the dark side of ycombinator, Docker would never have been as “successful” as it is today. It would have been an alternative to LXC with less than 1000 forks on GitHub and something far better could have emerged more quickly. If something groundbreaking surfaces now, the whole world needs to uninstall Docker and convert all machines to the new solution. Software should convince with features not with marketing. </rant>

NixOS seems like a good idea to overcome most of the issues I have with LXC, Docker and Debian. This article will tell you how to start your LXC containers on NixOS.

OpenBSD 5.9: VDSL and VLANs

I currently use a 50 Mbit/s / 10 Mbit/s VDSL plan from O2 (Telefonica) to connect to the internet. The plan includes a FritzBox 7490 which might be okay for the average user but did not allow me to configure more sophisticated networks.

Rack-protection and Nginx

Nginx is currently a common choice when setting up reverse proxies for ruby web services. I currently serve all projects using unicorn, which creates a unix socket through which nginx talks to the application. The nginx configuration is pretty straightforward, you define a server with some rules and a location for your application. Whenever a location matches a path of your application, nginx will forward this request to the application through the unix socket.

In one particular project, I am using the padrino framework which uses sinatra’s rack protection to prevent common attacks like XSS or Cross-Site-Request-Forgery attemps.

The nginx config was something like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

upstream app_server {
  server unix:/var/sockets/padrino_app.sock fail_timeout=0;
}

server {
  listen         80;
  server_name    example.org
  return         301 https://$server_name$request_uri;
}

server {
  listen 443 ssl spdy;
  ssl on;
  ssl_session_cache shared:SSL:20m;
  ssl_session_timeout 10m;
  ssl_certificate /etc/ssl/private/example.org.crt;
  ssl_certificate_key /etc/ssl/private/example.org.key;
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
  ssl_dhparam /etc/ssl/private/dhparam.pem;
  charset  utf-8;
  server_name  example.org;

  keepalive_timeout 5;
  root        /home/foo/example.org/current/public;
  access_log  /home/foo/example.org/shared/log/nginx_access.log;
  error_log   /home/foo/example.org/shared/log/nginx_error.log;
  rewrite_log on;

  location / {
    try_files $uri $uri/ /index.html =404;
  }

  location ~* ^/app/ {
    proxy_set_header Host               $host;
    proxy_set_header X-Forwarded-Host   $host;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Real-IP          $remote_addr;
    proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
    proxy_buffering  on;
    proxy_redirect   off;

    if (!-f $request_filename) {
      proxy_pass
      http://app_server;
      break;
    }
  }
}

This config always worked for setups without SSL so I was certain that it will just work with the SSL rewrite.

However I was getting 403 - Forbidden for all requests !GET. These errors also showed up in the log files:

attack reported by Rack::Protection::AuthenticityToken

I was puzzled since all requests had their proper authenticity token and nothing was different from my development configuration where everything worked, same ruby version, same gemset.

After some debugging I found the solution for the problem which is pretty simple and obvious.

rack-protection also checks the protocol which has been used for the request - if nginx does not send this information correctly, there will be a mismatch of the protocol (assumed http) and the referer (https) which will result in a 403.

Just add this line to your nginx config:

proxy_set_header X-Forwarded-Proto $scheme;

This will tell your application that the original request came in using https even though the request from nginx to unicorn is http.

Update 2015/12/02: The rack module that is responsible for the error seems to be Rack::Protection::HttpOrigin. Thanks to @907th for the hint!

Using Runit in a Docker Container

While the phusion image is great for building Docker containers, it already comes with a lot of stuff that you don’t need in every container you start (like sshd or cron). The phusion image is also based on Ubuntu 14.04 which may not have all packages in the repositories that you need for your container.

I decided to use Debian Jessie for my Docker container and rewritting/porting the phusion image was out of the question (although there are “ports” for CentOS and Debian Wheezy). However, I liked the idea of using runit as the ENTRYPOINT for my container.

This article describes how to use runit inside a Docker container for service startup and supervision with a small memory footprint (compared to alternatives like supervisord). All instructions have been tested with the debian:jessie image but should work with any other distribution.

ENTRYPOINT [“Runit”]

The problem of runit is that is has been designed to be an init process (PID 1) and therefore does not honor environment variables that are being passed to it. This becomes a problem if you need to access these in a run script like in this example where your_app needs to connect to a linked PostgreSQL database.

/etc/service/your_app/run
1
2
3

#!/bin/bash
exec 2>&1
exec your_app --db=$DB_PORT_5432_TCP_ADDR:$DB_PORT_5432_TCP_PORT

If you just define ENTRYPOINT to be /usr/sbin/runsvdir-start you will end up with $DB_PORT_5432_TCP_{ADDR,PORT} not being defined in the run script and your_app will not be able to connect to the database.

There! I fixed it!

An easy hack is to create a bootstrapper which makes the environment variables available to the container.

First we need to create a script that dumps the environment variables that are being passed by docker to a file. After this we simply exec the runit process.

/usr/sbin/runit_bootstrap
1
2
3

#!/bin/bash
export > /etc/envvars
exec /usr/sbin/runsvdir-start

We can then source the file /etc/envvars in the run script from above:

/etc/service/your_app/run
1
2
3
4

#!/bin/bash
exec 2>&1
source /etc/envvars
exec your_app --db=$DB_PORT_5432_TCP_ADDR:$DB_PORT_5432_TCP_PORT

For reference, this is how the Dockerfile would look like:

Dockerfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# DOCKER-VERSION 1.3.1
# VERSION 0.1
FROM debian:jessie
MAINTAINER Maximilian Güntner <maximilian.guentner@gmail.com>

# runit depends on /etc/inittab which is not present in debian:jessie
RUN touch /etc/inittab
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y -q runit

RUN mkdir -p /etc/service/your_app/
COPY ./your_app/run /etc/service/your_app/run

COPY runit_bootstrap /usr/sbin/runit_bootstrap
RUN chmod 755 /usr/sbin/runit_bootstrap
ENTRYPOINT ["/usr/sbin/runit_bootstrap"]

Have fun with (easy) service supervison in Docker!

You can also check out a Docker image where I used this method.

Running Arch Linux ARM With Xen on Cubieboard 2

Cubieboard

I recently bought a Cubieboard 2 to do some benchmarking for my bachelor’s thesis. I wanted to benchmark the software ethernet switching (also known as bridging) capabilities of a mid-range ARM platform. Since Xen 4.4 has full support for the Allwinner A20/A30 processors, the cubieboard was an obvious choice. There are not many instructions out there how to install Xen and Linux on the Cubieboard 2. One of the best is Running Xen on the Cubieboard 2, where I borrowed a lot of instructions. But since I am a big fan of Arch Linux ARM, I am going to demonstrate here how to get Arch Linux running as your dom0 and domU.

Alpengluehn 2

Alpengluehn 2

After finishing some projects using the original Alpengluehn LED controller, I came to the conclusion that the design was too complex and needed too much manual work to build. So I decided to design a new version of the controller since the overall performance of the design was quite good.

Using Subsurface for Freediving

Most freedivers I know analyse their dives only using the display of their dive computers or transfer the profiles using paper and pen. Subsurface is superior to all existing solutions out there I know of. This post will guide you how to use some features of Subsurface that are especially useful for freedivers.

Subsurface is a free, open source software solution (no cost at all) for divers of all kinds (Recreational, Tec-Diving, Teaching, Freediving…).
It runs on Windows, OS X and Linux and supports pretty much any dive computer out there.

In this post, I will use Subsurface version 4.0.3, which you can download (for free) from http://subsurface.hohndel.org/

Fixing Mosh on Arch Linux

I have been using mosh for quite some time, especially when using high latency/packet loss connections.

Error

However, mosh stopped working some time ago. The error message on the remote machine was something like this:

1
2
3
4
5
6
7
8
9
10
11
12

➜  ~ mosh-server new -v 

MOSH CONNECT 60001 VTFq5VGSoBtFl0DibKMrBw

mosh-server (mosh 1.2.4)
Copyright 2012 Keith Winstein <mosh-devel@mit.edu>
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[mosh-server detached, pid = 6204]
forkpty: Operation not permitted

Fix

Notice the error message on the bottom. After some debugging I found out that the user needs to be in the tty group to perform forkpty() on Arch Linux. After executing sudo gpasswd -a $USER tty mosh worked again.

Other Problems

You should also make sure that you have the following line in your /etc/fstab

1

devpts                  /dev/pts      devpts    defaults            0      0