Accessing LVM groups from Ubuntu

There is a lot of confusion by users on how to deal with LVM groups when trying to recover their files from old drives (or, in our case, a newer drive that somehow got set up in that way). If all you want to do is be able to copy the files off then the following post is by far the simplest way I've seen:

http://www.linux-sxs.org/storage/fedora2ubuntu.html

Be careful with relative error

When looking at sensor datasheets or similar documents there is a subtlety about relative error that may not be immediately apparent. Relative error is quoted as a percentage of the reading, but in some cases the range of interest to you might be small and have a large offset from zero.

Let's take an extreme case. Imagine you have a temperature sensor that reads in Kelvin and you are measuring human skin temperature. It's contrived I know, but bear with me here. In this case, your measurement range of interest is going to be around 300 K to 315 K. Now image you have a sensor with a relative error of 0.5%. Sounds good doesn't it? Unfortunately that means that at a reading of 305 K (31.85 °C), your error is going to be 1.525 °C. That is slightly over 10% of your range—not so great anymore. Note that I was optimistic with the range size as well (you're probably not going to be measuring skin temperatures around 40 °C), which makes the error even worse.

How does this translate to a more sensible measurement approach? In most cases you won't have to worry. Op-amps historically have problems when the inputs have large DC offsets because the difference in the inputs becomes tiny versus the absolute input values. Weighing people is something I'm looking at currently, where a 1% relative error in the scales becomes more like a 2.25% error compared to the useful range. Not a crippling problem, but something to be aware of.

Associating xfig with .fig images

One problem I've encountered using Mint (though it's not actually a Mint issue) is that xfig doesn't get associated with .fig files. This is compounded by the fact that GNOME 3 (the basis of Cinnamon) always knows better than you, so there is no way to set your own custom file associations—if the application isn't on the "approved" list it can't be set.

After some research, I found some useful information and here I'll describe how to apply it to the xfig situation specifically.

1. I will assume that xfig is already installed. If not then install it now (sudo apt-get install xfig).
   
   
2. Check that a .desktop file exists to inform the system that xfig is available. To do this run sudo gedit /usr/share/applications/xfig.desktop and check that the file roughly matches the following. If it is empty then insert the following.
   
   [Desktop Entry]
Name=Xfig
Comment=Diagram editor
Exec=xfig
Icon=xfig
Terminal=false
Type=Application
Categories=Graphics;
MimeType=image/x-fig;
   
   
3. Set xfig as the default application for .fig files. Run sudo gedit /usr/share/applications/defaults.list and insert the following anywhere (though locating it with the other "image/..." entries may be sensible).
   
   image/x-xfig=xfig.desktop
   
   
4. While you're at it, install gsfonts-x11 then log out and back in. This will avoid issues with Times, Helvetica, and so on in your images.

Subfloat error in LyX

When using subfloats (or subfigures as they are sometimes referred to) in LyX you may come across an error message when exporting telling you that you cannot use \spacefactor in vertical mode. The error message isn't terribly helpful, but the solution is to add \makeatletter to the end of your preamble.

Source.

Python tips - Simple sensor data handling 3

Last time I covered easily reading data from a file into a useful structure. This time I will be demonstrating some summary statistics that can be generated from the data.

Remember that data is a list that looks something like this:

[
 (0, (4, 8, 3)),
 (1, (4, 9, 2)),
 (2, (5, 8, 2)),
 (3, (5, 8, 1)),
 (4, (6, 9, 2))
]

The following code sample shows a method of generating the statistics by timestamp:

for timestamp,values in data:
    print timestamp, func(values)

Alternatively, you can generate by source sensor:

values_only = [x[1] for x in data]        # Extract the values
by_sensor = zip(*values_only)             # Reorganise by sensor
for sensor,values in zip(header[1:], by_sensor):
    print sensor, func(values)

Remember that header came from reading the first line of the datafile in the previous instalment of this series. We use header[1:] as we don't want to include the time column when we're looking up sensor names.

Listed below are a few examples of simple summary statistics that can be generated. A call to any one of these would replace func(values) in the code samples above. Note that these, and many more, are part of packages such as numpy. These packages inevitably offer more features and more robust implementations than home-cooked ones, but for simple cases (such as here) these examples may be sufficient.

Mean

def mean(values):
    return sum(values) / len(values)

Min / max (already exist)

min(values)
max(values)

Median

def median(values):
    values_s = sorted(values)
    n = len(values)
    if n % 2 == 0:
        value1 = values_s[n / 2]
        value2 = values_s[n / 2 - 1]
        return 0.5 * (value1 + value2)
    else:
        return values_s[n / 2]

RMS

import math

def rms(values):
    sq = map(x*x, values)
    mean = sum(sq) / len(sq)
    result = math.sqrt(mean)

Or, in one line:

import math

def rms(values):
    math.sqrt(sum(map(x*x, values)) / len(values))

As a more involved example, let's assume you want to write the data out to the terminal (to be redirected to a file) with an extra column that contains a weighted sum of the three values for each timestamp.

import sys
import csv

weights = [0.5, 0.3, 0.2]

def weighted_mean(values, weights):
    vw = zip(values, weights)
    weighted_values = [value * weight for value,weight in vw]
    return sum(weighted_values)

header = get_header()  # Returns the original header
data = get_values()    # Returns the data values

outfile = csv.writer(sys.stdout)
outfile.writerow(header + ['weighted'])

for timestamp,values in data:
    row = [timestamp] + list(values) + [weighted_mean(values, weights)]
    outfile.writerow(row)

Next time we'll look at generating some statistics that will help you see how well the system was working.

Convert images to video in Linux

On occasion I need to convert a series of images into a movie. For example, I have a data visualiser which generates an image for each timestamp and it nice to be able to use these to view the evolution of the sensed values over time. If you search on this topic then you'll find a lot of people suggest using ffmpeg with a command line such as the following:

ffmpeg -r 10 -b 1800 -i %03d.jpg test1800.mp4

However, this assumes that all the filenames are sequentially numbered and that there are none missing (it will finish when it reaches the first missing number). I needed something a little more flexible, and eventually found this page which suggests the following:

mencoder "mf://*.jpg" -mf fps=10 -o test.avi -ovc lavc -lavcopts vcodec=msmpeg4v2:vbitrate=800

This will simply read the files in the usual alphabetically sorted order. You can change "mf://*.jpg" to adjust which files it uses, along with tweaking the framerate and bitrate.

Contrary to the note on that page, more recent versions of mencoder support png files as input.

Installing Drupal 7 in Ubuntu

I had occasion to install Drupal 7 on my own Ubuntu server recently. Many of the guides I found went into some quite unecessary steps, involved liberal use of tasksel, or made a variety of assumptions about your environment. In the end, though, I found this guide, which is short, simple, and just worked.

The only addition I required was to run a2enmod rewrite and to edit /etc/apache2/sites-available/drupal to have AllowOverride All in the entry for the /var/www/drupal/ directory.

Job done in almost no time.