Plaintext
NODE.JS SNAPPY UBUNTU CORE QT + MORE!
TAKE BACK YOUR
PRIVACY
Hide from snoopers Encrypt your email Browse privately
Secure instant messaging Protect SMS messages
LITTLE BROTHER EMULATION LINUX
Cory Doctorow Play old games Inside the kernel
On copyright, Rediscover all What goes on in
Creative Commons your old Windows the project that
and why open is games – and play powers your Linux
always better them on Linux! machine
34+ PAGES OF TUTORIALS
GNOME BUILDER Write your own Gnome applications
OPENSUSE Peek into the world of our favourite German export
Augus t 2015 £5.99 Printed in the UK
ASTRONOMY Use raw images to find new worlds and new civilisations…
DESKTOP DISTRO CONVERGENCE!
NETHACK
FEDORA 22 KDE CONNECT
Yum is dead. Long live Link your phone to your
DNF! Plus super Gnome desktop and never miss
upgrades and more fun another call about PPI
WELCOME
The ministry of truth
The August issue
Linux Voice is different.
Linux Voice is special.
Here’s why… GRAHAM MORRISON
A free software advocate
At the end of each financial and writer since the late
1
year we’ll give 50% of our 1990s, Graham is a lapsed
profits to a selection of KDE contributor and author
organisations that support free of the Meeq MIDI step
software, decided by a vote among sequencer.
our readers (that’s you).
T SUBSCRIBE
he New York Times has an excellent online tool that tracks
No later than nine months its use of words, as a percentage of articles that contain
2
after first publication, we will
relicense all of our content under
them, throughout the newspaper’s history (see http://
chronicle.nytlabs.com). Use of the word ‘police’, for example, has
ON PAGE 64
the Creative Commons CC-BY-SA remained more or less static, despite the monumental differences
licence, so that old content can between the civilisations of 1850 and 2015. Search for the word
still be useful, and can live on even ‘privacy’ however, and its use bumps along at around 0.05% for
after the magazine has come off 100 years before ramping up in 1950 to the current high of
the shelves. appearing in 2% of all articles in the newspaper.
I don’t think it’s a coincidence that privacy usage follows that of
We’re a small company, so another word, ‘computer’. Privacy in 1850 could only be physically
3
we don’t have a board of intercepted – opening a letter, or reading a diary, for example,
directors or a bunch of making it a non-issue. In 2015, it’s big business. From loyalty cards
shareholders in the City of London to browser cookies, our privacy is a commodity and we are the
to keep happy. The only people product. I’m not a tin-hat wearing cynic. I’m not complaining: this is
that matter to us are the readers. a system that enables us to do some wonderful things. But I
absolutely must be in control of what I want to share. And the only
possible system for administering that control is open source.
THE LINUX VOICE TEAM
Graham Morrison
Editor Graham Morrison Editor, Linux Voice
graham@linuxvoice.com
Deputy editor Andrew Gregory
What’s hot in LV#017
andrew@linuxvoice.com
Technical editor Ben Everard
ben@linuxvoice.com
Editor at large Mike Saunders
mike@linuxvoice.com
Games editor Michel Loubet-Jambert
michel@linuxvoice.com
Creative director Stacey Black
stacey@linuxvoice.com
Malign puppetmaster Nick Veitch
nick@linuxvoice.com
Editorial contributors:
Mark Crutch, Andrew Conway, Juliet
Kemp, Michel Loubet-Jambert,Vincent ANDREW GREGORY BEN EVERARD MIKE SAUNDERS
Mealing, Travis Mooney, Simon “We delve into how the kernel is “Andrew Conway has written a “I love all things Nintendo – even
Phipps, Les Pounder, Mayank Sharma, created, because it’s so easy to fascinating look at using RAW geeky projects! That’s why our
Valentine Sinitsyn. overlook the gargantuan job of photos to grab infrared data to guide to taking images with the
putting it together” p34 map the stars.” p92 Wiimote is my choice.” p80
www.linuxvoice.com 3
CONTENTS
August LV017
As flies to wanton boys are we to the gods; they play with us for their sport. Oh well!
20 REGULARS
SUBSCRIBE News
06
ON PAGE 64 Kubuntu nonsense,
SourceForge silliness, and
farewell to Mandriva.
08 Distrohopper
The meddle-proof Porteus
Kiosk, the Systemd-free Antix
and the ancient SUSE 5.2.
10 Gaming
When is a shooter not a
shooter? When it mocks the
glorification of war.
12 Speak your brains
Vent your spleen, share your
opinions, let us know what
Many shady organisations want to know your you’re thinking.
details – keep them out, with Linux! 16 LV on tour
Reporting from the OpenStack
Summit in Vancouver on the,
er, ‘New Linux’.
34 Inside the kernel
The beating heart of every
42 Linux installation is controlled
here – the kernel project.
Cory Doctorow 58 Group test
The Raspberry Pi has
changed, and so have the
The information distros that run on it. Find
the best for you!
age’s heir to Orwell 64 Subscribe!
on open source, Save money, get Linux Voice
delivered to your door, and
copyright and get access to every singe one
of our back issues.
Creative Commons.
66 Core technologies
Uncover the firewall
technology at the heart of
every Linux box – iptables.
70 FOSSpicks
The free-est, freshest
software on the internet,
corralled into six pages of
pure excellence.
110 Masterclass
Organise your music
collection in style with Picard
28
PROFIT SHARE! 30
INSIDE OPENSUSE 40
FAQ: QT on the desktop and Beets on
Ages ago we said that There’s life in the old It’s been part of the the command line.
we’d share our profits gecko yet – one of Free Software 114 My Linux desktop
with the community the oldest distros in furniture for years, Charles Butler, Ubuntu’s man
who writes all the Juju charms.
– and here’s how we town is still doing but what is Qt, and
did it… awesome things. why do we need it?
4 www.linuxvoice.com
TUTORIALS REVIEWS
78 80
Cinnamon 2.6
KDE Connect: Never Build a Wiimote-triggered
50
Mint’s solution to the desktop
miss another phone call selfie machine crisis is all grown up and ready
to take on the big boys.
Get desktop notifications on your Put old hardware to frivolous
phone with the magic of KDE use with the Raspberry Pi
84 88
52 Fedora 22
Despite many changes behind
the scenes, this smart desktop
Snappy Ubuntu Core: Run DOS and Windows is still flying the flag for Gnome.
Next-gen packaging games on Linux 53 Gnu Octave 4.0
Sandbox your apps so you know Revisit your (or Graham’s) Stop press: mathematical
they’ll run perfectly every time misspent youth with old classics programming language gets an
excellent new GUI!
92 98
54 OnlyOffice
All the features of Google Docs
and more, without all that
unpleasant evil stuff.
55 Scribus 1.5
New features galore from
the latest in Free Software
design tools.
Imaging in the raw with Batch jobs: Automate
Books Ignore the lessons of
near infrared repetitive tasks
56
Skynet and build a learning
Blow your mind with science, Save time and effort – make the machine + more!
Python and a cheap camera computer take the strain
100
Olde Code: 104
Gnome Builder: 106
Node.js &
COBOL Craft Gnome apps JavaScript
Learn the language Make applications Create a secure
of business the easy way chatroom
www.linuxvoice.com 5
ANALYSIS
NEWSANALYSIS
The Linux Voice view on what’s going on in the world of Free Software.
Opinion
The business value of open source
When is something free worth more than something you pay for? All the time!
Simon Phipps In turn, lock-in to the vendor means you the software to anyone you want. In other
is president of the no longer have negotiating power when it words, it comes with software freedom.
Open Source Initiative
comes to paying for service. You have two These days, open source software is
and a board member
of the Open Rights options when they set a price in successive just as capable and comprehensive as any
Group and of Open years; you can pay what they ask, or you can proprietary solution, and there are plenty
Source for America. stop using their software. There’s no third of vendors who will provide you with the
option where you keep using the software same implementation support, ongoing
and buy service elsewhere, as you need service, warranty and all the other business
a licence to use their proprietary systems necessities you expect from a proprietary
which you’ll not keep if you don’t pay. Even vendor. As long as your supplier actually
W
hy do businesses – and indeed if you buy a ‘perpetual licence’ things aren’t delivers software freedom to you, and hasn’t
governments, educational a lot better, as no alternative vendor will be kept if for themselves while selling you a
institutions and other “enterprise” able to offer service without themselves proprietary “enterprise edition”, you then
software users – need open source? That having permission – in the form of a have the ability to choose whether to hire
sounds like a pretty simple question on the commercial relationship – with your vendor. in-house experts or buy service from the
face of it. But when I ask that question, the vendor, leaving you in control of how much
response is often in the vague “it’s obvious” Proprietary disadvantage you spend, when you spend and who you
category, with people making noises about Curiously, although buying proprietary pay. That’s how you ultimately save.
getting the software for free. That’s not the software involves surrendering business More than that, because true open source
real value of open source software, though. flexibility, ceding architectural control and software thrives in a diverse, multi-vendor
Free as in money is a benefit, but if that’s the losing budget control, none of these things community, there’s scope for a wide range
only advantage you’re peddling when you ever seem to show up as issues in of innovative approaches to using it. As
promote free software, it’s going to be very procurement negotiations (probably a developer, the world of open source
quickly undermined by a quick look at the because these are problems for another software offers rich choices every step of
total cost of ownership – things like training, year, and too many organisations are the way, and because permission to use
maintenance etc. focussed on the short term). If you could them is already granted in advance, it’s OK
When you buy proprietary software for procure software that protected your to prototype and iterate rather than having
your enterprise, you surrender control to business flexibility, left you or your to get special permission from your vendors
the proprietary vendor. They are the only implementation partner with architectural for each new approach you want to try.
ones who can change the software, so control and allowed you to set your own
you either have to give them architectural budget priorities every year, that software Open source: the business case
control of your enterprise software or you ought to be preferable? This is the business value of open source. It
have to commission custom extensions or Turns out you can. Open source software re-empowers the CIO, returning control of
even custom systems. Giving a software comes with permission granted in advance the budget and the enterprise architecture to
company architectural control over your to use the software for any purpose, have them. It re-energises the developer, opening
systems makes your lock-in to them ever full access to the source code, make up alternatives and liberating them to
deeper, while custom software just for your whatever changes you want and both prototype and iterate innovative solutions for
business is very expensive both to create contribute those changes to a community the problems they are solving. It’s about
and to maintain. to share in their maintenance and give regaining control, not avoiding license fees.
Given all the extra business value
delivered thanks to free software and open
“As a developer, the world of open source software source solutions, surely it’s the proprietary
offers rich choices every step of the way.” software that should be cheaper because of
all its disadvantages?
6 www.linuxvoice.com
ANALYSIS
Kubuntu • Raspberry Pi • Chip • Mandriva • OnlyOffice • Firefox
CATCHUP Summarised: the biggest news
stories from the last month
Canonical boots out SourceForge injects ads Raspberry Pi B+ price
1 Kubuntu lead developer 2 into Windows downloads 3 dropped down to $25
For the last two years, SourceForge, once the While everyone is talking
Kubuntu lead developer Jonathan premier host for free software projects, about the Raspberry Pi Model 2, the
Riddell has been asking the Ubuntu has taken a turn for the worse. It older version is still going strong and is
Community Council (UCC) questions declared the Gimp-for-Windows project now even cheaper at just $25. But how
about copyright and donations. Now as “abandoned” and took it over, will it fare against Chip (see below)?
the UCC has decided that Riddell’s modifying the installer to include “easy-
communication was too aggressive to-decline third-party offers”, which
and confrontational, and demanded in the real world means adware. This
that he leave his role for a year. But the doesn’t affect Linux users, but it could
Kubuntu Council defended Riddell, and negatively impact the overall perception
is concerned that the UCC believes it of free and open source software, and
can override derivative distros. This has caused much consternation in the
story is still unfolding as we speak… community.
Qemu floppy driver bug Distro maker Mandriva Nine-dollar Chip gadget
4 opens up security hole 5 goes into liquidation 6 scores big on Kickstarter
PC emulator Qemu is often This has been on the cards for Its specs are not amazing:
used in conjunction with hypervisors a few years, but it’s still sad news. 1GHz CPU, 512MB RAM and 4GB
such as KVM and Xen to provide virtual Mandriva (formerly Mandrake), the of onboard storage. Plus it only has
machines. Now a bug has been found company behind the eponymous composite video output and you’ll need
in the virtual floppy disk controller of desktop distro, has run out of cash and to connect your devices via Wi-Fi or
Qemu, opening up a potentially major closed up shop. All of us at Linux Voice Bluetooth. But at just $9, the Chip could
security vulnerability that already has its remember the shiny Mandrake boxed be the next Raspberry Pi: a tiny, cheap
own moniker: Venom. The bug could sets from the early 2000s, and the computer you can use in all manner of
allow code running in a virtual machine distro’s excellent installer and hardware projects. Its makers were looking for
to escape the confines of the VM and detection (back when those aspects of $50,000 via crowdfunding, but have
affect other services running on the Linux were somewhat lacking). Still, already snagged $1,893,355. Delivery is
same machine. Qemu has responded Mandriva lives in in two forked projects: due at the start of next year.
quickly and released a patch. Mageia and OpenMandriva. http://tinyurl.com/onqqech
Cloud-based OnlyOffice Firefox gets new Adobe
7 suite goes open source 8 DRM decryption module
Formerly known as Teamlab Digital Rights Management
Office, OnlyOffice is a suite of office (or as we prefer to call it, Digital
applications and collaboration tools Restrictions Management) aims to stop
that runs on a server and presents its people copying music and video that
user interface inside web browsers. It they view on the internet. Firefox maker
provides competition to Google Docs, Mozilla is in principle against DRM, but
LibreOffice Online and the editing also needs to maintain market share
component of OwnCloud, and it’s now in a world where Netflix and other
open source. To simplify deployment, media providers are dominant. So new
the OnlyOffice team has made the versions of Firefox can download a
server and related tools available as binary blob from Adobe to decrypt DRM
Docker containers. content – but it’s still possible to get
www.onlyoffice.org Firefox without it, thankfully.
www.linuxvoice.com 7
DISTROHOPPER
DISTROHOPPER
What’s hot and happening in the world of Linux distros (and BSD!).
Porteus Kiosk 3.4
Gentoo for web kiosks.
T
ake the rolling-release Gentoo
distribution, strip it down to a 36MB
ISO image, and what do you get? A
distro that doesn’t do very much out-of-the-
box, that’s for sure. But Porteus Kiosk
(http://porteus-kiosk.org) isn’t meant to be
a general-purpose flavour of Linux. It’s
designed to power internet kiosks – those
fixed-purpose machines used solely for
browsing the web, like you see in libraries,
museums and airports. A kiosk distro needs
to be as simple and minimal as possible, so
that users can’t escape from the browser
and start running SSH, GCC or (heaven
forbid) SuperTuxKart.
When you boot up Porteus Kiosk in live
mode, running directly from RAM, a The Kiosk Wizard lets you lock down the browser to prevent users from messing around.
graphical wizard appears asking you to
configure the network (eg Ethernet or Wi-Fi, “remote kiosk management”, which lets you The Porteus team is trying to monetise
manual or DHCP) and then choose a web store settings in a remote server and the distro by offering automatic support
browser. The distro doesn’t include a administer multiple kiosks simultaneously. updates for a price, after a three month trial
browser as standard, but you can ask it to It’s a very useful feature if you’re rolling out a period. But even without them, it’s a very
download Firefox or Google Chrome. In the bunch of kiosks and want to keep all your polished, easy to use and highly configurable
next step you’re asked if you want to enable settings in sync. kiosk distro – the best of its type we’ve seen.
Antix 15
Lightweight and free of Systemd.
W
e don’t mean to fan the Systemd use, but is targeted at older machines. It’s
flames here, and we know that available as a 670MB ISO download – so it
many people are happily using will burn to a CD-R – and can run on PCs
that init system replacement in their distro. with as little as 64MB of RAM. Of course, on
But equally, we know that many long-time such a low-spec machine you can’t expect Antix runs the svelte IceWM window manager,
Linux fans are looking for a more traditional an amazing experience, but to revive an old and also includes Dillo for web browsing.
Unix-like experience, with Bash-coded init box for the kids to play around with, it’s ideal.
scripts and plain text logs, so we always If you want to do something more building on the work of the Devuan project,
keep an eye out for Systemd-free distros. productive, such as web browsing or image the Debian fork that claims to “preserve init
There aren’t many around now, but a few editing, you really need at least 256MB RAM. freedom”. It remains to be seen whether
pop up from time to time, such as Antix Given that Antix is based on Debian, which Devuan is a serious long-term project or just
(http://antix.mepis.org). has recently switched to Systemd, how can a knee-jerk reaction to Systemd from a
This is a Debian Testing-based distro the distro stick with traditional SysVinit for bunch of frustrated Debian users, but Antix
that’s designed for general purpose desktop future releases? Well, the Antix team is is a sign of their efforts being appreciated.
8 www.linuxvoice.com
DISTROHOPPER
News from the *BSD camps
What’s going on in the world of FreeBSD, NetBSD and OpenBSD.
W
hile the major desktop
environments of KDE, Gnome
and Xfce tend to run fairly
smoothly on most of the *BSD variants, the
experience is often far from perfect. In many
cases the developers simply assume that
the target is a Linux system, and therefore
add snippets of Linux-specific code that the
*BSD teams later have to work around. It’s
not the end of the world, but it means that
integration with hardware and system
settings is often lacking when running these
desktops on a BSD flavour.
To combat this, the makers of PC-BSD (a
desktop-oriented FreeBSD spinoff) have
been working on Lumina, an environment
created from scratch (using C++ and Qt),
which aims to be much more integrated When Fluxbox is replaced Lumina will offer even more cosmetic customisability, including font
with the operating system. It uses Fluxbox scaling, plus compositing support for transparency effects and improved keyboard shortcuts.
for its window manager and a handful of
standard X utilities, but otherwise is a unique Still, Lumina aims to be portable to other major FOSS desktop apps such as Firefox
project. Examples of integration include the BSD flavours – and indeed Linux – and and LibreOffice run well on FreeBSD.
ability to restore ZFS snapshots directly in therefore avoids dependencies on D-Bus, Currently the PC-BSD team is working on
the file manager, and a configuration tool PolicyKit, Systemd and other frameworks. In a replacement window manager for Lumina.
that uses FreeBSD’s sysctl utility to change addition, the desktop isn’t bundled with any If it’s done right, this could really boost
things like screen brightness. specific end-user applications, because the PC-BSD as a great standalone OS.
SUSE 5.2 – A boxed set full of Linux love
Over on page 30 we look at the current state of the OpenSUSE
project, and see what the distro team is working on. We have a soft
spot for SUSE, because, as with Red Hat, it was the first distro for
many new Linux users in the late 1990s. With most of us running
painfully slow dialup modem connections, the boxed sets packed
with CDs and manuals were a joy – absolute treasure chests of free
software and Linux goodness, delivered to your doorstep.
For some top quality retrostalgia, we dug out SUSE Linux 5.2 (from
March 1998) and attempted to install it. But not on a real PC, mind
you – that would probably be impossible given the changes in
hardware over the last 17 years. Instead, we tried it in trusty old PC
emulator Qemu, and for the most part it went well. It’s fascinating to
look back and remember how complicated distro installers were back
then. You had to think about blocks on your hard drive, inodes on
your filesystem, what device node to use for your mouse, and so
forth. A long way from today’s graphical installers where you can get
a complete desktop distro running with fewer than 10 mouse clicks…
SUSE 5.2 installed correctly, but we had major problems setting up
X – which was also a bane at the time. XF86Setup created a usable
VGA16 configuration for us, and tested it, but when we tried to run X
independently (via startx), it bailed out with “no valid modelines
found” (even though we had specified one). Back in the day, using the
wrong modelines could make your monitor explode, but this isn’t a
problem now. We’ll keep battling away with it though!
SUSE had fantastic boxed sets back in the late 90s. Here’s the
German edition of version 4.2 (image: Samsara on Wikipedia).
www.linuxvoice.com 9
GAMING ON LINUX
GAMING ON LINUX
The tastiest brain candy to relax those tired neurons
GOOD OLD GAMES
Spec Ops: The Line
Turns the traditional modern shooter genre on its head, then makes it scream.
T
he problem with having a game based
on Heart of Darkness is that if you’ve read
the book or watched Apocalypse Now
then you pretty much know what the main
premise is going to be. On the other hand, we get
a game that reinvents the traditional modern
shooter genre, delivering instead something that
Michel Loubet-Jambert is our Games
Editor. He hasn’t had a decent night’s shows the grim brutalities of war.
sleep since Steam came out on Linux. The main character is pretty well fleshed out,
and having the third-person perspective helps Captain Martin Walker’s mental condition gradually
I
t makes sense that many Linux to deliver a character-driven plot. He and his two deteriorates as the horror unfolds before him.
gamers tend to stay away from US Army companions are tasked with finding a
DRM, and thus Steam as a
missing squadron and its leader in the Middle screen, then shows you the resulting carnage.
platform. Luckily we have GOG.com
and the countless Linux titles it offers East, after it was presumed that a sandstorm Perhaps the best executed aspect of the
DRM-free, both new and old. It has wiped them out, along with the city of Dubai. game is allowing for a great deal of fun in all the
recently released classic games such That all sounds pretty standard up to that shooting and tactical combat, before pulling the
as Cyberia 2, Star Wars Rebel Assault, point, but rather than being a shooter, Spec Ops: player back like a disapproving parent, reminding
Star Wars Tie Fighter & X-Wing,
The Line is the definitive anti-shooter, ridiculing you what you are in fact enjoying. Spec Ops then
Toonstruck and many more thanks to
FOSS projects such as DOSBox and the glorification of war. Instead of rewarding isn’t for the faint of heart, but for those who want
ScummVM. The store isn’t just limited the player when taking questionable actions, it a great story and change from the standard
to older releases though; you can find goes out of its way to show you what a terrible modern FPS games, it’s certainly worth getting.
more recent games like The Witcher 2 person you are. At one point the game has you
and Pillars of Eternity there completely Website http://store.steampowered.com/
wipe away an entire army of people using white
DRM-free. app/50300 Price £19.99
Even more exciting is that the phosphorus from the comfort of a computer
desktop client, GOG Galaxy, has
reached open Beta and the Linux
version is soon to follow. This client The game deals with a lot of
adds many of the features found on difficult issues, from clandestine
platforms like Steam, such as CIA operations to PTSD.
achievements and in-game chat, but
is completely optional, meaning you
don’t have to use the client to buy
games from GOG.com. This has been
eagerly anticipated by many who
want to manage their games without
all the desktop clutter, and to get
some of those Steam-like features.
Somewhat disappointingly though,
GOG.com has been extremely vague
with the release date of the Linux
version, when we had previously
expected that it would be released
alongside the Windows and OSX
versions. In the meantime though,
non-DRM gamers should pay them a
visit, while Steam gamers should
note that there are a lot of games
there that aren’t on Steam, so it’s
“Spec Ops: The Line is the definitive anti-shooter,
worth visiting. ridiculing the glorification of war.”
10 www.linuxvoice.com
GAMING ON LINUX
Victor Vran ALSO RELEASED…
An atmospheric action-RPG from the developers of Tropico 5.
T
his game is a great example of the Early Access version at the time of
Early Access done right, providing writing are voice acting and cutscenes,
an already entertaining game but the developers were kind enough to
to its players, which the developers can include placeholder images and a few
then use to get feedback and make written lines to give players an idea of
improvements before release, with lots of what the story will be like.
community involvement in the meantime. Not revealing much of the story yet is
Victor Vran puts gameplay above all another good move, allowing Early Access Vertiginous Golf
else; the combat mechanics are solid and players to enjoy themselves while not This weird and wacky mini-golf game is a lot of
incredibly satisfying, while having a wide spoiling the full experience when it comes fun and one of those rare cases where we get a
range of weapons, customisation and around this summer. local multiplayer game outside the console
character progression creates just the space. Vertiginous Golf is set in a dystopian
steampunk world and provides some nice
right balance between action and RPG. Website http://store.steampowered.com/
features like a map editor and a rewind ability
app/345180 Price £15.99
The two main elements missing from to be able to retake shots. It’s great to play at
home with friends or against others online.
http://store.steampowered.com/app/272890
Lux Delux
Levels are well crafted and This highly customisable Risk-like game is a
provide a great backdrop for lot of fun and provides a different experience
all the monster slaying. every time through all its options. The game
boasts over 900 maps and an active
community of players doing modding through
an open-source SDK and built-in map editor.
Windward
Online and even local bot games can be very
challenging, while the cross-platform
multiplayer and league tables exist to ensure
A procedurally generated trading and exploration game with pirates! plenty of competitive and nailbiting
experiences.
T
his fun little trading and http://store.steampowered.com/app/341950
exploration game is as addictive
as they come. You control a single
ship trading goods with different towns,
but as the game goes on there’s the
growing threat of rival factions and even
pirates to fight on the high seas.
There’s a tonne of things to do and
many different ways to play, from going Windward’s aesthetically pleasing maps are
the diplomatic route and slowly building full of adventure.
up the ship to carry more guns and Euro Truck Simulator 2: Scandinavia
cargo, or simply take on as many pirates on. For those who want an even greater Everyone’s favourite simulator game just got
its second major DLC, adding a bunch of
and factions as possible. Both options challenge, the online mode against other Scandinavian towns to visit, and also new
(and everything in between) are fun and players is probably the most fun, though cargo and companies. The new scenery is very
rewarding; the ship-to-ship combat is the game is worth it for the offline content pretty and the developers have done a great
great, while performing quests for towns alone. it is highly recommended to those job of capturing the region’s natural beauty. At
and seeing them grow gives a nice sense who enjoyed games like Taipan! or the same time, a new patch has come out
adding, among other things, an improved day/
of development. Sid Meier’s Pirates. night cycle and weather system, adding better
Windward is magnificently simple realism to the world.
and easy to get to grips with, while Website http://store.steampowered.com/ http://store.steampowered.com/app/304212
app/326410 Price £10.99
providing plenty of challenge as it goes
www.linuxvoice.com 11
MAIL
YOUR LETTERS
Got something to say? An idea for a new magazine feature?
Or a great discovery? Email us: letters@linuxvoice.com
LINUX VOICE STAR LETTER
LINUX ON THE HIGH STREET – A RESULT!
Further to my letter in LV014, it needed is a Linux laptop with a
appears that HP has decided virtual version of XP (or Vista/
to release three Ubuntu Windows 7/Windows 8) running
laptops in the UK via Ebuyer: inside it, courtesy of VirtualBox.
www.theinquirer.net/inquirer/ Linux would take care of
news/2406977/canonical- secure online transactions and a
and-ebuyer-team-up-to-bring- Windows-only program such as
ubuntu-powered-laptops-to- Adobe Photoshop could run offline
the-uk. in Windows XP.
The entry-level HP ProBook Godfrey Green, Cardigan
255 (£199.98) has an AMD A4 Andrew says: Well played Ubuntu! We
processor and 4GB RAM, the can’t fault Canonical for its ambition
ProBook 355 has an AMD A8 and attitude – the home PC
and 4GB, and the top-of-the-line market is there for the taking,
HP ProBook 455 has an A10 and we really hope that a Linux
with 8GB. All have Ubuntu 15.04 vendor or vendors step up to the
installed. Apparently HP has the challenge and target the (as you say)
“XP holdouts” in its sights: XP holdouts. They don’t want Vista, It’s bloody brilliant to see Linux (in
All recent CPUs support they don’t want Windows 7 – let’s the shape of Ubuntu on this HP
virtualisation, so all that is show them how good Linux can be. ProBook 445) as a retail offering.
RSYNC EH?
Just a quick note on the rsync happen if we added a slash to the end,
article from the June 2015 issue. as in home/bodhi/. Thankfully the
At one stage, you introduced technical point about using a trailing
the directory “bodhi” in the text, slash is unchanged; you’re right
in order to explain the trailing though that the change of directly
slash. I know what you meant, names could be confusing. I guess
but it was a bit confusing. Maybe Mayank, who wrote those excellent
a clarification in the next issue words, must have reinstalled Linux
would be good. with the username ‘mayank’ instead
Stevan Akrap of his usualy username ‘bodhi’.
That’s one of the perils of constantly
Andrew says: Now that I’ve gone tinkering…
back and had a look at the section
in question, I see what you mean.
They code example gave an example
directory as home/mayank, and we Bodhi Linux, not to be confused with
then referred in the text to home/ /home/bodhi (or /home/bodhi/ for that
bodhi, and explained what would matter)
12 www.linuxvoice.com
MAIL
FULL CIRCLE
In issue 16 Sarah McKie other resources both free and can access, might be helpful.
asked about beginner’s guide at a cost that might be useful Tony Hughes, Blackpool
to LibreOffice, she could do to Sarah. As Sarah states in her
worse than take a look at letter, LibreOffice is a powerful Andrew says: LibreOffice does indeed
Fullcirclemagazine.org; this is a application which many of us have excellent documentation, which
monthly community magazine underutilise, but with patience and is something we usually highlight
about Ubuntu and its derivatives. time many of its capabilities can whenever we review the most recent
They have been running a regular be uncovered. version. The flipside of this is that
Open/LibreOffice tutorial for many While I understand you’re it’s against the spirit of free software
years and are currently on part running a magazine that you to build on someone else’s work
48. There are also some user sell, you still have a limit to what and effectively keep it secret by not
guides on the LibreOffice website you can cover each month, so documenting it properly. Put like that,
but as I’ve not used these I don’t maybe a feature about community community resources like Full Circle
know how user friendly they are. resources, to help readers find are doing superb work, and deserve all
A look on the web should produce other sources of information they the recognition they get.
Full Circle is doing super
work – download the
latest edition for free
today.
www.linuxvoice.com 13
MAIL
DVDS ON LINUX NEWS FROM OUR AUSTRALIAN COLONIES
Once you know which packages Our (Australians’) glorious leader
to install and the appropriate shell Prime Minister Tony Abbott has
script to run, it’s easy enough to been giving us all the benefit of
watch DVDs on a Linux system. his considered opinions again, this
Not sure if you’ve already time on the subject of whether
published articles about this but kids should learn coding in school.
these things interest me:- It seems that this is a silly idea
Creating DVDs with home and 11-year-olds should not learn
grown content transferable skills.
Being able to test the above The irony of this is that it’s his
DVDs before burning a DVD-R party’s policy to adopt coding in
Recommendations for the Australian school curriculum;
packages to rip DVDs and to he was asked about it by a
“But it doesn’t mean
manage DVDs member of the opposition, and therefore easy to ignore (in fairness,
anything!” Ah Jeremy,
Being able to play back ripped instinctively chose to attack. I the person they had on to promote nature is a language,
DVDs on systems that don’t saw something similar on a UK programming literacy didn’t seem too can’t you read?
have a DVD drive news program, in which the host convincing – https://www.youtube.
Best wishes, (a man in his 60s) mocked the com/watch?v=-7x7GYItzS4).
Ian Bruntlett idea of children being taught Tony Abbott, on the other hand, has
programming languages, a responsibility to know what he’s
Graham says: In the caverns of old presumably because he’d learned talking about or keep his mouth
publishing houses, you can still Latin at grammar school and hit shut. The thing that baffles me is
hear grumpy old publishers say that had never done him any harm. that, if you add a bit of Python to
covering video editing in magazines Luke Milosevic, Melbourne the curriculum and kids don’t like it,
doesn’t sell. But I think you’ve made what’s the worst that can happen?
some good points, and we should Andrew says: The grumpy old man It’s only a kind of algebra – either you
do something, especially as the you remember is Jeremy Paxman, a get it or you don’t, but early exposure
applications for video editing and professional grumpy old man, and is vital.
production on Linux have suddenly
become awesome. We’ve also got
our own YouTube channel, which has
been a little neglected over recent
AU REVOIR, MANDRIVA
months, and this will give us the Raise a glass, oh my brothers, to
perfect excuse to dust off the lights our fallen comrade: Mandriva is no
in the studio. more. After years in the doldrums,
We’ve also needed to be cautious one of the foremost Linux vendors
about covering DVD ripping, as its has gone the way of all flesh,
legality was dubious in the UK, but leaving us with only memories. I
since October 2014, transferring both suspect a lot of your readers will
your music and movie collections share with me early memories of
has become legal, so we can give Mandriva (or Mandrake as was)
it some coverage with a clear as a first desktop Linux. Before
conscience. Thanks for the ideas! Ubuntu came along and forced
everyone to take new users into
consideration, Mandrake was the
only Linux distro to sugar the pill
of installing Linux, making it easier
for converts to Linux.
So farewell then,
Dave Moran, Basingstoke releases, Mandrake did a lot to make
Mandriva. Thanks for
Linux easier. And for a while it seemed all the socialist-realist
Andrew says: The first Linux I tried at though Mandriva was going to penguins.
was a version of Mandrake; I don’t make a success of commercialising
remember the version, because it Linux, with some big deals in the
didn’t work. I moved on to Ubuntu French public sector, it never really
If the thought of Netflix introducing 5.04 and didn’t look back. But for happened. Never mind though; the
advertising to your account has you people who tried Linux a year or two company may have fallen, but the
scared, it’s time to move to Kodi! before the first few brilliant Ubuntu code lives on. Viva Mandriva!
14
MAIL
Email andrew@linuxvoice.com to advertise here
www.linuxvoice.com 15
SHOW REPORT OPENSTACK
The elephant in the room is Amazon. AWS is by far
the leader in the public cloud, with five times the
cloud capacity of the next 14 competitors combined.
SUMMIT
Travis Mooney reports from from
2015
the clouds in Vancouver.
L
inux has grown up a lot since 1992, from a penetration. Making software easier to use decreases
hobbyist OS to powering datacentres, the requirement for a lot of supplementary services
smartphones, quadcopters and basically (consulting, especially). But as the market grows, so
everything else. Any technologist worth their salt do the number of opportunities. This is a balance that
knows that Linux displaced traditional Unix, and a lot of FOSS companies know well — Canonical has
pushed a lot of BSD (and even Windows) installs been quite open about riding the OpenStack wave to
aside. OpenStack has done the same thing among significant success.
cloud computing platforms, taking the market by
storm to become the de facto technology. What is OpenStack, anyway?
The crowds and marketing money going into Like Linux was as a three year-old (in 1995 I had an
OpenStack remind me of the explosion around Linux ancient Slackware install without X11), OpenStack is
in the late 90s. If anything, the OpenStack crowds are new, and can be a bit confusing. It is made up of a
more intense, the sessions more packed, and the number of components which provide: compute
vendor give-aways way better. And that’s because service (through Nova), image service (Glance),
OpenStack is a maturing technology, but the market storage (object by Swift or block by Cinder), identity
isn’t saturated yet, and as a result companies are service (Keystone), networking (Neutron),
making a lot of money off it. orchestration (Heat), telemetry (Ceilometer), database
Maturity is a double-edged sword in the technology (Trove), elastic map reduce (Sahara), bare metal
world. The FOSS market survives largely on provisioning (Ironic), multiple tenant cloud messaging
consulting, services, and paid development work. As (Zaqar), shared file system service (Manila), DNSaaS
software matures, it gets easier to use, gains (Designate), security API (Barbican), and wrap it all in a
additional features, and has greater market dashboard (Horizon). Just to make things more open,
16 www.linuxvoice.com
OPENSTACK SHOW REPORT
and a bit more complicated, there are different drivers OpenStack is making a lot
for various components, such as a choice of of money for companies
underlying filesystem driver for the storage like Canonical, Suse, Red
Hat, HP, and IBM.
components, or network driver for Neutron.
When you put all these programs together, you can
create a cluster of machines that behave very much
like any public cloud service, but completely under
your control. Although you still have to procure and
provision servers, you gain most of the flexibility of
cloud virtualisation, can run multiple tenants on the
same installation, and can have automatic scaling of
systems based on demand. Adding hardware to scale Thaks for having us,
out services also becomes less difficult. You can even Vancouver – see you soon,
Tokyo!
leverage public clouds that use OpenStack to create
hybrid clouds — for example to offload public web
services during peak hours.
Hot topics
And much like the early days of Linux, there are a lot of
companies building on the OpenStack ecosystem.
OpenStack has buy-in from Canonical, HP, and IBM
Softlayer, to name but a few. Smaller vendors at the Shifting from fat servers to a
OpenStack Summit were hawking: add-on billing cloud way of systems
engines (both for public-facing services and management and design was
departmental back-billing), integration of AWS as an also key to the summit.
external public cloud, additional automation and
orchestration tools, storage solutions, and many
other things.
Like a lot of technology conferences, the OpenStack
Summit moves, with a meeting in a new place every
six months. This meeting was in Vancouver, home to
Douglas Coupland, William Gibson, and Jewel Staite,
so you know it has some technology credibility. It’s a The lone LXD session was put on by Canonical, which
city of both modern glass and steel skyscrapers, and very correctly pointed out all the ways that LXD is
traditional brick buildings. Canada’s Hollywood, many lighter than KVM, but that’s not a surprise, as the
sci-fi favourites are shot in the city and surrounding reason that any container technology is hot is that it
countryside, including the The X-Files, BattleStar gives a stable platform a la virtualisation without as
Galactica (2004-onwards), and Supernatural. And, at much overhead. In the end it wasn’t terribly clear why
least during the OpenStack Summit, it’s a beautiful, Docker, Rocket or LXD would be a better choice for
walkable city, filled with people so friendly that you deployment on OpenStack, but it is definitely clear that
might wonder what they’re up to. containerisation has come to primetime.
Besides the design OpenStack is
summit, where real definitely a hot
code was really “OpenStack is one to watch; technology to
coded, the main
thrusts of the
changing to a cloud mentality means watch, and
changing to a cloud
OpenStack Summit that we run services, not systems.” mentality means
were around realising that we
containers, automation and devops, and building on run services, not systems — even if we have to make
the OpenStack ecosystem for fun and profit. sure the underlying systems are healthy. If you want to
Containers are hot on everyone’s plate these days, check out the newest in private cloud technology, take
and the OpenStack Summit featured at least three a look at the current DevStack single-computer
main container technologies: Docker, CoreOS, and OpenStack implementation, or run a full version up on
LXD. The most established of the three, Docker, not a spare cluster of six machines and take it for a test
only had overflowing talks where people competed for drive. It’s one of the ways the future is going!
standing room, but featured heavily in sessions aimed To see videos of the sessions from the OpenStack
at devops practitioners. CoreOS sessions were Summit in Vancouver, see https://www.openstack.
likewise overflowing, and added CoreOS technologies org/summit/vancouver-2015/summit-videos.
to Docker (or Rocket) with Kubertantes (an The next OpenStack Summit will be held in Tokyo,
automation layer for containers developed at Google). 27–30, October, 2015.
www.linuxvoice.com 17
18
19
TAKE BACK YOUR PRIVACY
TAKE BACK YOUR
PRIVACY
Hide from snoopers Encrypt your email Secure instant
messsaging Browse privately Protect text messages
T
he internet is a dream for snoopers. Almost Governments are also keen to investigate the
every action of our daily lives flow through it finest details of our lives. They claim this is for
at some point or another: emailed national security and to prevent crime, but there’s
appointments, streamed TV shows, web purchases, very little evidence that internet surveillance has ever
photos shared with friends. Much of this is prevented terrorism or made an impact on crime.
completely unencrypted and can be read by any of Instead, surveillance is used to harass critics and
the various companies who own the tubes the data entrench government control.
flows through. Even the bits that are encrypted are
usually only encrypted between the end user and the We can fight back!
company running the website. Once the data is Many of the underlying technologies of the
uploaded, it’s often mined, and the data is then sold internet come from a time when only a few
off to the highest bidder. people connected to the network, and no sensitive
Our privacy is so valuable information got shared.
that many companies build
their business models
“If you’re not paying for it, If encryption and security
were considered at all, they
on invading it. Facebook you’re not the customer: were considered a waste of
doesn’t provide free access
to a social network because
you’re the one being sold.” resources. This can make
it seem sometimes as if
it’s a charity; it does it to privacy on the internet is an
learn about your life so that it can sell advertising impossible achievement.
space more effectively. Google doesn’t index the All is not lost. You can’t get back the information
web just to make life easier for you; it does it so it that has already leaked out, but you can stop
can learn about your life and sell advertising space the invasions of privacy from continuing. All the
more effectively. Twitter doesn’t… well, you get evidence we have says that when it’s used properly,
the picture. The modern internet was eloquently modern encryption can’t be broken by anyone.
summed up by Andrew Lewis when users We’ll show you how to use it properly, and help
complained about changes to the Digg network: you understand what sort of security each form of
“If you’re not paying for it, you’re not the customer: encryption provides and which protocols can be
you’re the product being sold’’. trusted to keep your data private.
20 www.linuxvoice.com
TAKE BACK YOUR PRIVACY
Encryption
A digital toolkit for keeping data safe.
O
ur best tool against spying is
encryption. This is a complex
mathematical process of changing
data so that someone spying on us can’t
understand the data. There are three types
of encryption:
Shared key (aka symmetric key and
private key) encryption This is where the
same key is used to encrypt and decrypt
information. This means that if you’re
communicating with someone, both
parties need to know the key. This can
cause a chicken-and-egg problem
because you can’t communicate securely
until you both know the key, but you can’t
share the key until you have secure
communications.
Public Key Encryption Here, different keys The Matasanto crypto challenge is a great way to learn the challenges involved in encryption.
are used to encrypt and decrypt data. The
two keys are usually referred to as the property of hashing is that it’s consistent. When we talk of privacy, there are a
Public Key and the Private Key. The public That means that when you hash the same number of different things that we could
key is known to everyone, while the private value, it will always return the same result. mean. It’s important to understand the
key is known only to one person. If you For example, passwords should be stored different guarantees that each protocol
want to send someone a message, you hashed. When a user enters their attempts to establish so we know exactly
can encrypt it with their public key. password, the computer hashes what how private our communication is.
Alternatively, if someone wants to digitally they enter and checks that hash against Secrecy, where no-one can see the
sign a message, they can encrypt it with the stored value. If an attacker steals the contents of our communication. However,
their private key. Anyone can then decrypt stored password hashes (provided the it is possible that someone eavesdropping
it with their public key, and be sure that it password’s can’t be guessed), they can’t on a secret communication could find
came from the real sender. actually use them. some information out, like who is
Hashing (aka One-way encryption) This These three types of encryption are communicating with whom. They should
is unlike the other forms of encryption combined in various ways to form encrypted not, however, be able to see the data that’s
because once data’s been hashed, there’s protocols that we can use to secure our data being transmitted between two parties.
no way to un-hash it. The one redeeming and communications. Metadata secrecy, where no-one can see
who we’re communicating with. They may
Glossary of spying terms see that a stream of information comes
out of our machine, but can’t track where
Five-eyes An information sharing network made Cookie A piece of data stored by your browser it’s going, or even what form of
up of USA, UK, Canada, Australia and New that can be set by a web server. This can be
communication it is.
Zealand. used for tracking a user’s session (such as
Man-in-the-middle A form of attack where the keeping them logged in to a site), or tracking Non-repudiation/Tamper-proof, a way of
attacker positions themselves between the two their movements through the web. guaranteeing that the person who said
parties. Here they can both sniff and alter data Europe vs Facebook A legal case that’s being something really said it. This is useful
travelling in either direction. brought against Facebook for allegedly because it stops people impersonating
NSA The National Security Agency. The USA’s breaching European data protection law.
other people, and
spy agency tasked with foreign espionage and Fingerprinting A method of identifying a user
securing communications infrastructure. based on the settings in their web browser – see Anonymity. In a truly anonymous system,
GCHQ Government Communications Panopticlick by the EFF (https://panopticlick. no-one can tell who another person is
Headquarters. Britain’s communications spying eff.org/index.php). unless they deliberately reveal themselves.
agency headquartered in Cheltenham. Snooper’s Charter A proposed law in the UK that In some cases this is a good thing,
Metadata Data about data. In an email, the would bring in sweeping new powers to allow
because it allows whistleblowers to report
contents would be considered data, while the the government almost unfettered access to
sender, recipient, subject, date and associated IP internet data in the UK. on issues and even the person they’re
addresses would all be considered metadata. Human Rights Act 1998 A piece of UK blowing the whistle to can’t tell who they
FISA Foreign Intelligence Surveillance Act. A US legislation that the current government wishes are (and therefore can’t betray them). An
federal law that is used to legitimise much of the to repeal. It includes Article 8 (Everyone has the anonymous system could include some
NSA’s spying through a very flexible right to respect for his private and family life, his
form of online identity system, but not a
interpretation of the word Foreign. home and his correspondence).
way to link that identity to a real person.
www.linuxvoice.com 21
TAKE BACK YOUR PRIVACY
Spying programs
Governments are vacuuming up huge troves of data on civilians…
Prism
Prism Prism
XKeyscore
Almost all the communication unencrypted message, so it can XKeyscore isn’t a standalone server details are stored, it’s
gathered by Prism is sent relay it to a third party. In the surveillance program in itself, time to think again. All your
encrypted, but can still be case of Prism, the third party is but a front-end for all the data communications are linked to
gathered by the NSA because it’s the NSA, but Facebook also amassed by the NSA. It’s the all your others (unless you’re
not encrypted for its entire uses this information to tailor system that brings everything using carrier pigeons or smoke
journey. Take, for example, an adverts. The fact that your together and enables an analyst signals).
online chat in Facebook. The messages are stored in to instantly access all the
messages are sent via HTTPS Facebook’s data centre also information stored about Eye in the sky
communication from your means that your messages another person. Everything from The best defences against
browser to Facebook. They’re could be read by any hacker who mundane Facebook chats to the XKeyscore are end-to-end
then sent from Facebook to the manages to get access to this. details of your browsing history encryption to stop a
recipient via encrypted HTTPS, The only method of defeating to phone calls can be accessed communication appearing on
which again can’t be sniffed. this form of spying is end-to- from a single place. one of the back-end databases
This means that at no point is end encryption. This is where So, if you think that text- linked to the program, and true
the message transmitted a message is encrypted by the messaging a server password anonymity can mean that a
unencrypted. However, sender and not decrypted again won’t be linked to your particular communication can’t
Facebook has access to the until it reaches the recipient. online accounts where the be linked back to you.
Prism is a data source that feeds into many of the NSA’s analysis tools. Systems like XKeyscore allow agencies to analyse vast volumes of data.
Prism
Tempora
Much of the data travelling to taps on every major cable 46 of them at a time. So much Europe, it’s still quite likely that
and from the west of Europe coming into and out of Bude. data is collected through it will be picked up by British
goes via the Cornish seaside These cables contain telephone Tempora that GCHQ can’t store spooks. Many major internet
town of Bude. Here, and on (voice and SMS) data as well as it for long. It holds on to the full companies have their European
nearby beaches, cables that internet communications. data for three days, and the headquarters in Ireland, so most
travel to Canada, the east coast metadata for 30 days. At least, communications in or out of
of the USA, the west coast of Cream first or jam? that was the capability of these data centres go through
Africa and beyond slide below Project Tempora is run by GCHQ Tempora in 2012, according to GCHQ-monitored cables as well.
the waves and into the murky (with assistance from the NSA), information provided by NSA Anything that isn’t encrypted will
depth below. If you want to be and collects data directly from whistleblower Edward Snowden. be extracted. Anything that is
able to sniff global internet internet cables such as those If you’re using the internet encrypted will have any available
traffic, you need a presence at tapped at GCHQ Bude. GCHQ in the UK, it’s almost certain metadata extracted. There’s
Bude. It should come as no has tapped at least 200 that your connection will go little oversight of GCHQ, so it’s
surprise that GCHQ runs one of 10-Gigabit cables and can through a GCHQ-monitored impossible to know exactly what
its regional sites here, and has process information from up to cable. If you’re in mainland they’re doing with all this data.
22 www.linuxvoice.com
TAKE BACK YOUR PRIVACY
Prism
Dishfire
The NSA is attempting to collect doesn’t cover data that’s shared slides have shown just how transactions (from text–text
every SMS message in the world by a foreign intelligence agency. much information the system payments or credit cards
using a system known as The same loophole works the is gobbling up. Every day it linked to phones).
Dishfire. According to one GCHQ other way, since the NSA isn’t collects: 1,600,000 pieces of
document, “[Dishfire ] collects allowed unfettered access to US 200,000,000 text messages. information on border
pretty much everything it can, so citizens’ data. Each agency 76,000 geolocated text crossings (from roaming
you can see SMS from a collects data on the other messages. information texts).
selector which is not targeted.” country’s citizens, and they 800,000 financial 5,000,000 missed call alerts.
In this context, a ‘selector’ is a exchange it. Thus each
person, so the document is government follows the letter,
showing the system collecting but not the spirit, of the law.
text messages from people who
the agency have no reason to be They know where you are!
suspicious of. Perhaps the most disturbing
Usually, GCHQ isn’t allowed to aspect of Dishfire is that it
perform this sort of doesn’t just include the content
indiscriminate collection and of the text message, it also
analysis of British citizen’s data attempts to locate the position
(although oversight is minimal). from which they’re sent. This
However, in this case, GCHQ makes it also a database that
bypasses the Regulation of can be used to track people
Investigatory Powers Act (RIPA), (again, this is everyone, not just
since it’s technically the NSA those suspected of wrongdoing).
that collects the data (it then We don’t know how long they
shares it with the UK), and RIPA store the data for, but leaked Think before you text. SMSs provide a ‘goldmine’ to spy agencies.
Prism
Marina Mainway Prism
EU Data Retention Directive
In theory, US agencies aren’t information on someone using On 15 March 2006, the does open up the possibility that
allowed to spy on US citizens metadata alone. European Parliament and these national laws could also
unless they’re suspected of Marina is a database of Council issued a directive be invalidated at the European
some crime. However, there are internet metadata, while stating that all member states level (as yet, no nation’s blanket
many loopholes that the NSA Mainway stores phone must require telecommunication international surveillance has
exploits. Marina bypasses this metadata. Between the two, providers (such as phone been tested in court).
restriction by not storing the the NSA can build up a picture companies and ISPs) to store However, a legal study
content of the communication, of your life, from your friends, users’ data for at least six financed by The Greens and
but keeping the metadata to the places you frequent and months and at most two years. the European Free Alliance
instead. the websites you visit. All this This data should include things concluded that, “The Court
Lawyers may argue about the bypasses spying laws because, like IP addresses, email clearly rejects the blanket data
difference between data and technically, it’s not data. The addresses, phone numbers retention of unsuspicious
metadata, but in reality the NSA difference, though, only matters called, text messages sent, etc. persons as well as an indefinite
can build up a huge amount of to lawyers. On 8 April 2014, the European or even lengthy retention period
Court of Justice declared the of data retained.” This study
Data Retention Directive to isn’t legally binding – it’s the
be invalid, though, many EU opinion of legal experts. It states
member states still require that citizens of a nation could
telecommunications companies challenge the national laws
to collect information about all through the European Court
their customers. Indeed, the UK of Human Rights. Much of the
plans to bring in even more laws legal position on this is based
regarding surveillance. on Article 8 of the European
The invalidating of the Data Convention on Human Rights
Boundless Informant, shown here, is a front end for Marina and Mainway. Retention Directive by the ECJ (ECHR).
www.linuxvoice.com 23
TAKE BACK YOUR PRIVACY
Private web browsing
Don’t let everyone know what you do on the web.
N
ormally, when browsing the web,
nothing is encrypted. All traffic is
sent in the open and can be
intercepted by a huge number of people.
This includes the packets sent from your
computer requesting data from the server,
and the data the server sends back. This
open communication is known as the hyper
text transfer protocol (HTTP).
Even very early in the development
of the web, it was apparent that not all
traffic should be sent in the open. In 1995,
Netscape released SSL (a layer of encryption
that can be used to protect protocols that
are normally unencrypted), and for the
first time, browsers and web servers could
communicate privately. HTTPS (the S
stands for secure) is the protocol for this
data exchange.
When it’s working properly, HTTPS
The Tor Browser allows you to surf the web anonymously, but if you don’t get this screen when it
guarantees two things: no one can read the first starts, then something’s gone wrong and you may not be anonymous.
traffic, and no one can alter the traffic. There
are caveats to both of these, but HTTPS is
a huge improvement in security over HTTP. requests this page from the server, and that they delete logs (or don’t keep them at
Anyone intercepting traffic can see what sends the resulting page back to you. If the all), but there’s no way of confirming this. In
web servers you’re getting data from, but not connection between your computer and the many cases, proxy providers will be bound
the data they send. proxy is encrypted, no-one can see what by national laws to turn over information to
server you’re requesting pages from (except the authorities, or data could be stolen by
Rerouting connections the proxy itself). If the page also uses hackers. In other words, proxies only provide
Web proxies are servers that you route your HTTPS then no-one (except the proxy) can security if the organisation running the proxy
connection through. This means that see or alter the data from the server. behaves well. If they don’t, then proxies can
instead of your browser sending a message However, the proxy is in an extremely provide less security than plain HTTPS.
to a server saying what page you want to privileged position. They can see just about
view, it sends a message to the proxy saying everything you’re doing on the web. Many The onion router
what page you want to view. The proxy then companies that provide proxies promise If you need anonymity online, the most
robust option is to use Tor. This is a network
where you communicate through a chain of
Cookies, trackers, web beacons - Following your browser
three proxies. You first establish a
Advertising companies don’t need to resort to Google or Facebook, and the tracking cookies connection from your machine to one proxy.
monitoring data flowing through wires in order will be sent along with that request. Since a huge Then, through this proxy, establish a link to a
to track users: your web browser will tell them number of pages include content from advertising second, then through this establish a link to a
everything they need to know. Cookies are bits of companies, these companies get a very complete
third, then through the third, connect to the
data that can be set by a remote website and are picture of your browsing habits.
stored on your browser. They’re most commonly Most web browsers enable you to set how your web. In this chain, the first proxy can see
used to set an ID so that a website can tell which browser sends cookie information at three levels: your IP address, and it can see the IP
requests come from a single browser. Every time all cookies; no third-party cookies; and no cookies. address of the second proxy you’re using.
your browser requests a page from a server, it will The ‘All cookies’ option allows any advertisers The second proxy can see the IP addresses
send details of any cookies set by that domain to track you. ‘No third party cookies’ only allows
of the first proxy and the third proxy, and the
along with the request. cookies associated with the domain that the main
When used responsibly, they’re good for web web page you’re viewing is from. This is a good final proxy can see the IP address of the
users. For example, they enable web shops to option if you’re concerned about being tracked by second proxy and it can see the web traffic.
follow the user as they browse the store and add advertisers, but willing to accept less than 100% This means that even if one of the proxies in
items to their shopping cart. The real problem privacy for the convenience of websites being able the chain is spying on you, it can’t work out
with cookies comes when a website loads content to remember some information about you. Picking
who you are and what you’re viewing. Of
from more than one source. For example, if you go the ‘No cookies’ option may cause issues with
to a website with a Google advert or a Facebook- some websites, but will give you more confidence course, if an adversary controls a large
like button, your browser has sent a request to that you’re not being tracked. portion of the nodes in the network, then
they may be able to de-anonymise the traffic.
24 www.linuxvoice.com
TAKE BACK YOUR PRIVACY
The Tor network provides anonymity, but
not security. That means if you’re browsing Certificates
the web over unencrypted HTTP, people will
All encrypted communication requires some form sends a certificate that has been cryptographically
still be able to see what you’re reading (or of shared information to start. This could be a signed by a certificate authority. If the signature on
sending), but they won’t know who is passcode that both parties know or an encryption this certificate matches one of the root certificates
reading or sending it. Therefore, it’s key. In the case of HTTPS, it’s certificates. in your browser, then the page is accepted as valid.
important to use additional encryption These certificates include a public key for the This means that the entire basis for the security
organisation, and some information about how to of HTTPS lies in these root certificates. If some
appropriate to the type of communication
use the certificate (what organisation it’s valid for, malicious party manages to get the private key to
you want to do (eg HTTPS, GPG or ORT -- what dates it’s valid for etc). one, they can break every bit of security in HTTPS.
see next page) in order to get both When you install a web browser, it comes with This also means that if someone can install a
anonymity and privacy. some certificates installed by default. These new root certificate on your computer, they have
There are two ways of using Tor to are root certificates, and the browser trusts the complete control over your web traffic. Many
organisations that issued them completely not just companies install root certificates on employees’
browse the web: install the software on your
to encrypt traffic, but to verify other certificates. browsers to allow the organisation to monitor and
system, or use a live distro that comes with When you visit a HTTPS website, the web server control internet activity.
it preinstalled. To install the software, go
to https://www.torproject.org and select
the Download link. It should automatically this from a CD, USB stick, or as a virtual There are a number of privacy/
detect the operating system that you’re machine. It has everything set up and ready convenience tradeoffs when it comes to
running, but you’ll need to make sure you to run, but you do need to make sure that web browsing, such as which cookies to
use the correct link to get the 32- or 64-bit you download any updated versions as accept (see boxout). It can be hard for non-
version. You should then download the tar. they come out to ensure that you
xz file, which can be installed with your always have protection.
normal decompression utility or from the Whichever option you choose, “There are a number of privacy/
command line with:
tar xvf tor-browser-linux64-4.5.1_en-US.tar.xz
once you’ve started the Tor
Browser, you’ll see that it’s a
convenience tradeoffs when it
You may find that you need to install the modified version of Firefox ESR comes to web browsing.”
appropriate utilities to extract the xz file. You (Extended Support Release).
should find these in your file manager. In If everything has gone correctly, you technical people to understand what the
Debian- and Ubuntu-based systems, you’ll should see a green page that states, issues are, and decide where to draw the
need the xz-utils package. This will extract a “Congratulations! This browser is configured line. The Tor Browser has a slider to enable
folder called tor-browser_en-US (depending to use Tor.” If you see this, you can start you to increase or decrease privacy levels
on your language). In this folder you’ll find browsing the web anonymously. However, (and consequently decrease or increase the
a file called start-tor-browser. This is an it is worth reading the page linked as Tips functionality of the browser). If you go to the
executable script. Depending on your file On Staying Anonymous (https://www. onion drop-down menu in the top-left corner,
manager’s setup, you may be able to click torproject.org/download/download. and select Privacy and Security Settings,
on it to run it, or you may have to use the html.en#warning) to make sure you fully you’ll get a pop-up box that lets you adjust
command line. On our machine, we can run understand what Tor does and doesn’t do. the features you want.
it with:
~/Downloads/tor-browser_en-US/start-tor-browser
If you don’t want to install the software on
your machine, or don’t trust the operating
system not to spy on you, then running a
live distro is the best option. There are a
few options, but by far the most trusted is
Tails (https://tails.boum.org). You can run
Security settings can be confusing for non- As well as letting you browse the web anonymously, Tor lets you host web pages anonymously
technical people, so the Tor Browser makes using .onion domains. Here’s Facebook served from https://www.facebookcorewwwi.onion.
things simple with a slider.
www.linuxvoice.com 25
TAKE BACK YOUR PRIVACY
Communications
How to keep your online chats private.
T
he internet is about far more than
just browsing the web, and the most
important area for privacy on the net
is online communications. There’s a good
reason that you put letters in envelopes in
real life – you don’t want everyone reading
your mail. In the digital world you should
ensure the same level of privacy by using
strong encryption.
Email is still one of the most common
forms of digital communication; however,
it has no security built in. None. By
default, there’s no attempt to encrypt the
communication, and no attempt to even
verify that the person sending the message
is really who they say they are. Over time,
some solutions to these problems have For more details on GPG encryption, see our masterclass in Linux Voice issue 12.
emerged, but they’re not universally applied.
When sending or receiving an email, you Many email platforms offer encryption email, while signing means that anyone can
should assume that there’s no security at all. to the server. On web-based email this is a read it but it guarantees that the mail came
When using webmail, bear in mind that HTTPS web page; on a regular server, this from the person that signed it.
many webmail providers make their money will be something like STARTTLS. This is an GPG uses public-key encryption for
through advertising and may be mining essential bit of encryption, because without verification of identification and key-
your mail for information about you that it, the email is readable by anyone. However, exchange, symmetric encryption for privacy
can be used to better sell advertisements alone, it doesn’t offer any guarantees of and hashing for signing. In order to use GPG
to you. Therefore, the first thing you need privacy because the mail server could you have to create your own public key, and
to do if you want private email is to use be reading the email, and it could send it get the public key of anyone you wish to
a mail provider that’s not spying on your unencrypted to the recipient’s mail server. communicate privately with. You can either
mail. This means not using an advertising- End-to-end encryption is needed to ensure do this by exchanging key files in person, or
driven mail provider. Riseup.net is a good privacy. This means that you need to by using a key server.
option. Another is to host your own email encrypt it yourself before you send it, and
server, though this can be a little involved. this needs to be done in such a way that Thank GNU for privacy
You should be wary of any email provider only the person receiving it can decrypt it. The method of setting up GPG varies
that makes exaggerated claims about the The standard method for this is Gnu Privacy significantly depending on what mail client
total privacy of their system since, this Guard (GPG). This can be used in two ways: and mail server you’re using. Unfortunately,
isn’t possible using the current email setup encryption and signing. Encryption means there isn’t yet a simple solution that works
unless you use end-to-end encryption. that only the intended recipient can read the across the board. You should look up the
advice for your setup on the mail client’s
Next generation private communications website. When properly set up, GPG protects
the contents of the message, but doesn’t
All the methods of communication we’ve looked ricochet) uses Tor, and each peer has its own hide who is communicating with whom.
at in the main text are client–server. That means hidden service as its interface with the network. This, and other metadata stored in the email
that your messages are first sent to some central This provides a strong degree of anonymity header, may still be sent in plain text.
server, and then on to the intended recipient. They (though not perfect as law enforcement
While there’s no easy way of hiding the
can be secured through end-to-end encryption, but agencies have been able to de-anonymise
it’s hard (or even impossible) to protect metadata, hidden services in the past). metadata in an email (or a good alternative
and potentially, an encrypted service could be Tox (tox.im) focuses less on anonymity, and that can be guaranteed to be secure), there
forced offline by an overzealous government more on having a robust network that’s hard to are some options to mitigate the problem.
that wants to limit the options for secure shut down, and on secure encryption. You can completely hide your location
communications (as happened with the Lavabit While both these projects are potentially very
by accessing webmail through Tor. This
email service). valuable assets in the fight for privacy, at present
The alternative is a peer-to-peer setup, similar we can’t recommend either of them for secure means that it’s impossible to link the email
to how BitTorrent works. A service like this would communications because they are simply too to the physical location sending it. If you
be impossible to shut down. At present, there isn’t immature. They’re under rapid development, and do this, and use different email addresses
a widely-used peer-to-peer chat system, but there that could lead to bugs. However, in the future, for different things, you can achieve a
are a couple in development. when they settle down, they may provide good
reasonable level of anonymity even though
Ricochet (https://github.com/ricochet-im/ alternatives to the traditional client–server tools.
the metadata is still public.
26 www.linuxvoice.com
TAKE BACK YOUR PRIVACY
While email is still hugely popular, instant try to initiate an OTR session with someone
messaging (IM) can be more convenient. who doesn’t have it installed, they’ll get a
Like email, there’s often little security built message telling them how to install it).
into IM solutions by default, and many IM The first time you chat with someone, you
platforms are run by advertising companies need to make sure they are who they say
that mine the chat sessions for data. Many they are. OTR offers three different methods
proprietary IM platforms make claims about of authentication:
privacy and security, but are very hazy on Shared secret Using this method, both
the details. users see a text box and have to type in
some text. If they both enter the same
Getting chatty text, they’re authenticated with each other.
For privacy, you need end-to-end encryption, Question and answer One user poses a
not just encrypted communications to the question to the other, and enters what
control server. Off The Record (OTR) is a they think the answer should be. If the
layer of end-to-end encryption that runs on other person enters the same answer,
top of an IM session to provide privacy. It then they are authenticated.
can run on top of any instant messaging Fingerprint Each user has a hexadecimal
platform, but the developers of the Tails string linked to their username that’s
distro recommend that it’s only used with known as a fingerprint. They can share
IRC and Jabber (or other XMPP platform). this string with other people either when
OTR is a plugin for Pidgin, and you can they meet them in real life, or by some
other means of secure
Key servers are an important part of using GPG
“Like email, there’s often little communication. OTR displays
both users’ fingerprints, and if
effectively. You can add your keys to pgp.mit.
edu (one of the most popular) via their website.
security built into instant they match what the users are
messaging solutions by default.” expecting, they can authenticate
each other. a spy, but are garbage to anyone involved
The first two can be used in the chat). This means that, while it’s not
download the source code or Windows to authenticate someone you know, and truly anonymous, there is some deniability,
binaries from https://otr.cypherpunks.ca. don’t rely on you being able to exchange since no-one except the intended recipient
It’s in most distros’ repositories, but make cryptographic keys in any way. They just can prove a particular message was a real
sure that you have the latest version (check need you to be able to come up with message sent by you and not a fake.
the OTR website for up-to-date details). something that you’ll both know. The final
Another option is to use OTR and Pidgin method can be used if you’ve already On the move
through the Tails live distro. This is a good exchanged digital fingerprints. It’s not just messages sent via the internet
option if you plan to use OTR through Tor, OTR isn’t anonymous, and people can that are routinely intercepted: phone
since using Tails will ensure that everything still see who you’ve communicated with. communications are too. Both voice and
is set up correctly. There are details of the However, the messages are designed in SMS messages are sent unsecured and are
Tails OTR setup at https://tails.boum.org/ such a way that even though a spy can see intercepted by phone companies. Our
doc/anonymous_internet/pidgin/index. that a message has been sent, they can’t recommendation for private communication
en.html. verify that it was signed by a particular on the go are the tools by Open Whisper
Both parties in the communication need public key, and there’s a tool to generate fake Systems (https://whispersystems.org).
to have OTR installed for it to work (if you messages (ie messages that appear real to These include Text Secure (an encrypted
mobile instant messaging platform) and
RedPhone (an encrypted voice caller). Both
of these are available through the Google
Play store and iTunes.
An added advantage is that both of
these apps are free to install and use and
neither comes with advertising. Instead, the
software is funded by grants from privacy
advocates such as the Freedom of the
Press Foundation and the Shuttleworth
Foundation (as in Mark Shuttleworth, the
Self-Appointed Benevolent Dictator For Life
of the Ubuntu Foundation). It’s not just us
recommending these tools. They come with
an endorsement from Edward Snowden
In September 2013, Mailpile crowdfunded over $160,000 to develop a webmail client that makes himself who said, “Use anything by Open
GPG encryption simple. It’s still in beta, but we expect a 1.0 release later this year. Whisper Systems.”
www.linuxvoice.com 27
FEATURE GIVING MONEY BACK
LINUX VOICE PROFIT
DONATION SCHEME 2015:
THE WINNERS!
Thanks to you, Linux Voice readers, we
are able to help FOSS-related projects and
organisations. Here’s how it happened.
W
hen we decided to create Linux Voice back Second, we decided to give 50% of our year-end
in late 2013, one thing was very clear right profits back to FOSS projects, communities and
from the start: we should give something organisations, and let our readers choose exactly
back. We all get so much from the Free Software where the money goes. We asked our website visitors
world – not just great software, but an awesome to come up with a shortlist of candidates, and then
community willing to help and spread the word. in issue 13 we started the voting procedure. You
So we decided to do two things. First, as you’ll have might be wondering: why did it take so long? Well, we
seen if you’re a frequent wanted to make sure all readers
visitor to our website,
we’ve been making back
“We decided to give 50% of had the chance to vote, and it
takes a while for the magazine
issues available under our year-end profits back to to appear around the world.)
a Creative Commons
licence nine months
Free Software projects.” As we’ve spent the last 12
months getting the magazine
after they go off sale. established, sorting out printing
This means that anyone can read, share and modify and distribution, we don’t have a giant pot of money
our old content – and even sell it on – providing they to give away. But we have £3,000 to contribute,
give Linux Voice credit for creating it in the first place. and to make sure multiple projects benefited, we
We’ve already seen articles translated and updated split the winners up into two categories. The first is
to match the latest developments in FOSS, so this is software projects, while the second is distros and
working really well. organisations. So without further ado…
28 www.linuxvoice.com
GIVING MONEY BACK FEATURE
SOFTWARE
D
esktop publishing application
Scribus won by a considerable
margin, which isn’t surprising as
we’d like to make the whole magazine using
it some day. It’s an excellent program and is
being used by many professionals around
the world; see our report in issue 15 for the
features we need before we can make the
switch. Craig Bradney, one of the Scribus’s
lead developers, told us:
“I’d like to thank all the readers of Linux
Voice magazine, as well as all of the users
of Scribus out there. These kind of funds
go well towards allowing the team to meet series will have been released. We hope the that we use in the production of Linux Voice,
up and collaborate on features and bug readers enjoy the new development version so hopefully our cash injection can help
resolution. The best example is the Libre and all its new features. Thanks again!” those projects to add new features and
Graphics Meeting which has just happened Gimp and Inkscape are also valuable tools close bugs as well.
in Toronto, which was the 10th anniversary
of LGM, where Scribus has been represented 1st Place Scribus receives £1,000
every year and we’ve always had a good
meeting there. By the time this news hits,
£ 2nd Place Gimp receives £300
out first development version of the 1.5.x 3rd Place Inkscape receives £200
PROJECTS AND DISTROS
T
he Open Rights Group (ORG) and
Electronic Frontier Foundation
(EFF) have similar goals: they are
organisations working to preserve digital
rights and freedoms, in an age where
governments are using mass surveillance
programmes to spy on us all. We regard
the internet as one of the most important
developments in human history, a fantastic
way to share knowledge and ideas, and it’s
vital that it doesn’t become locked down or
massively restricted.
The Open Rights Group is based in the UK,
and Jim Killock, its executive director, told
us: “We are really grateful to all of the readers
who voted for ORG. We’re going to use the
money to print our report into surveillance
by GCHQ [Government Communications surveillance operations. Hopefully the Open long-time Linux Voice subscriber or this
Headquarters] and send copies to every Rights Group can show MPs that we can is the first copy you’ve grabbed from a
new member of parliament so that they can fight for our security and protect ourself newsagent. Your support for us has made
understand how privacy, free speech and the from terrorism without having to throw away this possible, and we’re really happy to give
security of the Internet is being threatened all our civil liberties. something back. Tell all your friends to
by our own government agencies. This will So, thanks to everyone who voted, and subscribe and next year we can contribute
give us an early start in challenging any new above all, thanks to you, whether you’re a to even more projects!
intrusive legislation proposed by the next
government. Thank you!” 1st Place Open Rights Group receives £1,000
This is especially good now that the UK
has a new government, and has already
£ 2nd Place Electronic Frontier Foundation receives £300
expressed a desire to expand its mass 3rd Place Free Software Foundation receives £200
www.linuxvoice.com 29
FEATURE SOFTWARE UND SYSTEMENTWICKLUNG
THE OLD MEETS THE NEW
Two decades of development have brought SUSE more than
just Yast and a slick KDE desktop. We visit the distro’s team in
Nuremberg to find out more.
B
ack in the late 90s, as Linux was starting to Enterprise (SLE) targets big business and provides
gain mindshare, distro boxed sets were all the competition for Red Hat Enterprise Linux (RHEL).
rage. Most internet users still had sluggish At the time of writing, SUSE was hiring 73 new
dial-up modem connections, so it felt like Christmas staff, and business appears to be booming. But we
when a chunky package crammed with Linux CDs also don’t hear much from the distro team. Everyone
and manuals arrived at the doorstep. And SUSE Linux knows that Canonical is doing flashy stuff with
(www.suse.com) was Ubuntu on phones, while
the best in this regard:
its documentation
“SUSE has always had a feeling Red Hat demonstrates
the most bleeding-edge
was excellent, the of refinement – precision German Linux technology in
boxes included
thousands of
engineering, you might say.” Fedora. So what’s SUSE
doing? Is it really a
packages spread conservative company
across multiple CDs or DVDs, and the distro had an that doesn’t have much to shout about? Or is there
overall feeling of quality and refinement to it – German more happening behind the scenes?
precision engineering, you might say. We paid a visit to SUSE’s offices in the Franconian
Since then, a lot has happened with SUSE. The capital of Nuremberg, home to around 200 staff
company was bought up by Novell, then Attachmate, including developers for both the OpenSUSE and
and is now an independent business unit under SUSE Linux Enterprise distros. Before we went, we
Micro Focus International. Home desktop users and asked Linux Voice readers on our website what
hobbyists are most likely familiar with OpenSUSE, questions we should put to the team, so read on to
a community supported distro, while SUSE Linux find out what’s next for SUSE…
30 www.linuxvoice.com
SOFTWARE UND SYSTEMENTWICKLUNG FEATURE
OpenSUSE vs SLE vs Tumbleweed
How do the different distros work together?
O
ne thing many of our readers wanted to know integrated into the Open Build Service (OBS), a system
is: what is the relationship between used for building SUSE (and other distro) releases.
OpenSUSE, SUSE Linux Enterprise, and the “The entire SLE codebase is now in OBS for
Tumbleweed rolling-release distro? To find out, we OpenSUSE to build upon. The relationship and role
talked to Douglas DeMaio and Richard Brown. of the different distributions inside SUSE is changing
Douglas is the only SUSE employee who works on – and where it’s going to end up, we’re not sure yet,
OpenSUSE full-time, and explained his role: as that’s the fun part of this stuff! The community
“I’m the OpenSUSE employee, the one and only, is shaping it as we speak, and it’s going to happen
so I basically do PR, marketing and a variety of other organically. There’s only one full-time OpenSUSE
things from a business outlook, so we can coordinate employee, and there are a few other SUSE employees
efforts and do things properly – save money where that have very OpenSUSE-centric roles. OpenSUSE is
it is appropriate. I look at the overall goal and how we very much its own thing, it’s very independent – much
want to project OpenSUSE going forward.” more so than comparable projects. The community
Richard Brown is more directly involved with the can really decide where it wants to go. Like, with KDE
technology in the distro: “I have two jobs. Four days being the default desktop, while SLE doesn’t have KDE
a week I’m a QA engineer working on SLE, and eight at all – it’s Gnome-only in SLE 12.”
hours a week I’m chairman of the OpenSUSE board.
The board leads the project, and I look after the board. Pick-n-mix
The rest of the board is elected by the community, Richard describes how parts of OpenSUSE are
and it has five elected seats. The charter forbids any making their way into SLE more often now, and in a
company from having a majority – so there can only more diverse way. And indeed, some parts of the
be two other SUSE employees on the board. The rest Tumbleweed distro are being used in the enterprise
have to be external community members.” product – which seems odd, when enterprise distros
Richard explained that this is different to other are usually very conservative and rolling releases are
projects which have more hierarchical decision typically bleeding edge. For instance, OpenSUSE didn’t
making models, such as Fedora. But what happens include support for the Btrfs filesystem by default, but
with major changes like the switch to Systemd? Was the SUSE team had tested it in Tumbleweed and
there a big conflict and a vote, like with Debian? found it ready for widespread use, so they included it
“No! We had some people who were willing to in SUSE Linux Enterprise 12.
make the switch, and nobody who was not willing to But now that the SLE sources are available to
do it. So we just did it. There was a little bit of a user everyone, does SUSE expect the community to create Bottom left Around 200
backlash, but that was easy to explain – we could rebuilds, like CentOS and Scientific Linux did with the staff work for SUSE in
show that Systemd actually worked. So the model RHEL sources? Nuremberg, primarily on
works really well for us, but our board is quite different Richard: “We want to do it very differently from that. the enterprise products but
partially on OpenSUSE too.
to that of Fedora with all its structure and special We realise that might happen, but we think we can
(Photos: Alyssa Mello)
interest groups and governance.” do something way far exciting than that. Basically,
Bottom right SUSE’s office
We asked Douglas and Richard to clarify the we want to redefine the OpenSUSE regular release includes a mini museum
position between the various SUSE distros. Recently, as effectively a hybrid, with the SLE base underlying with boxed sets dating
the company made available the source code for it, and community stuff on top. So you have a very back to the very earliest
its SUSE Linux Enterprise distro, and that has been stable SLE-like or CentOS-like base system, up to and releases.
www.linuxvoice.com 31
FEATURE SOFTWARE UND SYSTEMENTWICKLUNG
that. You see it in the kernel, and the model works.
We’ve managed to build a rolling distro that works to
that level – that people use every day. So for those
users who want something that’s stable, usable, but
also always getting the latest stuff, which we think
covers the enthusiast crowd, Tumbleweed is it.”
But what about breakage? Despite the best
efforts of the Arch team, for instance, major updates
occasionally break things and users are expected
to keep an eye on the distro’s news site and wiki.
Moreover, Arch users tend to be more technically
inclined and can fix issues manually. So how does the
SUSE team prevent breakages in Tumbleweed from
ruining the experience for general hobbyist users?
Advantage: SUSE
“We have secret sauce that Arch doesn’t have! We’re a
very tool-centric distribution, and we’ve always
SUSE people showed us
the latest updates to the including minimal X [graphical layer], but anything the thought in those terms. From pretty much day one
Open Build Service and community wants to maintain at a faster pace, we when OpenSUSE started 10 years ago, we started the
OpenQA projects. can put on top. From the community perspective we Open Build Service, because we needed some system
think that’s more exciting than something like CentOS, to build our distribution in a very open way. We made
which is simply an exact copy of RHEL, and also from it very cross-distro. Now we have OpenQA, our
the SUSE side; SUSE the company is more interested automated testing tool. In the case of Tumbleweed it’s
in the outcome of that because anything that’s built in totally paired in with the development process and
that way is a candidate to be included in the next SLE also the Open Build Service. So when we’re building
service pack.” something new for Tumbleweed, before it gets
Mixing components in an enterprise-level distro anywhere near Tumbleweed, it gets tested.”
is difficult though. On the one hand, users expect an Richard went on to describe the case with
extremely solid base system that doesn’t randomly GCC 5. It’s a big update to one of the most critical
change underneath you and break all of your components of the system, and it’s currently sitting
applications. On the other hand, software moves at in a staging area. With the Open Build Service and
such a rapid pace that you need regular updates to OpenQA, the SUSE team can constantly check how
some things, and not stick with the same versions much of Tumbleweed can be built correctly with GCC
for the whole 10-year support period. In SLE 12, 5, and what packages are still broken. The same goes
SUSE introduced “modules”, which are parts of the for kernel updates and other major changes – they
distro that move can’t be released to Tumbleweed users until they’ve
“In SLE 12, SUSE introduced at a faster pace passed OpenQA’s automated tests.
than the normal
modules, which are parts of the packages. They’re
distro that move at a faster pace.” distributed and
supported by SUSE,
but not to the same
level – in other words, you can’t phone up and get a
bug fixed within a guaranteed timeframe.
Still, even with this approach to mixing elements of
SLE and OpenSUSE, some people will simply want a
carbon copy of SLE without the commercial support
requirements. The SUSE team doesn’t expect to
build such a distro themselves, but can envisage the
community doing it.
Tumbleweed: the future?
And what about Tumbleweed? Will it become the
standard distro that SUSE expects hobbyists and
home desktop users to run?
Richard: “In my opinion, rolling releases are the
future of Linux development. It fits in with all the
trends we’re seeing everywhere else with devops and Richard Brown is chairman of the OpenSUSE board, and
continuous integration. It’s the Linux equivalent of works on QA for the distro.
32 www.linuxvoice.com
SOFTWARE UND SYSTEMENTWICKLUNG FEATURE
Beyond the distro
Open Build Service and OpenQA: tools for all distro makers.
C
onstructing a distro from the ground up is hard
work, as you’ll know if you’ve ever tried Linux
From Scratch. The more processes you can
automate, the better. For building OpenSUSE and SLE,
the SUSE team uses the Open Build Service (OBS,
http://openbuildservice.org), an open source project
that was originally created just for SUSE projects but
is used by many more around the world. OBS can
automatically generate distro ISOs from packages
and scripts, and upload them for testing.
Also, with the OBS you can create packages for
many different distros. Right now, third-party software
distribution on Linux is rather messy: if you’ve written
an awesome piece of software, how do you get it
to Linux users? You could wait until some distro
developers package it up, but that could take many
weeks or months before your program is in Debian,
Arch, Fedora and so forth. Or you could try to hack
The Open Build Service
together a statically linked binary, wrap it up in a this process as much as possible, you can save a reference server hosts
tarball and throw it onto your website – but that’s not huge amount of time. SUSE’s OpenQA suite (https:// a whopping 37,972
the most elegant solution. openqa.opensuse.org) lets you boot up distros in projects and over 290,000
With the Open Build Service, you can upload your virtual machines, send them virtual keypresses and packages.
code and various metadata (eg for dependencies), mouse clicks, and see how they respond. In this way,
and then build binary packages for specific distros. you can rebuild your distro every night, for instance,
This means that end users can download packages then boot it up in a virtual machine and test various
that have been compiled for their own distros, without functions of it.
having to do a lot of manual work extracting tarballs.
And this isn’t just a service for part-time coders Automated testing with OpenQA
packaging up their latest GitHub work – it’s used by But how does this work? OpenQA does some nifty
major projects as well. tricks like screen-scraping – that is, pulling in images
Go to the OwnCloud website (www.owncloud.org), from the virtual screen, using optical character
for instance, and then the Downloads section. If you recognition to capture some text, and comparing it
choose Linux packages, you’ll be redirected to the against an expected result. The SUSE team showed
OwnCloud section of the Open Build Service website. us this in action, with an OpenQA test suite that
Many developers build their packages on SUSE’s booted up a desktop Linux distro, attempted to open
Open Build Service machines (the reference servers), a terminal, and echo some text to the screen. OpenQA
but as OBS is open source, it’s possible for companies can find this text (even if it’s in a slightly different
to set up their own servers and perform package place due to desktop theming changes), or if it doesn’t
builds internally. appear, register a problem for a developer to fix.
Along with packaging, another vital part of distro So by creating an OpenQA test suite you can
development is testing. Again, if you can automate automatically test your distro in many ways: check
that it installs OK, try more complicated partitioning,
run all major apps after installation, and so forth.
While OpenQA started as a SUSE project, the team
showed us how other distros are now using it, and
it’s even possible to perform checks on non-Linux
platforms such as Windows.
The notion of “eating your own dogfood” (that
is, actually using the software you develop in your
daily work) is integral to the open source world,
so it’s good to see the SUSE team using OBS and
OpenQA extensively. And kudos to the company
for supporting the spirit of open source by making
Yes, OpenSUSE even has its own beer – get a taste of it these tools available under the GPL and helping other
by heading to the distro’s next conference. distributions utilise them.
www.linuxvoice.com 33
FEATURE THE
INSIDE
LINUX
THEKERNEL
LINUX KERNEL
Since he couldn’t find the blueprints for Atlantis, Mayank Sharma
looked at the next best thing – the Linux kernel.
I
t was the turn of the century but Allen Pais thousands of eyeballs and where thousands of lines
couldn’t get to the graphical desktop on his of code are modified every day. This mechanism
computer. That was because Red Hat Linux 6.0 enables contributors like Arun Raghavan, who has had
didn’t support his SiS 6215c graphics card. Instead of just one patch accepted to the kernel, to improve the
moaning, he decided to get his hands dirty, read the kernel with their “well-meaning ignorance”. Raghavan,
Linux kernel source and the kernel mailing list archives who is a developer on the GStreamer multimedia
and hacked together a driver. Although his driver didn’t framework and maintains the PulseAudio audio
work, the experience gave him wonderful exposure to server, has sent in patches related to issues with his
the kernel as well as a career – Pais is now the Macbook Pro that weren’t accepted but managed to
principal kernel engineer at Oracle. kickstart a discussion towards the proper fix.
Pais is just one of the thousands of contributors to If you’re not a developer and have never contributed
what’s been dubbed the world’s largest collaborative to an open source project, think of a patch as a
development project. According to estimates it record of changes to an existing piece of code. When
would cost billions of developers need to fix
pounds and thousand of
man years to redevelop
“It would cost billions of pounds a bug or add a new
feature to the kernel,
the kernel, which now and thousands of man years to they write up a patch
contains over 19,000,000
lines of code written
redevelop the kernel.” containing the list of
changes they want
in about a dozen to make to the kernel
programming languages. Most of the work is paid for either by replacing lines of code or adding new ones
by over a thousand multinational corporations that or both.
have developers like Pais to work on the kernel. The patches go through an elaborate vetting
The secret sauce behind the world’s largest open process before making their way into the kernel.
source project is an effective system of collaboration. Once a patch is submitted, other developers review
Unlike other open source communities, the kernel it for quality and whether, in fact, the change it
community has had to evolve its own distinct implements is something they want in the kernel.
mechanism of operating in an environment with If the change is minor and implemented nicely, it’s
34 www.linuxvoice.com
INSIDE THE LINUX KERNEL FEATURE
Top left Arun Raghavan
accepted without much delay, while others can linger maintainer in charge of approving patches for their
often reminisces about
for years. Raghavan has had experience with both: particular subsystem. These maintainers each his time as a Gentoo
“The first security patch I posted was a minor cleanup manage their own version of the kernel source tree. developer.
that I thought made the code more consistent. Once a patch is submitted for review, the developer Bottom left In addition
The maintainer felt that this introduced a bit more will receive all sorts of feedback on the submission to the kernel patches
overhead. That discussion just fizzled out. The second and should be prepared to make changes to their they receive, Josh Boyer
was incorrect, so wasn’t merged. And the third was code accordingly. For many developers, the review values the help from the
the minor API change that did get pulled [included], process is one of the most intimidating parts of the community in triaging
and was really quite painless.” kernel development process. “It took a lot of courage bugs which helps Fedora
as a whole.
to send the first patch,” says Pais. “I had to ensure I
Above right Jonathan
Tending the garden read every piece of documentation about how it has
Corbet’s guide to the kernel
A typical release gets over 10,000 patches, so it’s just to be sent.” development process is
impossible for all of them to be inspected by a single Despite his apprehensions Pais has had a a wonderful read for any
individual. Instead the kernel is segregated into several wonderful experience working with the developers developer who wants to
logical subsystems or trees such as networking, video who have been very helpful even when he’s made get started on the Linux
drivers, etc. Each subsystem tree has a specific mistakes, like the time his patch broke the kernel. “It kernel.
Image Credit: Linux Foundation,
Flickr
Linus’s lieutenants
The 2015 edition of the Linux Kernel Development Report by off by Linus Torvalds (329, or 0.4% of the total) continues to
the Linux Foundation gives a wonderful overview of the kernel decline. “That reflects the increasing amount of delegation to
development process. The report tracks the development subsystem maintainers who do the bulk of the patch review
of the Linux kernel during a specific period and analyses and merging.” Greg Kroah-Hartman tops the table by signing
various factors such as the developers doing the work and the off 13,028 patches (14.4%) that he didn’t author himself,
iStock photo © Eyematrix
companies sponsoring them. followed by David Miller (8.6%), Mark Brown (4.1%) and
According to the report, since the 2.6.11 release in 2005 Andrew Morton (4.1%).
more than 11,000 developers have contributed to the kernel. Ranking sponsored contributions by companies puts Intel
However, despite the large number of individual developers, (10.5%) at the top followed by Red Hat (8.4%) and Linaro
most of the work is done by a handful of developers. The (5.6%), while 12.4% of the contributions come from developers
report also notes that the total number of patches signed who aren’t paid for their time and contribution.
www.linuxvoice.com 35
FEATURE INSIDE THE LINUX KERNEL
tree at this point in time for various technical reasons.”
The staging tree resolves “the ‘hundreds of different
download sites’ problem that most out-of-tree drivers
have had in the past” and gives developers a singular
place to concentrate their efforts.
Releasing a kernel
Since 2005, the Linux kernel has switched to using a
merge window to incorporate patches into the kernel.
During this merge window Linus Torvalds accepts
patches into his kernel tree, which will eventually be
released as the next version of the kernel. The merge
window lasts for two weeks. When it ends, Torvalds
releases the first of the -rc kernels, such as 4.0.2-rc1.
Instead of reviewing each and every patch, Torvalds
trusts the maintainers to only send quality patches for
merging into his kernel. Before it is released, Torvalds’s
-rc kernel is put through rigorous testing. Patches that
causes a feature to stop functioning, which is known
as regression in software development, are the first
A year in the life of veteran kernel developer Greg Kroah-Hartman.
Image Credit: Linux Foundation ones to get the axe if they aren’t fixed immediately.
As the kernel releases go through a period of testing
was a learning experience and fortunately people from and stabilisation, Torvalds puts out new -rc releases
the Linux community have been really nice.” once every week. A typical kernel release cycle has
Once a maintainer approves a patch, it’s entered anywhere between six to nine of these -rc releases.
into their subsystem tree. A maintainer accepts a Once it has been through testing, this kernel is
patch by adding a ‘signed-off-by’ line to the code. released as a new version and is dubbed the mainline
Some maintainers might have multiple trees; one for kernel. The whole process usually takes about 10
an upcoming kernel release and another for a future weeks. The actual time between kernel releases tends
release, for example. After a patch is included in a to vary a bit, depending on the size of the release and
subsystem tree, it’s bound to get more eyeballs as the length of time it takes to weed out any bugs.
it’ll now attract the attention of developers who are In the current scheme of things, this kernel
working on that subsystem’s tree. release containing the latest features and fixes isn’t
But what happens when, say, a patch uses a considered a stable release as it hasn’t been tested
function that’s been changed by another patch? To long enough. Every release also has a corresponding
avoid such conflicts and to enable developers to “stable” release which contains just the security
preview all of the patches being prepared for the next updates and bugfixes. There might not be noticeable
kernel release, the Linux kernel uses staging trees. changes in a new kernel version, unless it adds
These trees are a collection of the patches that come support for a piece of hardware that you use. You can
Just some kernel in from the various subsystems. view a summary of the differences introduced in each
developers debugging an In a blog post, kernel developer Greg Kroah- new kernel on the KernelNewbies website
issue at the Linux Kernel Hartman explains that the staging tree “is used to hold (http://kernelnewbies.org/LinuxChanges).
Summit in 2008.
Image Credit: Jonathan Corbet,
standalone drivers and filesystems that are not ready While all Linux distributions have the Linux kernel
LWN.net to be merged into the main portion of the Linux kernel at their core, it’s very unusual for the major distros to
ship the upstream mainline kernel as is. To wrap our
Strip searching the kernel
One noteworthy user of the mainline kernel is the Linux-
libre project. This Free Software Foundation-sponsored
project strips the kernel of everything “that is included
without source code, with obfuscated or obscured source
code, under non-Free Software licences, that do not permit
you to change the software so that it does what you wish,
and that induces or requires you to install additional pieces
of non-Free Software.”
According to the project the kernel started to include
binary blobs in 1996. The project uses scripts to check and
remove proprietary firmware from the kernel and produce a
100% free software version. This kernel powers several free
software distros, including the FSF-endorsed Trisquel.
36 www.linuxvoice.com
INSIDE THE LINUX KERNEL FEATURE
Compile your own kernel
Although the kernel is the core component of a distribution,
you can replace it with a custom one without much effort.
Replacing the kernel with a custom one is a wonderful
way to get a peek at how it works. It’s also one of the first
things you should get a grip on if you wish to contribute to
kernel development. Compiling a custom kernel gives you
the ability to tweak the stock kernel in a particular way, for
example to enable an experimental feature.
Another reason for using a custom kernel is if you have
hardware that isn’t supported by your distro’s stock kernel
but is supported in the upstream kernel or a third-party
kernel. The process also gives you access to the various
compilation flags using which you can optimise it for your
needs by stripping away any excess.
You can find instructions for compiling a custom kernel
for your distro in its official documentation or on its wiki.
According to the Linux
heads around the work that distros do to incorporate which 19 have already been merged in the upcoming
Foundation report, the
Torvalds’s mainline kernel into a release we reached 4.1 branch or one of the upstream maintainer trees. FOSS Outreach Program
out to Josh Boyer, who is one of the three members of “10 patches are for additional ARMv7hl board support, for Women introduced
the Fedora Kernel team. 19 are for Secure Boot support and the rest are 24 new developers who
patches that provide simple default setting changes, contributed 1.5% changes
How distros use the kernel or are in place to help us debug weird issues when in the run up to the 3.18
Fedora adopts two different approaches to selecting they pop up.” Put together, these patches will change release.
Image Credit: Linux Foundation,
the kernel for a release. The project’s development 121 files: “The single largest addition is the Ethernet Flickr
branch, known as Rawhide, follows Torvalds’ tree on a device driver we carry for X-Gene AArch64 boards,
daily basis. “We build Git snapshots of his tree every which is 10,320 lines by itself.”
day and it simply keeps following along,” says Josh. Boyer also shares the team’s intention to increase
On the other hand, the stable Fedora releases are contribution to the
based around the latest stable release of the kernel: upstream kernel “From about 10,000 lines in
“Fedora 20 and 21 are both on 3.19.8 at the moment, development: “While
and Fedora 22 is on 4.0.2.” During a release’s lifetime, Red Hat employs a fair 1991, the kernel has grown to
Fedora rebases the kernel, which is to say it swaps number of upstream over 19 million lines of code.”
out the old kernel with the most recent stable release. kernel developers,
This means that Fedora 21 will soon be rebased to the Fedora is not the place
4.0 branch, and Fedora 20 will follow shortly. where we are doing heavy kernel development. One of
While Fedora, just like the other distros, applies the goals for our small immediate team is to increase
patches to the upstream mainline kernel, it does its our upstream contributions, whether it be for bugfixes
best to keep these to a minimum. According to Boyer, or small cleanups, or hardware enablement. We want
“the significant majority of the patches we carry are to continue to be participants in the upstream kernel
actually patches that are already headed upstream. community and help where we can.”
They are either taken from the mailing lists or linux-
next tree and backported to fix an issue we’ve found.” A massive undertaking
This approach has its benefits: “We’ve found that by It is difficult for a non-engineer to fathom the size of
staying as close to the latest upstream kernel version the Linux kernel. From just about 10,000 lines in 1991
as possible, it is easier to work with the upstream the kernel has grown to over 19 million lines. If you’ve
developers. The code is still fresh in their minds, and collaborated on a developmental project, you can
they tend to be very responsive when we discuss probably imagine the work and effort that goes into
issues with them. Using the newer kernel also brings building and maintaining the most critical component
in a significant number of bugfixes that our team of your distribution.
wouldn’t be able to scale to cover via backports.” The Linux kernel is an engineering marvel, and one
That being said, there are always exceptions: of its inflection points, as compared to the proprietary
“The most significant patchset we’re carrying today model of development, is community participation.
is the Secure Boot work that we did in the Fedora The Linux kernel is one of the best documented
18 timeframe. We hope that will also eventually be open source projects, and contrary to popular belief,
merged, but it has been a slow process. We’re also the kernel welcomes new contributors. In addition
currently looking at kdbus and what we can possibly to the comments within the code there’s plenty of
do to help the efforts there.” documentation to welcome new developers. So
To summarise, Fedora 22 will ship the upstream budding contributors: grab your compilers, download
4.0.2 kernel release with 73 additional patches, of a copy of the kernel and get cracking!
www.linuxvoice.com 37
SUBSCRIBE
SUBSCRIBE
UK READERS!
Did you know that you can subscribe to Linux Voice from
just £10 per quarter with Direct Debit? Get every issue
straight to your mailbox (or inbox) and spread the costs!
What you get
116 pages each month
of the best tutorials,
features and interviews
Access to all back issues
in DRM-free digital formats -
over 1,500 pages
Take part in our yearly
profit donating scheme,
and help FOSS projects
Yearly Direct Debit prices
UK print subscription – £55
Digital subscription – £38
Quarterly Direct Debit prices
UK print subscription – £15
Digital subscription – £10
Go here now to subscribe!
www.linuxvoice.com/shop
Payment is in Pounds Sterling. If you are dissatisfied in any way you can cancel your subscription at any time and receive a refund for all unmailed issues.
38 www.linuxvoice.com
Linux Voice wants your ideas for tutorials, guides, how-tos and insights from
the hacker world. If you’ve found something you want to tell the world about,
let us know
What material is Linux Voice interested in?
Most of the time we’re more interested in what you can do with software X,
rather than singing the praises of software X itself. Clever software is good
but useful software is better. Proprietary software that works on Linux is
acceptable, but what we’re most interested in is Free Software.
What don’t you want?
We sometime get submissions that go like “I’ve been using Linux for X years;
can I write for you?”. This isn’t very helpful, to us, because what we want to
see is that you:
Have an idea
Can explain it clearly
If you can point us to examples of something you’ve written, please do
– we’re not looking for Shakespeare; we value clear communication and
enthusiasm above all else.
What do you want?
Tutorials. We want tutorials, of around 3,300 words in length usually. We pay
money! All tutorials should have a clearly stated aim, so readers know at
first glance why they should follow it. “Get started with XX software” doesn’t
tell you anything; “Build a weather tracker with Python” is much more active
and informative.
These are common reasons why we reject ideas:
Something which has been covered repeatedly on Linux Voice
and/or elsewhere
Material not obviously related to Free Software
Incoherent writing
Email ben@linuxvoice.com
to write for Linux Voice
FAQ Qt
Qt
Its birthday gives us the perfect excuse to revisit a project used
by almost everyone, from Canonical to KDE.
Qt is a serious contender in the square root in C’s standard library, for
GRAHAM MORRISON open source silly names instance, which is ubiquitous and as old
competition. Its developers and as time(.h). But when you start creating
Hasn’t the Qt toolkit been community pronounce it ‘cute’, which libraries and grouping them together to
around for ages? makes it feel a little like a hazing ritual help achieve a set of related goals, you
Qt has indeed been around for for a developers’ fraternity. Most other have a toolkit.
ages. In fact, it’s just celebrated people unofficially call it ‘Q T’, which Eirik Chambe-Eng and Haavard Nord
its 20th anniversary, which is quite suits us fine. started Qt after working on ultrasound
remarkable. We can’t say the same for equipment in a regional hospital in
many open source projects. Qt’s first The terms Toolkit and API are Trondheim, Norway, in 1990. They
version, modestly called 0.90, was born used to a lot in these pages, noticed how much extra effort it was
in Norway on 20 May 1995, and was but what exactly are they? taking programmers to port their work
the culmination of four years work by Both terms are almost to different platforms and wanted to
its founders, Eirik Chambe-Eng and synonymous, with toolkit create a toolkit that would enable them
Haavard Nord. Both founders have becoming a more modern term for to deploy their code on multiple
since departed, and Qt itself has been what mostly used to be called an API. systems with very little extra effort.
through a few transitions to get here. In Both help programmers to develop their
2008 it was dramatically bought by software more efficiently. At their most What features does Qt group
Nokia, which was subsequently bought basic, they’re a library of functionality. together?
by Microsoft. It’s now in the safe hands For example, a programmer who wants If you want to create a toolkit
of a subsidiary of Digia called simply Qt to write something to calculate the that’s going to work across
Company – Digia was one of the square root of a number could research different operating systems and
largest users and contributors to the their own solution and write their own environments, there’s very little you can
entire framework before buying Qt. implementation. They could copy actually build upon that’s common to
someone else’s, or they could use a them all. Input, output, sound, graphics
How is it pronounced again? toolkit that offers the function along – these all need to be unified within a
Kew-tea? with lots of other related functions. toolkit so that the programmer doesn’t
need to worry about whether the user is
Surely there’s more to Qt than on Windows, OS X or X.org. In an ideal
“Each major revision of KDE calculating simple
mathematical functions?
world, the programmer wants to write
code to add menus to their application
has followed a major revision Absolutely. Maths functions and those menus would appear on
of the Qt toolkit.” belong in a programmer’s maths
library. There’s a function to calculate a
whatever system the application was
built for and running on.
40 www.linuxvoice.com
Qt FAQ
In reality, all the major operating
systems use dramatically different
systems and APIs for their own menu
generation, and almost every other
function they provide. And this is where
Qt comes in. Qt is the bridge between
these systems, covering everything
the programmer might need – from
user-interface design and layout, to
remote procedure calls and accessing
Bluetooth devices.
If Qt is so great, why isn’t
everyone using it?
More projects seem to be using
Qt than ever, but it could be akin
to cracking a nut with a sledgehammer.
If you just needed a library to split up Qt 5 has taken a lot of features and suggestions from the team behind the KDE desktop.
character strings, for instance, it
wouldn’t make much sense installing Qt faction of desktop developers who cross-platform commercial software
solely for its excellent string handling. didn’t agree with the Qt licence. As Eirik like Spotify, Skype, Maya and
It’s a large download that would then Chambe-Eng explained in an interview Mathematica. Qt’s commercial growth
become a large dependency. with dot.kde.org back in 2004, “When gave Trolltech, the company behind Qt
Incidentally, a huge area of we started Trolltech we were fascinated at the time, the confidence to re-licence
improvement in Qt 5 is its modularity, so by Linux and the idea of free software. Qt. First under the GPL, then with the
that some parts of Qt can be installed At the same time we had neither the release of Qt 4 in June 2005, GPL for
without installing the whole thing. expertise nor the finances to do sales Windows. Nokia made the final step to
and marketing. It was really a very LGPL 2.1 in 2009. Qt was even in talks
With all this cross-platform natural and logical thing for us to give with the FSF when it created GPL v3,
love, what does this have to do away Qt for free for free software and added this as an optional licence.
with Linux specifically? projects (open source as a term didn’t LGPL v3 was finally added with the
Qt has a long and chequered exist back then).” release of Qt 5.4 in late 2014.
relationship with Linux. Most Unfortunately, this dual licensed open
importantly, it was the toolkit chosen by source/proprietary solution was So everything is awesome
Matthias Ettrich when he created the confusing. Not only was the open now?
KDE desktop way back in 1996, and it’s source version limited to Unix and X11, We think so. Thanks to Qt being a
still a fundamental part of the KDE it was also delivered by its own ‘QPL’ (Q commercial success, it enjoys
desktop today. Each major revision of Public Licence), which failed the Debian better quality control and
KDE has followed major revisions to Qt. Free Software Guidelines and was documentation than many open source
Qt 5.5 should be released in June 2015. incompatible with the GNU General projects. The commercial and open
Public Licence, despite being source versions are identical, but Digia
Why did KDE choose Qt if it compatible with the Free Software is using the new modular system to
wasn’t specifically looking for Foundation’s Free Software Definition. experiment with paid-for proprietary
cross-platform compatibility? This ambiguity led Miguel de Icaza and modules. These include code
Matthias chose Qt because he Federico Mena to create Gnome in optimisers, a purchasing API and a Qt
was looking for a comprehensive 1997 and to provide an alternative to Quick Compiler. This has caused some
graphical toolkit that could generate the rapidly growing KDE. concern in the community, but no one
modern-looking applications. He has yet taken the trouble to rewrite
wanted KDE to be more than a window What about the KDE desktop open source versions of those
manager and widget kit, and to do – was that open source? modules. And there was another
everything from file management to Yes, but because KDE is tied so debacle recently when Qt made open
email reading. Qt’s libraries made that intrinsically to Qt, the complete source users register before getting
easier to achieve with a single toolkit. project’s licensing wasn’t access to a download (the project later
straightforward. Qt was becoming very said this was a mistake). Digia and the
If Qt is so good, why didn’t successful as a proprietary product, Qt community knows that open source,
Gnome choose it? despite the free version being available. KDE and the wider community has
Ooh, this is a good story. Qt It’s currently used by a large number of been integral to its success, and
inadvertently led to the creation institutions, including the European remains integral to its future, so we’re
of the GTK toolkit and even Gnome itself Space Agency, Dreamworks, Disney certain it will remain a vital pillar in the
because these were created by a Animation Studios, as well as in lots of success of Linux and open source.
www.linuxvoice.com 41
INTERVIEW CORY DOCTOROW
“Science fiction writers
shouldn’t predict the future in
the same way that drug dealers
shouldn’t take their own
product – it never ends well.”
CORY DOCTOROW
Ben Everard and Graham Morrison meet the author, blogger, journalist,
digital rights activist and crazed adversary of all things DRM.
C
ory Doctorow once ran a technology is taking us. And Cory is when she opens parliament with a
peer-to-peer download service, prepared to live by example. His first statement of intent written by the
helped found the Open Rights book was the first book ever to be election’s victors (the centre-right
Group, and is a fellow of the Electronic released under a Creative Commons Conservative, or Tory party). The
Frontier Foundation. One of the ideas licence, and he’s committed to how speech was widely rumoured to
that kept coming up in our chat is that ill suited copyright and DRM are to a contain profound reforms to the UK’s
information doesn’t really want to modern age never envisaged by early obligations to the Human Rights Act,
be free, in contrast to the old hacker legislators. reforms that were supposedly cut at
adage. His books, both fiction and We met Cory Doctorow in May after the 11th hour, and it was under the
non-fiction, are incredibly popular and a brilliant talk he gave in Bristol. It shadow of these rumours we started
challenge the authoritarian direction was the day of the Queen’s Speech, our interview.
42 www.linuxvoice.com
CORY DOCTOROW INTERVIEW
You’ve said that future because you have a good chance of things like TrueCrypt and Enigmail. You
generations will look back at being picked up, but because everyone know Enigmail is supported by one
this period as being one in which it who’s ever been picked up has treaded developer – it’s a key piece of software
was really hard to copy stuff. But water until rescue came along. And I and its UI is not great and it needs
with the new Tory government and have hope for the future. patches to be brought up to date.
trade deals going through, do you I think what the Tories have proposed We were just talking about problems
still have optimism for the in this election may subject British with UEFI and full disk encryption.
liberalising of copyright laws? people to inordinate computerised risk That’s another area where people who
Cory Doctorow: I’m very sceptical of and will be terrible policy. But I think that are coders and administrators can work
optimism. I think that optimism and the global forces that are making and contribute and pop out reports and
pessimism are effectively predictions copying easier – which is to say look at open bugs and so on. EFF and
about the future. I think science fiction making computers faster, storage more the Free Software Foundation both
writers shouldn’t try to predict the compact and the internet more maintain lists of free software projects
fault-tolerant – those forces are largely that are looking for contributors.
“If you believe you shouldn’t indifferent to whether or not David
Cameron expands lawful interception
There are probably areas in your life
where you feel some despair because
start unless you’re pure, you capacity into a realm that can only be every month you’re sending some
won’t get anywhere.”
dreamt of at Hogwarts. money to companies that are working
to destroy the world. Maybe the only
How can the average person DSL provider in your neighbourhood is
future in the same way that drug help copyright change go in the BT, which is an enthusiastic participant
dealers shouldn’t take their own right direction? in the Great Firewall of Britain. Or
product. It never ends well. After all, if CD: Well there’s a bunch of things. Larry maybe it’s that you have pre-flavoured
you were optimistic about the future, Lessig [professor at Harvard Law mobile devices or laptops, which are
you would get up every morning and do School] divides the factors that legislate harder and harder to get away from. Or
everything you could to make our world into four forces: law, code, maybe your friends require you to use
computers better. If you were norms and markets. And I imagine Facebook in order to stay in social
pessimistic about the future, you would people who read Linux Voice are touch with them. You’ll never be pure.
do the same thing. So optimism and positioned to do code, which is And if you believe that you shouldn’t
pessimism don’t really give you a something a lot of people are not start unless you’re pure, then you’ll
course of action. Hope is, for me, a lot positioned to do. But there are free never get anywhere. But one thing you
more interesting. Hope is kind of why software projects that work towards can do is hedge against these
you tread water if your ship sinks. Not freedom and that need your help. That’s compromises you’re making. You can
“I think that the future is
up for grabs. That we can
change the future. That what
we do affects the future that
arrives…”
www.linuxvoice.com 43
INTERVIEW CORY DOCTOROW
help make up for it by tithing some of someone in the party of your choosing whether you were in the entertainment
the money that you spend every month who cares about these issues in the industry, was whether you were making
making the world worse, with way you do and is looking for moral or handling copies. And making and
organisations that help make the world support from within the party, and handling copies always implied
better, whether that’s Open Rights parties are responsive to their base. industrial activity, because books
Group or Article 19 or Privacy One of the reasons we’re not seeing always had printing presses in their
International or the Electronic Frontier the Human Rights Act repeal being history, records always had record
Foundation or the Free Software introduced in the Queen’s speech is pressing plants. But when something
Foundation Europe. All of these groups because, within the Tory party, it’s a copied was non-industrial – everything
do amazing work on all the issues that very divisive issue. And so, being part of we do on the internet makes copies –
relate to the free and open internet, and a noisy voice inside some party is a rather than saying, “OK well you know
they could use your help. really good way to make a difference we just need a new way to figure out
In terms of markets, supporting there too. who copyright applies to”, we just say
companies and firms that use free and that copyright now applies to everyone!
open source software is a really good Our magazine is released
“Supporting firms that use free
way that you can make an impact. It under CC Attribution Share-
doesn’t solve all of the problems, but it Alike licence within nine months.
solves some of the problems. And then What are your thoughts on that sort
software is a really good way
that you can make an impact.”
politically, getting involved with political of embargo process?
parties and working with them on their CD: Well it sounds reasonable. I think
information policy is a really important the real crisis of copyright is that
thing. In the last election, I joined the copyright historically has been made as
Greens because I think they had a really a means of regulating the If you’re sending an email to your
good information policy. But I know entertainment industry. So if you look at daughter while you’re travelling for
people in the Tories, in the Lib Dems, in the history of copyright laws, usually work, copyright should be the system
Labour and in the SNP who really care what’s happened is that the to regulate that too because, like, why
about information policy. They come at entertainment industry has had some not!
it from a different angle to me, but they best practices that it used internally, Creative Commons, as useful as it is,
do care. And they are part of the forces and they went to parliament or is a hack to get around the fact that
that drive their party’s policies in one congress and then got it based into law. there’s just a bunch of people who are
direction or another. So whatever your And the way that we figured out now being bound by copyright who
political stripe, there’s probably whether copyright law applied to you, shouldn’t be bound by copyright. So if I
want to read your magazine, I buy your
magazine and read it. I don’t have to
sign a contract. I don’t have to
understand a contract. If I want to
share this magazine with a friend, I
don’t have to regulate that behaviour on
the basis of a contract. The fact is, if
you have to understand the law and
then form a legal agreement in order to
read a magazine, what’s in the
agreement is almost irrelevant to
whether or not that’s good or bad. That
situation is bad because it makes no
sense, that looking at art or listening to
music or reading a book should be a
contractually regulated activity.
When your daughter’s a few years
older [Graham’s seven-year-old
daughter was assisting at the interview]
and she wants to make her first Harry
Potter fan website, she will be regulated
by the same rules that regulate Warner
Bros and Universal, when Warner Bros
licenses Harry Potter to Universal to
If you rip a page out of a book make that Harry Potter theme park.
and stick it up on the wall, that’s
And your daughter will probably not be
covered by copyright law.
This is crazy… capable of understanding those rules,
not because she’s not an exceptionally
44 www.linuxvoice.com
CORY DOCTOROW INTERVIEW
Corey reckons the Pirate Party,
for all its lack of electoral
success in the UK, has played
a part in driving the agenda of
digital rights within the EU.
intelligent person, but because those was noncommercial, and then it like commercial activity and activity
laws are made to be passed out by became the most successful book of that is not cultural, so it should be both
people who have four-year law degrees all time. And those cases are cases commercial and not cultural.
and then five years of specialist where both sides can hire lawyers and At that intersection we can define
training. And even if she were the ask the judge to decide which law is statutorily or we can define them in
Doogie Howser of copyright law and at dispositive when. But in almost every principle, or we can define them with a
age 12 was able to comprehend the law other case, where we find ourselves set of examples and then a set of
and pick up the phone and ring Warner asking, “Is this the right Creative principles for evaluating new activities
Bros and say, “What are the terms Commons licence?”, “Is this the wrong that are made possible by technology. If
under which I get a licence for my Harry Creative Commons licence?”, the reality you are doing something that is both
Potter fan website?”, no one there is that if we were dealing with physical commercial and not cultural, you
would answer her call, because the goods, nobody would be asking about should be bound by copyright law.
terms are you can’t have one. licences anyway. And any answer to Otherwise, if you’re bound by any rules,
But kids have been doing fan-ish that question that doesn’t start with they should be cultural rules. Plagiarism
activity in relationship to the literature that is giving too much credit to sense isn’t always a copyright violation. If I
they love for longer than copyright has that is really a nonsense. claim to have written Shakespeare and
existed. And the fix to that is not put my name on the cover of The
figuring out how to streamline If copyright is an obfuscation Tempest, I’ve committed no copyright
copyright law so kids can understand it, and we’re always going to be law violation. Normatively, culturally, I’ve
the real fix for this, if we’re ever going to chasing that, and Creative Commons done something really bad. They might
make sense of this stuff, is to change is a hack, what should we do? be formal rules, they may be normative
who copyright law applies to by CD: I think we can define a suit of rules, but they won’t be copyright’s
redividing the realm of activities into activities that we think of as industrial, rules. Which rules does copyright have?
cultural and industrial. There will be whose litmus test isn’t ‘are you Well that matters a lot to me as
things that will be in the middle. Like 50 copying?’, but whose litmus test somebody in the industry supply chain,
Shades of Grey started as fanfic and it involves something more complicated but almost everyone else in the world
www.linuxvoice.com 45
INTERVIEW CORY DOCTOROW
can just ignore copyright. We should
start by saying, these activities are
commercial, these activities are
cultural; if you’re not doing something
commercial and you’re doing
something cultural, then you’re
governed by a different set of laws.
Is there something we could
create to run in parallel to
Creative Commons?
CD: No, unfortunately there isn’t. This is
an area where we need legislative
reform because the copyright law
works under international law… You
automatically get a copyright for your
life plus 50 years.
Maybe you could get some students
to say that some classes of works
Want to read more Doctorow? There are
aren’t in copyright, but I mean
plenty of books of his available on all
effectively we have to break the legal good internets.
deadlock in order to make sense of this.
That doesn’t mean we should give up
on Creative Commons. I believe in Moorfields Eye Hospital, which is the software trunking is in the walls, they
Creative Commons licences, support best eye hospital in the country, are effectively saying our business
them financially, I use their licences, I possibly in the EU, and after the NHS model trumps best engineering
promote their use because it’s the next eHealth record system collapsed, they practice, and for that reason alone,
best thing to fixing this untenable hired an open source developer named leaving aside all of the questions about
hairball of legal gubbins. To pretend Chris Reading to build a LAMP stack efficiency and code quality and cost,
that the problem isn’t an untenable glaucoma tracker called OpenEyes (see they should never ever be buying
hairball, is to ignore the problem. We www.openeyes.org.uk). And I went and proprietary software.
can only fix so much around the edges. gave a talk for them to other eye Firms have dealt their commercial
surgeons about why this makes sense advantage out on the basis of their
Why do you think licensing has and why you should use this open ability to deliver to deadline, their ability
really caught on in the tech version instead of paying one of the big to liaise with other firms and gather
world of open source software, but consultancy firms to build you a their requirements and reflect them
has been slow in other areas like proprietary glaucoma tracker. And I back to them in their projects that they
publishing and music? said, leaving aside all the commercial build and so on, but not on secrets
CD: Code is, on some fundamental considerations, when you put a wing on about how they accomplish their stuff.
level, maths. And maths is science. And the side of your hospital and the firm of The secrecy is in the culture of the firm,
science has, since the enlightenment, engineers comes in and says we’re not that’s their proprietary secret source,
operated on the basis that everyone going to tell you how to calculate the not in the standard tools that they use.
else has to be able to replicate what you load stresses on this RSJ (rolled steel No one uses a secret bulldozer, right?
do and build on it. So, if you look at the joist) because that’s proprietary, we’re And no one uses a secret RSJ. And
history of open licensing, it actually not going to tell you where the trunking nobody should use a secret operating
came about really when a commercial is in the walls, because we want to system for anything that matters.
entity started to assert copyright on make sure that you pay us when you
paper tapes. It started when Richard want a new mains outlet. For your books, you use a CC
Stallman walked into the lab one day You would say, commercial non-commercial licence.
and the paper tape drawer was locked, considerations aside, that this is not CD: Non-commercial, and then some of
and no one would give him a key to the about whether you deserve to make them are Share-alike and some are
drawer with the paper tapes in it. money, it is just not right. It is not non-derivatives.
It was a collision between an responsible for us to have a hospital
enlightenment ethos, that says that you where we can’t independently verify the Does much come out of the
have to tell other people what you know way that you calculated our load Share-alike books?
in order for knowledge to be collectively stresses. CD: Yeah, there’s a fair bit. Mostly, the
advanced, and this industrial drive to And so, when the consultants come only ‘problem’ I have is the order in
property-ise information. in and says our software engineers which the translations came out. So
Think of it from the perspective of aren’t going to tell you where the what I found is that foreign publishers
open eHealth records. I live near software RSJs are and where the were by and large OK with translations,
46 www.linuxvoice.com
CORY DOCTOROW INTERVIEW
provided that it didn’t surprise them. CD: Back then my publisher was like been sold to a giant multinational
Provided that I sat down and talked to ‘we’re paying you $7,500 for your first company. I’m not going to name them,
them and said ‘do you know when you novel, we’re printing 10,000 hard but they’re the largest publisher in the
buy this book and do a translation, fans covers, we break even if we sell 4,000 of world. The guy who bought it was this
are going to be able to do their own them, what’s the worst that could brilliant guy who was their head of
translations too’. There will be a varying happen here?’ And a lot of people said digital strategy. And he knew where I
quality, but their primary motivation is ‘Doctorow can afford to do this, no stood on DRM and I knew where he
not to compete with you, but to improve one’s ever heard of him, what does he stood, and we were all cool with it. We
access, to demonstrate and improve have to lose?’ Now I hear a lot of, ‘well worked on it for years, on this book, and
their own mastery of the language, to Doctorow can afford to do this, he’s so went through lots of revisions. It wasn’t
do something with a group of friends. well known, he can afford to do it’. all that we were doing, it was just every
My foreign publishers were totally OK You’re kind of damned if you do, now and again that he’d send me
with this, but they went into a meeting damned if you don’t. I don’t think revisions and I’d get to them and I’d
at the Bologna book fair with my foreign Creative Commons licences make send them back, or we’d get sketches
rights agents who pitched them on a people care about your books. But what from an artist, and then go to a different
book, and they said “but there’s already they do is they make it so that people artist. Finally we’re ready for the
this translation out there.” They could who do care about your books find it contract… And months went by and my
never make sense of how that easier to share them, to promote them. agent was bugging him and finally he
translation could operate alongside That’s been the guiding light of my CC sent me an email from his non-work
their commercial edition. philosophy. My editor at Tor Books, email address and said could you call
Patrick Nielsen Hayden, is super Linux me on my non-work phone. And he
Were you the first publisher to savy. Like, I met him on a BBS in the said, you know, I’ve been over it with
use the Creative Commons 80s, he administers his own Linux contracts but they won’t do a contract
licence? boxes. I went over to his place once and without DRM, and I’ve tried everything. I
CD: My book was the first ever Creative I was like “What browser is this!?”, and said look what if we just don’t buy the
he said “Oh, it’s Konqueror”. That was eBooks? And they said no, we have to
“Nobody should use a secret the first time I saw Konqueror! So he’s
pretty tech savy – he gets this stuff. He
now. And he said ‘OK, what if we use
the covenant not to use the eBook
operating system for anything was writing Google type plugins back rights unless we have mutual
that matters.”
when I was doing this stuff. So it was a agreement?’ And they said no we can’t
very easy sell. do that either. And I said look, here’s the
pro forma spreadsheet, nobody buys
Commons book (Down and Out in the Have you had to walk away picture books in electronic form. Our
Magic Kingdom). It came out the same from any deals because a pro forma earnings after we pay for the
week as the Creative Commons publisher wouldn’t accept it? conversion is minus £80 on this eBook,
licences. CD: Kind of. I have a picture book we’ve already sunk thousands into this
coming out next year from another eBook, are you crazy? And they said no,
Was that a difficult division of Macmillan, which is the we can’t do it.
conversation to have with your same publisher that does my novels, And that’s when he said, “so that’s
publisher? Tor. And the picture book had originally when I quit my job”. So he quit his job
that day to go to work for another
publishing start-up that was doing
amazing digital publishing stuff.
Macmillan snapped up the book and
now it’s coming out in 2016. It’s called
Posy the Monster Slayer and it’s going to
be a fun book. It’s about my daughter,
who is your age (talking to Graham’s
daughter), who one night, after she’s
been given all these super girlie toys for
her birthday like a Barbie®
Dreamhouse™ and a tiara and all this
junk, and she eats too much ice cream
and cake and when she goes to bed,
she has nightmares that the monsters
are coming. And so she takes all of her
girlie toys and she turns them into
weapons and she kills all of the
Next from Doctorow: a parable on the
dangers of eating too much ice cream. monsters with them. It’s pretty fun! So
anyway, all’s well that end’s well.
www.linuxvoice.com 47
LISTEN TO THE PODCAST
WWW.LINUXVOICE.COM
BUY MUGS AND T-SHIRTS!
shop.linuxvoice.com
INTRO REVIEWS
REVIEWS
The latest software and hardware for your Linux box, reviewed
and rated by the most experienced writers in the business
On test this issue...
50 52
Andrew Gregory
is hayfever-free after moving so far north that
nothing can grow.
J
ohn Deere is a company that
makes tractors. Those tractors Cinnamon 2.6 Fedora 22
have engine management Mike Saunders explores the desktop to Committed KDE fan Graham Morrison
systems, and the engine management end all desktops – it works as you expect steps out of his comfort zone to try a
systems need computer code in order it to, it looks great, and it won’t cause Linux distro with Gnome’s footprints all
that they may manage the engine. So Gnome vs KDE arguments. over it – and absolutely loves it.
far, so innocuous.
However, in the wonderful world of
proprietary software, John Deere claims 53 54 55
that, because of the presence of this
code, when you hand over your money
to buy your dream tractor, you don’t
actually own it – you’re merely licensing
it. Despite the tractor being physically
present in your field, it isn’t yours; it still
belongs to the company.
Gnu Octave 4.0 OnlyOffice Scribus 1.5
Ben Everard likes graphs, Ben Everard also likes What is still quaintly called
Property rights code, and playing with huge the convenience of a desktop publishing has a
This is really, really weird, as the EFF data sets. This cloud-based office suite – Free Software champion
has been telling us since this story programming language is especially when he’s not – Graham Morrison tests
broke in April. But what’s been right up his street. being spied on. the new features.
overlooked is the question: in what way
would John Deere lose out if it open
sourced the software in question? BOOKS AND GROUP TEST
I can’t imagine horny handed sons of
The Raspberry Pi has outgrown its makers’
toil sitting around a laptop in the barn
intentions several times, and the latest incarnation
debating the merits of Python 3 vs 2 in – Version 2 – now has enough grunt to function as a
the implementation of this year’s usable desktop computer, running a web browser,
sowing patterns. But I can imagine a productivity software and more, all for a bargain
scenario in which the increased price and minimal power consumption. Our challenge
this month is to find the best distro to take
serviceability of its tractors made John
advantage of this new power, whether that’s an old
Deere more popular among folk with an favourite optimised for ARM, or a completely new
eye for a bargain, as farmers usually creation. And in the book review pages, we learn that
are. Silly buggers. the printed page is still a viable medium for learning!
andrew@linuxvoice.com
www.linuxvoice.com 49
REVIEWS DESKTOP ENVIRONMENT
Cinnamon 2.6
Linux Mint’s Gnome 3 fork has come a long way in the last four years.
Mike Saunders explores this shiny new release.
F
orks in the free software world are often make it look like Gnome 2.x releases, with Cinnamon
DATA regrettable events, and can result from nothing 2.6 many individual Gnome programs have been
Web
more than personal squabbles between forked and the two codebases are now distinct.
http://cinnamon. developers, but Cinnamon demonstrates that forks Cinnamon is still very much associated with the
linuxmint.com can be productive as well. Back in April 2013, with the Linux Mint distribution, but the desktop is available in
Developer release of the Gnome 3 desktop and its radically other distros such as Fedora. We tried Cinnamon 2.6
Linux Mint and others redesigned Gnome Shell, the Linux Mint distro team by installing Mint 17.1, the latest available release at
Licence
GPL
was left with a dilemma. On the one hand, the Mint the time of writing, which included Cinnamon 2.4. We
developers wanted their distro to stay fresh with the then added the Romeo repositories for packages that
latest desktop technology – but on the other hand, are still undergoing testing – and one apt-get update
they were concerned about the impact of Gnome 3’s && apt-get upgrade later (and 500MB of downloads)
redesign. After all, Mint we had the sparkling new release to try out.
prided itself on offering
“Cinnamon 2.6’s user-facing a slick all-round So what’s new?
improvements, performance desktop OS with a
familiar user interface,
Desktop environment upgrades can often inflict a
noticeable hit on performance, but we found no
tweaks and buxfixes all add up.” whereas Gnome 3 change between Cinnamon 2.4 and 2.6. Both started
looked drastically up in the same time, and the stock Linux Mint
different and was installation, freshly booted up, occupied around
crafted with tablet and touch interface users in mind. 330MB of the RAM banks with both versions. So
So the Mint team took the difficult step of forking Cinnamon will run decently in 1GB of RAM, but it’s far
Gnome. This was received with mixed reactions by from being a lightweight desktop in the realms of Xfce
the wider Linux community: do we really need more and LXQt. Mint’s Cinnamon themeing is excellent; the
forks? Will it just die out when the Mint crew run out desktop looks polished and tidy, with subtle effects
of energy? Why don’t they just use Mate (a Gnome 2 and few distractions.
This release doesn’t look
drastically different to 2.4, fork) instead? Well, here we are over four years later, The main menu organises applications into
but there are many small and Cinnamon has gone from strength to strength. categories, and also features shortcuts to commonly
and subtle changes all Whereas early releases of the desktop were basically used programs down the left-hand side. Meanwhile,
around the interface. the Gnome 3 codebase with some design tweaks to the panel along the bottom is very much Windows-
esque, with a taskbar, system tray and clock. Some
Linux users might deride this setup as being rather
unimaginative, especially with Gnome 3 and Ubuntu’s
Unity adopting more novel approaches to window and
desktop management, but familiarity (and ease of
transition from Windows) was always one of Mint and
Cinnamon’s goals.
We’ve never had serious stability problems with
Cinnamon over the years, but the developers have
added a new shortcut for this release: Ctrl+Alt+Esc.
This restarts the Nemo file manager and cinnamon-
settings-daemon process, effectively restarting the
desktop environment as a whole, and is provided in
case of a freeze (which has been reported in earlier
releases, according to the developers). This key
combo doesn’t restart applications, however – so if
you’re doing something in LibreOffice and Cinnamon
locks up, you can restart the desktop without having
to log out and lose all your work.
In terms of user-facing improvements, the System
Settings dialog has been redesigned to be simpler
and more attractive. Options are neatly divided
into sections accessible via buttons along the top,
with silky transitions between them. New window
50 www.linuxvoice.com
DESKTOP ENVIRONMENT REVIEWS
animation effects have been added as well, with a
great deal of customisation available for how quickly
they perform. On the desktop, the power applet has
been rewritten to consolidate multiple features in
the same place: along with battery information, it
also provides controls for screen brightness (and if
applicable, the keyboard backlight).
With Cinnamon 2.6, it’s now possible to have
multiple panels operating independently, and a new
applet called Inhibit has been introduced which
prevents notifications and power management
from interrupting presentations and similar work.
The sound applet has been updated too, with better
PulseAudio integration and the ability to change sound
levels for individual applications.
Nemo, Cinnamon’s file manager, sports a new plugin
manager for single-click enabling actions, scripts and
extensions that can be applied to files and directories
via the right-click context menu. This context menu
has also been cleaned up to show only the most
common operations, while large file operations are
now queued up and performed sequentially, rather
Finally, a proper
than in parallel like in previous releases. usage by 40%. To speed up the desktop’s load time,
screensaver! Now
In previous versions of the desktop, the cinnamon- Cinnamon 2.6 now has a preload feature that caches Cinnamon can use fancy
screensaver tool did little more than lock the screen themes and application information earlier in the screen locking effects from
after a certain amount of time; with Cinnamon 2.6, boot sequence, so after login the desktop appears XScreenSaver.
screensaver support is much more extensive. You can much more quickly. Optimisation work is difficult
use fancy XScreenSaver modules along with HTML 5 and tedious, and most developers would rather be
screensavers, and customise many more settings. working on flashy user-wooing features, so we give a
Other user-facing changes include improvements to big thumbs-up to the Cinnamon crew for their efforts
the accessibility tools, including the magnifier, mouse in this area.
zoom modifier and on-screen keyboard (which now For application developers, Cinnamon 2.6 is the
has an auto-hide facility). first release to include proper documentation (see
http://developer.linuxmint.com/reference). It’s a
Beneath the surface mixed bag and has some major holes, but at least it’s
Under the hood, a significant change has been a start and provides some useful tutorials for creating
implemented to improve the desktop’s portability. applets for the desktop. Because multiple panels are
Cinnamon can use both ConsoleKit and Logind (the now supported, many applets will need to be updated,
latter is from Systemd) to handle user logins, but in especially if they assume they will only ever be
previous releases this had to be defined at compile installed on a single panel.
time. With version 2.6, you can choose which So, is Cinnamon 2.6 worth the upgrade from 2.4?
framework to use by changing a setting. This is a bigger question than it sounds, because for
Meanwhile, the Cinnamon team has done a lot most users it will involve more than just grabbing a
of work to reduce CPU usage and avoid duplicated few packages; it will mean a major distro upgrade.
actions. The main menu is now drawn six times Linux Mint 17.2 (“Rafaela”) is due to be released
less frequently than before, while optimisations in around the end of June, so it should be available
the window manager have reduced its overall CPU to download by the time you read this. It’s well
worth the upgrade, even if you have to grab other
Cinnamon vs Mate packages in the 17.2 update as well. The user-facing
improvements, performance tweaks and bugfixes add
Both Cinnamon and Mate were born out of dissatisfaction up, and with the new documentation it’s turning into a
with Gnome 3’s redesign, but the two desktops had very serious competitor to the established desktops.
different goals early on. Whereas Mate was happy to
continue with the Gnome 2.x codebase despite its age, the
Cinnamon team was eager to use code and components LINUX VOICE VERDICT
from Gnome 3 – just with a more familiar design. Mate
is still going strong today and the codebase is receiving Everything a desktop should be:
many updates, so in some respects it’s getting closer to attractive, fast, familiar but still
Cinnamon. There’s also now an official Ubuntu flavour that customisable. KDE and Gnome have
bundles Mate as the default desktop. But could the two some real competition now.
ever merge? One can dream…
www.linuxvoice.com 51
REVIEWS FEDORA 22
Fedora 22 Workstation
Graham Morrison tests a distro that uses the Gnome desktop – and likes it!
F
edora 22 was released at the very end of May,
DATA but we’ve been using the betas in earnest since
our monster distro roundup last issue. As is
Web
https://getfedora.org often the case with Fedora, this release is mostly a
Developer revision rather than a revolution, with a few modest
Fedora/Red Hat refinements and the latest packages. Those
Licence refinements this time are unlikely to have a huge
Free plus proprietary
impact on your Fedora experience, but they’re also
firmware
quite substantial in the way they change Fedora’s
internal plumbing. That we’ve experienced no major
issues, even with the betas, means these refinements
can only have happened after considerable effort and
planning.
The most significant of these refinements is of Fedora is now available as Workstation, Server and Cloud
course the new package manager. If you think about spins. Workstation is designed for desktop users.
how intrinsic package management is to the fabric of
your system, it’s a significant success that the switch your network. But they do require some local
from Yum to DNF (Dandified Yum) barely flutters a processing to create an installable package. From our
neuron. DNF is almost entirely equivalent to Yum on perspective in rural Hobbiton, we approve of this
the command line, accepting nearly all the same change, but it would be useful if the package manager
arguments. Of the few recognised from your network and CPU combination
“It’s a significant success when that are missing,
--skip-broken is the
that installation may be quicker from a full RPM
download rather than a reassembled RPM and offered
the switch from Yum to DNF guilty favourite we’re you the option to revert to full-fat RPMs.
barely flutters a neuron.” going to miss most.
Even when muscle Gnome Home
memory types yum by The other major feature that most users are going to
mistake, you’ll find yum is now a simple script that notice is the inclusion of the latest Gnome desktop,
politely reminds you of its own imminent demise version 3.16 (see our review issue 15). Gnome isn’t
before passing on your arguments to DNF. specific to Fedora, but Fedora always does a good job
Behind DNF is Hawkey, the new API for packaging at creating a default environment, and we’re really
There’s lots of cutting-edge
software in Fedora, that’s responsible for resolving dependencies. The starting to enjoy Gnome. We love the new grey look
including the very latest packages themselves are delta RPMS. Deltas contain and the new notifications. These appear from the
release of Gnome’s new only the difference between the installed and the new middle of the top of the screen and you can now
development environment. versions of a package, and take considerable load off interact with some, such as the calendar or an
incoming message.
It was reported that Python 3 would become default
in Fedora 22; that transition has been pushed back to
Fedora 23. We’re used to dealing with this problem in
Arch, where version 3 is already the default, and it
does create considerable difficulty for Python users.
Finally, the other major upgrades include GCC 5, which
is now the default compiler, and the inclusion of KDE
Plasma 5 - its most significant endorsement. We were
also impressed by the inclusion of the shiny new
Gnome Builder IDE. It’s in alpha but well worth a look
(see p104 for our getting started guide).
LINUX VOICE VERDICT
A strong release by virtue of good
upgrades and stability despite major
changes to its infrastructure.
52 www.linuxvoice.com
GNU OCTAVE 4.0 REVIEWS
Gnu Octave 4.0
Messing about with mathematics might not be everyone’s cup of tea, but
Ben Everard finds a way to make it more fun for everyone.
G
nu Octave is a mathematical programming
language that’s designed to help users analyse
and visualise numerical problems. It first came
out in 1988 and gets a major release on average once
every eight years. With such a conservative
development pace, 4.0 is a big release, and it comes
with a killer new feature: a graphical interface.
Prior to this version, Octave ran in a terminal
window. Octave 4.0 comes with a Qt graphical user
interface, which uses OpenGL to render graphics.
Since Octave is often used for visualising data, this
should help it take better advantage of graphics
hardware when rendering large data sets.
The GUI doesn’t add anything new to the
functionality of Octave; it just makes it a little nicer
to use. Most of the window is taken up with the text
interface to Octave that’s exactly the same as it
running in a terminal. This command window (which
isn’t an individual window, but a pane in the main
The new GUI has a clean,
window) also has tabs for a command editor and Perhaps the biggest draw bringing people to Octave
uncluttered interface
documentation. Additionally there are panes to show is its compatibility with Matlab. Matlab is another which is easy to use even
files, workspaces and the command history. All these high-level mathematical programming language for people unfamiliar with
are moveable to other parts of the window. that’s long been popular at universities. It does have Octave.
a Linux version, but it’s proprietary and licences can
Graphical gloss be expensive. Octave isn’t perfectly compatible with
New people coming to the software might find the Matlab, but it’s close. Version 4 brings even better
DATA
different interactions in the command window and the compatibility in quite a few areas. We won’t go into
editor a little confusing. In the command window, them all in detail, but all the improvements are listed in Web
pressing Tab completes a command, while in the the release notes at www.gnu.org/software/octave/ www.octave.org
Developer
editor, a drop-down list automatically appears giving NEWS-4.0.html.
John W Eaton et al
options for command completion. This is obviously Many pieces of Matlab code will work out of the Price
because of the different heritage of the two forms of box, and it’s usually not too hard to port those that Free under GPLv3
input, but the difference seems confusing. won’t. This makes Octave a great choice for people
Console fans haven’t been abandoned, and you can who have been trained in Matlab, but want to switch
start the software in terminal mode by starting Octave because of either financial or ethical concerns over
with the --no-gui flag, or --no-gui-libs if you also want the proprietary model.
to use terminal-based plotting. You can also change The graphical interface of Octave version 4 is a
the plotting library from the default Qt to either Fltk or major step forward, especially considering that all the
Gnuplot using the graphics_toolkit() function. alternatives (such as Matlab and iPython) have great,
well tested interfaces. Even though the core of the
language works well, without a GUI, it’s hard to see
how Octave could have stayed relevant in the face of
some excellent competition. The GUI lifts the software
from an obscure piece of command line software
used by a few geeks to a genuine option for teaching
and investigating mathematical phenomena.
LINUX VOICE VERDICT
The new graphical user interface
makes Octave more accessible to
new users.
Visualisations have always been a speciality of Octave,
and in version 4.0 they’re rendered in OpenGL.
www.linuxvoice.com 53
REVIEWS ONLYOFFICE
OnlyOffice
Ben Everard reclaims his privacy with a web-based office suite he controls.
O
nlyOffice is a web-based office suite similar to
DATA Google Docs or Microsoft’s Office 365.
Web
However, unlike its competitors, OnlyOffice is
www.onlyoffice.com open source (under AGPL) so you can run it on your
Developer own server (there’s a hosted version available as well).
Ascensio System SIA OnlyOffice is a rebranded version of TeamLab
Licence Office, which has been around in one form or another
AGPL
since 2009, so it’s had time to mature to a featureful,
stable platform. However, it was closed source and
Windows-only until the end of 2014, so is still fairly
unknown in the Linux world.
There are two parts to OnlyOffice: the community
server and the document server. The community There are some theming options available for anyone who
server is for collaboration and includes document doesn’t like the default OnlyOffice colour scheme.
sharing, and other tools for working as a team. The
document server is just for viewing and editing usual protocols. Again, this is perfectly functional,
documents. OnlyOffice also releases a though unremarkable, and probably not enough to
“The real star of mailserver, but this isn’t developed in-house,
instead it’s a build of open source mail tools
convince anyone to use a new collaboration tool.
OnlyOffice is the including iRedMail and SpamAssassin. Word processing power
online office suite.” You can install OnlyOffice on top of a
distribution just like you would any other
There’s also a calendar and an online chat tool. The
Community Server can link to any online storage that
software, however, there’s also a series of offers WebDav access (such as Box or OwnCloud).
Docker images, which means you can deploy the This is a great option if you want to take advantage of
software with a single command. The images are in OnlyOffice Documents while still using your existing
the Docker hub, but the Dockerfiles are on GitHub so cloud storage option.
you can modify them to customise the build (https:// The real star of OnlyOffice is the online office suite
github.com/ONLYOFFICE). – the word processing is the best open source web-
The OnlyOffice Community Server is most useful based document editing experience available. This
as a web-based document sharing tool. In this role, may change when LibreOffice online is released, but
The word processor uses a
it works well, but doesn’t offer much to distinguish it for now, nothing else we’ve seen comes close in terms
HTML 5 canvas element to
over the competition (such as Seafile or OwnCloud). of experience. It can handle complex layouts, it has
display the editor, so you’ll
need a modern browser – In addition to the document sharing, there’s also an plenty of features and it runs well (provided you have
we didn’t find any problems online email client. This connects to an email server a modern web browser). There’s also a document
in the common Linux that could be the official OnlyOffice build of iRedMail, viewer that can be embedded in other web pages to
options. or could be any other mail server that supports the allow you to share read-only access with the world.
Our biggest complaint is that OnlyOffice
Documents uses Microsoft’s DOCX, XSLS and PPTX
formats. It can handle others (such as ODT), but
only by first converting them into DOCX (they can be
converted back before downloading). This probably
makes sense from a pragmatic point of view, but it’s
disappointing from a document freedom perspective.
The spreadsheet and presentation editor are
similarly impressive. While none of the office suite has
quite the range of features you’d expect of a native
suite, they have enough capability for most tasks and
far more than other web-based office suites.
LINUX VOICE VERDICT
The best online office suite backed up
by a reasonable collaboration server.
54 www.linuxvoice.com
SCRIBUS 1.5.0 REVIEWS
Scribus 1.5.0
Graham Morrison can’t resist the lure of new features, even when they’re
from a development branch.
F
irst, a warning. This isn’t a stable release. Like
lots of open source projects, Scribus uses the
odd-minor-number versioning scheme to
differentiate a development release from a stable
release. The current stable release of Scribus is still
version 1.4.5 (the 4 being the minor number we’re
talking about), which was made available in February
to fix a few bugs.
The version we’re looking at, 1.5.0, is the first major
update to the development branch, and it’s not in any
way meant for a production environment, or for
people who rely on Scribus to get some real work done
– people like us! In particular, it’s no longer compatible
with the old file format, so you can’t try working with
1.5.0 and revert to a stable release if you encounter a
bug and need to continue with the same file. But now
that we’re trying to use Scribus more and more at the
magazine, it makes sense for us to also check out the
new features and see where development is headed,
The code to handle many
especially when this is already a huge update. calligraphic pen works brilliantly, especially with a
import and output formats,
In particular, and like many other applications, stylus. There’s a new picture browser plugin too, including PDF and
Scribus has finally made the transition from Qt 4 to Qt accessible from the ‘Extras’ menu. This enables Illustrator, has been
5.4. This is long overdue and makes Scribus snappier you to browse images and tag them with your own completely rewritten, with
and less resource hungry, at least if other Qt apps descriptions, which can then be searched or grouped already excellent results.
are anything to go by. The old Scribus was one of the together. Considering the number of images we get
last applications we had installed still using the old through in an issue, this is going to be a great addition.
version, so it will be good to see that dependency go. There’s also been a huge amount of effort put into
As for features, the big new addition is the rewrite file compatibility, and there’s a vast array of improved
DATA
for tables. In the old version, tables were a simple and new import/output filters. We tested the new PDF
frame where individual cells felt more like a hack. It import, for instance, and experienced no problems Web
was difficult to resize and realign cells, for example. with the translation (unlike with the previous version), www.scribus.net
Developer
You can now drag columns, insert and delete rows/ and it was faster. We also tested some of our Adobe
Scribus Team
columns and merge cells together, all options Illustrator files, and these also worked brilliantly. For Licence
available from their own menu. the first time, Adobe Indesign gets an import filter, GPL
but this is only for the XML format rather than the
Les Arcs proprietary binary format. We’re hopeful that InDesign
Then the new features start rolling in. There are support will come, because we’re sure this will help
new vector tools for arcs and spirals, and the new many designers who want to move away from their
chosen platform.
As well as the new features, we also experienced
great stability, despite this being a development
release, although we wouldn’t dare us it for production,
and we hope that Scribus can maintain this fantastic
form. We’re now eagerly looking forward to Scribus
1.6.0, and despite there being no release schedule for
this, it can’t come soon enough.
LINUX VOICE VERDICT
A great development release that
shows huge potential for the next
stable update.
Manage the huge number of images you typically have to
deal with in print with the new Picture Browser.
www.linuxvoice.com 55
REVIEWS BOOKS
Building Machine Learning Systems
with Python, Second Edition
Ben Everard makes computers learn so he doesn’t have to
M
achine Learning Systems with that it’s a complex field. As well as knowing
Python takes the reader on a tour how to use the techniques, you need to
of the SciPy module’s machine know when, and the subtleties of it.
learning routines. It doesn’t dwell too While this book gives you a good
much on what the algorithms do; instead introduction, it won’t make you an expert
it focuses far more on the practical side of in machine learning. That’s not necessarily
things. Because it’s mostly focused on using a criticism – no sensibly sized book could
the module, the code is quite simple, so you take you from beginner to expert in ML.
don’t need to be a particularly skilled Python However, before embarking on a machine
programmer to follow this book, though learning adventure, you need to be aware of
decent school-level maths will help. the challenge.
Most of the book covers the problem of
classification. That is, trying to identify what LINUX VOICE VERDICT
class a particular piece of data should be Author Luis Pedro Coelho and Willi Richert
in. The most famous classification problem Publisher Packt Publishing
is spam filtering, where a piece of software Price £32.99
ISBN 978-1784392772
has to classify whether a particular piece of
A solid introduction to the basics of implement-
software is spam or ham (ie not spam). ing machine learning with SciPy
Before you venture down the path of
machine learning, you need to understand We long for the day when we can import brain.
Seven Concurrency Models
Will the spinning top eventually fall? Graham Morrison may have an answer.
A
fter working on my own projects, I and when. It does this with Clojure, and while
have some form with concurrency. it borrows from things like Go’s concurrency
I spent months attempting to model, we’d have preferred to see Go actually
coerce a dozen different multithreaded used rather than name dropped. But that’s
processes to listen to the same clock, and our preference for what’s currently a very
then report things at the same time. My cool language.
code was left looking the opposite of the This book succeeds in teaching people
‘crystal clear, high-quality’ code promised by with no specific knowledge of concurrency,
Seven Concurrency Models, and even when it (but firm programming skills), what an ideal
worked as expected, I no longer understood solution looks like. It builds in complexity
my own solution. through its nine chapters but remains
Covering seven different solutions in seven readable and interesting, leaving us with a
weeks (to give the book its full title) is hugely much clearer idea for our own projects.
ambitious. The subject is complicated and
often mind-bending, making this a book with LINUX VOICE VERDICT
a very specific readership. And rather than Author Paul Butcher
my humble threaded code, the concurrency Publisher Pragmatic Bookshelf
described here is designed to scale: think ISBN 978-1-93778-565-9
Price £25.50
Twitter rather than tmux. The book’s great
A complicated subject, but one that’s all too
trick is that its examples and text uses easy to ignore without books like this.
functional programming, rather than If we had the choice of any super power, it
procedural, to explain what needs to be done would be the ability to infinitely multitask.
56 www.linuxvoice.com
REVIEWS BOOKS
The Open Organization ALSO RELEASED…
Graham Morrison is looking for a job at Red Hat.
A
few years ago, our esteemed
colleague and Friend Of Linux
Voice, Jonathan Roberts, was
writing a feature about Red Hat. He was
sitting at his desk, silently taking notes
from a phone call, occasionally interrupting
to ask a question or two. When the call All proceeds from
was over and we all wanted to know what If you can’t afford
the sale of The Open
had kept him enthralled for so long, he said Virgin Galactic, do
Organization will be
it yourself!
he’d been on the phone to Jim Whitehurst. donated to the EFF.
“Jim Whitehurst, CEO of Red Hat?!” we ‘d DIY Comms & Control for
half asked, half shouted. We can’t help but imagine what the world Amateur Space
“Yes”, he said. “That Jim Whitehurst. He’s of business would be like if other CEOs We’ve had some radio components sitting in a
an awesome guy.” had a similar attitude and vision towards Tupperware box for some time, so we’re
interested in the theory this book promises to
And here is a book by the very same success and innovation.
offer on creating an uplink and a data
Jim; eloquent, patient and readable. No download station for our space empire.
other CEO from any other company could LINUX VOICE VERDICT
have written it. It’s about how Red Hat has Author Jim Whitehurst
become incredibly successful while Publisher Harvard Business Review
always doing things the open source way. ISBN 978-1625275271
Price £19.73
“The best idea wins regardless of whether
Could only be improved if Jim donated the
the idea comes from the top executive or book’s proceeds to the EFF. Oh, he does.
a summer intern,” as Jim puts it. And it’s
this ethos that he’s put into his leadership.
The Unity
games engine
Learning Apache Cassandra is free to use.
Ben Everard is looking for work at either Apple, CERN, IBM or Netflix. Getting Started with Unity 5
The Unity programmer’s games engine, used by
A
many of the best games, is now free. And,
pache’s Cassandra database is drumroll, available for Linux. So if you’ve
designed to scale horizontally. always wanted to get into games development,
That means you don’t just have a now is the perfect time. All you now need is a
single database, you have five, or ten, or as book on getting started with Unity.
many as you like all managing the same
pool of data. If you need more space or
performance, you just get another server
and add it to the collection. Cassandra
manages this through a combination
of clever coding and (we’re pretty sure)
black magic. The important thing from a
user’s perspective isn’t how it works, but
how to get it to work, and it’s this process
that Matthew Brown looks at in Learning
Apache Cassandra. Cassandra was developed by Facebook. The eyes
Brown takes the use through the have it.
CQL query language, which is similar to LINUX VOICE VERDICT Raspberry Pi Computer Vision
Structured Query Language (SQL), but Author Matthew Brown Programming
different enough to cause problems for Publisher Packt Publishing If Andrew Conway’s awesome tutorial on
Price £27.99 infrared imaging with the Raspberry Pi (see
the uninitiated. As you work through the
ISBN 978-1783989201 p92) has whetted your appetite for more vision
book, you gradually build up a real-world
A great book for anyone switching from a projects, here’s a whole book’s worth that takes
application that demonstrates the power relational database to Cassandra. the same ideas further.
of Cassandra, and the issues you face
using it.
www.linuxvoice.com 57
GROUP TEST RASPBERRY PI 2 DISTROS
RASPBERRY PI 2 GROUP TEST
DISTROS Since you can now use the Raspberry Pi 2 as an everyday
desktop, Mayank Sharma needs a distro that fits the fill.
On test Raspberry Pi 2 distros
T
he original Raspberry Pi Pi 2-ready versions of their wares
Rasbian struck a chord with anyone that take advantage of the extended
URL www.raspberrypi.org/downloads who wanted a tiny little hardware on the device.
VERSION 2015-05-05 device that had enough juice for a We already know that this will
DESKTOP LXDE specialised task. Thanks to desktop be the first Pi that’ll be supported
Can the reigning champion maintain its distributions optimised for the Pi, in by official releases from both
winning streak on the new Pi 2?
particular Raspbian, you could also Microsoft and Canonical,
use the Pi as an underpowered although their releases might not
Ubuntu Mate desktop. But with the shiny new be what you expect. So in this
URL www.ubuntu-mate.org/raspberry-pi Raspberry Pi 2, the device for the group test we’ll take stock of the
VERSION 15.04 hobbyist has broken into the available distro options. Instead
DESKTOP Mate mainstream. With a quad-core of specialised builds, we’re on the
Will it extend its empire on to the mini PC processor and 1GB of RAM, the lookout for a distro that lets us use
as well? new version has the right kind of the Pi as a general purpose desktop
components and physical and extends all the benefits we’d
SARPi2 resources to outpace some
full-sized desktops produced in the
expect from a regular desktop Linux
distribution.
URL http://rpi2.fatdog.eu
last decade or so.
VERSION 13Mar15
DESKTOP KDE/Xfce However, the new Raspberry Absent friends
How does the grand-daddy of Linux Pi 2 uses a processor based on a The one omission we regret is
distros perform on the PYT? different ARM architecture than the Pidora, the Fedora flavoured distro
original Pi. Among other things, this for the Pi. Unfortunately the project
change means that you can’t use has lost steam over the course and
Arch Linux distros designed for the original Pi isn’t yet available for the Pi 2. We’ll
URL archlinuxarm.org/platforms/ on the new Pi 2 straight out of the also be leaving out RiscOS, which
armv7/broadcom/raspberry-pi-2 box. Since its release some months despite being an excellent OS, isn’t
VERSION NA
back, distros that target the Pi have Linux and might be unfamiliar to
DESKTOP NA
been working hard to put out many of our younger readers.
One of the most flexible Linux distros.
DietPi “The Pi 2 will be supported by official
URL http://fuzon.co.uk/phpbb/ releases from Microsoft and Canonical.”
viewtopic.php?f=8&t=6
VERSION 52
DESKTOP NA
Has it managed to shed the excess? Windows 10 on the Pi 2
When Eben Upton released the Pi 2 he wasn’t what many thought it to be.
Minibian also announced that the Raspberry Pi
Foundation would be collaborating with
Getting Windows 10 to work on the Pi is
part of Microsoft’s program for Internet
URL https://minibianpi.wordpress.com Microsoft to get Windows 10 on the new of Things [IoT] devices. The Pi version of
VERSION 2015-02-18 device. This is possibly because of Windows 10, which will be released at
DESKTOP NA Microsoft’s work on Windows RT for an as-yet unspecified time in the future,
Is this the best of cholesterol-free devices that runs ARMv7, such as the is meant for developing IoT apps and
one that now powers the new Pi. While will probably just boot to a command
distros for the Pi?
the announcement broke the internet, it line interface.
58 www.linuxvoice.com
RASPBERRY PI 2 DISTROS GROUP TEST
Ubuntu Snappy
Canonical wants a share of the Pi as well.
T
he other big announcement that craft custom images for specific needs or use the apt-get package management
accompanied the debut of for containers such as Docker. system. Instead it manages packages
Raspberry Pi 2 was the official Keeping in mind its intended goal and with the new containerised system, which
support from Canonical. However, just like purpose, Snappy is also conceptually Canonical claims to be more “snappier” and
the Windows 10 version, the official Ubuntu different from the other Ubuntu releases. gives the distro its name. The distro is still
release for the RPi 2, dubbed Ubuntu One of the design goals of the distro is to under active development and supports a
Snappy, isn’t a full-fledged desktop but keep the various apps isolated from one limited number of packages, which you can
rather a minimal server image. Ubuntu another. This means that in addition to interact with either via the command line
Snappy isn’t a desktop distro and is instead missing a graphical desktop, the biggest or the custom-built web-based package
designed for developers to enable them to difference in Ubuntu Snappy is that it doesn’t manager called WebDM.
SARPi2
For the Linux aficionados.
S
ARPi is the name of the sub-project
that produces the ARM port of the
Slackware distro. Just like its desktop
sibling, Slackware on the Pi is an acquired
taste that won’t appeal to everyone.
Unlike most other Pi distros, SARPi2 has a
very involved installation process. It involves
downloading a small boot image along with
an optional set of packages for a complete
network-less install. While for most distros
the tricky bit is booting the Pi after you’ve
transferred the disk image to the SD card,
for SARPi this is just the beginning. The
installation process involves laboriously Installing the full distro requires about 8GB of disk space, so you may need to prune the packages.
navigating an ncurses-based menu, defining
partitions, selecting packages, configuring users among us is the availability of detailed We also got SARPi2 to work with the
the network and mounting an external installation guides and other documentation. MicroNEXT wireless adapter and the RPi
source of packages, which takes about an The SARPi2 website hosts an illustrated camera module. However, unlike with other
hour to install depending on the source of guide that meticulously tracks the distros, getting anything to work is a chore
the packages and the speed of your card. installation process and helps you sail for a non Slackware users. Despite Linux
Then you go through another round of through the installation without any issues. inherently being more involved than other
steps manually defining the nitty gritties Also, if you need handholding there are operating systems, there are several things
of the distribution. And that’s just the active Slackware forums on websites such that users of mainstream distros take for
installation. Maintaining and administering as LinuxQuestions.org. granted. A task such as enabling the camera
the distro involves further geekery on the By default, SARPi2 installs the KDE module, which can be done with a single
CLI. For example, you’ll have to edit the desktop, but you can replace it with the keystroke using the raspi-config tool under
mirrors file and uncomment the entry for lightweight Xfce desktop. However, in our Raspbian, involves multiple trips to the
the mirror you wish to use before you can tests, not installing the KDE desktop breaks forums, loading modules, installing utilities,
update or install packages. the Xfce desktop as some tools such as the and editing files on SARPi2. That’s not a
wpa_gui insist on the presence of KDE utils criticism of the distro, but a reflection of how
Down the rabbit hole like kdesu. Also, the distro doesn’t give an good SARPi2 is at aping the behaviour of its
While this level of involvement might seem estimate of the installation size after you’re desktop cousin.
masochistic to most, Slackware users done customising the list of packages you
would have it no other way. Slackware wish to install. The default Xfce desktop
doesn’t make choices on behalf of its users includes apps such as Gimp, Pidgin, Xchat, VERDICT
Designed for Slackware
and SARPi2 follows the simplicity-in-system- GFTP, MPlayer and various Xfce utilities. The users – keep this one
design principle of the desktop version to the distro lacks a graphical browser, though you away from beginners.
letter. The good news for the non-Slackware can fetch one using the package manager.
www.linuxvoice.com 59
GROUP TEST RASPBERRY PI 2 DISTROS
Arch Linux
DIY on the RPi.
T
he venerable Arch Linux One of the best tools in Arch is its
distribution has impressive Pacman package manager. With the CLI
support for the ARM platform package manager you can assemble
and has been running on the Pi for as a fully functional desktop in under an
long as Raspbian itself. However, pretty hour. Yes, that might be 60 minutes
much like the Slackware-based SARPi2 more than the average desktop user
distro, Arch on the Pi sticks to the would be willing to invest in setting up
design principle of its desktop version a desktop, but it enables you to cut the
and puts the user in charge of building bloat on the installation. For example,
their own OS virtually from scratch. setting up a Mate-based desktop with
Arch Linux on the Pi isn’t meant for the usual collection of apps took us just You can implement the advice on the Arch Wiki for the
the average desktop user. You wouldn’t over an hour. But for our troubles we desktop version on the Pi installation as well.
even be able to transfer its image got a streamlined desktop that’s quick
onto the MicroSD card without Linux. off the heels and boots into the desktop again, enabling the support for these
Furthermore, you even have to partition in about 15 seconds. peripherals is a more involved process
the memory card yourself and manually Arch Linux also supports the Pi- than on Raspbian. However, one of the
copy the filesystem from the command specific functions including the camera strengths of Arch is its documentation
line. And that just gets you to the Arch module and the GPIO pins. But once and the distro’s DIY nature chimes with
command line interface with nothing the Raspberry Pi’s education ethos.
more than a bare-bones system
with a kernel. From here you have to “Arch Linux on the Raspberry VERDICT
laboriously build your system from the Pi isn’t meant for the average Another distro that
desktop Linux user.”
requires familiarity with
ground up. But just like Slackware, Arch its desktop version.
Linux would have it no other way.
Minibian
Raspbian from scratch.
T
he idea behind Minibian is to The stock Minibian image boots
create a minimal distro image up to the login prompt in about 15
with a small footprint that’s fully seconds. It ships with the DHCP
compatible with the official Raspbian daemon and the SSH server, so you can
distro, using the same underpinnings manage it remotely via the Ethernet.
as its latest release. Instead of stripping Since it uses the same mirrors as
away unnecessary components from Raspbian, you can apt-get useful scripts
the main distro, the developer of such as raspi-config and rpi-update,
Minibian assembles this distro from the though it’d be nice to have these useful
ground up by pulling in packages from scripts pre-installed. Once installed
the main Raspbian repository. you can use these scripts to enable Minibian isn’t designed for the average desktop user, but
According to the project’s website the camera module and work with the can be converted into one with a few keystrokes.
Minibian is designed for embedded GPIO pins. You can also use the mirrors
projects and makes available the to install the necessary bits to get Wi-Fi install. While Minibian is a wonderful
maximum amount of the physical to work or stuff the distro with graphical little distro, you’ll have to weigh it along
resources on the Pi. In our tests, a apps or even a full-fledged desktop. with its biggest competitor: the similarly
fresh install took slightly over 300MB But that’s not the use-case its positioned DietPi distro, which offers
on the MicroSD card and spared over developer had in mind. Minibian is a more convenient environment for
900MB of RAM. Even after installing meant for anyone familiar with Debian fleshing out the base distro.
the LXDE desktop, and a handful of who wants to use the Pi as a dedicated
graphical apps including the Iceweasel server. It’s ideal for running security- VERDICT
Pruned version of
web browser and the Synaptic package related web apps, since there are no Raspbian that can be
manager, the distro used only about unnecessary daemons and services used as a regular desktop.
111MB of RAM. besides the one that you choose to
60 www.linuxvoice.com
RASPBERRY PI 2 DISTROS GROUP TEST
DietPi Specialised builds
Is sugar-free any good?
Bespoke distributions for
your Raspberry Pi.
I
n addition to the mainstream distros
tested in this feature, there are several
purpose-built distros available for the
Raspberry Pi 2 as well. You can use the HDMI
port on the Pi to connect it to your HDTV and
use it to power your home theatre. Distros like
OpenELEC and the upcoming OSMC (Open
Source Media Centre) wrap the popular Kodi
media player into dedicated ready-to-use
home theatre PC (HTPC) appliance. Using
these distros you can easily move media
inside your Pi-powered HTPC and control
playback remotely.
Sticking with multimedia, you can turn
your Pi into the ultimate jukebox with the Pi
DietPi has a software installation tool, and you can also use the custom script to install MusicBox distro. The distro can handle all
additional software and resources such as a bunch of GPIO projects. sorts of media files stored locally and over
the network and can also fetch music from
O
ne of the most popular uses of the wizard to quiz you about some aspects streaming services like Spotify, Google Play
Raspberry Pi is as an always-on about the installation. The most crucial Music, SoundCloud, Last.FM and more. The
and efficient standalone server. information it seeks is whether you’d like distro interfaces with various desktop and
The DietPi distro installs the bare to use a USB drive with the installation. If mobile clients that you can use to control
minimum components you need to flesh you decide to skip this you won’t be able playback. Furthermore, Pi MusicBox can also
out the installation according to your to configure one later! This is an extremely convert the Pi into a DLNA compatible device
needs. It isn’t the only distro that does so. weird limitation, but at least the screen that can stream music from other DLNA
But unlike others, DietPi goes one step gives you verbose feedback. devices (DNLA is a manufacturers’ ‘standard’
further and offers a nice menu to help you Once you’ve configured the distro, for sharing data over a home network).
pick and choose a functionality for the Pi. you’re taken to the software selection If you’re a fan of retro gaming, RetroPie
For example, using DietPi’s custom screen. You can exit the tool at this point, will give you access to every open source
package management script you can turn which will only install the bare minimum gaming emulator on the planet and includes
the base installation into a filesharing base along with DietPi’s custom tools, drivers to let you hook up modern day
server, a web server, a file server, a VPN using which you can flesh out the gaming controllers. And if you ever need
server, a seed box and even into an installation later. At this point you can use storage space, get hold of a couple of large
LXDE-based desktop. the distro’s pre-installed SSH server to log capacity disks and hook them to the Pi,
The distro ships in a 7zip-compressed into the installation remotely. which you can then use as a power-efficient
archive, so Linux users will need to grab network attached storage device with the
p7zip from the repos of their distros to Easy to build on OpenMediaVault distro.
extract the image file. Another good thing In addition to the helpful software
about DietPi is that it lets you tweak its installation tool, DietPi includes a custom
configuration by editing a text file before configuration script for managing various
you boot the Pi with it. So if you plan to aspects of the Raspberry Pi and the
use a Wi-Fi adapter with your installation, connected hardware. You can use the
enter the SSID of your Wi-Fi network and script to overclock the Pi, change the
its password in the distro’s configuration resolution, mount remote shares, and
file after writing the image. even enable the Pi camera module. It also
When you boot from the card, the distro includes a tool for benchmarking the
will automatically resize itself to take over performance of the Micro SD card and
all the free space on the card and then any connected USB drives.
check for updates as soon as you log in
for the first time. This is good, as it takes VERDICT
The best minuscule distro
care of the two most important aspects with the right kind of tools
of using the Raspberry Pi as a server of for easily fleshing it out. Set up a home NAS with OpenMediaVault.
any sort. It then launches an installation
www.linuxvoice.com 61
GROUP TEST RASPBERRY PI 2 DISTROS
Raspbian vs Ubuntu
The fight of the century.
W
ith its first release in 2012,
Raspbian is one of the
oldest distros for the
Raspberry Pi that runs on both the
original Pi and the Pi 2. On the other
hand, Ubuntu is a greenhorn making its
debut on the tiny PC thanks to its
ARMv7 chip.
Due to its age-old support and strong
foundation, Raspbian is recognised
as the recommended distro for the
Pi. It’s also the easier to install of the
two thanks to it being part of the
Pi’s NOOBS installation mechanism.
Raspbian uses the LXDE desktop and
its default selection of a handful of
apps is tailored for young audiences,
particularly those who want to hone
their programming skills. There’s Sonic Raspbian boots to the desktop in about 21 seconds and leaves about 748MB RAM.
Pi, Scratch, Minecraft, Wolfram Language
& Mathematica along with interpreters enabling the camera. On first boot, lightweight apps as well. What’s
for both Python 2 and 3. Raspbian launches the script to enable surprising though is the inclusion of
While these apps make it a wonderful you to change the password for the feature-rich mainstream apps such
starter kit for educational purposes, it default user and expand the distro to as Pidgin, Thunderbird, Rhythmbox,
fails miserably as a regular desktop. take over the entire card among other VLC, Firefox and even LibreOffice! The
The included web browser is good for things. If you’re willing to put in some inclusion of LibreOffice might seems
reading HTML documentation but can time you can transform Raspbian for asinine at first (it did to us) but the fact
do little else as it ships with no plugins. everyday use thanks to the gazillions of that it starts up in under 10 seconds
Three apps that are of note are the apps at your disposal via its mirrors. is a testament to the Pi 2’s processing
graphical app for configuring wireless superiority over its predecessor.
adapters, the Pi Store client that pulls in Hiya Mate! Software management is handled by
apps from store.raspberrypi.com and In contrast to the blandish Raspbian, Ubuntu Software Centre, which, like the
the ncurses-based raspi-config script. Ubuntu Mate boots into a desktop other heavyweights, performs well.
You can think of raspi-config as the that’s chock full of apps. As its name But all these apps take a toll on the
BIOS for the Raspberry Pi. It helps suggests, the distro is based on the distro’s boot times (about 50 seconds
you tweak the hardware of the Pi; for Gnome 2-inspired Mate desktop and to boot to the desktop). Ubuntu also
example, changing its clock speed and includes quite a few of its default lacks the helpful raspi-config script, so
you have to configure extras manually.
For example, you can use the camera
module on Ubuntu after appending
a couple of lines in the config.txt file.
Also, GPIO works out of the box
Unlike other distros, Ubuntu Mate
includes a four-step installation wizard
that helps you create a user account.
The distro also picks up the attached
Wi-Fi adapter, though you’ll have to
install the OpenSSH server if you want
to manage the installation remotely.
VERDICT
RASPBIAN The UBUNTU MATE
recommended distro Chock-full of apps and
for the Pi is a good performs admirably
For a smooth video playback experience, either purchase the hardware accelerated desktop distro. well.
plugins from the Raspberry Pi store or use the included OMXPlayer.
62 www.linuxvoice.com
RASPBERRY PI 2 DISTROS GROUP TEST
OUR VERDICT
Raspberry Pi 2 Distros
W
For a speed boost, take advantage of Ubuntu Mate’s chassis and
e aren’t kidding when we such as a MiniDLNA server or a install a lighter desktop environment such as lubuntu-desktop.
say that we can find a seed box.
use for each one of
these distros. SARPi, Arch, DietPi It’s about the desktop, stupid
1st Ubuntu Mate
Version 15.04 Desktop Mate
and Minibian are all excellent However, this group test is about
choices for building headless finding a regular desktop distro that www.ubuntu-mate.org/raspberry-pi
servers, depending on your takes advantage of the pumped-up The best distro for using the Pi as a full fledged regular desktop.
familiarity with their respective base Raspberry Pi 2. Raspbian has done
distros. These distributions ship a commendable job as a desktop 2nd Raspbian
with a bare minimum base and give distro for the original Pi, which is Version 2015-05-05 Desktop LXDE
you complete autonomy over their also why it’s the recommended
package management. You can flavour. That said, we’d like to award www.raspberrypi.org/downloads
The Pi Foundation’s recommended distro is a wonderful Linux OS
use them to take full advantage of this test to the latest entrant on the
for educational purposes.
the Pi’s minuscule physical Raspberry Pi’s download page,
dimension to build a low footprint
server that’ll fit anywhere.
Ubuntu Mate. The distro has just
had its first release on the platform,
3rd DietPi
Furthermore, since SARPi and but it comes from a strong lineage Version 52 Desktop NA
http://fuzon.co.uk/phpbb/viewtopic.php?f=8&t=6
“Ubuntu Mate is the sincerest attampt to The best option for rolling out servers and network services over
ship a ready-to-use desktop for the Pi.”
the Raspberry Pi.
4th Arch Linux
Arch mimic their desktop variants, and is backed by an official Ubuntu Version NA Desktop NA
you can also use them to learn and spin. It also lacks the convenience
understand the workings of the of the raspi-config script, but all the http://archlinuxarm.org/platforms/armv7/broadcom/raspberry-
pi-2
geekier Linux distributions without features of the script are available
One of the two distros that’ll appeal to existing users of its
exposing them to your regular as long as you know where to look; desktop flavour.
desktop. plus, a version of raspi-config is on
Between the Debian-based mini the project’s to-do list, and knowing 5th SARPi2
distros, Minibian and DietPi, we the rate at which Ubuntu works you
Version 13Mar15 Desktop KDE/Xfce
favour the latter for its software may find this feature implemented
management scripts. They are easy by the time you read this. http://rpi2.fatdog.eu
to use and help lower the entry Ubuntu Mate on the Pi 2 is the Its installation process is just too involved to appeal to everyday
barrier making it possible for even sincerest attempt to ship a ready- desktop users.
an inexperienced user to transform to-use desktop distro for the Pi. The
the Pi into a server without messing distro doesn’t require a trip to the 6th Minibian
with the CLI. This isn’t how you’d package repository and can be put Version 2015-02-18 Desktop NA
ideally set up a web server for your to use straight after its desktop-
https://minibianpi.wordpress.com
company, but is ideal for deploying style installation, which earns it the
A minuscule distro that offers no incentives over its closest rival.
bite-sized home-based projects top spot.
Installation Desktop Based On Pi models supported
Raspbian Simplest LXDE Debian Wheezy Pi 1, Pi 2
Ubuntu Mate Straightforward Mate Ubuntu 15.01 Pi 2
SARPi Complex KDE/Xfce Slackware Pi 1, Pi 2
Arch Linux Complex - Arch Linux Pi 1, Pi 2
Minibian Involved - Raspbian Pi 1, Pi 2
DietPi Straightforward - Raspbian Pi 1, Pi 2
www.linuxvoice.com 63
SUBSCRIBE
SUBSCRIBE shop.linuxvoice.com
Introducing Linux Voice,
the magazine that:
Gives 50% of its profits
back to Free Software
Licenses its content
CC-BY-SA within 9 months
12-month subs prices
UK – £55
Europe – £85
US/Canada – £95
ROW – £99
7-month subs prices DIGITAL
UK – £38 SUBSCRIPTION
Europe – £53
US/Canada – £57
ONLY £38
ROW – £60
Get 114 pages Access our Save money on
of tutorials, rapidly growing the shop price
features, interviews back-issues archive and get each issue
and reviews – all DRM-free and delivered to
every month ready to download your door
Payment is in Pounds Sterling. 12-month subscribers will receive 12 issues of Linux Voice a year. 7-month
subscribers will receive 7 issue of Linux Voice. If you are dissatisfied in any way you can write to us to cancel your
subscription at subscriptions@linuxvoice.com and we will refund you for all unmailed issues.
64 www.linuxvoice.com
NEXT MONTH
NEXT MONTH IN
ON SALE EVEN MORE AWESOME!
THURSDAY
Robots!
30 JULY Ben has been locked
in the shed with his
soldering iron for a
while. We’re not sure
what he’s building,
but the shareholders
of Boston Dynamics
are looking worried…
Inside ORG
The Open Rights
Group do good
things on our behalf;
lobbying politicians,
campaigning and
fighting the good
fight. Here’s what
they’re up to now.
Drupal
GEEK UP YOUR SUMMER If your website is
anything more than
a static HTML page,
Enhance your summer with our clutch of you probably need
a content
things to make and do – all powered by management system
Linux and Free Software, of course – and Drupal is one
of the best there is.
LINUX VOICE IS BROUGHT TO YOU BY
Editor Graham Morrison Editorial consultant Nick Veitch through the use of advice in this magazine. Copyright Linux is a trademark of Linus
graham@linuxvoice.com nick@linuxvoice.com Experiment with Linux at your own risk! Torvalds, and is used with permission.
Deputy editor Andrew Gregory Distributed by Marketforce (UK) Ltd, Blue Anything in this magazine may not be
andrew@linuxvoice.com All code printed in this magazine is licensed Fin Building, 110 Southwark Street, London, reproduced without permission of the editor,
Technical editor Ben Everard under the GNU GPLv3 SE1 0SU until March 2016 when all content (including
ben@linuxvoice.com Tel: +44 (0) 20 3148 3300 our images) is re-licensed CC-BY-SA.
Editor at large Mike Saunders Printed in the UK by ©Linux Voice Ltd 2014
mike@linuxvoice.com Acorn Web Offset Ltd Circulation Marketing by Intermedia Brand ISSN 2054-3778
Creative director Stacey Black Marketing Ltd, registered office North Quay
stacey@linuxvoice.com Disclaimer We accept no liability for any House, Sutton Harbour, Plymouth PL4 0RA Subscribe: shop.linuxvoice.com
loss of data or damage to your hardware Tel: 01737 852166 subscriptions@linuxvoice.com
www.linuxvoice.com
CORETECHNOLOGY
CORE
Valentine Sinitsyn develops
high-loaded services and
teaches students completely
unrelated subjects. He also has
a KDE developer account that
TECHNOLOGY
he’s never really used. Prise the back off Linux and find out what really makes it tick.
Non-trivial Iptables
You know how to compose basic iptables rules – now take them further with these clever tricks.
L
ike any self-respecting operating and make your host invisible to “ping scans”. comes into play, and you can use it to
system, Linux comes with the built-in This is not recommended though, as ICMP prevent connections from being tracked via
firewall. If this statement makes you is not just a ping workhorse but an essential the NOTRACK target.
think of iptables, or Xtables in general (which protocol for networks to run smoothly. If you What’s conntrack, you ask? It’s a
refers to iptables, ip6tables etc), you’re right. block it blindly, you’re almost certain to run part of Netfilter (officially the Netfilter
However, that’s only the tip of the iceberg. into obscure bugs. connection tracking module) that tracks
Iptables is a userspace tool that relies on an In iptables, rules form chains that are network packets and determines which
in-kernel framework called Netfilter. The grouped into tables (hence the name). connection they are part of. With stream-
latter is what hooks into network subsystem, Packets traverse them in a predefined oriented protocols like TCP it’s relatively
analyses packets as they come in and out order (see the diagram below). If a packet straightforward. However, conntrack
and acts accordingly. matches no rule, the default policy also knows about some application-level
There are numerous good tutorials and (configured per-chain) determines the action protocols like FTP, and is smart enough
howtos on iptables (see http://netfilter.org/ to be taken. You are free to create your own to treat the FTP command and FTP data
documentation), and we’re not going to chains and even nest them, as possible connections as the same logical entity.
repeat them here. What we are going to do actions include switching to another chain Many parts of Linux rely on conntrack,
is to shed some light on lesser known, more or returning to the parent one. The table including iptables own state module:
obscure features you may find useful in real- set is fixed, however, at least if you are not iptables -A INPUT -m state --state NEW -j REJECT
world scenarios. comfortable with kernel hacking. Any new connection to the machine
A firewall is basically a set of rules The most frequently used table is filter: you’ve run this command on (root
containing conditions to select packets (or the iptables command implies it when permissions required) will be banned with
“matches”, in iptables parlance), and actions you leave the -t argument out. There is an ICMP Port Unreachable message. A
(or “targets”) to take on them. Target names also mangle for setting marks (covered connecting party will probably receive a
in iptables are, by convention, uppercase. For shortly), and nat to perform network address “Connection refused” error. All existing
instance, you can DROP (or silently discard) translations. A relative newcomer to this connections should continue, so you won’t
all ICMP type 8 (Echo Request) messages, family, raw is traversed before conntrack shut down your own SSH session, for
Other NF parts Packet flow in Netfilter and general networking
Other networking
Basic set of filtering
opportunities at the INPUT PATH FORWARD PATH OUTPUT PATH
network level
local
Application layer process
Protocol layer By Jan Engelhardt (based in part no clone to
xfrm
clone packet (eg ipsec) xfrm/socket
on Joshua Snyder’s graph). Last AF_PACKET xfrm
decode lookup
updated 2014-Feb-28; Linux 2.6.36+ encode
Network Layer filter
input
* “security” table left out
for brevity
routing
decision
raw
output conntrack
mangle
output
mangle * “nat” table only consulted filter nat reroute
input for “NEW” connections check
taps
(eg AF_PACKET) raw conntrack
routing mangle filter mangle nat xfrm
decision forward forward lookup
This diagram shows how network packets traverse built-in iptables chains and tables. Note that Netfilter is only part of the story.
66 www.linuxvoice.com
CORETECHNOLOGY
instance. Remember though that changing iptables -A INPUT -p tcp --dport 80 -j MARK 0x1
firewall rules over SSH is almost certainly a --set-mark 0x1
bad idea. iptables -A INPUT -m mark --mark 0x1/0xff
Connection tracking not only provides Packet marks are available even if your
states (so this type of firewall is called kernel was compiled without Netfilter
“stateful”). It also maintains arbitrary 32-bit connection tracking support.
integer marks associated with connections.
These are known as “connmarks”. To set a Digging deeper
connmark, do the following: Xtables sports many matchers, including The OpenDPI project was shut down three
iptables -t mangle -A PREROUTING -p tcp --dport 80 third-party extensions. However, most of years ago, but development continues at
-j CONNMARK --set-mark 0x1 them work only on the network packet ntop.org.
The mangle table is a traditional header, and there will be times you’d want
place to set marks. Here, we do it in the to peek into data payload. You may want to The homepage is in Russian, but you should
PREROUTING chain, or prior to routing be sure the packet targeting port 80/tcp is find the download link easily: look for the
decisions. All TCP traffic targeting port 80 really HTTP, or filter DNS requests by names topmost nDPI-something.tar.gz; for now, it
(presumably, HTTP) is assigned connmark they contain. These times, you’ll need Deep is nDPI-1.5.1.r9249.tar.gz. The rXXX part is
0x1. Here’s how to match against the Packet Inspection, or DPI. the nDPI SVN revision that the extension is
connmark: DPI techniques are complex and bundled with.
iptables -A INPUT -m connmark --mark 0x1/0xff -j performance-hungry. You don’t want them Once you have the tarball, unpack it and
ACCEPT unless absolutely necessary, and we’ll cover cd into the ndpi-netfilter directory under
0x1 is the target mark value, 0xff is a some alternatives shortly. However if you top-level nDPI-.... Now, run make: you’ll need
mask. A mask is how you define bits to find yourself looking for a way to block or the kernel and iptables headers installed on
consider when matching; here, only the prioritise Skype, BitTorrent or another tricky your machine. They are usually called linux-
lowest 8 bits are taken into account. Masks protocol that was designed to be hard to headers and iptables-dev in your package
are quite common in Netfilter, and are often firewall, DPI is the answer. manager.
used to effectively combine several marks DPI engines aren’t naively parsing all Wait for the build process to finish. Check
in one. traffic coming through. Instead, they look there were no errors, then copy ipt/libxt_
While you can check marks in iptables, ndpi.so to wherever your system stores
they are mainly useful for advanced routing
or traffic shaping (QoS). However, tools like
“Remember that changing Xtables extensions (usually /lib/xtables
or /usr/lib/iptables). Now, insmod src/xt_
ip or tc can’t work on connmarks directly. firewall rules over SSH is ndpi.ko (as root). If this complains about
Instead, they rely on per-packet marks.
These are different from connmarks, but
almost certainly a bad idea.” an unknown symbol, make sure you’ve
also loaded nf_conntrack and the x_tables
you can synchronise their values with: kernel modules.
iptables -t mangle -A PREROUTING -j CONNMARK for specific traits or signatures that uniquely The extension provides both a match and
--restore-mark identify the protocol. It is similar to how a target. You can see the options available
iptables -t mangle -A POSTROUTING -j CONNMARK antiviruses work, and “false positives” (or with iptables -m npdi --help. Consider
--save-mark misdetection) can happen here as well. piping it to less as an ndpi match provides a
The first command copies a connection Finding a good signature is a tough research command line switch per protocol, and the
mark to the packet, and the second does the problem, that takes time and money to list of supported protocols is quite lengthy.
opposite. solve, so the DPI market is dominated by This is how to block Skype on your router:
It is also perfectly legal to set or check proprietary (and very expensive) solutions. iptables -m ndpi -A FORWARD --skype -j DROP
“plain” marks manually like this: A German-based company named Ipoque Considering possible false positives, you
(www.ipoque.de) once open-sourced a may decide not to block Skype completely
stripped-down version of its Pace DPI engine but force it to use limited bandwidth. This
Debugging aids
and built the OpenDPI project around it. is straightforward: assign a mark instead
As the size of your iptables ruleset grows, Unfortunately, things didn’t go well and the of DROP and use tc for shaping. You may
debugging it may become troublesome. Luckily, OpenDPI project was shut down in 2012. readily convert the rule above to -j MARK,
there is one little tool to make the process
easier. The TRACE target forces Netfilter to log
The good news is that the guys at the ntop but there is better approach.
every rule the packet traverses. This is useful project forked OpenDPI as nDPI and develop The nDPI Xtables module provides an
only if -j TARGET is the first action taken on the it today. It’s available from www.ntop.org/ NDPI target that automatically assigns
packet; that’s probably why TRACK is valid only products/ndpi under LGPLv3. marks or priority to packets according to the
in the raw table. For each match, you get the While nDPI is a userspace library, it is protocol detected. Mark values and masks
table and chain names, and the rule number. If
the packet reaches the end of the chain or the
written in portable C suitable for kernel are stored in /proc/net/xt_ndpi/proto – read
default policy-defined action is taken, you also space execution. There have also been this file to get the current settings in simple
get a note. numerous attempts to wrap it as an Xtables tabbed format. The column named id stores
To use TRACK you’ll also need the ipt_LOG extension. They seem to die and rise from the identifier that nDPI assigns to supported
kernel module loaded. Traces are viewable ashes quite regularly, so finding one that’s protocols; mark and ~mask are mark and
in dmesg and /var/log/kern.log or similar,
depending on your logger settings.
steadily maintained is not trivial. My own mask values (negation ~ is a typo); and
favourite lives at http://devel.aanet.ru/ndpi. the last column contains a short protocol
www.linuxvoice.com 67
CORETECHNOLOGY
this looks familiar? distributed nameserver system, reported
Network packets inside filter tcpdump -i eth0 udp port 53 that it was able to filter 41 billion malicious
In This is how you limit tcpdump, and in fact DNS requests by names they contained
any other libpcap-based program’s output to overnight. Note that this result is obviously
DNS traffic. Internally, libpcap compiles this hardware-dependent.
Queue 1 filter into BPF. How do you make use of BPF filters in
By itself, BPF is an assembler-like your own ruleset, you ask? Don’t be afraid,
Kernel
Userspace language that is executed in a virtual being able to program at assembler level is
machine. There are no backjumps, so the not a strict requirement. Still, for those who
ACCEPT
language isn’t Turing-complete, but this is a find it fun, our ASM School (which started in
guarantee that a BPF program will not loop LV012) provides just enough background to
DROP Our code Final recipient forever. Here’s how a simple BPF program get started.
may look: The trick is to call the high-level syntax
ldh [12] compiler provided by libpcap. Some may
jne #0x806, drop advise you to use tcpdump -ddd, which
ret #-1 prints BPF opcodes, but this doesn’t seem
This is how the Netfilter queue operates: drop: ret #0 to work with newer iptables anymore.
packets are evicted from their normal flow and This loads the EtherType field (byte offset Better to stick with nfbpf_compile, which
re-injected as needed. 12) and compares it against 0x806 (the comes bundled with iptables. The only
ARP protocol type). If the values aren’t equal, inconvenience is that it is built only if iptables
name. Two special identifiers, all and any, execution continues at drop and returns is configured with --enable-bpf-compiler,
are wildcards: any stands for any recognised 0. Each BPF program yields a number and this is not what most Linux distributions
protocol, and all includes unknown ones. of bytes to keep from the packet, so this do. So, you may need to grab the sources
By default, mark values are equal to means discarding it completely. Otherwise, and recompile them yourself.
protocol IDs, but you can write to this file -1 (or unsigned 65535, a maximum packet If you want to filter NTP requests, this is
to change marks. For example, this is how size) is returned, and the packet continues possible with nDPI, but BPF will probably be
you assign the 0xdeadbeaf mark to Skype faster. NTP uses port 123/udp. Moreover,
(protocol ID 7d):
echo ‘7d deadbeaf/ffffffff’ > /proc/net/xt_ndpi/proto
“Berkeley Packet Filters is as per RFC958, bits 2..7 in the packet
payload are set to a value less than four
The ID, mark and mask values must be the de facto standard for (this is basically how nDPI detects NTP,
hexadecimal numbers. We’ll see further
examples of NDPI usage in the later section.
advanced packet filtering.” too). Call xt_bpf as follows:
iptables -A INPUT -m bpf --bytecode “$(nfbpf_
compile RAW ‘udp port 123 and ((udp[8] & 0x38)
Quick filters untruncated (or is accepted). You can find >> 3) <= 4’)” -j LOG
It would be unfair to say that iptables can’t more examples in the bpfc(1) man page, Now all NTP requests will end up in your
work with network packets at byte level. which is a BPF assembler from the netsniff- kernel log.
At least two modules, strings and u32, ng toolkit.
come bundled just for these purposes. With Starting at version 3.0, the Linux kernel Deciding in userspace
strings, you can search for given substring can JIT (Just In Time) compile BPF filters At the very end, you may want some really
(either ASCII or hexadecimal) in the packet. to native machine code. To enable this convoluted logic, not easily expressible even
u32 sports C-like expressions to check feature, just do echo 1 >/proc/sys/net/core/ in terms of BPF. Wouldn’t it be good to have
byte values at given offsets, and can even bpf_jit_enable. This makes BPF filters really a whole C (or even Python) function that
perform some bit operations like shifts. Both fast beasts. CloudFlare, which provides a decides on a packet’s destiny?
do their jobs well, but are somewhat limited
and don’t account for all possible scenarios.
u32 rules are also not easy to read, unless
you have a trained eye.
The alternative comes in the form of
Berkeley Packet Filters, or BPF. Despite the
name revealing its BSD origins, this is the
de facto standard technology for advanced
packet filtering in Unix. BPF is available to
many operating systems now, including
Linux, of course. The primary design goal
behind BPF was socket-level filtering for
network sniffers such as Wireshark. This was
later extended to seccomp, a Linux-specific
sandboxing technology.
It is quite possible you’ve already used Our toy firewall can block Skype already. Be prepared to click “No” many times, as it will eagerly
BPF filters without even knowing it. Does look for a way out.
68 www.linuxvoice.com
CORETECHNOLOGY
That’s exactly what the Netfilter queue (or
nfqueue for short) is for. It’s like libpcap in nftables: iptables reloaded
that it copies network packets to userspace. iptables were here for more than fifteen years. iptables) are optional, and rule may have more
The difference is that with libpcap you can They are great, but they’re starting to show their than one target. Nftables relies on new in-kernel
only see what’s in the wire, while nfqueue age. The project to redesign iptables is underway, infrastructure that provides optimized data
lets you mangle packets’ headers and and preview releases are available with Linux 3.13 structures (much like ipset) to match packets
and up. That’s what nftables are. faster. For migration path, there is a compatibility
data, accept them and even drop them at
Nftables replace the whole family of tools layer to run iptables or ip6tables on top of nftables
your own discretion. This comes in handy (`iptables`, `ip6tables`, `ebtables` etc) we have infrastructure.
in some cases, and Intrusion Prevention now with only one: `nftables`. The syntax is also nftables look very promising - please drop
Systems (IPS) like Snort or Suricata rely on changed to feel more natural: us a line if you want to know more about them.
this mechanism to implement so called nft add rule filter output ip daddr 1.2.3.4 counter Eventually, they will replace iptables in your
Both tables and chains are now user-definable. favourite Linux flavour, so keep an eye on the
“inline mode”.
Packet counters (sometimes a bottleneck in progress.
The libnetfilter_queue(3) man page
describes many options available with
nfqueue. You can choose to get only packet adds measurable overhead. As a quick test, mark = payload.get_nfmark()
metadata which is faster, if you only need to I tried a single rule that accepts ICMP Echo data = payload.get_data()
check marks, for instance. You can retrieve requests on lo in the kernel (-j ACCEPT) and packet = ip.IP(data)
packets partially or fully, change headers, in userspace with nfqueue. On 32K pings, the # Get dst and dport from packet
data payloads and even metadata (including latter is 2.9 times slower with C code and req_proto = ndpi_proto(mark) if mark else
firewall marks), and you can set your verdict, 3.8 times slower with Python (packets are ‘unknown’
like NF_ACCEPT to accept a packet or fully copied). Your mileage may vary, but you if ask_user(dst, dport, req_proto):
NF_DROP to discard it. Note that there’s no can use these figures as a guide. That’s the payload.set_verdict(nfqueue.NF_ACCEPT)
easy way to accept a packet and continue price to be paid for being able to program else:
the current chain, however. in userspace, and even use a high-level payload.set_verdict(nfqueue.NF_DROP)
To queue packets for userspace, we use language like Python. queue = nfqueue.queue()
the NFQUEUE iptables target. There are 64K For a complete example, let’s write a toy queue.set_callback(handle_packet)
queues available in Linux; --queue-num interactive GUI firewall. These things are queue.fast_open(QUEUE_NUM, AF_INET)
select the one you want. The kernel queues quite popular in Windows. First of all, set up queue.try_run()
up to 1024 packets by default; excess ones iptables as follows: The ndpi_proto() function (not shown
will be discarded silently, or accepted, if the iptables -N userfw here) reads /proc/net/xt_ndpi/proto and
queue was opened with the NFQA_CFG_F_ iptables -A userfw -j NFQUEUE --queue-num 1 gets the protocol name for a ‘mark’ that
FAIL_OPEN flag. Packet loss will also iptables -A OUTPUT -m ndpi --all -j NDPI --ndpi-id nDPI sets for us. Only the first packet
occur if no program processes the queue in --set-mark in a connection reaches our firewall, so
userspace: use --queue-bypass to bypass iptables -A OUTPUT -m state --state NEW -j userfw detection may be inaccurate. We use dpkt
queuing in such cases. In short, be careful Here, we create a userfw chain and pass to parse network protocols; scapy is also a
when working with nfqueue: your code is a all new outgoing traffic through it. Every reasonable choice. You will find complete
part of the Linux networking core. packet in userfw gets queued in queue source at www.linuxvoice.com.
Despite all its advantages, nfqueue is 1. We also use an NDPI target to classify Note this is really a toy, albeit one that
quite slow. First, it needs to copy packets packets before queuing them for userspace. runs with root permissions. Modern
to and from userspace. Netfilter uses All we need now is a Python program that applications make many simultaneous
Netlink sockets (the AF_NETLINK family, reads packets from the queue, and asks the requests. You may not like it, and they may
somewhat like AF_UNIX we discussed back user if it is OK to let them out. A stripped- not like you banning these connections
in LV015) for this purpose. Then, processing down version of this code may look like this: randomly. So, play wisely, and stop
is delayed until the kernel scheduler gives QUEUE_NUM = 1 programs having valuable data before trying
your program a chance to run. All of this def handle_packet(payload): this.
Command of the month: ipset
Quite often, iptables rules contain many IP are separate from rules, so you can update Use bitmaps to store address or port
addresses (think of Fail2ban). You can use a them dynamically, which is faster than ranges. Hashes are good for multiple disjoint
single rule per address, but the more rules reloading the ruleset. Internally, sets may values.
you have, the longer it takes to match use bitmap or hash representations to iptables -A INPUT -m set --match-set fail2ban src -j
packets against all of them. What should facilitate O(1) lookup times. DROP
you do? ipset should be in your distribution’s Here, set is used to drop all packets
ipset comes to rescue. It refers to both repositories. The syntax resembles ip(1) or whose source (src) address is in the
the in-kernel framework and the userspace tc(1): fail2ban set. To block another host, just do
utility to manage possibly disjointed sets ipset create fail2ban hash:ip ipset add fail2ban 5.6.7.8: no rule changes
of addresses and ports effectively. Sets ipset add fail2ban 1.2.3.4 are needed.
www.linuxvoice.com 69
FOSSPICKS
FOSSpicks Sparkling gems and new
releases from the world of
Free and Open Source Software
Our editor Graham Morrison is a fearless explorer of the internet – look,
he’s found some excellent Free Software on his travels!
Music player
Tomahawk 0.8.99
F
rom the minimalism of Google Play collection, and you can
XMMS to the lyric-wielding, play and build playlists with these
Wikipedia-reading Amarok, files and your local files.
music players all fundamentally What’s more impressive is that if
work the same way by playing your you search for an artist or a piece of
local music. Even if some add music, Tomahawk will scrape
online music services or streaming through all of your configured
internet radio, this is often as an sources, enabling you to construct
afterthought. But not with playlists and albums from more
Tomahawk. Tomahawk is trying to than one source. Tomahawk doesn’t
be a different kind of music player, make any distinction between
and it’s very much a product of our sources, making it a fascinating
modern age of interconnected, way to access and discover new
music streaming social networks. music. You can also share access
Its main difference is that while it to your music across the local LAN,
can (and does) scan your local or with your contacts through
music collection, it includes plugins Google and Jabber.
to access over 20 online music Interoperability, aggregation and cross-platform parity are the
sources, including Spotify, Google Groovy potential best reasons to use Tomahawk. There’s even an Android version.
Play Music, Jamendo, SoundCloud, The latest stable version was
Ampache (OwnCloud), YouTube released in April (0.8.3). There are another major update. The
and Beats. charts and new releases from developers have migrated to Qt 5
The Google Play Music plugin, for multiple sources, including iTunes, from Qt 4 and added more
instance, operates as an expansion Metacritic and Rovi. Clicking on a resolvers, including (an untested)
of your local music files. If you’ve release will fill in the music sources one for Amazon Music.
synced your collection of Leonard from your enabled resolvers, so it There’s no automatic import of
Cohen’s albums to Google Play (a won’t always work. The version your Spotify collection, but you can
service that’s free for up to 50,000 we’re using is a release candidate drag and drop playlists from the
songs), these will appear as your for version 0.9.0, which is itself main Spotify application. We’d like
more granular control over which
sources are prioritised, and whether
covers and live versions are
returned from a search, as these
options are only available when
configuring a source. We’d also like
to see more aggressive caching of
your pre-configured playlists and
searches. But these are small points
when the application itself is doing
You can view and play
so many new things. If you enjoy
chart music from all
over the world, thanks music, you need to check it out.
to iTunes. Here’s the
PROJECT WEBSITE
charts from New
www.tomahawk-player.org
Zealand, for instance.
70 www.linuxvoice.com
FOSSPICKS
Screen colour adjuster
Blueshift 1.90.1
W
e published a brief can set your location, for example,
tutorial on a wonderful as well as the specific colour
utility called Redshift in temperature you need from the
issue 14. Redshift adjusts the colour screen. You can also push hues into
temperature and intensity of your the blue frequency range, as hinted
screen to better reflect the at by Blueshift’s name, but we’re not
diminishing light of an evening or sure that this encourages
the darkness of night. This makes concentration (as implied by the
your screen easier on your eyes and developers).
your brain as there’s less contrast Blueshift is also a super-powered
and the hues are closer to dimmed version of Redshift, and that power
The only feature we
light than the Mercury-vapour lamp comes from its configuration files, off to bed. Another example makes
miss from the original
white of the typical display. It helps which give you complete control Redshift is the option to the light curves logarithmic, rather
many of us nocturnal workers get a over how colour transitions are get your location from than linear. The ‘xmonad’ config file
better night’s sleep. handled. One example your IP address. uses this window manager to map
It’s such a useful tool that there configuration is ‘bedtime’, for different light curves to different
are many forks and alternatives, instance. This adjusts the colour workspaces, and there’s a file for
and Blueshift is our favourite. Unlike temperature not by the light outside reversing the colour palette when
a couple of alternatives that attach but by the time you want to head you’re running low on battery power
a GUI configuration panel to – excellent if you’re running the
Redshift (redshift-gtk being one), screen on minimum brightness.
Blueshift is driven purely from the “Blueshift helps us nocturnal
command line. It takes many of the
same arguments as Redshift: - you
workers get a good night’s sleep.” PROJECT WEBSITE
https://github.com/maandree/blueshift
Digital darkroom
LightZone 4.1.0
L
inux has quietly become a competitors for both thumbnail
photographer’s dream studio. rendering and editing.
We now have more photo Rather than an editing process
post-processing and management based on Adobe’s Lightroom,
applications than ever, and they’re LightZone has a large list of ‘Styles’
nearly all brilliant. LightZone is one that can be added to process an
of these; so too are Darktable, image, along with the regular
RawTherapee and AfterShot Pro. sharpen, blur and colour balance
Since March last year, LightZone effects you’d expect. RAW photos
users have been asked to register start off with exposure, noise and LightZone is another great graphical application that was once
to be able download from the main tint controls, and any further styles proprietary and is now a fully fledged open source project.
site. This is presumably to better or processing you add can be
track users, and over 80,000 have moved up and down through the environment. The clone tool is
registered. But as the application processing order. particularly good because unlike
remains open source, most of us brushed cloning in something like
Linux users will simply need to Intuitive Gimp, in LightZone you create a
download the latest major update Hover over a style and you’ll see a blended zone that acts like a portal
from our package managers. preview of what the processes will between the source and the
There are two main modes – do – perfect for choosing one of destination, and it can be moved
browse and edit, and even when the nine black-and-white styles, for after adding it to your edit queue.
dealing with the large RAW files example. There’s even red-eye
produced by our DSLR, LightZone PROJECT WEBSITE
removal and a cloning tool, which
http://lightzoneproject.org
was much quicker than its are rare in a post-processing
www.linuxvoice.com 71
FOSSPICKS
Blogging platform
Ghost 0.6.4
G
host is a blogging platform default, the same minimalism is
that we’re sure you’ve carried forward into the published
already heard about. It’s the page, but Ghost is now well enough
result of a crowdfunding campaign, established that there are hundreds
and the team have just published of themes available.
an in-depth look at where their The software needs to run on a
$300,000 has gone in the two years server, and you can pay someone to
since it launched. run the server and host your blog,
What makes Ghost great is its or you can download and install the
simplicity. You enter your thoughts files yourself. The latest small
using the hipster’s language of update fixes more issues with the
choice, Markdown, and your text major 0.6.0 update, which appeared
and layout are updated in real time in mid-April. This included some
over on the right-hand side of the functionality that had held us back The Ghost blogging platform is already two years old, and it’s
window. Paragraphs, line breaks, from committing more time to come a long way from those early versions with little support.
emphasis, links, code and images Ghost, such as a spellchecker,
are all handled quickly and easily, mobile uploads and code injection, the overall user-experience, which is
without any of the open-and-close- plus lots of new API hooks that will what’s most important with Ghost.
element hassle you get with HTML. help developers without diminishing If you’re looking at writing a blog,
The simplicity and the use of we’d highly recommend you take a
markdown means that there’s very look at Ghost first.
little distraction between your “What makes Ghost a great
thoughts and the published story.
And the output looks fantastic. By
blogging engine is its simplicity.” PROJECT WEBSITE
https://ghost.org
Arch package manager
Octopi 0.7.0
I
n Arch distro land, we’re quite The main Octopi window is split
happy using the command line into three: one panel lists packages;
for most day-to-day uses of another Arch’s metagroups; while
Pacman (Arch’s package manager) the final panel is a tabbed view that
and Yaourt (its accompanying switches between specific package
package manager for the user information, the files it installs, and
repository). But sometimes we long what happens when you do.
for a decent graphical interface to An essential addendum here is
all those results and dependencies. the latest news from Arch itself,
This is perhaps why there are which is a prerequisite before any
several graphical interfaces to Arch distro upgrade you might perform.
package management, despite its There’s also a page on general
Octopi also includes a
hacker credentials, and Octopi is the application usage. Through the notifier that can be set packages need updating. Outdated
latest. The best thing about Octopi main window, there’s a neat series to sync the database at packages get their own lists and a
is that it works everywhere. It works of icons that quickly inform you if a an automatic interval numbered reminder so you don’t
well on KDE 3, 4 and 5 desktops, package is part of the main for updates. leave it too long before performing
LXDE, Mate, Trinity and Xfce, and repository or you’ve installed it an update. We like the visual style
across Arch-based distributions like yourself, and whether those very much. The small alien icons
ArchBang, Chakra, KaOS and aren’t over the top, and everything is
Manjaro. Built on Qt 5, it’s also very beautifully functional.
frugal with resources and we found “Sometimes we long for a graphical
database updates and refreshes as
fast as the command line.
interface for our Arch packages.” PROJECT WEBSITE
https://octopiproject.wordpress.com
72 www.linuxvoice.com
FOSSPICKS
Office for Android
LibreOffice Viewer for Android 5 alpha 1
L
ibreOffice Viewer is published even complex documents without
by the same people behind difficulty. Performance on our
LibreOffice – The Document Nexus 5 was good, enabling us to
Foundation – and it’s an important zoom around pages and skip
part of its strategy for getting open through slides with very little delay.
standards into the hands of as But the new and experimental
many people as possible, as well as feature that makes this release
keeping up with similar offerings significant is the ability to edit
from proprietary vendors. It’s documents. You need to enable this
available through Google Play and first in the settings panel, and after
as an APK that you can download doing so, a cursor appears on what
and install yourself. It should also was previously a passive viewer.
LibreOffice Viewer is
be available from the open source You can then move the cursor, type being done by Collabora, and one of
now more than a simple
F-Droid repository by the time you and make changes with the toolbar, viewer – with the new the side-effects of creating a
read this. just as you would on the desktop. It version, you can finally rendering engine that edits, works
It loads and views all the Open does need some refinement, but it edit documents. without X.org, and scales for
Document formats (.odt, .ods and works, and we can’t wait to see this Android devices is that it’s leading
.odp) as well as Microsoft’s .docx, . feature become more stable. This to the development of the browser
xlsx and .pptx formats, and it’s a development is thanks to the work version, which should also be
great reader for files you might keep available later this year.
on your phone. We tested it with our
own selection of documents and “You can move the cursor, type and
found the viewer to be just as good
as the desktop version, rendering
make changes with the toolbar.” PROJECT WEBSITE
www.libreoffice.org
CAD modeller
Antimony (Git revision)
A
ntimony is a tool for The great thing about applying
Computer Aided Design, this to Computer Aided Design is
but it’s like nothing we’ve that you’re forced to use
seen before. Instead of interactive mathematical models and
point-and-click modelling using constructs, which is exactly what
primitives, Antimony builds objects you need if you’re planning to build
by connecting nodes in a graph something. For example, to build a
while still letting you manipulate 3D ring you first add a 2D circle and
values with the mouse. It’s a little then connect the output of this to a
like a using a modular synthesizer, ‘revolve’ function from the
only instead of constructing sounds Transforms menu. This rotates the
you’re constructing 3D models. circle around the X axis, effectively
There are two main windows. making the circle follow a 360
The first is the graph, which is degree path. There’s also a script
where you add your objects, change interpreter, and you can open any You can have as many
their parameters and link attributes node and start editing the code windows and views powerful as the model output is
together. The second is the 3D view, used to render that specific element open as you need, algorithmically watertight and
which can also be split into front, using a slightly augmented version focusing on whichever perfect for output that’s going to be
bottom, side, back views or any of of Python 3. Even without specific parts of your model you used to produce a physical model.
want them to.
these separately. You can zoom CAD knowledge, Antimony is a lot of
around this view and move objects fun. But if you’re a mathematically PROJECT WEBSITE
around, changing the parameters minded designer, this kind of www.mattkeeter.com/projects/
antimony
back in the graph when you do so. package must be incredibly
www.linuxvoice.com 73
FOSSPICKS
Video editor
Shotcut 15.05
D
espite there being some well
established video editors on
Linux, none have yet been
able to make the process easy and
intuitive for us. Kino was one of the
most successful, however. It was
powerful and capable of great
results, but it was tricky to use.
Unfortunately, Kino ceased to be
developed in 2013. However, Dan
Dennedy, Kino’s lead developer, has
returned and created an all-new
application, Shotcut, built atop his
MLT framework. This means he
doesn’t need to completely
re-invent the wheel when it comes
to processing the video, and he can
spend his effort working on how the
user interface is going to work.
We really like Shotcut. The layout
and workflow is supremely logical,
and you can tell a lot of work has
gone into its design. You import
clips and add video tracks, split
clips and crossfade between them.
The edges of clips on the graphical
timeline can be grabbed to extend
them, and the cursor scrubs along A feature unique to the Linux version is the ability to capture your screen and import directly into Shotcut.
the edit to make finding the right
place to cut as simple as possible. effects, and they give you all you out the processing of those effects
You can drag elements out of the need for most projects. There’s to your GPU, which is the first time
background and into different colour balancing and grading, we’ve seen what’s normally a
areas, or leave them as floating rotation, overlays, glow and opacity, professional addition in a piece of
windows, making it easy to create a for example, and creative effects open source for Linux.
layout that works for you. include some excellent 3D text,
There’s a small group of well sepia tones and waves. Best of all, a PROJECT WEBSITE
www.shotcut.org
implemented audio and video new beta feature allows you to farm
How it works: Image stabilisation
1 Limit the clip Image stabilisation takes its 2 Add the effect Use the video filters tab to add 3 Process the video When the status reads
toll on processing and playback, so start by the stabilise effect and click on Analyse to let ‘Analysis complete’ you can adjust the options.
making your shaky clip as precise as possible. it work through the clip. It saves its output to a file. Render the clip to a new clip to limit the CPU load
74 www.linuxvoice.com
FOSSPICKS
FOSSPICKS Brain Relaxers
https://launchpad.net/pybik/
Interactive fiction
Fizmo 0.8.0 (b4)
M
any years ago, there programming environment that’s
was a humble games still used by interactive fiction
genre where you could ‘aficionados’ today. All you need to
interact with a story by typing in play these games is an interpreter.
commands. The computer might There have been many
output “You are inside a building, interpreters over the years, and our
a well house for a large spring. current favourite is Fizmo. Fizmo is
There are some keys on the still being actively developed and
ground here.” And you might version 0.8.0 is the first to take it
respond by typing, ‘get keys’. By away from the command line and
typing simple commands, into its own SDL window. It remains
moving through locations and absolutely minimal yet supports
solving riddles, you would nearly all of Infocom’s games,
Each year there’s a
become part of the story. including those with sound, and will often make their work available
competition to find the
This genre was known as load almost any of the brilliant best new interactive completely free, and some are
interactive fiction, and its most games that are still being written fiction, which is still better even than those old
famous publisher was Infocom. and given away for free. being written by fans classics. We recommend starting
In the process of creating games If you need some games, take a today. with Adam Cadre’s brilliant
like the Zork series, Hitchhiker’s look at the interactive fiction Photopia.
Guide to the Galaxy (with Douglas database (http://ifdb.tads.org).
Adams), or this writer’s favourite, This lists all known titles, and PROJECT WEBSITE
Stationfall, Infocom created a there’s still huge interest in creating https://christoph-ender.de/fizmo
virtual machine-alike new ones. The authors of these will
Role playing game
OpenMW
T
his isn’t quite free modify and even create their own
software, because you environments and adventures. For
need to source files from Morrowind players, this is brilliant
an original and still proprietary because there’s still a huge
game. But OpenMW is worth the community playing the game and
compromise because it’s a making modifications that improve
complete recreation of the nearly every aspect of the original.
games engine behind one of the OpenMW is also easy to use. You
best ever PC role playing games start with the launcher, which will
– Morrowind. Released in 2002, ask for the location of the installed
Morrowind is the third in the Elder files or the mounted drives, and it
Before OpenMW will
Scrolls series, coming after sucks up data from Morrowind itself launching the game, with or
work, you’ll need the
Daggerfall (see our tutorial on p88 and both of its expansion discs, if media assets and game without expansions and mods.
getting this to run for free) and you have them. A configuration data from an original The quality of the recreated
before Oblivion. It consists of a panel is then used to generate copy of Morrowind. engine is staggering, and while
huge open world viewed in first display settings with OpenGL for it’s a bit of a CPU and GPU hog,
person, and while this is a graphics rendering and for this will hopefully improve when
recreation of that original game, the migration to OpenSceneGraph
OpenMW has ambitions beyond is complete.
straightforward recreation. “The quality of the recreated game
OpenMW includes an editor, for
example, enabling players to
engine is staggering.” PROJECT WEBSITE
www.openmw.org
www.linuxvoice.com 75
TUTORIALS INTRO
TUTORIALS
Dip your toe into a pool full of Linux knowledge with eight
tutorials lovingly crafted to expand your Linux consciousness
In this issue…
78 80 84
Ben Everard
Is making a DOS game which runs batch jobs
on Ubuntu Core using Bluetooth and Android.
KDE Connect Wiimote Ubuntu Snappy
I
recently attended a talk by Cory Run double Linux for double Les Pounder links a Mike Saunders investigates
Doctorow, information freedom fun – Graham Morrisn Wiimote to a Rasbperry Canonical’s new package
campaigner and science fiction shows you how to link your Pi with Bluetooth and manager. Can it really bring
author (just before doing this month’s Android phone with your Python for some visual peace and harmony to a
interview). One of the things he spoke KDE desktop. entertainment. troubled server?
of was the importance of pragmatism
in the fight for digital freedom.
No one is perfect. It’s impossible to
88 92 98
avoid every piece of proprietary
software. Even if you use a fully free
distro (such as Trisquel), there’s still
proprietary microcode running on your
CPU. There’s still proprietary firmware
in your hard drive (and most likely on
other parts of your machine). Even if DOS games RAW images Batch Jobs
you somehow managed to overwrite
this firmware, the hardware is still In a never-ending search for With the right tools, image Ease the strain of repetitive
closed. Likewise, you almost certainly more ways to waste time, files give you more than jobs and get the computer
use some data harvesting web service Graham Morrison raids his just pretty pictures. to do the hard work.
such as Google search or Facebook. collection of DOS games Andrew Conway looks Mike Saunders introduces
The only way to be digitally free is to and brings them to Linux. beyond the visible. batch processing.
abandon the digital world altogether,
and although that may seem like an
attractive option at times, it doesn’t PROGRAMMING
really benefit anyone. A far better
solution is to recognise that you Cobol Gnome Builder Node.js
inevitably support some organisations 100 Once the most popular 104 Gnome is more than just a 106 Take a look at the new web
language in the world, Cobol desktop environment: it’s an platform in town: Node.js.
with bad practices and try to
has fallen from grace. It’s become entire suite of applications and This takes the JavaScript engine
counterbalance that by supporting a niche language, but still lives on the technologies used to build from Chrome and turns it into a
organisations with good practices. inside some of the biggest them (such as the GTK widgets). server powerhouse. It’s best suited
You can cleanse yourself of digital corporations in the world. We take Until recently, there hadn’t been an to event-driven applications that
guilt by donating to ORG, FSF, EFF, a look at this digital relic from a IDE to help developers work in this push data to the browser. With
SFLC or one of the other organisations time when computers weren’t area, but thanks to a crowdfunding Node.js, you can create complex,
personal and certainly didn’t sit campaign and some hard work, we interactive web apps using just
that campaigns for digital rights. atop desks or laps. now have Gnome Builder. one language.
ben@linuxvoice.com
www.linuxvoice.com 77
TUTORIAL KDE CONNECT
KDE CONNECT: GET DESKTOP
TUTORIAL
NOTIFICATIONS FROM A PHONE
Share files, check battery status, read notifications
GRAHAM MORRISON
and add remote control to and from your phone.
L
aptops and phones have become inseparable. phone to your desktop, including notifications,
WHY DO THIS?
They can often be found huddled together on messages and files, and lets you remote control your
• Check phone status
desks and breakfast bars across the land. But desktop or use your phone’s keyboard for input. Yes,
from your desktop
it’s only now that tools are being created to better KDE Connect does work best with KDE – it’s still
• Upload and download
files unify them. Canonical is trying its best with Ubuntu primarily a KDE 4 application, although we installed it
• Share clipboards Phones, and both Apple’s OS X and Google’s Chrome in Plasma 5 alongside the new widget and it worked
OS are starting to blend their desktop operating well. But an additional install called KDE Connect
systems with their mobile ones. Indicator adds much of the same magic to almost all
Linux has had these kinds of tools for some time, other desktops too, so everyone can get
and our favourite is KDE Connect. KDE Connect sends metaphorically closer to (or literally further away from)
all kinds of useful information about your Android their phones.
Step by step: Link your phone and your desktop
1
Installation 2
Get and configure the app
Most distributions will have a package for KDE … There are two sources for the Android ‘KDE
Connect and installation should be simple. However Connect’ app: the Google Play store and the F-Droid
we found the Ubuntu/Kubuntu package a little old, so open source package manager. The app is a free
we’d suggest using the following PPA: download on both (and open source too) and at the
https://code.launchpad.net/~vikoadi/+archive/ubuntu/ppa/ time of writing they both offered the same version for
If you’ve not used a PPA before, simply follow the download – version 0.8g.
instructions to add the required repositories to your After the app has installed and you’ve launched it,
system and you’ll find KDE Connect is now available. there’s a good chance your laptop/desktop will appear
On Arch, we installed kdeconnect-git from the user in the list of available devices, as detected by KDE’s
repository because it was a much more up-to-date auto-discovery. If not, you should click on the menu
version, and it didn’t require any weird dependencies. icon in the top-right and select ‘Add Devices By IP’. You
You’ll need to restart the KDE desktop after the can then use the button at the bottom of the screen to
installation because the tool itself adds a settings add and enter either an IP address for your computer,
panel and runs a background daemon that’s going or a hostname if it resolves across your network. We
to wait for incoming connections. If you open the found that after doing it once, our computer was
settings panel now from the System Settings menu, always detected, even without adding the IP address
no devices will be detected and you’ll be informed that manually. Type ifconfig -a or ip addr on your KDE
you need to install the app on your phone… machine’s command line to get its IP address.
78 www.linuxvoice.com
KDE CONNECT TUTORIAL
3
Make the connection 4
App configuration
At the same time that your desktop appears on your Back on the Android device, select the desktop
phone, you should find that KDE’s settings panel now machine and you’ll see a few options. The first opens
lists your phone/Android device too. You now need to a simple set of transport controls so you can play,
pair the two devices together, which you can do from pause, skip and change the volume of media being
your Android device or your computer running the played back from your computer – ideal if you’ve got
KDE desktop. On your Android device, select your KDE your laptop plugged into a television. The second is
machine and after it says ‘Device Not Paired’ press the labelled ‘ping’; pressing this will open a simple
‘Request Pairing’ button. You should immediately get notification window on the desktop.
a notification on your KDE desktop to say there’s a The last button in the list will turn your device’s
pairing request from your Android device and you touchscreen into a touchpad for your desktop, and it
need to click on ‘Accept’. works rather well. Moving the cursor is very sensitive
If you miss this, just press the back arrow on your and capable of offering good control. You can use the
Android and try again. As soon as you’ve accepted the touchpad to turn a single tap into a left-click, a
request, your Android device will now list your desktop two-finger tap into a right-click and a three-finger tap
beneath ‘Connected Devices’ rather than beneath into the middle button. These options are also
‘Available Devices’. Back at the KDE settings panel, available from the app’s menu.
your device will have turned green to show it’s
authenticated and you’ll no longer be able to choose
the ‘Pair Device’ option.
5
KDE configuration 6
Using KDE Connect
KDE gives you specific control over which parts of the There’s another part of the desktop, and that’s the
app you want imposing themselves on your desktop, widget that displays device-specific notifications/
and these options are available from the now- interactions and the remaining battery life of your
populated KDE settings panel for KDE Connect. Top of device, as well as giving quick access to the settings
the list is the ability to watch your Android device’s panel and the file sharing capabilities. For Plasma 5,
battery drain from the comfort of your desktop, just in you can install this just as you would any other widget,
case you weren’t paranoid about it enough. The using the ‘Add Widgets’ menu from the background.
second option is particularly awesome, as it shares You should now be able to share (for example )
the clipboard contents between the two devices. a URL on your phone and see it open the default
Select a URL on KDE, for example, and you can simply application on your desktop – usually a file browser.
paste the same string from your phone. It works like The folder icon on the widget will also open an SFTP
magic! You can also choose to enable notifications, connection to your phone, so you can browse its
ping or the multimedia controls, as well as whether filesystem from your KDE desktop – we had to add
incoming phone calls pause your music playback. /storage to the end of the path as there seemed to be
This works with almost any media player. a permissions problem with Android.
www.linuxvoice.com 79
TUTORIAL EDUCATION
WIIMOTE-TRIGGERED SELFIE
TUTORIAL
MACHINE
Les Pounder digs out his neglected old Nintendo Wii and hacks
LES POUNDER
together the latest in selfie technology.
T
he Nintendo Wii games console was released
WHY DO THIS?
in November 2006 to much fanfare due to a
• Learn Python novel method of input – it used a candybar
• Use new types of inputs shaped controller with a number of sensors such as
• Repurpose old an accelerometer, and an IR (infrared) sensor, which
technology
when used with the included sensor bar could locate
your position and use it to control your character. The
controller (which became known as the Wiimote) also
TOOLS REQUIRED featured a vibration motor for haptic feedback such as
• A Raspberry Pi Model Pi gunfire. But how did the Wiimote connect to the Wii?
2 or B+ Well it used good old Bluetooth to provide two-way
• Raspbian operating communication between it and the console, and the
system aforementioned sensor bar was really a series of IR Taking a picture with your Pi is really rather easy. You too
• PiCamera LEDs and a power supply. The Wiimote IR sensor can take high-quality cheesy selfies with ease!
• Nintendo Wiimote would calculate its position relative to the LED and
• A Bluetooth dongle then communicate that to the console. HDMI port and the blue band on the reverse facing the
• Monitor, keyboard, Sadly the Nintendo Wii has ceased production – Ethernet port. Once the cable is slid into place, gently
mouse and power
supply for the Pi but we're going to give its hardware a new lease of life, push the clip back into place to grip the camera cable
by building a Raspberry Pi-powered selfie machine to into the port.
trigger taking a picture and recording a short video. With the camera hardware installed, attach
your components and peripherals, then boot your
Setting up the camera Raspberry Pi to the desktop. With your Raspberry Pi
To install the Raspberry Pi camera module, your connected to the internet, open a Terminal (the icon
Raspberry Pi will need to be turned off, as the camera for which is located in the top-left of the screen and
is a rather delicate piece of kit. Locate the CAMERA looks like a black computer monitor).
The Camera has its own
dedicated port on the port on your board (between the HDMI and Ethernet In the terminal window, type the next two lines; at
Raspberry Pi and fits in ports). Gently pull the clip upwards to open the port the end of each line, press Enter:
rather securely, but be ready for the camera cable. The camera cable will sudo apt-get update
careful as it’s quite fragile. slide into the port, with the silver contacts facing the sudo apt-get upgrade
The first line updates the list of repositories, which
are places that contain Raspbian software packages,
to ensure that we have the latest software lists for
reference. The second line instructs our Raspberry Pi
to compare our installed software with that provided
by the repositories and, if there are any upgrades,
to download and install them. By completing this
step we can confirm that we have downloaded the
PiCamera Python package, which we will use later in
this project.
We will now issue another command in the terminal
to start the configuration tool:
sudo raspi-config
In the menu that appears, navigate to option 5,
Enable Camera, using the arrow keys, and press Enter
to confirm your entry. Choose Enable and press Enter,
then navigate to Finish to exit the config tool. If you
are prompted to reboot then do so and return to the
Raspbian desktop to continue.
Now we need to test that the camera is working.
80 www.linuxvoice.com
EDUCATION TUTORIAL
Taking a still image
Our first command is called raspistill, and as you may
have guessed, it uses the camera to take a still image.
To use the command we need to type the following
into the terminal:
raspistill -o test.jpg
This will open the camera preview and you should
see an image on screen. After around five seconds the
camera will take your pic and save it as test.jpg. Once
The raspi-config menu is
raspistill has completed it will return control of the the following:
a handy suite of tools to
terminal to you. If you open the File Manager, which sudo apt-get install bluetooth configure certain elements
can be found in the task bar, you'll see test.jpg in your This command will install all of the dependencies of your Raspberry Pi.
home directory, which is where we ran the original for using Bluetooth with our dongle – it will also take
raspistill command. some time, so perhaps pop off for a cup of tea and
To test video recording, we can use raspivid with come back in a few minutes.
the following command in the terminal: To enable Python to talk to the Wiimote, we need
raspivid -o test.h264 to install a library, and we do that using the following
Again this will launch the preview window but it command in the terminal:
is now recording video and will do so for the next sudo apt-get install python-cwiid
10 seconds. Once finished, the terminal control will CWIID, pronounced “seaweed”, is a Python library
be returned to you. To watch your video, type the that handles communication between your Raspberry
following into the terminal: Pi and the Wiimote. In this project we are using CWIID
omxplayer test.h264 with Python, but there are also packages available in
With our camera tested, it’s now time to set up the repositories to enable your Wiimote to be used
Bluetooth. as a mouse/presentation device – see https://help.
ubuntu.com/community/CWiiD for more details.
Setting up Bluetooth and CWIID With Bluetooth and CWIID installed, it's time for us
For our project we will need a Bluetooth USB dongle to move on to part 3 – putting it all together.
– we used an ORICO Bluetooth 4.0 dongle from
Amazon (http://bit.ly/LV17Bluetooth) as it had a Building the selfie machine
decent range of 10 metres and consistently In Parts 1 and 2 we have successfully set up our
connected first time with our Raspberry Pi/Wiimote camera and Bluetooth dongle and now our focus
combo. Plug your Bluetooth dongle into a spare USB shifts to creating the code that will control our selfie
port, then open a new Terminal window and type in machine. Our project will be written in Python 2.7, this
is due to CWIID not having a Python 3 library. So we
Raspberry Pi Camera start by opening the Idle text editor, which you can find
in the main menu under 'Programming' and then
The Raspberry Pi Camera comes in two flavours. In this
project we used the standard version that caters for all under 'Python 2'. Once Idle is open, click on File > New
scenes, but there is also the Pi Noir camera, which is used to open a blank document. Straight away save this file
in low light scenarios along with an infrared light source to as selfie.py by clicking on File > Save.
record video and take pictures at night. It is commonly used We start the code by importing the libraries that will
in nature photography such as bird boxes, as in this tutorial
form the basis of the project:
from the Raspberry Pi Foundation: www.raspberrypi.org/
learning/infrared-bird-box. import cwiid
The Raspberry Pi Camera can also shoot at high speed from time import sleep
using the raspivid command. Speeds of up to 90 fps are from datetime import datetime
possible at 640x480 resolution, enabling you to easily import picamera
create high-speed photography. You can try it by running
import os
raspivid -w 640 -h 480 -fps 90 -t 10000 -o test90fps.h264
This will capture 10 seconds of video at 90 fps. When We first import the cwiid library that we installed
played back, the video will be running at a third of its normal earlier; we then import a function from two libraries.
speed, due to the slower 29.97fps of normal video playback. From the time library we import the sleep function, to
The video will look like slow motion but will capture sharp control the speed of our project, and from datetime
images at every frame.
we import the datetime function, used to add a
If you would like to know more about high speed
photography read https://www.raspberrypi.org/new- timestamp to our images. Our last two imports start
camera-mode-released, and for general camera information with the picamera library, used to control the camera,
head over to Dave Jones’ great documentation at and finally the os library, used to run Linux shell
http://picamera.readthedocs.org/en/release-1.10. No commands in Python.
matter which camera you may choose, they both work with
Next, we create a variable called button_delay to
the PiCamera Python library in the same manner and they
also work with the raspistill and raspivid commands. ensure that input is read only once per press:
button_delay = 0.1
www.linuxvoice.com 81
TUTORIAL EDUCATION
the start of this function. We can add the time and
date that the picture was taken as text on the image:
camera.annotate_text = (pic)
Our last two lines of code handle capturing the
picture and saving it as the filename contained in the
pic variable. Finally we close the preview window:
camera.capture((pic))
camera.stop_preview()
Our next section handles creating three functions,
which will handle taking a picture, recording video and Recording video
Installing Bluetooth
finally displaying the picture on the screen. We start Our second function controls the recording of a short
requires a lot of
dependencies, but don’t with taking a picture. video, and the structure of this function is very similar
worry: the apt packaging Functions are defined, in that they are given a to that of the previous function.
tool will do all the hard name that can be called, and when that is the case We start with naming the function, creating an
work. the contents of the function are run. Our first function argument named vid which will later contain the
is called takepic, and it also has the word pic in timestamp for the video. We then also repeat the
brackets. This is an argument, an extra option passed shortening of the picamera function:
to the function when it is called. All the code indented def takevid(vid):
underneath the def takepic(pic): line is part of the with picamera.PiCamera() as camera:
takepick() function: Indented into the with statement, we have the next
def takepic(pic): two lines of code: the first sets the video resolution
with picamera.PiCamera() as camera: to the HD 720p, which give us the best compromise
camera.start_preview() between video quality and small filesize. We then
for i in range(5): start recording the video, passing the vid argument
wii.rumble = 1 that we will later create, and use string concatenation
sleep(1) to attach ‘.h264’, the video format which is used to
wii.rumble = 0 record the video, to the timestamp (vid):
sleep(1) camera.resolution = (1280, 720)
camera.annotate_text = (pic) camera.start_recording((vid)+'.h264')
camera.capture((pic)) We next add the timestamp to the video in the
camera.stop_preview() same manner as we did for the previous function.
Line 2 shortens the long picamera.PiCamera() into Then we start the preview window to help the user
camera, which is much easier to work with: frame their shot:
with picamera.PiCamera() as camera: camera.annotate_text = (vid)
We then indent once again and trigger the camera camera.start_preview()
preview function, which will show a live shot from the We now create a for loop that will iterate 10 times
camera: to rumble the Wiimote 10 times; you'll see on the last
camera.start_preview() line of this snippet that we have added camera.wait_
Our next section of code is still inside the function recording(1) this is a unique function for recording
and makes the Wiimote’s motor vibrate five times, video and is used in place of the sleep function. Using
giving the user a countdown until the photo is taken. the wait_recording function the program will check to
We use a for loop to iterate five times, turning on the ensure that there is enough disk space for your video:
motor for 1 second, then turning it off for 1 second: for i in range(10):
for i in range(5): wii.rumble = 1
wii.rumble = 1 sleep(1)
sleep(1) wii.rumble = 0
wii.rumble = 0 camera.wait_recording(1)
sleep(1) Finally we stop the preview window and recording:
Our next line of code is not part of the for loop, but camera.stop_preview()
is still inside the with conditional that we created at camera.stop_recording()
Our final function handles showing the user the last
picture that was taken; we name it showpic:
def showpic():
Our first line of code indented into the function runs
the system function from the os library; this function
enables us to run a shell command in Python, in
Running the code will this case the application gpicview. We then use
produce a series of outputs string concatenation to join the filename (selfie)
to the shell; these are to the command, and to append the string with an
instructions to the user. ampersand, which is Linux shorthand for running a
82 www.linuxvoice.com
EDUCATION TUTORIAL
command as a background process, releasing the
terminal back to the user. The entire command is
wrapped in a string (str) function, which formats the
contents into a string.
os.system(str('gpicview '+(selfie)+(' &')))
We next instruct the function to wait for five
seconds, giving our user time to view the picture;
finally we run another shell command that will stop
the picture viewer by killing its process:
sleep(5)
os.system('killall gpicview')
Bring it all together
The Wiimote is a really
With our functions complete, we now turn our buttons and save the value as a variable called
cheap method of input that
attention to the main body of code. First we create a buttons. We then create four conditional statements. can be used with robotics
method to handle connecting the Wiimote to your Pi. Our first is a method to close the connection between as well as the Raspberry Pi
We use a try..except construction to test that a the Wiimote and the Pi. By pressing the Minus and Camera.
connection is made. If there are errors, it will try three Plus buttons the program will exit:
times to connect before exiting: if (buttons - cwiid.BTN_PLUS - cwiid.BTN_MINUS == 0):
print('Press 1 + 2 on your Wii Remote now ...') print('\nClosing connection ...')
sleep(1) wii.rumble = 1
wii = None sleep(1)
i=1 wii.rumble = 0
while not wii: exit(wii)
try: If the user presses the Up button on the Wiimote, it
wii=cwiid.Wiimote() will launch the showpic() function we created earlier:
except RuntimeError: elif (buttons & cwiid.BTN_UP):
if (i>2): showpic()
quit() If the user presses the A button, a photograph is
break taken. Remember the pic variable we used as an
print('Error opening wiimote connection') argument for the takepic(pic) function? Well here we
print('Attempt '+str(i)) create it by asking Python to save the current time
print('Press 1 + 2 on your Wii Remote now ...') and date in a YEAR, MONTH, DAY, HOUR, MINUTE,
i=i+1 SECOND format. This is then printed in the shell, to
Our next block of code is instructions to the user show that it has worked. The takepic(pic) function is
on how they use the selfie machine; this is simply a called and a selfie is taken!
number of print functions with instructions on each elif (buttons & cwiid.BTN_A):
line. You will notice that each line ends in \n; this is an pic = datetime.now().strftime('%Y-%m-%d-%H:%M:%S')+(".jpg")
instruction to Python to move to a new line: print(pic)
print('Wii Remote connected...\n') sleep(3)
print('Press\n') takepic(pic)
… sleep(button_delay)
The final part of the main body is a while True loop Our last condition handles the user pressing the
that will constantly check the state of the Wiimote’s B button, which creates the vid variable used as an
argument in the takevid(vid) function we created
earlier. The vid variable is almost exactly the same as
Code for this project
pic: it records a timestamp for the video but replaces
All of the code for this project is housed in a GitHub the .jpg with .h264 instead:
repository. GitHub is a great way to store your code so that
elif (buttons & cwiid.BTN_B):
it is readily available and backed up to the cloud. GitHub
uses the Git version control framework to enable you to vid = datetime.now().strftime('%Y-%m-%d-%H:%M:%S')+(".
work on your code and then push it to the cloud; changes h264")
made on your machine can be pushed when ready, updating takevid(vid)
the code in the cloud. Others can fork your code and work sleep(button_delay)
on branches, for example creating new features. These are
So that's it! Save your work and when ready click on
then submitted to you for approval, and when you're ready
you can merge them with the main branch. Run > Run Module and have your Wiimote ready to
You can download the code for this project from test your selfie machine!
https://github.com/lesp/LV_Issue17_Education if you
are a GitHub user; if not you can download a Zip archive Les Pounder divides his time between tinkering with
containing all of the files used from https://github.com/ hardware and travelling the United Kingdom training teachers
lesp/LV_Issue17_Education/archive/master.zip. in the new IT curriculum.
www.linuxvoice.com 83
TUTORIAL SNAPPY UBUNTU CORE
SNAPPY UBUNTU CORE: NEXT-GEN
TUTORIAL
PACKAGE MANAGEMENT
Discover a bunch of Ubuntu technologies that could define
MIKE SAUNDERS
the Linux distributions of the future.
W
e all love to play around with new end-user Now, a bunch of these new technologies have
WHY DO THIS? features in Linux: new distro releases, worked their way into Snappy Ubuntu Core, a new
• Try a stripped-down updated desktop environments, and variant of the popular distro. We had a brief
Ubuntu variant
awesome graphical apps. But there’s a huge amount introduction to Snappy Ubuntu Core in issue 12’s FAQ;
• Learn about
transactional updates going on under the hood in Linux right now, affecting here we’ll look at it in more detail, explain why it’s
• Understand how Snappy the core of the operating system and the low-level useful, and show you how its packaging system
packages work plumbing that keeps it all ticking over. Some of these works. There’s a lot of cutting-edge technology in this
changes have been controversial and fiercely debated distro, but it’s worth learning about as it could make
(such as Systemd), but one thing’s for sure: Linux isn’t its way into the mainstream distros we’ll all be using
hanging around. It’s developing and modernising to be next year. Even if you don’t use Ubuntu yourself, given
the best all-round platform for desktops, mobile its prominent role in the Linux ecosystem it’s
devices and cloud deployments. important to keep track of developments.
1 UNDERSTANDING THE TECHNOLOGY
So, what is Snappy Ubuntu Core? First off, let’s focus Ubuntu onto it, you’d opt for Ubuntu Core and
on the second two words: Ubuntu Core. This is a customise it to your exact liking, choosing the
streamlined version of Ubuntu that weighs in at packages and interface that fit the limitations of
around 50MB and provides the bare essentials to get the device.
a Linux system up and running. You won’t find any So far so good – but it’s nothing special. There are a
graphical desktops or web browsers here; it includes zillion trimmed-down distros out there doing the same
just the base system with the usual command line thing. But this is where Snappy comes into play, which
tools, libraries and hardware drivers. Additionally, it radically changes the way software is installed and
includes apt-get for retrieving extra software. how updates are applied. Snappy Ubuntu Core uses
Ubuntu Core isn’t designed for end users, but rather transactional updates, which means that they are
Canonical is pushing
Snappy Ubuntu as the for distro and hardware developers. For instance, either applied in full, or not at all. It also means that
next-gen distro for cloud imagine you’re designing a new single-board updates can be rolled back very quickly and easily.
deployments and mobile computer like the Raspberry Pi, and you want it to run The best way to explain this is like so: consider the
devices. Ubuntu. Instead of throwing the full desktop version of current update mechanism in Ubuntu and other
Debian-based distros. As root, you enter the following:
apt-get update && apt-get upgrade
This has worked pretty well over the years, but it
has its limitations. What if you have a power cut or
kernel panic during the installation of a certain
package? The system files will be left in an undefined
state. You may end up with executables from FooApp
1.2 installed, but libraries from FooApp 1.1.
Configuration files may be out of sync. And if this is a
system critical tool or library, what happens when you
reboot? Your system may not come up properly. If
you’re willing to spend a lot of time, you could boot
from a live machine, chroot into the broken installation
and downgrade the offending package – if you can
find it. It all becomes incredibly messy, very quickly.
Transaction complete
Snappy Ubuntu Core aims to fix this by having
transactional updates. Instead of having several
84 www.linuxvoice.com
SNAPPY UBUNTU CORE TUTORIAL
hundred packages for the base system (kernel, Bash, A.) In this way, you always have a functioning
glibc etc.), this base system exists in a single package operating system partition on your drive, and you can
– so everything is updated at once. This base system roll back to the previous state very quickly – without
is also provided in a read-only root partition, called having to fiddle around with individual packages.
partition A, and it also has an unused copy in partition Now, a power outage during updates on a PC is very
B. When you update the base system, the working unlikely, but consider mobile devices. Canonical is
version in partition A is left alone; instead, the updates pushing hard to get Ubuntu onto mobile phones, and
are applied to partition B. you don’t want those getting bricked because
On the next boot, the machine tries to boot from someone’s battery ran out during a system update.
partition B, and if that works then partition B becomes We Linux geeks enjoy poking around in our OSes and
the default, and the next round of updates, when fixing problems when they come up, but mobile phone
available, will be applied to partition A. However, if the users? They demand that everything just works, so a
machine fails to boot from the updated partition B, it transactional update system with a simple rollback to
will revert to the known-as-working partition A. (Or if B the previous setup is essential – so that’s why we
boots but you have problems, you can switch back to have it.
2 TRYING IT OUT
Let’s give this a go. In a terminal, grab the latest the password, enter ubuntu. And that’s it – you’re
compressed filesystem image of Snappy Ubuntu Core running Snappy Ubuntu Core! You’ll notice that there’s
from the net and extract it like so: not a lot going on here, as it’s a very minimal
wget http://releases.ubuntu.com/15.04/ubuntu-15.04-snappy- installation. You can run commands as root by
amd64-generic.img.xz entering sudo before them, but note what happens if
unxz ubuntu-15.04-snappy-amd64-generic.img.xz you try to run apt-get – you’ll be told that this is a
(This download is 120MB, and will extract to a .img Snappy-only system.
file of 3.7GB in size.) Next, you need to boot this in a Yes, Snappy is the
package manager of
“Snappy is responsible for
PC emulator – and the simplest option is to use
Qemu-KVM. Find it in your distro’s package this distro, and is not keeping programs well isolated
repositories, install it and then run kvm-ok to check only responsible for
keeping the system in
from one another.”
that it’s working (you should see a message stating
“KVM acceleration can be used”). Then boot it up in a bootable state as discussed earlier, but also for
Qemu-KVM like so: keeping programs well isolated from one another.
kvm -m 512 -redir :8022::22 ubuntu-15.04-snappy-amd64-
generic.img System upgrades
Here, the -m 512 part says that we want our Enter the following command to view the filesystem
emulated PC to have 512MB of RAM, and the -redir layout of the virtual hard drive:
bit redirects a network port on the emulated PC to sudo cfdisk /dev/sda
ones on our host system. So once Qemu-KVM has You can see that the /dev/sda3 and /dev/sda4
finished booting Snappy Ubuntu Core, you can log into partitions are both 1GB in size, and these are the
it from another terminal using: read-only base system partitions (A and B) that we
ssh -p 8022 ubuntu@localhost mentioned earlier. Press Q to quit cfdisk, and then
(We redirected port 22, the SSH port, from the enter:
emulated PC to port 8022 on the host machine, which mount | grep /dev/sda3
is why you log in via localhost.) When prompted for Now you’ll see that /dev/sda3 is the root (/) partition,
Docking complete
Canonical is also pushing Snappy Ubuntu Core as
“the perfect system for large-scale cloud container
deployments”. Ubuntu is already one of the most
popular platforms for running Docker containers, and
in Snappy it’s available for installation with a single
command (sudo snappy install docker). The idea with
Snappy is that both the OS and the containerised
software benefit from transactional updates and
easy rollbacks. For more on Docker, check out our
tutorial on p96 of issue 16.
Docker: just a fad, or the future of software
distribution? Canonical is betting on the latter,
and it’s available in Snappy Ubuntu Core.
www.linuxvoice.com 85
TUTORIAL SNAPPY UBUNTU CORE
and the ro flag means that it’s mounted read-only. So Going back to Snappy, enter this to see a list of
no programs can tamper with the base system – packages installed on the machine:
compare this with the case of normal Linux sudo snappy list -v
installations, where everything can be modified by You’ll see that there aren’t many in comparison with
processes running as root. a typical Linux distribution. Pretty much everything is
But what happens when you need to make changes included in ubuntu-core. With Snappy, it’s possible to
to files in /etc and other directories? Well, repeat the have multiple versions of a package installed at the
above command but with /dev/sda5 instead of same time, and the active one is marked with an
/dev/sda3, and you’ll see that it’s mounted onto asterisk. To perform a system update, enter the
various places like /etc, /var and /home. It’s also following commands:
read-and-write, so this is where configuration files and sudo snappy list -uv
user files live. Additionally, /dev/sda5 is mounted onto sudo snappy update ubuntu-core
/apps, which holds self-contained applications as we’ll If an update to the ubuntu-core base system is
see in a moment. available, it will be downloaded and installed into the
So in summary: /dev/sda3 contains the alternative root partition – in this case, partition B
unchangeable base system for extra reliability and (/dev/sda4). Upon next reboot, the system will
security, while /dev/sda5 contains user-modifiable attempt to boot from partition B. To revert to the
directories that are mapped on top – in other words, previous version, use:
persistent data. sudo snappy rollback ubuntu-core
3 SNAPPY PACKAGES
So, what makes Snappy packages different to regular
.debs? Most crucially, they are designed to be
self-contained and not dependent on anything other
than base system tools and libraries. They include
everything they need, so they don’t have lots of
external library dependencies or are split up into many
different packages for documentation, artwork and so
forth. Ultimately, the goal here is to make software
distribution quick and easy – especially for third-party
app developers. You don’t need to worry about what
exact versions of everything are installed on a user’s
Ubuntu box; your program is pretty much guaranteed
to work and doesn’t care about the rest of the system. Use snappy search to find packages – but note that most
Additionally, you can have multiple versions of the of them are demos or hardware drivers right now.
same program installed as mentioned earlier. To see
how this all works, try installing an example Snappy into that and enter ls again: along with a directory
package: called 1.0.15, which contains the installed version of
sudo snappy install hello-world hello-world, you’ll see a symbolic link called current.
Now cd into the /apps directory, enter ls, and you’ll This link always points to the latest version, or if the
see a directory called hello-world.canonical. Switch user has rolled back a package after problems with an
Snappy vs RPMs/Debs: the trade-offs
With traditional Linux package management systems, On the downside, this introduces problems with security.
dependencies play a major role. A single program can depend Take a library that’s used by many different programs, such as
on hundreds of other packages – libraries, toolkits, artwork, Zlib (for compression). With traditional Linux packaging
documentation and so forth. Snappy packages, in contrast, systems, there’s one copy of Zlib on the system, and if a
have much more in common with software distribution on security hole is discovered in it, only that package needs to be
Windows or Mac OS X: everything that the program depends updated. All programs that depend on the library will
on should be distributed with the program itself. automatically be fixed.
There are upsides and downsides to this. On the upside, it If every program starts bundling its own copy of Zlib,
makes it very easy for third-party developers to distribute however, it gets complicated: if a security hole is discovered,
software: users can install a Snappy package and it will almost every program needs to be updated independently. Some
certainly work. The package doesn’t care which libraries are developers will respond quickly to security holes – others may
installed on the system, or where they are, or which versions take longer, or not bother to fix. As the user or sysadmin, it’s
they are. An update to Libfoobar which subtly changes the difficult to tell which is which, and whether your system is
behaviour of one of its routines can’t break other software in safe. Additionally, with every Snappy package bundling all of
mysterious ways, for instance. the libraries it needs, this takes up more disk space.
86 www.linuxvoice.com
SNAPPY UBUNTU CORE TUTORIAL
update, it will point to a previous version. This is how
multiple versions can live in a Snappy Ubuntu Core
installation side by side.
Enter cd 1.0.15/bin and run ./echo to execute the
main program in hello-world – and as you’d expect, it
prints “Hello World” to the screen. Enter ls and you’ll
see other executables in the directory, such as env
and showdev. You can run these executables from
any location in the filesystem by entering the name of
the package, a full stop, and the executable, eg:
hello-world.echo
But how does the system know where to find these
executables? If you look at your $PATH (eg echo
$PATH) you’ll see that there’s no entry for /apps/
hello-world.canonical/1.0.15/bin. If the $PATH
needed to be updated for every app you install, it
would become unwieldy. So instead, scripts are added
to /apps/bin whenever an app is installed, which call
the relevant programs. Have a look at /apps/bin/
Snappy package metadata is provided in YAML (Yet Another Markup Language –
hello-world.echo, for instance, and you’ll see a chunk www.yaml.org) format.
of boilerplate Snappy code to set up the execution
environment, while the last two lines execute
/apps/hello-world.canonical/1.0.15/bin/echo. meta directory, so if you look inside echo.apparmor
for instance, you’ll see that it’s assigned the “default”
Snibeti Snab AppArmor template. In other words, it runs with default
Switch back into the /apps/hello-world. permissions. With AppArmor, it’s possible to restrict
canonical/1.0.15 directory and enter ls again, and programs from accessing certain network resources
alongside the bin directory you’ll also see one called and filesystem locations – useful if you’re installing
meta. As you’d expect, this contains metadata for the software from an untrusted third-party source.
package – in other words, not the software itself, but As with operating system updates, Snappy package
information describing how it works. Switch into the updates are transactional so they can’t leave you with
meta directory and have a look around; you’ll see that a broken setup. Because newer versions are placed in
there’s a file called package.yaml, which contains the different directories
most important information about the package inside /apps/appname,
(name, version, vendor, icon and so forth). This you can always revert “Canonical has hinted that
package.yaml is one of two mandatory files in a to an older version if Snappy packages could become
Snappy package, the other being readme.md, a you come across any
description of the software in Markdown format. bugs. If you’ve ever part of the mainstream distro.”
For tighter security, Snappy packages are also tried to have multiple
restricted in their capabilities using AppArmor, a versions of the same program installed on your Linux
Mandatory Access Control (MAC) system that, among box, you’ll know just how difficult and messy that can
other things, stops executables from being able to be – so this is a welcome development. Ultimately,
access certain files. Each executable in the hello- users will have the freedom to try newer releases
world package has an associated .apparmor file in the without having to overwrite their old software and
potentially messing up the system.
So that’s an overview of Snappy Ubuntu Core,
exploring how its update and packaging systems
work. Much of the technology is still undergoing heavy
development, but Canonical has hinted that Snappy
packages could become part of the mainstream
desktop distribution in the future, and if it works well,
we could potentially see it adopted by other
distributions as well. For more on Snappy, see
Canonical’s documentation at https://developer.
ubuntu.com/en/snappy, and if you’d like us to cover
any aspect of it in more details (such as creating
Snappy packages by hand), drop us a line!
Mike Saunders has seen the future, although he has no idea
Much of the technology behind Snappy comes from
what next week’s lottery numbers will be. Sorry, everyone.
Canonical’s work on its Ubuntu Phone.
www.linuxvoice.com 87
TUTORIAL OLD GAMES
RUN DOS AND WINDOWS
TUTORIAL
GAMES ON LINUX
Broaden your games collection with a few classics from those
GRAHAM MORRISON
other forgotten operating systems.
T
hanks to Valve and its decision to switch from
WHY DO THIS?
Windows to Linux, our favourite operating
• Play some awesome system is in the process of becoming a major
games for free or cheap
gaming platform. Valve’s games portal, Steam,
• Brush up on essential
cross-platform skills currently lists over 1,100 Linux titles, including
• Work with Windows blockbuster releases like Team Fortress 2, Portal 2,
binaries on Linux Borderlands 2, The Witcher 2, Bioshock Infinite and many
more. This would have been unimaginable a couple of Even though there’s now a native version of Steam, and
years ago, and it’s likely to get even better as Valve natives versions of many of its games, you get access to
works towards releasing its own Linux-based platform many more when running the Windows version.
to compete with games consoles. It’s the reason why
many of us are looking at upgrading our machines intended to run. These are games that were typically
and moving them closer to the television. made for older versions of Microsoft Windows, and
Alongside the shiny new native version of games before that, Microsoft DOS. But we have the power to
that run on Linux is a vast library of classics yet to get these games running on your Linux desktop, and
be played, and almost any modern machine can that’s without running a virtual machine or needing
play them – even the humble Raspberry Pi or your a Windows licence. So raid your shelves, grab those
Android phone. The only slight hitch is that while games you’ve not played for a decade and spend a
these games can run on Linux, it’s not how they were wet afternoon enjoying some classics.
1 DOSBOX
Before Microsoft Windows (and Linux) there was DOS, from your browser while they’re being held in cold
the classic command prompt interface that turned storage at https://archive.org/details/
generic PC hardware into something useful. In the softwarelibrary_msdos_games.
1990s, DOS – the Disk Operating System – became a
game programmer’s playground because it gave Take a trip down memory lane…
direct access to hardware with rapidly accelerating The software that archive.org uses to drive its
performance and an equivalent drop in price, in-browser gaming engine is called DOSBox, which is
especially when compared with the Apple or itself a brilliant GPL-licensed problem solving tool
Commodore computers of the same time. All this new capable of running far more than just games. DOSBox
processing power led to the development of new is essentially an emulator for those earlier machines, in
types of games, such as Doom, as well as what much the same way CCS64 emulates all the hardware
became 3D acceleration devices for gaming, and intricacies of a Commodore 64. But while DOSBox can
there are all kinds of classics from this period. and does emulate the hardware environment of those
Remarkably, you can play thousands of them directly earlier machines, it can also pass CPU instructions on
to your native processor for massive speed and
efficiency gains. This is because the x86 CPUs used by
most computers are still derived from the CPUs in
those early PCs and share many of the same features
and instructions. Unfortunately, you don’t get the same
boost from a different architecture such as the
Raspberry Pi or ARM running Android, but you at least
get the same compatibility.
Daggerfall is one of the DOSBox is a simple point-and-click install from
best RPGs made, and it’s your favourite package manager. But like those early
a free download from the DOS environments, DOSBox can be a little cryptic
original publisher. to get running usefully. The first step is running the
88 www.linuxvoice.com
OLD GAMES TUTORIAL
executable. This will transport you from the modern own Daggerfall, we’ll include this step too. Whenever
world of social networking, pervasive data networks you make changes outside of DOSBox to a mounted
and clouds, and drop you into the world of Sound folder, you need to press Ctrl and F4 to refresh
Blaster, HIMEM.SYS and IRQ assignment. The first DOSBox, otherwise they won’t appear. You can then
line is already typed for you – SET BLASTER=A220 mount the ISO image from within DOSBox, and you do
I7 D1 H5 T6, configuring audio playback variables this with the following command:
for maximum compatibility, and you’ll notice the IMGMOUNT D DAGGER~1.ISO -t iso
command prompt flashing Z:\>. In the above line, the real filename of the disc
DOS is similar to the Bash command line, with some image, daggerfall.iso, has been truncated with the
important differences. The ls command is replaced ~1 symbols to accommodate the length restriction
by dir, for example, while cd will still change directory. on filenames in DOS. To get around the awkwardness PRO TIP
The internal mounted virtual drive is known as Z, of guessing and typing these names, just press Tab List all the commands
supported by DOSBox by
containing the simple tools required to boot most DOS to get name completion after entering the first few
typing help /all.
applications – you can see a list by typing dir. To do letters. With the above command executed, you’ll find
something meaningful, we’ll need to create a portal the contents of the ISO disc image hanging on the end
between DOSBox and our files back in the real world. of the D drive designation, which we can switch to by
You can do this by typing the following: typing d:. If you’ve extracted a rar archive, just cd into
mount c /home/graham/games -freesize 1000 its DFCD folder. The contents of both the ISO and the
We created a folder in our home directory called folder will be the same.
games, so you’ll need to modify the above command
for your own installation. The freesize argument is Installation
there because the storage capacity of our modern By 1995, the price of a Seagate hard drive the size we
drives is far in excess of what DOS is expecting, so created earlier (1000MB) was approximately $850
we’re pretending our folder is far more modestly – expensive but worth
capable. In this case, we’re mounting the folder and the outlay for most
providing just 1000MB of storage, which is still more computer users. “DOS is similar to the Bash
than enough for a CD installation. In DOSBox, you Storage was becoming command line in Linux, with
can change to this new drive by typing C:, and you affordable, and for that
can check its contents by typing dir – experience the reason, most mid- to some important differences.”
nostalgia of filenames limited to eight characters. late 90s DOS games
If you have the contents of your games media will need installing. In an age before standard toolkits,
handy, such as the files off a CD-ROM or a floppy most installers were different. You will need to briefly
drive, you can move them to your mounted folder and check any README.TXT files, documentation or even
access them just as you would the original media. the manual, if you’ve still got it. Most, including
However, we’re going to use an ISO image of one of Daggerfall, use their own installer, an install.exe
our favourite games – Daggerfall. This is a brilliant executable that’s run by typing install from the
3D RPG and a forerunner to both Oblivion and Skyrim. directory. Installers will enable you to select an
Daggerfall is available as a free download from the amount to install and a destination (you’re usually
publisher (see www.elderscrolls.com/daggerfall). best off sticking with the defaults). After those files
This download is a ‘rar’ archive that can be extracted have been copied, you’ll also need to negotiate
into your games folder and acted upon just as if you’d soundcard configuration. Daggerfall does this too, and
mounted an ISO or optical drive. But as most games thanks to the SET command that runs when you
you’ll own can be turned into an ISO, and because we launch DOSBox, selecting ‘Auto Detect’ should work. If
Step by step: Installing DOS games
1 Start DOSBox 2 Run install.exe 3 Configuration
When you run DOSBox, it will prepare the The installer is usually called install.exe, and All DOS games will require you to enter the
environment for running your games. You’ll need to they’re usually unique to each game. Here we’re details of your sound device, but auto-detect should
mount and navigate to the installer’s mounted folder. performing a full install of Daggerfall. almost always work out fine.
www.linuxvoice.com 89
TUTORIAL OLD GAMES
not, the default settings are Sound Blaster 16/ manually from the installation folder, after which we
AWE32, Port 220, DMA 1 and IRQ 7. After exiting any could type dagger to finally run the game. The name
installer, you can usually change these settings by of the executable should always be obvious and is
running setup or by editing a configuration file usually output by the installer. Just type this to run the
(usually ending with .cfg). One final step required by game. Many games also expect to have the
Daggerfall and many other games too, is to run an installation medium installed in the same location,
update. We downloaded ours from the publisher, and such as the ISO mounted, as this was their form of
you can usually find updates for popular titles. The copy protection, and you’ll need to make sure this is in
update was a dag123 executable, which we ran the same location each time you run the game.
PRO TIP
If you’re running DOSBox
2 RUNNING WINDOWS GAMES
in a window and you want
to get your mouse back While DOS games are lots of fun, many people have a alongside, just as you would on Windows itself, such
to the desktop, press Ctrl
and F10. larger collection that require Microsoft Windows, as DirectX for accelerated graphics, codecs for video
especially if you migrated to Linux from Redmond’s playback and fonts so that text looks the same. It
own OS. Many of the second-hand games you’re likely can get extremely complicated, and the configuration
to find are going to be from the last decade, rather and maintenance of a working Wine environment can
than the last two decades, which also makes take some time and effort, especially when changing
Windows rather unavoidable. settings can affect compatibility.
There are several solutions, with perhaps the
easiest being to install a legal copy of Windows into a PlayOnLinux
virtual machine. With this, you will get a perfect Fortunately, there’s an easy solution: PlayOnLinux
recreation of the operating system, but you won’t get is a wrapper around Wine installations and their
any meaningful hardware acceleration to help with configuration for specific applications and games.
game performance (although you will get some) and It makes installing something like a Windows
you’ll need a licence to use it. Virtual machines like this game much easier. Most distributions include the
are very important if you need 100% compatibility for PlayOnLinux package, and the package itself will
some Windows software, but there’s a better open handle Wine downloads, so you won’t find this as a
source option that works for many games and dependency. This is because specific versions of Wine
applications, and that’s called Wine. are tested with specific games and applications, and
Wine calls itself a ‘compatibility layer’ rather than the developers will only support versions they know
an emulator (Wine is an acronym for Wine Is Not an work well together. For that reason, PlayOnLinux will
Emulator, to make the point). Instead of emulating juggle several versions of Wine installed at the same
hardware, such as recreating the sound chip of a time, and it also means that installing the average title
Commodore 64, Wine recreates what Windows will take a considerable amount of network bandwidth
applications expect from the operating system, as it downloads the prerequisite fonts, libraries and
usually by replacing Windows functionality with Linux DirectX version for each successive version of Wine.
functionality. This functionality is augmented with When first launched, PlayOnLinux will download
Windows tools that are downloaded and installed the latest list of supported titles. Click on ‘Install’ and
Games are installed into their own virtual drives to avoid cross- PlayOnLinux will cleverly download any missing parts of Windows as
contamination of configurations and Wine versions. required by whatever game or application you’re trying to install.
90 www.linuxvoice.com
OLD GAMES TUTORIAL
switch to the Games list to see what’s available. There
are hundreds of compatible titles, including digital Games sources
downloads and games from CD and DVD. You’ll even
find some GOG.com titles listed, adding Linux support Buying pre-owned games is big business. games and applications. You could even
Whether it’s from your local store or eBay, try the free demo if you wanted to check
to your downloads from the main site, and the
there’s a huge selection of classics most of compatibility first. The great thing about
Windows Steam client can also be installed, adding us have missed. There are also many sites CrossOver is that it’s also a major contributor
many more titles that are yet to make it to Linux. We of dubious legality offering access to what to the Wine project itself, as all developments
installed both the Steam client and our favourite old they call ‘abandonware’, as well the stack of made to the commercial version are merged
classic, System Shock 2, which we own on CD. In both games hosted at the Internet Archive. into the open source version, helping the
GOG.com (formally Good Old Games) has development of both products.
cases, you’re guided through the installation via on-
turned playing old classics into a business
screen instructions. model, offering older games at a lower
price, and packages together with tools
Installing from CD like DOSBox to enable configuration-free
For System Shock 2, we needed to insert the CD before playback. What’s more important is that it
works with the original publishers to remove
we started. This is because the first question you’re
any DRM, which is often a stumbling block
asked by PlayOnLinux is where the optical drive is when playing old games and much more
mounted, and it will list the mounted volumes it likely to help those games work with Linux.
detects. Hopefully, one of these will be your drive. If Which is perhaps why beta Linux support is
not, use the ‘Other’ list item to point the requester at now available at GOG.com.
If you’re serious about playing old games,
the location of your mounted drive or ISO image. The
you might also want to look at CrossOver.
Windows installer will then be launched from the drive This is a commercial version of Wine that GOG.com is a games distribution
and you’ll need to run through the installation options uses a similar profiles system to PlayOnLinux service primarily for older games, and
for your games, including the entry of serial numbers to create point-and-click installers for many now offers Linux support.
if this is used to protect against copying.
If you get the option, we’d recommend choosing a
‘Full Install’ so that as many files as possible are automatically installed, but PlayOnLinux successfully
copied off the installation medium. We needed to use navigated the complicated third-party packages that it
Tab on the keyboard to select between some options needed and installed them at the same time.
in the Windows installer, and PlayOnLinux should also After games and applications are installed, they’ll
notice when other packages are installed. Most will appear as desktop icons (if you let them) and within
need fonts and DirectX, and System Shock 2 needed the main PlayOnLinux application window. Launching
an Intel media codec for video playback. You may also them is now just a click away, and they should
need to Alt Tab to hidden windows if the installer perform almost as
stops. Finally, you’ll be asked for your graphics card’s well as their natively
configuration, which is usually just the amount of installed counterparts. “Games and applications will
RAM on board. The Steam client required a few more From the Steam client,
you can install other
appear as desktop icons and
steps because the client itself needed to be launched
several times to enable a few updates to be titles but you’ll still find within the PlayOnLinux menus.”
compatibility problems
Getting CDs and DVDs onto your computer with the latest releases.
If you want to make changes to the Windows
Many of those old CDs and DVDs holding games are likely installation of a game, right click on its entry within
to be scratched and separated from their wallets, which PlayOnLinux and you can choose to open the directory
means their time as effective data sources is limited. Now
where the application/game is installed. This is the
that storage is so cheap, it makes good sense to move
them onto your hard drive. The simplest method is to copy place in your filesystem where the Windows files are PRO TIP
all the files and folder structure over from the mounted installed and is usually isolated from other games to With Wine installed, you
optical drive into a new folder on your machine. You can maintain compatibility. Opening the folder is useful if can run simple Windows
do this from the command line or from a file manager, and you need to manually add updates to an installation, binaries by typing wine
as long as you point Wine or DOSBox at the location of this followed by the name of
such as replacing an executable for a games update. the .exe file.
root folder to use as the optical drive, it will work fine.
A neater solution is to create an ISO image of the disc. You can also configure each Wine installation from
This is a single file that contains both the files and the the same menu. Wine’s configuration panel enables
filesystem of the disc, which can help with some game you to change the location of the optical drive and
compatibility when the game is checking for whether a the graphics resolution of the display, as well as the
disc is inserted. You can create an ISO using a GUI tool
version of Windows that’s being mimicked. This is
like Brasero by using the ‘Disc Copy’ option, or from the
command line with the dd command. If you’re going to usually Windows 98 for maximum compatibility,
store an ISO image, use 7zip to compress them as it can but you can choose anything between the ancient
also decompress an ISO’s contents. You can access an ISO Windows 2.0 and Windows 8.
directly with DOSBox, or mount it onto your filesystem with
the following command (/mnt/iso will need to exist first): Graham Morrison is the editor of Linux Voice, a lapsed KDE
sudo mount -t iso9660 -o loop cd.iso /mnt/iso/ contributor and a collector of old synthesizers.
www.linuxvoice.com 91
TUTORIAL IMAGING
IMAGING IN THE RAW
TUTORIAL
WITH NEAR INFRARED
ANDREW CONWAY
Still have your camera hooked up to your Raspberry Pi? Good
– put it to good use and learn some science at the same time.
I
n the heart of the Sun, nuclear fusion reactions This arrangement is called the Bayer pattern, after
WHY DO THIS?
produce energy. This leaks out to the surface of the person who first proposed it. The total number
• Make accurate the Sun where it is liberated in a huge number of of these pixels is usually the same as the advertised
measurements from
images – do amateur photons – particles of light. A photon can traverse image size of your camera; so if your camera boasts
science 150 million km unimpeded, make it through the 8 megapixels, which is about 8 million, then that is
• Understand your Earth’s atmosphere, through a chink in your curtains the number of Bayer pixels. In your final image, say
camera, get beautiful and strike your face in the morning. Then when you a JPEG, each pixel will be represented by a colour
images
open your eyes, an enormous number of photons will made up from mixing red, green and blue values, eg
• See in the infrared
enter them every second enabling your brain to an RGB of (255,0,0) is red, and (255,255,0) is yellow.
construct an image of the world you see. Since a Bayer pixel only records the intensity for
Our eyes are remarkable. What’s even more one colour, the colours need to be estimated from
remarkable is that for under £20/$20/€20 you surrounding pixels. The process of doing this is known
can buy a device that does a similar job that plugs as demosaicing or interpolation.
into a Raspberry Pi. In some ways it’s even more In images with gentle colour gradients, such
sensitive than the human eye in that it allows precise as a view across a grass field, the artefacts from
measurement and, if you buy the Pi NoIR version, it demosaicing will not be too noticeable. They may
works in the infrared. however become apparent if there is sudden change
We’re going to take a look at raw data produced by in colour and intensity, say around the edges of
a camera and understand how it can be processed for a window. More powerful CPUs and GPUs and
a number of purposes, both aesthetic and scientific. advanced interpolation reduce such problems, but the
Don’t worry if you don’t have a Raspberry Pi, because best solution is to increase the pixel resolution.
much of what we’ll explore can be applied to raw data Usually we want our photographs to look natural,
files from any camera. that is, to resemble what we’d see with our own eyes.
To achieve this we must balance the colours after
Bayer pixels demosaicing the Bayer pixels – a process known as
At the back of a camera is a grid of light sensors that white balancing. Using the red, green and blue values
you can think of as hardware pixels. Each one records as reported by the sensor will almost certainly result
the amount of light that enters it and this is read by in odd-looking colours. For example, a white object
the electronics as an integer, which will end up in the might appear slightly bluish. White balance varies
raw image file. However! These hardware pixels are from camera to camera, and depends on exposure
not sensitive to the colour of light. So to give us colour and lighting conditions. Although acceptable defaults
images, each one of these pixels has a tiny filter and algorithms for balancing are shipped with most
placed on top of it. In a typical modern camera half camera firmware, professional photographers will
the pixels have green filters, a quarter have red filters often want to take control of the colour balance for
and a quarter have blue filters, arranged as shown in the best results.
the image below. Having established that demosaicing and white
balancing are important for most photographs, let’s
now turn to an example where they would destroy
information, and working with raw data is a must.
Bayer pixels are arranged
on the camera sensor in Amateur science
groups of four with two Working with raw data enables you to make
green, one red and one quantitative measurements from your images. This
blue pixel in each 2 by 2
has applications across all branches of science, but
group. Green is over-
we’ll take a look at an astronomical one first.
represented to emulate
the colour response of It’s possible to use a normal digital camera, or the
the human eye, but it’s Raspberry Pi camera, to do useful astronomical work.
also due to geometry and The simplest question you can ask about a star is:
efficiency of manufacture. how bright is it? For this you must work with the raw
92 www.linuxvoice.com
IMAGING TUTORIAL
Star cluster colours
Here’s the star cluster M44, which you can see with brightness – known as the Hertzsprung-Russell from data published by professional astronomers
the naked eye in the constellation of Cancer. The diagram. The black squares are the values George (the trend is for brighter stars to be bluer). The
graph shows data obtained by amateur astronomer obtained using a Canon EOS 550D camera, and scales are logarithmic but run the opposite way to
George Johnston on how star colours correlate with show the same correlation as the red diamonds the magnitude system used in astronomy.
7
6
5
Star brightness
4
3
2
1
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Blue – Green
(image credit: Miguel Garcia)
data, because any attempt at demosaicing the Bayer along with the Python source code from https://
pixels will distort the results. github.com/mcnalu/linuxvoice-imaging.
Without even using a telescope you can measure There are two ways we can work with the
the changing brightness of a variable star such as Raspberry Pi camera: either from the Linux command
Algol. Put a normal digital camera on a tripod, take line, or using Python. We’re going to use the first, but
a few seconds of exposure and then measure the if you want to use Python then have a look at the
brightness of a star by adding up the values of all PiCamera module, and in particular the instructions
the pixels its image covers. Do this over a number on Raw Bayer capture that can be found here: http://
of nights and you’ll be able to plot a graph of Algol’s picamera.readthedocs.org/en/latest/recipes2.
variability over time. html#raw-bayer-data-captures.
It’s even feasible to discover a planet orbiting a star First, make sure your
by looking carefully at the star’s brightness. If the star
has a planet and its orbit is aligned so that it passes
Raspberry Pi is set up
with its camera and
“It’s even feasible to discover a
between the star and the Earth, then a dip in starlight ready to use. Next, point planet orbiting a star by looking
might be detectable. Of course, being feasible doesn’t
mean it’s easy: a decent telescope would be needed,
the camera at something
interesting and colourful,
at the star’s brightness.”
as well as a lot of patient searching and honing of open up a terminal and enter the following:
technique. raspistill --raw -o image.jpg
Stars have different colours, with blue stars being This will take a picture and save it to the file image.
hotter than red stars. In the early days of astronomy, jpg. The --raw option means that the raw information
the colour of a star was quantified by putting a blue from the camera sensor will be embedded in the file.
filter at the end of a telescope and taking a brightness Now we have the image data, you could continue to
measurement and then repeating the same work on your Raspberry Pi, but it’ll probably be faster
measurement but with a different filter. The difference to copy image.jpg to a desktop or laptop computer
in brightnesses was called the colour index. The running Linux to perform the raw data extraction and
Bayer pixels in a modern digital camera can be used processing. Next, we’ll need to extract the raw data
in much the same way – see the boxout above for an from the .jpg file. To do this we’ll need to make use of
example of this. a nifty utility called raspiraw. It’s not available in distro
repositories, so we’ll need to pull its source code (just
The Raspberry Pi camera one C file!) from GitHub and build it:
Let’s play with data ourselves using a Raspberry Pi git clone https://github.com/illes/raspiraw.github
camera. If you want to work with raw data from some cd raspiraw
other camera then you can skip to the next section. make
Alternatively, if you’d like to use the exact same data Copy image.jpg to the raspiraw directory and
as I’m using, then you can get my raw images of trees perform this command:
www.linuxvoice.com 93
TUTORIAL IMAGING
Everything’s gone green
This image shows the raw data captured around the My Little Pony’s head (is the is much too green because there are
from the camera sensor. Each pixel author a Brony?) is blown-up so you two green Bayer pixels for each red or
is either red, blue or green. The area can see the Bayer pattern. The image blue pixel.
./rpi2dng image.jpg python processraw.py
This will output a file called image.dng that only and you should see output showing the width and
contains the raw data. The file is missing important height of the raw image. For a Pi Camera the width
metadata that will be useful later on, but we can copy will be 2592 and height will be 1944. Next, let’s look at
it over from the .jpg using another handy utility called information about the camera’s Bayer pixels. Add the
ExifTool. First, install following lines to processraw.py and run it again:
“Stars have different colours, ExifTool, for example on print ‘Bayer pattern:\n’,raw.raw_pattern
with blue stars being hotter Debian-based distros
just do:
print ‘Indices 0,1,2,3: ‘,raw.color_desc
The first two lines give this output:
than red stars.” sudo apt-get install Bayer pattern:
libimage-exiftool-perl [[3 2]
And then copy across the Exif data with this: [0 1]]
exiftool -tagsFromFile image.jpg image.dng -o image.exif.dng Indices 0,1,2,3: RGBG
This copies the metadata from image.jpg and takes This tells us that the Bayer pattern in this sensor is
the raw data from image.dng and combines them in ordered so that in each group of 2x2 pixels, the two
the output file image.exif.dng. green pixels are top-left and bottom-right, the red
pixel is bottom-left and the blue pixel is top-right. Pixel
Using Rawpy co-ordinates are such that (0,0) is at the top-left, and
Rawpy is a Python module that provides tools for pixel (x,y) is x pixels to the right, and y pixels down. So,
working with raw images. We’ll also need NumPy for (0,0) will be green, and (1,0) will be blue, (0,1) will be
handling numbers and arrays, and Matplotlib so we red, and (1,1) green, and (2,2) will be green again, and
can display images from Python. On a Debian-based so on. We can confirm a particular pixel’s colour and
distro you can get all of these with this: extract its value as follows:
sudo apt-get install python-numpy python-matplotlib libraw-dev print ‘Colour at 100,100:’,raw.raw_color(101,100)
Open a text editor and enter the following lines: print ‘Value at 100,100:’,raw.raw_value(101,100)
import rawpy, matplotlib.pyplot as plt, numpy as np which gives output
raw=rawpy.imread(‘/home/foo/linuxvoice-imaging/image.exif. Colour of bayer pixel at 101,100: 0
dng’) Value of bayer pixel at 101,100: 32320
print ‘Sizes of the image:’,raw.sizes Note the arguments of these two methods are (y,x)
First we import the modules we need and set and from the above we can see that 0 corresponds
abbreviations called plt and np to save on typing later. to red. Now let’s attempt a graphical reconstruction
The second line reads the data from the dng files into of what the Bayer pixel array in the camera’s sensor
the raw object, and then prints out metadata on sizes. “saw”. You can just add this code to the end of
Save these three lines in a file called processraw. processraw.py and run it as before:
py, and then in a terminal window cd into the directory nx=raw.raw_image.shape[1]
where you saved it and run it like this: ny=raw.raw_image.shape[0]
94 www.linuxvoice.com
IMAGING TUTORIAL
ris=raw.raw_image.astype(float) To understand how this works, let’s take a step back
rismax=ris.max() and look at 1D arrays in Python. Let’s say we have an
rgb=np.zeros((ny,nx,3), ‘float’) array x=[1,2,3,4,5,6]. Then a=x[0::2] says to start at
rgb[1::2,0::2,0]=ris[1::2,0::2]/rismax index 0 in x and copy every second element to a, so
rgb[0::2,0::2,1]=ris[0::2,0::2]/rismax a will be [1,3,5]. Likewise b=x[1::2] will contain only
rgb[1::2,1::2,1]=ris[1::2,1::2]/rismax the even numbers from x. Now, if z is a six-element
rgb[0::2,1::2,2]=ris[0::2,1::2]/rismax array filled with zeroes, then z[0::2]=x[0::2] will result
plt.imshow(rgb, interpolation=’none’) in z being [1,0,3,0,5,0]. So, with this mind, we can
plt.show() translate the first line, ie rgb[1::2,0::2,0]=ris[1::2,0::2],
There’s a lot going on here so let’s break it down. into English as “copy values from ris to rgb starting at
The first two lines store the width of the raw_image (1,0) (which is a red pixel) and skipping 2 pixels in both
array in nx and height in ny. We need to give floating the x and y directions”.
point values scaled between 0.0 and 1.0 to the plot The final two lines plot the data. The
the routine, so the next two lines prepare for this by interpolation=’none’ setting ensures that the data This image was taken
turning the array into floats and setting rismax to is displayed as is and isn’t smoothed to make it look using the piece of filter
the maximum value. NumPy’s zeros method is used more “attractive” when we zoom in. To see the Bayer plastic that is shipped
to return a 3D array filled with zeros. You can think pattern, you need to enlarge a small part of the image, with the Pi NoIR camera.
of this as having three layers indexed 0, 1 and 2, which you can do by clicking the “Zoom to rectangle” By comparing it with the
corresponding to red, green or blue respectively, where button at the top of the window and then drawing a colour balanced image
each layer has the same dimensions as the image. small rectangle somewhere on the image. above you can see that
the leaves on the trees are
The next four lines might look like a terrifying
much more obvious. This
mess of colons, square brackets and numbers, but Build an image indicates that these trees
all they’re doing is placing red Bayer pixel values into We can now construct a more conventional image by are in rude health, as one
layer 0, green pixels into layer 1 and blue pixels into using a simple method to demosaic the Bayer pattern. might expect for a mature
layer 2 of the rgb array at the same co-ordinates as We’ll take each group of four Bayer pixels and map tree that’s just sprouted
they were in the original image. them into one pixel in which the red and blue values fresh spring foliage.
www.linuxvoice.com 95
TUTORIAL IMAGING
are used unchanged, but the average is taken of the plants convert light to chemical energy. It turns out
two green pixel values. We can do that with this code: that a healthy, photosynthesising leaf will show a
rgbi=np.zeros((ny/2,nx/2,3), ‘float’) strong signature not just in the green, but also in the
rgbi[::,::,0]=ris[1::2,0::2]/rismax near-infrared part of the spectrum.
rgbi[::,::,1]=0.5*(ris[0::2,0::2]+ris[1::2,1::2])/rismax
rgbi[::,::,2]=ris[0::2,1::2]/rismax Where next?
Notice that there are no numbers around the double If you’re intrigued by raw image data but put off by the
colons on the left-hand side now, because we intend command line and coding you might want to
the resulting image to be half the size of the original. experiment with some GUI tools. If you’re already
As you can see in the boxout, the image still doesn’t familiar with imaging-processing tools on Linux then
look quite right, because no white balancing has been the UFRaw plugin with Gimp or KDE’s Krita will both
done, and this is especially noticeable if you compare enable you to work with raw data. Alternatively, have a
it with the other image, which uses the Auto White look at Darktable or RawTherapee – both are
Balancing (AWB) of the camera. We’re not going to go specifically written for working with raw images.
any further into this subject here, but colour balancing If you’re happy on the command line then you
can be done in Python, using clues from the camera’s might like DCRaw, which lets you convert from many
metadata. Have a look at raw.daylight_whitebalance different raw formats to the common image formats,
and raw.camera_whitebalance and you’ll see giving you control over aspects such as colour
suggested coefficients for red, green and blue to balancing and demosaicing through command line
achieve a decent white balance. switches. In fact, DCRaw is the grandparent of Rawpy
because Rawpy relies on libraw, which arose from a
How green are leaves? project to turn the source code of DCRaw into a library.
The hardware pixels used in cameras are sensitive to If you wish to delve deeper into the complexities
light that human eyes can’t see that’s just off the red of raw camera data then there’s plenty more to
end of the spectrum, known as the near-infrared. explore with Python and Rawpy, but if you’re more
(Before you get too excited, it’s the far-infrared that’s comfortable with C++, you’ll want to work directly with
used for night vision intensifiers.) Most cameras have the libraw library.
a built-in filter to block this out because it will produce Even if you don’t need to work with raw data directly,
images that look quite unnatural to our eyes. However, an understanding of it not only gives insight into the
the Pi NoIR (No InfraRed filter) camera is shipped tremendous power and flexibility of modern imaging
without this filter. This makes it more suitable for devices, but also the nuances of how we perceive light
some scientific tasks. and colour with our own eyes.
Plant leaves contain a substance called chlorophyll.
Not only is it responsible for their green colour, but it Andrew Conway absorbs infrared, predicts election results,
watches the stars and uses Slackware Linux.
is essential to photosynthesis, the process by which
Demosaiced image
LEFT Simple demosaicing of the Bayer pixels. Each pixel in this image RIGHT This image has been demosaiced and auto-white coloured balanced with
corresponds to a group of four Bayer pixels. The red and blue component of default settings. It shows a much more natural range of colours, including the
each pixel is set equal to the red and blue Bayer pixel values, and the green vivid red, green, yellow and blues of the children’s toys in the foreground. The
component is the average of the two green Bayer pixels. The colours are not colours are still unnatural because the image was taken with a Pi NoIR camera,
natural because no white balancing has been performed. which is sensitive to the near-infrared which our eyes cannot see.
96 www.linuxvoice.com
SUBSCRIBE
SUBSCRIBE
shop.linuxvoice.com
Get your regular dose
of Linux Voice, the
magazine that:
DIGITAL Gives 50% of its profits
SUBSCRIPTION* back to Free Software
ONLY
*
£38
WHEREVER IN THE WORLD YOU Licenses its content
CC-BY-SA within 9 months
ARE – IT’S DIGITAL, SO THERE
ARE NO POSTAGE COSTS
Overseas subs prices All subscribers get
12-month print & digital: access to every
Europe: £85 single digital back
US/Canada: £95 issue – that’s about
Rest of world: £99 1,000,000 words of
tutorials, reviews
and free software
hackery at your
fingertips
Get 114 pages Access our Save money on
of tutorials, rapidly growing the shop price
features, interviews back-issues archive and get each issue
and reviews – all DRM-free and delivered to
every month ready to download your door
Payment is in Pounds Sterling. 12-month subscribers will receive 12 issues of Linux Voice a year. 7-month subscribers will receive 7 issue of Linux Voice.
If you are dissatisfied in any way you can write to us to cancel your subscription at subscriptions@linuxvoice.com and we will refund you for all unmailed issues.
www.linuxvoice.com 97
TUTORIAL AUTOMATION
BATCH JOBS:
TUTORIAL
AUTOMATE REPETITIVE TASKS
MIKE SAUNDERS
Need to do the same job on 1,000 different files? As always,
the Linux command line comes to the rescue.
N
ew Linux users are often baffled by the You might be tempted to rename (move) them with
WHY DO THIS?
command line interface (CLI). Why would something like this:
• Do hour-long jobs in anyone want to tap cryptic lines of code into mv austria*.jpg germany*.jpg
seconds
a black box, when modern Linux distros provide But! This won’t work at all. The asterisk wildcard is
• Automate your workflow
enough point-and-click goodness for even the expanded by the command line shell before the mv
• Avoid getting RSI
noobiest of noobs? Well, there are plenty of reasons to command is executed, so it becomes:
use the CLI, but our number one is this: it’s fantastic mv austria_001.jpg austria_002.jpg austria_003.jpg
for doing batch jobs. Whenever you need to perform germany_001.jpg germany_002.jpg...
the same task on hundreds (or even thousands) of This makes no sense – you can’t move multiple
files, nothing beats it. Sure, some graphical programs files to a single file (only into another directory). So we
and file managers let you do batch jobs with a lot of need a more canny way of doing this. Instead of trying
fiddling around, but the CLI makes it very to throw every filename at the mv command, we put
straightforward. So if you’re new to Linux or you’ve mv inside a for-do-done loop, so that the command is
never used the CLI to automate tasks before, open up executed for each file individually.
a terminal and read on… Consider this:
The key to performing a batch job is being able to for x in *; do file $x; done
iterate over a bunch of files. At the command line, the This does the same job as the file * command
asterisk (*) character is a wildcard used to refer to all mentioned earlier, but instead of passing all filenames
files, so if you enter a command like this: to the file command, it runs file on each one
file * individually. This bit starts the loop:
It will show information for every file in the current for x in *;
directory. Now let’s say you have a bunch of JPG It essentially says: for every file in the current
photo files from a recent trip called austria_001. directory (*), go through them one by one, and store
jpg, austria_002.jpg, austria_003.jpg and so forth. the filename in the x variable – aka storage space –
On closer inspection of the EXIF data, you realise each time. (We choose x as a name here, but you can
that the photos were taken just over the Bavarian use something else.) Then we have the middle part of
border (it happens) and you want to rename them to the loop, the bit that is executed for each file:
germany_001.jpg, germany_002.jpg etc. do file $x;
So, it runs file on the filename stored in the x
variable. This happens for each file, and done
signifies the end of the loop. Now, how do we modify
this command to make it perform batch rename
operations? Here we use text substitution, like this:
for x in *; do mv “${x}” “${x/austria/germany}”; done
Here, inside the loop we run an mv command to
rename the files, and we mv it from the filename
stored in the x variable to a modified version. The x/
austria/germany part performs text substitution,
taking the filename in x and replacing the part after
the first slash with the part after the second. So in the
end, this loop goes through all files and replaces all
instances of austria with germany.
You can use variants of this command in many
ways, eg for changing file extensions. You can also
add text to the beginning of filenames like so:
for x in *; do mv $x 2015_$x; done
Here we add 2015_ to the beginning of each
filename. Note that we use a simpler form of the x
The find command has a huge array of options for narrowing down your searches. variable here – $x instead of ${x} – because we’re not
98 www.linuxvoice.com
AUTOMATION TUTORIAL
Handling unknown files
Here’s a useful trick if you ever have a directory full Run file -i * in a directory with some PNG files, just those with the MIME type of image/png,
of random filenames without extensions, or with and you’ll see output like this: but we don’t want that extra information in the list
incorrect extensions. This situation can occur if you somefile.png: image/png; charset=binary we send to mv. So we use the cut tool to take the
use file retrieval software after deleting something, This shows the MIME type for the file (image/ first field of data (the filename), telling cut that
or in the case of corrupted filesystems. You can png), so imagine we have a directory full of fields are separated by colons (-d :). In the end, the
end up with thousands of files with names like thousands of randomly named files, and we want command inside the backticks generates a list of
NWI928AN – which isn’t particularly useful. to move all PNG images into a separate directory filenames that have been identified by the file utility
As you’ve seen in the main text for this guide, called new. We do this using the mv command as PNGs.
you can use the file utility to determine the format as usual, and using backticks (as explored in the This demonstrates how you can string together
of a file. This is a clever program that doesn’t use main text) we feed mv with a bunch of filenames operations to generate the information that you
file extensions to work out what a file is – after all, generated by another command: need, and build up extremely powerful commands
they can be changed to anything – but actually mv `file -i * | grep image/png | cut -f1 -d :` bar from a collection of small tools. This is the
looks inside the file and searches for sequences Here we have a series of commands inside Unix way: standalone tools working together to
of data. For instance, PNG files actually have the backticks (turning it into a larger command), and make something big, instead of giant monolithic
letters “PNG” in the first few bytes, so it’s possible we use pipe characters to move the data between applications that try to do everything at once (and
to identify them regardless of filename. them. The grep part narrows down the list of files to usually badly).
doing any text substitution, but merely adding some command inside another command, which sounds a
characters. bit strange at first, but look at this:
You can, of course, use these batch loops with for x in `find . -type f -mtime -1`; do file $x; done
other programs as well. One very popular command This runs the file command on every file that was
line suite for processing images is ImageMagick, modified in the last 24 hours, but nothing else. This is
which has a tool called convert that performs a wide the important component of the above command:
range of operations on images. For instance, say you `find . -type f -mtime -1`
have a directory full of PNG files, and you want to Here we use the find utility to search for files in
create thumbnails from them with a maximum width the current directory (.), and only normal files (-type
of 100 pixels: f, so not directories or special system files), with a
for x in *.png; do convert $x -resize 100 thumb-$x; done modification time of one day (-mtime 1). Because
Here, we run convert on all files ending with .png, we surround this command in backticks, its output
create versions of maximum 100 pixels wide, and is passed back to the “for x in...” loop in the full
save them with a prefix of thumb-. The convert utility command. There are
is capable of many other image editing operations, other ways to pass “Sometimes it’s useful to
such as cropping, adding text and changing formats,
so see the manual page (enter man convert at the
output around, such
as redirecting into text
perform batch operations on
command line) for more information. files, but this is a quick files that contain a certain date.”
way to do it.
Your time is precious So, let’s combine everything we’ve learnt into a
Sometimes it’s useful to perform batch operations on beast of a command that will create 150-pixel JPG
files that match a certain date, and here’s where we versions of all PNG files modified in the last three
can use another cool trick at the command line. days. And to top it off, we’ll restrict it to files that are at
Backtick characters (`) enable us to perform a least 2MB in size (using the -size +2M option to find).
Ready? Take a deep breath...
for x in `find *.png -type f -mtime -3 -size +2M`; do convert
“${x}” -resize 150 “${x/.png/.jpg}”; done
Not bad, eh? With the convert command, we don’t
need to specify an option to say which file format we
want to use – we just use a different extension for the
output file. So we take the original filename and using
text substitution, change .png to .jpg in the output file,
and convert therefore knows which format we want
to use.
So, now you know how to save heaps of time at
the command line, and next time your Windows
using friends or colleagues dismiss Linux because it
involves lots of tapping stuff into a black box, you can
laugh at how much time they waste with their pointy-
clicky desktop fluff.
You can perform batch file rename jobs using graphical
Mike Saunders finds it fascinating that this “mouse” gizmo
apps, but it’s better to learn the command line approach
thing actually took off.
to combine it with other tools.
www.linuxvoice.com 99
CODING COBOL
COBOL: THE LANGUAGE OF
TUTORIAL
BUSINESS
JULIET KEMP
COBOL wasn’t necessarily the best language, but as a tool aimed at
non-specialists it was ahead of its time.
C
OBOL is the last of the big four late-1950s
languages we’ve been looking at in this series.
Like ALGOL, it was designed by a committee;
but COBOL’s main distinction was that it was aimed at
businesses, and the design priority was to make it
English-like and easy to understand. It’s often seen as
an outdated language, but there are billions of lines of
COBOL still running on computers across the world,
and the language is still being actively developed.
One of the problems computer users faced in
the late 1950s was that programming was not only
very expensive, but also non-transferable. Programs
written for one computer couldn’t in general be run
on another computer, so your expensive program
was limited to one machine. Programming languages
that were semi-portable (like Fortran) were appearing, As it turns out, the reports of COBOL’s demise were
but these weren’t aimed at business so much as greatly exaggerated. It’s old, not dead.
at academics, and computer programming was of
increasing interest to large businesses. organisations – academics, computer users, and
Mary Hawes, a programmer at Burroughs computer manufacturers – met, and persuaded the
Corporation, was the first to call for something like US Department of Defense to sponsor the creation
COBOL. She wanted a cross-platform language of a common business language. The DoD had
that could run payroll, inventory, and other similar a vested interest in the matter: owning over 200
tasks: the sort of data processing that businesses computers already with another 175 on order, they
wanted. Fortran simply wasn’t set up to do this. Later were understandably keen to be able to run the same
in 1959, a group of people from various interested programs on all of them.
Design by committee
The initial meeting of the group, in May 1959,
saw the participants describing a language that
would be cross-platform; easy to use, maintain,
and alter; English-based; and work across multiple
environments. Various committees were set up,
including a short-range committee which was to
assess the currently available languages and begin to
specify an interim new language. Grace Hopper was
an advisor to this committee, and various computer
manufacturers and government agencies were
represented. Members included Jean Sammet (later
creator of algebra system FORMAC), Betty Holberton
(one of the first programmers of ENIAC and involved
later in FORTRAN 77), and Bob Bermer (inventor of
COMTRAN).
The committee was chiefly considering FLOW-
MATIC, an English-based language invented by
Grace Hopper and her team; AIMACO, a derivative of
FLOW-MATIC; and IBM’s COMTRAN, invented by Bob
The cover of the first Bemer. Jean Sammet, remembering it later, described
COBOL report, from 1960. a certain amount of anti-IBM bias, and certainly
100 www.linuxvoice.com
COBOL CODING
Y2K
You may remember, in the years before 2000, a lot of talk This wasn’t a COBOL-specific problem; it just happened
about the “Y2K problem”. This boils down to the fact that, that since COBOL was the most-used language in the 1960s
back in the 1960s and 1970s, when program space was at and 1970s, most of the surviving code (ie the problem
a premium, programmers chose to store dates as 6 figures: code) was COBOL. So, coming up to 2000, lots of COBOL
YYMMDD. So 1-Jan-1900 was 000101, and 1-Jan-1999 was programmers had many lucrative contracts to fix up all those
990101. The latter was larger than the former, and thus the 6-figure dates, either by changing them to 8-figure dates (ideal
second date was later than the first date. Even once space but expensive), or with a cheaper fix such as changing the
became less expensive, this continued to be the norm. 100-year ‘window’ to allow for the century change). This ran
Programmers simply weren’t expecting the code they were alongside a lot of media hysteria about planes falling out of
writing in 1975 or even 1985 to last until 2000. the sky and so forth.
As it turned out, lots of this code was indeed still running The date flipped round, nothing much happened, and there
come the 1990s. Which meant that 1-Jan-2000 would be was a new round of media hysteria about how over-hyped the
stored as 000101 and the whole thing would (or might) break. whole thing had been. On the other hand, you could see the
The consequences of this would depend on the particular fact that everything went smoothly as a credit to all those
piece of code in question (is it comparing, or just displaying, or coders who put the effort in to make the changes in time.
something else again?); which is to say, in any given case, no (For another take on this, see www.exit109.com/~ghealton/
one knew without going to look. (And if you’re going to look, y2k/y2k_humor/Cobol.html).
you might as well crack on and fix it.)
Grace Hopper seems to have pushed FLOW-MATIC’s in retrospect looks like a very bad design decision.
features quite heavily. Much later, in 1980, she claimed COBOL was also hard to modularise (in particular it
that COBOL was “95% FLOW-MATIC”. However, while was impossible to limit data access) at a time when
FLOW-MATIC’s English language naming and many modularisation was becoming popular. Further COBOL
other features were included in the new COBOL spec, updates improved some of this, and in 2002 object-
COMTRAN features were also included, such as some oriented programming was included in the spec.
mathematical formulas, the picture clause (allowing
data items to have both type and formatting defined), Installation and Hello World
and a much better IF statement. What COBOL didn’t COBOL is still actively maintained, with COBOL 2014
have were functions with parameters, which would be the most recent release, and COBOL programs are still
criticised later. used globally across many different operating
The size of the short-range committee made systems. Although some managers of these systems
for slow progress; at one point Howard Bromberg say they would like to migrate to another language,
bought a tombstone with “COBOL” engraved on it the bottom line is that migrating the billions of lines of
and sent it to the data systems director at the DoD. actively-used COBOL is expensive, time-consuming,
A further sub-committee was formed, which created and risky. An expensive rewrite of an operational
the COBOL spec. COBOL 60 was signed off by the system purely to achieve the same thing in a different
steering committee in January 1960, which meant language is a hard sell, in budgetary terms, especially
that computer manufacturers could begin to create when the old code can instead just be migrated off
COBOL compilers. An RCA 501 was the first machine aging mainframes onto modern kit. Not only that, but
to successfully run a COBOL program, on 17 August plenty of people would argue that the other
1960. The first cross-platform program was run in mainstream languages simply don’t replace the
December, on a UNIVAC and another RCA machine.
COBOL had achieved its aim: same program, different
machines.
These first compilers were very slow, although by
1964, speeds were increasing. Various improvements
were also made to the specification, in 1961, 1963,
and 1965. (In particular, COBOL 60 included several
logical flaws which were cleared up in the 1963
version.) By 1970, COBOL was the most-used
programming language in the world.
However, in the 1970s it came in for a certain
amount of criticism as ideas of structured
programming were developed. COBOL can be written
in a structured way, but COBOL programs of the time
often relied heavily on the GO TO statement. Attempts
to rewrite code used the PERFORM statement (as
used in the tutorial code below), but this didn’t always
work as clearly as it might. There was also no way to
pass parameters into a procedure/function, which It works! Our Hello World program in GnuCOBOL.
www.linuxvoice.com 101
CODING COBOL
PROCEDURE DIVISION.
Begin.
DISPLAY “Enter title”.
ACCEPT BookTitle.
DISPLAY “Enter author”.
ACCEPT BookAuthor.
DISPLAY “Have you read the book? (Y or N)”
ACCEPT BookRead.
ACCEPT BookFinishDate FROM DATE YYYYMMDD.
DISPLAY “Title is “ BookTitle “ author is “ BookAuthor.
DISPLAY “Date is “ BookFinishDate.
STOP RUN.
COBOL programs have four divisions:
1
Identification Division Identifies the code, and in OO
type COBOL, contains the class/interface definition.
2
Environment Division Contains configuration and
input/output setup.
Writing to file – Note that
business logic that COBOL does very well. COBOL is 3
Data Division Sets up all program data, in multiple
odd spacing, which you
can fix with the TRIM unlikely to be going anywhere in the near future. sections (files, working-storage, local-storage,
function. The best bet for running COBOL on Linux is GNU linkage, report, and screen).
Cobol, formerly known as Open COBOL and still 4
Procedure Division Contains the working code, in
often packaged under that name. Get it from http:// sections and paragraphs.
sourceforge.net/projects/open-cobol or via your The WORKING-STORAGE SECTION of the DATA
distro’s package manager (in Debian the package is DIVISION contains any program-internal data; here,
open-cobol). the fields that describe each book. ID will be auto-
Once you’ve installed it, here’s a Hello World generated; title and author will need to be supplied for
program to save as hello: each book; as will whether the book has been read
IDENTIFICATION DIVISION. or not; and the date that it was finished on will also
PROGRAM-ID. HELLO. be stored. Note that each field has a PIC (picture)
PROCEDURE DIVISION. clause. This gives the type and length of the field. X
DISPLAY ‘Hello World!’. stands for a character and 9 for a digit, and the length
STOP RUN. of the variable is given in brackets. So X(100) is a
100-character string.
Compile it with cobc -free -x -o hello-exe hello. The The PROCEDURE DIVISION is the actual code of
-free argument indicates you are using free source the program. DISPLAY displays text to the screen, and
code format (see boxout). -x tells the compiler to ACCEPT accepts user input and stores it in the given
build an executable program, and -o NAME saves the variable. ACCEPT name FROM DATE YYYYMMDD
executable as the given name. Execute with ./hello. uses a built-in function to get today’s date and store
it in the specified variable, in the given format (eg
Entering and displaying information 20150324 for 24 March 2015). We’ll then display the
Since COBOL is designed to handle data well, a stored data again before finishing the program with
database-type program seems like a good fit to try it STOP RUN.. Note that all COBOL statements in any
out. We’ll try a book database, storing book title, section of the program end with a full stop.
author, whether or not you’ve read the book, and date Compile and run with cobc -free -x -o book-
finished if so. exe books; ./book-exe. On running it, you’ll see an
Here’s a first iteration, which defines the data unattractive space between the end of the book title
formats, asks for information, and prints it: and the “author is” output. This is because the book
IDENTIFICATION DIVISION. title is saved as a 100-character string, and COBOL
PROGRAM-ID. Books. automatically pads the string to that length with
AUTHOR. Juliet Kemp. spaces. To avoid outputting those spaces, you can
use OpenCOBOL’s TRIM function:
DATA DIVISION. DISPLAY “Title is “ FUNCTION TRIM(BookTitle) “ author is “
WORKING-STORAGE SECTION. FUNCTION TRIM(BookAuthor).
01 BookDetails. The code so far just takes a single entry and
02 BookId PIC 9(7). outputs it; it doesn’t store it. Let’s write it to a file:
02 BookTitle PIC X(100). ENVIRONMENT DIVISION.
02 BookAuthor PIC X(50). INPUT-OUTPUT SECTION.
02 BookRead PIC X(1). FILE-CONTROL.
02 BookFinishDate PIC 9(8). SELECT BookFile ASSIGN TO “books.dat”
102 www.linuxvoice.com
COBOL CODING
ORGANIZATION IS LINE SEQUENTIAL.
DATA DIVISION.
FILE SECTION.
FD BookFile.
01 BookDetails.
* remaining sections of BookDetails as above
WORKING-STORAGE SECTION.
01 IdCount PIC 9(5) GLOBAL.
PROCEDURE DIVISION.
Begin.
OPEN OUTPUT BookFile.
DISPLAY “Enter each book as requested below. Enter no data to
end.”.
PERFORM GetBookDetails.
PERFORM UNTIL BookTitle = SPACES
WRITE BookDetails
Outputting and entering
PERFORM GetBookDetails to file, then PERFORM again asks for a new set
information. You need to
END-PERFORM. of information. Note that the first PERFORM call hit return through all three
CLOSE BookFile. is outside the loop, so the first WRITE call has fields to finish, which is
STOP RUN. something to write. This avoids writing a blank line something else that could
GetBookDetails. at the end of the file. Note too that there is no full be improved.
DISPLAY “Enter title”. stop until after END-PERFORM; PERFORM...END-
* get title, whether or not read, and finish date as before PERFORM is a single statement. Finally, remember to
ADD 1 TO IdCount. close the file at the end before stopping the program.
SET BookId TO IdCount. In COBOL, a paragraph is a block of code, of one
or more sentences, labelled with either a language-
This time there is an INPUT-OUTPUT SECTION, defined or a programmer-defined name. It continues
which controls the files used. The label BookFile until the next section or paragraph is encountered
is assigned to a specific filename, and the file (or the end of the code). So FILE-CONTROL. above
organisation is line sequential, which means that it labels a paragraph, and so does GetBookDetails.,
must be read line by line in the given order. (Files can which comes after the main body of the program.
also be indexed, which allows random access by key.) This paragraph mostly does the same as in the first
The data fields are the same, but under the FILE version of the code. However we also use the global
SECTION instead of the WORKING-STORAGE counter to set the BookId field. (Add one to it first to
SECTION, which just contains a global variable to start at one.) In effect this is a lot like a function, but
act as a counter. The FILE SECTION describes the unlike functions in most other languages, you can’t
records of any files used by the program, whereas pass parameters into it; instead it just has access to
the WORKING-STORAGE SECTION defines program- the global program variables.
internal variables, which will not be written out to a file. If you compile this and run it a couple of times,
The file is opened for output, and the initial user you’ll see that as stands, it overwrites the books.dat
information given. Then we use the PERFORM file each time. You can use OPEN EXTEND BookFile
keyword to call the GetBookDetails paragraph. to add to the end of the file, but this will fail if the file
PERFORM UNTIL provides a loop in which the doesn’t already exist. It isn’t perfect; we’ll leave it in as
program continues to ask for input until it gets an exercise for the reader to see if you can fix this.
an empty title field. WRITE writes the information A great resource to find more code to look at and
experiment with is the University of Limerick’s sample
programs (www.csis.ul.ie/cobol/examples), which
Free vs fixed format
were a great help to us when writing the tutorial
As with other languages of a similar age, COBOL originally code. There are also useful links at the University of
had a fixed format structure, in which source code was Michigan COBOL page (http://groups.engin.umd.
written in lines of 72 characters, consisting of a sequence umich.edu/CIS/course.des/cis400/cobol/cobol.
number, an indicator area, area A (in which section or
html). It may have a slightly elderly feel to it, but
paragraph names begin), and area B (in which other
code sentences begin; so code is indented). By default, COBOL is still an interesting, robust, and surprisingly
OpenCOBOL still expects this format. common language.
However, modern COBOL (since 2002) also accepts, with
the -free flag, free-format source code, which does not have Juliet Kemp is a scary polymath, and is the author of
the same limitations. The code here all uses free format. Apress’s Linux System Administration Recipes.
www.linuxvoice.com 103
CODING NINJA
GET STARTED WITH
TUTORIAL
GNOME BUILDER
We take a look at a new, clean and particularly awesome
GRAHAM MORRISON
integrated development environment for Gnome.
M
any people consider Gnome the closest Meanwhile, Qt and KDE developers get both
WHY DO THIS?
thing to a default desktop for Linux, but KDevelop and the wonderful Qt Creator, and there’s
• Install a cutting edge
Gnome and its toolkit, GTK, have never had Eclipse for everyone else, although we admit that
IDE with a wonderful UI
a great set of developer tools, nor a modern integrated Eclipse would easily win any competition for the most
• Create your own GTK+
and Gnome apps developer environment (unless you include Vim and intimidating IDE. But with Gnome’s current focus on
Emacs). Glade is still an excellent tool for user interface usability and user experience, it seems fitting that
design, but it doesn’t help developers take the next there should be a better option, and one that’s more in
step. Anjuta isn’t bad either, and it does integrate well tune with Gnome’s new principled user interface. And
with Glade. But Anjuta hasn’t been developed for a now there is – Gnome Builder. Let’s take a look at the
while and can be intimidating. future of Gnome development…
Step by step: Manage your projects with Builder
1
Installation 2
Open a project
Gnome Builder is the brainchild of Christian Hergert, For non-Fedora users, there’s also a PPA for Ubuntu
and the product of a phenomenally successful and Gnome Builder is easily installed through Arch. We
Indiegogo crowdfunding campaign. Launched in installed version 3.16.3, and while Builder’s user
December 2014, the campaign asked for $40,000 to interface is already well thought out, it’s also being
enable Christian to work full time on the project, after rapidly developed, so it’s possible that features may
he’d already quit his job and put four months into the have changed slightly by the time you read this. The
prototype. The campaign reached its target by raising best way to get a taste of what Builder can already do
$55,204, and the rate of development since the is to use its text editor because we think it’s already
campaign’s conclusion has been incredible – so one of the best we’ve seen.
much so that Gnome Builder has had several releases Builder is a strict adherent to Gnome’s minimalism,
and is already functional. It’s got a few rough edges which you’ll notice when you launch the application
and many features yet to be implemented, but it’s for the first time. Before you’ve even said hello it lists
great fun to work with, especially with its inspired any automake projects it finds within your home
design. It’s the perfect excuse to try a bit of Gnome folders, and you can simply click on one of these to
development, which is exactly what Christian wanted open a project. Alternatively, you can click on ‘New’ in
to achieve. Installation can be tricky if you don’t want the top-left. Here you get to select an existing folder,
to compile it yourself, which is fair enough for an empty or not, and call that a new project, or enter the
application designed for developers. The simplest Git URL of a project you’d like to download. We started
solution is to use the shiny new Fedora 22, where with an empty folder, which will then give you an
Builder is already in the repositories. empty editor view on the right.
104 www.linuxvoice.com
NINJA CODING
3
User-interface tour 4
Get editing!
There are three ways to configure what you see. Click Another of Builder’s nicest features is a built-in real
on the top-right menu, and the View option listed time preview for HTML and markdown documents –
enables and disables the left-hand panel, which is the latter being especially impressive, as there isn’t a
used to navigate files of a project folder. Beneath this, decent native open source Markdown editor for Linux
labelled ‘Plain Text’ by default, is a button that enables we know of. Markdown is an incredibly useful way to
you to specify the type of the file you’re editing. add formatting information to pure text files by using
There are more options here too, for adding things a variety of simple and intuitive symbols that don’t get
like line numbers or the excellent auto-indentation. in the way of the legibility of the original text.
Finally, our favourite features are hidden beneath To see the preview in action, start a document
the filename of the file you’re editing, just above the like this in the editor. If Builder doesn’t automatically
editing window. Click on this and another small panel change the file type, make sure you change it to read
appears. The top row of icons lets you split the view ‘Markdown’, or HTML if you’d rather preview that. Now
horizontally or vertically, allowing you to have more click on the filename to open the split window view
than one file open at a time, or the same file but and add a vertical or horizontal split. Select the panel
different locations. You can use the arrows beneath to you want to be the preview and use the same menu to
move between open files. enable the ‘Preview’ option.
5
Get coding! 6
Change the preferences
The easiest way to get a build environment working You can press F4 to switch between the source and
and start playing is to download a simple Gnome the header file, but there are more options in the
project with an environment in place. We went with preferences panel if you need to change the keyboard
gnome-calculator, which can be grabbed as a tarball layout, including both Emacs and Vim emulation.
from download.gnome.org. From a command line in There are options for all of the programming
the untarred calculator folder, type ./configure to languages that the editor supports. Each can have its
generate the autotools scripts. You can then type own margins and indentation. But our favourite
make to build the project, or use the side panel in feature is the night mode, which can be enabled from
Builder. You can now take advantage of all the features the ‘Theme’ page of the preferences panel. The Builder
currently implemented, including error highlighting, Dark theme is perfect for late-night coding sessions,
auto-completion for C and C++ and the global search. which are something Builder has inspired us to look
This will list everything related to the project and let into. Builder may still be at only an early stage of
you skip between search hits within your files, but it development, but the small team has already
also links to API references which will appear in a new produced an excellent IDE with a lovely, minimal and
panel to the right of your code. deceptively comprehensive user interface.
www.linuxvoice.com 105
TUTORIAL JAVASCRIPT
JAVASCRIPT: CREATE A SECURE,
TUTORIAL
ANONYMOUS CHATROOM
BEN EVERARD
Build a fully functional web app with enterprise-level
security using just one language.
T
his month we’re looking at privacy online, and
WHY DO THIS? to help with this we’re going to create a web
• Get started with Node.js, chatroom with end-to-end encryption and
the trendiest web tech
of 2015 anonymity. To keep things simple, we’ll do everything
• Learn to use web in a single language: JavaScript. This language has
sockets for two-way traditionally been used to add small functions to
communication in a web websites, but it’s grown considerably and is becoming
browser
popular on the server thanks to Node.js.
• Develop a secure
messaging service and Node.js is an event-driven non-blocking platform.
keep your chats private This makes it particularly suitable for interactive web
apps that have two-way communication between the
browser and the server. There are two bits of software
you’ll need to install before you can get started, A triple hello world using Node.js, Express.js and
Node.js itself and npm (the Node Package Manager) Socket.IO all in a single web app.
for installing the additional modules we’ll need.
You should find both in your distro’s repositories, everything a little cleaner, we’re going to use the
though make sure that you’re installing the right Express.js framework. You’ll need to install that with:
node software as there’s also a radio application sudo npm install express --save
called Node. On Ubuntu-based distros, Node.js is in a As is tradition, we’ll start our journey into Node.js
package called nodejs, and you can install everything with ‘Hello World’. To do this, we’ll need to start up a
you need with: server in Node.js that serves a single page which just
sudo apt-get install nodejs npm contains the words ‘Hello World’.
Node is based on the JavaScript engine from The Node.js code for this is in a file called app.js:
the Chrome browser, so the basic structure of the var app = require(‘express’)();
If you want to learn more
language is identical. However, it also comes with var server = app.listen(9999);
about Node.js, NodeSchool
has online tutorials, and additional features to let it do things like run a web app.get(‘/’, function (req, res) {
real-world workshops to server and read the filesystem. res.sendFile(__dirname + ‘/index.html’);
help you get started: Node.js does come with a web server that you can });
http://nodeschool.io. use without any additional modules, but to make The first line imports and initialises the
Express.js framework. The second line creates a
server that’s listening on port 9999 (you can use a
different port if you prefer, but our test machine is
running other servers on the normal web ports).
In Express.js, you have to tell the server what to
return for each path using app.get (there’s also app.
post, but we won’t be using that in this tutorial). app.
get() takes two parameters: the first is the path that
the browser requests; the second is the function
that is used to process this request. As you can
see, this is an anonymous function that itself takes
two parameters: the request and the response. The
request can be used to get more information about
the HTTP request, and the response is used to
formulate what is sent back to the browser. In this
case, we just send the file index.html that’s located in
the same directory that Node is being run from.
JavaScript code often uses functions passed
as parameters like this, known as callbacks. When
they’re used well, it can make it easy to write event-
106 www.linuxvoice.com
JAVASCRIPT TUTORIAL
driven code. When they’re used badly, they can lead to
unreadable spaghetti code.
That’s the Node.js code sorted. We just need the
HTML file called index.html, which in this case is
really simple.
<html>
<head>
<title>Hello World</title>
</head>
<body>
<h1>Hello World!</h1>
</body>
With those two files created, you can start Node with:
nodejs app.js
Then, if you point your web browser to http://
localhost:9999, you’ll see the Hello World web page.
Going both ways
The Open Source Web
Now, let’s move on to our chat application. The web event in the connected client (more on that later), and
Applications Security
has a client–server model where a client (a web second, it creates another callback event specific to Project (OWASP) has some
browser) requests some data (a web page) from a this socket. In this callback, every time a reply event resources to get you find
server. That content is sent, and the connection is triggered, it logs the output. This logged data will and stop XSS attacks:
finishes. For a chat session, though, we need to keep appear in the terminal where Node.js is running. www.owasp.org/index.
a line of communication open between the browser The code in index.html should be changed to: php/Cross-site_Scripting_
and the server, and have some way of pushing data <html> (XSS).
back and forth between the two in a manner that’s <head>
more like peer-peer software than client–server <title>Hello World</title>
software. For this we’ll use Socket.IO, a JavaScript <script src=”/socket.io/socket.io.js”></script>
framework with components for both the browser <script>
and the server. var socket = io.connect(‘http://localhost:9999’);
First, you need to install Socket.IO for Node.js with: socket.on(‘news’, function (data) {
sudo npm install socket.io console.log(data);
Now we need to add both the server and the socket.emit(‘reply’, { my: ‘Hello World!’ });
browser parts of Socket.IO to our Hello World app. });
First the server part. Change app.js to: </script>
var app = require(‘express’)(); </head>
var server = app.listen(9999); <body>
var io = require(‘socket.io’)(server); <h1>Hello World</h1>
app.get(‘/’, function (req, res) { </body>
res.sendFile(__dirname + ‘/index.html’); This loads the Socket.IO JavaScript library from
}); the server (this path is automatically set up when
io.on(‘connection’, function (socket) { you initialise Socket.IO in Node.js). This connects to
socket.emit(‘news’, { hello: ‘world’ }); the server, and creates a callback for the news event.
socket.on(‘reply’, function (data) { This is the news event that’s triggered by the socket.
console.log(data); emit line in our code. In the server code, we sent some
}); JSON data ({hello: ‘world’}). This is the data that’s
}); the parameter of the callback function. The callback
Socket.IO, like Express, uses callbacks to set sends this data to the console log, and then calls emit
functions that should run on certain events. The io.on to trigger a reply even in the socket on the server.
line (‘connection’, … is used to set callbacks that The console log in the browser can be viewed in
happen to every single browser that connects to the the developer tools.You can enable these in Firefox or
server. In this case, the line: Chrome by pressing Ctrl+Shift+I.
io.on(‘connection’, function (socket) { You can run this code exactly as before. You’ll need
creates an anonymous callback function that runs to end (with Ctrl+C) and re-run nodejs app.js, and then
every time a new client connects to the server. When reload http://localhost:9999 in your browser. This
a client connects, it creates a new socket, and it’s this time, though, you’ll get triple Hello World. You should
socket object that’s passed as a parameter to this see the greeting in the main browser window, in the
callback function. browser console and in the terminal where you’re
The callback function does two things. First, it sends running Node.js.s
(or emits) a chunk of data that will trigger a ‘news’ We now have all the pieces we need to build our
www.linuxvoice.com 107
TUTORIAL JAVASCRIPT
var socket = io.connect(‘http://localhost:9999’);
socket.on(‘news’, function (data) {
document.getElementById(“chat”).innerHTML +=
data.data + “</br>”;
});
function send() {
socket.emit(‘news’, {data:document.getElementById(“msg”).
value});
}
</script>
</head>
The two parts to this are setting a callback for
news events and the send function. When this page
receives a news event, it just adds the content of the
data portion of the JSON object to the chat <div>, and
appends a line break. The send function creates a
news event on the server, and attaches a JSON object
that includes a data element which contains the
content of the msg text input.
That’s all you need to create a simple chat server.
The final app sending
chat application. There’s a method for serving the If you make these changes, and restart nodejs app.
encrypted messages
between Chrome and HTML page and a method for sending data back and js, you’ll be able to chat between two people using the
Firefox. Take that NSA! forth between the server and the browser. web page. To test this out, open two web browsers
(such as Firefox and Chrome), and connect both of
Pulling it together them to http://localhost:9999, and you should be
The server side of our chat program is really simple able to send messages between them.
– all we need to do is receive data from a client, and
then send it out to all clients. This is done by replacing Securing the data
the io.on connection callback with the following: What we’ve just created is probably the least secure
io.on(‘connection’, function (socket) { chat tool ever. Not only can the server see every
socket.on(‘news’, function (data) { message that’s being sent, so too can anyone else on
io.emit(‘news’, data); the network and anyone else who happens to connect
}); to the server. What’s more, there’s no authentication,
}); so you’ve no idea who’s sending messages.
The browser code is a little more complex. This We said at the start that we would implement end-
has to accept user input to forward to the server, and to-end encryption to guarantee user’s privacy, and that
receive other users’ chat data and display it on the means that we have to encrypt the messages in the
screen. We’ll look at this in two parts. First, the HTML web browser, and not decrypt them again until they
for the web page <body> section is used for the user reach to the destination browser. Since the server just
interface. This should be: passes data around, it doesn’t matter if this data is
<body> plaintext or ciphertext, so we can add this client-side
<h1>Chat!</h1> encryption without changing the server at all.
<p>Welcome to Linux Voice’s anonymous encrypted chat. </p> For the encryption, we’ll use CryptoJS. This is
<div id=”chat”> a library that implements a number of standard
</div> encryption techniques. The project is hosted on
<input id=”msg” type=”text”> Google Code (https://code.google.com/p/crypto-js)
<input type=”button” value=”Send” onClick=”send();”> however, Google Code is shutting down, and there’s
This has a <div> with the ID chat. We’ll use this to
display the messages that we get from the server. Express.JS Getting to grips with a Node.js web framework.
There are also two inputs: a text field and a button.
The text field is where the user enters their message, We’ve barely touched on the power of the Express.js
and the button runs the send function when it’s framework, which can do far more than just serve up HTML
clicked to send this message to the server. pages. It’s based on the concept of middleware. In Express.
js, middleware is a series of functions that run one after the
The <head> section of the page then contains
other that can all access the request and response objects.
the JavaScript code to control these (nb this comes One could, for example, log the request, while another
before the body section in the index.html file): could make a decision based on the device that sent the
<head> request. At the end of the middleware stack, the response
<title>Linux Voice anonymous chat</title> object should be fully formed and can be sent. There’s more
information, and a guide to help get you started, on the
<script src=”/socket.io/socket.io.js”></script>
express.js website: http://expressjs.com.
<script>
108 www.linuxvoice.com
JAVASCRIPT TUTORIAL
a mirror on GitHub (https://github.com/sytelus/ the same encryption key. In normal usage, this would
CryptoJS). be text, but a malicious user could enter some HTML
First we need to add a text box for the user to enter code, or even JavaScript, that attacks the other users.
the encryption key into the <body> section of the For example, if someone enters:
HTML. You can add this directly under the <h1> line: <b onmouseover=alert(‘helloworld’)>click me!</b>
<p>Encryption Key: <input id=”key” type=”text”></input> They can inject a JavaScript alert (or potentially any
Then you need to include the JavaScript library by other code) into the other client’s browsers. This is
adding the following script line just below the <title> known as a Cross Site Scripting (XSS) attack. In order
line in index.html: to prevent them, we need to encode the contents of
<script src=”http://crypto-js.googlecode.com/svn/tags/3.1.2/ the textbox so that it shows as text, rather than HTML.
build/rollups/aes.js”></script> To do this we need an encoder. We’ll use the one at
Then you can change the contents of the main www.strictly-software.com/scripts/downloads/
<script> tag to; encoder.js rather than creating one ourselves. To use
<script> this, you need to change the socket.on callback in
var socket = io.connect(‘http://localhost:9999’); index.html to:
socket.on(‘news’, function (data) { socket.on(‘news’, function (data) {
console.log(data); console.log(data);
var decrypted = CryptoJS.AES.decrypt(data.data, document. var decrypted = CryptoJS.AES.decrypt(data.data, document.
getElementById(“key”).value); getElementById(“key”).value);
if (decrypted.toString(CryptoJS.enc.Utf8) != “”) { if (decrypted.toString(CryptoJS.enc.Utf8) != “”) {
document.getElementById(“chat”).innerHTML += var safeString = Encoder.htmlEncode(decrypted.
decrypted.toString(CryptoJS.enc.Utf8) + “</br>”; toString(CryptoJS.enc.Utf8));
} document.getElementById(“chat”).innerHTML +=
}); safeString + “</br>”;
function send() { }
var encrypted = CryptoJS.AES.encrypt(document. });
getElementById(“msg”).value, document.getElementById(“key”). If you do this, whatever the users enter (special
value); characters and all) will appear in the window, and any
socket.emit(‘news’, {data:encrypted.toString()}); code won’t be executed. If you make these changes,
} and restart your app, you should find that the XSS
</script> attack no longer works.
As you can see, this works in exactly the same We set out to create a private and anonymous chat
way as the unencrypted chat, but it uses CryptoJS’s system. Our simple app hasn’t been fully vetted by
encrypt and decrypt functions to protect the security experts, but we think that it fulfils this role
messages using AES encryption before sending well. It’s private because it has end-to-end encryption
them. This is a highly secure encryption standard and that’s supplied by (what we believe to be) a secure
should keep your messages safe from prying eyes. cryptographic library. It’s
Note that we’ve made no attempt to organise key anonymous because
exchange – this has to happen offline. Before two if you connect through “Our simple app hasn’t been
people can start chatting, they have to first agree on a
key, then both go to this website at the same time (it
Tor, no-one can tell
if you’re connected
fully vetted by security, but
doesn’t store messages, so if two users aren’t online to it, and anyone we think it fulfils its role well.”
at the same time, the message is lost). monitoring the server
(even the server operators themselves) can’t tell who
What about security? is speaking with whom. They only see encrypted text
The server will send all messages to all the browsers go back and forth, and have no way on knowing who’s
connected; however, there’s no guarantee that all decrypting which messages. Because it’s limited to
browsers are using the same key to communicate. text, it should scale well, but it won’t scale indefinitely.
This is deliberate, and enables several groups of It requires each user to have enough bandwidth to
people to communicate anonymously on the same receive every message, and the server to be able to
board. You will only be able to see the messages of send every message to every connected person.
people using the same key as you, but since all We should point out at this point that we’re relying
messages are sent to everyone, anyone monitoring on CryptoJS and the Encoder for security, and we
the board can’t prove who is chatting with whom. In haven’t fully vetted them for high security. If you want
fact, a spy can’t even prove that you’re chatting with to test out this app, or make any changes, you can
anyone (even if there’s data leaving your machine, find the full code for the finished web app on GitHub
they don’t know if it’s ever being decrypted). at https://github.com/linux-voice/issue17-node.
There’s a problem with this board. It will insert
whatever a user types in the text box directly into the
Ben Everard isn’t paranoid – they really are all out to get him.
HTML of every other connecting client that’s using
www.linuxvoice.com 109
MASTERCLASS PICARD
MASTERCLASS
Undo the wrongs of your 90s self by tagging your anarchic
BEN EVERARD music collection correctly.
TAKE CHARGE OF THE UNRULY
LIBRARY WITH PICARD
If your music’s grown like the Borg, you need a Picard to control it.
W
ho doesn’t love the sound of DRM-free
MAYANK SHARMA music, or the gigabytes of music you’ve
legally ripped from CDs and perhaps even
LPs for your own listening pleasure? However, the one
side-effect of owning so much music is
disorganisation. Picard is a nifty little app that can help
you get your music files back in shape. The app is
designed to sort your music library and fill in missing
tags, rename oddly named files and easily identify
incomplete albums. Picard is developed by the
MusicBrainz project, which is an online database that The idea with Picard is to get all your tracks into the
PRO TIP captures and hosts various information about artists right-most pane.
If you have dozens of and their recordings including track titles, album titles,
albums you may want to length of each track, cover art, and more. According to button on the toolbar. Depending on the album
break the process down
statistics published by the project, their database metadata present for each of the files, Picard will
into several runs, at least
until you’re comfortable contains information about 950,000 artists, 1.5 million group all songs from the same album together. Files
with Picard. releases, and 14.8 million recordings. that don’t belong in a clustered album will be listed as
Unmatched Files.
Picard to the bridge Review the clusters, and if a file has been wrongly
The software has a simple layout, though you’ll have added, you can drag it to another cluster or back into
to use it a couple of time to inculcate its methodology. the Unmatched Files folder. Similarly, you can
Start by adding a handful of manually drag any
music files. Picard supports all
popular formats including
“Picard is a nifty little app unmatched files into the
correct cluster if you know
Ogg, Speex, Opus, Flac, MP3, that can help you get your they belong there.
WMA and WAV. You can drag
the files from the file manager
music files back in shape.” Once you’ve clustered the
files based on the existing
into the left-hand pane of metadata, you can either click
Picard’s main interface, or head to View > File Browser Lookup or Scan for matching the files with the online
to search your filesystem from within Picard itself. database. The difference between the two options is
Any new files you add will initially be added under that a lookup is done on an entire cluster at once, and
the Unmatched Files folder. Click on one of these files uses whatever existing metadata is already in the files
to view its current metadata in the bottom panel. You to query the database. It’s quite fast and usually gets
can right-click on any of the fields and select Edit to the job done. If Picard’s lookup doesn’t work, you’ll
manually modify the tags. have to scan the files. Scanning is done on a file-by-file
The first step to get your music organised is to process, and uses each file’s audio fingerprint. It takes
cluster all files from the same album under one much longer, but it works well.
heading. For this, select the Unmatched Files folder or After fetching the relevant information, Picard
any of the files you’ve just added and click the Cluster displays the name of the album that every song
110 www.linuxvoice.com
MUSIC MANAGERS MASTERCLASS
belongs to in the panel on the right. The app makes Head to the AcoustID
intelligent guesses to pair the track with an album. It website and get an API key
also removes these songs from the Unmatched Files to contribute signatures of
list and moves them into this new album entry in the new tracks to its database.
panel on the right. Expand each album to view your
track (or tracks), which will be marked with a small
colour-coded rectangular icon. Green is a good match;
yellow, orange and red represent increasing degrees
of uncertainty. It also uses a colour-coded scheme to get the hang of its basic operations, you might want
point to album completeness. Complete albums are to tweak some its options for a better experience.
shown with a golden icon. The ones with a silver icon Head to Option > Options to access all of Picard’s
are missing some tracks, which is clearly noted next settings under various tabs. The Automatically Scan
to the album’s name. All New Files option under the General tab will save
Inspect each album and track. If the software has you a click by automatically looking up tracks as soon
identified it correctly, click on the Save button. This as you add them in Picard. Then there’s the Metadata
tells Picard to attach the new metadata to this song. tab, which has an option to translate foreign artists’
As a visual indicator, the coloured rectangle changes names into English. If you shuttle your tracks between
to a green checkmark to show the track information devices, switch to the Tags tab and make sure you set
has been saved. the Tag Compatibility to 2.3, which has a broader
support that the newer v2.4.
Disengage autopilot The most interesting option is under the File
For songs that are matched incorrectly, you may find Naming tab. Although Picard uses the metadata to
they’ve been confused with another track in the same update each file’s tags, you can also have it rename
album. In this case, you can manually rearrange the the files and place them in folders according to the
tracks into the correct order by dragging and dropping naming scheme you see fit. This functionality is
them. If the data is completely wrong, however, select disabled by default. You can enable it by toggling the
the song and drag it back to the Unmatched Files Rename Files When Saving option under the File
folder. To improve the chances of detection, add some Naming tab.
information to the song yourself (the artist’s name, Furthermore you can also customise how Picard
track length or album name are helpful). After that, formats the file name by specifying a pattern in PRO TIP
click on the Lookup icon in the toolbar at the top again Picard’s own scripting language.If you use the Drag a directory from the
and Picard is more likely to find an appropriate match. %artist% - %title% pattern, Picard will format the name file browser to an album
If the program can’t find the data automatically, you as The Beatles – Ticket to Ride.mp3. When you – this attempts to match
all the files from the
can also find track information manually by right- specify a pattern, Picard will preview how it will name directory to the album.
clicking the file and selecting the Lookup in Browser the files using sample tracks in the Examples section
option. This will fire up your browser and point you in below the pattern window.
the direction of the MusicBrainz database, which Here’s an interesting naming pattern adapted from
shows you a list of all possible matches. You can also an example we found on the web:
query this database by visiting the MusicBrainz $replace($if($eq($left(%albumartist%,4),The),%albumartistsort
website at musicbrainz.org. When you find an entry %,%albumartist%)/%albumartist%-$left(%date%,4)-
matching your track, click the Tag button adjacent to %album%$if(%discnumber%,-CD %discnumber%,)/$num(%track
the album name and a new folder should pop up in number%,2)-%title%,:,)
the right-hand side of the Picard window. This string will first generate a folder per artist using PRO TIP
When you’re done, hit the Save button in the toolbar the Album Artist tag. If the tag begins with the word Here’s a list of Picard’s
to associate the new metadata with the files. “The” it’ll strip that string and stick it at the end. This internal tag name and
their equivalents in other
You don’t need to tweak Picard in any way to fill in means “The Rolling Stones” will be filed under “Rolling
tagging formats (http://
the tags for your music collection. However, once you Stones, The” which makes it easy to sort the library in picard.musicbrainz.org/
the regular folder browser. docs/mappings).
Underneath the artist’s folder we ask Picard to
create one folder for each album. This folder will be
named in the format <Album Artist>-<Year>-<Album
Title>. If the album has multiple discs, the disc
number will be inserted into the folder name, such as
The Rolling Stones-1971-Sticky Fingers-CD 1.
When you hit the Save button after enabling the file
renaming option, Picard will save the tags and also
rename your tracks as per the naming pattern. You
can find various examples of Picard’s renaming
You can extend Picard’s already impressive functionality patterns on its wiki; it does take some getting used to.
by enabling a variety of plugins. but will help you better sort and organise your music.
www.linuxvoice.com 111
MASTERCLASS MUSIC MANAGERS
FIX THE TUNES FROM THE CLI
Beat your library into shape with Beets.
P
icard is a fabulous application. But its biggest
shortcoming is that it’s a graphical app! If you
MAYANK SHARMA want to fix the incomplete tags in your music
library from the confines of the familiar, versatile and
venerable command line interface, you need Beets,
which bills itself as the music geek’s media organiser.
Beets is available in the repositories of most popular
distributions. However, the version in your distro’s
repository might not be the latest. The recommended
way to install Beets is via PIP, which is a package
management system used exclusively for installing
PRO TIP packages written in Python. On Deb-based systems Beets resolves conflicts between settings in the config file
The tool is called Beets such as Ubuntu, install PIP and its dependencies with and the CLI by going with the latter.
but you interact with it sudo apt-get install python-pip
using the beet command. and on RPM-based distros with After creating the config file, we need to import our
sudo yum install python-pip music collection into Beets. By default, the tool
Once installed, you can use PIP to install Beets with assumes that we’ll organise all our music under the
sudo pip install beets directory specified in the configuration file. You can
either copy the music from where it resides currently,
Configure Beets or you can move the music to save disk space.
Unlike most command line tools that ship with a If you wish to move the files into the specified
working configuration file, you have to manually directory, add these lines to the configuration file:
create the configuration file for Beets. The import:
configuration is stored in a text file called config.yaml move: yes
that’s placed under the ~/.config/beets/ directory. On the other hand, if you don’t wish to change the
The configuration file will grow with time as you location of your music files, you can specify the path
become more familiar with Beets. To begin with, start to your music files in the directory option and use the
with the following entries: following lines to inform Beets that you don’t wish to
directory: ~/Music/beets-music copy or move files from the current location:
library: ~/Music/beets-music/musiclibrary.blb import:
Make sure you create the ~/Music/beets-music/ copy: no
directory beforehand. The directory option points to move: no
the directory where you wish to store your music Make note of the indents. The configuration file is in
collection. Remember, this isn’t the path to your the YAML language, which accepts spaces (and not
existing music collection. Rather, this is the directory tabs) to indent some lines. Refer to the official
Use the beet fields
command for a complete where Beets will store your music after the tool has documentation for more configuration options
list of items and album imported and reorganised it. The library path is where (http://beets.readthedocs.org/en/v1.3.13/reference/
fields that you can use in Beets will store the database file that stores the index config.html)
your queries. metadata of your music files.
Import music
With the configuration file in place, you’re finally ready
to import files. The command
beet import /path/to/music/files
will import your music. The command also copies or
moves the files into your specified directory
depending on the import options you’ve specified in
the configuration file. As the import command brings
in your music, it’ll also fix and fill any gaps in the
metadata from the MusicBrainz database.
If it finds multiple choices for a track, Beets will let
you select the one that matches the track. If none of
the candidates match your album, press U, which tells
Beets to import files as it is. Note that the import
process does not produce any output on the screen,
112 www.linuxvoice.com
MUSIC MANAGERS MASTERCLASS
If you’d like to fetch lyrics for songs, retrieve cover
art for albums, and provide new metadata sources,
and more, while Beets imports your music, you can
enable the concerned plugins by editing the config file.
$ nano ~/.config/beets/config.yaml
plugins: lyrics fetchart scrub
The plugins: line expects a space-separated list of
all the plugins you wish to enable. In the above
example, the Lyrics, FetchArt and the Scrub plugin are
enabled. You can also use the
beet lyrics <song name>
command to manually search for lyrics for a song.
The web interface is pretty basic but gets the job done. Beets will automatically store the lyrics in the
database. You can then use the
except for when it lists the possible candidates based beet lyrics -p <song name>
on metadata. The process can take a long time, so if command to print the lyrics on the screen. Beets will
you wish to import a large selection of files in one go, first search for the lyrics in the database, and if it
and don’t want to be prompted again and again for doesn’t find a match, it’ll fetch the lyrics from the
the metadata information, use the -A option, such as online sources.
beet import -A /path/to/many/music/files/ Beets also has a simple web UI. To use the web
which asks Beets not to auto-tag the files and is much interface you need the Flask framework, which you
faster. can install with
If you haven’t modified the configuration file to sudo pip install flask
instruct Beets on how to handle the imported music, Then put web in the plugins line in the configuration
it’ll stick to its default behaviour and copy the music. file and start the web server with
Use the -C option when importing music to ask Beets beet web
to update the tags without copying the music. Now launch your web browser and head to http:// PRO TIP
localhost:8337 to access the interface. Using the web Use beet ? import to list
Browse the library interface you can search through your imported all the import options that
After importing the files, you can use the beet ls music collection. Click on a song from the results to you can use.
command to query the music library. This command view its metadata including the lyrics if you’ve enabled
expects a query string, and if you don’t specify any the plugin and fetched them. The web interface also
delimiters, it will search all the metadata files for the has basic controls to play and pause music.
supplied query string. So for example, if you use the Beets can also fingerprint your music and query the
beet ls Rocky II command, Beets will list all songs AcoustID database to find a match. First grab the
where both the words Rocky and II appear in the Chromaprint library (https://acoustid.org/
metadata, whether in the title, album, artist, and so on. chromaprint) for your computer’s architecture and
When you supply multiple keywords as the query extract it to reveal the fpcalc binary, which you should
string, the words are automatically joined with a place in /usr/local/bin/ with
Boolean AND operator. That is, Beets will only display sudo mv ~/Downloads/fpcalc /usr/local/bin/
results where both the words appear. Then install the dependencies for the plugin with
You can also restrict the searches to specific fields sudo apt-get install python-gst0.10-dev
such as artist, album and year, such as beet ls and then install the plugin using PIP with
artist:Beatles or beet ls year:2004. You can also sudo pip install pyacoustid PRO TIP
specify a numeric range as a query, and combine it Once you have installed the dependencies, enable Use beet stats to get
with other list options. For example, the plugin by adding the word chroma and rerun the statistics about your
collection including the
beet ls -a year:2004..2005 import command to generate and match the
total number of tracks,
will list all albums released between 2004 and 2005. signatures for the music in your library. total number of artists,
The -a option queries albums instead of individual Both Beets and Picard are feature-rich tools that you the total play time and
more.
tracks. The command can use to organise your dishevelled music library.
beet ls format:MP3 bitrate:128000 Even though we’ve covered some of their most useful
will list all MP3s where the bitrate is more than 128k. and interesting plugins, they offer a lot more options
Refer to the official online documentation (https:// than what’s mentioned here. You can use either tool
beets.readthedocs.org/en/v1.3.13/reference/query. based on the level of comfort of their respective
html) for more query options. operational environments, though Beets can do
everything Picard can and more.
Extend Beets
You can extend the core functionality of Beets with
plugins. Beets ships with several plugins by default, Mayank Sharma has been finding productive new ways to
mess about with free software for years now.
but they need to be enabled before you can use them.
www.linuxvoice.com 113
/DEV/RANDOM/
Final thoughts, musings and reflections
Nick Veitch
was the original editor
Juju Deployments – automating
of Linux Format, a
all the major data centres. Bitwig Studio – the native
role he played until he
Linux DAW to get your
got bored and went pro audio work done.
to work at Canonical Great for everything from
instead. Splitter! composing to podcasting
(but expensive).
R
ecently on Twitter I congratulated the
Linux Voice readership on their prescient
choice to donate some of the LV profit
hoard towards the Open Rights Group (www.
Samson C01u condenser
openrightsgroup.org), an organisation dedicated mic. Great warm tones for
to protecting the digital rights of the individual. podcasting and announcing.
It was prescient, because the UK has elected a
Hercules DJ Control Instinct
new government, one which, unfettered by any USB Midi Controller. Great
sort of liberal conscience, wishes to pursue a for the DJ on the go.
new “Investigatory Powers Bill”, or the “Snooper’s
Charter” as some have dubbed it.
It remains to be seen what specific Orwellian
fantasy will manifest itself through such a bill,
but based on previous efforts, it seems likely it
My Linux Setup Charles Butler
will embody the right of the state to basically
read and spy on absolutely everything you do
Professional tech dabbler, Juju Charms wrangler and more.
online, with no accountability or justification.
Mandatory state-controlled backdoors in What version of Linux are you In the mid to late 90s I received a
messaging systems may seem like a grand idea currently using? SAMS Linux admin guide with three
to combat terrorists, but quis, as ever, custodiet? Ubuntu 14.04 – I love how distros. It’s hard to remember whether it
We can’t even rely on the age-old defence ubiquitous Ubuntu is, on all was Red Hat 5.1, Slackware 6, or Corel
against government meddling – their own hardware, everywhere. Most everything Linux – but one of those three.
incompetence. When David Cameron ‘just works’ these days with it, and I value
announced he was going to clean up the internet that above the enablement story, above What Free Software/open source
by blocking things he considered undesirable, anything else in terms of eye candy. can’t you live without?
the result was the predicted catastrophe. But Internet DJ Console and Icecast -
where a set of skeleton keys for encryption tech And what desktop do you between these two tech’s I’ve
is concerned, the foreknowledge that it will be currently use? managed to reach over 400 thousand
bungled isn’t much compensation – if I were to Unity. I’m a very keyboard-focused listeners. As an indie DJ, these tools are
bet on who would get my bank details first, I’m user. Every day I’m flying in and out reliable and pro grade.
pretty sure I’d rate the hacker skills of global of contexts between servers, my desktop
terrorists over those of the cabinet office. apps, and appliances. Having a keyboard- What do other people love but
So, if you didn’t vote to donate to ORG, there’s centric environment is crucial, especially you can’t get on with?
still time to take a look at the website linked when doing live streaming shows. I’m going to have to go there… but
above and get involved with some very sensible Reaching for the mouse can mean I miss Emacs. I’m a Vim user, through and
campaigns. But gosh, only if you believe in a critical transition. through – but I don’t openly hate on
freedom, openness and transparency of course. Emacs, to each their own.
And if you are a terrorist, I would like to point out What was the first Linux setup
that David Cameron has more money than I do. you ever used?
114 www.linuxvoice.com
CC BY 3.0 Zack Rusin