Plaintext
FIREFOX: DISCOVER ITS HIDDEN FEATURES
› The best
kids’ software
› Build a DIY
Oculus Rift
December 2015 www.linuxvoice.com
FREEDOM!
Switch to 100% Free Software today
Purge your PC of proprietary software
Get complete privacy and security
Be free as Richard Stallman intended!
LIBREBOOT X200
Certified by the Free Software
Foundation – could this be
your next laptop?
ANDROID ANATOMY
Discover the incredible power
in your pocket [ooh behave!] 32 PAGES OF TUTORIALS
INTERVIEW DESKTOP
ALLISON RANDAL GNOME 3.18
De cember 2015 £5.99 Printe d in the UK
The president of the You’re going to love
Open Source Initiative the shiny, slick desktop
talks Perl, linguistics, environment that beats
and the Free Software/ Apple’s OS X at its own
Open Source schism shiny, slick game
DATABASES › GITLAB › GARBAGE › ADA & MORE!
ISSUE 21 WELCOME
DANGER: GLOBAL WARNING
The December issue What’s hot in LV#021
ANDREW GREGORY
Building your own 3D head
tracking system with a few spare
parts and a Linux box sounds like
something from the A-Team. But
we’ve done it with just twelve
easy steps and lots of images.
GRAHAM MORRISON p80
A free software advocate
and writer since the late BEN EVERARD
1990s, Graham is a lapsed The Electronic Freedom
KDE contributor and author Foundation gets a lot of press
of the Meeq MIDI step and does some great work in the
sequencer. USA. Which is why it’s great to be
I
able to highlight a similar project
t doesn’t make it any easier to accept, but it’s no real surprise that based in the UK – the Open
Volkswagen has been caught fixing its emissions software. And Rights Group. p22
it’s probably just the tip of the iceberg. But if there is a positive
aspect to this revelation, it’s that Volkswagen has given us the MIKE SAUNDERS
perfect example of the dangers posed by some proprietary software The continuation of Juliet Kemp’s
and the unexpected consequences of overarching legislation like the wonderful Olde Code series (this
DMCA in the United States or the EUCD in Europe. month: Ada!) is a fascinating read
But this is also an example that can be used to illustrate the many for both old-school charm and for
advantages of open source without any of the ideological baggage, the modern languages that owe
and this is now more important than ever, as technology seeps into so much to the past.
p90
defibrillators, insulin pumps and autonomous cars. It’s unrealistic to
think that proprietary software will cease to exist, but it may give
companies like Volkswagen another reason to try open source.
Graham Morrison
Editor, Linux Voice
Linux Voice is different.
THE LINUX VOICE TEAM
Linux Voice is special. SUBSCRIBE
Editor Graham Morrison
graham@linuxvoice.com Here’s why… ON PAGE 56
Deputy editor Andrew Gregory
1 At the end of each financial year we’ll
andrew@linuxvoice.com
give 50% of our profits to a selection of
Technical editor Ben Everard organisations that support free
ben@linuxvoice.com software, decided by a vote among our
Editor at large Mike Saunders readers (that’s you).
mike@linuxvoice.com
Games editor Michel Loubet-Jambert 2 No later than nine months after first
michel@linuxvoice.com publication, we will relicense all of our
Creative director Stacey Black content under the Creative Commons
stacey@linuxvoice.com CC-BY-SA licence, so that old content
can still be useful, and can live on even
Malign puppetmaster Nick Veitch
after the magazine has come off the
nick@linuxvoice.com
shelves
Editorial contributors:
Mark Crutch, Robin Gower, Juliet 3 We’re a small company, so we don’t
Kemp, Vincent Mealing, Simon Phipps, have a board of directors or a bunch of
Les Pounder, Mayank Sharma, shareholders in the City of London to
Valentine Sinitsyn. keep happy. The only people that matter
to us are the readers.
www.linuxvoice.com 3
CONTENTS ISSUE 21 DECEMBER 2015
Contents
Software is born free, but everywhere it’s in chains. Blame the cheesemakers.
Regulars Cover Feature
News 06
The coming and goings on planet
Linux, including our assessment of the 14
Volkswagen mass fraud, encapsulated for
your reading pleasure.
Distrohopper 08
Manjaro, GNU Hurd, Calculate Linux and
the BSD flavours hitting your hard drive (or
virtualisation platform) right about now.
Speak your brains 10
Us Free Software types seem to be
obsessed with privacy, with good reason.
You’re not alone in your mistrust of Google.
Subscribe! 12/56
Save money, get the magazine delivered to
your door and get access to 21 issues of
Linux Voice, in lovely DRM-free PDFs.
Secrets of Firefox 26
For privacy, for security, and to make Richard Stallman happy – you
The flagship application in Free Software is too can ditch proprietary software completely and go 100% free today.
facing competition from Chrome, but still
has a few tricks up its furry sleeve.
Interview Feature
Core Tech 94
Programmers these days don’t have to think
too much about memory management, but
it never hurts to know the basics. So we 34 22
give you: garbage collection.
Geek Desktop 98
Mr Libreboot, Francis Rowe, doesn’t like
KDE. Mr Bushmills, Nick Veitch, doesn’t
like humans. Like ebony and ivory, they live
together in perfect harmony on page 98.
Allison Randal Inside ORG
SUBSCRIBE The boss of the OSI talks Open Source and Too busy to write to your elected representative?
ON PAGE 56 the schism at the heart of Free Software. Don’t worry, the Open Rights Group will do it for you!
FAQ Group Test FOSSPicks
OpenStack 32 Educational software 50 … yes, Fosspicks! 58
Take the modular, scalable Find the best games, projects and These six pages contain more
principles of Unix and apply them whimsy to keep kids entertained freedom than the Declaration of
to the cloud. Unlimited power! while stimulating their brains. Arbroath. That’s lot of freedom.
4 www.linuxvoice.com
ISSUE 21 DECEMBER 2015 CONTENTS
Feature Tutorials
28
GitLab 66
Don’t depend on Sourceforge – host your
world-beating project yourself!
Android anatomy
Install a custom OS on your Android phone and cut out all the rubbish you don’t want. MariaDB 68
In the kingdom of the database, the man
who understands MariaDB is king…
Reviews
Raspberry Pi 72
Link a funky HAT with some Python and
42 you’re got yourselves a newsreader.
Bitwig OpenVAS 76
Studio 1.2
Scan your network for vulnerabilities
automatically, saving time and money.
At last, there’s some competition on the
Hardware hacking 80
desktop audio production scene, in the
Craft a 3D motion tracking headset out of
shape of Bitwig Studio. But how does its
eBay’s cheap cast-offs.
latest Linux version hold up?
Coding
Gnome 3.18 43 Unity 44 Devolo 1200+ 45
The Gnome Foundation scores a Create games for any platform Extend the range of your home
huge 6 with its documentation, you like (as long as you don’t Wi-Fi throughout your sprawling
polish and panda emojis. mind a tangled web of licences). Dordogne château.
GNU R 84
Use the power of maths to reveal how safe
your favourite Free Software project is.
Rust 88
Launch yourself into the next web with
Mozilla’s new programming language.
Gaming on Linux 46 Books 48
This month, a spiritual successor to the awesome We’re going back to basics without book selection Olde Code: Ada 90
Syndicate, one of the coolest games ever to grace this month, with a look at electronics, programming The US military has given us many things;
the Amiga (spiritual predecessor to Linux). and the noble art of darts. our favourite might just be this language.
www.linuxvoice.com 5
NEWS ANALYSIS
NEWSANALYSIS
The Linux Voice view on what’s going on in the world of Free Software.
Opinion
Code, law and digital restrictions
The VW scandal reveals the self-defeating nature of the laws we use to protect ourselves
Simon Phipps key minds behind the GPLv3 and founder of trivially easily. Business models that depend
is ex-president of the the Software Freedom Law Centre – has on such measures are thus fatally flawed –
Open Source Initiative
frequently pointed out that proprietary unless shored up by draconian laws.
and a board member
of the Open Rights software is inherently challenging to the That’s what the DMCA in the USA and
Group and of Open trust society has in its administrators and laws implementing the EUCD in Europe have
Source for America. suppliers. When an agency or a supplier done. They make “circumventing” a
refuses to allow scrutiny of its source code, “technical measure” a crime, mostly
only its own staff can check it for errors. regardless of context. Sadly these laws omit
Open source is vital for the software used to exceptions for legitimate activities, with the
regulate the public good, because without it, result that activities the legislators never
T
he revelation that Volkswagen and we are forced to trust blindly those who are imagined have fallen into the dragnet of
other vehicle manufacturers had theoretically serving us. When it comes to anti-circumvention laws. Legislators thought
apparently been cheating the vehicle critical infrastructure, this is especially they were protecting the music and movie
emissions inspectorates of the world has
brought back into focus three observations
from the free culture movement. The lesson The right response to VW is to rethink digital
they teach? We must rethink the restrictions and make them reflect the real
criminalisation of citizen curiosity.
contours of the digital landscape
Code Is Law
Law professor and US presidential candidate
Lawrence Lessig first pointed out the impact challenging; as Moglen has said, “Proprietary industries against “pirates”; instead, they
of technology on the administration of law in software is an unsafe building material. You have criminalised the scrutiny of emissions
his book Code and Other Laws of Cyberspace can’t inspect it.” control software in vehicles, protected the
in 1999, where he coined the aphorism wasteful monopolies of printer ink
“Code Is Law”. By this he meant that the real Digital restrictions manufacturers and created an avenue for
regulation of the world arises not in the letter If code is law, and if our trust is only well unseen manipulation of voting machines. As
of the legislation, but rather in the way placed if we can verify, then the observations Doctorow has said, “DRM changes your
programmers interpret it and then express of science fiction author and digital rights computer from ‘What can I do for you, sir’ to
their interpretation in the software that campaigner Cory Doctorow illuminate the ‘I can’t let you do that, Dave.’”
applies the law. More casually, the software third facet of the situation. Technologically Vehicle manufacturers gamed emissions
in the systems around us is what creates illiterate legislators in thrall to the testing because they could. Undoubtedly
our context and environment. entertainment industry have mistakenly their games should be penalised. But we
passed laws that criminalise the would be foolish if that were the only fix we
Trust – but verify circumvention of “technical measures” that furnished. The gaming of vehicle emissions
If code is law, how can society remain obfuscate software. As any geek knows, hid behind the DMCA and the European
transparently governed? Law professor and measures that artificially obscure technical Copyright Directive, and is just one instance
former developer Eben Moglen – one of the details can always be worked around, often of unethical practice enabled by misguided
laws that try to criminalise an arbitrary kind
As any geek knows, measures that artificially of programming. The right response to VW
is to rethink digital restrictions and make
obscure technical details can always them reflect the real contours of the digital
be worked around, often trivially easily landscape – not the imaginary ones the
media industry dream could exist.
6 www.linuxvoice.com
ANALYSIS NEWS
LibreOffice • Gnome • Raspberry Pi • CHIP • Microsoft • Wikimedia
CATCHUP Summarised: the biggest news
stories from the last month
Italy switches 150,000 Gnome 3.18 released with Pi touch display is here
1 PCs to LibreOffice 2 new goodies galore 3 Yes, the Raspberry Pi now
While we’d like to see more After six months of hacking, has its very own official touch
governments and companies migrate including 25,112 source code changes display. It’s a 7-inch 800x480 screen
to Linux, any major moves towards Free from 772 contributors, version 3.18 with capacitive touch capabilities, and
Software (even on Windows) deserve of the Gnome desktop is ready for can connect to the Pi via HDMI, DPI, DSI
applause. Italy’s Ministry of Defence the masses. It includes Google Drive and DBI interfaces. The cost: £48/$60.
has announced that it will transition integration, firmware updates through http://tinyurl.com/pidisplay
150,000 PCs to LibreOffice, following a the Software Tool, automatic screen
law passed in 2012 which states that brightness changing (depending on
free and open source software should light levels around you), touchpad
be the default option in the country’s gestures and new Calendar and
public administration bodies. So Characters applications. Expect it in the
perhaps a switch to Linux will come… next round of distro releases.
http://tinyurl.com/nhsmzmf www.gnome.org
$9 CHIP single-board Raspbian updated to LibreOffice celebrates its
4 computer starts shipping 5 Debian 8, aka “Jessie” 6 fifth birthday
The Raspberry Pi is about to Raspberry Pi owners rejoice: a More good news for the
get a competitor, and a crazily cheap new version of Raspbian, the popular flagship open source office suite. On
one at that. CHIP – the “world’s first Debian-based distro, is here. This time 28 September 2010, a small group
nine dollar computer” – was it’s based on Debian 8 (“Jessie”), and of hackers and FOSS fans decided to
successfully Kickstarted earlier in the brings a boatload of improvements. split off from OpenOffice.org and create
year, and units are now being sent out Most notably, it boots to the desktop by a new project with a more rapid pace
to early backers. The CHIP has a 1GHz default now, and includes a GUI setup of development. LibreOffice has come
ARM CPU, 512MB of RAM and 4GB of tool as an alternative to raspi-config. This a long way in that half-decade: it has
onboard storage, and of course it runs new Raspbian release features plenty of replaced OpenOffice.org (now Apache
Linux. It’s considerably weaker than the optimisations too, to squeeze more OpenOffice) in virtually every major
Pi 2, but still capable of many jobs – performance out of the relatively distro, and is getting faster and more
we’ll get one in for review soon. low-spec machines. Download here: polished with every release.
http://tinyurl.com/q54s9kd www.raspberrypi.org/downloads https://t.co/AvQBPbtkx2
Microsoft makes an Wikimedia starts new
7 OS based on Linux 8 mapping project
What a time to be alive: The Wikimedia Foundation
Microsoft has unveiled Azure Cloud hosts many big-name sites like
Switch, a “cross-platform modular Wikipedia, Wikibooks and Wikivoyage,
operating system for data centre and is now collaborating with the rather
networking built on Linux”. So it’s a very awesome OpenStreetMap team to
specialised stack and a long way from create a new map rendering system.
a desktop distro, but underscores the The Foundation’s goal is to “encourage
changing attitudes within Microsoft our community to create tools and
towards Linux and Free Software. integrations to develop new ways
We’re still sceptical, but if Microsoft for readers and editors to discover
contributes useful code back to FOSS Wikimedia content”, and you can see
projects, we’ll give them some kudos. the early development work here:
http://tinyurl.com/qamw5se www.mediawiki.org/wiki/Maps
www.linuxvoice.com 7
DISTROHOPPER LINUX DISTROS
DISTROHOPPER
What’s hot and happening in the world of Linux distros (and BSD!).
Manjaro 15.09
User-friendly Arch goodness.
W
e use a bunch of different distros
at Linux Voice Towers, and one of
our favourites is Arch. But setting
it up manually can be a chore sometimes,
especially when we get a new bit of kit and
want to put Linux on it as quickly as
possible, so Manjaro has become one of our
favourites. It takes all the good bits of Arch
– the speed, the customisability – and adds
a click-and-go installer, an attractive desktop
setup and other polish. It still lets you tinker
with the system and provides access to the
hugely useful Arch User Repository – but it
doesn’t take as long to set up and provides
extra codecs and drivers out of the box.
The previous release of Manjaro was
numbered 0.8.13, but now the team has
switched to date-based versioning – hence
15.09. And the biggest change here is the Thanks to Calamares, Manjaro now has a graphical installer – although text-mode is still available.
inclusion of Calamares, a graphical
installation tool. Beforehand, Manjaro’s polished and user-friendly, and while it’s still If you’ve heard us waxing lyrical about
installer was a text-based affair that did an undergoing development, the Manjaro team Arch but never had the time to install it, give
adequate job, but wasn’t really all that has deemed it “stable enough for use on Manjaro a go. It’s an excellent way to get the
user-friendly. Calamares is much more production systems”. benefits of Arch without the learning curve.
Calculate Linux 15
Slick Gentoo-based distro for desktops, servers and media.
D
istros based on Debian, Ubuntu and Mate desktops plus configuration tools and
Fedora are ten a penny these days, other goodies.
but one distro that has relatively few Calculate Linux 15 is a major update
derivatives is Gentoo, the source-based providing improved reliability and
distro that was once hugely popular among performance during package updates, while
tinkerers and power users. the Calculate Utilities suite has been Calculate is available with KDE, Xfce and Mate,
Gentoo is still a good distro, but it requires updated with new system build tools, and a along with a Directory Server for businesses.
some effort to get working, so Calculate GUI has been added to Calculate Console.
Linux aims to make the job easier. It’s 100% It’s a well-presented distro with plenty of package update can potentially break the
compatible with Gentoo and features the documentation, and available as a live DVD system or introduce new problems but, as
same rolling-release model – so users image that can be installed to hard drives Arch shows, it’s possible to get it right most
always get the latest apps without having to and USB flash keys. We have some of the time. To grab the latest release, pop
perform major upgrades every six or 12 concerns about the long-term stability of over to the project’s website at
months – but with ready-built KDE, Xfce and rolling-release distros, when a system www.calculate-linux.org.
8 www.linuxvoice.com
LINUX DISTROS DISTROHOPPER
News from the *BSD camps
What’s going on in the world of FreeBSD, NetBSD and OpenBSD.
F
reeBSD is well known for being a
rather conservative project, waiting
for new features and technologies to
mature before rolling them in to the main
source code tree. This pace of development
has served FreeBSD well over the years, but
some developers are looking to make more
ambitious changes to the operating system.
NextBSD (www.nextbsd.org) is a “science
project” that aims to “adapt some of the
more interesting Open Source technologies
from Darwin/OS X to FreeBSD”.
So it’s not a fork of FreeBSD, but rather a
distribution that adds certain bleeding-edge
features onto the main codebase.The most
notable of these features is launchd, Apple’s
init system and service management
framework, which is responsible for starting
and stopping scripts, background daemons
and applications. Despite being part of OS X, NextBSD ISOs are available from http://build.nextbsd.org/data – and you get free ponies as well.
launchd is open source, like various low-level
components of that operating system. faster by starting services in parallel, and Some developers are looking to the future
But why does FreeBSD need a new provides much more flexibility so that and saying that FreeBSD needs to adapt to
service manager? Hasn’t the traditional services can be started when mount points stay competitive and relevant – so if there’s
BSD-style init worked well over the years? appear, sockets are opened, or filesystem useful open source code in Mac OS X that’s
Well, yes, but launchd aims to make booting changes are made. worth nabbing, why not?
What’s the deal with: GNU Hurd?
GNU/Linux, the GNU operating system with the Linux kernel, is what we
use today. But the GNU project started in 1984, seven years before Linus
Torvalds announced his first kernel release, and for many years it was
expected that GNU would have its own kernel. But this component of the
operating system wasn’t a priority for the GNU developers; they were
focusing on a C compiler, text editor and system libraries – things they
could run on other Unix platforms before theirs was fully ready.
However, in 1990 the GNU team decided they really needed to get
cracking on a kernel, and Hurd was born. The name is a doubly-recursive
acronym: “Hurd” stands for “Hird of Unix-Replacing Daemons”, while “Hird”
stands for “Hurd of Interfaces Representing Depth”. Try explaining that to
someone at the pub…
But anyway. Hurd’s most notable feature is that it’s a microkernel,
where only a small portion of the code (memory management, process
scheduling and some hardware drivers) runs close to the hardware. The
rest of the kernel runs in userspace, like a normal program, so if a network
stack, filesystem driver or other feature suffers a crash, it shouldn’t take
down the rest of the kernel.
Of course, the need to regularly switch between kernel and user modes
impacts performance, and GNU Hurd has lagged way behind Linux in
development activity. Even Richard Stallman has said that Hurd is not a
priority for the GNU project nowadays, because the Linux kernel does a
good job. Still, there are GNU distributions using the Hurd kernel, such as
Debian GNU/Hurd (www.debian.org/ports/hurd). For more information,
see www.gnu.org/software/hurd.
Hurd’s design makes it potentially more robust than Linux, but it has
far fewer active developers.
www.linuxvoice.com 9
MAIL YOUR LETTERS
YOUR LETTERS
Got an idea for the magazine? Or a great discovery? Email us: letters@linuxvoice.com
THINK BEFORE YOU SEARCH
DuckDuckGo’s bangs are great, and it’s easy to see why information as possible to search engines, and only ever
people love them. But they underline a bad habit we as what is necessary. And it’s going to be a hard, long, and
users have fallen into. Back in 2012 Eben Moglen [founder difficult fight to make that change.
STAR of the Software Freedom Law Centre] gave a talk at Patrick McDonough
LETTER Republica, where he used the term “Superfriend” – that
third party in the middle who knows everything because Andrew says: Hmm, that’s an angle I’d never
we willingly inform them in exchange for convenience. By thought of until now – use the minimum of
depending on bangs we are placing DuckDuckGo in keywords and click through the links until you
exactly that position. find what you’re looking for, rather than give the
“!g example” talks to DuckDuckGo and Google. “!w search engine all the keywords (I guess this is what
example” talks to DuckDuckGo and Wikipedia. “!r example” you mean by searching smarter, but I’m not very
talks to DuckDuckGo and Reddit. This isn’t malicious, it’s smart). It’s easier said than done though, and really
just how it works – DuckDuckGo needs to know what only works if you know in advance what you’re
you’re looking for so it can interpret the bang. looking for. With DuckDuckGo, at least we know
I’m not advocating people don’t use DuckDuckGo – you that if the injunction comes and Google has to
should use it. But bangs aren’t new – Firefox, Chrome, spill its guts to the police, there
Konqueror, and pretty much every browser implements are alternatives around
some form of them. For Firefox and Chrome they are that we can continue
“Keywords”, for KDE browsers they are “Web Shortcuts” – to use. Hopefully one
and in all of them you can add as many as you like and set day we’ll be able to
the prefix to anything you like. show the political
The solution to the private search problem isn’t to use a culture of mass
private search engine – to create a second, more surveillance to be
responsible Google – because a single point of failure only what it is: paranoid,
takes a single gagging order. The solution is to change illiberal, useless and
how we search – to search smarter. To give as little undemocratic.
ANDROID PRIVACY
Nothing stated in your article (issue 18, September 2015) Sailfish-based device becomes available soon via my
allays my privacy concerns about using Android. I had a mobile provider.
weather app on my tablet and a recent update wanted Kurt Meyer
access to my identity. Why does a weather app need
access to my identity? I’ve been using the app for about a Andrew says: “Why does a weather app need
year or more, so why does it now need to know who I am? access to my identity?” is a very good question
I decided just to completely remove the app from my indeed. I imagine if you were to ask the developer,
tablet. the answer would be some guff about providing
I also realise that there are ways to combat app you with an enhanced user experience, seamlessly
permissions, but such methods either entail not using integrating with your digital lifestyle and online
the app or rooting your device which may void the experience. Which seems a bit much when all you
warranty or cause issues with your mobile provider. want to know is whether to take an umbrella to
The only good thing I can say about Android vs work. I also feel uncomfortable giving too much
iOS is that at least you know what permissions apps information away, and now that my Nexus 5 is
have. I so much want to have a smartphone as they about to pass like tears in rain, I really want to go
are very useful devices, but I value my privacy much back to the simpler days when Snake didn’t need to
more. I’m very much hoping that an Ubuntu- or know your mother’s maiden name and blood type.
10 www.linuxvoice.com
YOUR LETTERS MAIL
CINNAMON Ubuntu isn’t
just a flashy
desktop – it’s
I have three old Dell 170L 32-bit machines with Celeron
hours of unseen
processors. One has Ubuntu 12.04 on it, another Kubuntu graft, ironing
12.04; in both cases 14.04 will barely run, with some parts out hardware
of Kubuntu not working at all on these machines, yet the problems and
third machine runs happily on 13.1 Mint Cinnamon. This making sensible
seems to be an exception to your comment ‘Avoid if you choices. It’s great.
are running old hardware’.
On a different issue; after reading your September issue
I tried Mageia 5 (this time on a Dell 210L machine) .
Despite the impressive looks I found it did not work The point being that for people like me who just want a
completely ‘out of the box’ : system that works without problems, any new system
Problem 1 – It would not download updates; this was must be complete enough for all the basic functions the
traced to two pre-ticked boxes telling the system to look in novice user is likely to perform to work, something that
the optical drive for updates, where it stopped looking Ubuntu and Mint do very well. Once sorted, Mageia 5
further. Unticking these boxes solved the problem . appears to be a good system, although I can’t help
Problem 2 – When trying to use the terminal with a wondering what else they have missed .
sudo command I received a message telling me that I was John Bourne
not on the sudo’ers list and had been reported. The reason
for this is because the sudo’ers list did not exist. The Graham says: Working out of the box is a feature. It
system installation did not include sudo (or Grub) can’t be overstated how important this is. Thanks
software, this has to be downloaded and installed after for your observations John, and I’m glad your old
installation of the main system . machines are getting a new lease of life.
TUX: SHOULD HE STAY OR SHOULD HE GO? PTII
Back in the day, in 1995 when I first started using Linux, of way. Let’s regain that idea of being able to laugh at
Tux fulfilled two very important functions that are still ourselves again. Today, for me, he is a symbol of continuity
appropriate today. – from the near 30-year-old me hacking Linux onto a
Firstly, a unifying theme. I used Red Hat, Slackware, 386dx66 with 4MB (yes children, 4 megabytes!) of RAM, to
SUSE, and Debian – all variously branded, all today and whacking a DVD into whatever and it just works.
Linux, and Tux was the link. Secondly, while Pass me my zimmer frame and false teeth.
we may have serious industrial-strength Richard Rowe
software to use, it is easy to take ourselves
too seriously, so Tux was always meant to Andrew says: The penguin is awful. We’ll just have
be a bit ironic in the ‘GNU’s not Unix’ sort to agree to disagree on this one.
www.linuxvoice.com 11
SUBSCRIBE
Subscribe
shop.linuxvoice.com
Get your regular dose
of Linux Voice, the
magazine that:
Gives 50% of its profits
back to Free Software
Licenses its content
SUBSCRIBE TO CC-BY-SA within 9 months
US/Canada subs prices
TODAY! 1-year print & digital: £95
12-month digital only: £38
Get many pages Access our Save money on
of tutorials, rapidly growing the shop price
features, interviews back-issues archive and get each issue
and reviews – all DRM-free and delivered to
every month ready to download your door
Payment is in Pounds Sterling. 12-month subscribers will receive 12 issues of Linux Voice a year. 7-month
subscribers will receive 7 issue of Linux Voice. If you are dissatisfied in any way you can write to us to cancel your
subscription at subscriptions@linuxvoice.com and we will refund you for all unmailed issues.
12 www.linuxvoice.com
SUBSCRIBE
All subscribers get access to every
single digital back issue –
that’s about 1,000,000 words of
tutorials, reviews and free software
hackery at your fingertips
Overseas subs prices
12-month print & digital:
Europe: £85
US/Canada: £95
Rest of world: £99 DIGITAL
SUBSCRIPTION*
ONLY
*
£38
WHEREVER IN THE WORLD YOU
ARE – IT’S DIGITAL, SO THERE ARE
NO POSTAGE COSTS
www.linuxvoice.com 13
FEATURE 100% FOSS FOREVER
FREEDOM Moving to 100% Free Software is essential in the fight for privacy,
security and freedom. Mike Saunders explains all.
W
ho really owns your computer? If you have
even the tiniest piece of non-free software
installed, it’s fair to say: you don’t own it. As
long as you’re using software that you can’t
investigate, take apart and modify – that is, closed-
source and proprietary software – then you don’t
really control your computer. Indeed, your computer
controls you. Richard Stallman, who created the Free
Software Foundation and GNU project (which is a
huge part of the GNU/Linux system we use today)
has been saying this for years.
Now, as you’re reading this magazine, the chances
are that you’re already using mostly free and open
source software – but there are probably some
programs or online services that you rely on that
are closed source and proprietary. These apps the freedom to study its
and services may seem harmless or claim to workings and make sure no
protect your data and privacy, but in a world dodgy three-letter agencies
where governments and corporations are from the government have put
performing dragnet operations to suck up backdoors in it; the freedom to
all our data and correspondence, the only help our friends and neighbours by
way to be truly secure is to run a 100% free giving them software that may make
software stack from the ground up. their lives easier.
So this issue we decided to go full-on
It ain’t about the money Free Software. We’ll show you how to purge every
If you’re new to Linux, it’s important to last proprietary byte from your machine, explore
note what “free” means when we talk some distros that only include 100% FOSS, look at
about Free Software. It’s free as in speech – not as in alternatives to proprietary online services, and take
beer. We GNU/Linux and FOSS fans don’t particularly the awesome Libreboot laptop – which is also 100%
care about the zero price aspect (although it’s good FOSS right down to the BIOS, and Richard Stallman-
that we can save money for other things – such as approved – for a test run. When you’ve experienced
beer). No, we care about the freedom it provides: total computing freedom, you (and your PC) will never
the freedom to use, modify and share the software; want to switch back.
14 www.linuxvoice.com
100% FOSS FOREVER FEATURE
FIRST STEPS
What’s free, what’s non-free, and how you can purge your system of the latter.
Most software in a typical Linux distro above freedoms and expands upon the
is provided under a Free Software fourth. The GPL states that derivative Get Stallman to check your PC
licence, but certain parts – such as software – ie programs based on code Wouldn’t it be great if Richard Stallman could come round
hardware drivers and games – may from another GPL program – must to your house and personally inspect your PC for non-free
not fall under the definition of Free retain the same licence and provisions programs? (Tip: have two cans of non-diet Pepsi ready, and if
Software. But what exactly is that for freedom. In this way the GPL is you have a parrot, even better – http://tinyurl.com/rmsrider.)
Unfortunately Stallman is too busy to make sure we’re not
definition? Richard Stallman is the best sometimes regarded as a “viral” licence,
being led astray by proprietary software, but there’s a program
authority on this matter, we feel, and he although we’re not complaining when it you can install that will scan your machine for programs that
says that a program can be classed as spreads software freedom. would furrow the great man’s brow.
Free Software if it provides users with If you wanted to run a purely GPL Vrms (Virtual Richard M Stallman, http://vrms.alioth.
four “essential freedoms”: GNU/Linux installation, however, debian.org) scans Debian and Ubuntu-based installations
for non-free packages, which is mightily helpful if you’ve just
1 The freedom to run the program as you’re out of luck. Many other major
done a default installation and want to purge everything that
isn’t full Free Software. Vrms lists the non-free packages on
While most of the Linux kernel is your system, gives you explanations about licence issues if
necessary, and then summarises the proportion of free to
released under the GPL, it also includes non-free software that you have installed. On a typical desktop
system, chances are that you’ll see the Adobe Flash Player
closed source firmware and drivers plugin listed; if you want to remove this but still really need
Flash support, try Lightspark (http://lightspark.github.io), an
open source plugin. It can’t handle all Flash content perfectly
you wish, for any purpose. components, such as the X Window yet, but it’s making good progress.
2 The freedom to study how the System and OpenSSH, have different
program works, and change it so it licences, such as MIT and BSD.
does your computing as you wish. These also provide the four essential
3 The freedom to redistribute copies so freedoms, but they don’t force them
you can help your neighbour. on others. Some would argue that
4 The freedom to distribute copies of these licences are actually more free
your modified versions to others. than the GPL, in that they have fewer
What’s absolutely essential to restrictions; others say they don’t
fulfilling the above freedoms is access protect the user’s freedom enough. It’s
to the source code. Not just so you can an argument that will no doubt go on
look at it – also that you can change it for years.
and redistribute your changes. Now, the The Linux kernel is a special case:
most successful and prominent Free while the vast majority of its source
Software licence is the GNU General code is licensed under the GPL and Use Vrms to get an at-a-glance overview of the
Public Licence (GPL). This is a long and is therefore Free Software, it also non-free packages on your system.
intricate document that guarantees the includes various “binary blobs” – that
is, closed source firmware and drivers
for certain chips and hardware. Many
users don’t object to these and use
them for pragmatic reasons, but as
we mentioned at the start, they put the
control of your computer in someone
else’s hands.
So a project called Linux-libre (www.
fsfla.org/ikiwiki/selibre/linux-libre)
has been developed by the Latin
America branch of the Free Software
Foundation. This removes all of the
binary blobs from the Linux source
Even when he’s rafting down
code tree, and publishes the results as
rivers, Richard Stallman is
a separate branch. Consequently, 100%
fighting to preserve the four
essential software freedoms. Free Software GNU/Linux distros can
(CC-ND, from www.stallman.org/photos)
use this kernel without compromising
any of the four essential freedoms.
www.linuxvoice.com 15
FEATURE 100% FOSS FOREVER
FULLY FREE DISTROS
GNU/Linux distributions that are pure FOSS – every single byte.
Most big-name distros – Ubuntu, Fedora, OpenSUSE, Arch Linux,
Debian – include some non-free software, or at least make such
software easily accessible. There are a handful of distros that are
completely Free Software, though, and which don’t provide easy
access to proprietary software, drivers or binary blobs.
That might seem overly restrictive, but the idea is to prevent you
from accidentally installing something non-free. With one of these
distros, you can be sure that your Linux installation is free to the core,
regardless of whichever checkboxes you tick during the installation.
Here’s a selection of the most notable fully free distros:
Above: Dynebolic is geared towards
multimedia, with audio and video editors
making up the bulk of its software.
Left: Trisquel’s default desktop is
Gnome 3, but with the Classic mode
enabled to make it behave more like
Gnome 2.
Dynebolic gNewSense Parabola GNU/Linux-libre Trisquel GNU/Linux
www.dyne.org/software/dynebolic www.gnewsense.org www.parabola.nu http://trisquel.info
Created for “media activists, artists Despite the terrible name (seriously, Parabola is a fork of Arch Linux, but Our #1 pick of the fully free distros
and creatives”, Dynebolic GNU/Linux try evangelising Free Software and uses the Linux-libre kernel rather the is Trisquel. It’s based on Ubuntu and
includes tools for recording, editing telling them you use something called standard kernel (so it has none of therefore inherits that distro’s plus
and encoding audio files, along “guh-nuisance”), this distro is very the aforementioned binary blobs). points: the user-friendly installer, the
with software for broadcasting the polished with a well-presented website. Additionally, it only provides access vast range of packages available in the
results on the net. It’s designed to be Early releases of gNewSense were to fully Free Software in the Arch repositories, and the wealth of help and
lightweight, requiring just 256MB of based on Ubuntu, but the most recent repositories, and maintains a blacklist support available on the net.
RAM, and is therefore especially useful version uses Debian as its base and of packages that don’t provide the four Trisquel’s developers have focused
if you want to revive an old PC. Gnome for the desktop. gNewSense freedoms (http://tinyurl.com/nzdvsct). on making the distro attractive and
The developers are keen to state is one of the few distros to achieve Like Arch, it’s a rolling-release distro accessible, the end result being a fully
that the distro doesn’t use any online official support from the FSF – indeed, so it’s being constantly updated – a free GNU/Linux flavour that you feel
cloud services. While Dynebolic has a Richard Stallman used it for a while. bonus for those who like new software, comfortable recommending even to
heavy focus on media applications, it’s The project has become somewhat but users need to keep an eye on the newcomers. Its latest release is based
also possible to install regular desktop dormant recently, though, with no website for potentially distro-breaking on Ubuntu 14.04 LTS – so not bleeding-
software as well. major releases for almost two years. changes that occur. edge, but still reasonably modern.
16 www.linuxvoice.com
100% FOSS FOREVER FEATURE
INSTALLING TRISQUEL
Switching to a fully free distro couldn’t be easier.
So, let’s install Trisquel GNU/Linux. Go to close to you and click Download ISO. Once you’ve downloaded the ISO image,
http://trisquel.info/en/download and select Recommended minimum system you can burn it to a DVD-R and boot your
the 1.5GB live DVD ISO image option. Scroll requirements are: a 1.8GHz CPU, 1GB of computer from it (ie start your PC with the
down and choose 64-bit or 32-bit (depending RAM and 10GB of hard drive space – but DVD in the drive), then follow the steps
on your PC’s CPU), then choose a mirror site the more, the merrier! below to install Trisquel.
1 2
2 Prepare When the installer appears, it
will check that you have enough hard
drive space, that you have a power
source and that you are connected to
the internet.
3
1 Boot At the initial boot screen, use the cursor keys to choose the Install Trisquel option and hit
Enter. You can also check the DVD for defects, in case there was a problem when it was burnt.
3 Partition You can choose whether to
4 4 Timezone Next, you’ll be asked to select your install Trisquel alongside an existing
timezone and keyboard layout. Trisquel will try to Linux or Windows installation, or use
guess where you are if you’re connected to the the entire hard drive. If you’re familiar
internet – otherwise you’ll have to choose manually. with advanced partitioning, choose the
When you’re done, click Continue. Something Else option.
6 6 Finish! After the files have been
copied over, you can reboot your
machine, remove the DVD or USB key,
and boot up your shiny new Trisquel
5 installation. Congratulations: you’re now
running 100% fully Free Software.
5 User account In order to identify
yourself to the Trisquel installation,
you’ll need to set up a user account and
password. You can choose whether to
log in automatically when you boot
Linux, or ask it to prompt you.
www.linuxvoice.com 17
FEATURE 100% FOSS FOREVER
FREEDOM ONLINE
Switch to online services that put your privacy and security first.
Even if all of the software installed Then it’s worth installing a browser
locally on your machine is Free and plugin that prevents websites from
open source, as soon as you start tracking you and following you around
visiting websites, you run the risk of the web: DoNotTrackMe, Blur and Privacy
losing control again. Pretty much every Badger are good examples in this field.
website uses JavaScript now so it’s You may be tempted to install a full-on
easy to fall back into the trap of running ad blocker, but many websites survive
code on your machine (in your browser) solely from advertising revenue, and
that you can’t study, modify and share. we don’t have anything against non-
Now, even though the web is intrusive, non-tracking ads relevant to
dominated by a handful of companies the site in question.
such as Google, Facebook and Twitter,
it’s perfectly possible to free yourself Search and social
from their clutches. The first step is Next up is switching to a FOSS-friendly
to install web browser add-ons that search engine. DuckDuckGo (www.
restrict the execution of JavaScript duckduckgo.com) is one of the better
from a website until you explicitly allow providers in this field – although not
it. NoScript (for Firefox) or ScriptSafe (for 100% FOSS on the back-end – with
Chrome) let you enable JavaScript on mostly decent results (if still lagging
an ad-hoc basis, so if a site doesn’t need behind Google). It has a repo of source
it and only uses it for fancy effects, you code at https://github.com/
can disable it. In this way, you can stop duckduckgo, and there’s also a plain
Top: OpenStreetMap is much better than Google Maps
third-party JavaScript from advertising HTML version of the main search page,
in many respect: its interface is saner, and the maps
more detailed. Middle: Most webmail services read your providers from running on your which doesn’t use any JavaScript at
messages and pester you with ads. Get privacy by hosting machine. There’s also LibreJS (www. www.duckduckgo.com/html.
your own with Roundcube. Above: OwnCloud features a gnu.org/software/librejs) from the free Google Maps has a FOSS-
rudimentary word processor, which is fine for basic tasks. Software Foundation with similar goals. friendly replacement in the form of
18 www.linuxvoice.com
100% FOSS FOREVER FEATURE
The colo alternative
If you choose commercial web hosting or a virtual private
server, you may still end up running it on hardware that’s
not completely Free Software down to the core (ie it uses
binary blobs or non-free firmware). If you really want to go
100% FOSS, you could always build your own machine and
have it co-located in a hosting centre. These centres provide
electricity, bandwidth, cooling and security for your machine,
but otherwise it’s yours to do with as you please – just as if it
were in your home.
This can be quite expensive – upwards of £30 a month –
and many co-location providers will only host machines in a
specific form factor (eg rack unit servers). Others are more
flexible, however, and for the right money they would be willing
to look after your 100% Free Software GNU/Linux box. Some
colo providers are now offering Raspberry Pi colocation, which
is relatively cheap as they’re small, they run cool and they use
barely any power – but note that the Raspberry Pi isn’t 100%
FOSS and FSF-approved due to the firmware it uses.
While the Raspberry Pi + Raspbian is mostly Free Software, the device uses non-free
firmware to boot the GPU, so it’s not the best choice if you want to go 100% free.
OpenStreetMap (www.openstreetmap. geeks on there and it has the potential which has a big impact if you want to
org). This service uses data contributed to grow into something much bigger. create your own private file-hosting
from the community, and the database cloud, and many ISPs block ports for
is free for everyone to share and modify. Hosting options outgoing mail. In this case, you’ll need
OpenStreetMap doesn’t have some of Of course, while using third-party a commercial hosting provider, and one
Google Maps’s frills, but it’s often more services is one option, for maximum of the best in terms of FOSS support is
up to date and has a much higher level privacy you should host your own. This Gandi (www.gandi.net).
of detail. Also, the OpenStreetMap data might sound like lot of work, but various Gandi offers domain name
is used by many smartphone mapping packages are available that simplify the registration, simple web
apps such as the open source OsmAnd, process considerably. First off, consider hosting facilities, and
which lets you download areas of the how you want to host your services: via VPSss (virtual private
world for navigation when you have no your home internet connection, or servers) for when you
internet connection. somewhere else? The former is want maximum control
certainly possible, but most home over your OS and software.
Social networking broadband connections don’t provide Once you have hosting or a
With every man Jack and his dog on static IPs. In other words, your IP VPS set up, you can install private
Facebook now, it’s hard for competitors address will change every time the cloud software such as OwnCloud
to get a foot in. Diaspora’s giving it a connection is dropped. (www.owncloud.org), which provides
go though, with an open source and To counter this, you can use a DNS file storage, calendars and document
distributed social network, in that service like http://freedns.afraid.org – collaboration. It’s a quick way to
anyone can contribute a server (aka this regularly updates a domain name replace Google Drive, Google Calendar
a “pod”) and host accounts on it. to point it at a frequently changing IP and Google Docs (or their Microsoft
(Contrast this to Facebook, which is address. FreeDNS has a wide range equivalents) with services that fully
massively centralised and entirely in the of domains you can use, with custom respect your freedom. Gandi has
hands of a single company.) To find a subdomains, so you could use it to useful documentation for setting up
pod that’s open for new user accounts, point blah.homenet.org to your home OwnCloud: http://tinyurl.com/pjhg9lu.
see https://podupti.me. IP address. This is simple to set up, and To replace Google Mail and co, you’ll
For something more Twitter-esque, involves running a program inside your need to host your own mail server,
try Pump.io (https://pumpit.info), home network at regular intervals (eg a somewhat more involved task but
another distributed social network via cron) to make sure that FreeDNS certainly still possible. There are many
that’s a continuation of the once mildly knows your current IP address. different mail server packages out
popular Identica network. Pump.io’s Hosting services from your home there, but we recommend using Cyrus,
userbase is very small in comparison has downsides though: your upload an IMAP server. We had a detailed
to Twitter, but you will find some FOSS speeds aren’t likely to be amazing, tutorial on setting up Cyrus in issue 8
of Linux Voice, so grab a copy from
Even though the web is dominated by http://shop.linuxvoice.com. Another
piece of software worth considering
a handful of companies it’s possible to is Roundcube, a slick webmail client
free yourself from their clutches that you can hook up to your own mail
server and run on your hosting provider.
www.linuxvoice.com 19
FEATURE 100% FOSS FOREVER
ON TEST: LIBREBOOT X200
For maximum freedom, you want a machine that’s FOSS right to the core.
The Libreboot X200 is a
refurbished ThinkPad X200,
and fully Free Software
right down to the BIOS.
So you’ve installed Trisquel (or refurbished ThinkPads, primarily
Other options another fully free GNU/Linux distro), ex-corporate machines from large
Currently, the Libreboot laptops from Minifree are the only you’ve purged all traces of proprietary companies, and Minifree lent us one
machines that are certified by the FSF’s Respects Your software from your machine, and of its Libreboot X200 units (originally
Freedom programme. So if you’re looking for something else you’ve switched to FOSS-friendly online a ThinkPad X200) for review. This
that’s confirmed as running FOSS at every level, you’re a bit services. What’s your next step? Well, particular model is the highest spec
out of luck. There’s the Purism (https://puri.sm) range of
chances are that there’s still some non- that Minifree offers, with an Intel Core 2
laptops, which have similar goals, but as of the time of writing,
they still included non-free software in the BIOS and some free software lurking in your machine. Duo P8400 CPU clocking in at 2.26GHz,
firmware, making them not eligible for Respect Your Freedom Most PCs and laptops have proprietary 8GB of RAM, a 240GB SSD drive and
certification. If you’re looking to build a PC from scratch, BIOSes (or UEFI implementations), a docking station – all for €578, so
check out www.h-node.org for a database of hardware that for instance – and they are far around £433 at the time of writing.
works with fully free GNU/Linux distributions.
from harmless. For instance, PC
manufacturer Lenovo was recently Business machine
caught forcing installation of bloatware Hardware-wise, the X200 is extremely
on its Windows-based laptops via the rigid and well-built. It’s a “business
machines’ BIOS. ultraportable” machine, and while its
This didn’t affect Linux users, of weight (1.66kg) and size is trounced
course, but it shows why it’s important by modern ultrabooks, it’s still not
to have a fully open source BIOS if you especially heavy. The X200 has three
want total control over your computer. USB 2.0 ports, VGA-out, SD card
Minifree (www.minifree.org) is a UK- and ExpressCard slots, headphone
based company selling Free Software and microphone jacks, plus ports
Foundation-endorsed laptops that for Ethernet, dialup modem and a
are FOSS right to the core, including Kensington lock. Its dimensions are
the BIOS. The laptops are actually 295mm (width) by 210mm (height)
and its thickness varies from 21mm to
There’s something very special about running 35mm. The docking station provides a
CD/DVD-RW drive, four additional USB
a completely Free Software stack on such a 2.0 ports and DisplayPort as well.
robust, well designed machine Along with the overall build quality,
the biggest highlight of the hardware is
20 www.linuxvoice.com
100% FOSS FOREVER FEATURE
While the X200 Verdict
isn’t as thin as an A couple of slightly dated components aside, the X200 is a
ultrabook, it’s still good value machine with superb build quality, almost flawless
fairly light and has a GNU/Linux support, and a heavenly keyboard.
superb keyboard.
on the web; this didn’t work properly
for many sites we tested, but YouTube
ran smoothly using its HTML 5 mode.
the keyboard. It’s absolutely fantastic, machine? Fortunately, it’s superb. Battery life was decent as well, with
as fans of the older ThinkPad models Trisquel boots and shuts down the 9-cell battery providing around 4.5
will attest to. very quickly thanks to the SSD, and hrs of usage with maximum screen
Wi-Fi, volume controls and power brightness and light browsing and
Pointing fingers management (eg suspend) work writing tasks. If you order an X200 and
No trackpad is installed in the X200; perfectly out of the box. This is quite receive a 6-cell battery, expect around
instead, you get a “nipple” TrackPoint an achievement for any laptop running three hours on the road.
device in the middle of the keyboard Linux, but when you consider that Overall, we can safely recommend
that can be used to push the pointer it’s 100% powered by Free Software, the Libreboot X200 as a solid
around the screen, along with three with no binary blobs in sight, it’s even workhorse with superb GNU/Linux
buttons underneath. A free USB mouse better. The X200 handles CPU load with and FOSS support. Despite being
is available on request. aplomb too: even with maxed out CPU refurbished, our review unit looked (and
The X200’s screen is perhaps the cores, the machine stayed cool and the smelled!) largely like a new machine,
least impressive part of the package, fan noise was barely perceptible. and only a small trace of dust under
and reflects on the machine’s age. It’s the keyboard reminded us that it had a
a 12.1-inch WXGA display (1280x800) Nice little runner previous life in an office somewhere.
that’s lacking in terms of resolution, In terms of performance, the 2008- There’s something very special about
brightness and viewing angle when era CPU and Intel 4500MHD graphics running a completely Free Software
compared to more modern laptops. It’s chip are a bit long in the tooth now, but stack on such a robust, well
not bad per se – it just takes a while they’re no slouches and the 8GB of designed machine
to adjust to and is a noticeable step RAM – coupled with the SSD – make it – and with
backwards when you’ve gotten use a generally zippy machine to use. In our the perfect
to HiDPI displays. Some units have testing we found that daily tasks such keyboard
webcams, but not all – it’s best to ask as web browsing, image editing, office for heavy
Minifree before ordering. work and coding were no problem on Emacs
Now onto the biggest question: the X200. Trisquel uses the GNU Flash (sorry, Vim)
how well does GNU/Linux run on this Player (Gnash) to view Flash content sessions.
Minifree includes a
power adaptor, Ethernet
cable and FOSS stickers
galore in the box.
www.linuxvoice.com 21
FEATURE MEET THE ORGWELLIANS
MEET THE ORGWELLIANS
Mayank Sharma embeds with the elite force that’s guarding the UK’s digital fence.
L
ike most good things that start with a question, and Privacy conference in the US every year from
it was a panel called “Where’s the British EFF?” 1994 onwards. ORG engages with those same sorts
at the Open Tech Conference in 2005 that led of topics,” explains Grossman in an email exchange. “It
to the creation of the Open Rights Group. The panel is much easier to shape the future of a medium at the
was hosted by Danny O’Brien, then the EFF’s activist beginning than it ever will be later on when interests
coordinator, and featured noted digital rights activists (and sometimes bad laws) have become entrenched.
Cory Doctorow (representing the EFF), Ian Brown from The internet is the one new medium of my lifetime,
European Digital Rights and Rufus Pollock of the Open and how we shape it will set the tone for generations
Knowledge Foundation Network. An encouraging to come. Bad policy decisions now will weigh on our
response from the attendees led O’Brien to set up a great-grandchildren.”
crowdfunding campaign on Pledgebank to secure
enough funding to pay for a couple of members of An ORGan for change
staff. Around 600 people signed up, and ORG was In an email exchange with LV, Pam Cowburn, ORG’s
born before the year was out. communications director, notes that it campaigns
The primary mandate of ORG is to preserve and for both free speech and privacy online: “We focus
promote your rights in this digital era. According to its on the intersection between technology and human
website the group does “whatever it takes to build and rights. ORG believes people have the right to control
support a movement for freedom in the digital age.” their technology, and oppose the use of technology
Wendy Grossman, who has been writing on to control people.” She reasons that given how
computers and privacy issues since the early 90s, has much technology has transformed the way we
been on ORG’s advisory council since the beginning. communicate, work and live, this is a massive remit,
She contacted O’Brien to get involved and he put so the group has to be selective in its approach. The
her in touch with ORG’s founding executive director, group focuses on the most important issues that they
Suw Charman, when she began bootstrapping the can influence before deciding the most effective way
organisation. to bring about change. ORG also works in partnership
“ORG engages with many interests that in my case with other organisations, as it helps them be more
were fostered by attending the Computers, Freedom, effective with their limited resources.
Cowburn explains that a lot of their work
The Open Rights Group focuses on involves reacting to external developments: “Our
supporters rely on us to respond to the behaviour of
the intersection between technology governments, corporations and others. Following the
and human rights Snowden revelations, we have inevitably focussed on
surveillance by the British and other governments.
22 www.linuxvoice.com
MEET THE ORGWELLIANS FEATUREL
This campaign has involved challenging government
legislation such as the Data Retention and
Investigatory Powers Act (DRIPA) and working with
other organisations to call for more transparency,
accountability and better oversight. With other
partners, we’re challenging the UK Government at
the European Court of Human Rights.” ORG also
intervened in a case brought by MPs Tom Watson and
David Davis, which saw the High Court rule that parts
of the Data Retention and Investigatory Powers Act
were unlawful.
Quis custodiet ipsos custodes?
The group has produced in-depth papers on a wide
variety of subjects. There’s a report detailing GCHQ’s
mass surveillance programs, the intrusion and
integration with the NSA, and the threats and risks Credit: Jim Killock
it creates. Another one highlights the group’s digital
rights concerns with the Transatlantic Trade and
Investment Partnership (TTIP). You can also read its
correspondences with the government and public
functionaries on issues such as the exchanges with
London’s Police Commander Steve Head on the
Police Intellectual Property Crime Unit (PIPCU).
Outlining the group’s upcoming tasks, Cowburn
shares ORG’s concerns over the new surveillance
bill on the horizon: “We are concerned that the
government will attempt to extend its powers and
bring back elements of the draft Communications
Data Bill, which ORG and our supporters helped to
defeat previously.”
Commenting on their course of action in case
this happens, she says the group will mobilise its
supporters and ask them to persuade their MPs Credit: Sheila Thomson
that human rights do matter: “Our supporters are
key to this campaign. We know that MPs listen to
their constituents, especially when they care enough Credit: Ed Lander
about an issue to visit them in person. We’ve already
held a lobby day in parliament to help people set up
meetings with their MPs. Once the new draft bill is
published, we’ll no doubt be asking supporters to raise
concerns with their MPs whether through emails or
face-to-face meetings.”
The ORGanisers
The Open Rights Group has a handful of paid staff
members, headed by executive director Jim Killock.
Before joining ORG in 2009, Killock worked as the
external communications co-ordinator of the Green
Party. A grassroots man himself, Killock had led
campaigns against the ‘three strikes’ and the Digital
Economy Act, the company Phorm and its plans to
snoop on UK users, and against pervasive
government internet surveillance.
Another member of staff is project manager
Top: Project Manager Richard King with ORG
Richard King, who oversees technical projects. While
supporters at an ORG hack day at Mozilla’s offices.
ORG might appear to be a campaigning organisation, Middle: Glyn Wintle, an ORG supporter from the very
King reasons that because ORG campaigns at the beginning, mans the group’s stall at LUG Radio Live.
intersection of human rights and technology, “it’s Above: Cory Doctorow interacting with attendees at an
natural for us to use technology in our campaigns.” event organised by ORG Bristol.
www.linuxvoice.com 23
FEATURE MEET THE ORGWELLIANS
our reports or to help us to give expert responses to
media requests.” It has over 40 members, including
John Buckman (founder of Magnatune), Alan Cox
(noted kernel hacker), Graham Linehan (writer and
director of The IT Crowd), Cory Doctorow, Wendy
Grossman and others.
Cowburn also points out that ORG also has a group
of lawyers to whom they look for advice and guidance
when required. She believes “legal interventions are
Credit: Peter Chamberlin
a very effective way of campaigning for change” and
highlights the work of David Allen Green, a lawyer at
Preiskel & Co LLP, who acted for ORG pro bono in a
trademark infringement case.
Circle of friends
An important proponent of ORG’s outreach to
individuals are the different local groups all over the
UK. “Our local groups are really vital to ORG’s success,”
says Cowburn. “They are a place where supporters
can meet and get together to discuss digital rights
issues. Like many organisations, we are based in
London and it’s easy for organisations to become very
London-centric. Our local groups help us to reach out
to people across the country and are invaluable in
putting pressure on their local MPs.”
Each local group is managed by local
representatives or organisers, and ORG provides
information and support to help anyone new to
campaigning. The group organisers are assisted by
Credit: Alec Muffett the Local Groups Co-ordinator and ORG staffer Lydia
Snodin, who helps the organisers with planning,
Top: Supporters of the administrative and perhaps even some financial
Open Rights Group
King coordinates the work of ORG’s community support. The group organisers are invited to join the
protest against the Digital
Economy Bill back in 2010. of technical volunteers, which is made up of ORG Supporter Council, where they can share ideas to build
Above: A section of the supporters who like to work on digital-rights projects their groups, brainstorm event ideas and help plan
crowd at ORGCon 2012. they think are interesting. “We’re not just a community future campaigns.
of coders,” he explains. “We encourage people with Art O Cathain is a software developer and local
any kind of interest in technology to join in. We organiser of the Bristol group and has planned and
especially love it when people suggest new projects hosted several ORG events in Bristol. “Organising
– and it’s my job to support these in whatever way events locally is a case of coordinating a speaker,
I can. I also organise occasional hack-days for the venue, and audience. It’s useful to have the Open
community.” King points to https://www.blocked. Rights Group name because they are a well-known
org.uk as example of a project built by the tech- organisation within the tech community. For example,
volunteers. The website makes the web filters of we recently held a talk by Cory Doctorow during the
home and mobile ISPs transparent for the first time. Bristol Festival of Ideas. ORG’s campaigns coordinator
The system is Free Software and the group is now helps out with suggesting speakers and event
working to deploy it in other countries as well. themes, and publicising events.”
ORG also taps the collective knowledge of its The local organiser of the Manchester group, Tom
advisory council for guidance and expertise. Cowburn Chiverton, narrates a similar tale: “The campaigns
points out that the advisory council is made up of tech team at HQ always seems to be available for last-
experts, MPs, academics and activists. “They have minute thoughts and sharing of ideas.” In addition to
vast amounts of knowledge, expertise and experience finding speakers, Chiverton says that ORG also helps
and we call on them regularly to ask their advice fund larger events, as well as travel costs for the more
about our strategy and campaigns, to comment on far flung speakers.
If we allow the internet to be taken Get involved
If ORG’s activities resonate with you, the group
over by those who don’t understand its presents several opportunities for contributions. While
importance, the impact would be huge one of the best ways to contribute is to find and join a
local group, there are other avenues. King invites all
24 www.linuxvoice.com
MEET THE ORGWELLIANS FEATUREL
Credit: Open Rights Group
those who are technically inclined to contribute to
ORG’s tech efforts by joining its mailing list at https:// varies between £5–10 a month. Members who pay Above: ORG is using
lists.openrightsgroup.org/mailman/listinfo/ £60 a year or more can opt to receive a welcome pack £1,000 donated by Linux
tech-volunteers. Those with an editorial bent of mind and gift, which currently is a copy of Becky Hogge’s A Voice to campaign about
can contribute to the internal research projects about Guide to the Internet for Human Rights Defenders. You surveillance in the run up
projects that interest ORG. can also make a one-time donation via PayPal and the to the publication of the
draft Investigatory Powers
ORG keeps track of developments in the UK group even accepts Bitcoins.
Bill this autumn.
parliament that may have an impact on digital rights
issues such as copyright, privacy, and open data. Defend the future!
When it finds something that needs to be addressed, The issue of digital rights has never been more
the ORG seeks opinions and inputs for drafting relevant in the UK than it is now. As companies and
consultations for the Government. A consultation is the government come up with more shrewd
a government request for opinions on various issues, technology and laws that will invariably intrude upon
be they proposed internal guidelines, draft legislation, our privacy, we need someone who can look after the
or simply the route the government wants to take in a interests of individuals. Navigating the digital
particular area. minefield requires knowledge of technology, policy
One sub-section on ORG’s wiki is the Adopt an and law and most of us are ill-equipped to grasp
MP scheme. This page hosts a list of the Members the implications.
of Parliament along with their contact details. You Cathain says he is a passionate believer in freedom
can read the views of the individual members on the and liberty, which is why he thinks the Open Rights
various digital rights issues. Volunteers can update Group is so important. “Because computers and
the page on the MP to include their public statements technology are generally not well understood, only
on issues that are of interest to ORG. This list helps a small fraction of the population appreciates the
when the groups needs to lobby MPs. significance of digital rights issues. The same
If you can earmark specific periods of time for problem applies to our elected MPs. Laws haven’t
helping out ORG, you can join the group as a volunteer kept up with changes in technology and it’s vital that
working on the things mentioned on the volunteer they can debate new laws such as the proposed
page (https://www.openrightsgroup.org/volunteer). Snoopers’ Charter with the necessary understanding
The group also offers some internship positions for of the issues.”
students during their summer break. If this interests Chiverton holds a similar view: “The online world
you, check out open positions at the internships page is no longer an optional part of modern life, and a
on the wiki (https://wiki.openrightsgroup.org/wiki/ functioning democracy requires a free flow of ideas
ORG_Wiki:Internships). and information. If we allow the internet to be taken
The Open Rights Group is funded by small grants over by those who don’t seem to understand its
and donations from individual supporters. You can essential character and importance, and would seek
become a member by giving as little as £2.50 a to turn it into a series of silos and blocks of bunkers,
month by Direct Debit, although the usual amount the impact would be huge.”
www.linuxvoice.com 25
SECRETS FIREFOX
SECRETS OF
FIREFOX
Master the web with the hidden features of the flagship open source browser.
I
f there’s one project that has brought more new users to open browsers have come along and challenged for online supremacy.
source software than any other, it’s Firefox. This web browser Here at Linux Voice, we think that Firefox is still the best web browser
rose out of the now-defunct Mozilla suite and came to dominate around, but takes a bit of experience to get the most out of it. Here
the web, though it now faces stiffer competition as other new web are some of the features that we find particularly useful…
01 02
01
Tab control privacy when using Firefox. Disabling inspect the HTML. Whether you’re
Most Firefox users are third-party cookies (Edit > Preferences > building a website or investigating
comfortable using tabs to Privacy > History > Use Custom someone else’s, there’s no better tool
keep multiple websites open at a time, Settings For History > Accept Third for understanding how the web works.
but Firefox tabs are more powerful than Party Cookies > Never) is a simple way
04
this. Pinning tabs (right-click on the tab of cutting down the number of sites Addons
and select Pin Tab) keeps the tab on the that track your browsing. If you want to Firefox gets a few more
left-hand side and the browser opens go further, addons like Privacy Badger features every release, but
that site automatically when launched. (from the EFF) and NoScript enable you it’ll never have everything that everyone
If you use large numbers of tabs, you to make intelligent choices about how wants. However, it does enable you to
may find it easier to keep track of all the you want your browser to work. get the features you’re missing by using
pages by grouping tabs. Press add-ons. Head to https://addons.
03
Ctrl+Shift+E to enter the Tab Group Developer tools mozilla.org to see what’s available.
screen to ease tab management. These enable you to get a
05
better insight into how the Themes
02
Privacy browser’s working – for instance, you If the default grey is a little
There are a wide range of can use them to analyse a web page’s boring for you, you can
options to increase your performance, debug its CSS and spice up your browser’s look with a new
theme. These are a special type of
addon that just affects the look of the
Addons like Privacy Badger (from the EFF) and browser, leaving its features
unchanged. You can take a look at the
NoScript enable you to make intelligent choices options (be warned, some of them
about how you want your browser to work aren’t too pretty) at https://addons.
mozilla.org/en-/firefox/themes.
34
26 www.linuxvoice.com
FIREFOX SECRETS
03 04
05 07
06
06
WebRTC want to save. Your portable device will
This black magic can send
data directly between web
then download the page and make it
available offline. 08
browsers. At the time of writing,
08
WebRTC is mostly used for video and Bookmark management
text chat that doesn’t rely on central Visit the same pages often?
servers, but the technology is data- Bookmarks make it easy to
agnostic, so it can be used to shuffle go back without having to remember
anything between browsers in real time. the address. You can also make it easy
to manage your bookmarks by
07
Pocket grouping them into folders. To enable
This is a mobile app that easy access to your bookmarks, you
enables you to save web can add a toolbar to the main Firefox
pages for offline viewing. If you install window (View > Toolbars > Bookmarks
Pocket on your mobile or tablet, then Toolbar) or adding them to the new tab
you can press the Pocket button in page (use the pin icon when they
Firefox on your desktop on a page you appear). Happy browsing!
www.linuxvoice.com 27
FEATURE ANDROID ANATOMY
ANDROID ANATOMY
Take control of the most widely distributed Linux-based computing device on the planet.
I
t may be just a phone. Or a tablet. And it may be
just a simple, perfunctory, useful tool; the
prerequisite accessory of our times. But our
phones hold many times the power and capacity of
even PCs from just a decade ago, and they’re always
connected, always talking to servers, masts and
satellites. And most remarkably of all, this frontier is
mostly running on Linux.
DANGER!
Beneath the surface, clinging to the beating
Messing around with
elements of your Android devices heart of its Linux kernel, there’s still a lot that
can cause them to break, void will feel familiar in Android. Security via
warranties and become worthless. SELinux plays a central role, for example,
Proceed with extreme caution, and the filesystem has the same general
or at the very least, proceed ADB needs to be enabled as ‘USB debugging’ in your
layout as your desktop.
only after you’ve copied and
But the smartphone is also the new phone’s system configuration window.
backed up your data.
proprietary frontier. Where once we railed
against PCs pre-sold with a closed operating THE FILESYSTEM
system that couldn’t be changed, we find ourselves
in even greater peril. These are PCs in our pocket, Remember: Everything is a file
continually connected and capable of running The key to understanding the similarities and the
software 24 hours a day, 7 days a week, and yet most differences between Android and your typical Linux
are locked down in ways many wouldn’t accept from installation is to understand the Android filesystem
PCs. They’re what computing would have looked and how storage is partitioned. Each Android device
like had UEFI and SecureBoot realised our worst is different, so the best way of seeing how your
nightmares. phone is configured is to check for yourself from
But there is hope. Android is open source, and the the command line. On the vast majority of Android-
first step to a better platform is understanding how based phones, you can install a terminal emulator
it works, and how you can use your Linux skills to and use the command line just as you would your
subvert and extend this computer in your pocket. This Linux box, and there are various terminal applications
is what we’re going to do over the next few pages, available in both the Google Play store and the open
giving you the confidence to do anything from make source F-Droid store. Simply running the terminal
a few changes to your phone, to completely replacing and typing cat /proc/partitions will return a long list
the operating system. of devices that all start with ‘mmc’, which originally
Memory allocation
Android’s greatest/most frustrating attribute or details of what’s possible, and there’s a
is that every device is different, and almost proliferation of third-party operating systems
every manufacturer does things slightly to install.
differently from other manufacturers from Canonical’s Ubuntu Phone operating
one generation to the next of each device. system is based on Android and embeds
The most open and experimental Android significant chunks within Ubuntu Touch,
devices we’ve found are Google’s own which means that much of the ‘under-the-
Nexus units, and these have become the hood’ Android specification is also applicable
closest thing to a standard reference point, to an Ubuntu Phone. The partitioning
whether or not other manufactures take any scheme is similar, for example, although
notice. Nexus devices have relatively little the ‘recovery’ partition currently doesn’t do
obfuscation of the operating system, the anything. Fastboot works, however, and the
bootloader is open, and it’s relatively easy to recovery partition can still be flashed with
get yourself root access (a process known an Android equivalent for the same phone,
as ‘rooting’ on many devices). You’ll also find as can the entire phone, and ADB can be Google has just announced its Nexus 5x and 6P
that lots of other people have been doing the enabled by activating ‘Developer Mode’ in the phones, but the original Nexus 5 is only marginally less
same thing, so you can often find support ‘System Settings > About This Phone’ pane. powerful and still a great option for hacking.
28 www.linuxvoice.com
ANDROID ANATOMY FEATURE
OPEN
DEVELOPMENT
PLATFORM – NO
WALLED GARDEN
PSEUDO-ANARCHIC
SYSTEM LAYOUT
LINUX KERNEL
HEART FOR
FLEXIBILITY AND
STABILITY
UPGRADE THE
OS WITHOUT
WAITING FOR YOUR
PHONE CARRIER
KNOW YOUR ANDROID
stood for ‘Multi-media card’ when flash memory Main partitions
was removable. The chips are now more usually boot The boot partition performs exactly the
fused onto the PCB of modern models and split same function as /boot on the Linux desktop,
into 512-byte sectors, much like hard drives. Our but it doesn’t contain the bootloader – that’s on a
Nexus 5, for example, listed mmcblk0Op1 through to different part of the flash memory. boot contains
mmcblk0Op29, plus a partition called mmcblk0rpmb. the kernel and the ramdisk for the operating system,
To see where these partitions are being used, you both of which are read almost as soon as the
can use the same mount command you would on system is turned on.
your desktop, and you should pipe the output through The ramdisk is a minimal root filesystem that’s
grep, searching for ‘msm,’ to avoid being inundated loaded into memory and used to boot the remainder
with text: of the system, and becomes the root filesystem of
1|u0_a184@hammerhead:/ $ mount | grep msm the operating system via the rootfs mount point after
/dev/block/platform/msm_sdcc.1/by-name/system /system ext4 booting. The init scripts, found in /init, are then run to
/dev/block/platform/msm_sdcc.1/by-name/userdata /data ext4 boot the remainder of the system. The kernel version
/dev/block/platform/msm_sdcc.1/by-name/cache /cache ext4 also needs to correspond with the drivers contained
/dev/block/platform/msm_sdcc.1/by-name/persist /persist ext4 in the system, or a mismatch may prevent Wi-Fi or
/dev/block/platform/msm_sdcc.1/by-name/modem /firmware Bluetooth from working, or even a successful boot.
vfat system Here’s where all the main Android
The device nodes in the above output match the components of the operating system are installed.
partitions that can be found in /dev/block/platform/ Before Android 5.0, it used to correspond directly to
msm_sdcc.1, although the contents of these device the contents of the system.img file that was part of
nodes are hidden unless you gave root access (see any Android update or installation, but it’s now rolled
later for details on how to get root), and these match into system.new.dat which needs to be opened and
against the partitions we listed earlier. MMC flash re-created using a specific tool.
devices appear to the system as hard drives, they’re As the mount command reveals, it’s also mounted
formatted using Android’s preferred filesystem, which read-only so that the user can’t mess around or
is currently ext4, as shown above. As these are the alter its contents. Removing and replacing files in
partitions used while the operating system is running,
there are a couple more that aren’t mounted and Android is open source, and you can use
aren’t visible from within Android itself. The first is
boot, which performs exactly the same function as your Linux skills to subvert and extend
/boot in GNU/Linux, and recovery, which is a special
partition used to update and recover your phone.
the computer in your pocket
www.linuxvoice.com 29
FEATURE ANDROID ANATOMY
internals is through the Android Debug Bridge, ADB.
ADB enables you to open a shell on your Android
device from a computer connected with USB, much
like connecting to a remote SSH server through a
network. This configuration means there are two
sides to this configuration. On the Android device,
you need to be able to activate its ‘debug mode.’ On
Above: Backing up through
the system partition, often referred to as a device’s many Android devices, including Google’s Nexus 5
ADB is the best way we’ve
found to keep your Android ROM, especially for devices that ship with their own and Samsung Galaxy phones, this is accomplished
device safe, especially as front-ends, is what the modding community spends by tapping 7 times on the ‘Build Number’ field in the
ADB can also work across a great deal of time doing. ‘About phone’ page, unlocking the Developer Options
Wi-Fi. recovery This partition loosely corresponds to the settings pane. Other phones, such as those running
Right: With ADB running, rescue partitions you find on laptops. It’s usually Meizu’s Flyme OS, include the option as a simple
you don’t even need to use booted into by holding down a button combination switch in the settings panel.
the command line. The when you turn your phone on – most commonly Before you can connect your Android device to your
brilliant QtADB lets you
power and volume-down at the same time, and it computer, you will need to install the ADB drivers and
perform lots of advanced
usually offers a way to restore the operating system do a little pre-configuration to make the connection
functions on your phone
with just a mouse. if it’s become corrupt or broken. But what the work. ADB is part of the Android development kit,
partition contains depends on the manufacturer. On which means if you’ve already done some Android
many devices, for example, after booting into the programming on your computer (such as following
recovery partition you can mount the partition on our short series on Android app development in
your computer and copy across an update/install LV004), you won’t need to install anything else.
file for Android. The recovery partition is also used Otherwise, you’ll need to either install the complete
to store and the ClockworkMod (CWM) or TWRP SDK or a cut-down version of the tools package, if
boot selectors/backup tools, as they can be booted your distribution provides one.
to with the recovery shortcut. You can find these by searching for adb or fastboot
data This will appear as userdata in the mount specifically (Ubuntu’s packages are android-tools-
command. This is where all those applications adb and android-tools-fastbootm, for instance).
you download are installed, keeping the system With everything installed, we need to explicitly state
partition untainted. This is also why you’ll find this which device we want to speak to. With the phone
partition mounted as read/write, although only by connected, type lsusb and look for something similar
‘root,’ which you’ll need to be if you want to look to the following:
around. If you need to back up your data, such as Bus 001 Device 008: ID 18d1:4ee1 Google Inc. Nexus 4 / 10
messages, emails and photos, this is the partition You need to add the Vendor ID (the 18d1 for our
you should focus on, as it’s also the partition that’s Nexus 5 above) to the file called adb_usb.ini, which
wiped by a factory reset. data is related to the cache should be within a .android folder in your home
partition, which stores temporary files that need to directory. As long as .android exists, you can do this
be accessed frequently. by typing echo 0x18d1 >> ~/.android/adb_usb.ini,
but you’ll need to change the Vendor ID for the one
ADB returned by your phone. Now connect your phone and
One of the easiest ways to access your phone’s type adb devices, which is a command that will list
ADB compatible devices.
All the exciting stuff happens with ADB is a daemon that runs in the background, so
this will need to start first. After a few moments, you’ll
Fastboot, which is most commonly see a prompt on your phone to accept the connection
used to reflash Android’s partitions from the computer and, after doing this, the output
from ADB will show your device is connected.
30 www.linuxvoice.com
ANDROID ANATOMY FEATURE
To open a shell that’s running on your device, type
adb shell. Just like running the Terminal app on the
phone itself, you’ll be able to run commands and
explore the filesystem. You can copy files to and
from your device using the adb pull and adb push
commands, with full path names for source and
destination as their arguments. You can also install
and uninstall packages with the install and uninstall
arguments, which is an essential utility if your phone
can’t connect to a network and doesn’t have any
removable storage. Anther great command is adb
logcat. This will output the system log of your device,
and continue to show new entries as they occur,
which is brilliant if you’re trying to troubleshoot some
element of your installation. Then there’s perhaps the
most essential:
adb backup -apk -shared -all -f backup.ab
This will back up all your self-installed applications If the worst happens, there’s a good chance you can even fix a phone that doesn’t turn
and the contents of your shared storage, putting on with SmartPhone Flash Tool – but it won’t restore the vital number for your device.
everything into the backup.ab file. When entering
this command, you’ll have to accept the request on your flash memory by typing fastboot flash recovery
your phone, optionally providing a password so that recovery.img. This only replaces the recovery
the backup file can be encrypted. Use adb restore partition, and you can perform similar tricks with both
backup.ab to write the contents of the file back to the boot and system partitions, if you have access to
your phone. Finally, to reboot your device into Fastboot, properly formatted image files.
type adb reboot bootloader. With older versions of Android, you could often
unzip any updates and see the files that were going to
Fastboot be copied to the system partition. It was sometimes
All the really exciting stuff happens with Fastboot, even possible to remove or replace those packages
which is a command most commonly used to reflash you didn’t want installed into the read-only part of
Android’s various partitions and to install different your phone. Recent updates for most phones will
operating systems and bootloaders. And because it’s now compress these files into a system.new.dat file
always messing with the filesystem, Fastboot needs that no longer corresponds to the contents of the
to be run outside of the operating system, which filesystem. Thanks to a great tool, however, you can
means restarting your device separate from both still unfold this date, change the contents, and roll it
the normal boot and the rescue partition. Many of back into a system.img file that can then be flashed
Fastboot’s functions will result in the loss of data, or using Fastboot. This tool is a Python script called
even a complete rewrite/erase of your system, so it’s sdat2img (https://github.com/xpirt/sdat2img), and
important you’re happy with these risks and have a you run it as follows:
current backup before going any further. Without the sdat2img system.transfer.list system.new.dat system.img
adb reboot, Fastboot is normally accessed by turning The end result is a system.img file that you can
on your device while holding down specific buttons send to your device with the fastboot flash system
– power and volume up on Nexus 5, Ubuntu Phone, system.img command. As system.img is also just a
and you’ll normally have to choose ‘fastboot’ from block device formatted as ext4, you can also mount
a primitive menu. You can then safely connect your it to take a look at its contents. Before your device
device to your desktop with a USB cable. can boot, or at least access the Bluetooth and Wi-Fi
With your phone/tablet in fastboot mode, type drivers, you will also need to make sure the ramdisk
fastboot devices. Just like the output to ADB, you and kernel for the system file flashed onto your phone/
should see your phone listed as a device and you can tablet. Luckily this is included in current update.zip
now execute fastboot commands. Fastboot itself is a files as a simple boot.img file that can be flashed by
USB protocol and command language for performing typing fastboot flash boot boot.img. It is possible
tasks on your device without any operating system. to update devices using just the system and boot
One of the first commands you should try is fastboot files, without going though the update process. This
oem unlock. Depending on your phone, this may allows you to update to Lollipop on the MX4 Ubuntu
erase everything and return your operating system Phone, for example, and it might allow you to create
to its default, but it also allows you to overwrite the your own installations by removing those apps you
recovery partition with something like the Clockwork don’t require, as well as a possible installation route
Mod utility. These will be downloadable as recovery. for Cyanogenmod for unsupported devices. But better
img files from places like the Cyanogen Project, built than anything, it gives you the ability to explore the
specifically for your device, and they can be written to inner workings of the computer you keep closest.
www.linuxvoice.com 31
FAQ OPENSTACK
OpenStack
Ben Everard’s favourite type of clouds provide snow.
His second favourite run on OpenStack.
and enables you to quickly and easily components has an HTTP API, so they
BEN EVERARD share databases between can be controlled from other software.
applications. This API is compatible with Amazon’s
At the time of writing, there were over web services, so much of the software
OpenStack? That seems to be 15 components that can be deployed, that’s been designed to work with this
in the tech news quite a bit. although any single OpenStack commercial stack can also work with
What’s it all about? deployment doesn’t have to use all OpenStack.
OpenStack started as a project of them.
between Rackspace and NASA. As you can see, many of OpenStack’s This all seems a little over the
The space agency wanted a private components are built on top of top. I can create virtual
cloud computing system and needed tried-and-tested data centre machines using VirtualBox, or
the technology to manage it. That was technologies. In most cases, they’re Gnome Boxes. Block Storage is easy
back in 2010. Since then the then, the also agnostic of the underlying enough to manage, and I’ve never
wider IT industry has adopted the technology, which means that you had a problem managing my
technology and it’s developed into a could use OpenStack to manage databases. What does OpenStack
complete stack for cloud deployments. proprietary services running in a hosted give me that I can’t do without it?
data centre, or you could host The important thing to realise
When you say ‘a complete everything yourself on servers you about OpenStack is the size of
stack for cloud deployment’, control. This puts the user in charge of the situation that it’s designed to deal
what exactly do you mean by that? technical decisions rather than locking with. If you have a single server running
OpenStack isn’t a single piece of them into a single technology. a few virtual machines and hosting all
software, but an ecosystem of the storage on it, then there’s no need
components. These components each How do all these components for anything this heavyweight.
deliver a single aspect of cloud come together from a user’s OpenStack is designed for enormous
infrastructure. To list just a few: point of view? Is there some form of organisations (like NASA) that need to
Nova, the compute component, OpenStack interface? manage whole data centres or whole
enables you to launch and control There’s an OpenStack groups of data centres.
virtual machines running on a variety component called Horizon, which When looking at it this way, it’s not
of technologies (including KVM, Xen is a web dashboard that brings that OpenStack makes things possible,
and Linux Containers). everything together into an easy-to- it’s that it makes them easy to run and
Cinder creates block-storage for manage website. This enables you to manage. There’s nothing that you can
use by the virtual machines created manage your entire cloud deployment do with OpenStack that you can’t do
in Nova. from your web browser. You don’t have something less buzzwordy, but rather
Trove is the database component to use Horizon though. Each of the than having to use different tools with
different interfaces for each thing, you
OpenStack is designed for enormous can use a standard tool for everything.
If you need to write custom software to
organisations (like NASA) that need to manage manage it all, you can use the standard
API for this rather than having to deal
whole data centres or groups of data centres. with each item separately.
32 www.linuxvoice.com
OPENSTACK FAQ
Let me get this straight:
OpenStack is a bunch of
pieces of software that I can use to
set up my own private cloud
services (virtual servers, backup
storage, authentication, etc) on my
own hardware?
Sort of. It’s the software to set up
a cloud on servers. This could be
a private cloud running on your own
data centre, or it could be something
you pay a hosting company to run for
you. It could even be a public cloud that
you rent out to a group of clients. The
point of OpenStack isn’t really that it’s
for public or private use, it’s that it’s an
open cloud platform that you can do
whatever you want with.
OpenStack sounds cool, but is
there any way of trying it out
without going to the not OpenStack’s Glance module enables you to store images that you can use to boot virtual machines at
inconsiderable expense of setting the click of a button. Think of yourself as the Ernst Stavro Blofeld of cloud infrastructure deployment.
up a whole data centre to run it?
Yes! Although in production, you’d HP and Cisco are all heavily involved as competitors, or does OpenStack
generally want several machines well. With so many of the big names in stand alone?
to run different parts of OpenStack server computing involved, you can be The most direct competitor is
(because if you don’t have that level of sure that OpenStack will be around for Apache’s CloudStack. This is
infrastructure, you probably don’t need quite some time to come. another cluster of open source
OpenStack), there’s a setup designed components designed to enable people
for getting set up and testing out on a With all those competing to roll their own cloud environments.
single machine. This is called DevStack, companies on the board, has Cloudstack, however, is significantly
and you can get it from http://docs. development ground to a halt as less mature than OpenStack and has
openstack.org/developer/devstack. bureaucracy and politics engulf far fewer contributors and
There you’ll find instructions on how to everything? deployments.
set everything up, but it’s all quite Not at all! There’s a new release The most widely used competitors to
straightforward. You just need to clone of OpenStack every six months. OpenStack are all closed source, hosted
a Git repository and run a script. As we write this, Kilo is the latest cloud infrastructures such as Amazon’s
Alternatively, there’s a test system release, but Liberty should be out by the Web Services (AWS), Google’s Cloud
available at http://trystack.openstack. time the mag hits the shelves (the Platform and the Microsoft Cloud. None
org. This enables you to spin up a few project follows an alphabetic naming of these are direct competitors though,
machines to see how everything works. process). Even with the six monthly because they all involve letting another
At the moment, you have to log in via releases, each new release packs quite company host your cloud for you, in
FaceBook to use TryStack, which make a lot of features. To give you an idea, contrast to the control that CloudStack
us love it a little bit less. the Kilo release notes were over 6,000 gives you. Using one of these options
words of dense feature descriptions. means potentially locking yourself into
You said that NASA and That’s an impressive amount of new a system that’s hard to leave and losing
Rackspace came together to stuff for just six months’ work. control of how your computing
make OpenStack, are they still The rapid advancement is no doubt resources are hosted.
developing it? due to the young nature of the project.
The project is now managed by Five years may seem like a long time, So, in a way, OpenStack is kind
the OpenStack Foundation. but it’s just the blink of an eye in of like the GNU/Linux of the
Rackspace is still heavily involved in enterprise software development. Over enterprise computing world? It’s a
this, NASA less so. The foundation is time, we would expect the number of set of open source components that
run by directors and a technical new features in each release to slowly let you take control of your
committee, whose members come tail off as the software becomes more information technology rather than
from a huge variety of companies. The mature. surrendering it all to a commercial
Linux world is represented by people entity.
from Canonical, Red Hat and SUSE. This all sounds very Exactly!
From the wider technology world, IBM, impressive. Are there any
www.linuxvoice.com 33
INTERVIEW ALLISON RANDAL
“The more effectively you
participate in Free Software,
the more benefit the project will
get out of it, and the more benefit
you’ll get out of it.”
34 www.linuxvoice.com
ALLISON RANDAL INTERVIEW
ALLISON RANDAL
PRESIDENT OF THE OSI
We meet the new president of the OSI, Ex-president of the Perl Foundation, chief
architect of Perl’s Parrot and long-time program chair for OSCON.
T
he open source conference, OSCON, been instrumental in Perl’s success, partly currently works at HP. Allison has also just
started out as the Perl Conference, thanks to her work on the Parrot virtual taken over from our own Simon Phipps as
which made Allison Randal’s machine. But Perl is just one aspect to her the president of the Open Source Institute,
tenure as its program chair particularly career in open source. As well as working which meant that when we discovered we
fitting. She’d worked with Larry Wall, the at O’Reilly, editing and publishing her own were in the same time zone has her, we had
creator of Perl, for a long time, and has books, she’s worked for Canonical and to grab a few moments of her time.
Do you miss being the program computer languages don’t really have dense legal document that’s almost
chair for OSCON? that in mind – instead, Iit’s just like, impossible to read and it’s the same
Allison Randal: I do, I do! It’s like a “how do I mathematically express this language as a children’s book, but
little nostalgia when I come here every problem”. So I do think it does appeal to, you’ve just expressed yourself in a way
year. It’s like the old home camp. not just linguists, but poets and writers for a certain circumstance.
that make that transition.
It must have been difficult Do you think the same will be
giving that up, and for every It’s not the easiest true for Perl 6 and its multi-
other outgoing chair. programming language though, paradigm capability?
AR: We have a tradition of skipping a so we’re surprised that any non- AR: I haven’t been following it closely
year. Nat (Torkington) skipped a year technical people would find some lately, but when I was involved in the
when he created it, I skipped a year kind of fluency in Perl. design of Perl v6, we did have a goal of
when I created it, I just didn’t even AR: Think about how complicated making it easy to understand. That
attend. But they’re doing a great job. English is! I think, if you don’t keep it whole getting started easily was still
Now it’s actually quite exciting to come very clean and simple, it can be very much a very big part of it, and I feel
back and see it. They’re carrying on complicated and you can end up like it still had that. It went through a
with it and still going strong. making this very complicated program cycle of getting very complicated and it
that no one can ever understand, and seems like it kind of came back around
You’ve done quite a few things that’s why Perl kind of ended up getting to being more focused on being easy to
in quite a few places, and it’s a bad reputation. You can write a very use. I don’t think it’s so much of a
fascinating that you started off as a
linguist – this seems to be a Perl
thing, what with Larry Wall doing
something similar. What is it about
Perl that attracts linguists?
AR: In my case it was a very direct
connection, which is where I met Larry
and we started talking about linguistics,
and that was why I got involved in Perl.
Although, I guess I was using it at a
startup before that, but that was why I
got involved in developing Perl. So in
that case, yes, it was absolutely
because I was a linguist that I got
pulled into it. I think the way Larry
thinks, he thinks about language and he OSCON, the gathering of geeks on America’s
West Coast, wouldn’t exist without Allison’s
thinks flow of language, so the way you
guiding hand, which means we wouldn’t have
wrote constructs for Perl was very like a developed such a taste for US-style IPAs.
natural language, whereas a lot of
www.linuxvoice.com 35
INTERVIEW ALLISON RANDAL
As the former Technical
Architect of Ubuntu, Allison’s
done more than most to make
Linux and Free Software
approachable by non-techies.
problem having those paradigms mixed HP was one of the biggest We’ve really struggled since then, since
in, in much the same way that the early users of Ubuntu and one of the that was about 2010, to find what we
Perl mixed in a lot of paradigms. It few to actually pay for it. In what want to do next. It turns out there hasn’t
mixed in like procedural and then it other ways does HP use open been an obstacle. There’s a survey by
added object orientation onto that. source? Black Duck [www.blackducksoftware.
AR: I don’t cover the whole company, I com] that shows corporate use. So
Do you still do much Perl cover HP Cloud. HP’s cloud product is 2010, it was 48% of respondents said
development? OpenStack and it’s almost pure they’d use open source, in 2015 it’s 78%
AR: Not much, no. My last job before I upstream OpenStack. There’s a few with about 68% contributing and 88%
moved to HP, I was a CTO at a startup patches, but they consciously push saying they plan to use or contribute to
that was all Perl. them back upstream as fast as it in future. So it happened anyway.
possible. And there’s also Cloud
So that was after Canonical?
AR: Yeah, it was in between the
Foundry, which is another big one that
we’ve heavily involved in.
Companies are driven to
two. But at HP I haven’t done a whole Recently we’ve switched over contribute upstream for
lot of development. My main role is to doing deployment with Ansible
open source strategy. One of the big (orchestrated configuration practical reasons
things I do is that I maintain this management for networks), that’s
30-page strategy document that covers another big one that we’ve been very For the current generation of
all the open source projects that HP involved in. Debian too, for the machine developers it’s the only way to
invests in. stuff like trying to explore new kinds of work. That’s amazing. We never
It starts by setting the principles, like hardware for storage, where you can thought that would happen. But it
contribute upstream first, and why merge memory on disk, so you get fast brings bigger challenges. We’ve
we’re doing what we’re doing, but then disk and flexible allocation. So they’re heard copyleft licences being talked
it also goes through each project and doing a lot of work on the Linux kernel about by open source developers as
lays out what the project is, here’s why and on Debian to try to have a fully being viral, almost for the first time
we care about it, here’s our priorities, functional distro on this hardware that’s in years (this was at a Lightning Talk
here’s where we’re investing and why, still very experimental. one evening at OSCON), which is
sort of keeping open source as a worrying. And maybe there’s a move
fundamental part of the business. It’s a What would you consider the towards permissive licences
place where you invest and get value. biggest successes and failures because they feel that the job has
Just like anything else, you need to of open source? already been done.
have that same consciousness that AR: The biggest success is that we did AR: I’m kind of in two minds on that.
you have with monetary resources or actually get corporate adoption. That Companies are driven to contribute
human resources. was the beginning of open source. their changes back upstream for
36 www.linuxvoice.com
ALLISON RANDAL INTERVIEW
practical reasons… for example, forks anyway, for practical reasons, will also Is that something that you
are very expensive to maintain, so make it them less afraid of copyleft want to do as part of your role
pushing your patches upstream because it’s no longer a scary thing to at HP and as president of the Open
reduces the cost of maintenance, and have to contribute your changes back, Source Initiative?
fixing bugs upstream and adding it’s just normal and what they have to AR: Yeah, it is. That’s kind of my whole
features upstream benefits the do anyway. role at HP, is like, “No, no. Look it’s in
company. So there’s that economic your best interests to contribute your
drive to contribute your changes back Do you think the initial changes back,”. Whatever the licence, it
anyway. To a certain extent, I think momentum for the GPL to doesn’t matter, you still need to
copyleft in the 80s, when it was make sure everything is contributed contribute back.
introduced, was essential. back might be lost?
AR: GitHub recently did a licensing And is HP happy about pushing
We wouldn’t have got here survey and they found that the MIT changes back up stream?
without it. Licence was the most common licence AR: Yeah.
AR: Not all the time, but in many cases on there. I think the important thing to
you can get the same effect without it, do is keep in mind that it is very Is that something that’s
so I’m not worried about permissive important to educate these consumers changed recently?
licences. On the flip side, I think the fact of open source and free software that AR: It’s an attitude that’s changing all
that companies have learned that they they need to contribute their changes over, it’s not specifically HP. But since
have to contribute their changes back back [to the projects]. about 2000, yeah, it’s an attitude that
has changed. It’s just coming to grips
with the nature of the software industry.
In some sense, they see their
competitors doing it, and they’re seeing
all this money, and then we’re going to
be left out if we don’t follow, and then it
ends up spreading and spreading.
So where is the challenge if
most companies get it?
AR: The challenge is that they all get it
but they don’t all get it to the same
degree. There’s a difference between
understanding software freedom and
just being driven to it by economic
necessity. That’s part of why I talk
about education. You still need to learn
the full benefit cycle, and you still need
to learn how to participate effectively
because the more effectively you
participate, then more benefits the
project will get out of it, the more
benefits you’ll get out of it.
The way I explain it that seems to
appeal to people is that it’s a
competitive edge. So using Free
Software and open source used to be
the competitive edge, but now
everybody does that. So now the
competitive edge is participating
effectively, and if you’re participating
more effectively than your competitors,
you will get more bugs fixed, you will
get more features added, you will get
your problems solved, you will have an
edge over your competitors and it will
Where there’s disagreement, it’s benefit business.
part of Allison’s job at the OSI to
remind us that we’re all on the
same side. Is there a good example of a
company that gains that
www.linuxvoice.com 37
INTERVIEW ALLISON RANDAL
system over in that direction.
Free Software needed the
formality of the rules system
that Richard Stallman created to be
able to create a framework, which is
probably becoming relaxed now.
AR: Yeah, that’s an interesting thought.
I’ve been thinking a lot about
inevitability of various things lately,
about specifically the open source and
Free Software movement and what
was inevitable. And maybe it was
inevitable that some kind of split would
happen. One grew under another name
or some kind of schism happened, but I
still hope that we merge back over time.
Is there anything that you can
do as president of the OSI to
be able to build that bridge?
AR: We work closely with the FSF (Free
Software Foundation) and we do
collaborate on things when we can, like
amicus briefs for various cases that
have significance for both movements.
Canonical is one of the
companies that totally
You can watch Allison’s
keynote from OSCON
gets software freedom
here: www.youtube.com/
watch?v=egEO1L8EHJU. The blog I’m writing, connected to my
keynote, explicitly talks a little bit about
the history of Free Software and open
competitive edge? For example, we AR: To my mind, the biggest failure has source and the drivers for the split, but
like Canonical but it’s often been the tension between Free it also ends on the note that, as free
criticised for not doing enough, Software and open source. I think that software and open source succeed
certainly upstream. has actually hampered both then we don’t really need to talk about
AR: If you look at the history of their movements in a way. The fact we have tactics so much anymore. We all agree
company and their business, a lot of the different emphasis is fine, I don’t think that we want software freedom to win
things they struggled with have been that’s a problem. But for a long time, and that there are inherent problems
specifically in the areas where, if they and not so much any more, it’s less and with proprietary software, but the free
had been contributing regularly and had less over time, but for a long time the software movement is focused on
been participating in the wider world, two movements kind of undercut each boycotting and a specific set of tactics.
they could have just eliminated that other, which was too bad. And I think it
entirely. They never would have had a slowed the overall progress of things. Do you think the FSF is
problem in the first place. We think you’re right, but it’s becoming more independent
still a difficult and thorny and using a more pragmatic
But they do get open source. subject. We greatly respect Richard approach before it becomes too
AR: They do, and that’s what’s so Stallman and, in an ideal world, he’s marginalised?
hard. They’re one of the companies that right. But if he’s not the best AR: I’ve been pretty pleased over the
totally gets software freedom, but ambassador, we perhaps need his past few years to see what’s coming
they’re not participating effectively and unmovable principles. out of the FSF. I’m good friends with
it’s hurting them, which is sad. They’re AR: Yeah, in some ways it’s like John John Sullivan, he’s the executive
doing fantastically. the Baptist. Not everyone’s where he is, director (see Linux Voice issue 19 for
and probably we’ll never be where he is, our interview with John). He’s a
Has there been any failure that but just the fact that he’s there and moderating influence. He really does
we could learn from? sticks to his principles, shifts the whole represent the future of Free Software.
38 www.linuxvoice.com
SUBSCRIBE
Subscribe UK READERS!
Did you know that you can subscribe to Linux Voice from
just £10 per quarter with Direct Debit? Get every issue
straight to your mailbox (or inbox) and spread the costs!
What you get
100 pages each month
of the best tutorials,
features and interviews
Access to all back issues
in DRM-free digital formats -
over 1,500 pages
Take part in our yearly
profit donating scheme,
and help FOSS projects
Yearly Direct Debit prices
UK print subscription – £55
Digital subscription – £38
Quarterly Direct Debit prices
UK print subscription – £15
Digital subscription – £10
Go here now to subscribe!
www.linuxvoice.com/shop
Payment is in Pounds Sterling. If you are dissatisfied in any way you can cancel your subscription at any time and receive a refund for all unmailed issues.
www.linuxvoice.com 39
LISTEN TO THE PODCAST
WWW.LINUXVOICE.COM
BUY MUGS AND T-SHIRTS!
shop.linuxvoice.com
INTRO REVIEWS
REVIEWS
The latest software and hardware, rigorously bashed against a wall by our crack team.
On test this issue . . .
42
Bitwig
Studio 1.2
Ableton Live gets a worthy
competitor, and, amazingly,
one that runs natively on
Linux. We can’t stress
enough how pleased we
are that this excellent audio
Andrew Gregory production software is here.
Studied economics and psychology at one of What a time to be alive –
the UK’s most prestigious universities. praise be to Saint Moroder!
Y
ou don’t need to have studied
economics and psychology at
one the UK’s most prestigious
universities to see that the egregious
fraud perpetrated by Volkswagen is
unlikely to be a one-off event. Any car
maker doing the right thing, working Gnome 3.18 43 Unity 44 Devolo 1200+ 45
hard and playing by the rules is going to More features, more refinement, If you feel like making squillions Powerline Ethernet has never
be at a disadvantage to the companies more silly characters – the little of pounds coding the next AAA been so reliable, so easy to use,
that cheat, so the incentive to cheat is desktop has all grown up. game, why not use this engine? so effortlessly jazzy.
enormous. I would be hugely surprised
if more auto makers weren’t involved in
this disgusting scandal, and if I were a
Saab shareholder I’d be even more
Group test and books
annoyed with VW than I am now.
Of course, they were only able to get
away with it for so long because the
software is closed source. If we were
allowed to look at the source code for
the software in our cars, the increased
transparency would put pressure on
manufacturers to be as efficient as
possible, knowing that any false claims
could be scrutinised.
Of course, that’s not the way they see
it. But if your reputation is built on
reliability, it’s an awful shame to throw it Group test – educational software 50 Books 48
away for the sake of an outdated Computers for kids aren’t just idiot-proof iPad Praise be, for Hello Ruby has arrived! Only a year
software development model. distractions – there’s some serious learning and three quarters since its author crowdfunded
andrew@linuxvoice.com potential in even the lowliest Linux machine. $380,000 to publish it. We hope it’s good.
www.linuxvoice.com 41
REVIEWS BITWIG STUDIO 1.2
Even though it’s complex, you can control
everything you see, which helps with
performance, control and responding
quickly to audio problems.
Bitwig Studio 1.2
Still in rapture, Graham Morrison tests an update to the best audio software around.
V
Web www.bitwig.com ersion 1.0 of Bitwig Studio, which we reviewed using your audio hardware’s ALSA drivers, which is
Developer Bitwig GmbH
Price €259.00 in issue 4, was a huge milestone for the audio especially useful for laptop music making, where
production landscape. Eighteen months later, the on-board audio rarely works well with Jack. Our
there have been several significant updates, but none second favourite change deals with the practicalities
have been on the scale of this 1.2 release. It’s a major of audio production, and that’s the ability to group
overhaul, adding all kinds of improvements to what tracks together. On medium-sized projects, and
was already a unique implementation of a music even our podcast, putting tracks together in what is
production environment. effectively a virtual folder is an essential. This is where
What Bitwig does (if you don’t have your CC-BY SA we keep the various parts of Brad Sucks’ awesome
copy of issue 4 handy) is build pieces of music out music, for instance, so that his tracks don’t get in the
of audio and MIDI clips, internal and external sound way of the other audio. Bitwig’s implementation is a
sources, effects and virtual instruments, allowing little clunky, especially compared against Cubase, but
them to trigger and arrange them into both scenes it’s a step in the right direction.
and more traditional timelines. And it’s brilliant.
The first impressive new feature is that, like Love to love you baby
Ardour, the Jack audio connection kit is no-longer We also love the way you can now audition audio,
a prerequisite. You can now start Bitwig normally effects and sound generators from the new browser.
It works perfectly and is the best implementation of
this idea we’ve seen. The new oscilloscope plugin is
also brilliant for troubleshooting, it lets you to visualise
audio even when you can’t hear it. The huge range of
new audio packs, the new Delay 4 effect, and presets
give you more options than ever for creating music.
Bitwig is expensive, but if music production is your
The preset browser now
thing, there’s nothing like it on Linux.
gives you an instant
preview of audio and
Turn your Linux laptop into an audio production,
effects, helping you choose nightclub DJ, synth and effects workstation.
the right sound for each
new track.
42 www.linuxvoice.com
DESKTOP ENVIRONMENT REVIEWS
Gnome 3.18
Like clockwork, a new Gnome release is here. Mike Saunders checks it out.
T
hese are good times for Gnome. Early Web www.gnome.org
Developer The Gnome Project
releases in the 3.x series caused much Licence GPL and LGPL
consternation, but after years of
improvements, we know many Linux users who
abandoned Gnome early in the 3.x cycle but have
since returned and are loving it.
Gnome 3.18 contains a mixture of new features
and refinements. The file manager now sports Google
Characters is one of the
Drive integration and has seen lots of work to improve
new additions to this
usability, with progress windows for file copy/move release: a trivial little tool
operations replaced by a single button in the toolbar, for finding Unicode
and an “Other Locations” entry in the sidebar for characters like, er,
commonly used network locations. File and folder snowmen.
renaming and creation is now smoother, using dialog
boxes and popovers. machine’s firmware (eg your BIOS) from a GUI thanks
Two new programs have been added to this to the Linux Vendor Firmware Service initiative,
release: Calendar does as its name suggests, drastically simplifying a job that normally requires
with support for linking to online calendars and much fiddling at the command line.
switching between month and year views. Then Gnome 3.18 is a solid update from the Gnome
there’s Characters, a browser for numbers, letters team, and we’ll give them extra credit for their
and Unicode symbols. If you like to paste pictures communication efforts. The Gnomers really work hard
of pandas into your emails or IRC chats, your life will to demonstrate what’s new in each release, with short
become slightly easier with this release. videos and attractive release notes. If you’re one of the
Hardware-wise, Gnome 3.18 has a couple of new ex-Gnome users who gave up on the desktop a couple
features worth shouting about: automatic screen of years ago, now’s the time to give it another shot.
brightness is now supported, so if you’re on a laptop
with a light sensor, Gnome will adjust the brightness Mostly refinements in this release, with a couple
of new goodies tossed in to round it off.
to match your surroundings (and potentially save
battery life). It’s also now possible to update your
www.linuxvoice.com 43
REVIEWS UNITY 5
The example projects show you how full
games are developed without your having
to go through the whole process yourself.
Unity 5
Now that a new games engine runs on Linux, Ben Everard gets interested in coding.
U
Developer Unity Technologies nity is an Integrated Development interactions, gravity and most of the aspects of
Web www.unity3d.com
Price Free or $75 per month Environment (IDE) for developing games, creating a world in which the game exists. C# scripts
which comes complete with an engine to can then be used to fine-tune this world and the
handle most of the tricky parts of games development gameplay within it. As a professional-standard games
(such as animations and physics). When combined engine, Unity gives the user a huge amount of control
with MonoDevelop (which handles the C# scripting), over how everything is done, and this leads to the
you get everything you need to make top-tier games. interface being quite complex. Fortunately, there’s
Unity is free (as in zero cost), but only if you or your good documentation, and a good selection of tutorials
company earns less than $100,000 per year. Once to help new users get started.
you pass this threshold, you need to pay $75 per Unity includes an asset store that enables users
month plus an additional $75 per month for each to buy and sell parts of games including 3D models,
mobile platform you want to support. Alternatively, animations and particle systems, which can help you
you can pay $1,500 (plus an additional $1,500 per to very rapidly build up a high-quality game even if you
mobile platform) to buy a licence outright. have little artistic flair. However, it does add another
Games development in Unity is mostly point and layer of proprietary licences to your game.
click to bring various assets and animations together Learning Unity is a serious task, and someone
in the game engine that can handle collisions, interesting in playing around with games development
would probably find it much easier to use something
simpler, perhaps even just a games library in a
language they already program in such as Pygame.
Freedom-loving developers will also be put off by the
licence. However, if you’re interested in serious games
development, and are happy relying on a proprietary
engine, Unity is probably the best option on Linux at
the moment.
The asset store is bundled
with Unity or viewable in
Linux gains a great development environment,
an external browser at but one that comes at the expense of freedom.
www.assetstore.unity3d.
com.
44 www.linuxvoice.com
POWERLINE ETHERNET REVIEWS
The size of these devices is a little prohibitive, and we’d
always like more Ethernet ports, but the extra power
socket has proven very useful in tight installations.
Devolo 1200+ Wi-Fi Starter Kit
Graham Morrison finds a way to work from the shed at the bottom of the garden.
B
roadband adoption is at critical mass and Web www.devolo.com
Developer devolo AG
everyone wants Netflix in high definition Price £160
delivered to their bedrooms. But Wi-Fi has
problems with both range and bandwidth, especially
in the average household with lots of devices,
unfriendly partitions and microwave dinners. Which is
Despite using the
why powerline Ethernet adaptors have become such
deprecated Adobe Air,
an essential part of home networking. They pipe your
Devolo’s Cockpit utility
network through your electricity cables, turning any does provide a useful
power sockets into RJ45 Ethernet ports. Or in the overview of speeds
case of the this package from Devolo, double Ethernet between devices and the
and a Wi-Fi access point. Ubuntu installation is easy.
The kit we’ve been sent includes two adaptors –
one intended to work as a hub and containing a single with a closer socket. These figures are way off the
Ethernet port, and another with two Ethernet ports 1200 hinted at in the product name, but we’re immune
and a comprehensive Wi-Fi access point. It’s perfect to this arbitrary rating and these units are much better
for extending your network more than a single floor than other devices we’ve tested. 2.4 and 5GHz
upwards, or into the garden. In fact, this was how we concurrent Wi-Fi was equally simple through the web
tested the kit, and the 1200+ Wi-Fi adaptor proved interface and we were very impressed by the access
faultless. You press the button on a device that’s point’s capabilities. A physical button toggles the
already part of the network, run down the garden, Wi-Fi, and there’s a scheduled timer too, plus guest
quickly plug in the new adaptor and press its own accounts and excellent parental control. This kit is
button, synchronising with the existing network. expensive, but we feel that in this bleached white
plastic case, you get what you pay for.
Devolo in the detail
There was a very real and repeatable improvement If you need network connectivity in a remote
over the pre-500av era of Devolo devices, and it was a part of your estate, these units perform most
excellently, and are worth the expense.
similar story with speed. Download speeds in the
shed were around 150Mbps, increasing to 250Mbps
www.linuxvoice.com 45
REVIEWS GAMING
GAMING ON LINUX
The tastiest brain candy to relax those tired neurons
PLAYING CATCH-UP
Company of Heroes 2
A solid World War II RTS game.
T
here aren’t a whole lot of Real Time
Strategy games on Linux, especially not
of the calibre of a franchise like Company
of Heroes. The game focuses primarily on the
eastern front playing as the Soviets, with a few
smaller single-player and multiplayer campaigns
taking place on the western front and more
Michel Loubet-Jambert is our Games
Editor. He hasn’t had a decent night’s available as downloadable content.
sleep since Steam came out on Linux. Focussing on the eastern front is a refreshing
choice, given that certain parts of the war All the major fronts in the European theatre of war
T
he 1,500 Linux games on have been done to death in WW2 games. The are playable – this looks like North Yorkshire.
Steam milestone has been story the game presents, where an imprisoned
reached, and though this
Soviet lieutenant recounts his experiences, isn’t The game focuses controlling small squads
doesn’t seem much compared with
the 6,500 or so games for Windows, it particularly profound, but does well in outlining rather than entire armies, however it is also
is indeed very significant. At the the events of the war to those not entirely not uncommon for numerous simultaneous
same time, this can only grow over familiar with it. skirmishes to occur, testing the player’s
the coming year, with SteamOS on The gameplay in Company of Heroes 2 is multitasking abilities under pressure. There are
the horizon and the coming
excellent, and shows that RTS is the best way also some great features which reflect the harsh
improvements on the AMD driver
front, as well as a greater of conveying warfare and all its chaos, with a realities of the front, such as the extreme cold
commitment to the Vulkan API, with persistent pressure on the player as fascist slowing troops down and eventually killing them,
the latter being increasingly troops storm the city or as soldiers desperately forcing the player to find sources of warmth on
discussed publicly by the big scorch earth before enemy troops advance. the battlefield.
hardware companies.
Similarly, the sheer expendability of the Soviet
What the numbers don’t show, Website http://store.steampowered.com/
however, is a fundamental change in troops portrays the harsh and desperate realities
app/231430/ Price £29.99
the types of games we’re seeing on of the front.
Linux. A year ago, the Linux gaming
space was dominated by indie titles,
but there are now a great deal of
large developers porting in-house or
outsourcing to a number of porting
houses, most of which also port
games to OS X. Along with this,
spending habits are also changing,
with Linux gamers buying fewer
games at higher prices as more
people move from bundle-style
games to AAA mainstream titles,
which are considerably pricier.
There are still some very big-name
developers missing though, such as
Blizzard, Bethesda and Rockstar. The
biggest is certainly EA, which joined The fast-paced action
Blizzard recently in stating that it has often necessitates a lot
no plans to support Linux until the
of multitasking.
market share is bigger – though the
acknowledgement suggests that
many out there are keeping an eye on
Linux Gaming to see how it’s evolving,
and waiting to see if SteamOS and Real-time strategy (RTS) is the best type of game
Steam Machines are a hit.
to convey warfare and all its chaos
46 www.linuxvoice.com
GAMING REVIEWS
Satellite Reign ALSO RELEASED…
A worthy spiritual successor to the Syndicate series
A
lot of game franchises have Unforeseen situations cause plans to
been brought back from the go out of the window as the player has
dead recently, and Syndicate to quickly adapt to circumstances. There
is the latest to undergo the “spiritual is also a good sense of progression as
successor” treatment. One of the things stats and weapons are upgraded and
Satellite Reign does best is portraying its new skills learned, but after a while, many
dystopian cyberpunk world ruled by mega- of the missions start to feel repetitive
corporations with private police forces. – something which could have easily Adventurezator: When Pigs Fly
Unfortunately, its superficial plot and been fixed with more dialogue and story. This “what you see is what you get” point-and-
lack of characters makes many of its However, Satellite Reign is a great game for click adventure game maker enables just about
positive points feel like lost opportunities. those who enjoy squad-based tactics. anyone to make their own game with no
programming skills whatsoever. At the same
Though there is a story, the game is more time there is also a (pretty good) adventure
of an open world where players attempt Website http://store.steampowered.com/ game included and the ability to play other
app/268870/ Price £22.99
break-ins into corporate facilities. people’s creations on the Steam Workshop.
There are a tonne of nice features that give the
ability to create professional-looking games.
http://store.steampowered.com/app/300280
Satellite Reign has one of the
Train Valley
most convincing cyberpunk On the surface, this game looks like a railroad
worlds seen in gaming. sim, like Railroad Tycoon, but it is actually a
micromanagement puzzle game. It’s a nice
little casual game that’s visually appealing and
tonnes of fun for those not willing to commit
The Stanley Parable tens of hours into a game. It still shouldn’t
disappoint fans of railroad simulators – if one
expects a different style of play – given that
Dark, puzzling and hilarious. there are many fundamental similarities.
I
http://store.steampowered.com/app/353640
n this game about choice and free
will, the player takes control of
Stanley, employee #427 of an newly
abandoned office whose employees
formerly followed orders from a screen,
mindlessly pressing buttons on terminals.
The game is short and simple, and
being too descriptive would give a lot
away, but in essence the story is told The story’s many paths and endings all begin
through the choices the player makes. at desk 427.
There are numerous possible endings,
Dropsy
and despite some taking only a matter of the fourth wall provided by the narrator A bit creepy, a bit strange, but also oddly
of minutes to find, it can take a few – the only character in the game with heartwarming, Dropsy is the tale of a clown
hours to find them all. As clichéd as this spoken dialogue and the closest thing who wants to help people and give hugs. The
has become in recent years, The Stanley it has to an antagonist. Combining the game employs traditional point-and-click
Parable is more of an experience than a humour with the experience is why this gameplay and graphics, though the story is far
from traditional, with many underlying themes,
game, and though it was one of the first in game is so highly recommended – it’s some of which are pretty dark. The game does
this new genre, it’s still one of the best. thought provoking and fun. a lot of interesting things and it’s worth getting
Other than the game’s portrayal of if you enjoy creativity in video games.
choice in video games, its major strength Website http://store.steampowered.com/ http://store.steampowered.com/app/274350
app/221910 Price £9.99
is its dry humour, with subtle breakings
www.linuxvoice.com 47
REVIEWS BOOKS
A Peek At Computer Electronics
Ben Everard learns not to poke around inside a desktop case using his finger.
Author Caleb Tennis
Publisher Pragmatic Bookshelf
Price $22
ISBN 978-0-9776-1668-8
A
uthor Caleb Tennis takes the into a gap by being too technical for a casual
reader on a 250-page journey from reading book yet too impractical to be a
the discovery of electronics to technical book.
semiconductors and the inside of a modern A Peek At Computer Electronics delivers
computer case. That’s a long journey and exactly what is promised – a peek at the
250 pages is not a lot to accomplish it in. subject – and it does this well. It’s engaging
There’s a clue in the name here. A Peek At and there are plenty of diagrams to help
Computer Electronics really does nothing explain the contents. However, this brief
more than give the reader the briefest of peek at electronics just isn’t what we, as
overviews of this enormous subject. readers, want. We can’t help but feel that
By being rushed through everything, the most people would be better off with a book
reader is left without any useful knowledge that focuses more on a single part of the
of any area. They may know what a subject area and covers it in enough detail to
transistor is, but they won’t have any real leave the reader with enough knowledge to
idea of how to wire one up, or in what way be of practical use.
computers are built from them. They may
know the voltage changes in a Ethernet Rushes through too many topics to leave
the reader with any useful information.
cable, but they don’t know how to use that Yes, valves are covered in this book, despite not
knowledge to improve their networks. It falls being used in computing for half a century.
Dart 1 For Everyone
Ben Everard steps up to the oche and learns a new language.
Author Chris Strom
Publisher Pragmatic Bookshelf
Price £15.99
ISBN 978-1-94122-225-6
L
et us be completely clear about entirely focussed on front-end development,
one thing: Dart 1 For Everyone is not yet many of the projects also need a back
suitable for everyone. The book dives end, and that’s not covered at all despite
straight into programming and assumes Dart having the capability to run on the
that the reader can follow along without server. Perhaps, then, the book should be
any explanation of how coding works. It titled Dart 1 Web Frontends for JavaScript
doesn’t even explain how Dart or JavaScript Programmers, but I suppose that’s a lot less
scripts run inside HTML in the browser. snappy. That’s the end of the caveats.
This book would be far better titled Dart 1 Dart 1 For Everyone particularly focuses
for Programmers, or better still, Dart 1 for on the aspects of Dart that are different to
JavaScript Programmers. With this caveat JavaScript (such as manipulating the DOM)
out of the way, let’s take a closer look. and on the aspects that are important for
Dart is designed for building web modern web apps (such as the Model-View-
applications, and Dart 1 For Everyone wastes Controller (MVC) architecture).
no time in showing off its power in this
regard. In the very first chapter, the book
dives into a web front-end for a database A good introduction to Dart 1, but makes
promises that it doesn’t keep.
that grabs the information it needs via AJAX. This book contains no useful advice on getting
This leads us to the second caveat: it’s an elusive 180.
48 www.linuxvoice.com
BOOKS REVIEWS
Hello Ruby
Graham Morrison rants a little at a crowdfunding prodigy. Also released…
Author Linda Liukas
Publisher Macmillan
November 2015
Price £11.23
ISBN 978-1250065001
Clojure for the Brave and True
H
ello Ruby started life as a
Kickstarter campaign just We admit, we chose this
book purely on the
as Linux Voice completed
strength of its cover. It
its own Indiegogo project in January looks like a demonic
2014. It was hugely successful, with Viking riding a wild boar,
thousands of backers pledging a of course. But it also
total of $380,747 to turn Linda Liukas’ made us realise that we
know very little about the
dream into reality. With hindsight,
Lisp-based Clojure
the project’s success is perhaps programming language
no surprise. Linda did an amazing other than how its
job at selling the idea of a children’s pronounced (like closure),
book that could teach programming Hello Ruby is about ideas rather than so perhaps we should read
the words as well as look Nobody tosses a Dwarf.
fundamentals through illustrations programming specifics.
at the nice pictures. Or, indeed, a Viking.
kids would respond to and a story
more in common with Julia Donaldson and pointing at things. It’s a simple
than Bjarne Stroustrup. story that follows a fierce little
But the book’s journey from idea girl called Ruby as she takes on
to reality has been a difficult one, some challenges set for her by Learning Predictive Analytics
especially for many of its backers her father. There’s nothing specific If you enjoyed this
(and we enthusiastically backed the about programming, or even about month’s tutorial on
project at the time). The book is over technology, other than character statistical analysis with
a year late, with many complaining names like Tux and Django, but the the GNU R programming
language (see page 84),
their children are now too old for plot is constructed in such a way there’s an entire book on
the narrative. More importantly, the that Ruby has to solve problems and the same subject, using
book now has the backing of a major create solutions, useful skills for both the same language. It’s
publisher, Macmillan. This has meant programming and every day life. going to be especially
Kickstarter backers and Amazon The book ends with an activity useful when you start
downloading all that open
pre-orderers are getting the book at section, originally promised as a data from your
almost the same time, yet Amazon’s separate workbook in the Kickstarter government, calculating
customers pay far less than the $60 campaign. This involves putting MPs expenses and the Turn to p84 to see if Linux
we pledged as Kickstarter backers. instructions together and does rise in rail fairs. survives without Linus.
introduce lots of programming
Continue rant terminology and ideas, testing the
We understand crowdfunding and the reader to come up with their own
huge challenges involved in getting solutions in a form of pseudo-code. It Unity AI Game Programming
things published. But it has been works well, but there’s a sharp jump in
Now that we have a native
very difficult getting over the feeling difficulty from the reading of the story Linux version of the Unity
that we, as backers, have subsidised to the challenges of the workbook. games engine editor (see
what should have been Macmillan’s Ignoring the negativity, Hello Ruby page 44), we need to
investment in the title, with money is a successful blend of storytelling, prove that this porting
that could have gone to other projects illustration and challenges. As the effort was worth the
trouble by creating
supporting children and technology. product of a dedicated individual who awesome games on Linux.
We’re disappointed that neither Linda has put all this together, it’s still hugely If only we knew how.
nor Macmillan seem to be addressing impressive. But as the work of a major Maybe we could code
a problem that we feel will only publishing company, we can’t help feel some AI that will do the
impact similar worthy crowdfunding a little short-changed. job for us, then we can
hire Linux Voice TNG and
campaigns in the future. retire to Monaco with The Unity games engine is
The book itself is beautifully As a crowdfunding campaign, it’s slightly
depressing. As a book, it’s rather good. Jenson Button. not the Unity desktop.
illustrated, with enough detail on
each page to keep children amused
www.linuxvoice.com 49
GROUP TEST EDUCATIONAL SOFTWARE
GROUP TEST
Graham Morrison takes a look at a range of software that may be slightly more
productive for children than hours spent watching ukulele videos on YouTube
On test Educational software
H
aving spent half a lifetime single age or schooling range, and
Scratch playing video games, we’d also because the software that is
URL https://scratch.mit.edu argue that there’s available is rather disparate.
Licence GPLv2 educational value in almost any There are applications that deal
Latest Release 1.4/2.0 (web) kind of software – even with a single category, for example,
The best-known tool for teaching SuperTuxKart. But for those of us such as astronomy or geography,
children how to program.
with responsibilities to the young while there are also tools that
folk, finding something a little more aim to provide an entire suite of
TuxPaint brain nourishing than reflex learning, cramming hundreds of
URL www.tuxpaint.org attunement is a good way of activities into a single application.
Licence GPL introducing computing technology, Then there are the programming
Latest Release 9.22 Linux and learning, before letting environments, or even flash cards,
The Linux equivalent to Deluxe Paint is your little ones run free with your that are completely open and
simple artistic fun. pile of Gentoo install discs. It’s also versatile to any use.
a great way of discussing some of To side-step these issues
TuxMath the ideas behind open source,
especially when so many children
while hopefully still providing a
great overview of the software
URL http://tux4kids.alioth.debian.org
are now used to the freemiuum available, we’re going to focus
Licence GPL
Latest Release 2.0 business model (free to download; our comparison on whether an
Kid-friendly graphics and sounds help pay through the nose if you want to application succeeds in engaging
gamify simple maths lessons. get anything out of it) thanks to the its target audience while being able
apps they play on tablets and to teach its users effectively. That’s
smartphones. a tough challenge, but equipped
Sonic Pi But creating a meaningful with some real-life young people
URL http://sonic-pi.net comparison of education software (see How We Tested boxout, below)
Licence MIT is difficult. This is because there and a fresh Ubuntu installation, we
Latest Release 2.7.0
isn’t enough software to focus on a think we can give it a jolly good go.
Used by serious musicians, it’s also a
great way to learn programming.
We’d argue that there’s educational
GCompris value in almost any kind of software
URL http://gcompris.net
Licence GPL – even Super TuxKart
Latest Release 15.02
This is the one to beat. Over a hundred
mini-games and a framework for adults How we tested
to supervise children’s development It wouldn’t make much sense testing all couple of weeks (and months, for some
Childsplay this software ourselves. Out collective
brain cells are too old, too cynical and
of our choices), we sat with a small
group of children aged 4-10 and let them
URL www.childsplay.mobi too few. And as all this wonderful is play with the software we showed them,
Licence GPL software designed for the delicate noting their comments and allowing their
Latest Release 2.6.3 sensibilities of The Next Generation, feedback to shape our own conclusions.
Another suite of activities, which can we’d be entering premature senility if It wasn’t a large enough group to allow
we didn’t put it to the test in front of its scientific rigour, but it gave us a great
log statistics to an external database.
target audience. Over the course of a idea of what worked, and what didn’t.
50 www.linuxvoice.com
EDUCATIONAL SOFTWARE GROUP TEST
Distros for children
Is it worth installing a distribution specifically designed for young people?
T
here are several Linux distributions Edubuntu is better for older children, as it complete control over which software is
designed specifically for children. avoids them having to install a load of installed and how the computer is used. All
Qimo and Edubuntu are both worth software before they can get started. the most popular education applications
a look, as they each take different approach However, what we’ve found is that it’s and tools will be a few clicks away, and you
for making a computer more friendly and usually much more practical to install a can also make sure the machine is locked
accessible. Qimo is more suitable for standard Linux distribution, such as Mint, down adequately – running through a
younger children, as its artwork and styling Ubuntu or Fedora, and let your children Dansguardian internet proxy filter, for
is designed to get kids clicking, but guide you in how they want the example, and with YouTube restrictions
development has been slow recently. configuration to be adapted. You then have in place.
Scratch
Itching to get started?
G
etting started with programming is
difficult. But it’s also vital. Code is the
agent that binds computing to
computing science. Even for students who
have no intention of taking their studies
further, unlocking this particular black box is
the key to understanding how technology
governs our lives, from ATMs to encryption.
It’s the equivalent of studying DNA in
Biology, or the Periodic Table of Elements in
Chemistry, or gravity in Physics, and we
think programming should be a prerequisite
for any computing course, regardless of age.
This is something that’s only going to
become more important. The huge
challenge, of course, is how best to teach
programming, and Scratch is one of best Scratch projects can also run online with Flash, and the project’s repository prepares young
attempts we’ve seen. developers for ideas like forking and version control.
Scratch is a visual programming
environment. A simple script-like syntax is The visual style is also much easier to programmers. For us, the biggest challenge
augmented by colourful blocks that encase understand than abstract source code. has been less about learning and more
the various variables and keywords. The But the best thing about Scratch is the about finding something to inspire learning,
colour of each block is used to denote what vast amount of support material you and Scratch doesn’t help with this. Just like
each command is doing – purple is for have access to. This is partly thanks to any other language, it’s there to get a job
visual changes such as output, for example, the sterling work of the Raspberry Pi done, but it’s up to you to find a job that will
whereas dark orange is for events. Most Foundation, which has pushed Scratch inspire your charges.
things you write in Scratch are going to be and even toyed with the idea of its devices And this is the biggest problem with both
driven by events, whether that’s keyboard booting into the environment in the same learning to program and Scratch; you need
input or waiting for a value to be achieved. way Acorn machines booted to BBC BASIC, to find the purpose that drives the learning.
The jigsaw-like design of each block makes but also through necessity. The new UK That said, as long as you’re comfortable
it easy to see the organisation, level and national curriculum for 5–7-year-olds devising a program of study (loops, lists,
flow of the code, as well as which blocks are includes abstraction logic, algorithms and control flow etc are all easy to implement)
wrapped within others. data representation, as well as writing Scratch has tons of learning potential.
All this means that you learn exactly ‘computer programs’ to solve problems, and
the same concepts as you would with a Scratch is going to be perfect for this. VERDICT
programming language like Python, without However, in practice, we have found that Wonderful as part of a
project, but older children
having to memorise lots of keywords or type while Scratch is a wonderful environment for may be better off diving
too much, and apply them just as you would potential programmers, it doesn’t do enough straight into Python.
with any other language. for self-learners, or perhaps for younger
www.linuxvoice.com 51
GROUP TEST EDUCATIONAL SOFTWARE
Tux Paint
It’s just like painting, only without the mess on the floor and the tidying up.
A
nyone with kids and a shape or a brush and then a colour.
smartphone knows that the Thanks to the icons, a three-year-old
first thing they want to do is can pick up the process quickly, and
paint onto your screen. And there’s a while the sounds are initially appealing,
good argument to suggest that the they quickly get annoying (for adults,
Amiga’s initial success was partly anyway). The application also defaults
because Commodore bundled the to full-screen, which works well for
brilliant drawing package, Deluxe Paint. most, but we couldn’t get window mode
The addictive pleasure of multi- working without a configuration file.
coloured point-and-click sketching was The stamps are good fun, but we’d
a great motivator for mastering the love to see the simple foreground/
mouse. And it’s the same for Tux Paint. background layers that Deluxe Paint
Despite its toddler-friendly icons and had, as well as a similar animation
sound effects, we’ve found that its engine – effectively a fast slideshow. Drawing is fun and helps children learn to use the mouse.
compulsive drawing engine has never Deluxe Paint could also animate the
failed to entertain children under 10. brushes (sprites), which made creating make TuxPaint more than a simple
They just love mucking about, much as animations as fun and as easy as distraction, and help potential artists
they do with an Etch A Sketch. drawing. Features like these would help and animators learn more skills than
simple mouse control.
Paint Thanks to Tux Paint’s
Tux Paint seems to take inspiration from VERDICT
Deluxe Paint. The three borders of the icons, a three-year-old Good for encouraging use
of a mouse, but limited in
canvas area where you paint are used
for choosing a drawing tool, selecting a
can pick up the process its educational potential.
TuxMath
Maths makes sense.
T
uxMath is similar to Tux Paint, Typing the answer and pressing return
thanks to the project being will shoot a laser beam to vaporise any
largely from the same threats whose answer you’ve hit.
development team and sharing the
same cartoony style and audio. But it Levels of challenge
takes learning further, and would make There are many difficulty levels to
a good next step for young children choose from, from simple addition
needing something more practical. We through to the division of positives and
found that when they’re old enough to negatives (-10/2, for example), and
have had some preliminary exposure to the objects’ descent gets quicker and
numbers and simple addition, perhaps quicker, turning the game into quite a
3–5 years old, TuxMath was both challenge. Even older kids will get a kick
entertaining and helped to build their out of beating their high scores as the
confidence with numbers. game itself becomes quickly addictive.
Through a series of mathematically There’s a shared screen multiplayer TuxMath includes a version of Asteroids where large
themed games, TuxMath becomes mode, which could do with a handicap asteroids break into their factors.
far more than a distraction, often setting for one player, and a client/
combining quick thinking with server mode that can be used to but sound can be disabled from the
keyboard and mouse skills. The games dispatch the same lessons across a command line.
themselves are variations on the arcade network of machines, making it ideal for
classic Missile Command, where you a workshop or club. We find the music VERDICT
Gameplay and learning
stop the imminent collision of a meteor slightly incongruent with mathematical potential make up for the
by typing in the answer to a sum study, especially as it loops back to lack of in-game options.
displayed across each descending foe. the beginning after a few minutes,
52 www.linuxvoice.com
EDUCATIONAL SOFTWARE GROUP TEST
Sonic Pi
Learn to code and write music all at the same time.
Online resources
The only reason to keep
Flash installed.
A
lthough we haven’t included them,
there are dozens of online resources
and educational games to play online
with children. Scratch 2.0, for example, can be
used, shared, modified and edited from its web
portal without installing a single package. This
could be a great option if you’re teaching kids
using a variety of operating systems, or if they
want to access their projects from home. The
BBC also hosts some wonderful content, from
games through to study guides. For pre-school
children, its CBeebies activities page is
essential (www.bbc.co.uk/cbeebies), as too is
the free Android and iOS app.
For older children, the BBC has a
comprehensive set of resources that covers
the national curriculum from key stage
1 through to GCSEs. The Raspberry Pi
Foundation has also done a brilliant job with
The flexibility in Sonic Pi comes from the powerful SuperCollider audio programming system. its education resources (www.raspberrypi.
org/resources), turning the creation of
W
hen it comes to technology, the way into Sonic Pi, as they’re used to play hardware into its own entertainment, from
thing that often comes after different sections of code simultaneously. teachers to makers. Even adult resources
messing about with virtual like Khan Academy (www.khanacademy.
paint is music and sound. Whether that’s Sonic the leap-frog org) or many of the TED talks (www.ted.com/
mashing the keys on a piano or banging a This level of coding is obviously going to talks) can provide great insight into a subject,
drum, there’s something special about its be most beneficial to older children, but as too can many of the free audio and ePub
spontaneity. Instant feedback and we found the audio feedback much more books you can download through projects like
familiarity make a great teacher, and the natural than the graphical style of Scratch, Project Gutenberg (www.gutenberg.org). The
formulaic nature of audio generation and and because you can change your code only problem is that many of these portals
music composition makes audio perfectly while it’s playing back, you can see how and games require Flash, which we’re loath to
suited to programming. Sonic Pi different parameters and keywords recommend, and your children will obviously
capitalises on this, presenting the user affect the feedback. For our kids, it added need supervision and support when left to the
with a real-time programming self-propelled exploration, despite the wilds of the internet.
environment that’s capable of some typing and text involved in writing code.
serious results. Its creator, Dr Sam Aaron Even younger children had fun changing
at the University of Cambridge, live-codes octaves and synth parameters, gaining an
Sonic Pi with his band, and there are many insight into the types of variables and their
complete algorithmic compositions to ranges, as well as what computers do
download as examples. behind their screens.
Sonic Pi is also easy to use. At its The only negative with Sonic Pi is
simplest, you can play a pitch by typing that because it’s partly funded by the
play 70 and clicking on Run in the toolbar. Raspberry Pi Foundation and is pre-
The syntax is always simple, and the configured to run on a Pi, it’s a little clunky
exceptional inline help and tutorials on a desktop. It also needs to run through
system make it easy to progress without the Jack audio layer, which the average
any external resources. The loop-based Linux geek will have problems getting to
nature of modern music lends itself well run correctly, let alone a 10-year-old.
to common structures like blocks, arrays,
functions and iteration, and the many VERDICT
This might be the best
sound generators and variables provide way to get musical kids We hope the BBC follows-up its iPlayer migration
huge scope for experimentation. Even into programming. from Flash by porting its ace edutainment too.
advanced topics like threads find their
www.linuxvoice.com 53
GROUP TEST EDUCATIONAL SOFTWARE
GCompris vs Childsplay
Two activities suites fight it out to control the mind.
W
hen it comes to educational
software for young children,
GCompris is often
considered the go-to package to install.
It’s tailored specifically for young
children – perhaps as their first
experience with computers, and it
bundles so many mini-games that you
could dedicate an old computer to
running GCompris and nothing else.
The games themselves are split into
various categories, and are navigated
using a simple folder system. Many
of the games are simple, but the ideas
behind them are clever. One of the
early activities has the player pressing
both Shift keys at the same time, for
example, while more activities practice
reading or playing strategy games like
chess or Mancala. Our test audience
didn’t always respond well to the
colourful and brash aesthetics, and the
simple animations don’t compare to The graphics may be simple, but with almost 150 activities, there’s hours of fun to be had in GCompris.
the smooth transitions found on tablets
and smartphones. We’d also like to see The best thing about GCompris, they spend on each game is logged
a button on each screen that repeats however, is the management interface. and listed, along with which levels
the instructions for each game, as the This is designed for an adult and it they played. You can also enable and
attention spans of small children often enables you to see exactly what your disable certain titles, and create group
mean they miss the description. children have been doing. The time and class categories to better track the
children playing the games.
Here’s Chucky
Childplay is another application that
offers a suite of activities for children.
It can’t compete with GCompris when
it comes to the number on offer, but its
design is slightly slicker and you don’t
have to make your first mission turning
off the inane background music. There
are fewer than 20 games, including
flash cards and putting animal sounds
to the images, and even a game of
Pong. But unlike GCompris, there’s no
structure to the games, no parental
control or management, and the
activities are often simplistic. However,
our kids had fun playing Simon Says
and the Pacman-style spelling games,
and it’s definitely worth a try.
VERDICT
GCOMPRIS A huge CHILDSPLAY Too few
suite of activities that activities, but it’s
succeeded in keeping worth installing as
younger wards quiet. another option.
Childsplay has less educational value, although getting children into Pac-Man via spelling is genius.
54 www.linuxvoice.com
EDUCATIONAL SOFTWARE GROUP TEST
OUR VERDICT
Educational software
N
one of us want our least until you’ve got the skills to
children to spend too work on your own projects. This
much time in front of a is why Sonic Pi wins. We loved
screen. But computing has become the user interface and the way its
such an important part of the world baby-step tutorials were written
around us that getting them used and embedded within the main
to the technology is just another application. The application itself
step in their educational journey. has developed so rapidly over the We loved Sonic Pi. If you’re looking for an entry point to
Our small team wanted to last 12 months, that it almost feels programming and creating sounds with instant feedback, it’s the
learn, but the messages from the like a modern IDE. But best of all, perfect experimental platform.
software were mixed. As wonderful sound production gives immediate
as GCompris is, for example, it’s feedback, inviting you to play with it
hard to pitch it against the glossy and mess with the values. 1st Sonic Pi
apps that many children spend time
with, and the disparate collection Music == code http://sonic-pi.net
of ‘brain training’ activities are Even if you don’t care for music or Fun and serious at the same time, this could prove the inspiration
difficult to justify. For us, this means sounds (and we’ve yet to meet a for the career of your own fame-obsessed pop stars.
asking what you want educational child like this), audio has evolved to
software to achieve, and the best give us this natural feedback, and 2nd GCompris
answer we have is that we want coding both timbre and rhythm are
this software to teach about brilliant ways of learning about
http://gcompris.net
computers. And the language of writing functions and manipulating
Worthy of second simply because of the number and breadth of
computers is programming. data structures. But best of all, it activities here. Also better suited for younger children.
That means, for us, the winner gives you a reason to try and to
of this test needs to be something take your skills to the next step, all
that teaches programming. It’s within a single application. And 3rd Scratch
widely accepted that Scratch does when you want to take your code
a brilliant job at this, especially in into the hipster stratosphere, you’re https://scratch.mit.edu
a classroom. But we can’t help ready to take on Pure Data, which is We still love Scratch and it’s capable of more than Sonic Pi, but
thinking that Scratch doesn’t make a kind of visual programming you need to put more effort into the preparation to inspire your
programming any easier, just environment that inadvertently children to use it.
because it’s visual. What we found teaches you all the subtlety of
with our children is that Scratch functional programming – 4th TuxMath
doesn’t inspire learning either, at hopefully before you’ve left school.
http://tux4kids.alioth.debian.org
Simple and effective at getting children to learn about numbers
Sonic Pi has developed so quickly while they think they’re being allowed to play video games.
over the last 12 months that it
5th TuxPaint
almost feels like a modern IDE
www.tuxpaint.org
This is still a brilliant application; creative, relaxing and good for
Other applications teaching mouse control; it’s just lacking any educational rigour.
So much of the software we use day- their knowledge, as too is Kalzium for
to-day is perfect for children and their exploring the periodic table of elements.
development. Online maps, for example, Leaving earth behind, Stellarium is a 6th Childsplay
are better than any old geography book, brilliant astronomy application that can
and applications like Blue Marble bring all do the same for space, and while Celestia www.childsplay.mobi
the wonder of the world to your desktop, development has stalled, we’ve yet to
The few activities in Childsplay are fun, and worth an installation,
allowing your kids to explore a virtual find another application that will let you
but they won’t keep anyone’s attention going for as long as the
globe with just a mouse. KGeography fly from one star system to another in
other tools here.
is a brilliant application for testing beautiful OpenGL-rendered graphics.
www.linuxvoice.com 55
SUBSCRIBE
Subscribe
shop.linuxvoice.com
Introducing Linux Voice,
the magazine that:
Gives 50% of its profits
back to Free Software
Licenses its content
CC-BY-SA within 9 months
12-month subs prices
UK – £55
Europe – £85
US/Canada – £95
ROW – £99
7-month subs prices DIGITAL
UK – £38 SUBSCRIPTION
Europe – £53
US/Canada – £57
ONLY £38
ROW – £60
Get 100 pages Access our Save money on
of tutorials, rapidly growing the shop price
features, interviews back-issues archive and get each issue
and reviews – all DRM-free and delivered to
every month ready to download your door
Payment is in Pounds Sterling. 12-month subscribers will receive 12 issues of Linux Voice a year. 7-month
subscribers will receive 7 issue of Linux Voice. If you are dissatisfied in any way you can write to us to cancel your
subscription at subscriptions@linuxvoice.com and we will refund you for all unmailed issues.
56 www.linuxvoice.com
NEXT MONTH
NEXT MONTH IN
ON SALE EVEN MORE AWESOME!
THURSDAY
Economic modelling
19 NOVEMBER Predict the next
catastrophic
financial collapse
with this open
source, GPLed
software. Interest
rates have never
been so much fun!
Microsoft!
We chew the fat with
Gianugo Rabellino,
Microsoft’s senior
director of open
source programs, to
find out what the
Windows giant really
thinks about Linux.
Bash
RISE OF THE ROBOTS Hack your Linux
shell to get more
power, more
Under the sea, in space, clearing minefields, features, and spend
looking for earthquake survivors, tracking more time walking
down survivors of the nuclear holocaust – the dog while your
computer does the
Linux-powered robots are everywhere! hard work for you.
LINUX VOICE IS BROUGHT TO YOU BY
Editor Graham Morrison Editorial consultant Nick Veitch through the use of advice in this magazine. Copyright Linux is a trademark of Linus
graham@linuxvoice.com nick@linuxvoice.com Experiment with Linux at your own risk! Torvalds, and is used with permission.
Deputy editor Andrew Gregory Distributed by Marketforce (UK) Ltd, 2nd Anything in this magazine may not be
andrew@linuxvoice.com All code printed in this magazine is licensed Floor, 5 Churchill Place, Canary Wharf, reproduced without permission of the editor,
Technical editor Ben Everard under the GNU GPLv3 London, E14 5HU until July 2016 when all content (including
ben@linuxvoice.com Tel: +44 (0) 20 3148 3300 our images) is re-licensed CC-BY-SA.
Editor at large Mike Saunders Printed in the UK by ©Linux Voice Ltd 2015
mike@linuxvoice.com Acorn Web Offset Ltd Circulation Marketing by Intermedia Brand ISSN 2054-3778
Creative director Stacey Black Marketing Ltd, registered office North Quay
stacey@linuxvoice.com Disclaimer We accept no liability for any House, Sutton Harbour, Plymouth PL4 0RA Subscribe: shop.linuxvoice.com
loss of data or damage to your hardware Tel: 01737 852166 subscriptions@linuxvoice.com
www.linuxvoice.com
FOSSPICKS
FOSSpicks Sparkling gems and new
releases from the world of
Free and Open Source Software
Between sessions foraging for mushrooms and apples, Ben Everard
snuffles through the forest looking for the best Free Software.
Web reader
Wallabag
T
he web is made for transform them into a different
browsing – but sometimes format (such as ePub) that may be
you come across more suitable for your device. It’s all
something fascinating that you open source, but you’ll need to run
don’t have time to read, and want to part of it on a server and part on the
come back to it later. This is where clients you connect with.
Wallabag comes in. It’s part Wallabag makes web pages
bookmark manager for available on all your devices when
synchronising bookmarks between you want them, not just when you
devices, part re-formatter and part have an internet connection, and
download manager. does so without using any
Wallabag links into your web proprietary software, so you don’t
browser, so that when you get to a have to surrender your data to the
page you want to save for later, you whims of advertisers.
activate it. This saves the pages,
and alters them to make them PROJECT WEBSITE You can test out Wallabag without installing it at
www.wallabag.org
more readable. It can even http://v2.wallabag.org/ with the login wallabag/wallabag/.
Secure chat
Ricochet
R
icochet (formerly known as Anonymity is created by routing
Torsion) is an anonymous, connections through the Tor
secure messaging platform. network, and your Ricochet
On installing the software, you username is in reality just the
automatically get a username that address of a hidden service running
looks like a random set of on your machine. The anonymity is
characters with the word ‘ricochet’ as good as Tor’s hidden services.
at the start. Using this, you can This is good, but not perfect. It is
connect with other users to send likely that users will be identifiable Protect yourself from unlikely to be the worst issue with
and receive instant messages. by organisations with large unwanted the security. It takes time and a lot
There are versions for Linux, resources, although this is still likely eavesdropping with of eyeballs to create software that
Ricochet’s Tor-based
Windows and Mac OS X, so you to be better than most options. completely protects the users. For
web chat.
can talk to people who haven’t yet Given the young status of now, Ricochet is probably secure
discovered the joys of Linux as well Ricochet, hidden services are enough for those who don’t want
as those who have. At the moment, the state spying on them routinely,
Ricochet is limited to just text
messages, but the protocol is
Ricochet is limited to just but who don’t have anything that
could be considered high-value.
extensible, and future releases may text messages, but the
include the ability to send files and PROJECT WEBSITE
potentially video chat. protocol is extensible https://ricochet.im
58 www.linuxvoice.com
FOSSPICKS
Map editor
iD
T
he Wikimedia Foundation The first time you sign in, you’ll be are already quite complete, there
has just launched a new invited to follow a walkthrough that are fewer major edits left to do.
map service that takes the will give you a reasonable idea of Gathering all these little edits, such
data from OpenStreetMap and how to use the software to edit as points of interest and details of
renders it in a format suitable for maps. It’s straightforward for basic buildings, is an important task in
Wikipedia and other resources. This editing and is ideal if you want to building up the maps.
is a fantastic idea, so we decided to add a point of interest, or add Open map data is important
take a look at OpenStreetMap. information to items that are because it enables anyone to create
We came across iD, which is the already on the map. geo-aware applications and
web-based editor for services. If we allow global
OpenStreetMap. It’s open source, Open data corporations to gather the most
but you don’t need to do anything to Maps based on this data are accurate maps of the world, we
install it, just point your browser to produced by rendering the raw allow them to dictate the terms
www.openstreetmap.org, and you information into images. Different with which we see our environment.
should find the Edit tab in the mapping services use different The beauty of iD is that there’s
top-left corner of your screen. rendering processes, so your nothing to install, and it doesn’t take
Clicking on this will open the iD changes may take some time to much technical skill to use. If you’re
editor and enable you to modify the appear in a particular set of maps. looking for a way to give back to the
map directly from your browser. If you’re doing significant edits to free software community, but you’re
You will need to create a login the map (for example, using a GPS not a programmer, open mapping
first, but this doesn’t try to harvest trace to alter or add a road), then it’s with iD is a fantastic option.
too much data (just a name and probably better to use a more
email address), or you can log in powerful piece of software, such as PROJECT WEBSITE
www.openstreetmap.org
using a social media account. JOSM. However, now that the maps
6
1 3 4
5
2
1 Edit tab 2 The features sidebox enables you to change the attributes of the currently selected item 3 Select the type of item you wish to add to the map 4 Undo and
redo changes 5 Save and upload your changes to OpenStreetMap 6 GPS traces are information added by people who have physically travelled the routes.
www.linuxvoice.com 59
FOSSPICKS
Emulator
Angel
A
NGEL Is Naturally Good at https://s-macke.github.io/jor1k/
Executing Linux is the demos/main.html. This emulator is
recursive name for this a little more useful, since it has
JavaScript implementation of the more utilities and even network
RISC-V instruction set. In other access, so you should be able to
words, it’s an emulator that can run download and compile software
in your web browser. Just head to should you need it.
http://riscv.org/angel and you can
boot up a Linux session. It should Linux in a browser
run in any modern browser, but we It’s hard to say whether these
found that it ran about four times emulators have any real value
faster in Chromium than Firefox. The beyond novelties. They could be
Get a full Linux command line in your browser with a JavaScript
session is fairly minimal, running used to port terminal applications
RISC emulator.
little more than the Linux kernel to the web, or they could be used to
and the BusyBox shell. There’s not help people get started with Linux cool about running a full operating
even network support, so it’s quite in a safe environment. system inside your browser. It’s
hard to get any more advanced In reality though, the main reason over the top, it’s geeky beyond
software running. for these is that there’s something words, it makes no sense from a
Angel isn’t the first emulator to practical point of view, and we love
run in JavaScipt. JSOR1K
(JavaScript OpenRisc 1K) is an
Angel is over the top, geeky it because of this.
implementation of the OpenRisc 1K beyond words, makes no
instruction set that you can boot by PROJECT WEBSITE
pointing your browser to sense, and we love it http://riscv.org/angel
Sandbox control
Firejail & Firetools
W
hen you launch an The solution to this is to run
application in Linux, the software in a restricted
software usually gets environment where it can’t access
the same privileges on the system the main system. Firejail is a piece
that you have. For example, when of software that is used to launch
you run Firefox, it gets permission to other pieces of software in limited
access the files in your home environments. You can open the
directory. Usually, this is a good web browser through Firejail with
thing. It means that you can access the following (you must have
your files and upload them to closed all browser sessions
websites, should you wish. beforehand):
However, it also means that if an firejail firefox
attacker is able to subvert Firefox, Firefox is now running in a Firejail can isolate any software, but the web browser gains the
they may also get access to your restricted environment and can’t most since it’s the most frequently attacked.
home directory. access the ~/.ssh directory (try to
This is exactly what happened in open this directory through the are loads of ways you can lock
August 2015. A bug in the pdf.js open file dialog and you’ll get a down the software you use, and
viewer allowed attackers to scan permissions error). There’s a there’s even a GUI to make it easy in
victims’ home folders and steal the specific Firefox profile that defaults the form of Firetools.
contents of the .ssh folder, which to this level of protection because,
potentially gave them access to any really, your web browser has no PROJECT WEBSITE
remote machines that the victim business looking in that directory. https://l3net.wordpress.com/projects/
firejail/
had access to. This is just a simple example. There
60 www.linuxvoice.com
FOSSPICKS
Source code hosting
GitLab
I
f you’re familiar with open to keep your energy bills to a
source software, you’re almost minimum, there’s also a version for
certainly familiar with the the Raspberry Pi, which is perfect
source code hosting platform for running at home, though we
GitHub. Even if you don’t code haven’t tried this one so we can’t
yourself, you’ve probably found comment on what the performance
yourself getting software from it at is like.
some time or another. GitLab offers The web interface is one of the
a very similar feature set to GitHub, nicest we’ve seen for project
but the entire setup is open source. hosting. It’s all very straightforward,
The web interface to
This means that you can host it and most users should be able to source platform, GitLab offers a
Gitlab is as good as any
yourself rather than relying on a use it without difficulty. Since it’s all project hosting we’ve series of options from free hosting
company to do it for you. based on Git, the workflow will be used, either open source (see https://about.gitlab.com/
There’s the web interface to Git familiar to anyone who’s used any or commercial. gitlab-com for details). There’s also
repositories, a wiki, issue tracker Git-based system in the past. an enterprise level of GitLab which
and more all in a single package If you’d rather someone else did has more features than the fully
that’s easy to install (take a look at the hosting for you, but you still open sourced community version
this month’s step-by-step tutorial on want to host your code on an open (including Kerberos integration and
page 66 for details of how to get up better Jira support and improved
and running).
There are versions for any distro
Gitlab’s web interface is one user controls), however there’s little
there for open source projects.
based on Debian, Ubuntu or Red of the nicest we’ve seen for PROJECT WEBSITE
Hat (unfortunately SUSE users
seem to be neglected). If you want project hosting https://gitlab.com
Web terminal
Reddit shell
F
or most of us, the CLI is JavaScript so should be easy to
nothing more than a useful run).
tool to make powerful This isn’t the first web-based
commands easy to use; for others terminal-style interface for a
it’s the centrepiece of a cult. Reddit website. On April Fool’s Day 2010,
Shell is a tool for these CLI fanatics. the geek web comic XKCD
It offers a terminal-style interface launched UNIXKCD (http://uni.xkcd.
for the popular Reddit news com) with a similar style interface.
aggregation site that enables you to We must admit, we can’t think of
use commands such as ls and cd a good reason to use either of
to view and move through the these. In both cases, the ordinary
various sections of the site (known website is easier and quicker to use,
as subreddits). and if you really had an aversion to
Reddit Shell runs inside a web using the mouse, it would probably
Reddit Shell leaves us
browser rather than in a normal still be easier to use a browser a stretch. This is all beside the point
one step closer to
terminal, but the style of interaction extension that’s designed for this getting rid of this pesky though – the reason that these
is the same. The hosted version rather than trying to find a terminal- mouse. exist isn’t because of any need, but
runs at https://redditshell.com, but style interface for every site you because they’re good, clean fun for
the software is open source, so you use. Perhaps, at a stretch, you could us terminal addicts.
can grab the code from https:// argue that these may make it a little
github.com/jasonbio/reddit-shell easier to use these websites at
and host it yourself if you’re that work and hide the fact that you’re PROJECT WEBSITE
https://redditshell.com
way inclined (it’s written in not really working, but that’s a bit of
www.linuxvoice.com 61
FOSSPICKS
Programming language
FreeBASIC
B
eginner’s All-purpose FreeBASIC is just a compiler, so
Symbolic Instruction Code you’re free to use it with any text
(BASIC) came out in 1964 as editor or IDE.
a product of Dartmouth University. BASIC may have grown up as the
It was designed to allow students language for beginners, but this
who didn’t specialise in a doesn’t mean that you can’t write
computing-related field to write serious software in it. If you don’t
software. For the next 30 years, it believe us, just take a look at the
was the dominant programming FreeBASIC Games Directory (http://
language for people new to games.freebasic.net). Some are
Geany can provide a
computing. Whole generations of simple, some are half-finished and to 3 … Next loops. Curly braces and
good development
programmers first learned their some are Windows-only (thanks to environment for more compressed grammar may
craft on dialects of this language: the QuickBASIC legacy) but many programming in make writing in other languages
AmigaBASIC, Commodore BASIC, are complex games that are well FreeBASIC. quicker, but it takes away a little of
BBCBASIC and QuickBASIC were written and run smoothly. the fundamental joy of
some of the most popular. programming that remains in
Almost all of us here at Linux Cosmic poetry BASIC. Even though we rarely
Voice first programmed in one of There’s a strange beauty in BASIC’s program in this language any more,
these, and we have fond memories language that, when written well, is it’s nice to know that it’s out there,
of the time. FreeBASIC continues somewhere between engineering still active and waiting for us should
this legacy. It’s a BASIC compiler and poetry. Other languages can we ever wish to return to the fold.
that loosely follows the QuickBASIC have elegant code, but few have the
version of the language. Unlike majesty of well written BASIC with PROJECT WEBSITE
www.freebasic.net
many versions of the language, it’s REMs (comments) and For X=1
Distro builder
Linux Respin
O
nce upon a time there was least, when we tried, the download
a tool called Remastersys link was broken and there was no
for building live and link to the original repository. We
installable ISO files from Debian can save you from the ordeal and
and Ubuntu systems. Unfortunately, point you directly to the project’s
distro development, like the tides, repository on GitHub, where you’ll
stops for no man, and new find Deb files for both Ubuntu and
developments mean that the Debian: https://github.com/
original tool no longer works. This chamuco/respin.
left Debian and Ubuntu without an Linux Respin comes in both
easy to use tool for customising command line and graphical
their distros. versions, and it enables you to clone
Linux Respin’s GTK
Fortunately, the story does not your system either to a full backup Linux Respin isn’t the only
interface makes
end there. The Copper Linux User or only including the programs and building a new distro as continuation of Remastersys. The
Group in Arizona has taken up the not the data. Both versions are easy as clicking a Pinguy Linux team have created
mantle and continued working on incredibility easy to use, and you button. Pinguy Builder in a similar manner.
the project under the new name of can have your live system ready in At the moment, both tools seem
Linux Respin. While this name is just a few clicks. There are loads of very similar, with the only real
fairly descriptive, it does make the uses for these customised live differentiation being that Pinguy
tool almost impossible to find via a distros: creating a custom install CD Builder doesn’t support Debian.
web search. Even when you have specifically for an organisation,
found the project’s web page, the create a backup of your current PROJECT WEBSITE
www.linuxrespin.org
task doesn’t get much easier. At setup, or even build your own distro.
62 www.linuxvoice.com
FOSSPICKS
FOSSPICKS Brain Relaxers
https://launchpad.net/pybik/
Car racer
Stunt Rally
T
he problem with the real
world is that it hurts too
much if you skid off a
road and hit a tree at 50 miles an
hour. It might be fun to try, but the
months of rehab and inflated car
insurance means that it’s just not
worth it. Fortunately, thanks to
computers, we can all experience
the thrill of rushing round a
corner in a safe, virtual world.
Stunt Rally gives us just such a
world to race around in a wholly
inappropriate manner. It’s all
about driving quickly round tracks
All the fun of reckless
with insufficient traction. In the Stunt Rally features impressive race around whatever you can
driving without the pain,
latest version (2.6), there are 172 graphics, even on our lowly test cost or legal trouble. imagine, and even contribute
tracks in a wide variety of machine. You can adjust the level back to help the game grow and
settings from jungles to deserts of detail to let the game to run well let other people enjoy the fruits of
to alien planets. You can race on whatever hardware you have. your creation.
around these using cars, If you fancy turning your hand to
PROJECT WEBSITE
motorbike and even Star game design, Stunt Rally comes
http://stuntrally.tuxfamily.org
Wars-esque land speeders. with a level designer, so you can
Real-time strategy
OpenDungeons
O
penDungeons is a real https://github.com/
time strategy game in OpenDungeons/OpenDungeons/
which you command a wiki. It can take a little while to get
legion of creatures that crawl, used to some of the nuances, so it’s
creep and fly around in darkened worth having a quick scan through
caverns underground. once you’ve got a feel for the game.
As the game progresses, you For example, you have to explicitly
can build libraries where workers set workers to task in the libraries
can research new items that you and workshops if you want to reap
Take on other dark lords
can use to expand and defend the fruits of these rooms, and this less frantic than some other
and build the ultimate
your dungeon. With food and won’t happen automatically. dungeon. games in this genre, and there
experience, your workers will The graphics are better than aren’t many maps available, so it
grow and develop into honed most games of this ilk, and help is a little short on content if you
fighting machines. Using your make OpenDungeon a good, fun plan on mostly playing against
dungeon as a base, you can then game to play once you’ve mastered the computer, but when playing
launch attacks on other dungeon the controls. The gameplay is a little against real people, there should
masters, with the aim of be enough variation to keep you
achieving dungeon supremacy.
This can be against real players in
OpenDungeon is a good, fun entertained (and avoiding work)
for quite some time.
a networked game, or AI players. game to play once you’ve
The documentation is a little PROJECT WEBSITE
hidden, but can be found at mastered the controls https://opendungeons.github.io
www.linuxvoice.com 63
INTRO TUTORIALS
TUTORIALS
Warning: excessive Linux knowledge may lead to fun and more efficient computing.
In this issue . . .
66 68
Ben Everard
Is embarking on a crusade to get open source
Host your own software Brush up your database
developers to document their work. projects with GitLab skills with MariaDB
Ditch commercial hosting providers and their Databases can be tricky, but fear not:
P
proprietary software, and follow Ben Everard’s Mike Saunders is here to show you how to set up
rogramming has changed since guide to open source hosting using GitLab. and use MariaDB as a web app back-end.
I was a lad. Back in those days,
if you wanted to do something,
you had to write the code to do it.
Nowadays, it seems, all you ever need
to do is link a couple of libraries
together and you can do anything.
The same thing, it seems, is
happening with electronics. Recently I
wanted to turn a broken 1940s radio
into a Bluetooth speaker. All it needed Raspberry Pi info centre 72 Vulnerability scanning 76 Head tracking 80
was a Bluetooth audio module, Les Pounder uses a Raspberry Pi Set a robotic penetration tester Improve your gaming experience
amplifier and power supply. Wire them and a Displayotron HAT to keep to work. Ben Everard automates by turning yourself into a cyborg,
all together and it works. At least, that’s himself up to date. finding vulnerabilities. like Graham Morrison.
the theory.
The problem with libraries and
electronics modules is documentation.
There are few things more frustrating
Coding
than trying to work out how a poorly
documented function works in a library,
or what on earth the pin labelled ‘EN’
does on an electronics module. It’s
starting to feel like everything I need is
already created, if only I could work out
how to use it.
Geeks of the world, I urge you, step
away from the IDE, put down your GNU R and Git 84 Get started with Rust 88 Olde Code: Ada 90
soldering iron, and take a little time to With GNU R , Git and a bit of Mike Saunders investigates the Juliet Kemp’s guide to computer
properly document your creations. statistical magic, Robin Gower new language from Mozilla that history reaches the 70s with the
ben@linuxvoice.com assesses popular FOSS projects. aims to make the web a bit safer. Ada programming language.
www.linuxvoice.com 65
TUTORIAL GITLAB
HOST YOUR OWN SOURCE
CODE WITH GITLAB
Run your open source project on your own open source server.
BEN EVERARD
S
ince its creation as a source code management a git server and giving everyone commandl-ine
tool built specifically for the Linux kernel, Git has access. This works, but having a web interface to
become the most popular option for hosting manage everything makes it all a bit nicer. GitLab
WHY DO THIS? open source projects. This is partly fuelled by several provides this web interface along with a wiki for
• Keep your source code on web-based hosting providers that offer free hosting to documentation and a bug tracker to keep an eye on
servers you control open source projects. However, there are times when any problems. It's a fully featured open source code
• Save on bandwidth by a public hosting system isn't what you need. Hosting hosting environment that you can set up and run on
hosting locally your own code repository can be as simple as running your own hardware. Let's get it up and running!
• Avoid using closed source
software in your workflow
STEP BY STEP: INSTALL GITLAB
1
Get the dependencies 2
Get GitLab
Before we get into GitLab properly, you need to get all With the dependencies installed, we can now get the
the dependencies. The key ones are the Curl library, main software. The GitLab team have put together a
the OpenSSH server, an SMTP (email) server such as script that will detect which distro you're running and
Postfix and the certificate authority certificates (if add the appropriate repositories. You can download
using a Debian-based system). In a Debian-based and run this in one go with:
system, you can grab these with: curl https://packages.gitlab.com/install/repositories/
sudo apt-get install curl openssh-server ca-certificates gitlab/gitlab-ce/script.deb.sh | sudo bash
postfix If you're using a Red Hat-based distro, you'll need to
Alternatively, you can install them through a package change script.deb.sh to script.rpm.sh.
manager GUI such as Synaptic. If you'd rather not pipe a script straight from the
If using Red Hat (or a clone), you'll also have to internet into sudo bash, you can perform the process
enable the SSH and Postfix services using systemctrl manually by following the steps here
(for Systemd versions) or service (for older systems). https://packages.gitlab.com/gitlab/gitlab-ce/install.
Additionally, Red Hat users will need to open the HTTP Once the script has finished running, you can install
port in their firewalls. See the GitLab documentation the gitlab-ce package. This script doesn't work in
for more details if you're unsure of how to do this. Ubuntu 15.04, so you'll need to edit the file /etc/apt/
During this install, you may be asked what sort of sources.list.d/gitlab_gitlab-ce.list and change any
profile you want for Postfix. In this case, select occurrences of vivid to trusty. After this, run sudo apt-
Internet Site. get update and you'll be able to install gitlab-ce.
66 www.linuxvoice.com
GITLAB TUTORIAL
3
Reconfigure 4
Login
By this point, all the GitLab software is installed, but it's That's all you need to do! GitLab is now installed and
not yet configured or running. Fortunately, GitLab running on port 80 of the machine. Just point your
comes with a tool to make this all really browser to http://localhost and you should see the
straightforward. You just need to run the following: login prompt. The default username and password is
sudo gitlab-ctl reconfigure root/5iveL!fe, however when you log in, you'll be
The gitlab-ctl program can be used to manage the prompted to immediately change this password.
GitLab server. You can see all the possible options by After logging in a second time (with the new
passing it the --help flag, but a few of the most useful password), you'll be taken to the admin panel. Here
are: restart, status and stop. These are all pretty self- you can see the status of GitLab . At this point, it'll be
explanatory. empty, but this will be a useful place to see what's
If you end up with a borked install, you can try happening. You can also keep an eye on updates here.
passing the cleanse option, which will delete all the You can get back to this admin page from other
data and leave you with a clean version (make sure pages in GitLab by clicking on the spanner icon in
you've got backups of any important data before the top right-hand corner of the screen. Using this
running this). If you're trying to debug a problem, the panel you can also set other options like alternative
tail option will output the logs of all the services, so authentication including OAuth to integrate GitLab
you can see a bit more about what's going on. with other systems.
5
Create project 6
Create users
The main reason for going to all this trouble is to host Now you've got your server running and created your
projects, so let's create one! Click in the plus icon in new project, the only thing left to do is to add users.
the top right-hand corner of the screen and you'll see As an administrator, you can create new users by
the New Project page. You can create a blank project, going to the admin area and clicking on New Users.
or import one from an external Git repository. This is fine if you want a few users, but it's not a great
Creating a project creates a Git store for the source idea if you need to add a lot. A better option is to allow
code. It also provides a wiki for documentation that users to register themselves. Go to the Settings menu
defaults to markdown formatting, an issue tracker to in the admin panel, and scroll down to Sign-In
help you fix bugs, milestone tracking and a few other Restrictions. Here you can set the options whereby
useful project management tools. All in all, it gives you users register themselves.
most features you'll need to keep your project ticking The best settings depend entirely on your use-case.
along smoothly. You may wish to open the system up to anyone who
The first step in a new project is usually creating wants an account, or you can control it more tightly –
the readme.md file to let everyone know what's going the best way to restrict users is by the domain of their
on with it. Click on the link on the main page to create email. You can also configure OAuth if you want to
this, and your project will be ready for contributions. enable restrictions like this.
www.linuxvoice.com 67
TUTORIAL DATABASES
SERVER 101: BRUSH UP
YOUR DATABASE SKILLS
Part 1: Most major websites (and apps) are built on databases.
Here's why they're important, how they work, and how to set one up from scratch.
MIKE SAUNDERS
W
hen compared with desktop environments,
distros, web browsers, games and other
fast-moving end-user software, databases
WHY DO THIS? may seem like the most tedious of topics that only
• Discover how web apps beard-endowed geeks get excited about. But
work under the hood databases are fascinating and well worth learning,
• Explore data stored by even if you never use one directly.
WordPress, OwnCloud
Without databases, most major websites
and co.
would be incredibly slow, inefficient and prone to
• Learn SQL to perform
powerful search queries major problems – and plenty of desktop apps use
databases as well. If you have a lot of information to
store in a reliable and easily searchable fashion, even
if it's your own personal beer mat collection, using a
database is a good idea.
But what exactly is a database? Imagine that they MariaDB has this peppy looking seal as its mascot,
didn't exist, and every program and website had to although it doesn't appear to have a name. Any
store data in its own format. Some software would suggestions, readers?
write data to text files, perhaps in CSV or XML format.
Other programs would use proprietary binary formats. included in a table, along with rows, which represent
Every piece of software would have its own data the entries. Look at this example:
storage, loading and searching routines, duplicating
a load of effort and making it extremely difficult for ID Name Login
other programs to share data.
(INTEGER) (VARCHAR(20)) (Date)
Databases fix this by handling all the dirty work
1 Mike 2015-01-12
Without databases, most major 2 Ben 2015-04-25
websites would be incredibly slow, 3 Graham 2015-10-02
inefficient and prone to major problems This table stores information on user logins for a
server, and has three columns: a unique ID for the
of storing, loading and searching data. A program login event, the username, and the date on which it
that uses the database – be it a website or a took place (year-month-day).
desktop application – speaks to the database in Databases can store all manner of information, but
a standardised language, so that other programs we can make them more reliable and easier to search
can query the same database and extract or upload by enforcing the type of data that a column can
information from it. The database worries about data contain (a bit like static typing in programming). So we
integrity, backups, duplicated entries and so forth, say here that the ID column can only contain integer
so the programs that use it can focus on their own numbers, while the Name column can contain varying
features instead. characters (ie a text string) up to a maximum of 20
characters, while the Login column can only contain
How it all works a date.
Most databases store information in tables, which A single database can contain many different
look somewhat like spreadsheets. A table consists of tables, and when a program (P) speaks to a
one or more columns, which define the types of data database server (D) to update data in a table (T), the
68 www.linuxvoice.com
DATABASES TUTORIAL
Choices, choices
MariaDB (and formerly MySQL) is the most common
database for low-to-mid-end tasks, such as blogs, simple
server applications and moderately popular websites. If
you need to chew through a lot of data, however, a good
alternative is PostgreSQL (www.postgresql.org). While
its name is rather a mouthful, PostgreSQL has enjoyed
a reputation for having advanced features way beyond
MariaDB, at the expense of some performance and ease-
of-use. In recent years, however, both databases have
come closer in terms of feature sets and speed, and they're
providing healthy competition for each other.
If you're writing some desktop software and want the
benefits of a database without having to run one as a
separate server, consider SQLite (www.sqlite.org). This is a
C library that lets you embed a database into a program to
get its benefits – data integrity, powerful search facilities
and so forth. As an example, the Firefox web browser uses
SQLite to store configuration data, bookmarks and other
information.
In this tutorial we're interacting with MariaDB via its command line tool, but web-based
software to manage databases, such as phpMyAdmin, is available as well.
conversation – usually over a network socket – goes
like this: This lets us create lots of tables containing different
P: Hi, I'm Program P. Can I have access to the login data, with all the entries linked together by the same
database please? ID. You could, of course, put everything together in
D: One sec... Let me check that you're allowed to a single table, but this is a more efficient and secure
access that particular database. Ah yes, you can – but approach. It's possible to restrict programs that
what's your password? access the database to specific tables
P: It's <password>. – so in a HR system, for example, the
D: OK, you're in. What do you want to do? kitchen staff are able to access an PRO TIP
P: Can you insert this data into table T please: "Mike, employee's dietary preferences table in Made a mistake creating a table? You
2015-01-12" the database without seeing their salary can delete it using the drop command.
For instance, if you entered a typo when
D: Done – that entry was added and has the ID 5. information and other personal details. creating login_dates, you can remove it
P: Thanks! I'm off now. It also gives you more flexibility with straight away by entering drop table login_
Note here that the program only supplied data backups, in that you can make regular dates;. Be careful with this command
though – it doesn't ask you if you're sure
for the Name and Login columns – but not for (daily) backups of the most important before deleting the table and its data!
the ID. That's because the database generates tables to save time, and less regular
(and increments) the ID itself, so that there are no (weekly) backups of the whole database
duplicates, and in this way we can turn the ID into a which may contain a lot of non-essential cruft.
primary key. What's that? Well, it's a unique reference A modification operation on a table – be it inserting
to a particular entry in the database, so we can have a new entry, modifying an existing entry or deleting
multiple entries with the same data (eg if Graham one – is known as a transaction, and good database
logged in many times on the same day) but with their software adheres to ACID principles. The letters mean:
own IDs. Atomicity A transaction should either work
And this is what makes relational databases work. completely, or not at all. You won't want an entry
With unique primary keys, we can create other tables containing a mixture of old and new data because
that contain additional data, for example: something failed during a modification.
Consistency The data must match the constraints
ID Command Exit code of the database. For instance, there cannot be
1 df -h 0 multiple duplicate primary key entries, and if a
column specifies that it should only store integer
2 crontab -e 1
numbers, a transaction which tries to insert text into
3 shutdown 1 that column will fail.
Isolation If multiple programs are connected to the
This table supplements the login one with extra same database and are trying to modify the same
information, and shares the same primary key in the table simultaneously, the database software should
ID column. So from the first table we know that Ben's handle this in an orderly fashion. Each transaction
login on 2015-04-25 has the ID number 2, and we can should be handled in isolation from the others, so
then use that ID in this table to see what command he that half-finished transactions don't get overwritten
issued (crontab -e). by others.
www.linuxvoice.com 69
TUTORIAL DATABASES
WordPress, OwnCloud or similar software that uses a
database, you may be familiar with part of this
process. Many web apps automate the job of creating
a database, which is a handy time-saver, but it shields
you from all the goings-on under the hood. Some
software requires that you create a database by hand
– so it's well worth learning the skills.
For this we're going to use MariaDB (www.mariadb.
org), a very popular open source database that was
forked off from MySQL after the latter was acquired
by Oracle and concerns were raised about its future.
MariaDB is used by many well-known web apps and
services such as the aforementioned WordPress and
OwnCloud, and it's fairly easy to get started with. Most
distros have it in their package repositories – so install
it using your package manager or via the command
line. For instance, in Ubuntu-based distros:
sudo apt-get install mariadb-server
This downloads and installs the database server,
and starts it running as a background service
Screenshot 1: Here we
Durability Once a transaction is done, the data (mysqld). Even though MariaDB is a fork of MySQL, it
create a table, specifying
its column names and the should be stored permanently – even if the OS still uses the same commands and executable names
type of data they should crashes or there's a power loss. as the old project, making it a drop-in replacement.
contain. Then we view a The isolation aspect is especially important when The first thing you'll want to do is to secure the
description of it. it comes to large-scale websites. Imagine a busy MariaDB installation as follows:
forum written in PHP with thousands of users online sudo mysql_secure_installation
simultaneously: you end up with thousands of PHP MariaDB has its own "root" administrator account
processes prodding the forum database at the same (separate from root on the operating system) which
is used to add new users and perform various
We're going to use MariaDB, a popular administrative tasks. MariaDB will ask you for the
current root password, which is blank after a fresh
database that was forked from MySQL installation, so just hit Enter. It will then ask you to
create a new password for root, so type something
after the latter was acquired by Oracle suitable here.
time, all trying to update tables as users write and edit Clean up your accounts
new posts, and the database needs to handle this Next up, the installation script will offer to remove the
sanely. anonymous testing user account – definitely a good
idea – and disallow root logins over the network.
Let's get practical Follow the remaining steps by entering Y, and then
So, that's enough theory – let's put it into practice by you'll be dropped back at the shell prompt. Now we
creating a database ourselves. If you've ever installed want to create a normal (non-root) user account and
a database, and set up permissions so that the new
user can modify the database:
Speaking the same language
sudo mysql -uroot -p
The commands that we use here to interact with the This logs us into MariaDB as the root user with
database are from SQL, the Structured Query Language. password authentication, so enter the password you
Not only does this provide a human-readable way to work
provided during the secure installation phase. Then
with data, but it's also standardised and lets you use the
same commands across different database programs. This enter these commands:
is a huge benefit if you need to switch to more powerful create database lvtest;
database software – for instance, if you've built a website create user 'lvuser'@'localhost' identified by 'pass123';
that is growing rapidly and your current database is grant all privileges on lvtest.* to 'lvuser'@'localhost';
crumbling under the load.
These steps are largely self-explanatory: we create
Many database programs also provide multiple options
for storing data on disk. These "storage engines" vary in the a new database called lvtest, create a user called
way they structure and save data onto the disk, so some are lvuser on the local machine with the password
optimised for maximum performance, whereas others are 'pass123', and then give this user all privileges (read,
designed with extreme reliability in mind. It's not something write, delete etc) for the lvtest database. In the lvtest.*
you generally need to concern yourself, but if you're
part, the asterisk is a wildcard and refers to every
interested in learning more, see https://mariadb.com/kb/
en/mariadb/storage-engines. table. You could replace the asterisk with the name of
a specific table if you wanted to restrict access.
70 www.linuxvoice.com
DATABASES TUTORIAL
But we don't have any tables yet, so enter exit (or
hit Ctrl+D like in a terminal window) to log out of the
MariaDB root account, and then log in with the new
account we created like so:
mysql -ulvuser -p
Enter pass123 as the password and you'll land at
the MariaDB prompt. Let's start building a database!
We'll create the example login information tables that
we used early on in the tutorial. First up we need to
select the database to work with:
use lvtest;
Now we create a table, specifying the columns and
types of data they should contain:
create table login_dates(ID int auto_increment primary
key, Name varchar(20), Login date);
When creating a table, we first specify the name of
the column (ID, Name, Login as per earlier) followed
by its data type (int for an integer number, varchar(20)
for text up to 20 characters, and a date). We can also
provide extra parameters for the column – like in the
Screenshot 2: After carefully inputting the data into the table, we can view it using the
ID column, where we say that the number should go
select command.
up by one each time a row is added, and that it should
also be the primary key.
If you now enter show tables; you'll see that login_ where operations using and keywords, allowing you
dates is now provided in the lvtest database, and if to perform more complicated searches. Indeed, some
you enter desc login_dates you'll be presented with search queries are so long and intricate that they have
a description of the table, showing its columns (aka more in common with a programming language than
fields) and the type of data that they can contain, as in a simple query language...
screenshot 1. Now let's populate the table with data: Of course, it's possible to use other operators with
insert into login_dates values (0, 'Mike', '2015-01-12'); searches as well. Take this for instance:
Repeat this command twice more with the data select * from login_dates where Name != 'Ben';
from the table earlier in the tutorial (Ben 2015-04-25 This shows the rows which don't contain 'Ben' in
and Graham 2015-10-02), and note that we specify the Name column. You can perform searches using
zero for the ID column. This forces the database greater-than or less-than parameters as well:
to generate its own numbers starting from 1 and select * from login_dates where ID > 2;
automatically incremented as described when we Note that text strings are placed
PRO TIP
created the table. Once you've inputted the data, view inside single quotation marks, whereas
MariaDB commands and parameters
it like so: numbers don't require them. Also note are not case-sensitive, and many
select * from login_dates; that the MariaDB command line client administrators use upper-case to
This select command is mightily useful and lets supports many time-saving features that distinguish them from actual data. So if
you look at other command examples on
you extract all kinds of information from a database. are included in Bash, such as command the web, you may see them written like
In this case it shows all data in a table (the asterisk history (use the arrow keys to cycle CREATE DATABASE lvtest;. It's purely a
being a wildcard again), as in screenshot 2. But why backwards through previously entered matter of taste, but if you're doing a lot of
database work – especially in scripts – it
is the command called select and not something commands), and Ctrl+R followed by text can be useful for clarity.
more meaningful like show or view? Well, in this case to find the most recent command that
we're doing nothing especially important with the included that text. Tab completion is
data – we're just looking at it. But in a typical real- also supported for table and column names.
world scenario, where a program retrieves data from a
database, it will go on to do other things with the data: Coming up next!
process it, combine it with something else, and so So there we have it: you now know the basics of how
forth. So you ask the database to select the items you databases work, how to create them, and how to
want according to your search terms, and then you manage data inside them. The skills you're learning
perform an action on the selection. here are applicable to other database servers, and
It's also possible to narrow down the data that's next issue we'll go further by looking at more
retrieved from a select command, eg: advanced commands to manipulate and search
select * from login_dates where Name = 'Ben'; through data and integrate with websites.
In this case, we're adding a restriction to the data
that's returned. We're saying: select all data from the
table login_dates, but only where 'Ben' appears in Mike Saunders is working on his own database, MikeSQL,
written entirely in 16-bit x86 assembly language.
the name column. You can string together multiple
www.linuxvoice.com 71
TUTORIAL EDUCATION
RASPBERRY PI:
INFORMATION CENTRE
Les Pounder hacks together an information centre to provide the latest Morrissey news.
LES POUNDER
I
nformation is all around us and it manifests itself
in many different ways. But what remains
constant is our need to consume and understand
WHY DO THIS? information, and in this project we shall use a
• Learn Python Raspberry Pi and a Displayotron HAT from Pimoroni
• How to parse RSS feeds (https://shop.pimoroni.com/products/
• Create functions displayotron-3000) as an appliance that delivers
• Use loops news and information in two ways: firstly it will act as
• Create variables an interface for streaming internet radio, and secondly
it will be an output device that shows the latest news
from the BBC as a scrolling “ticker tape” on the LCD
screen. We shall get the news via an RSS (Really
Simple Syndication) feed and use a Python library to
TOOLS REQUIRED
interpret the information and then present it on the
• A Raspberry Pi 2 or A+ B+
LCD screen.
• Displayotron HAT from
The Displayotron is easily fitted to all 40 of the The finished project can be easily powered from a USB
Pimoroni
• Ethernet or Wi-Fi dongle
extended GPIO pins found on the Raspberry Pi A+, battery giving a portable solution.
• 3.5mm speaker
B+ and Pi 2. Add-on boards should only be attached
• Power supply or USB
or removed while the power is off, otherwise you Pimoroni. You are safe to answer Yes to all of the
battery may damage either your Pi or the board. With installation questions. Once this is complete we shall
the Displayotron fitted, connect the rest of your test that it is working correctly. In the terminal type.
peripherals and boot the Raspberry Pi to the desktop. sudo idle &
You'll need to ensure that your Raspberry Pi is also With the Idle Python editor open we click on File
connected to the internet via an Ethernet or Wi-Fi > New to create a blank document. Save this as
LCDTest.py.
import dothat.lcd as l
Our device will show the latest news import dothat.backlight as b
from the BBC as a scrolling ticker tape b.rgb(255,0,0)
l.write(“Hello World”)
on the LCD screen In the blank document we enter the code, which
will import the LCD and backlight libraries as l and
dongle. Audio playback can also be configured using b. We then set the backlight colour to full red using
the new menu located in the top-right of the Raspbian the RGB values 255,0,0. Lastly we write the famous
desktop. Just right-click on the Volume icon and words “Hello World” to the LCD. Save the code and
select the correct output. We used a small 3.5mm click on Run > Run Module to see the output on the
speaker for our project. Displayotron.
With a successful test we now move to installing
Set up the software the Python library that will handle RSS feeds; this
After booting to the desktop we will need to install the is called Feedparser, and we can install it from the
software that powers the project, and this comes in terminal as so.
three stages. sudo pip install feedparser
We start by installing the Displayotron HAT library. This uses the pip Python package manager to install
With the terminal open enter the following command and configure the Feedparser library.
and press the Enter key to start. Our last stage of software installation is the VLC
curl get.pimoroni.com/dot3k | bash library for Python, which can be downloaded via the
This will run an installation script created by terminal. Create a new directory called Infocentre
72 www.linuxvoice.com
EDUCATION TUTORIAL
inside of the function and not between functions. The
Taking your project anywhere other variables are the names of radio stations that I
For this project we don’t need a powerful computer (we've like to listen to, we use their names to contain the link
used a Raspberry Pi 2). Rather we would benefit from a to the audio stream.
lower power drain that would enable our project to become We now move to create a series of functions that
mobile. Step forward the Raspberry Pi A+, which has all
will handle different aspects of the project. Our first
of the GPIO pins and runs the same Raspbian operating
system but uses the older BCM2835 system on a chip function handles reading RSS feeds.
found on the older Raspberry Pis (up to the B+ in late def feedme(feed):
2014). Retailing at under £20 and very power efficient, the feed = feedparser.parse(feed.encode('utf-8'))
A+ is a great board for building embedded projects such for i in range(6):
as robotics, weather stations and our information centre
b.graph_set_led_state(i,1)
project. Using the A+ with a USB dongle is possible, but
as the A+ only has one USB port you'll need a powered print(feed['entries'][i]['title'])
USB hub in order to configure your dongle and connect a scrollText(feed['entries'][i]['title'])
keyboard and mouse. It would also be prudent to set up b.graph_off()
SSH access, via the raspi-config terminal command, and fix We start our function by giving it a name, in this
the IP address of your Pi. Once configuration is complete
case feedme. This feed will take an argument, which
you can plug the Wi-Fi dongle directly into your Pi and
SSH in to the Pi to start your script. Or you could use cron will be the web address for the RSS feed. The RSS
to launch your script on boot. USB battery packs can be feed passed to the function is then parsed using
picked up for around £10 and should supply enough power feedparser and encoded to utf-8, which sanitises the
for your project. data so that we minimise any weird characters on the
LCD.
We then save the results to the variable feed. Next
and ensure that you are inside that directory before we create a for loop which will repeat six times, giving
downloading the file. us the top six headlines from the RSS feed. Why six?
wget http://bit.ly/LV21-VLC -O vlc.py Well the Displayotron has six bar graph LEDs located
This will download the file and rename it to vlc.py, to the right of the LCD screen, and we shall use those
which is required for this project. The vlc.py library to indicate our progress through the feed. We do that
needs to be in the same directory as our project code, by changing the LED state to 1, or on, for each LED in
as the library has not been installed system-wide; the column. With the LED lit we then print the RSS title
rather it is an external file that we are importing into to the shell for debug purposes and then call another
our code. function called scrollText, which we shall create later.
So our installation is complete, now we start our Once the for loop has iterated six times, the loop will
project. In the terminal we will open the Idle Python end and the bar graph LED will be turned off.
editor again. def scrollText(scrollBlurb):
sudo idle & if len(scrollBlurb) > 16:
With Idle open, click on File > New to create a new padding = " " * 16
blank document and save it as infocentre.py before scrollBlurb = padding + scrollBlurb + " "
continuing. for i in range(0,len(scrollBlurb)):
import vlc, time l.set_cursor_position(0,0)
import dothat.touch as j
import dothat.lcd as l
import dothat.backlight as b
import signal
import feedparser
Our first section of code starts with a series of
import commands; these bring in the various external
libraries that we shall be using. Firstly we import the
vlc library and the time library, which we use to control
timings for various functions. From the dothat library,
which is the Displayotron's library, we import the
touch, LCD and backlight functions. For each of these
imports we use a short reference, in this case j for
touch, l for lcd and b for backlight. Next we import the
signal library and lastly the feedparser library.
We now move on to creating a series of variables,
our first of which is a global variable called p that will
be used between functions that we shall create later.
A global variable is needed as otherwise the variable It’s important to test that the Displayotron works before you start the project. The best
would be “local”: in other words it could only be used way is to write "Hello World" on the LCD screen.
www.linuxvoice.com 73
TUTORIAL EDUCATION
the text to be scrolled. We start by checking the length
of the text, and if it is greater than 16 characters
then we enter an If..Else conditional statement,
which for if creates a variable called padding that will
contain 16 spaces. We then change the contents of
the scrollBlurb variable so that it now contains the
padding and text to scroll with an additional space at
the end of the text.
Next we use a for loop that will iterate the same
number of times as there are characters in the text to
scroll. We write the text starting in the top-left of the
screen, which is column 0 row 0. We then write the
text starting with the character that is at position i in
the text, where i is a an integer created by the for loop
that increments by 1 each time the loop is run. We
then add 16 to the i value to create an endpoint. The
script then sleeps for 0.25 seconds before repeating.
If the text is shorter than 16 characters then the else
part of the conditional statement is true and the
Scrolling text is an old cursor position is set to the top-left of the screen, and
method of displaying l.write(scrollBlurb[i:(i+16)]) the text is written to the LCD and printed to the shell
information in a small time.sleep(0.25) for debug.
amount of space, but you else: def player(radio):
need to ensure the speed l.set_cursor_position(0,0) global p
is just right.
l.write(scrollBlurb) p = vlc.MediaPlayer(radio)
print(scrollBlurb) p.play()
Our next function is called scrollText, and as its Our next function handles playback of the radio
name suggests we use this to scroll text across the streams. We call the function player and it takes the
LCD screen. This function takes an argument which is argument radio, which will be the URL of the radio
station that we wish to listen to. We use the global
variable p and then store the radio station details
External sources of data
ready for VLC to use as p. Next we instruct VLC to play
Pulling data from the internet is actually quite easy thanks the radio station stored in the variable p.
to the many different APIs (Application Programming def stop():
Interface) and many sources of information. So let's take a
p.stop()
look at a few…
b.rgb(255,0,0)
1) Pyjokes – http://pyjok.es
We all love a good joke and with this library you have l.write("S T O P")
access to many programming based jokes. You can install b.graph_off()
the library via pip with: Our last function handles stopping any audio
sudo pip install pyjokes playback. It calls the vlc.MediaPlayer(radio) function
2) OpenWeatherMap – http://openweathermap.org to stop, then changes the LCD screen backlight to red
Do you need an umbrella or sunscreen? Well fear not: using
Open Weather Map and pyowm, available from https://pypi. before printing “S T O P” on the LCD and ensuring that
python.org/pypi/pyowm/0.2.0 you can find out the weather any of the bar graph LED are turned off.
for any location and it can be easily used in this project. l.clear()
3) Fortune – https://pypi.python.org/pypi/fortune/1.0.1 b.graph_off()
Your own computerised fortune teller in Python, and We now move to the main body of code, and we
very similar to the fortune bash command. When used it
start by ensuring that the LCD is clear and that the bar
provides a random fortune or quote.
4) ihackernews – https://pypi.python.org/pypi/
ihackernews/1.0.0 Even Hacker News has its own API.
5) narwal – https://pypi.python.org/pypi/narwal/0.3.2b
Now you can procrastinate on reddit using Python, or pull
the latest news from your favourite subreddit.
There are many more Python libraries that can be used
with your information centre and you can find out more
from https://pypi.python.org/pypi/pip.
As for RSS feeds, there are many available covering
multiple topics, but some of our favourites are:
BBC RSS Feeds – www.bbc.co.uk/news/10628494
NASA – www.nasa.gov/content/nasa-rss-feeds
Raspberry Pi Foundation Blog – www.raspberrypi.org/feed
The Displayotron's six touch-sensitive buttons will even
work through acrylic, enabling projects to be enclosed.
74 www.linuxvoice.com
EDUCATION TUTORIAL
graph LEDs are off.
@j.on(j.UP)
def handle_up(ch,evt):
print("Playing BBC Radio 2")
l.clear()
b.rgb(255,0,255)
l.write("BBC Radio 2")
player(radio2)
Next we have blocks of code that handle the user
pressing the capacitive touch keys located on the
Displayotron HAT. To illustrate we will show how we
can use the Up key, located on the left of the LCD, to
play a radio station. We renamed dothat.touch as j
when we imported the library, so we instruct the code
to look for the up button to be pressed, and when that
is true it prints the radio station to the Python shell for
debug, clears the LCD screen, changes the backlight
so that Red and Blue are at full brightness, writes
“BBC Radio 2” to the LCD and then calls the player
This project is well suited to the Raspberry Pi A+, as it's power efficient and cheap.
function with the argument radio2, which will instruct
the player to load that station. The code for the down
button is similar but replaces the radio station with we print that it is working to the Python shell. The
BBC 6 Music and changes the backlight colour to the LCD screen is cleared and the backlight is set to off.
exact colour used for the station's branding. We then ensure that the LCD cursor position is in the
@j.on(j.LEFT) top-left of the screen and then write “Linux Voice” to
def handle_left(ch,evt): the screen. Next we use a for loop that will repeat 255
print("Left pressed!") times, each time it loops each of the three LEDs that
l.clear() make up the backlight, have their value increased by
l.write("BBC News Feed") one. A delay of 0.01 seconds is applied between each
time.sleep(1) change in value and this creates a gradual glow that
l.clear() slowly fades to life.
b.rgb(0,0,128) @j.on(j.CANCEL)
feedme("http://feeds.bbci.co.uk/news/rss.xml") def handle_cancel(ch,evt):
The code for the left and right buttons is similar print("Stopping Music")
to up and down. We print to the Python shell that l.clear()
the button has been pressed, then we clear the LCD b.rgb(0,0,0)
screen and write the name of the feed that we will be stop()
reading. This stays on the LCD for 1 second before Finally we create a function that handles the button
the LCD screen is cleared. The backlight colour is located at the top-left of the LCD screen. We shall
changed to a light blue, and then the feedme function use this to stop audio playback. Once the button
is called with the web address of the feed, in this case is pressed, output for debugging is passed to the
the BBC’s top news stories. Pressing the right button shell, the LCD screen is cleared and the
will call another feed and change the colour of the backlight is turned off. We then launch PRO TIP
backlight. the stop function and that handles All of the code for this project can be found
@j.on(j.BUTTON) turning off the audio and alerting the via GitHub http://bit.ly/LV21-Tutorial or
def handle_button(ch,evt): user. you can download the project as a ZIP file
from http://bit.ly/LV21-Tutorial-ZIP.
print("Button pressed!") signal.pause()
l.clear() Our last line of code ensures that the
b.rgb(0,0,0) project does not automatically shut down.
l.set_cursor_position(0,0) When you're ready, launch the code by clicking Run
l.write("Linux Voice") > Run Module. Press the up button to play a radio
for i in range(256): station, then press the cancel button on the top-left of
b.left_rgb(i,0,0) the Displayotron to stop playback before pressing the
time.sleep(0.01) down button. While the audio is playing you can press
b.mid_rgb(i,0,0) the left or right buttons to read the latest RSS feeds.
time.sleep(0.01) Congratulations! You have an information centre!
b.right_rgb(i,0,0)
time.sleep(0.01) Les Pounder divides his time between tinkering with
Our next function handles the button in between hardware and travelling the United Kingdom training teachers
in the new IT curriculum.
the left and right buttons. When this button is pressed
www.linuxvoice.com 75
TUTORIAL OPENVAS
AUTOMATIC VULNERABILITY
SCANNING WITH OPENVAS
Keeping track of security issues can be exhausting – so let your computer do it.
BEN EVERARD
I
f you run a server that's accessible from the
internet, whether it's a home server or one running
a multi-billion pound business, you need to make
WHY DO THIS? sure it's secure. If it's not, the it's just a matter of time
• Find vulnerabilities on before it falls prey to attackers. Keeping safe means
your servers always ensuring that your public-facing software is up
• Stop hackers breaking in to date and making sure it's configured correctly. In
• Keep an ever-watchful this tutorial, we're going to look at a piece of software
robotic eye on your that can automate this task – Open Vulnerability
machines to make sure
that they don't become Assessment Scanner (OpenVas), which is a tool to run
vulnerable checks on servers to see if they have any weaknesses
that attackers could exploit.
It can be a little tricky to set up OpenVas correctly,
but to make it easier, the project provides a pre-
configured virtual machine image that you can
download. You can grab this from www.openvas.org/ The Greenbone documentation covers almost everything
vm.html. you need to know, so if you ever get stuck, just drop into
As well as OpenVas, we'll need a machine to scan the Help menu and you should find out how to proceed.
for vulnerabilities. This could be a physical or a virtual
machine, but it's easier to understand OpenVas if the tab, make sure that Enable Server is checked and
machine has some security holes that the software that all the boxes are populated. If they're not, enter
can detect. We opted to use Metasploitable 2. This 192.168.56.1, 255.255.255.0, 192.168.56.2 and
is a virtual machine that's deliberately insecure for 192.168.56.100 from top to bottom.
practising attacks on. You can get this from http:// The OpenVAS file came as a VirtualBox appliance,
sourceforge.net/projects/metasploitable. so all you have to do is import the machine. Go to
File > Import Appliance and select the OVA file that
Keeping safe means always ensuring you've just downloaded, and it will create the virtual
machine. The only thing you need to do is set the new
that your public-facing software is up to machine to use the host-only network. Highlight the
new machine, then press the settings button. In the
date and configured correctly
To use these two virtual machines, you need to Keeping OpenVAS updated
install VirtualBox, which you should find in your distro's Network Vulnerability Tests (NVTs) are the scripts that
repositories. Once you have this installed, you need OpenVAS uses to identify any security problems. They're
to set up a host-only network. This will enable the constantly being updated, so if you want to check your
two virtual machines to talk to each other, but won't machines against the very latest security issues, you need
to make sure you have the latest NVTs. You can get these
expose either one to the network outside the machine.
by running the following in a terminal on your OpenVAS box:
This is important because the exploitable machine is openvas-nvt-sync
deliberately vulnerable and could easily be attacked if This will grab the latest NVTs from the official OpenVAS
it were public. feed. NSTs are written in NASL, a C like language that
First, create a host-only network by going to File originated with the Nessus security scanner. You can
create your own, although the process isn't particularly
> Preferences > Network > Host-only Networks
well documented. There's some information on how to
and clicking Add. Then, highlight the newly created get started writing NVTs on the OpenVAS website at www.
network (which will probably be called bvoxnet0), openvas.org/nvt-dev.html.
and click on the screwdriver icon. In the DHCP Server
76 www.linuxvoice.com
OPENVAS TUTORIAL
Automatic vs manual pen testing
Vulnerability scanning with a tool like OpenVAS is a great
way of improving the security of a server. It's quick,
easy and cheap. However, it's not a simple solution to all
security problems. The scans that OpenVAS runs try to
identify known security vulnerabilities. There may be other
issues that haven't been publicly reported that hackers
are exploiting. Another limit of automatic vulnerability
scanning is that software that's custom written won't be
adequately checked for security holes. Because of all these
reasons, a vulnerability scanner should never be seen as a
guarantee that there aren't any issues on the server, and all
normal security precautions should still be taken even if the
scanner doesn't flag up any issues.
In contrast, a manual penetration test performed by
an experienced tester should be far more thorough and
ultimately lead to a more secure system than an automatic
scanner. A good manual penetration test should identify
areas of potential vulnerability that don't have published
security holes (this could be particularly important if
you have unusual configurations or custom code). It
may also test out the effectiveness of mitigation tools
such as intrusion detection systems that won't be tested
using automatic scanners. OpenVAS can form a useful
part of your security policy, but shouldn't be considered a
substitute for having robust security.
You'll then need to log in. The username and There is a wizard that's
supposed to ease the task
password are both admin.
of creating tasks, but we
new window, go to Network, and change Attached To find it easier to build them
to Host Only Adapter. Greenbone manually, as the wizard
Metasploitable 2 comes as a virtual disk image The user interface of the Greenbone Security can be a little confusing.
rather than an appliance, so you have to manually Assistant leaves a little to be desired, but provided
create a virtual machine for it. Click New to open the you do everything in the right order, it works well.
new machine wizard, then enter a name and select The interface revolves around targets, tasks and
The SecInfo Dashboard
Type: Linux and Version: Ubuntu (64 bit). The default reports. Targets are machines (usually identified will give you details of
512MB of RAM is fine, as we won't be using this by IP address) that you wish to scan. Tasks are a all the vulnerabilities and
machine for anything taxing. particular setup of target and type of scan that can tests that OpenVAS knows
In the Hard Drive screen, select Use An Existing be run one or more times. Reports are the results of about.
Hard Drive, then select the hard drive image that
you've just downloaded and uncompressed. With this
done, you can click on Create. The final thing needed
to set up this virtual machine is changing the network
adaptor to Host-Only in the same way as with the
OpenVAS virtual machine.
You can now start both the virtual machines by
right-clicking on them and selecting Start. This will
open new windows for the virtual machines, and you
can interact with them through this window.
OpenVAS runs as a daemon, which enables you to
schedule and queue jobs to run in whatever fashion
you want. Happily, there's a web-based interface to
OpenVAS called Greenbone Security Assistant, so we'll
interact with this rather than OpenVAS itself.
Once the OpenVAS virtual machine has started, you
will see something similar to:
The web interface is now available at: 192.168.56.2
The numbers at the end might be different when
you run it. Whatever they are, this is the web address
for the Greenbone Security Assistant, so open up
a web browser and point it to http://192.168.56.2
(or whatever IP address is displayed in the virtual
machine).
www.linuxvoice.com 77
TUTORIAL OPENVAS
You can view the content tasks running. The process of scanning a machine is the Greenbone screens. Click on this to create a new
of the report before the creating the target, creating the task, scheduling the Target.
scan is finished, which
task to run and then investigating the report. Give the target a name (we went with the rather
is particularly useful for
The first task, then, is to create a target. Targets are unimaginative 'Metasploitable'), make sure Manual is
long-running, complex
scan profiles like Full and basically names given to IP addresses, so we need to checked and enter the IP address in the host box. With
Very Deep. find out the IP address of the Metasploitable virtual this done, just click Create Target.
machine. Switch to the Metasploitable window and The next step is to create a task. These are scan
log in with msfadmin/msfadmin and enter: configurations that can be executed. In normal use
ip addr (which we'll look at a bit later), tasks can be run at
In the output, you should see a line that looks regular intervals to make sure that a given machine
something like this: stays secure.
inet 192.168.56.3/24 Go to Scan Management > Tasks and click on the
The numbers may be different, but this new icon to set up a task. The key items on this form
(192.168.56.3) is the IP address of the Metasploitable are the Name (this can be anything that you use to
machine. In Greenbone, you can now go to identify the scan), the Scan Target, which should
Configuration/Targets. The Add New icon is a white be the target that you've just set up, and the Scan
star on a blue background. It's small and easy to miss, Config. This final item dictates the type of scan being
but once you've found it, it's in the same place on all run. It comes down to which checks are run against
the host. The more checks that are run, the more
thorough the scan will be, but the longer it will take
You can run periodic scans against your and the more noise it will generate on the network
machines and receive the reports (see boxout on Automatic and Manual Testing). We'll
go for a Full and Fast test.
detailing how they have performed
Testing for exploits
Click on Create Task (in the New Task box, not in the
Alternative vulnerability scanners New Container Task box), and you'll go through to the
OpenVAS is a fork of the Nessus security scanner. Nessus
Task screen. Here you should see all the details of the
was originally released under the GPL, but its developers task that you've just created, but it won't yet have run
decided to close the source code (since they owned the so there aren't any results. Click on the green play icon
copyright, they could legally stop releasing it under the at the top to start the task running straight away.
GPL, but couldn't revoke the open source access to earlier The page will hold the progress of the scan, but may
code). Nessus remains a very popular scanner and is free
(as in zero cost, not as in free software) for personal use.
not automatically update. Click on the refresh icon in
There many other commercial vulnerability scanners the page (not the main browser refresh, but the icon
available, including Nexpose from Rapid 7, and there are on the web page), or change No Auto Refresh. The
some specialised open source scanners that are good at status entry will update to show you how far the scan
identifying problems with particular pieces of software has progressed. It may take a little while to complete,
such as WPScan (http://wpscan.org), which checks
WordPress sites for vulnerabilities, and OWASP's ZAP, which
but once it has, you'll be able to view the report.
can check websites for potential security issues. In Scan Management > Reports, you'll see a list
of every task that has run. At the moment, it will just
78 www.linuxvoice.com
OPENVAS TUTORIAL
Metasploitable includes a
variety of vulnerable web
applications including
TikiWiki. Take a look at the
OpenVAS report for details
of how to attack them.
show a single entry. In this screen, you can see a Vulnerabilities and Exposures database, which is
summary of the results breaking down any issues it accessible at https://cve.mitre.org. This database
finds into high, medium and low severity. There are enables security researchers and software developers
also items listed as Log, which means that there isn't to track any problems. They're particularly useful for
a vulnerability, but information that you may wish to identifying which release of a program fixes a specific
look at. In this report, there should be lots of each level issues, as the CVE should be in the release notes.
of issue. Click on the date to get the detailed contents There will often be a link to the software's website that
of the report. should have details about how the vulnerability can
If this were a scan on a machine that was network- be fixed.
accessible, all the entries listed should be fixed. If
you click on an entry in the Vulnerability column, then Schedules and alerts
you'll see more details about the particular issue. This One of the great features of Greenbone is the ability
Vulnerability Details screen holds a few key pieces of to schedule scans. Using this, you can run periodic
information. As well as a general description of the scans against your machines and receive the reports
problem, you should see details of how to fix the issue detailing how the machines have performed. You
(typically, this is upgrading the problem software, but can even set up the system so that it runs the scans
it could be filtering ports in the firewall or changing silently and only alerts you when vulnerabilities
configurations). There should also be a CVE reference. are discovered. These features are set up using
This is an identification number in the Common Schedules and Alerts.
In OpenVAS, schedules and alerts are both entities
Exploiting Metasploitable that are created and then added to the task. This is a
little counter intuitive, since it's more natural to think
If you run a scan of Metasploitable using the profile Full of a schedule as something to which you add a task,
And Very Deep Ultimate, it will check very thoroughly for
rather than the other way around. As long as you do
vulnerabilities. In fact, it should find 19 high-priority issues.
Let's take a look at just how serious some of these are. One everything in the right order, though, everything is
of the severity 10 issues in the report is NFS Export. If you relatively easy.
click on that, you'll see that the problem is that the root Both schedules and alerts have entries in the
directory (/) is being shared using the Network File System Configuration menu that take you to the appropriate
(NFS). At you might imaging, this is quite a serious issue
forms, which are easy to fill in. Once you've entered
that can be exploited very easily.
To see how a hacker can break in, you'll first need to the details you need, you can go back to the task
make sure that you have the relevent nfs tools installed on (Scan Management > Tasks and then click on the
your machine. On an Ubuntu-based machine, this is done appropriate task), and use the spanner icon to edit the
with the command: task. In the task, you can apply whichever schedule
sudo apt-get install rpcbind nfs-common
and alerts you wish. Using this, you can make sure
Once you've got these, you can create a new directory
and mount the exposed network share with: that servers don't regress into unsafe states or miss
mkdir /tmp/nfs-test out on critical security updates. And that's about it –
sudo mount.nfs 192.168.56.3:/ /tmp/nfs-test/ automatic vulnerability scanning, made easy!
After this, you should find that you have complete access
to Metasploitable's filesystem. Not all of the vulnerabilities
are as serious as this, but there are many serious problems Ben Everard is the best-selling co-author of the best-selling
that can allow hackers to gain access to the system. book, Learning Python with Raspberry Pi.
www.linuxvoice.com 79
TUTORIAL HARDWARE HACKING
BUILD A 3D HEAD TRACKER
WITH OPENTRACK
Map your head movements to mouse control with a handful of open source.
GRAHAM MORRISON
A
fter experimenting with the kind of gyro among gamers where head tracking provides an
sensors that festoon most smartphones, the additional level of immersion in first-person games.
Oculus Rift virtual reality headset, due to be Turn your head left, for example, and your view turns
WHY DO THIS? released in Q1 2016, opted instead for a series of to the left.
• It’s cheap and very cool infrared LEDs surrounding the headset itself, coupled But there’s also a great open source project (called
• Generates X, Y, Z, yaw, with external cameras that translate the movement of Opentrack) that uses a DIY headset and translates
pitch and roll data those LEDs into 3D space. This is because it’s often head movement into up/down (y), left/right (x), in/out
• Augment virtual reality… quicker and more accurate, especially when cameras (x), roll, pitch and yaw parameters, which you can then
• …or play games with your operate at 60 or 100 frames per second. The Oculus use for anything from playing games to controlling
head Rift is cutting edge, but the principles behind this your desktop. And it’s exactly this kind of low-cost
tracking have been around for a while, especially configuration we’re going to build now.
What you’ll need
1 6
4
5
7
8
2
3
1 3 x infrared LEDs it up with a hair dryer and it shrinks around the entire package.
We bought OSRAM SFH485P LEDs from Farnell. Because they radiate light at 5 Metal coat hanger/wire
infrared range, they’re not visible to the human eye. There are may ways to construct a frame. Some projects have used pen cases.
2 1 x 6.8Ω resistor But we’ve had good results with a wire hanger.
The circuit for powering the LEDs is incredibly simple. You need a single resistor 6 Soldering iron/solder
and some wire. We’re taking 5v power from a USB power adaptor. We’re only going to make a few soldering points – connecting the LEDs to wire
3 USB lead and the resistor, making this project ideal for a beginner.
We also butchered a USB lead so we can connect the headset to a USB port on 7 Visible light filter
a computer or a phone charger, but batteries are another option. Yes, we’ve used a cutout from an old 3.5-inch floppy disc. Any old floppy will do
4 6.4mm heat shrink but you get extra bonus points for using Amiga Format's Cannon Soccer
This stuff is amazing. Put your wires and even resistors into the tubing and heat coverdisc. You could also used an exposed piece of old school negative film.
80 www.linuxvoice.com
HARDWARE HACKING TUTORIAL
1
Which webcam 2
Disassemble webcam
The Playstation 3 webcam we’ve chosen is commonly We’re now going to do some physical hacking. Taking
used for head tracking and works perfectly with Linux. apart the webcam is relatively easy, although you do
It’s also easy to take apart, which we need to do so need to use a little brute force at one point. Start with
that we can remove the infrared blocking filter and the small rubber covers hiding the four screws on the
add our infrared bandpass filter (blocking visible light), rear, followed by the screws themselves.
which can be slotted in front of the lens, blocking The front and rear now need to be pried open with
everything but infrared light from getting to the sensor. a flat-headed screwdriver. Be careful as the main PCB
It’s also cheap, with new models selling for around is held in between these two parts and you’ll need
£4 delivered on eBay. But the best thing about this some patience as you work around the gap, gently
camera is that it’s capable of 100 frames per second, making it larger until it comes apart. You’ll now see
and you can change the field of view by twisting the the PCB, and you need to remove the screws that
lens surround. Unfortunately, the process of taking the connect the PCB to the internal plastic. The PCB
camera apart and placing a filter across the lens should then come out without difficulty, revealing the
renders it useless as a normal webcam, unless you’re lens component, which can be taken off after
happy capturing out of focus images of infrared light. removing the two screws holding it on.
3
Removing and adding the filter 4
Testing the webcam
Be careful not to smudge the lens or the sensor, or to The Playstation 3 Eye camera works in Linux without
allow dust to settle on the sensor. There are now two any drivers. Just connect the camera’s cable to a
steps to perform before the lens will allow only the spare USB port. We used VLC, selecting
infrared light from our LEDs. The first is to remove the Open Capture Device as a media source
PRO TIP
infrared blocking filter, which is the thin sliver of glass and making sure the Video Device Name
Use the V4L2 Test Bench utility to change
closest to the sensor in the lens. This is best removed is pointed to the webcam. If the infrared webcam parameters, such as contrast,
with a sharp knife, going around the edge of the filter filter is working, the image will have brightness, exposure and frame rate, to
before levering the glass from inside. washed out black and white appearance. improve the tracking algorithm.
We now want to replace this with our bandpass You may notice that the camera only
filter, allowing only the light from the LEDs. We took finds a focus somewhere in the middle
apart an old Amiga 3.5-inch floppy disc and cut a of switching between the two field-of-view modes
small section that would fit exactly into the gap left by because removing the infrared blocking filter has
the blocking filter. You then need to retrace your steps, changed the focal length of the lens. There are two
putting the entire camera back together. solutions; the first is to do nothing, as even without
focus the LEDs will be detected: the second is to
carefully twist the lens until it sticks to the focus point.
www.linuxvoice.com 81
TUTORIAL HARDWARE HACKING
5
Tracking design 6
Connect LEDs via wire and resistor
With the webcam working, it’s now time to construct This couldn’t be a simpler circuit. Cut open the spare
the frame that will hold our LED circuit; this is because USB cable and strip the ends of the red (+5v) and
the length of the wires you use will depend on the black (GND) wires. This will give our unit power.
shape of the frame. Its optimal shape has already Connect the red +5v wire to one leg of the resistor and
been calculated, and you should be able to see the the other leg of the resistor to the shorter leg of one of
dimensions below. It’s an L shape, and two LEDs will the LEDs. LEDs have polarity, which means they need
sit on the ends while a third is at the corner of the to be connected the correct way around. The short leg
shape. We’d recommend laying out the components is usually positive while the short leg (along with a flat
and the wires so that they all sit close together. side on the LED) is usually negative. Connect the short
leg of this first LED to the long leg of the next and do
the same for the final LED, connecting its short
negative to the GND (black) wire on the USB cable to
complete the circuit. You can quickly check it works
by connecting to a USB port and viewing using a
smartphone camera.
7
Shrink 8
Install Opentrack
With the circuit built and connected, and the basic The software that we’re going to use to take the
design of the headset complete, it’s time to put it all realtime video of our headset and convert the
together. Our head tracking design is for a clip that's movement of the LEDs into raw data is called
intended to connect vertically to the side of a cap, or Opentrack. You may be able to find a package for your
best of all, a pair of headphones. It’s easy to use a distribution, but we needed to compile the latest
large clip to temporarily connect the headset, but we release for Arch, which has a few dependencies,
decided to dismantle an old pair of headphones and including Qt5 and Qt5-serial port, cmake and opencv.
permanently attach the head strap to the LED head To build the project, download the source code from
tracking device. It’s worth putting the camera above https://github.com/opentrack. Unzip the file and cd
your screen and positioning yourself with the headset into the directory, then type mkdir build to create a
to make sure all three LEDs are visible as you turn directory called build and cd into this. Now type
your head left/right and up/down. The orientation of cmake .. to generate a Makefile from the source code;
the headset is such that the corner of the L shape sits when this has finished, type make to start the build
near your ear, with the long edge going down to your process. When complete, you can run the binary from
chin and the short edge up to your eyebrow. the build folder by typing ./opentrack.bin.
82 www.linuxvoice.com
HARDWARE HACKING TUTORIAL
9
Configuring Opentrack 10
PointTracker 1.1
Opentrack uses three different sets of protocols, each The only plugin that needs configuration is
of which is selected and configured using the PointTracker. To do this, click on the … button to open
drop-down menus on the left. Tracker is the plugin the options view. First make sure your USB camera is
that generates the raw data, Filter is the plugin that selected in Device and click on Start in the main
smooths out the rough edges from the data, and application window. This will start the tracking and the
Protocol is the plugin that delivers this data to its final video input, enabling you to fine-tune the plugin while
destination. We’re going to use the PointTracker 1.1 seeing the output. Natural light is a big source of
tracker to generate the data, as this is the one that infrared, so you may need to close curtains or doors
derives positional data from the video of our LEDs, but until your LEDs are the only sources visible.
there’s also a plugin for an Arduino-based sensor if The main parameters to adjust are Threshold,
you wanted to take the project further. We’ve found which governs the strength of infrared declared a
the best filter plugin is Accela, which is usually point, and min/max sizes. We set the threshold to
selected by default, and the easiest protocol to get almost its maximum, shielding out background light,
working is libevdev joystick receiver, which turns all and a min/max size of 5x/25px. When the tracking is
of your movements into a virtual joystick that can working, you’ll see the game data and the octopus
then be reconfigured to control anything. image move in sequence with your own movements.
11
Mapping 12
Taking it further
Before all this data becomes useful, you need to Another essential step is to map 'Center' to a key from
fine-tune the scale of each axis. This is done by the Options panel in the main Opentrack window. This
clicking on the ‘Mapping’ button of the main enables you to set your straight ahead central point at
application. With the tracker running, you’ll see a any time, effectively resetting the
cursor move along a line. Twist your head to the left, tracking data. The simplest way to play PRO TIP
for example, and the roll angle will change, look down with the output is to use the joystick The Model view in the PointTracker plugin
at the keyboard, or left and right, and pitch and yaw driver in your favourite game – manually can be used to fine-tune the size of your
will change. The straight line can be edited to amplify mapping the input to the controls used LED clip and its position relative to the
camera.
the angle by dragging down the top-right point, and to change the first-person view. It’s also
new Bézier points can be used to fine-tune the curve. possible to map these inputs into
By default, the curves are identical when you cross desktop controls, moving the mouse around and even
over the central point, but you make them asymmetric selecting things with a gesture. We used a tool called
by enabling the mapping curve below. There’s no jtest-gtk to calibrate the inputs, and antimicro to map
correct way of doing this, but play with the curves until the inputs to mouse and keyboard controls. Finally,
you can make best use of your movement. you can map the control to an external virtual reality
headset using the ‘UDP receiver’ protocol.
www.linuxvoice.com 83
CODING GIT AND GNU R
CALCULATE YOUR BUS
FACTOR WITH GIT AND R
Discover what would happen if open source lost some of its core developers.
ROBIN GOWER
A
ll open source projects rely on their
developers, but some rely on a handful of
people so much that the project would be in
WHY DO THIS? serious jeopardy if one or two people stopped coding.
• Find out if the kernel The degree to which a project is reliant on just a few
could cope without developers can be measured by something called the
Torvalds... bus factor, which is the number of developers that
• Compare your work with would need to be hit by a bus before a project is in
successful open source
projects serious trouble. A low bus factor means that
• Read between the lines – knowledge is concentrated in a few vital people, while
learn about your codebase A high bus factor means that a broad range of R Studio (AGPLv3) is a popular user interface built with
with data analysis individuals know enough to carry on with the project the Qt and GWT graphical toolkits
even if others leave.
In this tutorial we'll show you how to calculate Ctrl+Shift+Enter to run the whole file (see the Code
the bus factor of your project, or of the open source menu for related short-cuts).
projects you love (and perhaps depend upon). We'll Let's try this now. Create a new script (File > New
get our data from Git commit logs and analyse it File > R Script) and type 1:5, with your cursor still
using GNU R, the statistical language and computing on this line, hit Ctrl+Enter. In the console pane the
environment. To set-up an R environment you'll first command 1:5 is run and the result – a vector of
need to download R, either from your distro's package integers from 1 to 5 – is displayed beneath. The
manager or from r-project.org. R comes with its own command lines have a prompt string > and the result
REPL (read, evaluate, print, loop) interpreter, which you line, in this case, starts with [1]. This is a guide that
can start by running the command R at your prompt. becomes useful when the output spans multiple lines.
However, instead of using this, we recommend you Try changing the command to 1:100*2, either in the
use an IDE. In recent years RStudio has established source view or directly in the console (you can hit the
up arrow key to retrieve historical commands). Notice
The degree to which a project is reliant that the results now spread over multiple lines with
the numbers at the left indicating the index number of
on a few developers can be measured the first element on that row. You can see that arrays
indices in R start from 1 (not 0).
by something called the bus factor Let's bring in the other panes. To do this we need to
declare some variables. Add the following code:
itself as the de facto standard (rstudio.com). In x <- 1:10
addition to the console you get an editor with syntax y <- rnorm(10)
highlighting, plotting windows, a help browser and d <- data.frame(x,y)
other tools. Once you're up and running you can install This assigns the range from 1 to 10 to the variable
the libraries you'll need with R's package manager. named x. Into the y variable we've inserted 10 random
(normally distributed) numbers. Finally we create
The RStudio IDE a data frame with these two variables as columns.
Assuming you've also already installed the Git client Data frames are the fundamental data structure in
from your distro's package manager, we're ready to R. To see the contents of d simply call it alone on
go. We'll start with a brief tour of the RStudio interface. the console or use View(d) to get a spreadsheet
In the default layout you'll have four panes. In the representation.
top-left we have the source pane for editing code and In the top-right pane you'll see the environment
sending it to the R interpreter displayed in the console view. This is a list of the currently declared variables
beneath: press Ctrl+Enter to run the current line, or (ls()) alongside a summary of their contents that
84 www.linuxvoice.com
GIT AND GNU R CODING
system(command, intern=T)
Getting help }
To get documentation on a given function in R you can call If you call, for example, author <- get_author_for_
for help. To find out about the parameters and return value lines_in_file("/path/to/repo", "README") then you'll
of the linear model function, for example, call help("lm"). get a vector of strings with the author of
You can also use the shorthand ?lm, although this won't
each line. PRO TIP
work with control syntax, meaning you have to call
help("for"), for example. In RStudio, the F1 key will bring up As you might expect from a statistical R recognises arguments in function
help for the function under the cursor. computing environment, there are quite calls by either position or name. Here
If you don't already know the name of the function a few different ways to count things. the system function has several default
you can use help.search("distribution") or ??distribution arguments, you can override a given
Here we will use table providing a parameter by name without having to
to search the documentation. Sometimes you'll want to
“responseName” (rather than having the specify all of the others (as you would
learn by exploring rather than having a particular goal need to do with positional arguments).
in mind. You can browse package documentation with default column name “Freq”) and we'll
help(package="ggplot2"). coerce the output to a data frame so we
In addition to the basic API documentation, some can combine results more easily.
packages also provide vignettes, which are PDF or count_line_authors <- function(author) {
HTML documents that provide an overview or some
as.data.frame(table(author), responseName="line_
demonstrations. You can find them with browseVignettes(p
ackage="colorspace"). There are also demos, which you can count")
find with demo() and run with eg demo(graphics)`. }
We should also keep a track of which file we used
to derive these counts. We can do this by adding a
has been generated by the str command, which is column to the data frame with the filename:
designed to give a reasonable, compact description of file_blame <- count_line_authors(line_authors)
the structure of any object. file_blame$filename <- rep(filename,nrow(file_blame))
If we now execute the command plot(d), the Encapsulating this in a function we get:
bottom-right pane will change to the plot view and count_of_lines_by_author_in_file <- function(repo_path,
you'll see a basic dot-plot of the random numbers. filename) {
line_authors <- get_author_for_lines_in_file(repo_path,
Let's crank out some stats! filename)
To run this shell command on different repositories file_blame <- count_line_authors(line_authors)
we can create a function that will take a path and file_blame$filename <- rep(filename,nrow(file_blame))
return a vector of file names. We will use interpolation file_blame
to feed the path into a string then call this using }
system. The paste() command takes any number Now we need to call this function on each file and
of arguments, converts them to character vectors, sum the results. Although R does have for loops, the
and concatenates them interleaving a separator idiomatic way of iterating like this is to use an apply
– this defaults to the space character (if you don't function. The base R system comes with a handful of
want spaces you can use paste(..., sep="") or just functions for this purpose, but instead we're going to
paste0(...) for short. This function returns a list of files use the plyr package, because it provides a consistent
in the repository (in all sub-directories): and intuitive interface. You can install this package
enumerate_files_in_repo <- function(repo_path) { with install.packages("plyr") and then load it with
command = paste("cd", repo_path, "; git ls-tree HEAD -r | library(plyr). The library follows a pattern described by
awk -F '\t' '{print $2}'")
system(, intern=T)
}
Once we have enumerated a list of files, we want
to figure out who committed (ie "knows about") each
line. We can use git blame with the --line-porcelain
parameter to get a machine readable attribution for
each line. We're only interested in the lines beginning
with author, so we pipe the output via the grep
'^author ' command, and for each of these we only
care about the bit after the field name, so we then pipe
this via sed -e 's/^author //'. This gives us a long list of
author names, one for each line in the source file.
get_author_for_lines_in_file <- function(repo_path,
filename) { ggplot2 is a powerful
command = paste("cd", repo_path, "; git blame --line- visualisation library for
porcelain", filename, "| grep '^author ' | sed -e 's/^author R that implements the
//'") “Grammar of Graphics".
www.linuxvoice.com 85
CODING GIT AND GNU R
count_of_lines_by_author_in_repo <- function(repo_
path) {
repo_tree <- enumerate_files_in_repo(repo_path)
lines_by_file <- adply(repo_tree, 1, count_of_lines_by_
author_in_file, repo_path=repo_path, .progress = "text")
ddply(lines_by_file, .(author), summarise, line_
count=sum(line_count))
}
We now have a data frame with two columns:
authors and line_count. You can see from this which
authors have made the biggest contribution (and hold
the most knowledge) by sorting the data frame:
author_lines[order(author_lines$line_count,
decreasing=T),]
The order function returns a vector of indices that we
use to extract rows from the data frame in the order
we wish. If you have a very large list of contributors
you might want to wrap this with the head function.
The bus factors of a
handful of open source
the author, Hadley Wickham, as split-apply-combine: I am become death, destroyer of developers
projects (higher is better/ take something, split it into partitions according to If you'll forgive a morbid extension of our metaphor,
more robust). Low bus some feature, apply a function to each piece, then this is a bit like lining up our authors in front of the bus
factors could be a sign combine the results. with the biggest contributor first. We need to know
that there aren't enough We'll use the adply function as we want to take how far the bus has to travel before the project is in
developers to spot security an array of filenames and turn it into a data frame of serious trouble. We're less interested in knowing who
bugs in critical projects line counts by author. The first argument is our list they are, and instead want to know how many we can
like OpenSSL. of filenames; adply also needs to know how to split afford to lose (in the worst case scenario). We also
this up. In our case this second argument is simply need to know what we're losing in terms of knowledge
"1" (ie “by row”), the third argument is the function as the bus makes its grisly journey. In analytical
that will be applied to each entry in the split-up array terms this means we'll sort the authors in order of
(count_of_lines_by_author_in_file). Since we also contribution, then calculate the cumulative number
need to specify the repo_path we add this as a named of authors and lines of code as we add (or remove)
argument. The git blame command can take a non- additional contributors. Since we want to compare
negligible amount of time to run, and it's this project to others we also want to calculate
being run many times, so to have some percentages for this variables. The following function
PRO TIP feedback during the process we provide wraps up these details – note that before we sort the
There's more than one way to skin an the final argument to get a progress bar. data frame, we're also sorting the authors dimension
analytical cat so it's important to explain
lines_by_file <- adply(repo_tree, 1, count_ (which will be useful if you want to plot this):
your methodology and the implications it
has for your conclusions. The beauty of of_lines_by_author_in_file, repo_ calculate_author_contribution <- function(author_lines) {
writing and sharing R code is that your path=repo_path, .progress = "text") author_lines$author <- reorder(as.factor(author_
research can be reproduced and validated
lines$author), -author_lines$line_count)
by others.
This gives us a data frame with a sorted_contributions <- author_lines[order(author_
row for each file and author (many lines$line_count, decreasing=T),]
authors will appear multiple times as they contribute sorted_contributions$cumulative_line_count <-
to different files). To calculate a grand total across all cumsum(sorted_contributions$line_count)
the files in the repository we use another plyr function, sorted_contributions$cumulative_author_count <-
this time from one data frame to another: ddply. 1:nrow(sorted_contributions)
The signature for this function is similar to adply, sorted_contributions$cumulative_line_percent <-
but we specify the split with column names, not an sorted_contributions$cumulative_line_count/
integer, here provided with a quote function .(author) max(sorted_contributions$cumulative_line_count)
that prevents author from being eagerly evaluated sorted_contributions$cumulative_author_percent
when the function is called, instead evaluating it in <- sorted_contributions$cumulative_author_count/
the context of the data frame (as a column name). max(sorted_contributions$cumulative_author_count)
Again we pass a function to apply; in this case we use return(sorted_contributions) }
summarise, which also expects pairs of var=value to Before we can calculate the bus factor, we must
describe variables that it should add to the result. Here make a decision: how much knowledge do we need
we want to sum the line_counts: to lose before we would say the project is in serious
ddply(lines_by_file, .(author), summarise, line_ trouble? For simplicity's sake we'll say half (the charts
count=sum(line_count)) show that this is a fair assumption for most projects).
We can tie all this together with another function: To see 50% of the line_count:
86 www.linuxvoice.com
GIT AND GNU R CODING
Projects with larger teams
can better afford to lose
individual developers and
have higher bus factors
overall.
calculate_bus_factor <- function(author_contribution, ggplot(all_ac, aes(cumulative_author_count,
critical_threshold=0.5) { cumulative_line_percent)) +
critical_contributions <- author_contribution[author_ Add a geom to determine how the data will be
contribution$cumulative_line_percent < critical_ displayed on the screen. Here we choose a simple line
threshold, ] to connect the data points:
nrow(critical_contributions) geom_line() +
} Expand the limits of the graph so that the origin of
So, now that we have an algorithm, let's calculate 0,0 is in view (ie no authors, no lines):
the bus factor for some open source projects. expand_limits(y=0, x=0) +
Furthermore we can also visualise the relationship Specify the scales. In the case of the x axis, we want
between number of people and proportion of the code a logarithmic transformation, as the interesting bit is
base rather than simply looking at the 50% mark. first few authors. We also add titles, and set the y axis
We've gathered data from a range of open source to use percentages for labels (10% rather than 0.1):
projects and bound these author_contribution data scale_x_log10("Number of Authors") + scale_y_
frames together into a single data frame with an continuous("Lines of Code", labels=percent) +
additional repo column that distinguishes the project Draw a horizontal line at 50% to remind us of the
(for brevity we've omitted the code here, but you can critical threshold:
find it on GitHub). To visualise the data we first need geom_hline(y=0.5, colour="darkgrey") +
to load two libraries: Divide the chart up into separate versions (or
library(ggplot2) facets) one for each repository:
library(scales) facet_wrap(~ repo) +
We also need a function to calculate the bus factor Annotate each facet with a red point and a label to
for a vector of cumulative_line_percent values: highlight the bus factor:
bus_factor_for_line_percent <- function(lp, critical_line_ geom_point(data=bus_factors, aes(bus_factor, critical_
percent=0.5) { line_percent), colour="red") + geom_text(data=bus_
length(lp[lp<critical_line_percent]) factors, colour="red", aes(bus_factor, critical_line_
} percent,label=bus_factor), hjust=-1) +
Then we calculate the bus factors for each projects: Provide an overall title:
bus_factors <- ddply(all_ac, .(repo), summarise, bus_ labs(title="Bus Factor for Open Source Projects") +
factor=bus_factor_for_line_percent(cumulative_line_ Then finally set the theme to minimal (this is a
percent), critical_line_percent=0.5) personal preference – check out the xkcd package for
Finally we can plot a graphic. We'll step through a familiar-looking alternative):
this line by line (the + is used to compose plots theme_minimal()
with ggplot; we have it on the end of the line to You can see in the images on these pages that
let the interpreter know we're not done with the some projects are quite safe. The Linux Kernel, for
specification). The ggplot function expects a data example, should continue on if the worst happens,
frame, then an aesthetic mapping specified with the while others such as OpenSSL are a little more
aes function. We want to plot the number of authors precarious. So do you fare any better?
against the proportion of the code base. Tshis call to
aes maps the cumulative_author_count variable to Robin Gower became interested in the statistical language of
GNU-R after looking for languages suitable for pirates. Ar.
the "x" axis and cumulative_line_percent to the "y":
www.linuxvoice.com 87
CODING RUST
RUST: GET STARTED WITH
MOZILLA'S NEW LANGUAGE
Mozilla may be famous for Firefox, but it's also given us a great language to play with.
MIKE SAUNDERS
D
oes the world really need another
programming language? We already have C,
C++, C#, Objective C, Swift, Java, Perl, Python,
WHY DO THIS? Ruby, Go and hundreds of lesser-known languages
• Expand your skills with with just a handful of users. Surely that's enough for
a new programming everyone, right? Well, the Mozilla team doesn't think
language so, and in the last few years has been working on
• Discover why Rust is the Rust, a "systems programming language that runs
hot topic among coders
blazingly fast, prevents nearly all segfaults, and
• Write memory-safe
and highly concurrent guarantees thread safety". In other words: it lets you
software write low-level software with C-like performance, but
without the problems that plague that language. Many editors support syntax highlighting for Rust, such
Rust is designed to be multi-paradigm, in that it as Vim (see https://github.com/rust-lang/rust.vim).
can be used in procedural, functional and object-
oriented programming. It doesn't allow the use println!("Hello world");
of null or dangling pointers, making it much more }
memory safe than C (see Core Tech on page 94 for You'll notice straight away that the syntax is rather
more on why these are bad), and provides a set of like C/C++, with double slashes preceding comments,
streamlined runtime libraries handling I/O, concurrent code blocks surrounded by curly brackets, and
programming and other features. Rust is a relatively parentheses used for the arguments in a function
new language, with its 1.0 release only arriving in May definition. Note here that functions are defined with
2015, but it has been in development for over five the fn keyword, and every program must include a
years. And it's well worth learning it now – at least the main() function. The exclamation mark after println
basics – as it could prove to be a major competitor for here indicates that it's a macro (it provides a friendly
C and C++ in the coming years. wrapper around the Rust runtime's own print routine).
Rust may be available in your distro's package To compile it, simply enter:
repositories, but as the language is undergoing rustc hello.rs
development it's worth getting the latest binaries from You'll see a resulting binary, hello, which you can
execute using ./hello. But if you look at the size of the
Rust is designed to be multi-paradigm, binary, you may be rather shocked: it's over 800kB! All
that for such a simple program? Well, by default the
in that it can be used in procedural, Rust compiler statically links in a good chunk of the
language's runtime, so that you can copy the binary
functional and OO programming. to an installation without Rust installed and run it
without problems. You can force the compiler to use
the project's website at www.rust-lang.org. Grab the optimisations and dynamic linking, however:
rustup.sh script and run it, eg: rustc -O C prefer-dynamic hello.rs
wget -O - https://static.rust-lang.org/rustup.sh | sh Now the binary is a much more respectable 8kB
(This uses wget to retrieve rustup.sh, spit it to in size, but if you run ldd on it you'll see that it needs
stdout, and pipe the results to a shell to execute it.) libstd-<version>.so to be installed.
After a few minutes of downloading, Rust will be
installed and you can start compiling programs using Language syntax
the rustc command. Let's kick off with a classic – Now that we can compile and execute Rust programs,
type this in and save it as hello.rs: let's look at the language's syntax and see how it
// Hello world differs from C, C++ & co:
fn main () { fn doubler (x: i32) -> i32 {
88 www.linuxvoice.com
RUST CODING
x*2 Rust's documentation is
} detailed and well written –
a major selling point for a
fn main () { fairly new language.
let a: i32 = 5;
let b;
b = doubler(a);
println!("a doubled is {}", b);
match b {
1 ... 10 => println!("From 1 to 10"),
_ => println!("Another number"),
}
} between 1 to 10 inclusive), while the underscore (_)
If you come from a C/C++ background, you may refers to everything else.
think this looks rather messy, but there's logic behind In Rust, the char variable type is actually four bytes
it. Let's kick off with the main() function: in the first and can contain any Unicode value, so the language
let line, we declare a variable, a, to be a 32-bit signed is designed out-of-the-box to handled a wide range
integer containing the value 5. We can omit the of writing scripts and special characters. Another
variable type (i32 is the default) and the initial value, useful type is the tuple, which is a list of variables with
in which case it will be zero. Note that if you declare a different types:
variable and assign it a specific value like we do with a let x = (1, 2.0, "Hello");
here, you can't change it later, so this would generate Here we have an integer, a floating
an error: point number, and a string – all in PRO TIP
let a: i32 = 5; the same tuple. These elements are If you're ready to explore more of Rust's
a = 10; immutable, and we can access them features on your own, check out the
official documentation at
By default, variables in Rust are not mutable – that like so: https://doc.rust-lang.org/book.
is, they can't be changed. You have to explicitly declare println!("{}", x.2);
them as mutable like so: This prints the third element in the
let mut a: i32 = 5; x tuple, so Hello.As in traditional arrays, which Rust
What's the point of this? Isn't it just extra typing also supports, indexing begins with zero. You can use
work? Well, yes, but it's all about writing safe tuples to return multiple values from a function:
programs. You should only make things mutable that fn switch(input: (i32, i32)) -> (i32, i32) {
absolutely have to be changed. Rust encourages you (input.1, input.0)
to be verbose, but it's all for clarity: in the above line, }
we know that a is a signed integer of exactly 32 bits,
with permission to change its value in the future. fn main() {
Next, we call our doubler function with a as the let x = (10, 50);
calling value, and b to store the result. Note the let y = switch(x);
function definition for doubler at the top of the code: println!("{}, {}", y.0, y.1);
the type of the passed parameters is specified (an }
i32), and the return value type is given after the -> Here we have a function called switch(), which
characters. You'll also notice that the only code we takes a tuple of two i32 values and stores them in
have in the function is x * 2, which isn't even followed the variable input. It also returns a tuple of two i32s.
by a semi-colon as in normal Rust code – so what's Inside this function we have a simple expression,
going on here? a tuple with the items back-to-front – that's what's
It's perfectly possible to return a value in normal returned to the calling code.
C-like fashion, but you can also achieve the same In our main() function, we create a tuple called x
result by providing an expression as the last line in containing the values 10 and 50, and a tuple called
the function, as we do here. And because it's just y, which contains the results after switch() is called.
an expression, it doesn't need the semi-colon to Then we print the results to the screen (50, 10).
terminate it. So that's a brief introduction to the syntax and
Back in the main() function, we use the println!() features of Rust – if you'd like us to go more in-depth
macro to print the result; note the variable substitution with a tutorial series, let us know!
in the {} curly brackets. Finally, we demonstrate
Rust's very useful "match" keyword, which saves a
lot of typing if you need to perform a lot of if/else Mike Saunders codes to the baroque rhythms of Bach,
Buxtehude and the original Gameboy's Tetris music.
operations. Here, 1 ... 10 refers to a range (numbers
www.linuxvoice.com 89
CODING ADA
ADA: MILITARY-GRADE
PROGRAMMING
Rock solid reliabilty, a solid safety record in embedded systems, and a cool name.
JULIET KEMP
I
n the 1970s, the US Department of Defense (DoD)
had lots of embedded computer system projects,
and over 2,000 languages being used for them.
The languages were often either obsolete or
hardware-dependent, and none of them were safely
modular. There were obvious concerns about safety
and long-term usability/maintenance (finding
maintainers for 2,000 obsolete languages is a tough
gig), not to mention the possibility of having to scrap
perfectly good hardware because the software
couldn't be safely altered. In 1975, they created a
working group to find or create a suitable
programming language to standardise on for
themselves and for the UK Ministry of Defence, with
the intention of solving the 'software crisis'. The Ada Language Mascot,
In the customary manner of government projects, by Leah Goodreau, inspired
committees, and computer language projects, it by Babbage's nickname for
Lovelace, "Lady Fairy".
took a while. In fact, it took three years and a number
of drafts before they had even reached Steelman
language requirements for their proposed language; Ada was very fully-featured, but that meant that it
but after those three years, they were also able was also quite complex, and early compilers tended
to conclude that there was no existing language to be slow. The language validation suite that was
that met them. They wanted a focus on reliability, mandated for Ada compilers made for a very reliable
maintenance, and efficiency, aimed as the language language, which was the aim; but it may also have
would be at embedded systems; and they wanted a tended to make the compilers slower, or at least
language that could be used for any of the systems slow down their improvement, as the focus when
they needed. The ideal (which they never quite developing compilers was on passing the validation
reached) would be a single language across every suite rather than on speeding up compilation or
DoD computer project there was. run-time performance. On the other hand, compiler
Four contractors were hired to develop proposals, validation, and the numerous compile-time checks
one of which would be chosen to develop fully. that compilers were mandated to perform, ensured (or
Interestingly, all of the four proposals had a basis in tried to) accuracy and reliability, clearly important in
Pascal, the Algol-like language developed in the late safety-critical systems. But slow compilation and poor
1960s as a teaching language. In 1979, the Green run-time performance can also have maintenance
team proposal was chosen and named Ada, after Ada and safety implications, making it slow to improve and
Lovelace. It had been developed by a French team at debug systems, so it's not a straightforward trade.
CII Honeywell Bull, led by Jean Ichbiah. Ada's focus on safety-critical support has made
Ada 83 was structured and strongly typed, with it popular for other projects with low risk tolerance.
built-in support for concurrency, run-time checking, Although it's less used in military situations now, it's
parallel processing, and exception handling. It also still widespread in transport (air and rail, including UK
provided task/parallel processing support, with the trains and air traffic control), space software, banking,
rendezvous mechanism allowing tasks to synchronise and medical applications.
with one another. Real-time support was part of the
tasking features. As it was designed for developing Getting started
large systems, it was also highly modular, and very Gnat is the best Ada compiler for Linux, and should be
readable and maintainable if written properly. available via your package manager. Once you've
90 www.linuxvoice.com
ADA CODING
installed it, create a file hello.adb: with Ada.Calendar, Ada.Calendar.Formatting, Ada.Text_
with Ada.Text_IO; use Ada.Text_IO; IO,
procedure Hello is Ada.Strings.Unbounded, Ada.Strings.Unbounded.
begin Text_IO;
Put_Line ("Hello World"); use Ada.Calendar, Ada.Calendar.Formatting, Ada.Text_
end Hello; IO,
Ada is designed to have a long life, and to be Ada.Strings.Unbounded, Ada.Strings.Unbounded.
maintained by people other than the original Text_IO;
programmer, so it's considered vital to make code as
readable as possible. (Anyone who's ever been faced procedure Tracking is
with an aging and incomprehensible code base will track : Unbounded_String;
understand the value of this.) So the code should be date : Time := Clock;
pretty easy to read through. begin
Ada.Text_IO is a standard package, which deals Put_Line("What are you tracking?");
(as expected) with text input/output. with imports track := Get_Line;
the package, and use is one of the ways to make Put_Line(track);
the package visible, enabling you to use procedures Put_Line(Image(Date => date));
(like Put_Line) from the package. Ada was designed New_Line;
to be highly modular and to encourage modular end Tracking;
programming, so packages are a major feature. You'll notice that we've imported a whole raft of
The main procedure can be called whatever you like, libraries this time, mostly to do with unbounded
but it's good practice to match the filename. Semi- strings and the calendar.
colons are used to end statements, and also to end a Normal Ada strings have to be exactly the length
procedure. Compile the file with gnatmake hello.adb, that they are specified to be. So if you put in too
and run it with ./hello. many characters, you'll lose some; if you put in too
few characters, your program will halt, waiting for
Tracking time the string to become full. The standard libraries Ada.
For something a bit more involved, let's try writing a Strings.Bounded and Ada.Strings.Unbounded solve
program to track how much time you've spent on a this problem. Bounded strings have a maximum size,
particular task. Set up a new project in GPS (the and are a bit faster, whereas unbounded strings have
default options should all be fine), and open a new file no maximum size and are much more flexible, but
called tracking.adb: because they're dynamically allocated are a bit slower.
We'll use unbounded strings for their flexibility.
So at the top of the procedure, we declare two
Building and running within GPS
variables, an unbounded string, and a time value. We
If you want an Ada IDE, you can open up GPS (gnat_gps), also set date, the time value, to the current system
which is used for the tutorial screenshots. clock time, thus declaring it (with : and assigning it
To build a file within GPS, first you need to add it to the
project file. Go to Project – Edit Project Properties, and
(with :=) in a single line. In the body of
choose the Main Files tab. Click add, choose the filename the procedure, we output a line and use
(eg hello.adb), and click OK. Once the file is added, go to Get_Line to get the unbounded string PRO TIP
Build > Project > hello.adb. You'll get a build window, but as input by the user. (The regular String Note that if you need to refer to Procedure
a rule the default options are fine. equivalents are Put() and Get().) A in Procedure B, Procedure A must be
You can then go to Build > Run > hello, and run it either declared before Procedure B, or GNAT
in the GPS Run window (as in the screenshot below) or in
Finally we output the input string won't be able to find it.
an external terminal. and the current time to the screen.
Image is an Ada attribute which can
be used to provide a string representation of its input,
which is handy in cases like this. We'll use more
attributes later on.
However, it would make more sense to store our
tracking data – string, date, and hours completed – in
a single entity. The Ada Record composite type, which
lets you define your own records, is useful for this:
procedure Tracking is
type Track_Record is
record
Name : Unbounded_String;
Date : Time;
Building and running a file within GPS. Hours : Integer;
end record;
www.linuxvoice.com 91
CODING ADA
use Ada.Directories, Ada.Streams.Stream_IO;
procedure Tracking is
-- ... as before ...
filehandle : Ada.Streams.Stream_IO.File_Type;
fileaccess : Ada.Streams.Stream_IO.Stream_Access;
procedure Save_Records_To_File(File : in out Ada.
Streams.Stream_IO.File_Type;
Name : in String) is
begin
Create(File, Out_File, Name);
fileaccess := Stream(File);
Track_Record'Write(fileaccess, track);
Close(File);
end Save_Records_To_File;
begin
-- as before
Save_Records_To_File(filehandle, "trackingdb");
Hello World edited in the
track : Track_Record; end Tracking;
IDE and running on the
command line. There are several ways in which you can write out
procedure Output_Track_Record(track : in Track_ binary data in Ada, but the most flexible is Stream_IO,
Record) is which also works with Unbounded_String, so we'll
begin use it here.
Put_Line(track.Name); To set up, we need a filehandle variable and an
Put_Line(Image(Date => track.Date)); access stream through which we'll access the data
Put_Line(Item => Integer'Image(track.Hours)); once the file is open. We then create a new sub-
New_Line; procedure, which has two arguments. File is the
end Output_Track_Record; filehandle, and Name is the string name of the file.
We create a new file, define fileaccess as the access
begin stream of that file, then write the record using the
Put_Line("What are you tracking?"); 'Write attribute of the Track_Record type. This will
track.Name := Get_Line; convert the Track_Record object into something that
track.Date := Clock; can be written to the stream, given the stream and
Put_Line("How many hours have you done?"); the object to write as arguments. Close the file again,
track.Hours := Integer'Value(Get_Line); and we're done. Remember to add a line to the main
Output_Track_Record(track); procedure which calls Save_Records_To_File.
end Tracking; What we can't do yet is to read the data back out
The first section just defines a new record type, with again, or to append new data to an existing file; Create
a name, the date of the last update, and the hours will overwrite the file each time. So let's add a couple
spent on this particular thing. The next section defines of functions to fix these problems:
a sub-procedure. Ada has a couple of ways in which with Ada.Directories, Ada.Streams.Stream_IO;
you can do this; see the boxout for more information. use Ada.Directories, Ada.Streams.Stream_IO;
Here, the procedure has a single input (in) value, procedure Tracking is
named track and with type Track_Record. type Track_Record is -- as before
Most of the rest of this will be familiar. The
Integer'Image and Integer'Value are ways of type Record_Array is array(1..100) of Track_Record;
translating from integer to string, and back, using records : Record_Array := Record_Array'(others =>
attributes. 'Value (you can also use Float'Value, etc) (Hours => -1,
gives the scalar value of a string, and 'Image gives a Name => To_Unbounded_
string representation of a scalar. Attributes in general String(""),
are things that can apply to different types of objects. Date => Clock));
counter : Natural := 1;
Output to file track_file : String := "trackingdb";
So far so good, but at present we don't have any way -- other variables as before
of saving this information. To fix that, we could take First up, we'll need a structure to read our data into.
the record apart, write it to a text file, and then Ada arrays are very powerful, but they're not dynamic:
reconstruct it when reading in. However, a better (and the array is of a fixed size. Array elements can be of
neater) bet is to write it out as a binary record object: any type, but they must all be the same type; and
with Ada.Directories, Ada.Streams.Stream_IO; array indexes can be of any type, not just numbers.
92 www.linuxvoice.com
ADA CODING
Subprograms: procedures and functions
Ada has two types of sub-programs. Procedures have no
return value, whereas functions do return a value. The basic
structure of both procedures and functions is exactly the
same as of the main procedure, with declarations at the
start and then a begin...end statement block.
Procedures and functions have three modes for their
input variables:
in means that the parameter will not be changed. It is
treated as a constant within the procedure.
in out means that the parameter may be changed. It is
treated as a variable within the procedure and any
changes will be permanent.
out means that the parameter will be given a value during
the procedure, and its previous value won't matter. This
gives you a way of 'returning' a value from a procedure. Name : in String) is
Look for these in the various procedures and functions in Successfully running! Two
begin
the tutorial. items stored, asking for
Create(File, Out_File, Name); another input.
fileaccess := Stream(File);
We'll use numbers here, though. for T in records'Range loop
When initialising an Ada array, you can initialise if (records(T).Hours /= -1) then
specific members of the array. You can also (since Track_Record'Write(fileaccess, records(T));
Ada 95) use the others keyword to initialise the rest end if;
of the array. Here, as there's no separate initialisation end loop;
values, others applies to the whole thing. If you don't Close(File);
do this, you don't know what will go into the array end Save_Records_To_File;
before you write into it.
Next, reading in the records: begin
procedure Read_In_Records(File : in out Ada.Streams. if(Exists(track_file)) then
Stream_IO.File_Type; Read_In_Records(filehandle, track_file);
Name : in String) is end if;
begin Save_Records_To_File(filehandle, track_file);
Open(File, In_File, Name); end Tracking;
fileaccess := Ada.Streams.Stream_IO.Stream(File); Get_New_Track is just breaking out the code that
While not End_Of_File(File) loop was in the main procedure, with the addition of a line
Track_Record'Read(fileaccess, track); to store the new record in the array. Save_Records_
Output_Track_Record(track); To_File is pretty similar to our previous single-record
records(counter) := track; code, too; we just use a for...loop...end loop; structure
counter := counter +1 ; to write out the whole array one at a time. We're still
end loop; overwriting the file, but as we've already read all the
Close(File); records into the array, we're overwriting it with all
end Read_In_Records; the available data. (To improve performance, look
It's structured much like our previous output into changing this to append only a single value.)
procedure; take a filehandle and a filename, open The 'Range attribute here does what you'd expect,
the file, and create an access stream. You can use providing the range of indices into the array (in our
While...loop...end loop; to iterate over a filestream, not case, 1–100). You might have noticed by now that
just an array or similar structure, and it's a neat way to attributes are ubiquitous in Ada and are both neat
do it, especially with the End_Of_File function. We use and powerful. Note too the if...then...end if; structure.
the Read attribute to read in records, output each one Finally, a couple of lines in the main procedure to kick
to the screen, store it in the array, and increment the the whole thing off.
counter. Don't forget to close the file! Here's the last bit: Ada is a huge and fully-featured language, so there
procedure Get_New_Track is are many more advanced aspects to explore, such as
begin the concurrency and other multithreading features, or
Put_Line ("What are you tracking?"); the more advanced OO features. It's well-documented
-- Get name, date, and hours as in main procedure (as it's ISO-standard compliant), and there is still an
earlier active community, especially around Ada 2012. As the
records(counter) := track; mascot logo says, it's time-tested, safe, and secure;
end Get_New_Track; the language for a complex world.
procedure Save_Records_To_File(File: in out Ada. Juliet Kemp is a friendly polymath, and is the author of
Apress’s Linux System Administration Recipes.
Streams.Stream_IO.File_Type;
www.linuxvoice.com 93
CORETECHNOLOGY
CORE
Valentine Sinitsyn develops
high-loaded services and
teaches students completely
unrelated subjects. He also has
a KDE developer account that
he’s never really used.
TECHNOLOGY
Prise the back off Linux and find out what really makes it tick.
Garbage Collection
Learn how high-level languages like Python reclaim memory, so
you can concentrate on your job and not memory management.
N
obody seems to enjoy cleaning up after char * get_name() { /* Implementation */ };
themselves. If you have children, or void somefunc()
remember yourself being a kid, you'd {
probably agree. Adults aren't too keen on char *name = get_name();
housekeeping activities either, as robotic vacuum printf("Your name is %s\n", name);
cleaners sales figures suggest. Programmers are also }
humans, so it's no wonder they prefer computers to The tricky part is, who should free the memory
keep the environment clean after their programs. This pointed by name when it's no longer needed? The
makes development easier, and hard-to-debug-yet- correct answer is that it's up to the get_name()
easy-to-exploit bugs less likely to occur. Of course, I'm implementation. If it allocates memory, a caller
speaking of automatic memory management and its should free() it. This may sound trivial, but real-world
thickest pillar – garbage collection. programs have more than two functions, and pointer
ownership is rather difficult to track.
Do it yourself Okay, what about this code?
The C language tutorial we ran in LV018 dealt void print_name() { /* Implementation */ };
with how memory management should work. void somefunc()
Unfortunately, things can go wrong. Take a look here: {
char *name = get_name();
frame print_name(name);
Figure 1: The objgraph module module
_main_ figure1.py:15 free(name);
(https://mg.pov.lt/objgraph)
showing two reference cycles. a }
_dict_ f_locals
refers to itself through a.self_ref; It may work just fine. Or, crash randomly if
b and c form another cycle via the dict print_name() remembers name to re-use it later. This
outer_ref attribute. 10 items situation is known as a "dangling pointer", and the
b problem is that the pointer's value doesn't say whether
a
the object is still alive.
C C These are just two problems, but hopefully
<_main_.C object at 0x7fd5c3acf250> <_main_.C object at 0x7fd5c3acf1d0>
you've got their taste. And they aren't unsolvable:
self_ref _dict_ _dict_ c for example, some C++ libraries provide so called
"guarded pointers" that can't dangle. Manual memory
dict dict management is somewhat like driving a car with
1 items 1 items
manual transmission: tedious, but fine if you know
out_ref out_ref what you are doing. Pulling the lever randomly is
almost certainly a way to get into trouble.
C
<_main_.C object at 0x7fd5c3a678d0> This analogy goes even further: just like automatic
gear, automatic memory management comes with
_dict_ a performance hit. Quite often, it also introduces
random pauses during the program's execution,
dict
1 items which is bad for time-critical code. Still, automatic
memory management makes programming easier
94 www.linuxvoice.com
CORETECHNOLOGY
and less error-prone, which is usually enough to justify
its costs. The majority of high-level languages feature
Root#1 Root#2
automatic memory management. In this Core Tech,
we'll stick to Python's implementation, as it is full-
featured yet relatively simple.
Garbage
Reference counting
The simplest form of automatic memory
management is reference counting, which the Linux
kernel uses to track which modules are safe to unload.
Garbage Garbage Garbage
The idea is simple: each object receives a counter.
Code that creates a reference to the object increments
it. When this reference vanishes, the counter is
Figure 2: Tracing GC
decremented. When it reaches zero, the object has no pointer operations; this can be optimised as well.
sample. Yellow boxes
users and is safe to delete. The real problem is reference cycles:
are roots; green ones are
Reference counting is often available out of the box. >>> a = list() objects that have been
For example, the C++ standard library includes smart >>> a.append(a) marked. Grey boxes are
pointer classes for this purpose. Python also uses Here, a holds a reference to itself. Even if it is not unmarked objects, hence
reference counting to keep track of allocated objects: reachable from outside, its "refcount" will never drop to garbage.
>>> a = 1024 zero. Now you have a memory leak; congratulations!
>>> b = a It is possible to implement automatic memory
The reference counter of a starts with 1. b adds management using reference counters only. Perl 5
another reference, as we can see: almost does this, for instance. But memory leaks
>>> import sys
>>> sys.getrefcount(a)
3
A garbage collector's goal is to identify
This returns 3, not 2, because getrefcount() also objects that are no longer reachable, and
references a through its argument. If you break the
reference through b, the result will drop by one: reclaim their memory.
>>> b = None
>>> sys.getrefcount(a) aren't good, so more often than not an elaborated
2 garbage collector (GC) is a highly welcome addition.
Note that executing del b has the same effect.
Actually, del neither deletes an object nor calls its Trace. Mark. Sweep.
destructor. It just decrements reference counter and A garbage collector's goal is to identify objects that
wipes the name: are no longer reachable, and reclaim their memory.
>>> b = a You can argue that reference counting already does
>>> del b this; it's true. In fact, it is one of the simplest garbage
>>> sys.getrefcount(a) collection algorithms. Still, it's possible to implement
2 garbage collection with no reference counting at all.
>>> b Tracing garbage collection is a good example.
NameError: name 'b' is not defined As the name suggests, it traces objects reachable
Nobody is perfect, and reference counting is from so called "roots" (global and stack variables).
no exception. At very minimum, you sacrifice the Everything else is considered garbage and deleted.
memory that the counters consume, though this isn't The process is depicted on Figure 2.
usually a big deal, as objects tend to be large. Keeping With tracing garbage collection (let's call garbage
counters updated also adds overhead to common collection GC from now on), memory isn't reclaimed
immediately when an object becomes unused:
Memory allocation instead, a dedicated routine frees it when it gets a
chance to run. Quite often, when GC starts, all other
Garbage collection and memory reclaiming are naturally operations are halted to keep the heap unmodified.
coupled to memory allocation. So how does Python gain
This is known as a "stop-the-world pause", and is
memory for its objects?
For objects larger than 512 bytes in size, Python rather harmful. More sophisticated concurrent and
simply uses the malloc() and free() functions from libc. incremental garbage collectors exist that keep pauses
Smaller objects are allocated in 256kB page-aligned at minimum. For simplicity's sake, we won't touch
chunks called "arenas". On Linux, arenas are created with them in this Core Tech.
the mmap(2) syscall as anonymous mappings to avoid
When GC starts, it visits objects reachable from
heap fragmentation (LV018). As objects are deallocated,
their arenas may become unused. In this case, memory is the roots first, and sets some flag on them. This is a
unmapped and returned to the system. "mark phase". Then it loops over *all* objects in the
heap, clearing the flag and freeing those not having it
www.linuxvoice.com 95
CORETECHNOLOGY
younger than the one being collected together. Then,
it follows object references within the generation and
Older generation Older generation decrements GC counters for each object it visits.
When it finishes, some GC counters will have non-zero
values. This means these objects are reachable from
outside, and should stay alive. So, GC moves them
(and objects they reference) to the next generation.
All other objects are unreachable. Note however
3/2 1/2 that their original reference counters were non-zero, as
they were staying alive. This means that these objects
form reference cycles (see Figure 3). By construction,
no object that's staying alive references them now,
1/0 1/0 1/0 2/0 and it's safe to destroy these cycles. The order of
destruction is undetermined – in practice, the garbage
collector just "clears" objects, breaking links between
them. This way, their original reference counters
Figure 3: An example
set. This is a "sweep phase", and the whole algorithm eventually reach zero and objects are freed.
of Python's garbage
collection process. n/m is called "mark-and-sweep GC". Python's garbage collector has three generations.
are GC refcounts before Some variations are possible. First, you can copy As usual, there are no guarantees as to when the
and after subtraction. reachable ("live") objects into one contiguous memory GC will run. Generally, it happens when the size of
Grey objects are dead, and area. This doubles memory requirements, but the youngest generation grows above a threshold
green ones will stay alive eliminates the sweep phase. Memory fragmentation (700, by default) or when a third-party C extension
in the next generation, is also reduced, but you need to spend time copying or the program itself decides. Very few programs
shown in yellow. possibly large objects. and extensions run garbage collection themselves;
Another trick is to introduce "generations". An object most often you rely on runtime to do the right thing.
that has already survived some sweeps will probably Time-critical application are notable exceptions.
stay in memory longer than others. New objects are Sometimes, they run with GC disabled (Python makes
created in the first (youngest) generation, and each this possible) and clean memory at appropriate times.
In theory, you can ditch garbage collector altogether:
just be careful not to make reference cycles in your
Python objects may have finalisers or Python code, or break them manually.
destructors, which are functions called To control the garbage collection process, Python
provides the gc module. Be careful with it, as if you
when the object is deallocated. call its functions without thinking, you can easily
screw up your program. Perhaps the most obvious
sweep promotes them to next generation until the thing you can do with the gc module is to disable
oldest one is reached. Collection usually starts at garbage collection, with the gc.disable() function.
the youngest generation, and stops there unless the Even when garbage collection is off, you can run it
system is starving for memory. This way, fewer objects manually with gc.collect([n]); n is the generation
need to be collected, resulting in smaller pauses. number and is optional. gc.collect() returns the
Now that we've looked at the basics of how number of unreachable objects it found. Play with it,
garbage collection works, we'll dissect the inner then re-enable garbage collection with gc.enable().
workings of Python's garbage collector.
Weak references
A Python's way
As you already know, Python (or, strictly speaking, Finalisers aren't the only things that a garbage collector
CPython) uses a garbage collector to free objects in should take care of. Another such thing is weak references
implemented in the weakref module.
reference cycles. Simple types like integers or strings
Weak references are like normal (strong) references,
do not produce cycles, as they don't hold references to except that they don't increase the reference counter for an
other objects. So, GC is concerned only with object they point to. This means that objects having only
containers (like tuples or lists), objects, functions and weak references are eligible for garbage collection. When
generators. Functions hold references to local and this happens, weak references don't become dangling;
instead, they return `None` when dereferenced. Weak
global variables, and can also grab outer scope
references are quite useful for caches that access objects
variables via closure. This makes them less obvious, but don't care about managing their lifetimes.
but real candidates for garbage collection. When an object having weak references is collected,
Python already has reference counting, so its GC Python generally does two things. First, it clears the weak
is naturally counter-based. Actually, each GC-tracked references. Second, it determines if it is safe to run weak
references callbacks, and executes them if possible. By
object has two counters: when Python decides it's
definition, weak references never prevent an object from
cleaning time, the original refcounts are copied to GC being collected, as destructors do sometimes.
counters. The collector then merges all generations
96 www.linuxvoice.com
CORETECHNOLOGY
There are also some tunables. gc.set_threshold()
sets GC thresholds for each of three generations, if
700 seems too much. Changing this affects collection
frequency and the amount of memory your program
consumes. gc.set_debug() makes garbage collection
verbose, which is useful when you experiment with it.
Uncollectable stuff
The previous explanation of a garbage collector's
operation implied that you can free objects in
reference cycles in an arbitrary order. This is often the
case, but there is one notable exception.
Python objects may have finalisers or destructors,
which are functions called when the object
is deallocated. In the simplest case, they are
implemented as __del__ methods:
With htop, you'll easily
class C(object): >>> del a, b
track all processes on your
def __del__(self): >>> gc.collect() # Attempt to collect Linux box. If you remember
print('I am dying...') >>> gc.garbage the days of two-panel file
Finalisers are meant to free resources like file [<__main__.C object at 0x7f785a28ce10>, <__main__.C managers, you'll feel right
handles or sockets. For instance, the pymongo. object at 0x7f785a29d090>] at home.
cursor.Cursor destructor closes the cursor on the >>> gc.garbage[0].cousin = 'Impostor'
database side. However, a plain context manager >>> gc.garbage[:] = [] # Clear references in gc.garbage
(with instance: ...) is often a better alternative to I am dying, my dear cousin <__main__.C object at
a destructor. Some tutorials may even advise you 0x7fedf7687e10>
against using finalisers anywhere in your code. So I am dying, my dear cousin Impostor
what's wrong with them? >>> gc.garbage
The problem is that finalisers may contain arbitrary []
code, which possibly references other objects. Think We reset the first object's `cousin` to break the
of the following example of two objects in a reference cycle. Now a and b refcounts become zero and both
cycle: destructors run. Note that this won't happen without
class C(object): some help from our side. A destructor that belongs to
def __del__(self): object in a cycle can in fact never run! You avoided the
print('I am dying, my dear cousin %s' % self.cousin) crash but got a resource leak.
You may now decide not to use __del__, but
a, b = C(), C() unfortunately it's only part of the story. __del__ is
a.cousin = b just a tip of an iceberg, as Python really looks for a C
b.cousin = a level destructor called tp_del. Generators implement
Suppose a is destroyed first. Now, the last reference it, among standard Python objects. If a generator
to b has gone, and it is also freed. This calls b.__ involves a try ... catch block, or in fact anything but a
del__() and bang! The interpreter crashes, because loop, and it was started, it needs finalisation. If such
b.cousin is a dangling pointer. a generator appears in a reference cycle, it won't
To prevent this from happening, Python (up to be collected. And as many popular Python network
version 3.4) refused to collect objects with finalisers. libraries (like Tornado or Twisted) rely on generators
They became "uncollectable garbage" and GC stored to make callback-based code look like sequential
them in the gc.garbage list, so you break references code, this could be a problem – but not if you're using
manually: Python 3.4 and up. Hurray for updates!
Command of the month: htop
When you want to monitor a process (including its can view the process list flat or as a tree, show and
memory usage), you'd probably use top. Everyone hide threads (including kernel ones), sort processes
knows about top. So this month we nominate by whatever column you need, filter them, and send
lesser-known yet friendlier alternative – htop. signals. Columns and meters are also adjustable, so
htop is like top but with a fancier look and more if you want to see, say, a session ID (LV019), it's easy
natural (if you ask me) key bindings. The interface to do. Bars and the process tree are updated live, and
mimics Midnight Commander, with various features if you need a quick answer on what is hogging the
mapped to `Fx` keys and listed in the bottom bar. You processor or memory, htop is here to help.
www.linuxvoice.com 97
/DEV/RANDOM/ FINAL THOUGHTS
Final thoughts, musings and reflections
Nick Veitch
was the original editor Libreboot X60 laptop
running Trisquel Screws from a
of Linux Format, a disassembled laptop
role he played until he GNU/Linux that hosts
www.libreboot.org. (ThinkPad T500), onto
got bored and went which I’m installing the
to work at Canonical Libreboot BIOS.
instead. Splitter!
AKG K 272 HD headphones
– high-quality and great
for music.
I
t is a well known fact that the world’s top
supercomputers run Linux. I haven’t
checked the list recently myself (be my
Libreboot X200, my
guest: www.top500.org) but it has been true main workstation and
for as long as people have made lists of the trusted companion, with
fastest computers in the world and I don’t Dvorak keyboard layout.
see that changing anytime soon. What
wasn’t quite so well know, until recently, is
that probably the world’s fastest distributed
process also runs on Linux. Unfortunately, it
MY LINUX SETUP
is a botnet that mainly concerns itself with
DDoS attacks. If you thought botnets were FRANCIS ROWE
the domain of less secure OSes, in general,
you are correct. The discovery of the Lead developer of FOSS BIOS replacement Libreboot, and also
malware behind XOR (https://goo.gl/ runs the www.minifree.org shop.
DB4bVN) and the attacks it is responsible
for puts it in a new league of effectiveness. What version of Linux are you I first started using OpenSUSE when I
But the more disturbing news, as security currently using? was 14. Over the years, I also
folks scramble to find wares and services to Trisquel GNU/Linux, version 7.0 LTS. gravitated towards other distros: Fedora,
market in its wake, is how it is spread. This is a fully free software Ubuntu, Debian, Puppy, Yopper, Arch and so
Basically, by brute force attacks on poorly distribution, with no binary blobs, either in on. They all have their strengths and
secured SSH servers. The weakness is not the default install or in the repos. The Free weaknesses. Over time, I settled on Debian,
per-se in the OS, but in the users (or at least Software Foundation staff use this in their which I used as my main distro for many
the OEMs). There is, at the time of writing, no offices. Richard Stallman also uses it. years. Since early 2013, I have used Trisquel,
patch for ‘stupid’. Many of the compromised one of the distributions that are fully
devices probably fit into the ‘appliance’ And what desktop are you using at endorsed by the Free Software Foundation,
category, where some embedded Linux has the moment? and based on Ubuntu so it still has the
been shovelled on to network facing devices A highly tweaked Gnome “fallback” Debian base that I’m familiar with.
with little thought about robust security. desktop, with 3D effects re-enabled.
Many users may not even be aware their This is a very practical version of Gnome, What Free Software/open source
appliance even has an active SSH server, let which Trisquel includes. I’ve further modified can’t you live without?
alone how to protect it. I’m not suggesting the one that I use, so that it more closely Libreboot! I won’t use any system
that patching such devices, adopting better resembles the Gnome 2.x desktop that I’m unless it has a fully free BIOS. I’m also
practices and warning users is a bad idea, more comfortable with. I’ve also used Xfce the lead developer of Libreboot.
but it won’t fix the problem. Cloud-based and Openbox in the past, but I find Gnome to
filtering and other DDoS mitigation may save be highly practical, with some tweaking. What do other people love but you
the most prepared sites, but the XOR botnet can’t get on with?
is likely to continue to be as effective as its What was the first Linux setup you KDE. It just sucks.
supercomputer cousins. ever used?
98 www.linuxvoice.com
CC BY-SA 4.0