Plaintext
PROUDLY INDEPENDENT SINCE 2013
SCRIPTING
LANGUAGES
JavaScript
Python, Perl –
which is right
March 2016 www.linuxvoice.com
ENCRYPT
for you?
EVERYTHING
The complete guide to keeping your data
private – no matter who’s looking
CONTROL
A MARIADB
Cover image: iStock
DATABASE
OPERATION
Build a practice
cadaver for home
surgery fun
THE SNOOPER’S YOUR BEST
CHARTER DESKTOP EYE
CANDY p22
Why it’s an utterly
rubbish idea 31 PAGES OF TUTORIALS
JOIN US NOW FIAT LUX
March 2016 £5.99 Printe d in the UK
LESLIE HAWTHORN PHILIPS HUE
On community, human Control the lighting
nature, leadership in your hollowed-out
and the importance volcano base with
of failing properly Linux and Python
FOG › VLC › SQUID › WINE › TAILS › SED & MORE!
COMING SOON FROM THE TEAM AT LINUX VOICE
*
Beep Beep Yarr!
An adventure story for young minds – featuring pirates,
robots and computer programming.
Order from
25 Jan on
Kickstarter!
www.linuxvoice.com/book
*With your help!
ISSUE 24 WELCOME
TWO YEARS OF AWESOME
The March issue What’s hot in LV#024
ANDREW GREGORY
Most of us don’t have the time to
go through bureaucratic edicts to
check their sanity. But this is
what Ben’s done with the UK’s
‘Snoopers’ Charter’ and the
ineptitude he’s found is stagging.
GRAHAM MORRISON p28
A free software advocate
and writer since the late BEN EVERARD
1990s, Graham is a lapsed As I’m in the process of
KDE contributor and author automating my home, our tutorial
of the Meeq MIDI step on controlling Philips Hue
sequencer. lightbulbs with Linux comes at
T
exactly the right time. Forget
his is our 24th issue. That means it’s been two years since Netflix and chill – UNIX and chill!
we launched, exactly as promised, after a successful p80
crowdfunding campaign. It’s a cliché and too often repeated,
but we couldn’t have done this without you. Without your backing we MIKE SAUNDERS
simply wouldn’t be here. I know of no other magazine, anywhere, I loved playing the board game
that has been able to do this, let alone one that gives old issues away ‘Operation’ when I was a lad, and
for free (even for commercial use!) and shares in any profits. it’s great to see our own DIY
These are ideas that run totally against traditional publishing Operation game is way better
wisdom, and once again, it’s the Free Software, Linux and open than the modern version of the
source communities that have made the impossible possible. It’s same game. Upgrades!
p72
you that has made this possible. So, thank you. All of us here at Linux
Voice are incredibly grateful.
Graham Morrison
Editor, Linux Voice
(ps we’re attempting to crowdfund a book that will teach children
programming via pirates and robots – see our advert on the left.
THE LINUX VOICE TEAM
Linux Voice is different. SUBSCRIBE
Linux Voice is special.
Editor Graham Morrison
Here’s why…
ON PAGE 56
graham@linuxvoice.com
Deputy editor Andrew Gregory
1 At the end of each financial year we’ll
andrew@linuxvoice.com
give 50% of our profits to a selection of
Technical editor Ben Everard organisations that support free
ben@linuxvoice.com software, decided by a vote among our
Editor at large Mike Saunders readers (that’s you).
mike@linuxvoice.com
Games editor Michel Loubet-Jambert 2 No later than nine months after first
michel@linuxvoice.com publication, we will relicense all of our
Creative director Stacey Black content under the Creative Commons
stacey@linuxvoice.com CC-BY-SA licence, so that old content
can still be useful, and can live on even
Malign puppetmaster Nick Veitch
after the magazine has come off the
nick@linuxvoice.com
shelves
Editorial contributors:
Mark Crutch, Marco Fioretti, 3 We’re a small company, so we don’t
Juliet Kemp, Vincent Mealing, Simon have a board of directors or a bunch of
Phipps, Les Pounder, Mayank Sharma, shareholders in the City of London to
Valentine Sinitsyn. keep happy. The only people that matter
to us are the readers.
www.linuxvoice.com 3
CONTENTS ISSUE 24 MARCH 2016
Contents
Welcome to the 24th issue of Linux Voice. Blimey, two years eh?
Regulars Cover Feature
News 06
The PlayStation 4 has been hacked to run
Linux, the Dutch government has donated 14
€500,000 to OpenSSL, and Apple has open
sourced its Swift language.
Distrohopper 08
Featuring a genuine advancement in Linux
distros – Solus 1.0. You’re going to hear a
lot more about this new Linux flavour.
Speak your brains 10
Is Microsoft still the enemy? Should your
ENCRYPT
fridge watch your waistline for you? And
what’s Zen for Slackware?
EVERYTHING
Subscribe! 56/93
Save money, get the magazine delivered to
your door and get access to 24 issues of
Linux Voice, in lovely DRM-free PDFs.
FOSSPicks 58
Keep the man out of your personal emails and off your hard drive with our
Wild picks, purse-seined off the cold in-depth guide to data encryption. Information wants to be private!
Icelandic waters, battered and served with
Maris Piper chips. Fresh, and also Free.
Interview Feature
Core Tech 94
DNS is more than just the addressing
system that tells the internet where you are
– it’s the key to understanding what’s going 34 22
on in cyberspace.
Geek Desktop 98
Andrew Gregory, permanent newbie of
the Linux Voice team, shows off his repro
mahogany furniture.
Leslie Hawthorn Desktop showcase
SUBSCRIBE Fail properly, then get up and carry on being Send us your best-looking Linux desktops, so we can drool
ON PAGE 56 awesome again. Cheers Leslie! over them – and print them in the magazine.
FAQ Group Test
CHIP 32 Software licences 50
It’s got a rubbish, un-Googleable If you’re ready to set your code
name, but this $9 computer is free in the wild, think about what
absolutely worth knowing about. licence you’re going to use.
Luxembourgeoise
LUGS p12
4 www.linuxvoice.com
ISSUE 24 MARCH 2016 CONTENTS
Feature Tutorials
28
Fog 66
Clone machines by transmitting entire hard
drives over a network.
Inside the Snoopers’ Charter Squid 68
Control your web traffic using a proxy server
Why the UK Government’s plans to spy on its subjects is misguided, muddled and mad. to save on precious bandwidth.
Reviews
42
Ardour 4.6 Raspberry Pi
Bring computing into the physical world
72
The leading digital audio workstation for with some tin foil and a cardboard box.
Linux has several new features, plugins
and other improvements to help you MariaDB 76
unleash your inner Visconti/Ronson/ Data, data everywhere, and you can control
Rogers/Stock/Aitken/Waterman. it all with a powerful open source database.
Tails 2.0 43 Wine 1.8 44 NodeMCU 45 Philips Hue 80
If you want privacy the easy way, Remember, Wine is not an Single-board computing with Control the lights in your smart dwelling
here’s the Tor network, with all the emulator. It just smells, looks and built-in Wi-Fi for only $9? Shut up with Python and these hackable bulbs.
hard work done for you. tastes very much like one. and TAKE MY MONEY!
Coding
Sed 84
More than just a text processing language –
infact, we’ve turned it into a web server.
Gaming on Linux 46 Books 48
Turn off your mind, relax and float downstream. Or Make your users awesome by going beyond legacy Scripting languages 88
kill things, shoot big guns and put loads of salt on code and reacting to Java. Or something like that – Python, JavaScript and Perl – the ins, the
the popcorn to sell more soft drinks. you’ll never know unless you turn to p48. outs, and the what’s that squiggle doings?
www.linuxvoice.com 5
NEWS ANALYSIS
NEWSANALYSIS
The Linux Voice view on what’s going on in the world of Free Software.
Opinion
Remembering Ian
Without Ian Murdock, Linux would be nowhere near the success story it is today.
Simon Phipps
is ex-president of the
Open Source Initiative
and a board member
of the Open Rights
Group and of Open
Source for America.
A
long with my former colleagues at
Sun Microsystems, I was horrified
at the start of the year to hear of the
death of Ian Murdock. There have been
plenty of other comments written about him,
so you will be aware of his work establishing
Debian, and maybe of his own company
(Progeny) and his work at what would
become the Linux Foundation. While you’ll
remember Ian as the spirit and name behind directions of their previous management. He discussed and addressed in advance is the
the Debian project, I got to know him in 2004 showed determination and vision, and used key legacy Ian leaves us.
when he joined Sun Microsystems. I was the his experience at the convergence point of We take it for granted today that we can
head of open source activities at Sun, while business and technology to talk round both simply add a package to our operating
Ian was hired a little later to head up the the engineering genius and the pointy- system with a single command and without
transformation of Solaris into an open headed boss. destroying the system or spending the rest
source operating system. The key vector from Debian that was of the day resolving dependencies and
introduced to OpenSolaris was the need for incompatibilities, and we are extending that
Linux for the people the software to be delivered in the form of concept further into containers and cloud
The outcomes from each of our jobs are installable packages from an open deployment. But when Ian started his Linux
stories for another time, but I was struck by repository, rather than the previous vision of distribution, that was not the received
the quiet force Ian brought to his role. Amid a tightly-controlled single source for wisdom, and I credit him with proving it was
a good deal of scepticism and hostility – componentry. crucial and showing how to lead a
after all, Ian’s creation was a driving force community to make it real.
behind the demise of Solaris – Ian was able The importance of packages More than that, Ian was also a great friend
to shape the work that the OpenSolaris team That vision of an operating system built to many of us. While his history with Linux
was doing, driving them in directions far from packages with their dependencies may have aroused suspicions in the team at
more likely to result in community automatically resolved and with the legal a traditionally proprietary software vendor,
engagement than the instincts and and social issues related to each element we all quickly realised that Ian was a warm,
kind and complex person who we enjoyed
We take it for granted today that we can being with.
To lose him so early is a tragedy; while I
simply add a package to our operating have the greatest respect for his technical
system with a single command legacy, what I will miss in the future is the
opportunity for his friendship.
6 www.linuxvoice.com
ANALYSIS NEWS
Kernel • Swift • IPv6 • LibreOffice • Ian Murdock • PlayStation 4
CATCHUP Summarised: the biggest news
stories from the last month
Linux kernel 4.4 released Debian Founder Ian PlayStation 4 gets hacked
1 To ring in the new year, we 2 Murdock passes away 3 to run Linux
have a shiny new kernel Just after Christmas 2015, It’s been a while coming, but
release. Linus Torvalds announced Ian Murdock, founder of the Debian finally an exploit for Sony’s console
kernel 4.4 on 10 January 2016 with GNU/Linux distribution, died in San has been developed to enable low-level
a boatload of new features: support Francisco at the age of 42. The cause of access to the hardware, and therefore
for direct I/O and asynchronous I/O death is as yet unknown: hours before the ability to run Linux on the machine.
in the loop block device, 3D support he had been threatening suicide on http://tinyurl.com/zp3g5bn
in the virtual GPU driver (for hardware Twitter after alleged mistreatment by
accelerated graphics in virtualisation police forces. According to the police,
guests), support for open-channel he had been detailed for attempting to
SSDs, and many improvements in break in to an apartment while
drivers, filesystems and memory drunk. It’s a sad loss for the Linux
management. See here for details: community – for a look back on Ian’s
http://kernelnewbies.org/Linux_4.4 work, see Simon’s words left and p9.
Dutch government IPv6 celebrates 20th Apple’s Swift language
4 supports encryption 5 birthday with 10% usage 6 comes to Linux
At at time when many Version 4 of the internet Originally unveiled at Apple’s
governments are eager to sneak protocol has done us well over the 2014 Worldwide Developer Conference,
backdoors into encryption software, decades, but its mere 4.3 billion IP Swift is a programming language
ostensibly to “protect us”, the Dutch are addresses is becoming a severe created for iOS, OS X and the Apple
taking a different approach. The powers limitation with the Internet of Things Watch. It was designed to be more
that be there have stated that “it is coming into full flow. IPv6 offers vastly concise and safe than Objective C,
currently not appropriate to adopt more addresses (and many other which the company had been using for
restrictive legal measures against the features) but has seen slow adoption. many years. Now the language is open
development, availability and use of Still, as of December 2015 it is now 20 source, and can be used to develop
encryption within the Netherlands”. And years old and has reached 10% software on Linux as well as Apple’s
on top of that, the government has adoption according to Google. Is this the proprietary platforms:
approved a €500,000 grant to the start of a widespread switchover? The https://developer.apple.com/swift/
OpenSSL project. next 12 months will tell… blog/?id=34
Mozilla stops working on LibreOffice Online joins
7 Firefox OS smartphones 8 up with OwnCloud
This isn’t entirely unsurprising Collabora, the company
news, given how utterly dominant iOS working on an online version of the
and Android are in the mobile space, LibreOffice suite, has teamed up with
but it’s a shame nonetheless. The OwnCloud to create a virtual machine
Mozilla Foundation has announced test image that combines both projects.
that it will stop developing and selling This lets you share, sync and manage
smartphones with Firefox OS, leaving your files in OwnCloud, and also
the fledgling platform a possible future edit them from your web browser in
on smart TVs and other devices. Which LibreOffice. It’s early days, but it shows
poses the question: is there room in the great potential and could be a big
smartphone market for another OS? challenger to Microsoft Office 365.
Can anyone else break the duopoly that https://owncloud.org/blog/libreoffice-
iOS and Android enjoy? online-has-arrived-in-owncloud/
www.linuxvoice.com 7
DISTROHOPPER LINUX DISTROS
DISTROHOPPER
What’s hot and happening in the world of Linux distros (and BSD!).
Solus 1.0
Featuring the Budgie desktop.
M
ost “new” Linux distributions that
come onto the scene are based
on other distros – standing on
the shoulders of giants, if you will. This
makes a lot of sense in most cases, but
sometimes it’s good to start completely
from scratch and build a distro from the
ground up. That’s what the developers of
Solus have done, and now they’ve finally got
version 1.0 out of the door.
Codenamed “Shannon” (after the longest
river in Ireland), Solus 1.0 aims to be an
attractive and user-friendly desktop distro
that’s focused on the x86-64 architecture.
One of Solus’ most notable features is
Budgie, a custom desktop environment
that’s built on top of GTK and Gnome. Budgie Raven (on the right-hand side) is the Budgie desktop’s panel containing applets and notifications.
is designed with accessibility in mind
(especially for visually impaired users). Given that there are so many top-quality vision for their distro, and have paid plenty of
One feature unique to Solus is its package Linux distros out there, we’re often sceptical attention to presentation and marketing,
manager, called eopkg. This supports the when we see new ones being created from something lacking from so many smaller-
usual functionality of adding, removing and scratch, as they tend to exhibit a lot of “not name projects.
searching for packages, and these packages invented here” syndrome – ie being built for We wish the project luck. To try out Solus
are neatly organised into categories that the sake of it, rather than to fix any particular and read more about it, visit the distro’s main
show what they provide). problem. But the Solus team have a solid website at www.solus-project.com.
OpenSUSE Li-f-e 42.1
Education distro with long-term support.
R
elease numbering and naming in the What makes Li-f-e shine for education use
Linux distro world is often… creative, is the software selection. Li-f-e includes a
to put it mildly. Not only did range of packages designed for learning, Why should schools pay for Windows when
OpenSUSE recently make the giant leap such as the Parley vocabulary trainer and there are better, cheaper alternatives?
from version 13.2 to 42.1, now a bunch of Marble virtual globe.
developers have created a new spin-off Some of the other education-oriented Linux distros such as Li-f-e are much safer
focused on education and schools. packages include GCompris, a set of learning than Windows and provide boatloads of
OpenSUSE Li-f-e 42.1 (the Li-f-e means games geared towards young children, and great software for kids out of the box, so we
“Linux for education”) is described as “the Little Wizard, a kid-friendly introduction to the welcome attempts by the major distro
only enterprise-grade long-term supported concepts of programming. Then there’s vendors to get involved. To find out more
Linux distribution for education”, and it’s a TuxPaint, iGNUit (a flash card tool), and download the release, check out
live DVD/USB image that can also be gElemental (a periodic table viewer) and https://lizards.opensuse.org/2015/12/21/
installed to a hard drive. Stellarium (for exploring the stars). announcing-li-f-e-42-1.
8 www.linuxvoice.com
LINUX DISTROS DISTROHOPPER
News from the *BSD camps
What’s going on in the world of FreeBSD, NetBSD and OpenBSD.
H
ow small can a BSD be? Well, if you
take the (rather ancient) source
code to 2.11 BSD, update it for
modern compilers and hardware, and
squeeze it down a bit more, you can get it
running on a device with just 128k of RAM.
That’s a full BSD operating system with
memory protection, multitasking and POSIX
compatibility. All of this is thanks to the work
of the RetroBSD project (www.retrobsd.org),
which is targeting microcontrollers such as
the Microchip PIC32.
This might seem like a rather pointless
task in this day and age, but it’s quite the
opposite – consider how much work is
going into embedded devices, especially as While DragonFly BSD is often overshadowed by FreeBSD, it has plenty of unique features to
the much-lauded “Internet of Things” starts warrant trying it out.
to take off.
Meanwhile, DragonFly BSD saw a new different approach to FreeBSD in dealing and checksums to ensure data integrity.
release in December: 4.4. For those who’ve with some key architectural issues, such as Version 4.4 of the operating system includes
never heard of it before, DragonFly is a fork threading and symmetric multiprocessing better support for Intel and Radeon graphics
of FreeBSD that came to life in June 2003 (SMP). (thanks to drivers from the Linux kernel),
after a fall-out among developers. Its chief Although DragonFly is still a much smaller improved CPU power saving settings, and a
coder is Matt Dillon, who some long-time project than FreeBSD, it has pioneered plenty vastly enhanced regular expression library.
hackers may remember from his work on of technologies such as the 64-bit HAMMER For the full release notes and links to
the Amiga (such as the DICE C compiler). filesystem featuring infinite NFS-exportable downloads, visit the project’s website:
With DragonFly BSD, Dillon wanted to take a snapshots, configurable history retention, www.dragonflybsd.org/release44.
In memory of Ian Murdock (1973–2015)
Anyone can start a Linux distribution – but few can make it a long-term
success. Back in the early 1990s, as the GNU/Linux combination was still
very much a baby, the only ‘distributions’ around were small, hobbyist
hackish projects for getting the operating system onto your hard drive one
way or the other. But Purdue University student Ian Murdock saw the
chance to create a new kind of distro: one that focused on the community
and spirit of Free Software. As he wrote in the Debian Manifesto in 1993:
“Debian Linux is a brand-new kind of Linux distribution. Rather than
being developed by one isolated individual or group, as other distributions
of Linux have been developed in the past, Debian is being developed
openly in the spirit of Linux and GNU. The primary purpose of the Debian
project is to finally create a distribution that lives up to the Linux name.
Debian is being carefully and conscientiously put together and will be
maintained and supported with similar care.”
Debian was named after Ian and his then-partner Debra, and the
distribution picked up support from the Free Software Foundation early
on. Since then it has become an enormous success, not just as a distro in
its own right, but as the basis for Ubuntu, Mint and many other derivatives.
Debian’s consistent and stringent focus on democracy, community,
engineering and freedom has produced a distro that millions now rely on
for their day-to-day work.
So thanks, Ian, for having the foresight and skills to start such a
monumental project. And although everyone will remember you for
Debian, thanks for your additional work in the Free Standards Group, Linux
Standards Base and Linux Foundation. The world of Free Software is so
much stronger due to your efforts. Ian Murdock started Debian, arguably the most important Linux distro ever
(photo: Ilya Schurov, CC-BY-SA, www.flickr.com/photos/39112057@N00).
www.linuxvoice.com 9
MAIL YOUR LETTERS
YOUR LETTERS
Got an idea for the magazine? Or a great discovery? Email us: letters@linuxvoice.com
HALLOWEEN IS OVER
LV issue 023 contained, as have prior numbers, many jabs
at Microsoft as the natural enemy of the Free Software
believer. It’s time to accept that the world has changed.
STAR Like many among your staff and readers, I remember
LETTER that period when the infamous Halloween memos were
leaked, and we realised joyfully that the Free Software
movement was big enough to concern the biggest
software company in the world.
I remember this not because it was recent, but because
I am old: this happened in 1998. Large companies like
Microsoft can be slow to change, so it is right that we
remain sceptical of their intentions with Free and open
source software, but we need to remember that if we
define our movement as Anti-Microsoft, it will live or die by
their fortunes alone.
While we jab at Azure for their plush Tux swag, Apple
has become one of the largest companies on the planet. It
has done this with its proprietary iPhone and iOS
platforms, which lock in more first-party applications than
1990s Windows did when the antitrust cases started
flying. You can download alternatives from its store (and
its store alone), but the terms of business on that store
prohibit copyleft software. The downloads obtained by
Apple’s users are restricted by DRM to particular Apple
accounts. Should we move on from our obsession with Microsoft, or is it
Meanwhile, Apple co-opts open source projects like useful to have such an incompetent enemy?
Clang and LLVM to replace successful Free Software
components like GCC. How does the availability of a movement, then we will not win. Rather we will become
cuddly Tux with Microsoft branding stack up to these irrelevant at the same time as our nemesis in Redmond.
actions in respect to the FSF’s four freedoms? You may think that Steve Jobs is an unlikely role model
We celebrate Google for popularising the Linux kernel for someone in my position, but I will end by paraphrasing
through its Android mobile OS, and companies like it, his statement on his return to Apple. We need to get out of
including Facebook and Twitter, for their contributions to the mindset that for the Four Freedoms to win, Microsoft
open source software. However, these companies thrive has to lose.
by providing proprietary services from their own server Graham Lee
farms. None has embraced the AGPL, a licence that
extends freedom to remote users of a hosted service [and Andrew says: I had never stopped to consider
which Mike talks about in this issue’s Group Test on page this, but what you say makes 100% sense. In
50]. Is it meaningful to have the freedom to use a browser practice though, for most people Microsoft is still
or a mobile device for any purpose, if the available the embodiment of proprietary software. Apple
purposes involve using non-free services? is arguably a more serious threat, but Microsoft
So yes, Microsoft is still important, and its proprietary keeps shooting itself in the foot, so it’s an easier
Windows and Office products are still huge obstacles to target for us. Apple at least makes a lot of good
the freedom of computer users everywhere. On the other products along with its egregious attitudes towards
hand, Microsoft is no longer the headline company compatibility, planned obsolescence and forced
defining the computing landscape for many people. If the upgrades; Microsoft seems to be successful only by
Free Software movement is the “say no to Microsoft” abusing its market position.
10
YOUR LETTERS MAIL
INTERNET OF FOOD
My idea is to make dumb fridges smarter by creating an
inventory management gadget from a Raspberry Pi or
Arduino, a barcode scanner and a display.
People waste a lot of money because they throw away
food that’s gone off. I think in most cases they forget
about the expiry date of food they put into the fridge.
That’s why I think a little computer could help out, if it
could keep track of expiration dates of various foods and
issued alerts before a configured number of days.
I’ve found a post about this idea, but I think the work
was either never done or never published. See
https://www.raspberrypi.org/forums/viewtopic.
In the internet
php?f=41&t=50916. all food in my house goes in my belly. However,
of things, your
So I would like to see a tutorial about something like this project does sound like a far more worthwhile healthcare
that in the magazine. project than the ludicrous sugar app that’s recently provider knows
Géza Búza been developed at UK taxpayers’ expense. We’ll see how much sugar
if Ben wants to give it a go in his Internet-of-Things you’re consuming.
Andrew says: I would have no interest in this, as robo-house.
MINIMALISM
I was rather taken aback to see your reference (February Frank Bell
2016 issue, page 8) to Slackware as a “fairly minimalist”
distro. Slackware comes with much more out-of-the-box Graham says: The minimalism to which
capability that most distros. In addition to a full KDE we referred was to do with the way it treats
desktop and application suite, six desktop environments, its packages. Rather than adding bits and
multiple media players, and almost every text editor bobs, renaming features and fiddling with
known to geekdom, Slackware includes the complete configuration files (like some distros do to the
LAMP stack, the OpenSSH server, CUPS server, Samba software in their repositories) Slackware
server, all the libraries needed for compiling from sources, keeps them clean, unadorned and as the
and other functionality that, with most distros, requires developer intended. One might almost
obtaining additional packages after the initial install. say that this approach to packaging is
Slackware may be many things, but “minimalist” is not minimalist, but ‘elegantly simple’ fits the
one. Let me suggest instead “elegantly simple.” bill much better.
www.linuxvoice.com 11
LUGS ON TOUR LINUXBIERWANDERUNG
LUGS ON TOUR
The 2015 LinuxBierWanderung
Jason Irwin reports from an event that combines three of our favourite things.
T
he LinuxBierWanderung is
an “unconference” with no
fixed fee, just a voluntary
donation towards costs. There’s no
central organisation beyond those
who have taken it upon themselves
to arrange that year’s event. The
three components mesh together
to provide an event with wide-
ranging scope that suits the geek,
the family and even the dog. Yes,
delegates from various species
were in attendance.
Most delegates to this year’s
LinuxBierWanderung stayed in
‘Camping Kaul’, which offered
excellent facilties and even laid on
a Letzebuergian three-course meal
on Friday that was simply superb.
Camping Kaul also provided a
Wiltz is a charming town
decent Wi-Fi service over the site, hands-on sessions with “Microtek” exceptionally popular as not only
in the Ardennes where,
this meant there was no need for a routers (including how to break through a quirk of was the beer of good quality, but
rustled-up radio link back to the hall everything by creating packet geography, the direction there was plenty of it!
as in previous years. storm) and a detailed run-down on of travel between two There was the traditional “Pot
how the hall infrastructure had points is always uphill. Luck” where delegates used the
Linux been installed and configured after well appointed kitchen to lay on
Talks this year covered everything it had been lifted off the roof of a veritable banquet of dishes from
from designing for extreme trusty Land Rover. their home nations.
performance, the legalities and Below left: Iggy Pup Usually a “Pub of the Day” is
and Alice Pooper
intricacies of professional drone Bier appointed, but this year it began as
contemplate BRAAS
flying, computer vision with a This year’s organisers had arranged more of a “Restaurant of the Day”,
(Belly-Rubs As A
Lego-sorting Lego-bot and all the a hot-line to the local brewery, Service). and over the course of the week
way to packing for multi-day Simon, which was always prepared Below right: A delegates mostly elected to simply
adventures with less than 5kg of to slake the thirst of the delegates. hexacopter on display – hang out at the hall to chat, play
equipment. There were also This meant the in-hall bar proved assemble before use. games and (of course) do their very
best to drain Simon.
Wanderung
The final component in this triptych
are the walks and other outdoors
activities. This year took in the
stunning scenery around Lac Haute
Sûre, as well as the historic towns
of Esch-sur-Sûre and Bastogne.
The 2016 LBW will be held in
Laxley, Isle of Man from 30 July to 7
August. See you there!
12 www.linuxvoice.com
FEATURE ENCRYPT EVERYTHING
ENCRYPT
EVERYTHING
Keep your data secure, safe and private, with a little help from Linux
and Graham Morrison.
E
ncryption isn’t just important: it’s vital. It’s as of knowing when software is sending information to
vital as free speech. It enables free speech. Not a third party, or when a data vulnerability hasn’t been
just for people struggling in places where their patched, or whether the encryption is as strong as
voices can’t be heard, but for people who want to keep its developer says it is. Without open source, there’s
their voices private. It’s also vital for security, keeping no peer review, transparency or even accountability.
your data locked when even the data itself is stolen. But open source encryption is still a minefield of
It’s vital for online commerce, banking, mobile phones technology, terminology and complexity, diminishing
and digital media, and it’s vital for almost everything its effectiveness and veracity. We should all be able
else we do in the 21st century. Without encryption, we to use encryption without any specialist knowledge
wouldn’t have the internet revolution. or training, and that’s our target for the this feature –
But the only way to trust encryption is through Linux demystifying what encryption is all about so we can
and open source. Without open source there’s no way all take advantage and put these ideas into practice.
14 www.linuxvoice.com
ENCRYPT EVERYTHING FEATURE
KEY CONCEPTS
Never again get your private keys muddled with your public keys.
T
here are a couple of reasons
for the aura of complexity that
hangs over encryption. The first
is that the complicated maths behind
encryption. But unlike the complex
management systems that control your
car, for example, in encryption those
systems aren’t always hidden, exposing
the user to ideas and concepts that
aren’t necessary for its use.
Another reason for this complexity
is the difficulty in creating a system
that effectively hides these parts while
maintaining trust. Without trust, there’s
no encryption. So, until some clever
startup solves the problem of creating
easy encryption that everyone can
trust without requiring any specific
knowledge, it’s worth taking a little
time to understand some of the key
It was Whitfield Diffie and Martin Hellman’s hugely influential 1976 paper on key
concepts behind encryption, and how exchange that led to the proliferation of asymmetric encryption algorithms.
you can make it work for you.
Encryption in three terms between two types: asymmetric and Public Key/Private Key These terms
Encryption This is the idea that you can symmetric. Asymmetric algorithms often cause confusion. It may be
encode something such that it can only are perhaps better known, thanks because most of us expect encryption
be decoded by someone holding a mostly to SSH and GnuPG/OpenPGP. to behave the way doors do where
specific decoding key, an idea that has These use what’s known as a private/ the same key is used to both lock and
been around for thousands of years. public key system implemented by unlock the door. This is the equivalent
This is the art of cryptography, the DSA algorithm, although this can of a symmetric algorithm (see above),
practised by everyone from the ancient be changed. The most important but it’s not the only case. Asymmetric
Egyptians to the Third Reich. characteristic of this algorithm is that algorithms allow data to be decrypted
But it’s the mathematical prowess of one key can’t be derived from the other, with a key that’s different to the key
computers that have defined the and because the private and public used to encrypt the data, and vice-
versa, to use the key to encrypt a
Asymmetric algorithms allow data to message that can only be decrypted
by another. Both these keys could be
be decrypted with a key that’s different secret, but the revolution came when
to the key used to encrypt the data one of the keys was allowed to be
pubic. This meant you didn’t need a
secret exchange of keys to be able to
modern era of cryptography, replacing keys are different, the algorithm is decrypt a message. You could grab just
initiate hieroglyphics or positional rotors ‘asymmetric’. the public key. And you could use the
with a complex series of substitution Symmetric encryption uses the same public key to send a message, or more
and permutation processes (depending key to encode and decode the data. The importantly your own public key, to
on the algorithm) that aim to make the best known use of a symmetric system someone you knew held the private key.
translation of values from ‘plaintext’ to is HTTPS, used to secure the web via
‘ciphertext’ and back to ‘plaintext’ as either the Transport Layer Security or
difficult and as provably mathematically the Secure Sockets Layer. Although a
rigorous as possible. certified public key is used as part of
Algorithm This is the part that turns the negotiation process, they keys for
plaintext into ciphertext and back. the communication itself are generated
The algorithm will depend on how uniquely for each connection. This
you want your encryption to be used. is because symmetric encryption is
Mostly, these algorithms are split simpler and faster to implement.
www.linuxvoice.com 15
FEATURE ENCRYPT EVERYTHING
GNU PRIVACY GUARD, AKA GNUPG
Start with the best known technology for chatting as securely as in your living room.
F
or Linux users and open source difficult subject. For that reason, we’d
enthusiasts, GnuPG is the recommend starting your encryption
default choice for nearly all adventure with a GUI front end that
encryption duties. It’s capable of both enables you to point and click your way
symmetric and asymmetric encryption, through the configuration – that way
and while it’s often used to encrypt local you’ll always have a visual overview of
data, you’ll mostly find it being used to your settings.
encrypt communication channels. Of course, there’s a huge variety to
GnuPG is compliant with OpenPGP, choose between, all in various stages of
the encryption standard derived from development, using Gnome, KDE and While the command line version of GnuPG
Phil Zimmermann’s work on the original lots of other different environments. is relatively straightforward to use, we’d
PGP. That means GnuPG works with But because they’re all using GnuPG, recommend beginners use a GUI at first.
any OpenPG-compliant application – and placing their key files and
including email clients such as KMail, configurations into the same .gnupg You can do this by meeting in person by
Evolution and Sylpheed, password folder, you’ll be able to switch between accepting a key over a network such as
managers and chat applications such them without breaking compatibility. email, OwnCloud, or file transfer.
as Gabber, and compatible proprietary In order to have ultimate trust in
software too. Graphical tools those networked methods, you’ll need
GnuPG support is also built into both Both Gnome’s Seahorse and KDE’s KGpg to verify the key that was received
Gnome (via its keyring manager, are excellent applications, but we’d was the key transmitted and from the
recommend GPA, because it’s as close person you know. This is called a ‘web
GnuPG support is built in to a default user interface as you can of trust’, and while HTTPS solves this
get. Standard applications will detect with certificates signed by an authority,
to Gnome and KDE, so you your configuration and use your keys personal exchanges are different. The
may be using it already automatically, asking for your
passphrase when needed. Some,
simplest way is to check via a phone
call, verifying the short fingerprint that
including most email clients, will also be accompanies any public key. If the
Seahorse) and KDE’s KWallet, which able to import a contact’s public key. fingerprint is the same then you have
means you may already be using it But you can always go back and check the same keys and you can encrypt
without realising it. GPA to see what’s been added and messages to your contact safe in
The default way of interacting with whether the configuration is working. the knowledge that only they can
GnuPG is via the command line, where After you’ve created your secure decrypt the message. Similarly, if you
the minutiae of every feature and key pair, you’re capable of secure and get a message that’s signed with the
component is best exploited. But the trusted communication with your a private key that the public key can
command line can add to the contacts. For this to work, you’ll also decrypt, you can be certain it can only
complexity of dealing with an already need their public GnuPG/OpenPGP key. be from your contact.
STEP BY STEP: CREATE YOUR KEY PAIR
1
Install GPA 2
Enter your details 3
Generate the keys
The closest thing to a standard GUI for GnuPG is a When asked for your name and email address, you This is where you need to enter the passphrase, and
tool called GPA (GNU Privacy Assistant). You need should use your real details. These are used by your this is more than just a password: not only is it used
to install this, and when first launched, it will ask contacts to ascertain the validity of your public to encrypt your keys, it’s used to unlock your identity.
whether you need a key pair generating if a pair key. Names and email addresses are obviously not The strength of GnuPG relies on this being unique
wasn’t already found. Select ‘Generate Key Now’ to enough, but they’re a good reminder. You should also and unguessable, while at the same time, you need to
launch the wizard that will create the key pair for you. accept the option to create a backup key. remember it without writing it down.
16 www.linuxvoice.com
EMAIL WITH GNUPG
Public and private keys in practice.
W
e’ve seen how important it is
to encrypt your data, as well keybase.io
as some of the theories As we’ve mentioned, one of the problems with public repository of public keys linked to email addresses
behind how it all works. You’ll be keys is being certain the copy you hold is the definitive and names. You can publish your own to a keyserver
reassured to learn that putting these key of the contact you want to email. Meeting up or directly from GnuPG’s GPA tool and most GUI clients,
ideas to use is straightforward; but how speaking via Skype is only really viable for contacts as well as download the public keys of contacts.
you have a relatively close relationship with, which But key servers don’t solve the fundamental trust
you do it will depend on your email
means there’s a real problem if you want to send a problem: you can’t be sure the person who uploaded
client of choice. message encrypted with the public key of someone the key is the person you think they are. This is
KDE’s KMail, for example, integrates else you want to email, or if you’ve received a signed where Keybase (https://keybase.io) comes in. It’s a
GnuPG without any further email from them and want to verify the veracity of your key server that works by linking public keys to their
requirements. All you need to do is sender. One solution is called a key server, a simple respective online personas. If you follow someone
on Reddit, Twitter and GitHub, and their accounts are
associate the key you’ve just generated
linked via Keybase, you can be almost certain that
with your email account, and you can those keys are for the correct person. There’s even a
do this from the Configure dialog by keybase command line tool for extracting keys and
clicking on Identities followed by the sending messages – all you need to import those
Cryptography tab. Use the Change keys into your system and use them yourself. Adding
yourself to Keybase is almost as simple (although it’s
buttons next to the OpenPGP
currently Private Beta). You link your Keybase account
encryption and signing fields to choose to your various networks by posting a Keybase-
your key. generated proof to those channels. This is validated
You can change how messages are automatically and your credentials are added to your
handled with the Security > Composing account. The final important ingredient works a little
like a social network. People who trust your account
panel. Signed messages will glow green You can even use Keybase to generate key pairs link their identity to yours, building an all-new peer-to-
if you’ve got the correspondent’s public and encrypt stuff, but we’d still recommend peer web of trust out of the humble GnuPG.
key in your GnuPG configuration, yellow keeping your private key decryption local. https://keybase.io.
if not, and you can use the Sign and
Encrypt toolbar buttons to sign or
encrypt your own emails. When signing restart of the client, you’ll be presented Write Message toolbar, and encrypted
or encrypting, you’ll be prompted to with the Enigmail Setup Wizard. Choose messages you receive will be decrypted
enter your key’s passphrase before the the Standard configuration and, thanks automatically, just like with KMail.
message is sent to your contact. to the magic of open standards, the Both apps will attempt to download
next step should automatically detect a public key from a key server if you
Encryption add-ons your default key pair. Select this and don’t have someone’s public key in
If your email client is Mozilla’s Thunderbird is now configured. your configuration. This is convenient if
Thunderbird, you’ll first need to install When you go back to the inbox view you need to read a message, but can’t
the Enigmail add-on via the Settings > you’ll now have the option to encrypt as trusted as a key you’ve exchanged
Addons menu. With this installed and a or sign your own emails from the yourself or through some other method.
STEP BY STEP: SHARING KEYS
1
Export your key 2
Import a key 3
Viewing a key
One of the best ways of swapping public keys with Your contact will have done the opposite, exporting With the third-party key imported, it will be listed
someone is to swap while you’re both physically their public key to device from which you can now within GPA along with some information about the
together. That way you can be certain the key you’re import it using Import Keys from the Keys menu. A contact, such as the encryption used and their key’s
getting belongs to who you think it does. To share status pane will inform you if they key was imported fingerprint. This is a ‘hash’ value for the entire key
your key, select Keys > Export Keys from GPA and successfully. You can always import more than one, if that’s often used a shortcut to quickly check whether
place the resulting file on a USB stick. you’re at a gathering where keys are typically shared. someone’s key is valid or has changed.
www.linuxvoice.com 17
FEATURE ENCRYPT EVERYTHING
SECURING YOUR DATA
Even if you’re not sending files across the internet, it’s worth keeping your data safe.
T
here are many tools created
specifically for encrypting your
data, and one of the best things
you should do is take advantage of your
distribution’s installer to encrypt your
home folder/partition. Ubuntu, in
particular, has a great system for
encrypting your home folder using
eCryptfs (see next page). That way, if
you ever lose your laptop, your data will
remain secure. But it’s also reasonable
to take a more ad-hoc approach, only
encrypting those files or folders you
want to keep secure. That way, those
files are easier to back up, copy or
transfer. As you’ve now invested lots of
mental energy in understanding how There should be no patterns or discernible information in encrypted data. It should
GnuPG works, this is probably the appear as random as the output from /dev/urandom, as shown here.
easiest way to start with file encryption.
GnuPG can use the same public and is whether you need to compress your file manager. This simple utility lets
private keys to sign or encrypt files as it files first. Properly encrypted data you add unencrypted files using a
does email. But equally, there’s nothing should look no different to a random simple requester, and a button in
stopping you creating a separate key stream of data. But file compression the toolbar lets you use your GnuPG
pair purely for dealing with files, only works when there’s duplication, keys to create an encrypted version,
although you will lose the ability to sign which makes post-encryption saved automatically with the gpg
files you may want other people to compression a waste of time. For that extension. It will also decrypt files
authenticate. reason, you’ll need to gzip or bzip2 your in the same way. And this being
It’s also possible to create sub-keys files and folders before encrypting GnuPG, anyone will be able to work
off your main key, using sub-keys for them, or use your favourite GUI tool. with the same files, regardless of the
different purposes or devices without Encrypting the files is easy, especially application or utility, if they’ve got the
losing the original authenticity. The only now we’ve already created a pair prerequisite private or public keys. It’s
special consideration you should make of keys and the GPA tool includes a a great way of backing up important
files, for example, but it does make
it more vital that you remember the
The configuration files of GnuPG passphrase to your key. If this is lost,
there’s no way of decrypting your
Even with the GUI approach, it still helps to have some you’ll make a mistake (such as publishing your private privately encrypted files. To solve this,
understanding of what files and folders are typically keys on GitHub).
ensure your encrypted key is saved
stored in .gnupg and what part of the encryption pubring.gpg This file contains your collection of
process they’re responsible for, regardless of whether public keys. These are public keys from people you in several places. Due to the strength
you use the command line or a GUI tool. In doing so, meet, people who contact you with their key, and keys of the encryption some people even
you’ll find using GnuPG easier, and it will be less likely downloaded from a trusted key server. leave these in plain sight, such as
secring.gpg Along with pubring.gpg, this is the other synchronised to their public GitHub
vital component in a configured GnuPG setup. It
accounts, but cloud-based email is
contains your secret key. However, from GnuPG version
2.1 onwards, this file becomes deprecated in favour of another good option.
a new agent that automatically places secret keys into
a folder called private-keys-v1.d (any files you have will Encrypt your home folder
be automatically moved). Taking file encryption a step further, you
crls.d The crl in this folder name represents ‘Certificate
might want to consider encrypting your
Revocation List’. It’s a way of marking keys you have
generated as invalid - you might have sent a private key home folder, decrypting it live as you
by mistake, or lost the passphrase, for example. log in. This is what Ubuntu does when
gpg.conf This is standard Linux procedure – the you enable the aforementioned
configuration file that tells GnuPG what it needs to installation-time encryption.
GnuPG can be used to encrypt and sign your know. If you use the command line, you can use it to
files, using the command line or one of its front- Its obvious advantage over per-file or
hold your preferred values – such as which key to use
ends. by default, or which keyserver you prefer. per-folder encryption is that you don’t
have to think about what you need to
18 www.linuxvoice.com
ENCRYPT EVERYTHING FEATURE
keep secure: everything is secure. If your system starts, making it much Encrypting passwords
your laptop is off, your data is easier to manage and copy encrypted There are many ways of storing important information like
inaccessible without your passphrase, data. Before mounting, the encrypted your various password and passphrase, but the best we’ve
which is usually configured to be your data can be found in a user’s .Private found is a small tool called simply pass. The great advantage
with pass is that it uses your filesystem to store and organise
login password. However, you still need folder, but after mounting, your home
files that contain the data. You could create a folder called
to make sure temporary files or caches folder will look just as it should. Shopping, for instance, and use this to hold password to your
used by your system are either saved to Amazon, eBay and Google accounts.
your home folder or deleted on John the Revelator As the passwords themselves are encrypted using
shutdown. You also need to make sure Choose a user whose home folder you the same GnuPG utilities we’ve already covered, you can
access their contents even when you’re not using the pass
you enable swap partition encryption, want to encrypt (we’re going with ‘john’),
command. It works particularly well when you use a remote
as the swap file contains data shuffled and without the user being logged in, synchronisation service, such as the one offered by OwnCloud,
in and out of your system memory. check to make sure they’ve no to automatically copy the contents of your password folders
We’ve been using an SSD laptop processes running. You can do this by to an online backup or remotely shared storage. We covered
without a swap partition for a couple of typing ps -u john. John should also the specifics of pass in issue 19’s FOSSPicks, but you can
skip the command line entirely by using one of the many GUIs
years without issue, which does make sure their password is tough yet
created to handle the complexity automatically. Our favourite
sidestep this problem. memorable, as this will be used as the for any desktop is QtPass, and we use an app called Password
Store on Android to access the same shared passwords.
If your laptop is off, your data is www.passwordstore.org
inaccessible without your passphrase,
which is usually your login password
The tool we’re going to use is the passphrase to encrypt their data.
aforementioned eCryptfs. Ubuntu Depending on the size of that user’s
configuration is easy, thanks to home folder – you can check by typing
Canonical’s developers. But those easy du -h /home/john – it may be quicker
scripts for creating and managing and easier to move out the majority of
eCryptfs have been migrated to most that user’s data. The first stage of
other distributions too, so it’s almost encryption will be far quicker, and the
as easy on Fedora and even Arch. Just user can then move the data back as Pass transparently uses the power of GnuPG and
search for and install the ecryptfs-utils they need it. There’s no point encrypting your filesystem to intelligently store passwords and
package, which should also ask for a download folder full of ISOs, for any other in data you want to keep secret.
rsync and lsof to help with locating a example. You can now instantiate the
user’s open files. eCryptfs itself is part migration process with a single
of the kernel, and the module needs to command: it’s hidden in the /home folder for
be loaded before you can start (sudo sudo ecryptfs-migrate-home -u john now, just in case something has gone
modprobe ecryptfs), although this You’ll be asked for John’s login wrong. John should log in and type
will be done automatically after you’ve password before the encryption ecryptfs-mount-private to mount the
created an encrypted folder. process kicks off. When it’s finished, decrypted folder and check it contains
It’s also important to note that text output will inform you of the next the expected files. Follow this with
eCryptfs sits on top of the filesystem – step, and if you check, you’ll see the ecryptfs-unwrap-passphrase to reveal
it’s not formatting your data at the block contents of John’s home folder has the master key in the encryption. Keep
level. It’s mounted using FUSE after now disappeared. Don’t worry, though, a note of this, as it will be required if you
need to access the data outside of the
login account environment, perhaps
using GnuPG to keep it safe and stored
somewhere else.
Finally, when you’re happy that
the configuration is working, remove
the hidden /home folder with the
unencrypted versions and reboot.
Thanks to some excellent eCryptfs utility scripts, encrypting any folder is an easy task.
www.linuxvoice.com 19
FEATURE ENCRYPT EVERYTHING
NETWORKING
Secure your connection with the magic of OpenVPN.
Y
our home network is mostly a or access machines behind a firewall. The vast majority of VPNs support
trusted environment. You can When you’re connected to a VPN, your OpenVPN, an open source VPN
plug devices into your network local connection behaves exactly as if it that uses OpenSSL for encryption.
and not worry too much about the data were physically relocated to the remote OpenVPN is almost ubiquitous on
as it travels between your NAS, your location, accessing those boxes and Linux systems, which means that
light switches and your Raspberry Pi. services without further configuration. configuration should be straightforward
Your router to the internet is your If you access your BrewPi at home with whether you’re connecting via the
firewall, and most things behind it are the IP address 192.168.1.177, you’ll use desktop, a Cyanogenmod phone
safe. But as soon as your data passes the same IP address if you use a VPN or a Linux router. We’ve found the
through that firewall, as soon as you to access your home network from a most reliable method for configuring
try to turn up the heating from your remote location. and enabling OpenVPN is via the
office before heading home, that data openvpn command line tool. With
is subjected to the wilds of the internet, Connecting to a VPN this installed and the ovpn file bundle
a place where no packet is safe. And It’s highly likely you’ve already used a downloaded from your VPN provider
that’s before we’ve even factored in the VPN for work, as they enable remote and unzipped, simply configure an
nightmare of open Wi-Fi access points.
The solution is to tunnel your data The vast majority of VPNs support
from your machine to your remote
trusted network. This is what a VPN OpenVPN, an open source VPN that
does, and it’s what SSH does too, to a
lesser extent. The tunnel is protected uses SSL for encryption
from the wilds of the internet by
encryption, with keys at either end of workers to access the same facilities OpenVPN connection by typing sudo
the tunnel encrypting and decrypting as when they’re in the office, only openvpn --config ‘path-to-file.ovpn’.
the data for transmission and upon securely from a remote location. But Within a few moments your network
arrival. Anything looking at the data over the last couple of years there’s connections will become re-routed
in-between will see a stream of random been a proliferation of low-cost private through the VPN. The best way of
bits and bytes. They won’t even be able VPN providers offering to tunnel proving this from the command line
to tell which protocols you’re using or your connection to some remote is to try and access a few sites, or
how you’re using them. geographical location while also vetting use a website to check your IP and
VPNs are becoming incredibly your data for security and privacy, as geographical location. If the VPN is
popular because they’re the most well as ad blocking. These are perfect working, your location will appear to be
comprehensive solution when you for side-stepping online restrictions and where the VPN exit is located, not the
need to leapfrog an insecure network local security issues. location of your local network provider.
STEP BY STEP: OPENVPN VIA A NETWORKMANAGER GUI
1
Connection editor 2
Configuration 3
Test VPN
Most distros have now moved to Systemd’s network Taking KDE as an example, open the connection After connecting to your network, you can activate
manager for their networking duties. Both Gnome’s editor and use the File menu to select Import VPN. the OpenVPN connection by clicking on the
and KDE’s GUI for this will let you add OpenVPN Navigate to the location of your ovpn file – unzipping configuration in Network Manager. When this is
connections and import a config file. You’ll first need the download archive if necessary, and let the running, you should see a padlock added to your
to download the certificate and ovpn file bundle connection manager import your settings. We needed connection and it should be obvious your data is
available from most OpenVPN servers. to to add a username and password. going through this tunnel.
20 www.linuxvoice.com
OPENSSH
Talk to your Linux machines across the wild, wild internet.
W
e couldn’t write about makes your server more secure, as you
encryption without can’t get access simply by guessing Further reading
looking into the open a password. To do this, on your client Encryption, security and privacy have been an important
implementation of the SSH protocol, machine (not the server), type ssh- part of our editorial coverage these last two years, and our
which is also a great place to end our keygen -t rsa to generate the keypair. old articles are the best place to start if you want to take
encryption coverage for this issue. This Enter a passphrase if you want the encryption further (quick plug: all subscribers get access to
our entire back issue catalogue).
wonderful offshoot of the OpenBSD added security of requiring both the
In particular, in issue 15 Jake Margason wrote a brilliant
project is one of the most used and key and a password. By default, it will tutorial on hiding encrypted volumes in plain site, so that
useful tools of the Linux stack, enabling save both keys into your .ssh folder. even the presence of your data becomes ambiguous. We
any two Linux boxes, or OpenBSD You can automatically copy the public also covered the popular password store, KeePassX, in issue
boxes, to talk securely to one another key to your server with the following 19, which is a great alternative to the humble pass tool we
mention here. Issue 17 was dedicated to privacy, and Ben’s
across the wilds of the internet. command: ssh-copy-id -i .ssh/id_rda.
excellent cover feature goes in-depth on how snoopers get
Because SSH uses very little pub user@server. access to your data and how how to use tools like Tor to keep
bandwidth to give you a remote You can now connect to your remote prying eyes away. There’s also a great interview with Cory
terminal, you can perform almost any server without entering a password. If Doctorow. Finally, as one last tip – keep your distribution up
task that you can locally, even on a not, make sure the permissions are set to date. Vulnerabilities will be found even in tools like GnuPG
and SSH, and the only solution is to replace those old versions
desktop, using nothing but a mobile correctly (chmod 770 .ssh). Finally, if
with new ones.
phone and a local GPRS network. We everything is working, disable password
doubt you need reminding of how logins on the server by commenting
it works, but things don’t get much out the PasswordAuthentication line
simpler than typing ssh followed by the in the server’s /etc/ssh/sshd_config
IP address of your server. If you’ve got a configuration file – and don’t lose your
user account with the same username, local SSH keys!
you’ll only need to enter your password One feature of OpenSSH that isn’t
to get access. often mentioned is that it’s also capable
of acting as a cheap VPN, tunnelling
Passwordless login general requests through the same
A much quoted improvement you SSH session you’re using for terminal
should make is to disable password access. It’s not a fully fledged solution
logins like these and replace them with in the same way as OpenVPN – it uses
the magic of a public/private key pair TCP, for a start, and OpenVPN should
that not only automates SSH login, but ideally be configured to use UDP, as it’s
faster and better suited
to passing through
NAT connections, but
it’s ideal for ad-hoc If you want to stay ahead of security and privacy
connections when issues, we can think of no better source than Bruce
you’re away from home, Schneier’s blog (https://www.schneier.com).
or need to jump a
geographical restriction
to your low-end-box. and click on the Settings button to the
The command for doing right of Connection. From the window
this is ssd -D 8080 that appears, enable manual proxy
username@server, configuration and enter ‘localhost’
setting up a SOCKS and 8080 as the port. This will re-
proxy on port 8080 of route all web traffic through the SSH
your local machine. tunnel. You can also do this for a
You then need to single port with the command ssh
configure either your -L 8080:localhost:8080 username@
computer (desktop) server, which is useful if you want to
or web browser to tunnel a single service from far away to
pass traffic through your local machine. We use this feature
this. In Firefox, open to access our router configuration page
You can easily tunnel web requests through SSH with the the Advanced settings from a firewall we’ve configured to only
-D option to create a SOCKS proxy. pane, click on Network allow through SSH.
www.linuxvoice.com 21
FEATURE DESKTOP SHOWCASE
DESKTOP SHOWCASE
Want to change the way your desktop looks? Searching for a new Linux
setup? Here’s a selection from Linux Voice readers to inspire you…
Michael White:
Nothing too fancy.
Just Ubuntu Mate
15.10 with the Numix
Circle icon set and
flat window borders.
I like it all to look like
paper. It pleases me.
Qkiel: Ubuntu 14.04
LTS with little tweaks
to the Unity launcher.
The icons are Gorts
Icons Vol. 6 (www.
forrestwalter.com/
icons). Wallpaper:
“My Cathedral by the
Sea” by Artur Rosa
(http://arthurblue.
deviantart.com/art/
My-Cathedral-by-the-
Sea-322219214).
22 www.linuxvoice.com
DESKTOP SHOWCASE FEATURE
Mike Saunders:
Xubuntu 15.10 on my
Libreboot-modified
ThinkPad X200. I’ve
to made the theme
lighter and brighter,
and I’m checking out
#linuxvoice on chat.
freenode.net, waiting
for the next SpaceX
launch…
Matt B: I’m currently
running Cinnamon
on Debian testing,
along with the Orchis
GTK theme and
Numix circle icons,
and Conky in the
background.
www.linuxvoice.com 23
FEATURE DESKTOP SHOWCASE
Sandersson: Xubuntu
with panel in the
button. Wallpaper
from WallBase.
Conky/Gotham and
TeejeeTech Process
Panel. GreyBird Style.
Elementary icons.
Frank Bell:
Mint 17.1 with
Fluxbox, Terminator,
and GKrellM.
24 www.linuxvoice.com
DESKTOP SHOWCASE FEATURE
Want your desk
top or
window manag
er featured in th
magazine? Take e
a full-screen
screenshot in PN
G format and
mail it to mike@
linuxvoice.com
Add some word .
s (max 50) abou
your setup, such t
as the WM/
desktop you’re
using and any
customisations
you’ve made!
Steve Newbury:
Ubuntu 15.10 with
Gnome 3.16 – added
Numix-Circle Icons,
Dash-To-Dock and
Dropdown terminal
extensions. Solarized
Dark terminal theme
in Gnome terminal.
Pickfire: DWM tiling
window manager
(http://dwm.suckless.
org), doing some work
on Makefiles.
www.linuxvoice.com 25
SECRETS VLC
SECRETS OF VLC
Master your media playback.
V
LC is among the most popular open source tools on any Fear not! For brave Sir Ben has trawled the graphical and
platform. It works even with files that many other video command line interfaces of this incredibly useful media player to
players struggle to read, and has a simple, clean interface find the eight most useful of these hidden features covering
that’s easy to use. Below this simple interface, however, there lurk everything from new media sources to easy control when away from
many more powerful features, but they’re not always easy to find. your desk. Never before has watching Monty Python been easier!
01 02
01
Play from YouTube web server that can host a simple web Convert / Save, then select the file to be
The YouTube web page page with controls for the video. The converted and then press, Convert/
works well for short videos, easiest way to start this is by launching Save. In the new window, select the
but if you’re settling down for VLC from the command line with the output options and press start Start.
something longer, you might want a following: This method can also be used to save
little more control, perhaps over the vlc -I http --http-password pass -I Qt network videos such as those from
audio and video (see tip 7) or over This uses pass as the password; if YouTube.
playback control (see tip 2). In these you’re on a public network, you may
04
cases, you can use VLC as the playback want to use something more secure. Capture video
device for YouTube videos. First, you Open http://<ip-address>:8080 to use As well as playing video,
need to use your browser to navigate to the controls. VLC can help you shoot
the video you want to watch, then open one. Go to File > Open Capture Device,
03
VLC and go to Media > Open Network Convert formats select the capture mode you want
Stream. Paste in the YouTube link, and Video players are fickle (probably either Video Camera or
it will play in the application. beasts, and just because Desktop), and you can start recording. It
you use VLC, which plays just about won’t be able to replace proper
02
Browser remote everything under the sun, that doesn’t recording software for serious users,
When watching videos, it’s mean you can ignore the problems of but it’s more than capable for simple
often not convenient to use others. Fortunately, VLC can convert messing about.
your mouse or keyboard to control between formats to make movies more
05
playback, but fortunately VLC includes a friendly for other devices. Go to File > Listen to radio and
podcasts
Sometimes videos may need cropping, noise VLC is a general media
player, and as well as video, it plays
reduction or adjusting the sync between video audio files. You can play these in the
same way you would with videos, or
and audio – VLC can do all this on the fly grab them from the internet. If you
26 www.linuxvoice.com
VLC SECRETS
03 04
06
05
07
listen to podcasts (such as the they’re good to watch. This could be
fortnightly Linux Voice cast), go to View some judicial cropping, noise removal
> Playlist, then in the internet section, from the audio or adjusting the sync
click the plus icon next to Podcasts and between the video and audio. There’s
enter the RSS URL for the cast. You can
also use this section to listen to an
no need to reach from some complex
video editor though, because VLC can 08
internet radio station. do all this on the fly. Just go to Tools >
Effects and Filters to set up whatever
06
Record current video you need.
You can record sections of
08
the currently playing video Bookmark a video
by clicking on the record button (a red If you find yourself needing
circle) in the control panel at the bottom to go to specific scenes in a
of the VLC window. If you don’t see the video frequently (perhaps you’re
button, go to the View menu and click watching Gladiator and need time to
on Advanced Controls. take in the magnificent ‘husband to a Manage to set up the bookmarks, and
murdered wife’ speech), you can make you’ll be able to return to those points
07
Audio and video effects your life a little easier by bookmarking with a single click. There’s no need to
Sometimes, videos need a particular points in the playback. Go to scroll back and forwards seeking out a
little treatment before Playback > Custom Bookmarks > particular scene again.
www.linuxvoice.com 27
FEATURE THE SNOOPER’S CHARTER
THE SNOOPER’S
CHARTER
The UK government
want to watch what
its subjects do on the
internet – wrongly,
says Ben Everard.
T
echnology changes quickly.
Law, on the other hand, changes
slowly. The laws we abide by
today are roughly the same as the ones we
followed last year, and the year before; in
contrast, if we look back 10 years, the way we
use tech has changed hugely.
Take smartphones, for instance. In 2006
Google showed the world its first prototype law determining the UK government’s access to
phone, and the Android operating system hadn’t even our communications – was passed. It was written
been released. 2006 was also the year rumours at a time when few could even conceive of a world
surfaced that Apple was working on a phone (which where people routinely checked a collectively edited
encyclopedia from their
phones, and the idea of a
The UK Home Secretary says that we social network hadn’t yet
need a new law to govern surveillance germinated.
Theresa May, the UK
on the internet, and she’s right Home Secretary, says that
we need a new law to govern
would be release the following year). Contrast that surveillance on the internet, and she’s right. However,
with 2016, when Google and Apple combined have we need to be careful: any new powers granted to the
over half the world market share for mobile phones. state are likely to stay for a long time, and will apply
Look at the web and a similar thing is true. In 2006, even when our use of technology has changed even
Facebook opened to the public after having previously more than it has since 2000.
been limited to academic institutions. YouTube, now
the third most visited site on the web, was less than a Who knows what tomorrow brings?
year old and first caught the public imagination. 2006 It’s impossible to say what the future will hold, but
was also the first year of Twitter. looking back at the change from 2000 to 2016 and
Go back a further six years and things seem even extrapolating forward, it seems almost certain that
more distant. Only four of 2015’s most popular technology will become even more integrated into our
websites even existed in 2000 (Google, Yahoo, daily lives in ways we can’t yet comprehend. It’s with
Tencent QQ and Amazon). There wouldn’t be a this technology of the future that any new law will
Wikipedia for another year. Windows 2000 and ME interact, so we need to ensure that it will enable us
were the dominant new operating systems. Sun privacy and security in a future world in which the
open sourced OpenOffice.org, though the 1.0 release amount of data about us stored online will be almost
wouldn’t be out for another two years. Napster was inconceivably large. The draft Investigatory Powers
still less than a year old. (IP) bill released by the government in November 2015
This year, 2000, was also the year that the gives the state sweeping new powers, but does it
Regulation of Investigatory Powers act – the main adequately protect our privacy?
28 www.linuxvoice.com
THE SNOOPER’S CHARTER FEATURE
THE BILL
What’s in the new law that the powers that be want to pass?
If the draft IP bill came into effect, it would allow the Global Internet Traffic
state to compel “anyone providing communications 180,000
services” to take “all reasonably practicable steps” to
160,000
intercept the data of their users. Here there are two
terms that have very loose definitions that will allow 140,000
the bill to be applied in a huge range of cases.
Petabytes per month
120,000
“Communications service provider” is a term that can
be applied to just about anyone who has anything to 100,000
do with any form of data exchange, from an internet
80,000
service provider to an app developer, and “reasonably
practicable” covers a lot of things that are easy to do 60,000
but ethically wrong, such as deliberately switching to
a weak encryption algorithm. 40,000
20,000
Dragnet!
The section of the bill covering bulk data collection 0
2014 2015 2016 2017 2018 2019
is the most worrying aspect to us at Linux Voice.
Year
It allows the government to require anyone who
handles communications to hand over vast swathes
Global internet traffic is
of data about people who aren’t suspected of any terrorists and serious criminals. While this is true, growing by 23% per year.
wrongdoing. The explanatory notes accompanying it is based on the logic that violating people’s rights If this continues, then
the bill attempt to justify these with the following: is acceptable if you violate enough people that you any interception laws will
“Access to bulk data is crucial to monitor known happen to also violate the rights of criminals and become 23% more invasive
and high priority threats but is also a vital tool in terrorists – also know as the ‘if you arrest everyone, every year.
discovering new targets and identifying emerging you’re sure to arrest all the criminals’ argument.
threats. The law provides for the use of interception, Perhaps the most revealing aspect of this passage,
communications data and equipment interference however, is revealed when you look at it as a whole.
powers in bulk. These can be used to obtain “Access to bulk data… is also a vital tool in discovering
large volumes of data that are likely to include
communications or other data relating to terrorists
and serious criminals. Robust safeguards govern
Access to bulk data is absolutely not
access to this data to ensure it is only examined crucial to monitoring known and
where it is necessary and proportionate to do so.”
This is probably the most misleading passage high-priority threats
in the notes. The first part of the first sentence is
demonstrably false: access to bulk data is absolutely new targets and identifying emerging threats…
not crucial to monitoring known and high-priority Robust safeguards govern access to this data to
threats, because if a threat is both know and high- ensure it is only examined where it is necessary and
priority (it can’t be high-priority without being known), proportionate to do so.”
then there is no need for bulk collection – an This implies that the government wants to use
individual warrant for the data on the known the bulk data to look for targets and threats that it
threat would equally suffice. doesn’t yet know about. However, the assurance that
The above quote then points out that if they bulk data will be used only where “necessary and
grab enough data, proportionate” creates a contradiction, since you can
they’ll probably get never know when it’s necessary and
some relating to proportionate to examine
data if you don’t know
the targets and threats
that you’re examining it
for. The only way that bulk
data can be used to identify
new targets and emerging
threats is if there aren’t robust
safeguards ensuring that it’s
www.linuxvoice.com 29
FEATURE THE SNOOPER’S CHARTER
only examined where bulk equipment interference powers that this new bill
it is necessary and confers, they’re unlikely to back off when they have
proportionate. Any more powers to attack computers in vast numbers.
robust safeguard Another aspect of the bill requires ISPs to hold onto
would ensure that everyone’s internet connection records (ICRs) for a
security services only year. These ICRs will detail every communication
filtered the data for you sent, including web requests. This part of the bill
known targets and already makes a distinction between metadata (that is, data
identified threats. about the communication, including which machines
took part and when it happened), and the contents
Hacked off of the communication. In the case of web browsing,
The equipment interference aspects of the bill this means that the ISP will record every site you
allow the government to alter either the software or requested data from, but not the actual page you
hardware of your machines in order to extract requested. In other words, if you visit a BBC news
information. In other words, it gives them the right to page, the ISP will record a visit to www.bbc.co.uk, but
hack computers. not the story that you visited. The explanatory notes
It may be useful to target individual equipment describe ICRs thus:
when done in accordance with proper safeguards; “An ICR is not a person’s full internet browsing
however, there is no situation where bulk equipment history. It is a record of the services that they have
interference can ever be justified. It is never connected to, which can provide vital investigative
appropriate for security services to routinely hack the leads. It would not reveal every web page that they
visit or anything that they do on that web page.”
The government will be able to build This requirement for only the metadata means
that sites can still be served over secure connections
up a far more detailed picture of your (HTTPS) without violating the law. However, it’s
actually a more nuanced situation than this, because
browsing habits than first appears a web page isn’t a single entity that you download in
one go: it’s made up of images, scripts and styles that
software and hardware of innocent people who aren’t all come in different requests.
suspected of any wrongdoing. It was under a similar Each of these requests will go on your ICR, and
bulk warrant that GCHQ hacked into the computers by looking at the pattern of all these requests, the
of Privacy International, a charity dedicated to the government can build up a far more detailed picture
right to privacy. At the time of writing, this hacking of your browsing habits than first appears. Depending
is under investigation by the Investigatory Powers on the website visited, our tests showed that this
Tribunal, because GCHQ didn’t have the powers that pattern of request was often sufficient to distinguish
it would have under the new bill. If GCHQ hacked the category of page visited within the site, and
into the computers of charities before they had the sometimes the actual pages visited.
Left: Theresa May is the current Home Secretary, but the powers in the bill will allow any
future Home Secretary to authorise spying of the entire populus. Above: GCHQ, based in
Cheltenham, carries out much of the government’s most invasive spying.
30 www.linuxvoice.com
THE SNOOPER’S CHARTER FEATURE
THE STATISTICS OF MASS SPYING
Bulk collection does not and cannot stop terrorism.
The current UK government is trying hard to convince
the population that bulk collection is an essential tool
in the fight against terrorism: that by collecting
enough information, and by analysing it carefully
enough, the security services will be able to work out
who’s a terrorist and stop them before they commit
an act of terror. The problem with this is that it’s
mathematically wrong.
The first thing to take into account is that terrorists
are rare. In the year ending in March 2015, 118 people
were charged with terrorism offences in the UK. No
attacks happened in the UK in the same period, so Thomas Bayes developed
there can’t be a significantly higher number of active his theorem in the 18th
terrorists than this. The UK population is about 64 century, and it still has
million people, which means that, even if we play safe applications in bulk
and scale up our estimate of terrorists by a factor of surveillance and spam
10, terrorists only account for one in 64,000 people. filtering.
Now let’s consider the algorithm that the state will
use to process this bulk data and find terrorists. This effective algorithm will actually be wrong 99.8% of
isn’t an especially easy problem, since terrorists will the time. The reason for this apparent contradiction
be attempting to blend in, and many ordinary people’s is because there are so few terrorists. The 1% of the
curiosity takes them to websites about terrorism. time that the algorithm falsely flags a non terrorist
The algorithm will never be 100% accurate, so what as a terrorist (1% of 64 million) vastly outweighs the
happens if it’s not? What happens if the algorithm is, proportion of the time that it correctly flags a terrorist
say, 99% effective at finding terrorists? (99% of 1,000).
If this terrorist search filter were used as an input
Enter Bayes’ theorem for a more involved investigation, the problems
Bayes’ theorem tells us that the probability of a person wouldn’t get any better. With a success rate of 0.0015,
being both a terrorist and flagged by an algorithm the security services would have to investigate on
that’s 99% effective can be calculated by multiplying average 667 people for every terrorist they caught.
the probability of a person being a terrorist by the With the UK police officer numbers having declined
probability of a terrorist being flagged by the algorithm by 17,000 (about 12%) between 2010 and 2015, the
(0.99), then dividing the result by the probability of police don’t have the manpower to chase up this
someone being flagged regardless of whether or not number of inaccurate leads. GCHQ is expanding
they’re a terrorist (0.01). The result is 0.0015. In other by 1,900 anti-terror spies, but even these numbers
words, the chances are that your theoretical 99% fall far short of the number required to make their
way through the sheer number of leads that a 99%
Safe spaces effective algorithm would generate. Even if all these
Speaking in the wake of the Charlie Hebdo attacks in leads were followed up completely correctly and every
January 2015, David Cameron reiterated his desire to be flagged terrorist were apprehended, the algorithm still
able to intercept the entirety of online communications: missed 1% of terrorists, so this would leave 10
“But the question is are we going to allow a means of free to carry out an attack.
communications which it simply isn’t possible to read.
Not only does bulk
My answer to that question is: no, we must not.” The Draft
Investigatory Powers Bill is the fruit of that desire. However, collection not work,
the Prime Minister’s ambitions are not met by this new bill but it cannot work.
for two simple reasons: Targeted, individual
• It places the requirement to intercept data on the surveillance coupled
telecommunications operator, but in the case of user-
with traditional
implemented end-to-end encryption (such as GPG), this
operator has no capabilities to intercept the plain text. methods of intelligence
• It only has the capability to force companies with gathering and police work
operations in the UK to comply with warrants. Any non-UK are the only effective ways
citizen could set up an organisation outside the UK offer of tackling the problem.
truly private communications without the need to comply
Now write to your MP and tell
with this law.
them so!
www.linuxvoice.com 31
FAQ CHIP
CHIP
The Raspberry Pi Zero isn’t the only ultra-cheap single board computer in town.
Say hello to the $9 CHIP.
Next Thing Co, a small startup to sound wasteful, but there was
MIKE SAUNDERS founded in 2013 and based in something especially appealing about a
Oakland, California. The company’s first computer that you can break, throw
product was the Otto, a customisable away and replace for a pretty small
Isn’t a chip just a part of a digital camera that’s powered by – of all outlay. The CHIP takes this to the next
computer? What’s the deal things – a Raspberry Pi. By connecting level: want to use a tiny Linux-based
with the name? the Otto to your smartphone, you can computer for home brewing, monitoring
We don’t think it’s a great name add filters, perform post-processing wildlife or doing other tasks where it
either – the CHIP has several effects and do other tricks without could end up broken? Buy a few CHIPs
chips on it, and searching for support having to be an expert in photography. and if one buys the farm, just swap it
on the internet is going to be tricky. You can even use the camera to take out for another. (And go without the
Googling for “CHIP not working” is videos in animated GIF form, which is couple of beers at the pub that night to
going to bring up lots of unrelated ideal if you’re looking to build up sweet, claw back the cash.)
results, although it’s not as bad as the sweet karma on Reddit’s /r/gifs forum.
situation in Gnome (“Web won’t connect Fair enough, but the Pi Zero
to the, er, web…”) What is it that makes the CHIP beats it in that respect now,
But anyway. CHIP (or officially so special? doesn’t it? Doesn’t that mean the
C.H.I.P., but we get a headache from First and foremost, the price. We CHIP is obsolete?
seeing it written like that all the time) is may all be cooing over the $5 Well, yes and no. Both prices are
an ultra-cheap, $9 single-board Raspberry Pi Zero now, but back when astonishingly low, and it’s true
computer very much like the Raspberry it was announced, the $9 price point for that the Pi Zero has the edge if you
Pi Zero. Indeed, it was announced and the CHIP raised plenty of eyebrows in really want to save every penny. But
crowdfunded on Kickstarter back in the wider computing world. Just the there are also some substantial
May 2015, half a year before the Pi Zero fact that a fully functional, usable differences to the hardware. Both
was revealed to the world, and the team computer (running Linux of course) devices have single-core 1GHz ARM
behind it managed to get over would be available for under a tenner CPUs backed up with 512MB of RAM
$2,000,000 from almost 40,000 was brain-bending for many people. – but while the Pi Zero requires a Micro
backers. That’s a real computer for the price of a SD card for storage, the CHIP has 4GB
couple of beers at the pub. of flash memory built in.
Wow, that’s not bad. Who’s Part of the original Raspberry Pi’s Similarly, CHIP has a single full-size
making it? appeal was its low price. We don’t want USB port, in comparison to the Micro
USB socket on the Pi Zero, so that’s one
CHIP supports wireless networking and fewer adaptor you may need to buy.
And then the CHIP supports wireless
Bluetooth out of the box, whereas the networking and Bluetooth out of the
box, whereas the Pi Zero has neither of
Raspberry Pi Zero has neither of these these. So while the Pi Zero may look like
32 www.linuxvoice.com
CHIP FAQ
the cheaper option at first glance, when
you start to think of the adaptors you
may need to kit it out with USB and
Wi-Fi support, its price can match (or
even exceed) that of the CHIP.
Video-wise, the Pi Zero has a Mini
HDMI port (again, requiring an adaptor
in most use cases) whereas the CHIP
just has composite video with the
option to convert to HDMI via another
adaptor. And in terms of GPIO, the Pi
Zero has the lead with 40 pins, in
contrast to the CHIP’s 8 (although this
may change as the product reaches
widespread production).
So both boards have their upsides
and downsides. The CHIP is more
capable out of the box, but the lack of a
Micro SD card slot means you have to
deal with the 4GB storage limit or start
adding a USB flash drive or SD card
reader – which again bumps up the
total price. You’ll have to wait a few months to get your hands on a CHIP, but it will give the Pi Zero some friendly
competition when it comes to embeddable computing power.
And what about those
all-important shipping costs? hacking and others. We look forward to QWERTY keyboard, GPIO breakouts
At the time of writing, delivery of seeing the CHIP hacked into old ZX and 5-hour battery. You can order one
CHIP to addresses in the United Spectrum and Commodore 64 cases, now, but it costs more than five times
Kingdom and United States costs $6.22 running emulators and hooked up to as much as the CHIP itself at $49.
using the charmingly named Super televisions. In addition, Next Thing Co sells
Standard Shipping method. For the Pi various converters and add-ons such
Zero, our friends at Pimoroni charge OK, I’m sold! How do I get hold as a HDMI adaptor for $15, VGA
£2.50 to post orders under £50 to the of one? adaptor for $10, composite video cable
UK, or £4 to the US. So again, the prices Well, you’ll have to be patient. for $5 and a case for $2. As with the
aren’t drastically far apart in this Next Thing Co is working on Raspberry Pi, we expect to see plenty of
department, and you could save on the getting its first batch of shipments out third-party add-ons and accessories
postage by ordering a bunch of CHIPs to the Kickstarter backers – so those once the supply chain for the device
or Pi Zeros at the same time. are taking priority. You can order a CHIP itself becomes stable and lots of people
from the website at www.getchip.com, have the devices in their hands.
What versions of Linux run on but you may have to wait until at least
the CHIP? June before it will arrive in your letter I don’t know what to buy – tell
Out of the box, the device will run box. In the meantime, Next Thing Co me how to spend my money!
Debian GNU/Linux. Next Thing has promised to send a review unit to Why not get both? Just like with
Co is keen to position the CHIP as a Linux Voice Towers, so we’ll have a KDE vs Gnome, Emacs vs Vim
general-purpose computer, capable of closer look at the machine next issue and Z80 vs 6502 (at least for us retro
surfing the web, playing games and and you can start thinking about CHIP fans), there are zillions of debates
editing documents in LibreOffice. Given projects before yours arrives. concerning CHIP vs the Pi Zero raging
the computer’s very modest on the internet. Teams have assembled
specifications (and our experiences And what comes next? Is there on both sides trying to win over
with the original Raspberry Pi), we think going to be a CHIP 2? Or a undecided buyers, but given the crazily
this might be overly ambitious – but CHIP Zero for 99 cents? low prices, we really don’t see the need
we’ll know for sure when we spend Maybe one day! There are no for flame wars.
more time with it. public plans for a followup device Competition is good, so we hope this
But outside of home desktop usage, right now, but the Next Thing Co team spurs on the development and release
the CHIP could be a great alternative for aren’t sitting around twiddling their of even more ultra-cheap single board
many of the jobs at which the thumbs. The PocketCHIP (www. computers. Unlike with high-end
Raspberry Pi excelled: robotics, home getchip.com/pages/pocketchip) smartphones or laptops, you don’t just
media servers, simple network attached converts the device into a fully fledged need to settle for one – build up a little
storage devices, classic 8/16-bit handheld computer, adding a resistive arsenal of low-cost Linux-powered
console emulation, ARM assembly 4.3-inch touchscreen, miniature devices and share the love.
www.linuxvoice.com 33
INTERVIEW LESLIE HAWTHORN
LESLIE HAWTHORN
COMMUNITY CURATOR
Graham Morrison meets an award-winning Free Software/open source advocate
and community manager extraordinaire.
L
eslie Hawthorn has been involved the Oregon State University, as its Open Wikimedia, StumbleUpon, Mozilla and Etsy.
with open source for a long time. Source Lab Community Manager, Leslie Her personal emphasis has always been
She spent five years as Google’s is now the Community Manager for on cultivating open source communities
program manager for its brilliant Summer Elasticsearch, the open source company to create productive and inclusive
of Code project. After stints at Red Hat, responsible for the ultra-scalable search environments for Free Software. And that’s
as its Engineering Team Manager, and platform that powers search on sites like exactly where our conversation starts.
Do you think the open source be difficult is that because discussions and open source software is that you
community is becoming more are more prominent, they are much can give back, and this is another way
inclusive? more politicised. If the dialog constantly in which you can give back.
Leslie Hawthorn: I think the is perceived by you as being ‘you are a
community is slowly and surely bad person because you fall into a It’s becoming increasingly
becoming more inclusive, which I find privileged group’, you’re just not going intimidating to get involved in
to be quite good. There are certain to listen. You simply will not listen. Why the community – everything is out
debates that we were never having should anyone feel bad for having had a in the open now and your CV is
before that I think are important. The good life? That’s not a reasonable online. Do you think this is a good
idea of, “Let’s talk about a lack of request to make. place for us to have gone?
women in tech,” or, “Let’s talk about a I think it’s more about changing the LH: I think that’s tough. On the one
lack of people of colour in tech”. This is dialog from ‘as a privileged person, you hand, it’s wonderful to be able to say
not a new debate by any stretch of the are an oppressor’, to ‘you are a your work is available publicly and you
imagination, or Anita Borg wouldn’t privileged person, you are in a powerful can stand on your own merits because
have created the Systers email list, position to be able to ensure that other everyone has access to see what you
what, 30 or 25 years ago. It’s certainly people can enjoy the successes that are capable of doing, and to some
not new. One of the things that I find to you have’. One of the beauties of free degree you can say that this removes
Leslie won O’Reilly’s
prestigious open source
award way back 2010 for her
work making Free Software
communities better.
34 www.linuxvoice.com
LESLIE HAWTHORN INTERVIEW
“People get over their intimidation
factor a lot more quickly when they
feel like they’re dealing with a human
being who cares about their success.”
www.linuxvoice.com 35
INTERVIEW LESLIE HAWTHORN
bias, because then people can don’t necessarily have time to prioritise to help everyone get into the tent and
objectively judge the quality of my work. putting together cool sample be successful. It’s not because Python
And yet there is a whole lot of applications on GitHub so everyone is easy: it’s because we as geeks value
problematic stuff involved in the idea of knows that I’m fantastic. I’m fortunate things that are really difficult and really
saying ‘well, I don’t need you CV in that I don’t have to worry about that, I challenging and for which you have to
anymore, I just want to see what’s have other ways in which I’ve built a be a super genius, because that’s part
available to you on GitHub’. reputation within the community that of how we define ourselves, and there’s
Not everyone has had the privileged speak for themselves and not everyone
position of working in a company in
which they were encouraged to do
is in that position. I think it’s a double-
edged sword.
Fundamentally, geeks are
open source software development. profoundly insecure; we
That’s gaining traction and much more Especially for people who want
prominence now but that certainly has to get in who may not be just don’t want to admit it
not always been the case. Depending versed in the way that it works.
on the organisation you work with, they LH: If you look at the rise in programs a kind of mismatch there.
may have a wonderful dedication to to help new people get into open source Fundamentally, geeks are profoundly
publishing open source software but – so this is everything like the Google insecure; we just don’t want to admit it.
that doesn’t mean that all of their Summer of Code spin offs like Rails And part of what drives us and our
software is published as open source, Girls Summer of Code, Django Girls etc success is that insecurity. I think it’s OK
so that should not diminish the quality – they are a phenomenal set of tools to to be insecure; just own that insecurity,
of your work. help people begin the on-boarding know what it is, know how it acts and
And then there’s the question of how process and get into open source operates in your life, and don’t let your
much time you have to devote to software development. tools use you. But if it’s a tool to
effectively writing software for fun or On the other hand, if you also look at become more ambitious, more
for a hobby or to scratch your own itch the (there’s no nice way to say this) successful, to learn new things, then
if that’s not part of what you’re paid to critiques of people who go through great – that’s wonderful. If it puts you in
do during your day job. I know that I those programs… Python is considered a position where you need people to be
have commitments to taking care of to be an easy programming language, less successful than you in order for
family members, I have commitments and I think it’s considered an easy you to feel good about yourself, that’s
to volunteer work that are not about language because the Python when it becomes problematic. If you’re
coding projects but they are about community has very deliberately, from competing with yourself, that’s one
helping more software developers. I very early on, made a concerted effort thing. If you’re competing with everyone
A San Fransisco Bay area
native, Leslie has upped sticks
to beautiful Amsterdam, the
home of ElasticSearch BV.
36 www.linuxvoice.com
LESLIE HAWTHORN INTERVIEW
Code program, we tell people just
Outreach programmes such Google it and you can find the answer.
as Google’s aren’t enough If it’s a specific error message then, fine,
to foster a more inclusive sometimes that will work for you, but
environment in Free how well are error messages written?
Software – a deeper cultural [much laughter]. Someone who is not
shift is required. intimately familiar with your project – to
the point where they know all of the
gaps that you’ve already forgotten
because you’ve created workarounds in
your head already – they don’t know
what to search for.
So a new user or contributor
might have no knowledge or
context of anything. Being in that
situation is awful because you don’t
want to look ignorant, like you’ve
missed something on the FAQ, or
whatever people will say.
LH: Absolutely. There are a couple of
different things that I’ve seen be very
successful and there are arguments for
and against each of these tasks, but
there are some projects, like the Drupal
world where they have a channel that’s
for Drupal newbies if hanging out in the
regular Drupal channel is difficult for
you either because it’s noisy, because
there’s several people on there on any
else in situations where it’s not really a documentation is broken or, potentially, given day, or it’s difficult because you’re
competition, it’s a situation where a the software is broken. intimidated because the creator of the
rising tide lifts all boats, that’s when it Drupal project is in there and maybe
becomes problematic. But the documentation is you don’t want to ask the silly question
nearly always broken. in front of [him]. Fair enough, then
So to give people the toolkits LH: Exactly. There’s a gentleman by the maybe you can ask in the Drupal
to improve their communities name of Rich Bowen who is a docs guy Newbies channel. And there’s also
and make them more approachable for the Apache Software Foundation. I always the encouragement that it’s
and less intimidating, what should just ran across a presentation he did great that you’re in here, you can also
those communities do? More and it was so incredibly good about the go over there too, you can play on
outreach programs like Google’s importance of documentation to your Drupal, it’s cool.
Summer of Code? free and open source software project. Other projects have a human who is
LH: I think Google’s pretty much alone About two slides in, when he’s talking dedicated to be the welcome wagon, so
doing projects like that for a couple of about DocBook and what not, he talks if someone new turns up on a mailing
reasons. One, Google being Google, about the importance of empathy and list or IRC or the issue tracker, and
people figure ‘OK, Google’s got it’: creating good documentation, being they’re asking questions that are clearly
Google’s got my email, Google’s got my able to see things through the eyes of well thought out but under informed
documents. your user. He also says that the then it’s their responsibility to welcome
And I don’t think that outreach response ‘RTFM’ is the height of this person [and let them know that]
programs like that are the sole means arrogance because, one, you’re 1) their ideas are valued, and 2) they’re
to make communities more welcoming. assuming that someone hasn’t read the not dealing with a bunch of people who
I also think it’s as simple as making manual, which isn’t fair because often were sprung full born from the head of
your documentation approachable and documentation is broken, and you’re Zeus knowing absolutely everything
also inviting people who don’t have also making a whole huge load of (because that’s not actually a thing).
expertise in your project whatsoever to assumptions about what someone has Amazingly enough, people get over
go through your documentation, as knowledge in their heads. their intimidation factor a lot more
because if somebody is not able to One thing that that I think that we quickly when they feel like they’re
effectively do the task that is forget constantly, and this was brought dealing with a human being who cares
documented, it’s not because this home to me very sharply during my about their success instead of dealing
person is foolish, it’s because the time running the Google Summer of with a wall of super geniuses sort of
www.linuxvoice.com 37
INTERVIEW LESLIE HAWTHORN
“That Thomas Edison quote: ‘I didn’t
fail 10,000 times, I found 10,000 ways
that didn’t work on the way to making it
work’ – that’s on posters for a reason.”
staring down at them saying ‘why are holding and hugs when they’re doing a former site reliability engineer there
you wasting my time’. well. They need it when they’re doing and still has great ties there, and I have
poorly. I think as human beings we’re friends there because, you know, the
And the same thing could be neurally geared as well as financially ex pat life. They actually find that 19
said of failure in all of those incentivised to focus on all of the bad out of 20 of their deployments that are
projects and workplaces. parts about failure, as opposed to what customer facing have a net negative
LH: Absolutely. And the thing that I we get out of it, which is new impact on revenue, sometimes into
would like to tease out of these knowledge. the millions of dollars over the space
examinations of failure is that a good of 5–10 minutes before they’re rolled
leader’s job is to help the people around People have been thinking back, and that they have a great deal
them get through their failures. If you about this for a very long time. of tolerance for that risk because the
are a capable competent human – and LH: Yes, and clearly it’s easy to say and 1 out of 20 deploys that actually have
really all colleagues are capable hard to do, as with many good things. If a positive impact on revenue can
competent humans – you don’t need it were easy to do, everyone would move the needle in terms of millions or
your manager to help you be already be doing it. billions because they have found that
successful. You don’t, you can go We’re moving into an era where we one thing that actually works.
succeed on your own. have a wonderful set of guidelines, such
You all need to get together and form as the Agile manifesto or DevOps That’s a cultural thing. Your
a plan about what needs to happen, but methodologies, you know: fail fast/ theme mostly seems to be
it is not your manager’s role to help you recover quickly; make it graceful; it’s not about empathy, because empathy is
succeed. It’s your manager’s role to help when your systems will fail or when you something you can’t put in a
you when you have failed to figure out as a human being will fail. And that’s all spreadsheet.
what that graceful recovery looks like, lovely and we pay lip service to all these LH: Absolutely. Interestingly enough, I
and also to help you realise that the fact ideas, but then we still operate in these will be giving a talk at Eurocamp with a
that this issue has come to pass is not ways that clearly subvert that as an wonderful gal named Dajana Günther
the end of the world, it is not a reflection idea, such as telling all of your team who is on the board of Ruby Berlin,
on you, it is a reflection on the outcome that it’s totally OK to fail – that’s not a which is the non-profit organisation that
of that situation. OK, so you made a problem, we just fix the problem, and looks after all of the Ruby activities in,
mistake, OK so you failed; well that’s then, instead of that actually being the obviously, the city of Berlin, but they’re
fine, what did you learn from it? If you way things are executed, instead we also highly involved in Rails Girls
didn’t learn from it, here are some of the micro-manage every bit of the process Summer of Code and it’s all about
things I learned from it, perhaps I and so people get the same message, empathy and why this is a valuable skill,
should not assign you a deadline two which is that’s it’s not OK to fail. but also how to cultivate empathy if you
hours in the past, for example. Again, And I also have a great case study don’t necessary think of yourself as
people don’t need a lot of wisdom, hand in booking.com. So, my partner was terribly empathetic.
38 www.linuxvoice.com
LESLIE HAWTHORN INTERVIEW
How can you cultivate that people often chose not to be it’s easy to succeed in the free and open
empathy?!? empathetic because they’re not source software world. I was fortunate
LH: This is terrific! This is actually financially incentivised to do so. The to have a mother who worked for one
something you can learn, so this is interesting thing that rolls out of that of the large telecommunications firms.
great because just telling people to be is the need to create team structures, I had access to the command line when
empathetic is not terribly helpful. I have organisations or structures of any kind I was three, and I’m (now) almost 40.
a personal pet peeve against going to that specifically incentivise empathetic
talks and people are like, do this, and behaviours, because clearly as humans And you used the command
we are hard-wired to do this, but again, line at the age of three?
it needs to be learned and we need to LH: Oh yeah! I think I was a strange
A leader’s job is to help have it proven that it will benefit us. three-year-old, and I stayed strange! It’s
the people around them Do you think recent inclusivity
very easy to think, “Well, I succeeded so
anyone can succeed.” And we like to
get through their failures issues within the open source think of ourselves as being special but
community are unique to the not the special snowflakes, right?
personality types it attracts? We’re special human beings because
give no information on how. So there LH: Yes and no. The challenges to we have this great gift, which is the
are actually a couple of great ways to empathy are not unique to the Free FOSS ecosystem, both in terms of the
cultivate empathy, one of which is fairly Software/open source world at all. I great tools we can use and the great
easy for us, which is read fiction. think they are the result of increasing community we have. And yet none of
pressures on all social systems period. us wants to think that maybe we were
Really? I think we can just blow this out to deeply successful, not because we
LH: Researchers have the whole world and go, wow, there’s were super awesome but because of
demonstrated that reading fiction helps now seven billion+ people, if you have the tools we had to be successful with
you to be more empathetic because the concern over your livelihood you will that no-one else had. That’s not a
process of fiction and actually going tend to be less empathetic. One of the comfortable position to be in.
through the cycles of imagining the things that I think is a unique challenge Frequently people look at
scenario in your head, while you’re for our domain is that we have a meritocracy, and meritocracy is a very
consuming the text, causes you to go rhetoric of openness, inclusivity, comforting concept, because it means
through the thought exercise of, “What meritocracy. Anyone can come and that I have succeeded because I am
is this character experiencing?”. I can participate in this process, and that is terrific and it’s very hard to disassociate
empathise with that because I’m true up to a point. There are certainly no the notion of that you are terrific and
internalising it – use those same rules saying that you may not you also have more advantages that
lessons you learn while reading fiction participate. But, because we have that allowed you to succeed. That’s not a set
and apply them to the real world as our rubric for understanding of ideas that people are very
[Reading Literary Fiction Improves Theory participation in FOSS, we don’t realise comfortable holding their minds on.
of Mind, David Comer Kidd and that that in itself is a problematic
Emanuele Castano, www.sciencemag. statement to make. It’s not as simple as It requires a personal level of
org/content/342/6156/377.abstract). [assuming that] everyone is able to do abstraction that a lot of people
There’s also some other cool this. There are various types of privilege aren’t equipped for.
scientific studies where people have that mean people are not able to LH: Yes, and self awareness is hard,
noted that empathy is a choice and engage, so it’s easy for me to say that just in general. And I think we’re not
taught exercises in self awareness
frequently [enough]. It’s empathy for
oneself and one’s willingness to be
humane towards oneself rather than
simply brush all of our feelings under
the carpet and pretend they don’t exist.
And honestly, empathy starts with the
ability to show care for the self and
really understand: these are my areas
of success; these are my areas of
failure; here’s what I want to do to
increase my successes; mitigate the
places where I’m not doing as well as I
want to do. Until you’re able to do that
“These are my areas of
with yourself, being able to provide that
failure; here’s what I want
as a baseline of compassion for other
to do to increase my
successes.” people is, well, I won’t say impossible,
but it’s certainly much harder.
www.linuxvoice.com 39
LISTEN TO THE PODCAST
WWW.LINUXVOICE.COM
BUY MUGS AND T-SHIRTS!
shop.linuxvoice.com
INTRO REVIEWS
REVIEWS
The latest software and hardware, rigorously bashed against a wall by our crack team.
On test this issue . . .
42
Ardour
4.6
Better plugin integration,
hardware controllers
for faders, easy track
duplication and the ability
to save plugin templates are
the stonking new features
Andrew Gregory in the latest release of
is 82.3972223% of the way through this essential digital audio
Dry January. But who’s counting? workstation.
I
t’s a new year, so let’s make some
predictions! I predict that Apple will
release a cable for its devices that
won’t fit any of its old devices, so
‘forcing’ its customers to buy a load of
new stuff to replace the perfectly good
old stuff. This will take place, ooh, in Tails 2.0 43 Wine 1.8 44 NodeMCU 45
spring. March maybe. The Anonymous, Incognito Live Need Windows software to run A single-board micocontroller
In August Microsoft will launch a System gets even easier. Privacy on Linux? Try Wine – but only if with Wi-Fi for only £3? This is the
partnership/foundation/call it what you for the masses, here we come. you have the patience… stuff of a madman’s dreams!
will, with an organisation nobody’s
heard of, that purports to represent Free
Software standards. This organisation
will conclusively prove that MS loves
Group test and books
Linux. Again.
A company with the info security of a
wet paper bag will leak a load of
customer data some time in October.
This will be blamed on ‘hackers’, and
not the use of passw0rd123 as their
admin password. And more car makers
will be sucked into the emissions
scandal, as we pretend that the
combination of the human nature and
closed source software makes this
anything other than inevitable.
Meanwhile, unnoticed, Linux will Booooooooooooooks!!!! 48 Group test – software licences 50
continue to power everything from A welcome return to badass form from Kathy Sierra, The licence under which software is released
fridges to the internet. Carry on! plus some other books that aren’t by Kathy Sierra, can make or break it. Here are some of the most
andrew@linuxvoice.com but will also help you write better code. important in Free Software and beyond.
www.linuxvoice.com 41
REVIEWS ARDOUR 4.6
Ardour 4.6
Graham Morrison finds another excuse to turn on his immense synthesizer collection.
W
Web http://ardour.org e don’t always agree with the monetising studio. We use it to record our podcast. It’s brilliant,
Developer Paul Davis methods of Ardour’s author, Paul Davis. At and recent releases have taken the application from
Licence GPLv2
the moment, for example, it’s impossible to a difficult niche to what we feel is imminent
download a binary of Ardour from its main website mainstream recognition.
without going through several nag screens, paying a 4.6 is a huge step forward too, mostly in the way
one-off (low) fee or buying a subscription. Ardour handles plugins. These essential components,
It’s completely within Paul’s rights to do this, and like filters in Gimp, enable you to process audio,
there’s nothing stopping anyone downloading the changing the perceived volume of a track, add echo,
code from GitHub and compiling the packages or distortion, or whatever else the plugin developer
themselves (as we did). But binary obfuscation isn’t has imagined. There are hundreds available for Linux
going to help Ardour’s cause. We don’t think making and they’re fundamental to Ardour’s functionality – we
something harder to access is going to improve Paul’s can’t believe Ardour doesn’t include its own
profits, especially when Linux distributions are going compressor, gate and equaliser, for instance. But
to bundle their own versions anyway and OS X or these external plugins are now a lot easier to use. The
Windows users will just pirate something else. We main mixer view even lists your favourites, right where
humbly suggest that Ardour needs to get more, rather you can drag them into your audio tracks. You can
than less, exposure, and another solid update like this even drag and drop presets too, enabling you to
is a great advertisement. quickly save and retrieve your hard-tweaked
Ardour is a multi-track audio editor. Unlike Audacity, parameters. The mixer view in general is now 25%
it’s designed to record and mix multiple recordings smaller, and the GUI has had lots of minor tweaks to
together, much as an engineer would in a recording make things look better. We also appreciated the
ability to turn off all plugins when loading a project, as
some plugins can cause a project to crash. Most
similar applications have had this feature for years.
Ardour is developing at an incredible rate, and the
quality of each release is staggering. All we can add is
that if you use Ardour professionally, it’s entirely worth
spending your money on.
This release also has Paul Davis and the Ardour team are doing an
support for a major amazing job, creating a powerful open source
audio editor to rival Cubase, Logic or Pro Tools.
external controller, the
Presonus Faderport.
42 www.linuxvoice.com
PRIVACY-CENTRIC DISTRO REVIEWS
Tails 2.0
A new release of this Tor-based distro keeps Mike Saunders safe online.
T
or is a fascinating project. It aims to provide a Web https://tails.boum.org
Developer Tor and Tails teams
certain level of privacy and anonymity online Platforms IA32
by routing traffic through a network of
thousands of relays scattered around the globe, but
it’s being attacked from all sides. Sure, there may be a
few bad eggs using Tor for nefarious purposes, but it
helps countless people in not-so-democratic
While Tor Browser is the
countries communicate and get access to
star of the show, there are
information that would normally be blocked.
other programs included
Installing Tor manually can be rather fiddly, so one for communicating
solution is the Tor Browser, a pre-packaged bundle of securely online, such as
Firefox and Tor. Just launch it and start browsing – it’s Icedove (a rebranded
then very difficult for websites you visit to determine Thunderbird).
who you are. But still, if there’s other unsecure
software on your system (especially if you’re running Icedove (an unbranded version of Mozilla Thunderbird),
Windows), using Tor Browser alone isn’t enough. while the Tor Browser itself has been updated to
Here’s where Tails comes into play. “The Amnesic version 5.5a6.
Incognito Live System” is, as its name suggests, a live We’ve always been fans of Tails, as it really focuses
Linux distro that routes all its network traffic through on making privacy and security accessible to the
Tor. Because it runs in live mode from a DVD or USB masses. This new release upgrades the distro with a
key, it leaves no trace on your hard drive (unless you better desktop environment and email client (nothing
explicitly choose to save files manually), so you can against Claws, but Thunderbird is a better choice
boot up a PC with it, do your work, and then power for less technical users). With certain governments
down the machine as if it hadn’t been used. desperate to monitor everything we do and put back
So, what’s new in Tails 2.0? The biggest change doors into encryption, we need distros like this.
is the switch to Gnome 3 as the desktop. Sensibly,
the Tails team is using the “Classic” mode of Gnome, The switch to Gnome 3 and Thunderbird keeps Tails
fresh, modern and ideal for private browsing.
so that it looks and behaves more like a traditional
desktop. Claws Mail has been junked in favour of
www.linuxvoice.com 43
REVIEWS WINE 1.8
Wine 1.8
Ben Everard messes about with some Windows software – all in a good cause!
T
Web https://www.winehq.org he name Wine originally stood for Windows and changing. Wine is incomplete and probably
Developer Wine Authors
Licence LGPL Emulator – however, this changed when it always will be, but it doesn’t have to be perfect: it just
became clear that the project wasn’t, and has to be good enough to run the software you want
wouldn’t become, an emulator. The project kept the to run, and each release, Wine gets a little closer to
same name, but switched to the recursive acronym matching Windows, and so more and more software
Wine Is Not an Emulator. Instead, it’s a compatibility runs. You can find out whether a particular piece of
layer that enables the user to run Windows software software runs or not by looking at the application
on Linux. database (https://appdb.winehq.org), but bear in
At its heart, any piece of software (whether it’s mind that this is based on people’s experiences and
compiled to run on Linux or Windows), is just made may be wrong or out of date.
up of machine code instructions. These are for the Perhaps the biggest challenge for anyone using
CPU and don’t care about the operating system Wine is working out how to configure it for each
running. The only compatibility problems arise when application. This can be a significant challenge, and
this machine code tries to access the library of code the simplest way around it is to use Play On Linux
that the operating system supplies, as these libraries (POL). This contains configurations for many popular
differ between Linux and Windows. Wine is an attempt applications (as you may guess from the name,
to recreate the Windows libraries (and other APIs) in many of them are games). POL uses known-good
Linux so that executable files for Windows will run in versions of Wine for each application, so you won’t
Linux. The problem for the Wine developers is that this immediately benefit from Wine 1.8 using this until the
set of libraries is huge, complex, poorly documented configurations are updated.
The development of Wine never stops, and version
1.9 is already out, however, this is a development
snapshot rather than a stable release. The next main
version will be 1.10 which is scheduled for release in
December 2016.
Wine 1.8 is the most complete release to date, and
well worth an upgrade if you need to run Windows
software. However, until the configuration is easier, we
can’t recommend vanilla Wine for regular users.
With Wine you can enjoy An excellent way to run Windows software
without having to dual boot, but let down by
classic Windows games complex configuration.
such as Microsoft’s Pinball
on Linux.
44 www.linuxvoice.com
NODEMCU REVIEWS
NodeMCU
Ben Everard tests out a small board with big ambitions.
A
computer with Wi-Fi for £3 including tax and Web www.NodeMCU.com/
index_en.html
delivery might seem like an impossible dream Developer NodeMCU team
– after all, most USB Wi-Fi dongles cost more Price From £3
than this. However, the NodeMCU really does deliver
this as promised with one caveat: it’s not a fully
featured desktop computer, but a microcontroller.
Don’t expect to be able to run your favourite Linux
distro on this, or any other Linux distro for that matter,
but you can program it to run a single application.
The ESP8266 Wi-Fi module
The primary way of interacting with the NodeMCU
provides all features of the
is via a Lua command line interface that you can NodeMCU. Both the
access over the USB port using either standard serial processor and the Wi-Fi
port software (such as screen), or specialist software connection come from
designed for this module such as ESPlora. Using here.
either of these methods, it’s possible to program your
NodeMCU to send and request data over the network. The downside to the NodeMCU is that it’s still
Coupled with the networking functions are eight quite new and not as popular as some other
digital input/output pins, one analogue output, an SPI microcontrollers (such as the Arduino), so there’s
bus and an I2C bus. To make using the module easy, not much other hardware designed to work with
version 2 of the board comes with a USB port for it at the moment. There’s also not much useful
power and communication, so no other hardware is documentation for people new to microcontrollers.
needed to program the device. However, if you have some experience with similar
The connectivity, small size and low cost of the devices and are looking for a Wi-Fi platform, the
NodeMCU make it ideal for building internet-of-things NodeMCU may be the best option available.
devices. For just a few pounds more, you could buy
a whole host of sensors to keep track of almost any A price and performance that is unequalled in
aspect of your environment. The microcontroller is wireless microcontrollers, but better
documentation for beginners is needed.
perfectly capable of running a web server to provide
an interface to any device you can connect it up to.
www.linuxvoice.com 45
REVIEWS GAMING
GAMING ON LINUX
The tastiest brain candy to relax those tired neurons
PASS ME THE WINE Divinity: Original Sin – Enhanced Edition
A true overhaul of a classic formula.
M
any games have claimed to bring the
dated CRPG genre into the 21st
century, but few have really followed
up on those claims as well as Divinity: Original Sin.
From its fantastic fully-3D graphics to its usage
of full voice acting, it is clear that the game
doesn’t simply seek to cash in on nostalgia.
Michel Loubet-Jambert is our Games
Editor. He hasn’t had a decent night’s This is most evident from the incorporation
sleep since Steam came out on Linux. of controller support, something that previously
seemed impossible to do in such a game The game is highly immersive, but does not shy
O
nce a must-have for Linux without dumbing it down significantly. Though away from humour.
gaming, Wine has become there are still needlessly clunky inventory menus
increasingly redundant of
found in CRPGs, navigation is streamlined and pacing that has traditionally been the bane of
late. This is not just due to the
increasing number of native titles easy to use. The drawback of using a controller such games.
available, but also to the lack of though is that text boxes get enlarged during Divinity is still nonetheless very immersive,
DirectX 11 support, which is now dialogue and cover the whole screen, killing with plenty of digressing and sidequesting.
essential for most AAA titles. much of the immersion – something that seems This is enhanced further with the rich, colourful
While the compatibility layer is still
completely redundant given the full voice acting. world, which is second-to-none in the genre.
excellent for older titles that are
unlikely to ever get Linux support, Though the story follows many of the usual Furthermore, the ability to zoom right into
such as Skyrim, gamers are going to fantasy tropes involving dark guilds and plots the characters to the point of a third-person
be bitterly disappointed if trying out to doom the world, it doesn’t come off as overly perspective is a very nice touch, adding even
something like Fallout 4, which dense given the presence of lighthearted gags. more immersion.
doesn’t support DirectX 9. There is
Similarly, the game at no point bombards the
simply no way to run such games, let Website http://store.steampowered.com/
alone well, and with the release of player with countless text boxes describing
app/230230 Price £29.99
DirectX 12 right around the corner, banal details, and thus does away with the slow
Wine is lagging behind.
Support for the API has been in
“experimental” stages for a while
now, and developers have been
working hard to address this elephant
in the room. That said, once support
is available, we’re unlikely to see a
bunch of DX11 games working
overnight. With this in mind, the
software has been taking more of a
“legacy” role for older titles.
While a cross-platform API such as
Vulkan (successor to OpenGL) would
be the ideal long-term solution, and is
likely to gain traction over DX12 (if
only for its cross platform support)
this will also take a while to become
an industry standard.
Even up close, the
With this in mind, Wine is unlikely
imgery is stunningly
to disappear any time soon, since
easily portable AAA games are still a vibrant.
while off, and there will always be a
demand to play classic titles of
yesteryear such as The Sims without The story follows the usual fantasy tropes
the need to install Windows.
involving dark guilds and plots to doom the world
46 www.linuxvoice.com
GAMING REVIEWS
Saints Row IV ALSO RELEASED…
Some crass sandbox silliness.
S
aints Row goes all out with the the game’s brash humour, silliness and
sandbox genre, giving the player parodies of popular culture. Every aspect,
unbridled freedom in its world, down to the intricate character creation,
which includes the ability to jump atop its dialogue and characters, are designed
skyscrapers and run faster than cars. to have the player in stitches.
However, the game’s biggest selling point In the end, Saints Row IV delivers exactly
is also one of its biggest weaknesses, what it promises: a healthy dose of fun.
since these abilities are available near the It is for this reason, along with its very Valhalla Hills
beginning of the game, which gives little attractive price tag, which has made this This nice little city builder requires the player
sense of progression and challenge. game so popular. to expand a Viking settlement up a hill to the
As is expected with the genre, the story point where it reaches Valhalla. Inspired by the
likes of The Settlers, it incorporates strategy
is lacklustre, though this is made up for by Website http://store.steampowered.com/
and resource management elements. However,
app/206420 Price £10.99
giving the player plenty to do and through be warned that a few of its mechanics,
particularly the pathfinding, tend to be a little
on the clunky side and can often be frustrating,
though its graphics are certainly relaxing.
http://store.steampowered.com/app/351910
Saints Row IV doesn’t reinvent
Dreamfall Chapters
the wheel, but instead straps This game perhaps deserves more attention,
fireworks to it. but was plagued with performance issues
which have only recently been solved. It’s a
worthy successor to the Longest Journey
adventure games, featuring a rich story,
Grid Autosport memorable characters and extremely
intriguing world. Though the puzzles can be a
bit hit and miss, the game more than makes up
Easily the most realistic racing simulator to be released on Linux. for it with its excellent plot and narrative.
T
http://store.steampowered.com/app/237850
he car-shaped gap in Linux
gaming is gradually being filled,
and Grid addresses the part of
that gap concerned with realistic driving
simulation, adding to the battle and
arcade racers released recently.
Grid features everything one could
want from a racing game: plenty of tracks
and cars, realistic physics and satisfying The game features many well known cars
graphics. There’s a wide range of classes and tracks to enjoy.
to choose from, including touring cars and Vendetta – Curse of Raven’s Cry
This game has a troubled history, being
endurance. following a crash and display ideal racing
released early last year and universally panned
The online multplayer can be very lines on the track. for its numerous bugs. This re-release
enthralling, but a little intimidating to It’s hard to pick any holes in Grid addresses those issues and demonstrates that
begin with. For the lone wolves, there’s a Autosport. It’s a must have for those more behind all the issues was a very decent RPG
hefty career mode where the player can serious about their racing games, while involving pirates, ship-to-ship combat and a
satisfying open world. However, it still isn’t
advance among the different classes and also recommended to those seeking
without its issues and those looking to dabble
get signed to bigger and better teams something more casual. in a bit of piracy may want to wait for a sale or
as the career progresses. This can be price drop.
challenging, but for casual players there Website http://store.steampowered.com/ http://store.steampowered.com/app/386280
app/255220 Price £24.99
are options such as the ability to rewind
www.linuxvoice.com 47
REVIEWS BOOKS
Beyond legacy code
Ben Everard avoids legacy code problems by using low-quality hard drives.
Author David Scott Bernstein
Publisher Pragmatic Bookshelf
Price £25.50
ISBN 978-1680500790
L
egacy code refers to the vast number out the importance of actually being agile,
of programs running today that not just adding an agile veneer to outdated
are hard to maintain and prone to software development methodologies. His
breaking. They’re a huge source of problems approach is summed up in nine practices
for anyone working in the IT industry, from that, when properly understood, ensure that
developers who have to wrestle with them in not only is agile development used, but its
order to add new features to sysadmins who benefits are actually realised.
have to keep old distros running because Bernstein’s advice is based on a huge
the legacy code relies on an end-of-lifed amount of experience, and it’s not only
package. Beyond Legacy Code isn’t a book easy to follow, but easy to understand.
about how to solve the problem with the This is important because it’s only with this
legacy code we have now, it’s about how to understanding that the principles can be
solve the problem we will have with legacy properly applied to each project.
code in the future. In other words, it’s about
Fewer than half of all software projects are
writing code that will stand the test of time successful. If more people followed the
and remain useful for years. book, that would increase dramatically.
David Bernstein prescribes an agile Learning to write easy-to-maintain code is a gift
approach to this problem. He is keen to point to your future self.
Reactive Programming with RxJS
Ben Everard attempts to code using Newton’s third law of motion.
Author Sergi Mansilla
Publisher Pragmatic Bookshelf
Price £12.18
ISBN 978-1680501292
D
on’t be confused: this is a book wastes no time in getting down to the
about JavaScript with the word nitty gritty of RxJS and how it can create
Reactive in the title, but it’s not readable, clean code where before lay a
a another book about building reactive mess of callbacks and event listeners.
websites. RxJS is about a the reactive While Reactive Development exists in
style of programming (this is not unique to most languages, this book is probably a little
JavaScript; it originated in .NET), rather than too tied to JavaScript to be useful to anyone
website design. In this style of programming, looking to gain the advantages in other
data is dealt with in streams that can be languages. However, Reactive Programming
filtered and acted upon. These streams with RxJS does cover RxJS in Node.js
(known as observables) are particularly as well as the browser, so with this one
effective for dealing with asynchronous data tome, you gain the skills to implement the
sources. programming style both on the server and
Reactive Programming with RxJS focusses for your end users.
specifically on Reactive development
rather than on general JavaScript, so the
A straightforward guide to get you
reader has to be familiar with the language, started with RxJS quickly.
and it will help if they have experience in RxJS: Code as neat as a knot of rainbow-
asynchronous programming. Sergi Mansilla coloured cables.
48 www.linuxvoice.com
BOOKS REVIEWS
Badass: Making Users Awesome
Graham Morrison finally gets another Kathy Sierra book. Also released…
Author Kathy Sierra
Publisher O’Reilly
Price £19.99
March 2016
ISBN 978-1491919019
Building a Quadcopter
W
e’ve been huge fans of
Kathy Sierra for a long This book promises to go
through the nuts and bolts
time. We first discovered
of building a quadcopter
her work through O’Reilly’s ‘Head First’ based on the Arduino
series of books, which Kathy helped micro-controller. We really
to formulate after co-authoring the like this idea, because
first title, Head First Java, back in 2003. quadcopters are becoming
very popular, which has
Written by Kathy and her partner,
made the hardware cheap
Bert Bates, it combined her extensive and accessible. Putting
Java training experience (from one together yourself or
Sun Microsystems) with her ability as a project with your kids
maximise attention and concentration, “Too often, the goals of a company and is a great way of breaking
the goals of its users aren’t just different down a complex problem Build your own open
perhaps thanks to her prior experience
into simple parts. source quadcopter!
as a games programmer. but mutually exclusive.”
That first book contained many
of the elements that have gone on to stepped out of the limelight, and it
make the series so useful. But mostly, has understandably taken her a long
it was successful because of what time to venture back – first with some Android Programming
it did not contain: in comparison to excellent blog posts, then with some This is a book aimed at
the average O’Reilly book, there were presentations, and now finally with beginners, written by an
far, far fewer words. There were no a new book very much in the Head author who we like, with
lengthy essays, no wordy diatribes on First… style. plenty of experience. This
is all good news, because
language syntax, no backgrounds or Badass: Making Users Awesome is mobile operating systems
biography and definitely no reference a worthy return, and it’s wonderful to like Android are the new
material. Each book was composed see her and her trademark teaching semi-closed PC platform.
of simple ideas linked together by approach back in print. Badass doesn’t The more of us writing
meme-like images, phrases, charts take on the complexity of a language open source software for
it, the better the future of
and remarkable insight. These like Java, but it does attempt to the platform and our
components were sewn together to answer a vital question relevant to hardware. It’s also a great
engage the reader from the beginning many of us, “Given competing equally- way of making you more Need a new career? Why
right through to the end. priced, equally- promoted products, employable. not try Android coding?
In theory, tackling big subjects like why are some products far more
Java in this way shouldn’t work. It’s a successful than others?”.
complex language that can take years Whether your product is an app, a
to master. But Head First… succeeds, game, a blog post or a print magazine, BIND/DNS Admin Reference
not because it’s a sugar rush of Badass takes an othogonal approach
This month’s sysadmin
quick and clever ideas, but because to helping you understand how best to pages, at the back of the
it understands the limitations of reach your audience, purely from your magazine, dive into the
teaching a subject through words. The potential user’s/reader’s perspective. It complex world Domain
writers probably assume, for example, does this by persuading you to forget Name Servers and the
that most people who start a complex the ‘brand’ and concentrate on making Berkeley Internet Name
Daemon, two fascinating
O’Reilly book don’t finish a complex your users ‘feel’ awesome after using and powerful services that
O’Reilly book. Yet most of us shouldn’t your creation. Only then will they hold the web together.
have trouble finishing any Head First… become your ‘badass’ users and help There are plenty of books
title, and even re-reading it several your endeavour to succeed. It’s a noble on the subject, for further
times over. And that’s the series’ killer approach that’s utterly right. reading, but this second
edition is the latest if you
feature; accessibility via a ‘natural want to take your DNS It’s difficult, but that
language’ style of teaching. learning further. makes it special.
A fabulous read, although it’s unlikely to
After suffering serious online make us awesome overnight.
harassment in 2007, Kathy Sierra
www.linuxvoice.com 49
GROUP TEST FREE SOFTWARE LICENCES
GROUP TEST
If you’re starting a new Free Software or open source project, you need
to pick an appropriate license for it. Here are our recommendations.
On test Free software licences
I
n the proprietary software rather than take them away, and
GNU GPL world, very few people read they’re usually written with real
URL www.gnu.org/licenses/ software licences. Many of us people in mind and not just lawyers.
gpl-3.0.en.html who occasionally spend time on They’re shorter, they’re clearer,
Originator Free Software Foundation Windows or Mac OS X are familiar and they try to explain how they
The license behind the Linux kernel and with huge EULAs (End User Licence want to help developers, users and
other big-name projects.
Agreements) which go on for (sometimes) software vendors
thousands of words and contain benefit from access to source code.
Affero GPL ridiculous amounts of jargon and But there are many licences –
URL www.gnu.org/licenses/agpl.html legalese. So virtually everyone just and the differences can have a big
Originator Free Software Foundation clicks “I accept” without paying impact on the success of an open
Like the GNU GPL, but with extra clauses proper attention. source project. So for this month’s
to make sure our freedoms are not In fact, the situation is so bad Group Test, we thought we’d step
curtailed by “software as a service”. that one company, PC Pitstop, put a aside from software and look at
note in its software’s EULA saying: the licences behind it. It’s good to
BSD Licence the first person to email a specific
address will get $1,000. It took four
be aware of the differences as a
regular Linux users, but it’s even
URL www.opensource.org/licenses/BSD-
months and over 3,000 downloads more important if you plan to start
3-Clause
Originator Regents of the University of before someone actually read the an open source project in the future,
California EULA in full and requested the prize. or at least contribute to one. We’ll
An open source licence allows code to be Here in the GNU/Linux world, focus on picking out the ones
pulled in to proprietary apps. it’s very different. Our licences are that are the most beneficial to our
intended to protect our freedoms, software ecosystem as a whole.
Artistic Licence 2.0
URL www.perlfoundation.org/artistic_
license_2_0 There are many licences, and the
Originator The Perl Foundation
Written by Larry Wall and used for many
differences can have a big impact on
Perl implementations and modules. the success of a project
GNU Lesser GPL Free Software vs Open Source
URL www.gnu.org/copyleft/lesser.html
Originator Free Software Foundation The terms “Free Software” and “open term that was developed in the 1990s
A version of the GPL that lets source” may seem interchangeable, but to make the concept of Free Software
Free Software libraries be used by there are some subtle differences in their more business friendly. It focuses on
philosophies. the practical aspects of software whose
proprietary software.
Free Software is the name used by source code you can view, change
the Free Software Foundation and GNU and distribute: better reliability, more
Mozilla Public Licence project from their early days, and it
means “free as in speech, not as in beer”.
security, and more eyeballs looking at
the code. Many people use “FOSS” (Free
URL www.mozilla.org/en-US/MPL/2.0/ The Free Software Foundation is keen and Open Source Software) as a term to
Originator Mozilla Foundation to stress the importance of freedom, encapsulate both approaches. You may
Used by Firefox, Thunderbird, community and sharing in its software even come across “FLOSS”, where the “L”
LibreOffice and other apps, aiming to – social benefits that improve the world. stands for “libre” – emphasising freedom
bridge the gap between GPL and BSD. Open source, on the other hand, is a of speech and not just zero cost.
50 www.linuxvoice.com
FREE SOFTWARE LICENCES GROUP TEST
A single-clause licence
Do What the ‘Flip’ You Want to Public Licence.
M
ost of the licences in this Group with your work (especially if it’s longer than In other words, no worrying about rights
Test are geared towards large the code itself)? for distribution, modification, linking,
Free Software and open source Here’s where you could use the WTFPL, patents or anything like that – developers
projects with multiple developers and or Do What the ‘Flip’ You Want to Public can simply take the code and do anything
thousands or millions of lines of source Licence. (You’ve probably gathered by now with it. Effectively, it’s very much the same
code. But what if you’ve written some small that ‘Flip’ is actually another word in the real as putting your code in the public domain.
snippet of code that you want to share with licence text…) It’s a single-clause licence that It’s not very popular, though, as most open
the world, but you don’t want the complexity has a tiny preamble and then says: source developers still want some credit for
of picking a licence, making sure all your 0. You just DO WHAT THE ‘FLIP’ YOU WANT their work, even if they don’t want to impose
code is compliant with it, and distributing it TO. any other restrictions.
GNU GPL
Freedom through copyleft.
T
he GNU project has contributed so GPL, it will always remain open
much to what we call ‘Linux’ today for anyone to download, study,
(hence the use of ‘GNU/Linux’ by modify and redistribute
some advocates): the GNU C library, GCC But what part guarantees
compiler suite, Emacs and many other major these freedoms? Well, Stallman
software components that make up a free craftily used copyright law –
operating system. But equally important is usually intended to prevent people
the licence that GNU created: the General from copying material – and used
Public Licence. This is not just a piece of text it to ensure that freedoms can’t be
describing what you can and cannot do with taken away. Because GPL-licenced
source code, but a powerful document software is copyrighted (eg by the
taking important social and ethical positions, program’s author or by the Free Software
with a goal of preserving and furthering our Foundation), and the GPL is a copyrighted
software freedoms. document as well, users of GPLed code
Back when Richard Stallman originally cannot omit any of the four essential rights
developed the GPL, the main other ‘open when modifying or distributing it.
source’ licence was the BSD Licence. This
was just a few paragraphs saying: here’s The four freedoms The GNU Project has brought us many awesome
some source code, do what you want with And these four rights are: the right to run components for a free OS, but also a philosophy
it, but give the original developers credit the program as you wish; the right to study and licences to match.
(we cover this licence and its impact in how it works and modify it; the right to
detail later). While this licence provided distribute copies; and the right to distribute Ultimately, the GPL is a rather long
developers with plenty of freedoms, it didn’t your modified version. The GPL guarantees document at 5,645 words (although still
forcibly preserve those freedoms. Anyone that these freedoms are preserved, which is far shorter than many proprietary app
could take BSD-licenced code and tuck it a boon for most of us, but can cause issues EULAs), and some developers take issue
away in proprietary software, so the original for some developers working on projects with its political stance and “enforcement”
developers couldn’t benefit from updates. containing a mixture of open source and (rather than optional offering) of freedoms.
Right from the start, the GPL takes a proprietary code. But we think it’s an incredibly well thought-
different approach. One of its opening lines Indeed, many proprietary companies have out document that has made the FOSS
is: “The licences for most software and put into place a “no GPL” policy – don’t even community so strong, and just keeps getting
other practical works are designed to take look at or touch any GPLed code, because if more useful with each year.
away your freedom to share and change it somehow ends up in our product, we may
the works. By contrast, the GNU General have to release the whole thing under the VERDICT
Public Licence is intended to guarantee your GPL. We don’t find this a big issue, however The GPL embodies the
principles of sharing and
freedom to share and change all versions – the GPL is all about our freedoms as users community that are so
of a program.” This “guarantee” is crucially and hackers, and not to make life easier for vital to Free Software.
important: if you release your code under the megacorps.
www.linuxvoice.com 51
GROUP TEST FREE SOFTWARE LICENCES
Artistic Licence
As used by Perl and many modules.
P
erl may look rather old-school Super Nintendo emulator already use
today when everyone is raving this newer version.
about hot and trendy languages So, what does the licence specify?
such as Go and Rust, but it’s still an Well, it’s very heavily focused on what
important language that’s doing plenty you can do with modified versions
of work on servers around the world of a program. You’re allowed to take
– especially in text processing jobs. some code released under the Artistic
Some coders criticise Perl for looking Licence and make another program
like line noise, whereas others praise out of it, providing that you “clearly
how much functionality the language document how it differs from the
can pack into just a few characters. standard version”. In other words, you Perl 6 will use version 2.0 of the Artistic Licence. Here’s
Perl is released under the Artistic demonstrate what you’ve changed, the language’s cheerful mascot, Camelia.
Licence, a curious text that has gone what you’ve added, what features
through some major modifications you’ve implemented and so forth. Perl, for instance, you need to make
over the years. The original 1.0 version You must also ensure that users sure that your new version doesn’t
received a lot of criticism for being can run both the original version of the trample all over the official release (and
too vague in places; Bradley Kuhn software and your modified version Perl programs on a freshly installed
(as interviewed last issue) worked simultaneously. So if you decide to fork Linux installation don’t run your version
with the Perl team and Free Software by default).
Foundation to create the Clarified It’s focused on what you
Artistic Licence, which is the current VERDICT
one used by Perl. Future versions of can do with modified Worth using if your code
is likely to be modified or
Perl will use Artistic Licence 2.0, and
some other apps such as the SNEsE
versions of a program forked in future.
Mozilla Public Licence
As used by Firefox and LibreOffice.
A
s we’ve seen, the GPL is the Software developers. So the Mozilla
best choice of license if your Public Licence (MPL) was born, and it
priority is freedom for end takes a clever approach: code that is
users – that is, the freedom to study licensed under the MPL must remain
and modify all parts of your software, under the MPL, even after modification,
and these freedoms cannot be taken so it always remains free like in GPLed
away. The BSD Licence, in contrast, code. But! It is possible to mix MPLed
focuses more on freedom for code with proprietary code, creating a
developers; specifically, the freedom to proprietary product. If you make such
use code in proprietary products if a product, you don’t have to release the
desired. Both of these licences serve proprietary code you wrote yourself, but
their target groups well, but is there a you must make available the MPL code
middle point? you used along with any changes to it.
The Mozilla Public Licence may be Want to keep your code open, but don’t mind it being
the solution here. This was originally Caudine fork combined with proprietary code? Mozilla has the solution.
written by Mitchell Baker of Netscape In this way, commercial developers can
back in the late 90s, as the browser use MPLed code such as the source code created by the community, we
developer was toying with the idea tree for Firefox, LibreOffice and other must get the changes back to benefit
of going open source. In its first flagship open source apps, and build us too.
incarnation as the Netscape Public products on top of it. If they add their
Licence, it allowed code developed own separate features, they don’t have VERDICT
A healthy compromise
in the open source community to be to release the code for those, and they between Free Software
incorporated into proprietary products, get a commercial benefit over the and proprietary.
which didn’t win applause from Free competition. But if they modify MPLed
52 www.linuxvoice.com
FREE SOFTWARE LICENCES GROUP TEST
BSD Licence Another perspective
Permissive and proprietary friendly.
on BSD
How the licence encourages
adoption.
O
ne argument in favour of BSD-style
permissive licences that often comes
up is adoption of key technology and
standards. If you want to get the whole world
using a certain protocol, library or piece of
software, you want to use the licence that’s
palatable to as many people as possible.
One example of this is OpenSSH, the
secure remote shell tool ubiquitous in every
modern-ish operating system (even Microsoft
is starting to use it). BSD fans have often
claimed that the permissive licence has helped
with its adoption, because any company can
start using it without having to think of legal
issues or releasing changes back to the world.
OpenSSH is a well tested and engineered
The FreeBSD operating system is the most prominent example of a BSD Licensed project. piece of software, and thanks to the BSD
licence, very few people have a beef using it.
L
ike the GNU GPL, this licence was spin-off. Note that unlike the GNU GPL, Had it been released under the GPL, a lot of
originally drafted with a specific there is no clause here saying that source companies and software vendors could have
operating system in mind, although code must be supplied. It’s a permissive had concerns using it, and so created their
today it is used by thousands of projects. licence – there are very few requirements own versions and we’d have lots of potentially
BSD refers to the Berkeley Software compared to the GPL. insecure, slightly incompatible versions all over
Distribution, a flavour of Unix that was the internet. Maybe some companies would
developed at the University of California, Who wins in the end? have forked older BSD Licensed versions and
Berkeley, from 1977 to the mid 90s. Now, the side effect of this licence is clear: the situation would be a jolly big mess.
Although BSD itself is no longer any company can take BSD-licensed code But as it is, OpenSSH has become the de
developed, a handful of forks developed and incorporate it in a proprietary product. facto standard for command line access to
from it in the early 90s, most notably This has happened many times, such as remote machines, and with pretty much the
FreeBSD, OpenBSD and NetBSD (famous when Microsoft built a networking stack whole world using (and examining) the same
for being portable to almost anything). for Windows using BSD code, or more codebase, we have a strong project with a
As it stands today, the licence has recently with the PlayStation 4, which runs good security track record and no major forks.
three requirements. You’re allowed a modified version of FreeBSD. For some
to redistribute the software in source people, this makes the BSD licence
and binary formats, with or without severely flawed – after all, why should we
modification, providing that: help companies that often don’t provide a
1
The source code contains a copyright penny or single line of code back?
notice and disclaimer (the latter saying Well, it depends on your perspective.
that the software is provided as-is, and The BSD Licence is arguably better
with no warranty). than the GNU GPL in that it gives other
2
If distributed in binary only (ie developers increased freedom and
proprietary software), the same choice (although at the expense of user
copyright notice and disclaimer is freedoms). So if you’re writing some code
included in the documentation. and simply want to get it out there and
3
The names of the original developers used as much as possible, without any
cannot be used to endorse or promote philosophy or politics, it makes sense.
software derived from it without specific
written permission. VERDICT
A minimal, simple choice
So in short: do what you want with it, if you just want to get The BSD Licence has helped one open source
give us credit for writing it, don’t sue us if your code out there. project to utterly dominate in a certain task.
it breaks, and don’t claim we support your
www.linuxvoice.com 53
GROUP TEST FREE SOFTWARE LICENCES
GNU Lesser GPL vs Affero GPL
What do these two derivatives offer?
R
ichard Stallman and the Free
Software Foundation often get
flak for not budging one inch
when it comes to other philosophies
and ideas. But we think this is unfair,
and one example of when the FSF has
been able to compromise is the GNU
Lesser GPL (aka LGPL).
Essentially, the LGPL is very much
identical to the regular GPL, but
includes an exception: it lets proprietary
software link to LGPLed software so
that the former can use the facilities of
the latter. This is especially important
in terms of libraries – collections of
software routines that an end-user
doesn’t run on their own, but which
provide support for other programs.
For instance, the GNU C Library
(Glibc), which provides various input,
maths and other routines, is released
under the LGPL. So a proprietary
program can link to it and use its The GNU C Library is an important component of the GNU/Linux system, and is released under the LGPL.
routines, without that program having
to be Free Software (released under the proprietary app developers? Well, when your computer, they must (on request)
GPL) as well. Many other libraries have a Free Software library doesn’t offer give you the source code and licence
been made available under the LGPL, many more features than a proprietary as well. But what if you’re not running
so it’s not uncommon to have a piece of one, it’s the lesser of two evils if a GPLed software directly, but over the
proprietary software that spends most proprietary app uses the free library. internet? Think of all the web apps out
of its execution time inside routines there, like web-based email clients:
provided by LGPLed libraries. The Affero alternative you’re not technically running the
Now, given that the goal of the GNU The Affero licence tackles a different software on your machine, but merely
project is to have a completely open problem. If someone gives you the viewing the results of it running on a
source operating system, why cater to binary of a GPLed program to run on remote one, so should you then be able
to request access to the source if that
code is GPLed?
Long before the emergence of cloud
computing, which brings up this very
issue, Richard Stallman and others
had noticed that it could become a
problem. So work began on the Affero
Public Licence (aka AGPL), which is
very similar to the standard GPL but
includes an extra clause dealing with
“remote network interaction” and says:
if people interact with your AGPLed
software over a network, they have the
right to request the source code to that
software – and at no extra charge.
VERDICT
LGPL Makes some AFFERO A simple and
compromises to effective solution to
proprietary developers the use of GPLed
for the greater good. software in web apps.
Launchpad, Canonical’s service to help developers collaborate, is released under the AGPL.
54 www.linuxvoice.com
FREE SOFTWARE LICENCES GROUP TEST
OUR VERDICT
Free software licences
W
hich license is GPL, the world would look very
intrinsically ‘best’ is very different today. Sure, we’d still have
much a matter of some open source projects like
opinion, and we know that some the BSDs, but thanks to the GPL
readers may disagree with our we have an enormous tapestry of
findings here. And ultimately, source code out there that was, is,
different types of project have and always will be free. The GPL
different licensing requirements, so makes us think about the social
we won’t say that one is simply benefits of Free Software, not just The GNU General Public License does the most to preserve and
worth using over another in every the practical ones, so for that it promote our freedoms in the long run.
single possible circumstance. What should be praised highly.
we at Linux Voice can say, however, Of course, the other licences
is that the GNU GPL is the one that have their benefits as well. Although 1st GNU GPL
has really helped us to get where we regard the GPL as the ultimate
we are today. licence for our freedoms, the BSD www.gnu.org/licenses/gpl-3.0.en.html
Yes, it’s wordy. Yes, it’s political. Licence is still a great choice when A cornerstone of the modern Free Software movement, and a
And yes, it causes consternation you simply want your code to license that puts users’ freedoms at the forefront.
for some software developers who be used anywhere, regardless of
also work on proprietary products. whether the end result is open or
But it’s not some crusty old text that closed. The BSD Licence is clear, 2nd BSD Licence
we still use out of habit: it’s a very simple and short, and in many
well thought-out document that cases companies that use BSD www.opensource.org/licenses/BSD-3-Clause
deals with today’s concerns despite code still contribute their changes Has very few requirements and permits use in proprietary
originally being written decades back (such as LLVM/Clang). products. But as we’ve seen, that’s not always bad.
ago. Richard Stallman foresaw how Artistic, Mozilla, LGPL and AGPL
the software world would develop,
and created a licence to protect his
all have their goals and purposes
too. But ultimately, the original GPL
3rd Mozilla Public Licence
work against it. embodies the spirit and community
www.mozilla.org/en-US/MPL/2.0
Without a powerful, copyleft, of Free Software to the largest
A clever licence that helps to bridge the gaps between two
freedom-centric licence like the extent, and that’s why we love it. approaches to software development.
Without a powerful, freedom-centric
4th Affero GPL
licence like the GPL, the world would
look very different today www.gnu.org/licenses/agpl.html
An important fork of the GPL that deals with something we’re
seeing more of: apps running in our browsers.
The MIT Licence
Some readers may be wondering why we
haven’t included the MIT Licence here,
original developers cannot be used to
endorse software based on it. So it’s
5th GNU Lesser GPL
given that it has become one of the most a slightly simpler version of the BSD
popular for open source development in Licence, and is otherwise the same. Is www.gnu.org/copyleft/lesser.html
the last few years. Well, we’ve omitted it it a big deal? Well, in most cases, no. If GPLed software is the best, we feel, but sometimes an LGPLed
for the simple reason that it’s almost the you have a popular piece of MIT Licenced library is the lesser of two evils.
same as the BSD Licence, but with one source code and someone makes a fork
clause removed. Like the BSD Licence, that’s utterly rubbish but says you’re
the MIT Licence says you can do what
you want with the software, there’s
championing it, very few people will
believe that person.
6th Artistic Licence
no warranty provided with it, and you If you’re working on something new,
must include a copyright notice and the however, and don’t want lots of half- www.perlfoundation.org/artistic_license_2_0
licence with the software. quality forks cropping up with people This licence has worked well for Perl and many of its modules
However, the MIT Licence omits the using your name to promote them, the over the last few decades.
clause stating that the names of the BSD Licence is a more sensible choice.
www.linuxvoice.com 55
SUBSCRIBE
Subscribe
shop.linuxvoice.com
Introducing Linux Voice,
the magazine that:
Gives 50% of its profits
back to Free Software
Licenses its content
CC-BY-SA within 9 months
12-month subs prices
UK – £55
Europe – £85
US/Canada – £95
ROW – £99
7-month subs prices DIGITAL
UK – £38 SUBSCRIPTION
Europe – £53
US/Canada – £57
ONLY £38
ROW – £60
Get 100 pages Access our Save money on
of tutorials, rapidly growing the shop price
features, interviews back-issues archive and get each issue
and reviews – all DRM-free and delivered to
every month ready to download your door
Payment is in Pounds Sterling. 12-month subscribers will receive 12 issues of Linux Voice a year. 7-month
subscribers will receive 7 issue of Linux Voice. If you are dissatisfied in any way you can write to us to cancel your
subscription at subscriptions@linuxvoice.com and we will refund you for all unmailed issues.
56 www.linuxvoice.com
NEXT MONTH
NEXT MONTH IN
ON SALE EVEN MORE AWESOME!
THURSDAY
SASS
25 FEBRUARY Like the idea of CSS
(and the attractive,
consistent web
layouts it produces)
but hate the arbitrary
fiddliness of it? We
have the tool for you
and your website.
Media players
Find the tools Free
Software has to offer
with which to
depress yourself
watching Truly Madly
Deeply and console
yourself listening to
side 1 of Low.
Image: iStock
Fosdem
Mike Saunders
HACK EVERYTHING
braves waffles, beer
and moules-frites to
report on Europe’s
The hackers don’t stand still, and neither biggest FLOSS
should you – learn their tricks to keep gathering. Will he
come back in one
yourself safe online. piece? Find out!
LINUX VOICE IS BROUGHT TO YOU BY
Editor Graham Morrison Editorial consultant Nick Veitch through the use of advice in this magazine. Copyright Linux is a trademark of Linus
graham@linuxvoice.com nick@linuxvoice.com Experiment with Linux at your own risk! Torvalds, and is used with permission.
Deputy editor Andrew Gregory Distributed by Marketforce (UK) Ltd, 2nd Anything in this magazine may not be
andrew@linuxvoice.com All code printed in this magazine is licensed Floor, 5 Churchill Place, Canary Wharf, reproduced without permission of the editor,
Technical editor Ben Everard under the GNU GPLv3 London, E14 5HU until September 2016 when all content
ben@linuxvoice.com Tel: +44 (0) 20 3148 3300 (including our images) is re-licensed
Editor at large Mike Saunders Printed in the UK by CC-BY-SA.
mike@linuxvoice.com Acorn Web Offset Ltd Circulation Marketing by Intermedia Brand ©Linux Voice Ltd 2016
Creative director Stacey Black Marketing Ltd, registered office North Quay ISSN 2054-3778
stacey@linuxvoice.com Disclaimer We accept no liability for any House, Sutton Harbour, Plymouth PL4 0RA
loss of data or damage to your hardware Tel: 01737 852166 Subscribe: shop.linuxvoice.com
www.linuxvoice.com 57
FOSSPICKS
FOSSpicks Sparkling gems and new
releases from the world of
Free and Open Source Software
Out benevolent editorial overlord Graham Morrison tears himself away
from updating Arch Linux to search for the best new free software.
Circuit simulator
Hardware designer
M
ore of us than ever you’re using. They’re roughly to PCB layout, and while it will
before are now messing analogous to IDEs for struggle with large projects it’s more
around with electronics, programmers, with automated than capable of handling
whether that’s building a 3D printer layout and component libraries homegrown projects. There’s even a
for your local hackspace or creating replacing syntax highlighting and 3D viewer, which is perfect for
your own circuits for short API reference material. studying other designs or
production runs. But unlike learning visualising your own prototypes.
to program, where mistakes are a Better by design The only disadvantage is that these
part of the learning process, the KiCad is one of the most capabilities come with complexity,
consequences for mistakes in comprehensive EAD suits we’ve and it’s a difficult suite to get started
circuits can be more serious. found. It’s got a long history – the with. The schematic designer is
Circuit simulators (or Electronic initial release was in 1992 – and very powerful, for example, letting
Design Automation suites) can help recent development has been you drag and drop components into
with this, not only by error checking partly funded by CERN. your designs and change their
your designs, but by understanding It’s capable of serious results too, values easily, but you still need a
the properties of the components taking you from schematics design good grounding in electronics for
those circuits to make sense, even
with the Electrical Rules Checker,
which checks for logic errors
against component and pin types.
2 Even if you are a beginner, KiCad
1 is a worthwhile installation, and this
being open source, many designers
have shared their own schematics
3
which can be altered relatively
easily. Version 4.0 is a major update
and includes OpenGL accelerated
Cairo-enabled rendering, new
libraries and the excellent 3D board
rendering, which looks brilliant on
4
7
5 complex circuits. There’s also
CERN’s interactive Push and Shove
6
router, which has been part of the
alpha/beta since 2013. This is for
advanced users, and way above our
electronics ability, but it promises
automatically ‘pushed’ track routing
when creating your own designs,
1 Project Overview Each project contains lots of different elements, from the schematics to a Bill of Materials
which at least looks really
2 Library You can install additional component libraries that list capability and common part numbers
3 Schematics Designer Drag and drop components and join them together to make circuits, then test them using the ERC
impressive in the YouTube videos
4 Router The push and shove router can make sure your connections are placed as efficiently as possibly explaining its function.
5 3D View Visualise your project, including all your components, in lovely 3D OpenGL 6 Circuit Diagrams Dropped
PROJECT WEBSITE
components from the library can be connected to each other, or imported from other libraries
http://kicad-pcb.org
7 PCB Calculator Great for quick reference and for checking and calculating the values of any components you need
58 www.linuxvoice.com
FOSSPICKS
Latex editor
Texmaker 4.5
L
atex is a brilliant tool for xelatex generated predictable
styling documents, despite output from our Latex files first, for
sometimes feeling more like example. This is one of the
programming than authorship. It’s configuration paths that Texmaker
capable of stunning results, and can use to generate its previews.
unlike the output from a word But with everything installed,
processor, layouts are dynamic, creating documents is as a simple
responsive and adaptable, as editing HTML.
especially when citing or cross- The preview window defaults to
referencing other documents. the right, while the text editor lives
The package for Arch
While some purists write Latex on the left. Clicking ‘Quick Build’ document. The vast majority of
Linux is the only one to
from Emacs and Vim, there are generates a PDF by default, which have made the leap markup options are easily
several great graphical applications is then previewed. For most from Qt 4 to Qt 5, accessible, with the most common
that make things easier, with documents this is almost although you can build embedded within the left-hand
Texmaker being our current instantaneous, and Texmaker will your own from source if border of the editor itself, and many
favourite. It doesn’t exactly make highlight any problems it finds, you want to. more are listed in the border of the
Latex easy – you’ll still need to hunt such as text taking up too much structure pane. This really helps if
down and install your own space or syntax errors in your you’re just beginning with Latex, as
packages and dependencies, and you don’t need to worry about
the real time previews within the
application depend on a working
Creating documents in memorising syntax or using two
windows for writing.
configuration outside of the Texmaker is as simple as
application. We spent some time PROJECT WEBSITE
making sure the command line tool editing HTML www.xm1math.net/texmaker
Screen capture
Spectacle
T
here’s been little recent Spectacle. Spectacle has been
innovation in screen freshly developed to take advantage
capture tools. This might of the new KDE 5 frameworks, and
seem a little unfair considering their was originally called KScreenGenie.
basic and utilitarian function, but Spectacle’s developer, Boudhayan
we feel there’s still plenty of Gupta, had intended to simply clean
potential for new features, up KSnapshot for the new KDE
especially as we typically take versions, but thought it better to
dozens of grabs a week. start from scratch after finding bits
Our screengrabber of choice, at of code as old as KDE 3.5 in its
least for the KDE desktop, is the source tree.
default KSnapshot. It does
The only problem we
everything we need quickly and Taking pictures have with replacing
such a problem while Wayland
efficiently – it’s triggered by a Despite the complete overhaul, it’s KSnapshot with remains in beta, but will become
hotkey, enables you to grab the almost functionally identical to Spectacle is important as more KDE users
contents of the entire screen, a KSnapshot. The only differences we remembering switch over to the already functional
window or a draggable rectangle, can spot are a neater GUI and more Spectacle’s name. technology preview supported by
and saves consecutive images naming options for automatically the last few KDE releases. At least
without interaction and with saved screenshots. It’s potentially Spectacle has this potential,
sensible file names. capable of taking shots of other whereas KSnapshot does not.
But KSnapshot’s days appear to desktop back-ends, such as
be over. A recent KDE 5 update has Wayland, though support for this PROJECT WEBSITE
https://github.com/KDE/spectacle
swapped it with a tool called hasn’t been added yet. This isn’t
www.linuxvoice.com 59
FOSSPICKS
Command trigger
entr(1)
E
vent Notify Test Runner – detects changes in the filesystem.
entr – is one of those You could, for instance, force a
command line tools that document viewer to reload when a
sticks to tradition – doing one thing file is updated, or rebuild a project
well so that lots of other tools can from source when the code is
be augmented with its functionality. modified. Or trigger an automatic
The old-school charm even applies backup. Or copy files to a remote
to the project name, where the server. Or transcode a new video or
entr will watch for
developer has included the ‘(1)’ to audio download from one format to this is cat hello.txt. This simple
changes or new files in
show where the man page another. Or shut down your system a folder and execute a construction will display the
documentation should be found on after a file appears. command when a updated contents of the hello.txt
your system, and subsequently, What’s more impressive is that change is detected. file whenever the file is changed.
what category of tool you’re using. it’s really easy to use. Typing ls Additional arguments include -d for
(1) is for general commands such hello.txt | entr cat hello.txt will ask watching a directory of files, -p to
as this, for example, while (6) is for entr to watch the piped output from avoid running the output command
games like Nethack and (8) for the ls command, and when a until a file is modified, -c to clear the
system administration. It’s a quick change is detected, run the display before running the output
way of knowing what kind of tool following command. In our example and -r to reload a persistent child
you’re looking at, and entr definitely process. With these simple
fits into the general category. When
you understand what it can do,
entr enables you to run a additions, you can automate almost
any task with a single command.
there’s no limit to its possible uses. command when it detects PROJECT WEBSITE
Put at its simplest, entr enables
you to run a command when it changes in a filesystem http://entrproject.org
Compiler
Free Pascal Compiler 3
R
etro gaming is big business. fundamentals of programming. Like
There are online stores built BASIC’s revival, it’s just as good at
specifically to sell old teaching those procedural
games, and re-issues and concepts today as it was in the
remasters of classic titles seem to 1980s. Which is where this major
be released every week. The same update to the Free Pascal Compiler
is also true of old computers, comes in. Unlike Turbo Pascal, it’s
operating systems and languages. just a compiler. There’s no IDE or
Or perhaps, more accurately, the editor for working with Pascal, but it
Relive those magical
decline of old programming will turn your source code into an decade chasing after languages like
days where your code
languages just hasn’t happened. executable binary. And it can build needed to be compiled JavaScript, C# and Swift. Version 3
Lots of us still play with old binaries on almost any platform, and linked before being of FPC, as it’s known and used on
languages for a variety of reasons. from the Raspberry Pi to the executed. the command line, is a big update
Our recent series on x86 assembler Nintendo Gameboy. that has excellent compatibility with
has been incredibly popular, for Turbo Pascal 7, and while we’ve
example, despite assembler having Learn from the past joked about Pascal’s age, we know
limited modern uses. Free Pascal Compiler also comes it’s still being used in many serious
Pascal is another language worth with a wonderful set of applications and projects, which
reminiscing over. It’s still used in documentation, and while really gives you the perfect excuse
lots of places, but most of us programming in Pascal does to revisit this gem of a language.
remember it fondly as Turbo indeed bring us a sense of
Pascal. It was used in many nostalgia, it’s also educational, PROJECT WEBSITE
www.freepascal.org
universities to teach the especially if you’ve spent the last
60 www.linuxvoice.com
FOSSPICKS
Audio effects
EQ10Q Plugin Pack v2
W
hen listening to music or
working with audio, most
of us will adjust the EQ
of the audio to better suit our tastes
and listening environment, even if
that’s just the dub crowd ramping
up the bass for their speaker stack.
But outside of boom boxes and
Hi-Fis, equalisation is a serious tool
that’s used both creatively and
practically. It’s the audio equivalent
of adjusting the colour saturation
and contrast in an image.
Adding bass and high frequency
boost is the audio equivalent of
making the black and white
elements of an image more
pronounced, for example. As such,
EQ is an absolutely essential step in
any audio production. It’s why
there’s EQ on each of the 24 inputs
found on the EMI TG12345, the first
solid-state mixing console and the
one used to mix both The Beatles’
Abbey Road and Pink Floyd’s
Any LV2 compatible
The Dark Side of the Moon. for modifying 1, 4, 6 and 10 those in between. You can quickly
host, such as Audacity
frequency ranges – you might need or Ardour, can be used switch between two alternate
The Equaliser a single filter to cut out a 50Hz to pipe audio through configurations with A/B.
Equalisation is basically filtering. buzz, for example, or roll off the wonderful effects in The equaliser is the main
You highlight a range of frequencies frequencies higher than 18kHz, this bundle. component in a small package of
in the audio spectrum and choose whereas 10 will let you sculpt the effects that also includes a brilliant
to either boost or lessen these sound into whatever shape you compressor and an absolutely
frequencies. need. Each filter can be enabled essential noise gate. The
Old hardware was cherished for and disabled, dragged across any compressor will make the quieter
the character that this process frequency and boosted/diminished parts of audio louder using a curve
exerted on the audio in much the with a drag up or down. to describe which amplitudes are
same way that a synthesizer’s But there’s a lot more on offer curtailed and by how much. The
filters define its sound, and modern too. You can enable a spectral view noise gate is best used on a noisy
digital audio engineers have similar of the audio, or a real-time mic or guitar input, as it will mute
preferences for the computational frequency plot, both of which the audio when the level falls below
algorithms doing the same thing in respond to your changes. These a certain threshold.
software. This is why there’s such a help you easily spot hums and All three filters are vital for any
huge difference in sound between buzzes, for example, and work just audio work, especially with an
the practical qualities of Audacity’s like colour curves in Gimp. Each application like Ardour that doesn’t
default EQ, and the sound you get filter can be switched between one include any if its own effects at all
from EQ10Q, a spectacular open of six different types – low-pass for – the gate, for example, is
source EQ LV2 plugin that’s part of high frequencies, high-pass for low, absolutely necessary for podcasts
a small bundle with the same and peaks, notches and shelves for and yet there’s no other open
emphasis on sound and control. source gate we’ve been able to find
EQ10Q can genuinely rival for our own recordings. As a result,
expensive alternatives, not only in EQ10Q rivals alternatives for this is a brilliant package.
its sound characteristics but also in its sound characteristics and PROJECT WEBSITE
the incredible amount of control it
offers. There are different instances the control it offers http://eq10q.sourceforge.net
www.linuxvoice.com 61
FOSSPICKS
Digital television recorder
TVHeadend 4.1
W
e’ve been using frequency, polarity and FEC for
TVHeadend for a couple satellite transmissions. This data is
of years. For us, it is all available online, and TVHeadend
without doubt the best system for is good at scanning automatically.
recording digital television, whether After a scan, your mux should yield
that’s terrestrial, cable or satellite. services that can be mapped to
Its best feature is a low resource actual channels that should then
overhead. TVHeadend takes up very begin to populate themselves with
little RAM and CPU and can easily programme data. If not, you’ll need
be installed on a Raspberry Pi or to install and configure XMLTV too.
even a humble NAS.
You start off by making sure your After the setup…
TV-grabbing hardware is You’ll be able to browse the EPG,
Whether you want to
automatically detected by Linux. record individual programs or an with. We use ours in combination
record a TV series or
With TVHeadend installed and entire series, or even set up a stream live to VLC and with a TVHeadend PVR plugin for
running, you’ll then be able to select search term to schedule recordings Kodi, TVHeadend is the Kodi, allowing us to hide the
this as a DVB input though its web automatically. Profiles can be used best television recorder recording equipment away from the
interface. After this, create a to process these files differently, or we’ve ever used. Raspberry Pi connected to the
network and add a ‘mux’ that save them in different locations screen, and it works as if both were
connects to the network and the – we use one for movies, for local to one another. There’s also a
device itself. Muxes are bundles of example. You can watch recordings great Android app.
digitally transmitted channels, and or live transmissions from a
you may need specific information browser and create accounts for PROJECT WEBSITE
https://tvheadend.org
on their formatting, such as the people you’d like to share access
Image viewer
imv 1.2
T
here have been many image Not only can imv display images
viewers. We fondly using Wayland, it can display
remember Xv on the Amiga, animated gifs and many different
and the way the same program image formats including RAW and
could be found in Mandrake Linux, Photoshop PSD files. It can overlay
helping with the transition from one information about an image and
system to another. This is where change the way it scales an image
imv is going to help too. Not only is to fit. Thanks to the command line,
this a rather nifty image viewer that you can get lots of added
Imv is the first image
you mostly launch from the functionality for free. You can give it background according to whichever
viewer we’ve found
command line, it’s also compatible a wildcard, such as *.jpg, and jump that’s developed to work image you choose. Most
with both X11 and Wayland. This between a collection of images on the new windowing importantly, it’s the quickest image
means it’s going to be a perfect using the cursor keys to create an system, Wayland viewer we’ve used for years, even
utility as we hope to move from the ad hoc slideshow, for instance, or with large images, and isn’t tied to
increasingly decrepit X11 to the use the output of the find any particular desktop or toolkit. It
new Wayland display system. command to display an image would be ideal on a low-powered
Despite stability still being some without knowing its location. device such as a Raspberry Pi,
way off, tentative Wayland support Cleverly, you can also use the P especially as it’s likely that Wayland
can already be found in GTK and Qt, key to send the current image to the will run must faster on the Pi and
and subsequently, Gnome and KDE, standard output, which lets you will need an image viewer.
and its slimmed down remit and construct commands that will
high-performance code promises to automatically email the selected PROJECT WEBSITE
https://github.com/eXeC64/imv
revolutionise desktop performance. image or change your desktop
62 www.linuxvoice.com
FOSSPICKS
FOSSPICKS Brain Relaxers
https://launchpad.net/pybik/
First person shooter
Warsow 2
W
e know there are still comparison than two different first
people devoted to person shooter types – Unreal
Counter-Strike after 15 Tournament was a stalwart of the
years; people who flinch when a old Linux games scene before
fizzy drink can is opened thinking Unreal Tournament 3 failed to make
it’s a grenade. People like these its way to Linux in 2007, despite
still meet virtually every week and years of vague promises that
chat over Teamspeak while they followed. Team Fortress 2 was one
plan their next conquest. This of Valve’s flagship titles when it
Unless they cripple you
doesn’t quite describe us; instead, started its great SteamOS we’ve always had a soft spot for
with motion sickness,
we enjoy first person shooters for campaign, turning Linux into a first person shooters Warsow, which has just
what they offer the casual gamer games console and changing many like Warsow are brilliant celebrated a major milestone
– an intense rush of adrenaline. attitudes towards Linux games fun. with the release of 2.0.
No other type of game can development almost overnight.
bring a group of strangers Between those two games, the Atrocity Exhibition
together to defend a pretend flag, vacuum of commercial releases Despite our always mistaking the
or run from one side of the map was taken up by the open source title with the capital of Poland (or
to the other while trying to target community. Alien Arena and Red the prototype Joy Division),
an almost identically equipped Eclipse are two open source Warsow is a brilliant FPS, similar
opposition. They can be great shooters that we love to play, but in playing style and frenzy to the
fun, which is why they’re still a much loved Unreal Tournament.
major part of the games industry. Its combat is well enough
Linux has had its fair share of Warsow is similar in playing established that even those old
classics, from Unreal Tournament
to the more recent Team Fortress
style to the much-loved Counter-Strike players will
recognise its tournament
2. But there’s more to this Unreal Tournament potential. There’s a great tutorial
to ease you into the game, and
you can then join what of the
several games that always seem
to be running, or create your own
local instance for friends and
colleagues to join.
The cell shading effect used for
the graphics looks fantastic, even
on modest modern hardware
running at a high frame rate.
There’s also more to the game
than shooting, with some
excellent platform elements such
as double-jumps and wall jumps,
Top: Even if you’ve and thanks to the JavaScript-
never played a game based scripting engine, there are
like this before, lots of different game types. The
Warsow’s excellent icing on the cake is that unlike
beginner’s tutorial will many other open source games,
guide you through the nearly all the media assets are
basics of control and
released under the terms of the
combat.
Creative Commons, making this
Bottom: The combat in
Warsow is exceptionally an exceptionally open FPS.
fast, and relies on quick
PROJECT WEBSITE
reactions and mastery
www.warsow.gg
of the weapons on offer.
www.linuxvoice.com 63
INTRO TUTORIALS
TUTORIALS
Warning: excessive Linux knowledge may lead to fun and more efficient computing.
In this issue . . .
66 68
Ben Everard
believes in a future where all technology is built
to make us happy.
Transmit entire hard Control your web traffic
drives over a network with a Squid proxy
W
Recreate your perfect setup on every machine With Squid as a gatekeeper to the web, Ben Everard
orking on a magazine about with Mayank Sharma’s guide to cloning disks. saves bandwidth, protects innocent eyes, and
Linux, you have to assess a Sheep not supported. avoids unnecessary distractions.
huge range of technology.
I’ve come to the conclusion that the
only thing that matters when deciding if
a device is good or bad is the answer to
the question ‘Does it improve my life in
any way?’ Here are the three bits of tech
that improve my life the most.
An Intel CPU with integrated 3D
graphics. I install a lot of distros, and
(even without taking free software Physical games in Scratch 72 Unleash MariaDB 76 Let there be light 80
concerns into account), life’s too Wannabe surgeon Les Pounder Dive deep into your server and Mark Crutch links his lightbulbs
short to have to worry about builds a practice cadaver using a grapple with the database. with his Linux box for penguin-
proprietary drivers. Raspberry Pi. Marco Fioretti leads the way. based illumination.
An Arduino Uno. This microcontroller
opened up a whole world of tinkering
with programmable circuits that I
thought was beyond my reach.
Coding
Plugging your own circuits into a USB
port and running custom code on
Get access to ev
them is my view of geek heaven. ery
Linux Voice tuto
Over-ear head phones. They they rial ever
published in ou
make listening to Linuxy podcasts r digital
library of back-i
while shopping in Tesco far more ssues
available exclus
enjoyable. ively to
subscribers – tu
I would encourage you to think about rn to
page p56 to join
which bits of tech you have that Build a webserver in Sed 84 Scripting languages 88 .
improve your life the most to help you Ben Everard tests the limits of Find your perfect language and
make better choices in the future. this text processing language by accelerate your coding with
ben@linuxvoice.com serving HTML. Juliet Kemps guide to scripting.
www.linuxvoice.com 65
TUTORIAL FOG
FOG: CLONE COMPUTERS
OVER THE NETWORK
Walk the fine line between being lazy and slick with the agility of a tightrope walker.
MAYANK SHARMA
M
anaging a lab full of computers can be most useful features of the Fog server, especially for
quite tiring. The constant barrage of admins of larger networks, is the multicast ability.
repetitive tasks can sap the energy out of Using this feature you can deploy multiple machines
WHY DO THIS? any sysadmin, irrespective of the size of their realm. in one go. To supplement it on such large networks,
• Deploy any number of Thank heavens then for the Fog project, takes the pain you can have multiple Fog installations configured as
machines without any out of regular admin tasks such as installing software, storage servers that help take the load of the main
hassle and can even manage printers on the network. Fog server when imaging computers. However, Fog’s
• Easily image critical The Fog server is scalable and can manage large most essential task is to image an installation and to
installations to minimise
downtime networks spread over multiple locations in the same deploy it to other computers on the network, which is
• Works with heterogeneous building or on the other side of the planet. One of the what we’ll cover in this tutorial.
networks comprising
Linux, Windows and OS X
STEP BY STEP: IMAGE AND CLONE A COMPUTER
1
Set up the image server 2
Create base image
Before installing Fog, make sure the server has a static After it’s fetched and installed any required
IP address, which can be easily ensured from your components, the installation script will display a URL
router’s admin page. Also make sure that all the for Fog’s dashboard. Open the link in your web
machines in your network are configured to boot from browser and log in with the default credentials
the network card. Finally, remember to disable any (fog:password). Before going further head to User
existing DHCP servers on the network, as we’ll set up Management > Create New User to define a new
the Fog server as a DHCP server and dole out administrator.
addresses to all the computers on the network. To begin the process of imaging a computer, head
Once you have your network set up, head to the to Image Management > Create New Image. Use the
machine that you’ve earmarked as the Fog server fields in the form to describe the image. For example,
and download the latest stable Fog release from let’s assume we're creating an image of an OpenSUSE
SourceForge (http://sourceforge.net/projects/ 42.1 installation that we’ll then use on all our
freeghost/files/FOG). Then fire up a terminal and workstations. We can name the image ‘Workstations/
extract the downloaded tarball with Desktops’ and use the Operating System pull-down
tar zxvf Fog_1.2.0.tar.gz -C /opt menu to specify the operating system of this image,
Change into the bin/ directory under the extracted that is Linux. Finally, select the correct disk layout
tarball, and fire up the installation script with scheme from the Image Type pull-down menu. Our
sudo ./installfog.sh OpenSUSE installation is on a single disk with multiple
The installation script will prompt you for several partition so we’ll select the second option.
bits of information. They are self-explanatory, and in
most cases it’s best to go with the default options.
66 www.linuxvoice.com
FOG TUTORIAL
3
Register host and associate image 4
Image the host
Now head to the computer with OpenSUSE 42.1 that We’re now all set to image the OpenSUSE installation.
you wish to use as the base image and boot it up. Head to Task Management > List All Hosts, which will
Since the computer is set to boot from the network list the recently added OpenSUSE 42.1 machine.
card, it’ll display the PXE boot environment from the Under the Task section, click on the green upload
Fog server. Scroll down the Fog menu and select the arrow corresponding to this image. Fog will give you
‘Quick Registration and Inventory’ option. The Fog multiple options to schedule the upload task. You can
server will now scan the computer and add it to its tinker with these in the future, but for now it’s best to
repository of known hosts. go with the default option for instant deployment.
When it’s done, power down the OpenSUSE Now head to the OpenSUSE machine and boot
computer and head back to the Fog server. Fire up it up. It’ll again detect Fog’s PXE and automatically
the dashboard and head to Host Management > image the machine and upload it to the Fog server.
List All Hosts. You should now see the OpenSUSE The process will take some time depending on
machine listed here, which by default is identified by the size of the disk it has to image, the processing
its MAC address. Click on the edit icon to change it capabilities of the computers involved and the speed
to something more identifiable, like ‘OpenSUSE 42.1’. of the local network. The OpenSUSE computer will
Most importantly, use the Host Image pull-down restart once it’s done uploading the image. That’s all
menu and select the Workstations/Desktops option there’s to it. Repeat steps 2–4 to similarly image any
for the image you created earlier. other computer on the network.
5
Register target machines 6
Deploy the image
Before you can deploy an image to another computer, Now to replicate the OpenSUSE image onto the other
you need to first register it with the Fog server. The computers, head to Task Management > List All
registration process is the same as before. Boot the Hosts. Browse the list of hosts to find the entry for the
new computer from the network which should detect computer to which you wish to deploy and hit the
Fog's PXE environment. And when it does, select the corresponding down arrow Download image button.
‘Quick Registration and Inventory’ option. Now head to this workstation and power it on. The
Once the computer has been added to Fog’s computer’s PXE environment will automatically detect
repository of known computers, log in to the Fog the task from the Fog server and begin copying the
dashboard and head to Host Management > List image from the server on to the local machine. When
All Hosts. Just like before, click on the edit icon it’s done, you’ll end up with a mirror copy of the
corresponding to the newly added machine and OpenSUSE 42.1 installation on this workstation.
rename it so that it’s more identifiable, and associate Besides deployment tasks, you can create various
the Workstations/Desktops image with this computer other types of tasks to check up on the computer and
using the Host Image pull-down menu. Repeat its installation. Click on the gears icon to bring up a
the process to register all the computers with the list of several deployment options, such as Test Disk
Fog server, then edit them in the Fog dashboard or Password Reset. Select one and then power on the
to give them a name and associate them with the machine to automatically launch the task.
appropriate image.
www.linuxvoice.com 67
TUTORIAL SQUID PROXY
SQUID: CONTROL WEB
BROWSING BY PROXY
Shield your browser from the ravages of the web and use a proxy as protection.
BEN EVERARD
A
proxy is something (or someone) that before, you get the best out of Squid when it's shared
performs an action on your behalf. In the case between many computers. In order for this to work,
of a web proxy such as Squid, it fetches web you need to have a computer that's usually turned on
WHY DO THIS? pages for you. If you configure your browser to and attached to your local network. If you've already
• Save bandwidth connect to a proxy rather than the internet, it doesn't got a home server, that's the ideal machine to use as
• Block web adverts download the pages directly. Instead, it sends a a Squid proxy. It's really easy to copy the configuration
• Be more productive message to the proxy indicating which pages are over from one machine to another, so if you're not
needed, and the proxy gets them and sends them to sure, you can just install it on your main machine to
the web browser. try it out, and then change over to a different machine
This may all sound very pointless, but by having a later if you find it useful.
proxy between your browser and the web, you can Once you've installed Squid from your package
fine-tune the way you connect. Since many machines manager, you'll need to make sure the service is
can share a proxy (it doesn't have to be running on started. The method for this varies a little, but the
the same machine that uses it), you can quickly following will work on most modern distros:
configure the way your web connections work across sudo service squid3 restart
a range of devices. You can check that it's running correctly by trying
Squid is the most popular proxy for Linux, and is to connect your web browser. In Firefox this is in Edit >
Preferences > Advanced > Network > Settings. Select
As many machines can share a proxy, Manual Proxy Configuration and enter the HTTP Proxy
as localhost (or wherever you installed Squid) and the
you can configure the way your web Port as 3128. Press OK to accept the settings. If you
use Chrome or Chromium, you'll have to set the proxy
connections work on several devices details at the OS level. The method for this differs
between distros and desktops, but should be possible
available in most distros' package managers in a in your network settings app.
package called squid or squid3. Before jumping in and If you can still view web pages, then everything's
Configuring your proxy at
installing Squid, it's worth thinking a little about what worked. You can also double-check that everything's
OS level will ensure that it
machine you want to install it on. If you only want going through the proxy by looking in the access log.
gets picked up by all your
software that connects to to use a proxy on a single machine, you may as well You can do this with:
the internet. just install it on that machine. However, as we said sudo tail /var/log/squid3/access.log
Distros other than Debian or Ubuntu may store the log
in a different place.
That's all there is to getting a web proxy running
on Linux. However, to really get the most our of your
proxy you need to configure it to your needs.
Getting personal
Traditionally, the most common use for a proxy has
been to reduce the bandwidth used, by sharing a
temporary store. If one person connected to the proxy
requests, for example, www.linuxvoice.com, then the
proxy fetches this from the internet and passes it on.
If a second person requests the same page, the proxy
already has the data for this site, so it doesn't need to
request it from the website again: it just passes it
straight from the proxy cache to the second person's
68 www.linuxvoice.com
SQUID PROXY TUTORIAL
web browser. The advantage of this varies a lot
depending on who's using the web connection. If
you're managing a corporate or school network,
there's a good chance that you could save quite a bit
of bandwidth. If you're managing a home network for
just one or two people then the benefits are likely to be
less, but it can still be worth doing especially if it's not
as fast a connection as you'd like.
By default, Squid will only proxy files in 256MB of
memory. This isn't very much, so it probably won't
have a noticeable effect on your web browsing. There
are two options to increase it: you can increase the
amount of memory available or you can configure
Squid to use the hard drive for the cache. Both of
these are configured in the squid.conf file, which is
usually found in /etc/squid3.
To make more memory available (for example,
1GB), open this config file with your favourite text
editor, such as with:
sudo nano /etc/squid3/squid.conf
and find the line:
# cache_mem 256 MB
The hash at the start of the line means that it's
commented out; however this is also the default
amount, so deleting the hash won't immediately
change anything. Instead, change the line to give a
different amount of memory to the proxy such as:
cache_mem 1024 MB
To change the disk caching, find the line that starts
The Firefox configuration
with: people to create content, so in this regard, they're
needs both HTTP and SSL
# cache_dir something that we all benefit from. However, through proxies set to capture all
The options on this configuration line allow for very 2014 and 2015 there was a massive increase in traffic.
fine-tuned control over the cache. The first argument invasive advertising, where adverts blocked large
is the storage method to use. There are several portions of the site and aggressively tracked people
options that each have different payoffs in terms on the site in an attempt to squeeze pennies out of
of space efficiency and time efficiency and other them. The result is that now a sizeable proportion of
aspects, but unless you're running a really high-traffic bandwidth and CPU power is spent on rendering
proxy, the default of usf (Unix File System) should adverts that the viewer didn't want in the first place.
work fine.
The second argument is the location of the cache. Admonish ads
It will be a directory structure rather than just a single There are already a number of options for blocking
file, and the default location is in /var/spool, though adverts, including web browser plugins; however,
anywhere to which the Squid user has write access many of these plugins are themselves nefariously
is fine. Following this are the arguments for UFS. tracking users and feeding this data back to the
These are three numbers, the first of which is of advertisers. If you run a network either at home or
most interest to us, as it's the total amount of space work, you probably also have a number of devices
that Squid can use on the disk. The second number – and wouldn't it be much easier if all of them could
is the number of subdirectories that can be in the have their adverts blocked at a central point? If you
cache root, and the third number is the number of do it with your proxy, every machine that connects
subdirectories allowed inside the first set. The defaults through the proxy automatically has adverts removed.
for these two are fine for most uses. To create a 1GB Squid uses Access Control Lists (ACLs) to decide
disk cache, change the line to the following: which traffic to let through and which to block, so to
cache_dir ufs /var/spool/squid3 1024 16 256 block adverts, we just need an ACL that will identify
Once you've saved your changes, you just need to which bits of traffic are adverts, and block these.
tell Squid to reload the configuration file with: In web access lingo, this is termed a blacklist. The
sudo squid3 -k reconfigure opposite of a blacklist is a whitelist, which contains
Advertising provides a good way for a website to details of traffic that we do want to let through.
make a little money to cover the cost of hosting the Blacklists are available online both for free and
site. Popular websites can also earn enough to pay commercially, and which one is right for you will
www.linuxvoice.com 69
TUTORIAL SQUID PROXY
ad-domains of the type dstdom_regex and loads
the data from the file /etc/squid3/advertsdomains.
The quote marks around the file name are important,
because without them, Squid will attempt to interpret
the file location rather than the file contents as the
blacklist data. The second line creates a rule for http_
access. If you wanted to create a whitelist rather than
a blacklist, you could use allow rather than deny.
Once you've saved those changes, you can reload
the Squid configuration using:
sudo squid3 -k reconfigre
Now adverts should be blocked on all browsers that
are routing through the proxy.
Access denied
Our above method of blocking adverts was easy to
set up because we had a ready-made blacklist in the
right format. However, if you want to block other
categories of content, you won't always find blacklists
in formats that Squid can understand. Fortunately,
By default, users visiting
depend on what level of blocking you need. If you're Linux gives us plenty of text processing tools that we
blocked sites will be
served an error page. blocking adverts (like we are here), a free blacklist can use to transform everything we need into a
This is configurable in is probably sufficient, since it's not a problem if the format that Squid can work with.
squid.conf. occasional advert slips through. If you find yourself There's a great set of blacklists put together by
relying on Squid to enforce network policies, then it's Shalla Secure Services are available at
worth investigating further options. www.shallalist.de. This website has a download link
We'll use the blacklist from http://pgl.yoyo.org/as/. that will grab a tarball you can extract to get blacklists
It's quite comprehensive and uses regular expressions for a range of different categories of material. In each
to keep up to date with changing subdomains. subfolder of the main BL folder, you'll find a domains
You'll need to select the type as 'Squid – As Squid text file containing one domain per line of content
Dstdom_regex File', tick the View List option as Plain that should be avoided in this category. There are two
problems with this. First, Squid won't automatically
exclude subdomains unless the domain is preceded
You may want to block Facebook during with a dot; and second, Squid will throw an error if the
weekdays, and only allow access in the list includes both a subdomain and a higher domain
preceded by a dot. If we want to use the Shalla list, we
evening and at weekends need to remedy these two problems. Open a terminal
and navigate to the subfolder you want to ban.
Text and then press Go. This will open the file in your To add a leading dot to all the lines, use the
browser, so save it to your hard drive with the name following command, which matches the start of a line
advertdomains. character (^) and inserts a dot:
You can configure Squid to look anywhere you like cat domains | sed 's/^/./' >domainsquid
for this file, but in the interests of keeping everything Solving the second problem is a little trickier. We
contained, it's best to place it in the Squid config need to remove any lines that are subdomains of
directory, so copy it and change the owner of the file domains that are included in the file. The first task
to the Squid user with:
sudo cp advertdomains /etc/squid3/advertdomains
sudo chown proxy /etc/squid3/advertdomains
This should work on Ubuntu- and Debian-based
systems. If you're on a different distro, you may need
to change the location of the config directory and
the username of the Squid user. Check your distro's
documentation if you have issues with this.
Now you need to open squid.conf in your favourite
text editor and add the lines to load and act on the
blacklist:
acl ad-domains dstdom_regex "/etc/squid3/
advertdomains" Sarg (Squid Analysis Report Generator) will parse your
http_access deny ad-domains Squid logfile to create a report to help you understand
The first line creates an access control list called how your internet connection is used.
70 www.linuxvoice.com
SQUID PROXY TUTORIAL
here is to make sure that the file is sorted so that all
the subdomains are listed underneath the original
domains. A normal sort won't help us here, because
the subdomains are on the left of the domain, and
sort organises a list using the leftmost character. The
easiest way around this is just to reverse each line
before the sort, and then reverse again at the end. We
can do this using the rev command. Once this is done,
we can use awk to match the domain against the
same number of characters on the line below. If the
two match then the second line is a subdomain of the
first line and it shouldn't be output. All together, this is
done with the following command:
rev domainsquid | sort | awk 'NR!=1&&substr($0,0,length(
p))==p{next}{p=$0".";print}' | rev > newdomainsquid
This list is a dstdomain type in Squid, so you need to
add the following lines to your squid.conf:
acl blocksites dstdomain "/etc/squid3/newdomainsquid"
http_access deny blocksites
When you reload the Squid configuration, this will
block all the domains listed in newdomainsquid.
The squid.conf file
You can have as many of these blacklists as you which enable you to alter which sites are blocked
includes detailed
like provided you give each of them a different name based on the time of day or day of the week. For comments about all the
(blocksites is the name here). example, you may want to block the Facebook options, so you should be
website during weekdays, and only allow access in able to get things working
Fine tuning the evening and at weekends. You could do this by even if you've broken your
Access Control Lists enable more nuanced control adding the following to your squid.conf file. internet connection.
than simply allowing or blocking. One of the most acl facebooktime time MTWHF 18:00-23:59
useful alternatives is the use of time-based controls, acl facebookdomain dstdomain .facebook.com
http_access allow facebookdomain facebooktime
Running a dedicated proxy server
http_access deny facebookdomain
The setup we've done here has been based on running your The first line defines an ACL with the time type, and
proxy locally, but if you want your proxy to be available to
this takes two arguments. The first argument is the
all devices on the network, you'll need a machine that's
always on (or, at least, is turned on as often as you want list of days to match on (this is the first letter of the
the network to be on). Obviously, when choosing a machine day, except for Thursday, which is a H, and Saturday,
to be turned on constantly, power consumption is an issue, which is an A). The second argument is the time
so any of the small ARM boards (such as a Raspberry Pi or period to allow.
Odroid) make an excellent choice. Another option is to run
The first http_access line combines two ACLs and
Squid directly on your network router. Most ISP-supplied
router firmware doesn't come with this option, but if your only lets a web request through if it matches both of
router is capable of running OpenWRT (https://openwrt.org) the ACLs. We also need the final line to deny other
you can run Squid on the same box that that handles your requests to this domain. In this example, we've just
main network connection. used a single domain, but exactly the same approach
Other than running the machine, the configuration is
could be used with one of the domain lists from Shalla
exactly the same as we've described here, so you just need
to copy across your squid.conf file and any associated to (for example) block all web chat at certain times.
black and whitelists, and Squid will be ready to run. Squid's http_access doesn't allow a way of
combining ACLs so that access is granted if either
one or the other ACLs is matched; instead, this can
be done by creating a http_access allow line for each
allowable combination of ACLs, followed by a
http_access deny line. Facebook is served over
HTTPS, so in order for this block to work, you have
to make sure that your browser is configured to
send encrypted pages to the proxy as well as
unencrypted ones. In Firefox, this is set in the proxy
configuration page on the SSL line.
Ben Everard intends to monetise Squid by selling Daily Mail
blockers for people who need to keep their minds clean.
www.linuxvoice.com 71
TUTORIAL EDUCATION
BUILD A GAME WITH
GPIO ZERO AND SCRATCH
Remember Operation? Of course you do – now make your own!
LES POUNDER
F
or this issue we are going to create one
hardware project, in this case a homemade
"Operation" game where we have to save the
WHY DO THIS? robot. We have three lives and if we touch the metal of
• Control hardware using the robot we lose a life, which is indicated by one of
the GPIO three LEDs turning off and the buzzer sounding. This
• Learn logic project uses a simple method of input: the surgery
• Transfer knowledge tool is turned on with current, and when we touch the
across languages foil around the robot, we connect to Ground, causing
the tool to turn off, triggering the code to execute.
To code this project we'll use two methods, each
one aimed at a different level of user. For example,
TOOLS REQUIRED
beginners can learn to hack with Scratch, whereas
• A Raspberry Pi running
the latest Raspbian more competent coders can use the new GPIO Zero Our finished project is a mix of arts materials, electronics
release Python library, which removes a lot of the hassle of and aluminium foil circuits and uses many different skills
• 3 x LED using the old RPi.GPIO Python library. across the curriculum.
• 3 x 220Ω resistors The circuit diagram for this project, along with the
• Male–female jumper full code listings and other images, can be found to turn on three LEDs – more on that later. So you
cables via our GitHub repository at https://github.com/ should have
• Male–male jumper cables lesp/LV24-Dr-Robot, or you can download a Zip file When Green Flag Clicked
• A buzzer containing all of the project files from https://github. broadcast gpioserveron
• Breadboard com/lesp/LV24-Dr-Robot/archive/master.zip. broadcast config17out
• Masking Tape broadcast config27out
• Aluminium Foil PROJECT 1 – SCRATCH broadcast config22out
• Wire You'll find Scratch in the Programming menu of the broadcast config10out
• Glue latest version of Raspbian (Jessie). To start coding we broadcast config9in
• A plastic case (we used an shall use blocks from the left of the screen and drag broadcast 3LED
A4 document holder)
them into the centre coding area. We start with a hat Now let's create a new section of code. Grab
block: in the Control palette, Look for "When Green another When Green Flag Clicked block from the
Flag Clicked" and drag it into the coding area. Still in Control palette, then a Forever loop and attach it to
the Control palette look for "broadcast"; we're going to the Green Flag block. In the Control palette you will
create seven broadcast blocks, each used to see Repeat 10 – drag it inside the Forever loop and
configure the GPIO. change 10 to 2, as we're going to sound the buzzer
Our first turns on the GPIO server, a script behind attached to GPIO 10 twice for half second intervals,
add the following blocks.
Beginners can learn with Scratch, while broadcast gpio10on
wait 0.5 secs
more competent hackers can use the broadcast gpio10off
Now we come out of the Repeat 2 loop, but still
new GPIO Zero Python library inside the Forever loop. Drag a Repeat Until loop from
the Control palette so that it is under the Repeat 2
the scenes that enables Scratch to talk to the GPIO. loop, and inside the Forever loop. We now need to
We are next going to create four configurations for create a variable, so go to the Variables palette and
GPIO pins numbered 17,27,22 and 10, turning them create a variable for all sprites called Lives. Once
into outputs. We configure GPIO 9 to be an input for it's created, drag the Lives block to the coding area
our surgery tool, before finally we create a broadcast and put it somewhere safe. Now from the Operators
72 www.linuxvoice.com
EDUCATION TUTORIAL
palette drag the \_\_ = \_\_ block and place it inside
the blank space at the top of the Repeat Until block.
In the right-hand blank space of \_\_ = \_\_ type the
number zero, and in the left drag the Lives variable
and drop it inside. Now any code inside this section
will run until the user runs out of lives.
We next need to drag an If block, placing it inside
the Repeat Until block. We also need another \_\_ =
\_\_ in the blank space of the If block. Add a zero (0)
to the right-hand side of the \_\_ = \_\_ block. For the
left-hand side we need to read the state of our surgery
Connects to all foil
tool, which will be 1 (on) or when touching the foil, 0 using multiple wires
(off). This block will be in Sensing, but first we need Connects to surgery
to activate our code to register the GPIO pins, so click tool (Tweezers)
on the Green Flag in the top-right of your screen. Now
Our project is a rather
return to Sensing and look for the Slider Sensor Value for 2 secs" and use that to say Game Over; we next simple circuit that sits
block. Click on the drop-down and you'll see GPIO9. change the Lives variable so that we have 3 lives. inside an A4 document
Change the block to GPIO9 and drag it into the left Lastly we create a broadcast "3LED". holder, with a lovely picture
blank of \_\_ = \_\_. Remember all those 0LED, 1LED broadcasts of a robot stuck upon it.
Inside this If condition we'll create another Repeat that we created earlier? Now we're going to create
loop, this time for three iterations. Each time it will turn sequences of code that respond to those broadcasts.
on GPIO 10, wait for 0.2 seconds, and then turn it off In the Control palette look for the "When I receive"
before waiting and repeating the sequence. hat block. Drag it to the coding area and change it to
Breaking out of the Repeat 3 loop but still inside 3LED, then underneath the block add three broadcasts
the If condition we now need to change the value of and edit them as follows.
the Lives variable by -1. In the Variables palette you broadcast gpio17on
will see "change lives by 1" – drag it under the repeat broadcast gpio27on
3 and change the value to -1. Now drag a "Say for 2 broadcast gpi22on
secs" block from the Looks palette and use it to tell This turns every LED on; to turn an LED off we simply
the player they have lost a life. swap the on for off.
Under the Say block we now place an If condition Now repeat this structure for 2LED, 1LED and 0LED,
that will compare the value of lives against an integer, remember to be consistent in which LED are turned
we will need the \_\_ = \_\_ block from Operators and off. So that's it, save your work and click on the Green
the Lives variable, which goes in the left \_\_ = \_\_ (in flag to play the game.
the right-hand side, type "2"). Inside the If condition
we will create a new broadcast block called 2LED. PROJECT 2 – GPIO ZERO
Repeat these steps for 1LED and 0LED. Place each If We covered GPIO Zero in LV23, and since then it has
condition under one another. been included as standard in the latest version of
We now break out of the "Repeat Until Lives" loop Raspbian, so no installation is required. For those
but stay in the Forever loop. We drag another "Say using an older version, please refer to https://
pythonhosted.org/gpiozero/#install for installation
instructions.
Crafty hacking From the Programming menu open the Python 3
> Idle application. You'll see a Python shell open; click
Using craft equipment from art and design lessons is a
great way for children to incorporate a hack into their on File > New Window to open a new project window.
curriculum. For example, in science lessons the class could Before you progress, save the blank document as
learn about Mars and its inhospitable terrain; in art they Dr-Robot-GPIO-Zero.py and remember to save your
could design the terrain of Mars; and in computing they work regularly.
can learn to program a Raspberry Pi robot. This joined up
We start the project by importing three classes
"cross curricular" methodology can help put the classes
knowledge into practice and still remain a fun activity. from the GPIO Zero library, handling the LEDs, the
At the Picademy training events we have a session buzzer and our surgery tool, which is classed as an
where the class are shown how to use a motor controller input (button). We also import the time library to
and are then given 30 minutes to build a moving gadget control our game pace.
using nothing more than arts and craft materials that can
from gpiozero import LED
be found in any school. The results of this experiential
from gpiozero import Button
activity are wide ranging– from simple roulette wheels to
interactive goal line technology – but the most important from gpiozero import Buzzer
outcome of the session is for learners to try out hacking a import time
project to life and solving problems as they occur. Next we create five variables that will each store the
GPIO pin used for the LEDs, buzzer and surgery tool.
www.linuxvoice.com 73
TUTORIAL EDUCATION
Programming the project
with the latest version of We call the relevant class LED, Buzzer and Button, and lives = 3
Scratch is an excellent pass the class the GPIO pin as an argument. life_counter(lives)
introduction to coding for life1 = LED(17) So now we move into the logic that forms our
beginners.
life2 = LED(27) game. We start by using an infinite loop, while True,
life3 = LED(22) which will constantly run the code. We introduce a
buzzer = Buzzer(10) 0.01 second delay to reduce the hit on the CPU and
tool = Button(9) ensure that our code runs smoothly.
Now we create a function to handle turning on while True:
multiple LEDs to represent the number of lives we time.sleep(0.01)
have left. The function is passed an argument, Still inside the infinite loop we now create a for
being the number of lives we have. Then an if..elif loop, a loop that will iterate a set number of times. In
conditional statement is used, the number of lives this for loop we shall instruct the buzzer to sound,
passed as an argument is compared to hard-coded indicating that the game is ready to be played. We turn
values, and the correct number of LEDs are lit. on the buzzer, then create a 0.5 second delay before
def life_counter(lives): turning it off, followed by another delay.
if lives == 3: for i in range(2):
life1.on() buzzer.on()
life2.on() time.sleep(0.5)
life3.on()
elif lives == 2:
life1.on()
life2.on()
life3.off()
elif lives == 1:
life1.on()
life2.off()
life3.off()
elif lives == 0:
life1.off()
life2.off()
life3.off()
Next we create another variable to store the number
of lives, in this case three. We then use the variable as Between the robot picture and plastic case we have
an argument and call the function that we have just aluminium foil connected to the Ground of our Pi.
created. Touching the foil is hazardous to the robot's health.
74 www.linuxvoice.com
EDUCATION TUTORIAL
buzzer.off() More foil is used for each
time.sleep(0.5) hole, to create a single
We now come out of the for loop and create a new circuit for each hole.
loop inside the infinite loop. Here we use a 'while the Each circuit connects to a
number of lives is greater than zero' loop. We evaluate Ground pin on your Pi, but
you could link all the holes
the value of the lives variable each time and check
together and use just one
that it is greater than 0; if that is the case, the loop
Ground pin.
repeats. We also introduce a delay to pace our code.
while lives > 0:
time.sleep(0.01)
Inside the while lives > 0 loop we create a condition
to check against; this time we are checking to see if
the surgeon has touched the aluminium foil, triggering
a loss of life. When the foil is touched, the GPIO pin
attached to the surgery tool goes from on to off (True
to false, 1 to 0) and this is a change of state used to
indicate an error.
if tool.is_pressed:
So if the surgeon makes a mistake and touches the
metal, the tool.is_pressed condition is true and we
create a for loop that will beep the buzzer three times
in quick succession.
for i in range(3):
buzzer.on() lives = lives - 1
time.sleep(0.2) life_counter(lives)
buzzer.off() In our final section of code we break out of if tool.
time.sleep(0.2) is_pressed condition and return to the infinite loop.
Breaking out of the for loop we next create a delay We now have a condition that will activate when the
before printing that the user has lost a life. We adjust user has no lives left. If that's true, the text 'Game Over'
the lives variable and one life from its current value. is printed to the screen. A three-second delay takes
Finally we call the life_counter function and pass it place before we change the lives variable, restoring
the number of lives that the player has left. the three lives that the player receives; this is then
time.sleep(0.1) indicated by illuminating the LEDS.
print("You lost a life") if lives == 0:
print("Game Over")
time.sleep(3)
lives=3
life_counter(lives)
Ensure that your code is saved and when ready
click on Run > Run Module to start the game.
What have we learned?
For this issue we highlighted how one hardware
project can be coded using two different methods.
Scratch is great at illustrating how the sequence of
code works as it is very visual with the design of
blocks and loops, and GPIO Zero enables anyone to
dip their toe into hardware hacking with Python, which
is great for children who want immediate results from
their projects.
Coding is a great activity but we should not remain
tied to just one language. Once you understand the
logic of programming then this knowledge can be
transferred to other languages such as Ruby, Perl or
JavaScript, and this is key skill for children to grasp in
the new Computing curriculum.
A breadboard is an ideal place to build your project. Les Pounder divides his time between tinkering with
hardware and travelling the United Kingdom training teachers
You can quickly test and take apart your project for the
in the new IT curriculum.
optimum layout.
www.linuxvoice.com 75
TUTORIAL MARIA DB
MARIA DB: LEARN THE
POWER OF DATA
How to live happy with MariaDB, and reuse its data in ways you didn't know existed.
I
testdb friends table
MARCO FIORETTI f you use any Linux desktop, or have any kind of
dynamic website, you're using a database. And E_ID Birthdate Place Name Address
the more you go on, the more likely it is that you
5 1980-03-25 college Fred Smith Cambridge St, 540
WHY DO THIS? will have to recover those databases from some
6 1985-06-30 work Nick Williams Oxford St, 34
• MariaDB databases disaster, or process them in ways that are not
7 1987-01-14 work Joan Walters Cambridge St, 601
probably power websites possible using only database software.
that you already have. The A database is an archive of data, structured in Foreign Key? Foreign Key?
sooner you learn how they
work, the better for you tables and stored in low-level binary containers. If you
• Because being able to could look inside a database, each of its tables would hobbies table
quickly get every kind of resemble a spreadsheet, with data of the same kind in H_ID Past_hobbies Present_hobbies Friend Name
data in or out of a real their separate columns, and different data about the
database is a skill that 5 Cooking, jogging Hockey, chess Fred Smith
may save you a lot of same entity (in database lingo: entry, or record) all in
6 Writing Cooking, carpentry Nick Williams
work the same row. The "Customers" table of a shop
7 Chess, bowling Free climbing, Joan Walters
• MySQL/MariaDB database, for example, may have columns like
crossword puzzles
knowledge is something "Customer code", "Customer name" and "Unpaid
employers like…
orders", and each row (each record), would describe The logical partitioning of data in tables and, above
one different customer. all, the possibility to define, and enforce, relationships
A database is what's known as relational when between tables by means of keys, that is related fields
you can define and use precise relationships among that connect them.
its tables, and the software that manages this is
called an RDBMS (Relational DataBase Management MariaDB is relatively easy to use thanks to its
System). A column that uniquely identifies each programming interfaces in many languages. The
record in a table of a relational database is called client–server architecture means that, in order to
its primary key. Relations between tables can be create and use any database with MariaDB, you must
enforced by defining some columns of one table install and run at least two programs. The first of
as foreign keys in another. In the shop database these is the default command line MariaDB client,
which, somewhat confusingly, is called mysql, for
compatibility with MySQL. The server, which for the
A database is an archive of data, same reason is still called mysqld, is the part that is
structured in tables and stored in always running, and that accesses each database, to
execute the requests coming from different clients,
low-level binary containers which may be on the same or on another computer.
above, this would ensure that you cannot enter, in the MariaDB commands
"Orders" table, a new order from a customer who does Installing the MariaDB server and/or client is no
not already exist in the "Customers" table. problem at all, unless you are using a niche, or
The standard way to work with relational databases experts-only version of Linux. In 2016, all the most
is via commands in an ad hoc Structured Query popular distributions should offer binary packages for
Language, or SQL for short. You may issue those both programs in their standard repositories. You may
commands indirectly, through a graphical interface, or have to explicitly enable the server to start at each
directly type them inside some MariaDB client. boot, but that's all there is to it.
The SQL commands to create or delete databases,
Enter MariaDB tables and users are all thoroughly explained in the
MariaDB (https://mariadb.org) is the Open Source, Documentation section of the MariaDB Knowledge
client–server RDBMS most popular on Linux these Base (https://mariadb.com/kb/en/mariadb/
days. Born as a drop-in replacement for MySQL documentation) or in the latest MySQL Reference
76 www.linuxvoice.com
MARIA DB TUTORIAL
manual (https://dev.MySql.com/doc). We're only
going to look at their most basic forms, to give you
an idea of how they work and help you understand
the manuals (throughout the tutorial, #> indicates the
Linux prompt; & is the MariaDB one. Most MariaDB
output was also removed for brevity):
#> mysql
& USE MySql;
& CREATE DATABASE testdb;
& CREATE USER 'linuxvoice'@'localhost' IDENTIFIED
BY PASSWORD 'testpw';
& GRANT ALL ON testdb.* TO 'linuxvoice'@'localhost';
& FLUSH PRIVILEGES;
The first line launches the MariaDB client.
Depending on your MariaDB default settings, you may You can browse and
have to run it as root, whenever you want to create or `place` enum('work','high school', edit MariaDB tables with
delete databases or users. Remember that MariaDB 'college','neighborhood') DEFAULT 'neighborhood', LibreOffice, as in this
users only exist inside MariaDB, but have nothing `name` varchar(255) DEFAULT NULL, screenshot, but only raw
to do with ordinary Linux accounts, including the `address` varchar(255) DEFAULT NULL, SQL and the other methods
administrator one which, like in Linux, is called "root". ... described here can process
The five commands above are what you should type These commands create a table inside testdb called them automatically.
at the MariaDB prompt in order to: friends, along with its first five columns, each with
1
Go into the "main" database of your installation, a different meaning and a matching
which stores metadata about all the others. MariaDB data type: a unique numeric
2
Create a new database, called testdb. identifier, a date in YYYY-MM-DD format, PRO TIP
3
Add a MariaDB user called linuxvoice, with a predefined set of text constants and You can use everything here to deal with
MySQL database, for now – be prepared
password testpw. finally two strings, to hold each friend's
for the future, and keep an eye on the
4
Grant that same user all privileges on all the tables name and address. As above, this is an MariaDB/MySQL incompatibilities that will
in the testdb database. You can create as many incomplete snippet of code, only meant surely arrive in the future
databases and users as you want, and then restrict to show the look and feel of MariaDB.
what each user can see or do, with the right options
to the GRANT command. Playing with records
5
Tell the MariaDB server to flush the old privileges If the friends table above only had those five columns,
configuration, to load the new one. you could add a friend named Fred like this:
Once you have created a database, you can log in & INSERT INTO friends VALUES(NULL, '1980-03-25',
to MariaDB as a normal user, type the corresponding 'college', 'Fred Smith', 'Oxford Street, 21');
password, and start creating tables, which you may where the NULL value tells MariaDB to increment and
then fill with data: set the numeric identifier itself. Now uppose Fred
#> mysql -u linuxvoice -p moves house? No problem. Assuming that MariaDB
& USE testdb; gave Fred 5 as F_ID value, just update Fred's record:
& CREATE TABLE friends ( & UPDATE friends set address = 'Cambridge Street, 540'
`F_ID` int(11) NOT NULL AUTO_INCREMENT, WHERE F_ID = 5;
`Birthdate` date NOT NULL DEFAULT '0000-00-00', Should you and Fred cease to be friends, you may
MariaDB/MySQL compatibility
The MariaDB project was launched six years ago with the goal MariaDB server (but not the MySQL one) restart automatically
of becoming “a complete drop-in-replacement for MySQL”. at every reboot. MariaDB also has more storage engines
Some day that aim will be abandoned, but today it still is 100% available, at least in the official package, than MySQL. In a
true between corresponding versions of the two databases. MariaDB/MySQL installation, the storage engine is the set of
The names of the server and client programs, as well as those algorithms and low-level libraries that actually create and
of all the configuration and raw, binary files, and of all the manage the binary data structures.
common configuration variables, are identical. There are storage engines optimizsd for speed, others for
The same applies to the SQL syntax, and to the binary robustness and many other scenarios. If you used MySQL to
interfaces that were originally developed to talk directly to a manage a database with certain non-default storage engines,
MySQL server from most programming languages. It is still you may be forced to manually recompile MariaDB to migrate
possible to talk to a MariaDB server from the MySQL client, that same database. Finally, MariaDB has different defaults
and vice versa. In spite of all these efforts, really 100% values for certain variables, and more options than MySQL.
compatibility is not guaranteed. On some Linux distributions, To know more, read https://mariadb.com/kb/en/mariadb/
for example, some extra manual work is required to make the mariadb-vs-mysql-compatibility.
www.linuxvoice.com 77
TUTORIAL MARIA DB
All commands concatenate rows in such a way that a
friends row with a certain value goes side by side with
Your custom script the row from hobbies in which friend_name has that
text file with and/or commands
mysqldump
same value. What changes from one JOIN example to
MariaDB
output the other (and remember, these are just three of the
database "Process available variants) is which rows are considered, and
& Format
how "holes" in the temporary table are filled.
code"
SQL The temporary table created by the INNER JOIN
SELECT concatenates only the rows of friends and hobbies
statement(s) that a) have the same value in the name and friend_
name fields (without the ON clause, you'd get every
Possible destinations row of friends concatenated to every row of hobbies)
and b) a list of present_hobbies that includes the
string hockey.
The second command produces a table that has
all the rows from the table on the left of the JOIN
keyword (that is friends) each concatenated with:
SQL INSERT HTML pages GNUPLOT csv text files for Twitter text
The row with the same friend_name, if such a row
commands to for dynamic Graphs! spreadsheets Command or slides in
OTHER DBs Websites Line clients OpenDocument
exists;
format A row with some or all fields set to NULL otherwise.
The RIGHT JOIN does the same thing, just reversing
the tables.
remove him from your table: Now, before moving to the next part, let's learn the
From graphs to web pages,
& DELETE FROM friends WHERE F_ID = 5; most important things, that is how to back up your
when you know the basics
of SQL and scripting, it's To list the name and addresses of all your friends MariaDB database with mysqldump. Typing this in a
easy to transform MariaDB who are more than 25 years old, sorted by street, you Linux shell:
database, or their textual could ask MariaDB to: #> mysqldump -u linuxvoice -p --extended-insert=false
backups, into almost & SELECT address as A, name as N FROM friends friends> friends-backup.sql
everything. WHERE Birthdate <= '1990-01-01' ORDER by address; would save all the SQL instructions to rebuild from
+-----------------------+---------------+ scratch all the content of your friends database in the
|A |N | text file friends-backup.sql, when loaded from the
+-----------------------+---------------+ MariaDB prompt:
| Cambridge Street, 540 | Fred Smith | #> mysql -u linuxvoice -p friends
| Cambridge Street, 601 | Joan Walters | & source friends-backup.sql;
| Oxford Street, 34 | Nick Williams |
The WHERE clause is essential. Without Why bother with all this?
PRO TIP
it, MariaDB would display, delete or alter There are lots of programming languages with
every record of the selected table! MariaDB/MySQL interfaces. Raw SQL commands are
Always try a SELECT before an UPDATE
or DELETE query, with the same clauses, All SQL statements share the same plain text strings, and the whole content of a MariaDB
to be sure that those operations will only basic structure shown above: you first database can be dumped into one file, as a sequence
alter the records you do want to alter!
define which columns you want to see, of plain text commands, each with a well defined
how to format each field if necessary, structure. Taking all that into account, it's easy to
and from which table(s) MariaDB should realise that creation, recovery and reuse of MariaSQL
get them. To see only the records that match certain databases, are much easier than you may suspect.
conditions, describe all of them in the WHERE clause. Because the commands are plain text, we don't
need to use SQL to manipulate them; we could, for
The real power of SQL: joining tables example, dump the whole content of the database, or
The SQL JOIN operator places two or more tables of of a single table, into a flat text file, one record per line,
a database side by side, to create one temporary, whith MySqldump. At that point, it will be easy to cut,
virtual table, that you may filter and display as if it slice, combine, alter and reformat that text with tools
were a normal one. These examples introduce three like sed, awk or grep, inside shell scripts.
common types of JOIN: The other method consists of querying the
& SELECT * FROM friends INNER JOIN hobbies ON database from inside a script, using the MariaDB/
friends.name = hobbies.friend_name WHERE hobbies. MySQL libraries for the chosen language, and playing
present_hobbies like '%hockey%'; with the results. Listing 1 shows some lines of Perl
& SELECT * FROM friends LEFT JOIN hobbies ON that detect, format and display, more flexibly than SQL
friends.name = hobbies.friend_namename; could do, the duplicate entries in a budget database:
& SELECT * FROM friends RIGHT JOIN hobbies ON Listing 1
friends.name = hobbies.friend_name; 1 use DBI;
78 www.linuxvoice.com
MARIA DB TUTORIAL
2 use DBD::MySql;
Unix log fixes OpenDocument Backups EXIF/IPTC Website mirrors
4 my $database = "budget"; spreadsheets texts, of other metadata from made with wget
5 my $host = "localhost"; etc databases photographs
6 my $port = "3306";
7 my $user = "budget_user";
8 my $pw = "budget_password";
9 my $dsn = "dbi:MySql:$database:localho
st:3306";
10 my $DB = DBI->connect($dsn, $user, $pw);
12 my $query = qq~select b_date, b_item, b_ Possible sources
amount, b_id from budget_list where b_date >= '2006-01-
01' order by b_date;~;
13 my $query_handle = $DB->prepare($query);
"Process SQL INSERT
14 $query_handle->execute(); statement(s)
& Format
15 $query_handle->bind_columns(\$b_date, code"
\$b_item, \$b_amount, \$b_id);
Your custom script MariaDB
17 while($query_handle->fetch()) {
and/or commands
19 if ($DOUBLE_ITEMS{"$b_date|$b_amount"} database
{'exists'} eq 'Y') {
20 print $DOUBLE_ITEMS{"$b_date|$b_
amount"}{'value'}; Even the inverse passage from CSV to database is A few lines of SQL
21 printf " %10.10s : %-40.40s %8.2f %-8s\n", simple, once you know the basic trick. These few lines commands and glue code
22 $b_date, $b_item, $b_amount, $b_id; of code, again in Perl: can load everything, from
photo metadata to system
23 } else { 1 while (<>) {
logs, into a MariaDB
24 $DOUBLE_ITEMS{"$b_date|$b_amount"} 2 chomp;
database, for further
{'exists'} = 'Y'; 3 ($b_date, $b_item, b_amount) = split/\s*,\s*/; analysis.
25 $DOUBLE_ITEMS{"$b_date|$b_amount"} 4 # process date, item and amount as you wish!
{'value'} = 5 print "INSERT INTO budget_list VALUES(NULL,'$b_
26 sprintf "ORIG: %10.10s : %-40.40s %8.2f date', '$b_item','$b_amount');\n";
%-8s\n", 6}
27 $b_date, $b_item, $b_amount, $b_id; are (almost) all you'd need to read a CSV file, one line
28 } at a time, load its fields in Perl variables, process them
29 } as needed, and generate the valid SQL statements
The first ten 10 of Listing 1 show one way to load that would insert them in a MariaDB
Perl modules that talk to MariaDB, initialising all the database.
necessary variables and using them to connect to a Last but not least, websites. Many PRO TIP
database called budget. The next four statements dynamic websites are powered by The MariaDB command line client keeps a
history of its commands, just like the Unix
prepare a query, execute it and bind the columns it MariaDB or MySQL. When we had to shells. Use it!
returns to Perl variables with the same names. In line resurrect an old Drupal website, for
17, the script loops through each row returned by the example, a quick look at its mysqldump
query. If the %DOUBLE_ITEMS array already contains backups showed that these three Perl statements:
an entry with the same combination of date and $PRINT = 'y' if ($_ =~ m/^CREATE TABLE `node`/);
amount, both that and the current record are printed, $PRINT = 'n' if ($_ =~ m/^CREATE TABLE `performance_
as possible duplicates. Otherwise, the current record detail`/);
is reformatted, and inserted into %DOUBLE_ITEMS. print "$_" if ('y' eq $PRINT);
With the same base technique you may quickly were all we needed to print only the SQL commands
write code (in Perl or many other languages!) that, containing the fields with the full, actual text, and
in the same run, connects to different MariaDB nothing more, of all its pages. After that, less than one
databases (or other data sources!), processes all their hour of trial and error later we had other 50 lines of
contents as you wish, and prints out the results as code that gave us a folder with all those web pages,
you want. More detailed, useful outputs that a script each in a separate file named after their title and
like that may generate include: publication date, ready for reuse. The gory details
Actual SQL statements that you may execute from and full source code are available online at http://
the same script, with the same connection freesoftware.zona-m.net/?p=124. Take it from us:
technique, or print out to a text file. knowing how to talk to MariaDB and reuse its data can
A CSV (Comma Separated Values) text version, save you lots of time, and be a lot of fun.
directly usable in spreadsheets like Calc or Excel, of
any data generated from processing the same Marco Fioretti is a campaigner and writer on issues
surrounding free software, ethics and the environment.
records
www.linuxvoice.com 79
TUTORIAL CONTROL SMART BULBS
ILLUMINATE YOUR LIFE
WITH LINUX PART 1
Take your first steps in home automation by using Linux to control your lights.
MARK CRUTCH
W
e're fed up with the term “Internet of
Things”. Yes, a new wave of low-power
devices and ubiquitous wireless
WHY DO THIS? connections mean that ever more devices are able to
• Gain control over your report their status to the world. However, we've been
lights using Arduinos and Raspberry Pis to add “intelligence”
• Get on the IoT bandwagon to dumb devices for long enough that this isn't really
• Party like an old school anything new. To be quite frank, we're very much
disco looking forward to the day when internet
connectedness comes as standard, and we can go
back to just using the term “things”.
As jaded as we are by the terminology, we still feel a
thrill every time we're able to control a physical device
with just a few lines of code. It's an empowering
skill – being able to link the wider world of data and
messages with changes in your physical environment.
Perhaps there's no easier way to get a feel for that The default bridge homepage lists the open source
power than to control your own domestic lights. projects that it's based on.
We opted to use the Philips Hue range of “smart”
bulbs. These sell for a ridiculous £50 per bulb, or a To control the lights from a Linux box, we'll use the
slightly more reasonable £15 for the Lux range – RESTful API exposed by the bridge, which requires
white bulbs that can be dimmed but can't change us to find the bridge's IP address or hostname. Ours
colour. You also need a “bridge” to control the lights, exposed itself via Zeroconf as philips-hue.lan, but
which can only be purchased as part of a Starter Kit, you can probably get its IP address from your router's
setting you back £150 just to get going (three Hue web interface. The bridge also communicates back
bulbs and a bridge), or £50 for the Lux option (two to Philips, telling them its local IP address. This does
mean that you can find it by browsing to https://
The bulbs form a mesh network, which www.meethue.com/api/nupnp from within your
network, but we'd still prefer it if our hardware didn't
makes it easier to extend your lighting talk to an external server behind our backs! Pointing a
web browser at either the IP address or the Zeroconf
system around the house hostname displays the default web page for the
bridge, which just shows some details about the open
bulbs and a bridge). You can use Hue bulbs with the source components it uses.
bridge from a Lux kit, and vice versa, so you can at
least start off with a Lux kit and add colour later. Let there be light!
The system uses a low-power wireless protocol Although it's refreshing to see a product so openly
called ZigBee. Although it's wireless, it's not the presenting this information, we can't help feeling that
same as Wi-Fi – hence the need for the bridge, the bridge could benefit from serving a password-
which connects the lights' ZigBee network to a wired protected administration interface to let you easily
Ethernet cable that plugs into your router or switch. turn lights on and off from a browser. What it does
This enables any device on your network to send data have, however, is a debugging screen that lets
to the bridge, which then routes it to the lights. The you submit commands to the bridge and view its
bulbs themselves form a mesh network, which makes responses. It's not linked to from the main page, so
it easy to extend your lighting system around the you'll have to modify the URL in the browser to visit
house, even to those corners that Wi-Fi never reaches. http://philips-hue.lan/debug/clip.html – replacing
80 www.linuxvoice.com
CONTROL SMART BULBS TUTORIAL
philips-hue.lan with the bridge's IP address, if The CLIP screen lets
necessary. You should see the “CLIP API Debugger”, you send arbitrary data
packets to the bridge for
which we'll simply refer to as the CLIP screen from
testing and debugging.
now on.
Most things in the API are restricted to registered
users to prevent a malicious program from playing
havoc with your lights, so we need to create a user
account on the bridge. New users can only be created
in a 30-second window after pressing the hardware
button on the bridge, so we'll prepare the data we
want to send first, then press the button, and then
finally send the request. The address to enter into the
“URL” field of the CLIP screen couldn't be simpler:
http://philips-hue.lan/api/
The API requires a devicetype parameter, which
consists of a pair of strings to identify the name of
your application, and the name of the device you're
using it on. For our simple test purposes the values
aren't terribly important, so we'll use LV_Hue as the
application name, and linux_box as the device. These
are separated by a hash character, then wrapped as a which contains just a single entry: an object with
JSON object, resulting in the following string that has a property called success whose value is another
to be entered in the Message Body box: object with a property called username. The value of
{"devicetype": "LV_Hue#linux_box"} that property is the long hexadecimal string, which is
Now's the time to put on your running shoes, enlist the real payload we need. That string will form part
the help of a glamorous assistant, or just move your of every other call we make, so that the bridge will
laptop closer to the bridge; firmly press the button in accept our instructions as coming from an authorised
the middle of the bridge then, within 30 seconds, click application.
on the “POST” button in the CLIP screen. You should
see the Command Response panel fill with something Put some colour in your life
similar to this: From now on you'll need to not only replace the
[ hostname in our code, but also swap the hexadecimal
{ string for the one that your own bridge
"success": { returned. For the sake of space we've
PRO TIP
"username": "1c4eb44d1be8dc071e7bed091946e023" abbreviated ours to 1c4...023, but you
You can use the CLIP screen to test any
} should use your full value. Let's start of the Hue's APIs without writing a single
} by getting a list of all the lights that are line of code.
] connected to the bridge. Enter your
Both this and the parameter you sent in the version of this URL into the CLIP screen:
Message Body are encoded in JSON format. This is http://philips-hue.lan/api/1c4…023/lights
a simple serialisation of JavaScript data structures, Then click the GET button. You should receive a
where items in curly braces represent objects lengthy response detailing the state and capabilities
(collections of name:value pairs), and items in square of the lights that your bridge knows about. Each
brackets represent arrays. In this case we sent an light is represented by a numbered property in the
object with a single property called devicetype whose top-level JSON object, and the value of each property
value was LV_Hue#linux_box. We received an array is an object, which descends even further into more
Why choose Hue?
There are plenty of manufacturers selling allegedly that we've used in this tutorial, and LIFX bulbs scheduled lighting changes continue to occur even
“smart” bulbs. At the cheap end of the market (www.lifx.com), which started out as a Kickstarter if our network is down. The mesh network formed
you can find colour-changing bulbs that are project but are now available for general purchase. by the bulbs means that we have no problem with
controlled via a dedicated infrared remote control. In both cases the price per bulb is similar, but the signal propagation, even in areas that have no Wi-Fi
Slightly more expensive are similar bulbs with LIFX bulbs are connected via Wi-Fi, with no need for signal. Finally the bridge offers a simple RESTful
Bluetooth controls. These are intended to appeal a bridge. You can therefore get started with a single API that's available locally. LIFX does have a similar
to smartphone users, but rarely offer an open or LIFX bulb, compared with Philips' requirement for API, but only via its own servers; if your internet
documented API, and suffer from range limitations. you to buy a whole starter kit. connection is down, or their servers are unavailable,
We prefer something more sophisticated which Although it's more expensive to get started, there you can't use it to control the lights. There is a
is designed to work as part of a larger network. Two are some advantages to the Hue bulbs, which led to UDP-based protocol that can be used for local
contenders stand out: the Philips Hue/Lux range us choosing them. The use of a bridge means that applications, but it's not so developer-friendly.
www.linuxvoice.com 81
TUTORIAL CONTROL SMART BULBS
properties and objects. You can request the details might choose to do something else as well. Finally,
for a single light by appending its number to the end press Control+D to end. The at command can take
of your URL (eg http://philips-hue.lan/api/1c4…023/ a variety of different time definitions, allowing you to
lights/1) and sending a GET request. To switch the execute the command at a specific time, or to include
light on, we need to PUT a vaue of true into the on a relative offset – which is what we've done here by
property of the state object. Enter the setting the specific time to now, then adding a 20
following two lines into the URL and minute offset to it.
PRO TIP
Message Body fields, respectively, then Controlling your lights from the command line has
Check the dimensions of the bulbs before
press the PUT button. its uses, but we want to be able to trigger changes
ordering – not all light fittings will take
them, even if the connector is the right http://philips-hue.lan/api/1c4e…023/ from a variety of different inputs. For that kind of task,
type. lights/1/state we prefer to use a language such as Python. To make
{"on":true} the code simpler we'll use the requests library, so the
Note that there are no quotes around first step is to install that:
the value, as this is a JavaScript Boolean, not a string. pip install requests
To turn the light off again, change the value to false, Now we can write a simple Python program to turn
then press the PUT button again. If the light's too a light on at half brightness. Save the following as
bright, try this payload to both turn it on and set it to hue_light_on.py:
half brightess (on a scale of 1 to 254): #!/usr/bin/env python
{"on":true, "bri":127} import requests
The CLIP screen is great for experimenting with host = 'philips-hue.lan'
the API, but we really want to put the power of our user = '1c4eb44d1be8dc071e7bed091946e023'
lights to practical use. Because all we're doing is light = 1
sending HTTP requests, you can control the lights payload = '{"on":true, "bri":127}'
from just about any programming language, or even api = 'lights/{}/state'.format(light)
directly from the command line. Create the following url = 'http://{}/api/{}/{}'.format(host, user, api)
Bash script as hue_alert.sh, changing the first three r = requests.put(url, payload)
variables to reflect your own setup and the number of You can run this using python hue_light_on.py, or
the light you want to affect. by making the file executable as we did with the shell
#!/bin/bash script previously. If you're having trouble getting the
HOST=philips-hue.lan code to work, and want to see the error messages
USER=1c4eb44d1be8dc071e7bed091946e023 coming back from the bridge, add the following line to
LIGHT=1 the end of the code.
API=lights/$LIGHT/state print r.text
PAYLOAD='{"alert":"select"}' You'll notice a lot of similarity between the Bash
URL=http://$HOST/api/$USER/$API script and the Python version. It makes sense to split
curl -X PUT -d $PAYLOAD $URL the host, user and light values out into variables at the
sleep 1 top of the file, to make it easier to reuse the code. After
PRO TIP curl -X PUT -d $PAYLOAD $URL that, it's just a case of compiling the parts together to
Make sure you order the correct lights sleep 1
for the fittings you have. You can buy
curl -X PUT -d $PAYLOAD $URL
adaptors but they increase the overall Accessing the complete API
length of the bulb. Make it executable using chmod u+x
hue_alert.sh. You'll also need to install The Hue API is documented at www.developers.meethue.
curl from your package manager if it's com. There is a “Getting Started” section which describes
enough of the API to let you create a user on the bridge
not already on your system. Running this command
and change the state of your lights. More extensive APIs,
– using ./hue_alert.sh from the directory it's saved to control groups of lights or whole lighting scenes are
in – should flash your light three times. Where it really available if you create a free account on the site. This
becomes useful, however, is in a situation like this: requires you to agree to some terms and conditions, which
./long_running_script.sh && ./hue_alert.sh won't affect you when developing normal applications,
but it does prevent anyone legitimately creating a bridge
Start your long-running script like this and you can
emulator or proxy using the documented APIs on the site. A
head down to the living room to watch TV knowing proxy would greatly open up the possibilities of the system,
that the lamp in the corner will flash when your script allowing commands to be intercepted and modified, or
has completed. Or how about using the venerable at offering additional input and control options. It would
command to flash the lights at a particular time. Need also allow Hue apps to control non-Philips lights, which
probably explains the restriction, but we reckon Philips
a reminder to do something in 20 minutes?
would actually sell even more lights if they make it easier to
at now+20min interface them with other software and hardware.
at> ./hue_alert.sh Despite this restriction, there are a couple of Hue
at> ^D emulators available on the internet, so a Pandora's box of
Enter the first line and press the Return key. Then bulbs has already been opened. Come on Philips, remove
this silly restriction and get behind the creativity of your
type in a series of commands to execute, one per
developer community!
line. This example just flashes the lights, but you
82 www.linuxvoice.com
CONTROL SMART BULBS TUTORIAL
make a simple HTTP request. It is important to use
the correct request type, though, as the bridge will
just return an error message if you try to send a GET
request to an API that expects a POST, or a PUT to
one that expects a DELETE.
So far we've used commands that work with either
Hue or Lux lights, but if you've spent the extra money
on a Hue bulb you probably want to know how to get
some colour out of it. Let's start by giving you a bright
blue light. Modify the payload line to this:
payload = '{"on":true, "hue":46920, "sat":254, "bri":254}'
Run the script and you should find that your light
changes to blue. Obviously we've turned it on and
set it to full brightness, but what of the hue and sat
values? The former sets the basic colour of the light
on a scale from 0 to 65535. The scale wraps round,
so that both ends are red, green is at 25500, and blue
is the value of 46920 that we've used in our script.
Try modifying it to turn the light green or red – then
try other values to find oranges, purples and more.
The sat value, refers to the saturation of the colour:
254 gets you 100% colour, while lower values result in
We used a Raspberry Pi
less colour and more white. At a value of 0 you'll get a of 0.1 to whatever random number is used for the
A+, an E-Ink screen (www.
completely white light. sleep() call. As we've used an infinite loop for the percheron-electronics.uk)
main part of the code, you'll need to press Ctrl+C to and some custom code
Shades of light and dark quit. We'll leave it up to you to put in nicer keyboard based on the Python in
Sometimes there's a need to increase or decrease handling or extend it to multiple lights. this article to create our
the brightness of a bulb by a relative amount, rather The snippets of code we've presented here are only own wall-mounted lighting
than set it to an absolute value. We could do this in intended as the basic building blocks of something controller.
code using the parameters we've looked at so far, more comprehensive. A real application should find
but there's also a bri_inc property that simplifies
the process. Despite the name, it can also be used
to decrease the brightness by passing a negative
You can watch TV knowing that the
number. This payload will nudge the brightness down: lamp in the corner will flash when your
payload = '{"on":true, "bri_inc":-30}'
There are equivalent parameters for adjusting the script has completed
hue (hue_inc) or saturation (sat_inc). The hue_inc
parameter wraps its result, so adding 45,000 to a red the bridge and register a new user ID automatically
hue will give you something blue, whether your red the first time it is run. It should also query the bridge
value is 10 or already up at 65,000. for information about the lights, to determine how
With our hue_inc parameter and a little randomness many are available, as well as their capabilities. Each
it's easy to simulate some disco lights. Save this as light has a “friendly” name that should be presented to
hue_disco.py and execute it as usual: the user, rather than just referring to them by number.
#!/usr/bin/env python But don't let all that put you off throwing together
import requests a few little scripts for your own use, with hard-coded
import random user IDs and bridge addresses. The Internet of Things
import time is all about getting disparate devices to talk to each
host = 'philips-hue.lan' other to make our lives a little better. It's about lights
user = '1c4eb44d1be8dc071e7bed091946e023' that change colour when you're mentioned in a tweet,
light = 1 or that turn off automatically when your phone is
payload = '{"on":true, "hue":0, "sat":254, "bri":254}' taken out of range of your Wi-Fi access point. We're
api = 'lights/{}/state'.format(light) a long way from cross-device protocols that enable
url = 'http://{}/api/{}/{}'.format(host, user, api) such capabilities with ease, so until then we may have
while True: little choice but to hack things together with some
r = requests.put(url, payload) less than perfect shell scripts. Which suits us fine,
payload = '{{"hue_inc":{}}}'.format(random. because that's the fun bit.
randint(0,65000))
time.sleep(random.random() + 0.1) Mark Crutch has recently cancelled his holiday to Iceland,
Philips recommends that a light shouldn't be changed now that he can recreate the aurora borealis in his living
room, any time he wants.
more than 10 times a second, hence the addition
www.linuxvoice.com 83
CODING SED
SED: BUILD A WEB
FRAMEWORK
Who needs Apache and PHP when you've got Bash and Sed?
BEN EVERARD
S
ed, the Stream Editor, is one of the tools that issue a simple print statement. We have to write a
you find on almost every Linux system. Its program to manipulate the lines until we get as far as
basic purpose is to edit a piece of text based on 'hello world'.
WHY DO THIS? a set of rules that are usually built around regular Let's take a look at one option:
• Get a better expressions (often shortened to the snappier echo "hello" | sed "s/$/ world/"
understanding of text 'regexes'). In this tutorial, we're going to abuse Sed to Here, we pipe the text hello into the Sed program
processing on Linux build a web framework that can serve up HTML pages “s/$/ world/”. The s character means search and
• Learn how the web works and generate dynamic content, but before we dive into replace. Following the s we need two arguments that
the web side of things, let's start by taking a look at are separated by forward slashes – the first is the
Sed itself. regular expression to search for and the second is the
Sed works by taking input either from a file or text to substitute for this expression. In this particular
standard input, applying a set of transformations, then case, the $ character matches the end of a line, so
outputting the result. The format for this is: this program simply adds <space>world to the end
sed <program> <file> of a line. If you run this command, it will output hello
The program is written in the Sed programming world, but it's not a true 'hello world' because you have
language, and if <file> is omitted, Sed takes its to pipe in hello in order for it to work.
input from Standard Input (Stdin). Sed is a highly
specialised programming language designed just to Primitive, yet functional
apply transformations to text; however, it is Turing A slightly more advanced option is as follows:
complete, which is a computer science way of saying echo "anything" | sed "s/.*/hello world/"
that anything that can be programmed in another This uses the regular expression .*, where the dot
Wireshark's 'follow TCP
stream' feature is a great language can also be programmed using Sed . character matches any single character and the
way to get to know text- The standard first program with any language is asterisk following it tells Sed to match that character
based protocols such as 'hello world', but this poses us a little bit of a problem zero more times. In other words, .* matches anything
HTTP. in Sed because, unlike most languages, we can't just at all including nothing. In this case, Sed has to decide
what to match .* to. It could match the first character,
or any number of characters on the line. When it has
a choice like this, Sed will always match the highest
number of characters possible; therefore .* will always
match the entire line.
This is a little better than our first example, but it's
still not a proper hello world program because it will
output hello world once for every line that's sent into
the program. For example, if you run the following,
you'll get a screen full of hello worlds:
dmesg | sed "s/.*/hello world/"
The dmesg command outputs all the messages
from the Linux kernel. It can be useful in diagnosing
hardware problems, but here we're just using it to
output large amounts of text.
We can make our 'hello world' program a little better
by chaining commands. A semicolon can be used to
join more than one Sed command together, and they
will then run one after another. The q command quits
Sed, so we can match, then replace the first line and
then quit with the following:
84 www.linuxvoice.com
SED CODING
dmesg | sed "s/.*/hello world/;q"
Another approach that gets the same result is to
use Sed's range function to only match the first line.
The problem with this that even if you only tell Sed to
change the first line, it will still print every other line. To
get around this we need two things: the command line
argument -n, which tells Sed not to print out every line
by default; and the p argument to s, which tells Sed to
explicitly print any lines that have been changed.
dmesg | sed -n "1,1 s/.*/hello world/p"
The range of lines that we want to search comes
before the main command. In this case, we've used
line numbers, so this matches just the first line (1,1).
Another way of using ranges in Sed is to enter two
regular expressions separated by a comma. When
you do this, Sed will start processing at the first
instance of the first regular expression and finish at
the last instance of the last regular expression.
That's enough about Sed to get us started. Let's see
how to misuse this editor to make a web server. request to an external process. On Ubuntu-based The sed command at the
systems, you'll need to install the netcat-traditional bottom replaces every
Built to serve package and use the command nc.traditional as we input line with "hello world"
Our Sed server needs a way of attaching to a TCP port have done here. On other distros, you may be able to
so that anything that's sent into the port gets sent to use the regular nc command.
standard input, and anything that Sed outputs gets All this script does is run an infinite loop passing
sent through the TCP stream. If we were building a each new request onto the run.sh script (which will
server to be deployed across many machines that
needed to be managed, we'd use the init process for
this. Both init.d scripts and Systemd services can
A semicolon can be used to join more
handle this easily. However, since we're just using our than one Sed command together, and
server for a bit of fun, there's no need to go to this
effort. The Netcat (nc) tool offers us the functionality they will then run one after another
we need, so we just need a wrapper script to attach
our server to the port: contain our Sed program). We've used port 8889 here,
#!/bin/bash but it will work equally well on any port that's not
while true ; do nc.traditional -l -p 8889 -e 'run.sh'; done currently in use.
There's more than one version of nc and not all We now just need to create a Sed script that takes
of them understand the -e flag, which attaches the incoming HTTP requests and replies with the data
we want. To do this, we need to know what these
HTTP requests will look like, so let's take a look at the
Regular expressions requests that are coming in. We can grab the HTTP
Much of Sed's power comes from regular expressions request by creating run.sh and entering the following:
(or regex). This allows us a very powerful method of #!/bin/bash
defining what text we want to match our Sed rules. Regular cat >> httpoutput
expressions define a pattern against which a given piece of Save this as run.sh, save the first Netcat script
text is matched. Here, we'll give a really quick overview, but
as server.sh and make them both executable with
for a more detailed look at regular expressions, see the Grep
tutorial from Linux Voice issue 1, which is available at chmod +x *.sh. Then start our web server with
www.linuxvoice.com/issues/001/bash.pdf. ./server.sh and point your browser to localhost:8889.
Any letter matches itself (the matches are case This won't actually load a page (since our server is
sensitive), but many symbols have special meanings. A only saving the contents of the request). Once the
dot matches any single character; ^ matches the start of a
page is trying to load, take a look in httpoutput and
line; $ matches the end of a line; and * means 'match the
previous character zero or more times'. you should see something like:
Here are a few examples: GET / HTTP/1.1
• ^hello will match any line beginning with hello. Host: localhost:8889
• ^hello$ will match a line that contains only hello. User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64;
• ^a.*b$ will match any line that begins with an a and ends
rv:42.0) Gecko/20100101 Firefox/42.0
with a b regardless of the characters in between.
• ^...$ will match any line containing exactly three Accept: text/html,application/xhtml+xml,application/
characters. xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
www.linuxvoice.com 85
CODING SED
The only difference between the above response
and a web page response is that a web page is
written in HTML. We could put all of our web page
HTML in the Sed line, but it would rapidly become
unmanageable. To make like a little easier, we'll allow
ourselves to use Bash variables as well as Sed. A
simple HTML hello world can then be done with:
httpheader="HTTP\/1.1 200 OK \n\n"
htmlheader="<html><head><title>hello<\/title><\/head>"
htmlbody="<body><h1>Hello World<\/h1><\/body><\/
html>"
sed "s/.*/$httpheader$htmlheader$htmlbody/;q"
We have to escape the forward slashes, but otherwise,
we can pass normal HTML straight to Sed.
We've now created a web server capable of
serving a single page. This is fairly impressive given
The developer tools in
Accept-Encoding: gzip, deflate the limitations of Sed, but it's far short of the web
Firefox will give you details
of the HTTP requests DNT: 1 framework we set out to create. In order to be able
sent and the responses Connection: keep-alive to serve up different pages, we need to be able to run
received. Cache-Control: max-age=0 multiple search and replace rules. We can do this in
This is an HTTP request. It contains all sorts of Sed by putting each rule on a separate line, but we
useful information for a web server, and fully featured can't do it with our current setup because we end the
web servers will use the different parts in different rule with the quit (q) command. Using this, it will exit
ways. Our server isn't going to speak the full HTTP Sed whether the line is matched or not after the first
protocol, but just enough of it to allow a web browser rule. Sed can selectively run blocks of code only on
to successfully request a page. For this, we'll use a lines that match a particular regular expression using
single bit of information: the first line. In the above the format:
example, it tells us that the web browser is requesting /<regex>/ { <sed code> }
the root (/) of our web page. The Sed script just has We can now write our code to serve up different
to read this and reply with an appropriate HTTP pages when the browser requests different URLs in
response containing our web page. different code blocks. The below code serves up two
An HTTP response containing just the text 'hello different pages, one at localhost:8889/test and one
world' would look like this: at localhost:8889/helloworld. Since both of these will
HTTP/1.1 200 OK quit Sed if they match the URL, we know that we'll only
reach the bottom of the Sed script if none of the URLs
hello world match. If this is the case, we'll serve up a 404 error
We only need to send this once, so this is actually message.
the same as our 'hello world' example from earlier. httpheader='HTTP\/1.1 200 OK \n\n'
The Sed program for this is: htmlheader="<html><head><title>hello<\/title><\/head>"
sed 's/.*/HTTP\/1.1 200 OK\n\nhello world\n/;q' htmlbody1="<body><h1>"
You can see here that the forward slash has to htmlbody2="<\/h1><\/body><\/html>"
be escaped with a backslash, and new lines can be httpnotfound='HTTP\/1.1 404 Not Found \n\n404-not
introduced with \n. If you put this into run.sh in place found'
of the cat line, you should find that you can reload
localhost:8889 in your browser and see the 'hello
Extra Sed features
world' message. There's no need to stop and restart
server.sh: as soon as the changes to run.sh are We've focussed on the features of Sed that we need to
saved, any new requests will go to the new script. make our web framework. It just so happens that this
also covers the parts of Sed that make is useful for text
TCP streams are a little different to files, because processing, but there are a couple of useful features that
they don't terminate in the same way. When the we've overlooked. These are:
server sends the HTTP request, it doesn't close the • The -i command line flag is used to edit a file in place.
stream, because if it did, it wouldn't be able to get With this, the changes are written back to the source of
the response. Instead, it expects the server to close the text rather than send to standard output.
• The g flag to the s command applies the transformation to
the stream once the entire HTTP response is sent. all occurrences of the regular expression on a line rather
Because of this, we have to explicitly quit Sed before than just the first. For example:
the browser will display the data. If we used the $ echo 'eeeee' | sed 's/e/f/'
alternative Sed hello world (based on ranges), then feeee
our server would send the data to the browser, but the $ echo 'eeeee' | sed 's/e/f/g'
fffff
browser wouldn't load it because it'd be constantly
waiting for more HTTP data from the server.
86 www.linuxvoice.com
SED CODING
Who would have thought
you could build a complete
web framework in 14 lines
of Bash and Sed ?
sed -n "s/GET// references. If you put an ampersand character in the
s/HTTP\/1.1// replace section of an s line, Sed will put in the entire
/^ \/test / {s/.*/$httpheader$htmlheader$htmlb text that the regular expression matched. For this to
ody1 test $htmlbody2/p;q} work properly, we first have to strip out everything
/^ \/helloworld / {s/.*/$httpheader$htmlheader from the URL that we don't want included in the final
$htmlbody1 helloworld $htmlbody2/p;q} page. Our final Sed script is:
s/.*/$httpnotfound/p;q" sed -n "s/GET//
The first two lines of this code strip out the GET s/HTTP\/1.1//
and the HTTP/1.1 from the request by replacing them /^ \/test / {s/.*/$httpheader$htmlheader$htmlb
with empty strings. This isn't completely necessary, ody1 test $htmlbody2/p;q}
but it makes our URL matching a little less cluttered. /^ \/helloworld / {s/.*/$httpheader$htmlheader
The caret character (^) at the start of the regular $htmlbody1 helloworld $htmlbody2/p;q}
expression that matches /test and /helloworld is /^ \/name.*/ {
there to match the start of the line. Matching in this s/\/name//
way means that there can't be an accidental match s/.*/$httpheader$htmlheader$htmlb
against another URL that also includes the string ody1 Hello & $htmlbody2/p;q}
/test. There's also a space after the URL, which s/.*/$httpnotfound/p;q"
means that the URL will only match if it's complete, as This simple Sed web framework could easily be
spaces aren't allowed in URLs. (Actually, the situation expanded to serve complex web apps, though it's
is a little more nuanced than this. While you can't have probably best to stick to a more conventional web
a space character in a URL, you can encode a space framework for this. We created this as a fun way to
as %20, which your browser will display as a space, so
even though it sometimes looks like there are spaces
in URLs, the space is never sent to the server in a We created this as a fun way to learn
HTTP request).
This Sed web server can be expanded to serve up
Sed, not as a safe, secure web server
as many pages as you like, and although our pages
are quite simple (to save space in the magazine), they learn Sed, not as a safe, secure web server.
could be as complex as you like. The Bash variables Technically, it's not written completely in Sed, since
make it quite easy to reuse different parts of the it requires a little help from Bash and Netcat, but Sed
page (such as, for example, creating a standard does all the heavy lifting. The small codebase shows
sidebar), however, it's still not quite a web framework. just how powerful Sed can be once you properly
We should enable our server to create dynamic understand the paradigm of stream editing. Using the
content based on values that the user supplies. A techniques we've used here, you could easily convert
simple example of this is where a browser goes to a logfile into a report, tidy up your source code, or do
the URL localhost:8889/nameben and the server all manner of text processing tasks.
responds with hello ben. In order to do this, we have
to be able to capture part of the URL and include it in Ben Everard is porting linuxvoice.com to Sed. He expects to
finish the project shortly after hurd 1.0 is released.
the response. Sed allows us to do this using back-
www.linuxvoice.com 87
CODING SCRIPTING LANGUAGES
SCRIPTING LANGUAGES:
WHAT AND WHY
Why go to the extra effort of compiling your code?
JULIET KEMP
S
cripting languages seem to be everywhere
these days; but what exactly is a scripting
language? Ultimately, it's not so much any
WHY DO THIS? feature of the language itself, as what it's used for:
• Pick the right language creating and running scripts (roughly, an automated
for you series of commands).
• Spread information This generally means that a language used for
asynchronously over the scripting will be interpreted rather than compiled, and
web
dynamically typed. (In theory you could write a 'script'
in pretty much any language, but Java, for example,
would be a bad fit.) The usage-based definition does
mean that the line between a scripting language and
a general-purpose language is a little vague. A 'script'
usually means a single piece of code, which runs
start to finish, but general-purpose high level scripting Running the Perl script
languages like Perl, Python, and Ruby can be used
to write 'scripts' that are thousands of lines long and compiled programs matters; and the more important
have many different components. the development speed becomes. Missing out the
As a rule, scripting languages are designed to compiling step makes for a much faster development
be fast to learn and fast to write. Being interpreted cycle, which also fits well with modern programming
languages (in general; Ruby can be either, depending practices like XP. The larger and more complex the
software, though, the more sensible it may become to
The increasing popularity of use a compiled language again.
The big-hitters in the scripting language world
scripting languages reflects gradual are high-level general-purpose dynamic languages,
improvements in computer hardware often thought of as 'glue' languages, connecting
things together. Many other scripting languages are
domain-specific, such as Sed or Awk, or (rather more
on its implementation), however, they're likely to be up-to-date) JavaScript. JavaScript is an embedded
slower to execute than a compiled language. Their language, used only within a particular application or
increasing popularity reflects gradual improvements set of applications. Emacs Lisp is another application-
in computing hardware. The faster a computer runs, specific language, and many games have their own
the less the speed difference between interpreted and scripting language or dialect. Finally, job control
languages and shells, like Bash, are another form of
scripting language, which are often also used as glue
languages and command-line interpreters.
The Start of Scripting
Early mainframes didn't have significant (or,
sometimes, any) direct user access; instead, they
batch processed jobs. To make this easier, various
languages including IBM's Job Control Language were
developed.
Batch processing languages were followed in
the 1960s by interactive shells, and shell scripts to
Running the Python script automate running programs. These accreted more
88 www.linuxvoice.com
SCRIPTING LANGUAGES CODING
and better features as programmers hacked on from the Enlightened Perl Organisation are attempts
them. TRAC, by Calvin Mooers, invented the idea of to make this easier.)
command substitution, which interprets a command Perl 6 is in active development, but is now
within a script and uses its output in the containing considered to be a separate language; backwards
script – like the backtick operator in a modern shell. compatibility with Perl is not a goal, but Perl 6 is
Stuart Madnick at MIT wrote the CMS EXEC supposed to be recognisably "a Perl programming
scripting language (originally called COMMAND) for language". The changes are largely aimed at
the IBM VM/CMS OS in the late 60s. This included normalising the language. It's been in development
control statements of various types (including since 2000 and still isn't fully ready.
conditional statements and loops) and a few built-in Perl is still in active use, and will
functions. You can see the bones of modern scripting doubtless remain so for a long time. PRO TIP
languages beginning to emerge. However, it's less popular than it used sed and awk might be domain-specific
languages, but if you want to see some
to be, especially for new projects. The general-purpose sed and awk code, check
Perl Perl motto, There's More Than One Way out the Rosetta Code website!
Larry Wall released version 1.0 of Perl in 1987. His aim To Do It, still sums up the immense
was to create a general-purpose Unix scripting flexibility which is both Perl's biggest
language to make report processing easier. advantage and its biggest disadvantage.
Improvements in computer hardware meant that
efficient programming practices were becoming more Perl code
important than super-efficient code, and Wall wanted Nearly all Linux systems will already have Perl
to support this. Perl's ancestors include Awk (1977), installed; if not, get it via your package manager.
and the Unix shell sh (1978), and it predates Linux Here's Hello World:
(1991) by several years. (See http://history.perl.org/ #!/usr/bin/perl -w
PerlTimeline.html for a cool timeline including Unix/ print "Hello World\n";
Linux and newsgroup history.) #! is the shebang, which tells the system to treat the
Perl 2 (1988) and Perl 3 (1989) followed quickly. rest of the line as the interpreter and pass the script
Perl 4 (1991) got its version number bumped solely pathname into it. So when we run hello.pl, the first
to clearly identify it as the version documented by line tells the system to fire up /usr/bin/perl, and feed
Programming Perl (aka the Camel Book), the canonical hello.pl in to be interpreted. Language interpreters
Perl reference. ignore the shebang, either because # is a comment
Perl 5 (1994) included a comprehensive rewrite of indicator, or as a special case.
the interpreter, and a whole bunch of new features, The -w switch turns on warnings, which is good
including objects and module support. This in turn practice. (You can also add the line use strict; just
gave rise to CPAN, the Comprehensive Perl Archive underneath that, to catch certain types of compile-
Network, created as a Perl and Perl module repository and run-time errors in advance.) You'll see that Perl
in 1995. CPAN is both an incredible resource, and a statements end with ;. Make the file executable and
place to get catastrophically lost; it has incredible run it with ./hello.pl.
157,742 (at time of writing) modules, but the standard This slightly more interesting code outputs the
of those modules varies dramatically, and it can be Mandelbrot set to your terminal screen:
hard to navigate and to find high-standard modules. #!/usr/bin/perl -w
(The MetaCPAN project and the Task::Kensho module use strict;
The Mandelbrot Set
The Mandelbrot Set is defined on the plane of complex
numbers (represented as an (x,y) grid). For each complex
number c, iterate:
zk+1 = zk2 + c
If the iterative sequence stays close to c, then c is in the
Mandelbrot set. If the sequence spirals off and away, c is
not in the set.
When calculating this, we'll iterate the sequence 50
times, and if zk+1 leaves a circle of radius 2 around c, we'll
conclude that c is not in the set.
Our grid will have the x axis running from -2 to 1, and
the y axis running from -1 to 1. We'll draw it as ASCII art,
starting from the top-left corner (x=-2, y=1), one y-line at
a time. Each ASCII character (space or *) will represent
a move of 0.05 along the grid. You can change the hard- With an appropriate program, you can generate the set
coded values to see what happens, or add colour to make it and then zoom in to see more and more fractal detail.
more visually complex. Licence: CC-SA
www.linuxvoice.com 89
CODING SCRIPTING LANGUAGES
core datatypes, and a module system. In 1994,
version 1.0 was finally released, including the lambda
and map, filter, and reduce functions, courtesy of a
Lisp hacker's patches. Initially, Python's clean syntax
was intended to make it accessible to non-
programmers; these days it is still designed to be
accessible and easy to learn, but non-programmers
are no longer a specific target market.
Python is highly extensible, and modules can be
written in either Python or C; which meant that Van
Rossum's C programmer colleagues could start
working with it straight away. One of Van Rossum's
aims was to create a bridge between the shell and
C; a second language for C/C++ programmers to
use in situations where C would be overkill. Python's
ancestors include ABC (a language intended for non-
programmers), C, Bash, Lisp, Perl, and Java.
Version 2.0, in 2000, introduced list
comprehensions, an idea borrowed from functional
programming that enables you to easily create
lists. Version 3.0, released in 2008, was a bigger
Bugfixing can be difficult
use Math::Complex; # deals with complex numbers change, aiming to reduce the redundancy that had
as errors don't show up
on the displayed page; try sub mandelbrot { accumulated in previous versions of Python (and
JSLint my $z = 0; ditching backwards-compatibility in the process). In
(www.jslint.com) or the my $c = shift; contrast to the Perl approach, the design philosophy
JavaScript console in your for (1 .. 50) { behind Python 3 was "there should be one – and
browser to help. $z = $z * $z + $c; preferably only one – obvious way to do it". But it
return 1 if abs $z > 2; # if it's bigger than two it retained its multiple paradigms, an approach common
escapes to general-purpose scripting languages; you can
} still write object-oriented Python, structured Python,
return 0; functional Python, and so on. This flexibility is great,
} but it can make it hard, especially for beginners, to get
for (my $y = 1; $y >= -1; $y -= 0.05) { to grips with other people's code.
for (my $x = -2; $x <= 1; $x += 0.05) { Python code is compact (you need fewer lines of
print mandelbrot($x + $y * i) ? ' ' : '*'; code to do the same amount of work than, say, Java)
} and readable, runs fast, is quick to develop (like other
print "\n"; scripting languages), works well with object-oriented
} programming, and has broad applicability. However,
Functions/subroutines are identified with sub it's still slower than compiled languages, though
name. Parameters are passed in as a list, and can optimisation can help, and it's not a client-side web
be retrieved by Shift-ing off the list one at a time. language. The huge recent increase in the number
However, they don't have to be identified of libraries available has greatly helped its uptake; it's
in the function name. The subroutine always been accessible for beginners but now it is
PRO TIP
here returns non-zero if the number is more powerful for pros.
Perl's unofficial backronym is Practical
Extraction and Reporting Language, not part of the Mandelbrot set, and zero There's a fascinating interview with Van Rossum
though Wall prefers his own Pathologically if it is. at www.artima.com/intv/pythonP.html on the
Eclectic Rubbish Lister. for loops have a three-part structure: beginnings of Python.
for (start, end, increment) {..}. You can
increment or decrement in a for loop, Python code
and you can nest them, as here. You'll probably have Python already on your Linux
The ?: operator shown here is a shorthand way of system too (if not, check your package manager).
writing if-then-else. If the function call here returns Here's Hello World:
true (the value escapes, so is not in the set), we print #!/usr/bin/python
an empty space; otherwise (else), we print an asterisk. print "Hello World"
You don't need to specify the newline at the end
Python of the output string. Nor do you need a semicolon to
Python is nearly as old as Perl; Guido van Rossum terminate the statement, as Python uses whitespace
(Python's Benevolent Dictator for Life) started to terminate statements.
implementing it in 1989, and released version 0.9.0 in Here's the Mandelbrot Set code in Python:
1991. It had classes and class inheritance, functions, #!/usr/bin/python
90 www.linuxvoice.com
SCRIPTING LANGUAGES CODING
Bash
Bash is slightly different to Perl, Python, and JavaScript,
being a job control/command execution languages. But it's
still a complete language, although not a great choice for
more than a page or two of code.
The Bourne shell, sh, was released in 1977 in Version
7 of UNIX, for use as an interactive command interpreter
and scripting language. It became the default UNIX shell
and was used for a whole host of practical jobs across the
system.
The GNU project, aiming to produce an entirely free
software system, needed a FOSS drop-in replacement for
sh. Brian Fox, funded by the Free Software Foundation,
released Bash (standing for Bourne-again shell) in beta
in 1989. It's been the standard Linux shell ever since
(though shells such as tcsh and zsh are also popular). To
check which shell you're using, type echo $SHELL. If it's
something else, type /bin/bash to try Bash.
Bash handles command-line input:
$ echo "Hello World"
Alternatively, you can create a file:
#!/bin/bash Bash doesn't require semi-colons, but instead treats a
echo "Hello World"
newline as starting a new command.
import math as a marketing ploy. It's interpreted, client-side, and is
def mandelbrot(z, c, n=50): executed directly in the web browser.
for a in range(1, n): Microsoft reverse-engineered JavaScript to produce
z = z ** 2 + c JScript, which they released in 1996 as part of IE3 and
if abs(z) > 2: IIS. The differences between the two implementations
return z made it hard to design websites that worked well
return 0 in both browsers, and JavaScript began to get a
print("\n".join(["".join(["*" if not mandelbrot(0, x + y * 1j) bad reputation for blocking cross-browser support.
else " " Netscape submitted it to the international standards
for x in [a * 0.05 for a in range (-40, 20)]]) organisation ECMA, and the ECMAScript official
for y in [a * -0.05 for a in range (-20, 20)]]) standard was released in 1997. JavaScript is the best
) known implementation of this standard; ActionScript
As with Perl, we import a library (here the math 3 is another.
library) to handle complex numbers. In Python you JavaScript relies on the browser providing objects
write x + yi as x + y * 1j. and methods to enable it to interact with the browser
Functions are defined with the def keyword. for environment. This does make for potential security
loops use the range keyword. The value in the range risks, which is part of why JavaScript has had
must be an integer, but if, as here, you want a non- problems in the past. Bad coding can make users
integer loop, you can multiply the range value by a vulnerable to malicious scripts, and there have been a
non-integer and feed that into your loop value. lot of problem developers and companies.
As Python automatically adds a newline after a print This was perhaps made worse because JavaScript
statement, to get the output we want we have to use was initially seen as the province of 'amateur' creators
join. Note that the for loops are inside the print/join of websites, not professional programmers. Many of
statement, and also the use of if not … else. This uses these 'amateurs' just pasted someone else's scripts
list comprehensions to create a list of all the outputs straight into their website, without checking the
from the Mandelbrot function, and join them together. quality or security of those scripts.
However, with the rise and rise of the web, and
JavaScript helped by the introduction of Ajax (Asynchronous
JavaScript was, famously, developed in 10 days in JavaScript and XML: basically a way to communicate
1995 by Brendan Eich at Netscape. Netscape at the with the server asynchronously, thus updating a page
time was offering a server with a portable version of without having to refresh it, which is great from a user
Java to run server-side applets, and wanted an easier, experience perspective), JavaScript has become a
interpreted, language to go alongside it, aimed at more respected, and more professional, language.
non-programmers. Initially it was called LiveScript and Dynamic web apps and web pages rely heavily on
first shipped in Netscape Navigator in 1995, but it was JavaScript -- it's nearly essential to use JavaScript if
later renamed JavaScript, at the same time as you want your webpage to look remotely up-to-date.
Netscape added Java support to Navigator, possibly Node.js has also finally allowed JavaScript to jump
www.linuxvoice.com 91
CODING SCRIPTING LANGUAGES
from client-side to server-side, and the new HTML }
5 APIs offer more options for controlling webpages, document.write("\n")
making it ever more useful – as long as you're careful }
with the security.
</script>
JavaScript code </body>
You can't run JavaScript from the command line; to </html>
try it out, open this hello.html page in your browser. As in the first line of the script, you can set
<!DOCTYPE HTML> document values in JS. (Note that the body element
<html> has to have an id value for this line to work.)
<head></head> Functions in JS use the function keyword, and you
<body> can pass in multiple parameters.
As with Python, JS variables don't need any
<script> signifiers, but should be declared with var the first
alert("Hello World") time they're used (though this isn't essential).
document.write("Hello World") for loops work in very much the same way as
</script> the other two languages. Since JS lacks a complex
</body> number library, we have to fake it by handling the real
</html> and complex parts separately. document.write didn't
This very minimal HTML just runs the script, want to output whitespace, so the space is a backtick
wrapped in a <script> tag. In fact, this says Hello here instead.
World twice: the alert line pops up an alert dialog (with If you'd prefer to make better use of HTML and draw
a prettier (non-ASCII) version of this, there's some
Stats from 2015 suggest that JavaScript, code on the Rosetta Code website (rosettacode.org).
And, as with Perl and Python, there are plenty of online
Python, Ruby and PHP are all in the top resources if you want to explore JavaScript further.
10 most popular languages And there's more…
The advantage of scripting languages is their speed of
an OK button automatically attached); the document. development, and a big part of that is their dynamic
write line writes it as HTML body text. Like Python, typing. The popularity of scripting languages is thus
JavaScript doesn't generally require (but will accept) arguably associated with the rise of unit tests in
a semicolon to end a statement, but there is some modern programming; unit tests make type safety
debate about whether it's better to include them or less valuable, because they (should…) pick up that sort
not. You can also write a separate file.js script and of error, and others besides. Dynamic languages are
include it in the HTML. faster to write, the lack of compile time with
Here's the Mandelbrot set: interpreted languages makes it faster still, and the
// HTML and head tags to start file as above speed of modern computer hardware means that run
<body id="mandelbrot"> speed (which is slower in interpreted languages) is
<script> less important. Or that's the theory; it certainly doesn't
apply to every project.
document.getElementById("mandelbrot").style. GitHub and StackOverflow stats from early 2015
fontFamily = "courier"; suggest that JavaScript, Python, Ruby, and PHP are
all in the top 10 most popular languages, more than
function mandelbrot(c_real, c_img) { holding their own with Java and C/C++. Job surveys
var x = 0 also show much the same thing. (Stats compiled
var y = 0 by www.sitepoint.com/whats-best-programming-
for (var i = 0; i < 50; i++) { language-learn-2015 and www.codingdojo.com/
var x_tmp = x * x - y * y + c_real blog/8-most-in-demand-programming-languages-
y = 2 * x * y + c_img of-2015.) Web 2.0 and associated new technologies
x = x_tmp place an emphasis on scripting langauges, and new
if (x * x + y * y >= 4) { return 1 } languages such as Lua, a very light and adaptable,
} dynamically typed, general embeddable extension
return 0 language, are on the rise. Scripting languages may
} not, contrary to some of the hype, be about to take
over the programming world, but 30 or more years in,
for (var y = 1.0; y >= -1.0; y -= 0.05) { they're more important and more flexible than ever.
for (var x = -2.0; x <= 1.0; x += 0.05) {
if (mandelbrot(x, y) == 1) { document.write("`") } Juliet Kemp is a computing polyglot, having fun with coding
else { document.write("*") } one language at a time.
92 www.linuxvoice.com
SUBSCRIBE
Subscribe
shop.linuxvoice.com
Get your regular
dose of Linux Voice,
the magazine that:
Gives 50% of its profits
back to Free Software
Licenses its content
CC-BY-SA within 9 months
All subscribers get access to
every single digital back issue –
that’s about 1,000,000 words of
tutorials, reviews and free software
hackery at your fingertips
Overseas subs prices
12-month print & digital:
Europe: £85
US/Canada: £95
Rest of world: £99 DIGITAL
SUBSCRIPTION*
ONLY £38
* WHEREVER IN THE WORLD YOU
ARE – IT’S DIGITAL, SO THERE ARE
NO POSTAGE COSTS
www.linuxvoice.com 93
CORETECHNOLOGY
CORE
Valentine Sinitsyn develops
high-loaded services and
teaches students completely
unrelated subjects. He also has
a KDE developer account that
he’s never really used.
TECHNOLOGY
Prise the back off Linux and find out what really makes it tick.
Domain names
Explore the nuts and bolts of DNS, the quiet workhorse of the
internet that too many of us take for granted.
W
hile many people think of DNS exclusively A name server can handle one or more zones, and
in the context of mapping host names to a single zone is often serviced by multiple servers
IP addresses, in fact, "DNS is a general (if for reliability reasons. A server that stores data for a
somewhat limited) hierarchical database, and can given zone is referred to as being "authoritative" for
store almost any kind of data, for almost any it. Other name servers may cache its responses and
purpose". By analogy, responding to DNS queries re-use them in their own answers; those answers are
often involves talking to several sources. In this Core non-authoritative. They are no evil; in fact, we often
Tech, we'll learn how it happens, both internet-wide use name servers on wireless routers that aren't
and in your home network. authoritative for anything. Sometimes, these servers
Once upon a time, there was a single file named are called "caching".
HOSTS.TXT. It contained names for all hosts on the A domain name consists of one or more dot-
internet, and the network information centre (NIC) separated labels. Names read right to left: com,
kept it current. Connected hosts updated this file via linuxvoice, www. In fact, each name ends with the dot
FTP. Everything went fine, until things started to grow that separates an empty label denoting namespace
big. Needless to say, the HOSTS.TXT approach didn't root. User-friendly resolvers (that is, programs that
scale well. Networks of the 80s had limited capacity, resolve names) don't force us to type it, but many
and HOSTS.TXT updates consumed their scarce other tools prefer this final dot explicit. A domain
bandwidth. A new system (DNS) had to be global, name that starts at root is called "fully qualified"
distributed to handle increasing load, and generic, to (FQDN); everything else is unqualified or relative.
support emerging applications. Today, DNS facilitates When Linux encounters an unqualified name, it tries to
name resolution, but also spam protection and service make it into an FQDN. To do so, it appends domains
discovery, among other things. from the system-configured search list. So, www on
DNS builds on the concept of a hierarchical, a corporate network may open company's website,
single-rooted namespace. A section of this tree as company.com often comes first in the search list.
(spanning one or more domains) is called a "zone". Search lists are convenient, but may cause resolution
slowdowns if configured improperly.
(namespace root)
On a good record
As the DNS is the database, it can also be viewed as
Top-level Delegation
domains (TLD) a set of records. A resource record (RR) essentially
maps a name to some data (RDATA). Each RR also
com. net. org.
has a class (typically 'IN', or Internet), a TTL (time to
Delegation live, a number of seconds for the record to stay valid
in cache), and, last but not least, type. There are many
domain1.net domain2.net RR types defined, and in this Core Tech we'll cover
most important ones.
DNS namespace in a DNS
nutshell. Different colours Zone Let's begin with 'A'. Records of this type map
represent separate zones, sub.domain2.net domain names to IPv4 addresses. A's cousin, 'AAAA',
solid arrows indicate does the same for IPv6:
domain delegation. www.linuxvoice.com. 300 IN A 104.28.6.18
94 www.linuxvoice.com
CORETECHNOLOGY
www.linuxvoice.com. 300 IN A 104.28.7.18
The above is standard textual representation of
an RR. It begins with the owner's name (note the
trailing dot), followed by TTL, class, type and RDATA,
which is just an IP address in this case. Note that
www.linuxvoice.com really has two IPv4 addresses.
This yields a simple load balancing scheme: one
connecting client will resolve it as 104.28.6.18, while
another will get 104.28.7.18. It would spread the load
on two boxes. Also note that addresses may change
by the time you read this, yet names will stay the
same. DNS provides an abstraction that makes IP
address changes transparent to end users.
The MX (Mail eXchange) record is what mail agents
use to find a host accepting email for a given domain:
linuxvoice.com. 300 IN MX 10 smtp.
linuxvoice.com.
Here, RDATA is the host name and 16-bit number
(10) dubbed priority. A mailer should contact a
highest-priority (lowest numbered) mailserver first. So,
if one wants to deliver mail for info@linuxvoice.com,
Wireshark network
it should talk to smtp.linuxvoice.com, port 25/tcp. 140.20.145.198.in-addr.arpa. 5 IN PTR tiz-korg-pub.
analyser can easily decode
Now, consider what www.kernel.org really is: kernel.org. both DNS queries and their
www.kernel.org. 599 IN CNAME pub.all.kernel.org. PTR is for Pointer; such records just point to some responses.
pub.all.kernel.org. 599 IN A 198.145.20.140 other RR. To do a reverse mapping, reverse the
CNAME stands for Canonical Name. Put simply, this IPv4 address octets and append the in-addr.arpa.
record says that www.kernel.org is just an alias for suffix. Recall that right-to-left is the natural DNS label
pub.all.kernel.org, which resolves to 198.145.20.140. processing order. So, one can map IPV4 subnets to
Note that IP address comes from a separate 'A' record. DNS zones and delegate them to different servers.
DNS also supports reverse mapping, or finding a Resource records may even contain some arbitrary
name by IP address. Naturally, there are hosts that data. The TXT type is reserved for that. Several
don't have a registered name, but those that do should technologies rely on TXT records, including DKIM and
also have a PTR record in the in-addr.arpa domain: SPF, which help fight spam.
I hope you'd agree now that DNS is little bit more
than an internet-scale phonebook. If so, let's learn how
Resolving in Glibc
DNS questions get their answers.
Glibc, the most popular C library in Linux so far, provides its
own stub DNS resolver. Available via gethostbyname() and A day with DNS resolver
related library functions, it comes as a part of the Name
Service Switch (NSS) subsystem.
DNS messages are usually UDP datagrams coming
NSS exists because DNS is not the only way to resolve from (or sent to) port 53. A maximum message size
host names in Glibc. Other means, say, looking up the /etc/ is 512 bytes. TCP connections are supported as well.
hosts file, are also possible. New methods are installed as However, they are mainly useful for data-intensive
NSS plugins and configured via /etc/nsswitch.conf: operations, like zone transfers which happen between
hosts: files mdns4_minimal [NOTFOUND=return] dns
This is the typical configuration for host name resolver.
name servers.
The routine starts at /etc/hosts then proceeds to mDNS. If RFC 1035 defines a single message format for
the name is in the local. domain, but nss-mdns was unable queries and responses. It includes a header with
to find it, the procedure terminates. Otherwise, a "normal" several bit fields (flags) and four sections: Question,
DNS query is made. Glibc uses another configuration file, Answer, Authority and Additional. Question consists
/etc/resolv.conf, to learn which DNS servers to contact and
which domains to try for unqualified names:
of a class, type and name of the record we're looking
search company.com for. The answer section contains records (if any)
nameserver 10.1.0.1 that directly answers the query. Authority refers to
You are unlikely to edit /etc/resolv.conf on your Linux authoritative name servers for the target domain. It
desktop, as various tools like NetworkManager take care of doesn't mean that any of this servers answered the
it automatically. To resolve a name via NSS, use the getent
command we introduced back in LV023:
query; if so, an AA bit would be set in the response
$ getent hosts www.linuxvoice.com flags. An additional section provides related records
2400:cb00:2048:1::681c:612 www.linuxvoice.com that the name server thought would be helpful for the
You see that Linux prefers IPv6 addresses where they are client. For instance, it may contain A records for name
available. /etc/nsswitch.conf is also "in charge" for other servers referenced in the Authority section.
system names, like users and groups. See nsswitch.conf(5)
for details.
Queries are resolved recursively. If a server is not
authoritative for the domain in query, and it doesn't
www.linuxvoice.com 95
CORETECHNOLOGY
VeriSign hosts com., but many of its subdomains
are delegated. So dig learns that there are two name
servers at CloudFlare that are authoritative for the
linuxvoice.com. subdomain:
linuxvoice.com. 172800 IN NS heather.ns.
cloudflare.com.
linuxvoice.com. 172800 IN NS yichun.ns.
cloudflare.com.
;; Received 677 bytes from 192.43.172.30#53(i.gtld-
servers.net) in 97 ms
www.example.com
The only thing left is to ask one of them for the
www.linuxvoice.com. IPv4 address:
www.linuxvoice.com. 300 IN A 104.28.7.18
www.linuxvoice.com. 300 IN A 104.28.6.18
;; Received 79 bytes from 173.245.59.248#53(yichun.ns.
cloudflare.com) in 86 ms
Phew! That was quite a job, and it's probably why
A simple DNS-based load most operating systems (Linux/glibc included) today
balancer: connecting come with a stub resolver.
clients resolve one 1.2.3.4 1.2.3.5 1.2.3.6 DNS servers are very important bits of the internet.
hostname to different IPs. But in fact, you can create a working DNS system
without any servers at all. Moreover, this thing is
have a relevant record in its cache, it refers to another essential for zero-configuration networking. And it
server that may have the information. It could be an involves lemurs. Prepare to say:
authoritative server for some "intermediate" domain,
like co.uk, or one of the root DNS servers. A recursive "Bonjour, Avahi!"
DNS server would follow the referral itself (and cache Apple calls it Bonjour, Linux implementation is
the response), while non-recursive may simply return codenamed Avahi. Either is a piece of software that
it to the client. Server-side recursion is optional, and wraps Multicast DNS (mDNS) and DNS-Based Service
clients indicate their intention to use it with the RD Discovery (DNS-SD) protocols.
(Recursion Desired) bit in the header. An RA bit set IP multicasting is like broadcasting in that a
in response means Recursion is Available. Iterative single packet reaches multiple recipients. But unlike
clients can do recursion themselves; stub resolvers broadcasting, a host must explicitly subscribe to a
rely on servers. multicast group before it gets any data. It is akin to a
Let's do a live experiment. We'll take dig and radio exchange where all peers must tune to the same
trace how it resolves a name. dig is a sophisticated frequency channel to hear each other. IPv4 reserves
recursive resolver that builds on the same codebase addresses in the range 224.0.0.0–239.255.255.255
as BIND, "the most widely used name server software". for multicast traffic.
Resolving a name with dig is as simple as typing: mDNS uses IP multicast as a primary transport. All
$ dig www.linuxvoice.com. +trace mDNS-capable hosts join the 224.0.0.251 multicast
The +trace bit commands dig to trace the domain group and send messages to port 5353/udp. Peers
delegation path from the root. It produces a lengthy can see each others' questions and answers and learn
output, so I've cut many lines from the samples below. from them, effectively working as a distributed DNS
; <<>> DiG 9.10.3 <<>> www.linuxvoice.com. +trace server with no central authority.
;; global options: +cmd
. 142 IN NS d.root-servers.net.
Name servers galore
;; Received 913 bytes from 192.168.101.1#53(192.168.101.1)
in 21 ms Linux isn't short of name server implementations. BIND
These are NS records for root DNS servers. They (https://www.isc.org/downloads/bind) is the de-facto
standard. It has many features, and you may think it's a way
are authoritative for the '.' domain, and play a crucial too much for your small home or office network.
role in internet operation. Thirteen root DNS servers If this is the case, try Dnsmasq (www.thekelleys.org.
exist, so their names fit in one DNS message. A uk/dnsmasq/doc.html). This daemon provides all-in-one
cluster of boxes is really running under each name, so infrastructure for small networks, including caching DNS
real redundancy is even greater. and DHCP server. Chances are you already use Dnsmasq
without even noticing it, as it comes bundled in many
dig chooses one of these servers, which happens wireless routers. Virt-manager also relies on Dnsmasq to
to be from University of Maryland, and asks it for a provide name services in its virtual networks. There is
referral for the com. domain: also Unbound (www.unbound.net), a caching DNS resolver
com. 172800 IN NS i.gtld-servers.net. library. It provides C and Python APIs, along with a caching
;; Received 742 bytes from 199.7.91.13#53(d.root-servers. recursive resolver daemon built on top of these APIs.
net) in 500 ms
96 www.linuxvoice.com
CORETECHNOLOGY
mDNS operates in a special domain, local.. It Avahi includes graphical
provides flat namespace, so laptop.local. is permitted, tools to browse networks
but mediaserver.livingroom.local. isn't, although for various services,
DNS-SD introduces more levels. mDNS names don't including SSH.
have to be unique. In fact, DNS-SD PTR records are
purposely shared. Shared host names don't make any
sense though, and there are mechanisms to ensure
this doesn't happen.
When an mDNS host becomes online, it sends
probe queries for names it wants to be unique. Any
host on the net that already owns such a name
should promptly reply to "defend" it. If this doesn't
happen within a second, the first host assumes there
is no conflict and makes an announcement. Should a
conflict arise at some later point, there is an arbitration
algorithm to solve it. printing support. Underscores aren't permitted in DNS,
mDNS also introduces new "ongoing query" so they are introduced to prevent name clashes.
mechanics, which fits well into the network browsing This PTR record resolves to instance._printer._tcp.
use case. The query doesn't stop with the first answer, local. The instance part is end-user visible, so it
but repeats at predefined intervals to learn about should be descriptive. An SRV record must exist for
changes in the network. To advertise support for this name, providing both host and port the service is
this feature, mDNS peers send datagrams from port available. Optionally, the name may also have a TXT
5353/udp. Otherwise, mDNS messages are almost record, containing additional connection data. This is
the same as DNS messages, so you can use dig to useful for legacy protocols, like LPR.
make mDNS "legacy" (ie non-continuous) queries. Let's try it in action. First, I'll run dig to see which
Just use 224.0.0.251 as the 'server' and 5353 as printers are available in the vicinity:
target port. $ dig @224.0.0.251 -p 5353 -t ptr _printer._tcp.local.
In Linux, two components implement mDNS. nss- ;; ANSWER SECTION:
mdns is an mDNS resolver wrapped as a Glibc NSS _printer._tcp.local. 10 IN PTR EPSON\032WF-
plugin. Add it to /etc/nsswitch.conf, and you'll be able 2010\032Series._printer._tcp.local
to use local. names like any other: -t ptr tells dig that I want a PTR record; -p specifies
[val@y550p ~]$ ping -c 1 y550p.local the port. Now we know there is at least one printer
PING y550p.local. (10.1.0.206) 56(84) bytes of data. nearby. There could be more, but dig doesn't speak
64 bytes from y550p.local (10.1.0.206): icmp_seq=1 ttl=64 mDNS well enough to find them all. Still, I can use it to
time=0.082 ms query for the printer's details:
Avahi provides an mDNS responder that registers $ dig @224.0.0.251 -p 5353 -t any 'EPSON WF-2010
your names and services on the network and answers Series._printer._tcp.local.'
mDNS queries as they come. Usually, it requires no ;; ANSWER SECTION:
manual setup: that's the point of zero-configuration, EPSON\032WF-2010\032Series._printer._tcp.local. 10 IN
after all. Avahi also provides some tools, like avahi- SRV 0 0 515 EPSON4E85C9.local.
browse to browse for services, or avahi-publish to EPSON\032WF-2010\032Series._printer._tcp.local. 10 IN
publish them. Although the examples below use dig, TXT "txtvers=1" "priority=50" "rp=auto" ...
it's purely for illustration purposes. Avahi tools are the ;; ADDITIONAL SECTION:
way to go in real-world scenarios. EPSON4E85C9.local. 10 IN A 192.168.101.158
In mDNS, the ANY record type yields all records
Hunting for printers for the given name. As you might expect, there
DNS-SD is a generic DNS-based mechanism to are two. The SRV record says that the printer is at
browse for network services. It's not tied to mDNS, EPSON4E85C9.local., port 515. Two zeroes are the
but both are often found together. Put simply, mDNS service's priority and weight. The TXT record contains
is about naming in general, and DNS-SD is about many key-value pairs I omitted for brevity. Among
naming (and locating) services. A service can be them, rp stores the queue name for the printer driver
almost anything: a workstation, a file server, a printer, to use. An mDNS responder on the printer was also
or even an SSH instance running at some Linux host. kind enough to include an A record in the Additional
Imagine you have a network printer. When you start section and save us an extra DNS query.
it, the printer claims ownership over some unique That's it! Now we can open a TCP connection to
mDNS hostname, but also announces shared a PTR 192.168.101.158:515 and submit a printer job. No
record for a specifically crafted name, _printer._tcp. end-user setup was necessary. Also note how the
local.`The . _tcp means that the printer is available instance name differs from abbreviations like prn120
over TCP. Anything else would use _udp, regardless of that you usually encounter in a "big" DNS. Zero-
the actual transport protocol. _printer advertises LPR configuration magic in action!
www.linuxvoice.com 97
/DEV/RANDOM/ FINAL THOUGHTS
Final thoughts, musings and reflections
Nick Veitch
was the original editor Headphones and Beer Street, by Hogarth. A
of Linux Format, a microphone – my desk reminder of what can be achieved
role he played until he is also the northern if you drink enough beer.
got bored and went studio of the Linux
to work at Canonical Voice podcast.
instead. Splitter!
Acer Aspire 5742 running Ubuntu
14.04 Trusty. 320GB hard drive, still
not filled after about five years’ use.
I
remember an acquaintance of mine
telling me about his unfortunate
experience with a sort-of-smart car. It
The shatter’d visage
was the old story of the “service me now” of my Nexus 5 Google
light coming on, and being ignored to the spy device. Fixing it
was enormous fun. Begbie Thin White Duke
point where, once stopped for a rest-break, Henderson-Gregory, my black
the vehicle decided it was no longer going to pudding-loving spirit animal.
move until it had been properly seen to by a
qualified engineer with the magic box
necessary to talk to the car into resuming
normal service.
Sadly in this case, the rest-stop was just MY LINUX SETUP
the side of the road in the middle of a South
African desert, a good few miles from the
nearest qualified engineer, any engineer, or
REV. ANDREW J GREGORY
possibly any other person at all. Podcaster, editorial director, permanent Linux newbie.
Smart cars should be smart, not just at
leveraging profit for their overlords. Maybe in What version of Linux are you others until I got SUSE working. I stuck with
the future when we can talk to our vehicles, currently using? SLED until Novell signed a patent
we may be able to talk them out of being so These days I flit back and forth partnership with Microsoft in late 2006,
stubborn. In the meantime, one of the best between Ubuntu and Mint, and at the which led me to Debian, then to Ubuntu.
ways to ensure that the ever-more moment it’s Ubuntu 14.04. It’s an old version You’ve got to vote with your feet sometimes.
computerised cars of the future aren’t because I fear change.
crippled by DRM-like lock-in, crippleware and What Free Software/open source
worse is to promote the idea of open source And what desktop are you using at can’t you live without?
and open standards. Not everyone is looking the moment? LibreOffice. It’s amazing that this
forward to the day when you have to run a That would be Unity. It has its faults, brilliantly useful software is available
virus-check or watch a sponsored ad before and I really should get round to to us all for free. And it’s not so much
the school run in the morning. switching to Mate, which has become the software, but the lack of viruses means that
It is gratifying then that at least some future of the Linux desktop by sticking with I can eke out hardware for longer than I
collaborative efforts are taking place in this and refining all the things that were good would otherwise, which saves me £££s.
arena. The Linux Foundation has its about desktops in the past – namely, it’s
Automotive Grade Linux project, which simple to use and looks good. What do other people love but you
seems to have the backing of many players can’t get on with?
in this space, including several big-name What was the first Linux setup you I only ever use a text editor to tweak
manufacturers. you may get some idea of ever used? configuration files now and then, so
what the commute of the Tux-based future Oooh, Mandrake something or other, the continuing saga of Vim vs Emacs leaves
might look like here: in 2005. Well, I say used – there was a me baffled. Gedit does the job just fine, (or
www.automotivelinux.org graphics incompatibility, so I tried a few Sublime Text if you’re feeling fancy).
98 www.linuxvoice.com
This is what we’ve done in the last 24 issues.
Subscribe to the next 12 from just £38.
Every subscription includes access to every PDF, ePub and audio edition we’ve ever published.
shop.linuxvoice.com
SUBSCRIBE
www.linuxvoice.com
SUBSCRIBE
www.linuxvoice.com