Internationalization (i18n)

We're using gettext for translations. All the actual translations are carried out via Pontoon.

Some commands wrap standard gettext tools. To run these commands you'll need to ensure you have done the following steps:

  • Run yarn to install all the project dependencies.
  • Install the gettext tools for your platform and make sure they're on your $PATH by checking the output of which gettext.

NOTE: All the instructions below show [MY_APP]; replace that with the name of the app you are updating, e.g. NODE_APP_INSTANCE=disco bin/create-locales

Adding a new language/locale

The supported languages are defined in the configuration. See config/default.js and look for the langs list.

Add the new language to the list and then run:

# create the locale for a newly added language.
NODE_APP_INSTANCE=[MY_APP] NODE_PATH='./:./src' bin/create-locales

Updating locales

Once a week right after the forthcoming release is tagged, the locales for each app must be generated.

This is a semi-automated process: a team member must create one pull request per application with the following commits:

  1. A commit containing the extraction of newly added strings
  2. A commit containing a merge of localizations

Each one of these steps are detailed in the sections below. Let's begin...

Extracting newly added strings

Start the process by creating a git branch and extracting the locales for a given app. This uses amo as an example app but you would need to repeat the process in a new branch for disco and any other activate application (example: NODE_APP_INSTANCE=disco ...).

git checkout master
git pull
git checkout -b amo-locales
NODE_APP_INSTANCE=amo bin/extract-locales

This extracts all strings wrapped with i18n.gettext() or any other function supported by Jed (the library we use in JavaScript to carry out replacements for the string keys in the current locale).

The strings are extracted using a babel plugin via webpack. Extracted strings are added to a pot template file. This file is used to seed the po for each locale with the strings needing translating when merging locales.

Run git diff to see what the extraction did. If no strings were updated then you do not have to continue creating the pull request. You can revert the changes made to the pot timestamp. Here is an example of a diff where no strings were changed. It just shows a single change to the timestamp:

diff --git a/locale/templates/LC_MESSAGES/amo.pot b/locale/templates/LC_MESSAGES/amo.pot
index 31e113f2..c7da4e34 100644
--- a/locale/templates/LC_MESSAGES/amo.pot
+++ b/locale/templates/LC_MESSAGES/amo.pot
@@ -2,7 +2,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: amo\n"
 "Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
-"POT-Creation-Date: 2017-06-08 14:01+0000\n"
+"POT-Creation-Date: 2017-06-08 14:43+0000\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language-Team: LANGUAGE <LL@li.org>\n"

When the application is under active development it's more likely that you will see a diff containing new strings or at least strings that have shifted to different line numbers in the source. If so, commit your change and continue to the next step:

git commit -a -m "Extract AMO locales"

Merging locale files

After extracting new strings, you have to merge them into the existing locale files. Do this in your branch and commit:

NODE_APP_INSTANCE=amo bin/merge-locales

Keep an eye out for fuzzy strings by running git diff and searching for a comment that looks like # fuzzy. This comment means the localization may not exactly match the source text; a localizer needs to review it. As per our configuration, the application will not display fuzzy translations. These strings will fall back to English.

In some rare cases you may wish to remove the fuzzy marker to prevent falling back to English. Discuss it with a team member before removing fuzzy markers.

Commit and continue to the next step:

git commit -a -m "Merged AMO locales"

Finalizing the extract/merge process

Now that you have extracted and merged locales for one application, it's time to create a pull request for your branch. For example:

git push origin amo-locales

If the pull request passes all of our CI tests it is likely good to merge. You don't need to ask for a review unless you're unsure of something because often locale updates will be thousands of lines of minor diffs that can't be reasonably reviewed by a human. 🙂 If the pull request passes all of our CI tests it is likely good to merge. If necessary, repeat the process for the next application (eg. amo, disco, etc.).

Building the JS locale files

This command creates the JSON files which are then built into JS bundles by webpack when the build step is run. This happens automatically as part of the deployment process.

Since dist files are created when needed you only need to build and commit the JSON to the repo.

# build the JSON.
NODE_APP_INSTANCE=[MY_APP] bin/build-locales

Setting up translations

To set up a component to be translated there are two pieces of code to know about.

Jed

We use Jed as the API for providing gettext functions inside React components. An initialized Jed instance has all the gettext related functionality exposed as methods. There is a fancy chained API but we've stuck to a more traditional approach.

Before we get into how to make use of these functions let's take a look at how the Jed instance is exposed to our components.

The Translation Provider

The translation provider is used to pass down a Jed instance via context to components lower down in the component hierarchy. This part is already done for you in addons-frontend. So you should only need to worry about wrapping your components as detailed in the next section.

The translate component wrapper.

The translate Higher Order Component is a helper that wraps any component and takes the Jed i18n instance from context and makes it available in the wrapped component's props.

Here's an example of a basic component setup for translation:

import * as React from 'react';
import PropTypes from 'prop-types';

import translate from 'core/i18n/translate';

export class MyTranslatedComponent extends React.Component {
  static propTypes = {
    i18n: PropTypes.object.isRequired,
  };

  render() {
    const { i18n } = this.props;
    return (
      <div>
        <p>{i18n.gettext('Something translated')}</p>
      </div>
    );
  }
}

export default translate()(MyTranslatedComponent);

That's pretty much all there is to it.

Using the Jed API

Once you have i18n available to your component you can then use any of the Jed methods exposed on the i18n object.

gettext = function ( key )
dgettext = function ( domain, key )
dcgettext = function ( domain, key, category )
ngettext = function ( singular_key, plural_key, value )
dngettext = function ( domain, singular_ley, plural_key, value )
dcngettext = function ( domain, singular_key, plural_key, value, category )
pgettext = function ( context, key )
dpgettext = function ( domain, context, key )
npgettext = function ( context, singular_key, plural_key, value )
dnpgettext = function ( domain, context, singular_key, plural_key, value )
dcnpgettext = function ( domain, context, singular_key, plural_key, value, category )
sprintf = function ( string, substitutions)

Using sprintf

As you can see a sprintf function is also available. You can use this to provide substitutions in gettext wrapped strings.

There are two flavours to this, numbered placeholders or named ones.

Here's the numbered approach:

i18n.sprintf(i18n.gettext('I like your %1$s %2$s.'), 'red', 'shirt'));

and here's the named arg approach:

i18n.sprintf(i18n.gettext('I like your %(colour)s %(garment)s.'), { colour: 'red', garment: 'shirt' }));

Both of these approaches allow for translators to re-order the substitution vars.

Guidance on HTML in translations

Generally we're looking to avoid having HTML in the middle of translation strings as much as possible.

If you need HTML it's better to use substitutions to add the HTML than leave HTML in the translation. Take the following string as an example:

i18n.gettext('Take a look at the <a href="#">documentation</a>');

Using sprintf we can provide use start and end substitutions. This way there's no HTML in the extracted string.

i18n.sprintf(
  i18n.gettext('Take a look at the %(start_link)sdocumentation%(end_link)s'),
  { start_link: '<a href="#">', end_link: '</a>' },
);

You can also use DOMPurify to sanitize strings that may contain HTML following substitutions so that anything not explicitly allowed is removed. DOMPurify will also help protect against malformed HTML in case opening and closing tag substitutions vars get swapped around inadvertently.