Development

Quickstart

cp .env-dev.dist .env
# Edit .env if need be
cp docker-compose-dev.yml docker-compose.yml
docker-compose build
docker-compose run --rm web update.sh
docker-compose run --rm web load_data.sh
docker-compose run --rm web ./manage.py createsuperuser
docker-compose up -d

Go to http://localhost:8000

PDF generation might not work unless you add the following entry to /etc/hosts and use geotrek.local:8000 to access Geotrek.

127.0.0.1   localhost geotrek.local

Contribution guide

Conventions

• Use flake8

• KISS & DRY as much as possible

• Elegant and generic is good, simple is better

• Commits messages are explicit and mention issue number ((ref #12) or (fixes #23))

• Features are developed in a branch and merged from Github pull-requests. A git hook to is available to prevent pushing to master, to enable it, developpers should run the following command from root directory (Geotrek-admin/) : ln -s -f ../../.githooks/pre-push .git/hooks/pre-push

Definition of done

• docs/changelog.rst is up-to-date

• A unit-test covers the bugfix or the new feature

• A frontend test (:path:jstests/nav-*.js) covers the navigation bug fix or feature

• A JS Mocha test (:path:jstests/tests.*.js) covers the JavaScript bug fix or feature

• Unit-tests coverage is above or at least equal with previous commits

• Settings have default value in settings/base.py or conf/settings-default.ini

• Installation instructions are up-to-date

Check TODO in the source tree:

find geotrek | xargs egrep -n -i '(TODO|XXX|temporary|FIXME)'

Release

On master branch:

• If need be, merge translations branch managed with https://weblate.makina-corpus.net

• Update files VERSION, docs/conf.py and docs/changelog.rst to remove +dev suffix and increment version (please use semver rules)

• Run dch -r -D RELEASED, update version in the same way and save

• Commit with message ‘Release x.y.z’ to merge in master branch before release

• Add git tag X.Y.Z

• Update files VERSION, docs/conf.py and docs/changelog.rst to add +dev suffix

• Run dch -v <version>+dev --no-force-save-on-release and save

• Commit with message ‘Back to development’

• Push branch and tag

• When pushing a release tag ‘x.y.z’, CircleCI will generate the .deb package file, and publish it on https://packages.geotrek.fr (see .circleci/config.yml file for details)

• Add release on Github (copy-paste doc/changelog.rst paragraph)

Developement

Model modification

docker-compose run web ./manage.py makemigrations <appName>
docker-compose run web ./manage.py migrate

Note

Add migration file to source control.

Model modification - Definition of Done for new model fields

When updating or adding a new field my_field to a model MyModel, please proceed with the following changes to ensure this field is included in existing functionalities.

In MyModel class :

• If my_field is a ForeignKey, make sure to override related_name with an explicit set name

• Make sure to set verbose_name on the field and add proper translations in .po files

Outside of model class :

• To display my_field in detail views, add a row in template mymodel_detail_attributes.html

• Look for form class MyModelForm(CommonForm) :

  • If it exists, and field needs to be included in form, add my_field to form attributes (fields on the Meta class, sometimes fieldslayout as well).

  • If field is added to the form and is optional, please add my_field to the documentation for hideable form fields : in docs/advanced-configuration.rst look for HIDDEN_FORM_FIELDS['mymodel'] and add your field to the list.

• Look for list view class MyModelList(CustomColumnsMixin, MapEntityList) :

  • If it exists, please add my_field to the documentation for custom list view columns : in docs/advanced-configuration.rst look for COLUMNS_LISTS['mymodel_view'] and add your field to the list.

  • If it exists, and if you wish to display a column for my_field in the list view for this model by default, simply add my_field to default_extra_colums on this class.

• Look for exports view class MyModelFormatList(MapEntityFormat, MyModelList) :

  • If it exists, please add my_field to the documentation for custom list exports columns : in docs/advanced-configuration.rst look for COLUMNS_LISTS['mymodel_export'] and add your field to the list.

  • If it exists, and if you wish to display a column for my_field in CSV/SHP exports for this model by default, simply add my_field to default_extra_colums on this class.

• Follow the documentation you just edited to test that custom columns and hideable fields do work properly with your new field.

• Look for sql file defaults geotrek/{app_name}/sql/post_90_defaults.sql :

  • If it exists find your modelname in the list and depending on the default value alter column my_field or add -- my_field

  • If the modelname doesn’t exist, create a new section (even if you don’t need to alter column)

• Look for sql view file geotrek/{app_name}/sql/post_20_views.sql and update the view for your model with an alias for the new field

In API v2 :

If MyModel is served by APIv2, make sure to add a serializer for the new field in geotrek/api/v2/serializers.py and if you wish to filter on this field, create a new filter and add it to the right ViewSet under geotrek/api/v2/views, using attribute filter_backends.

When updating a field my_field in a model MyModel for new_field, check if this field is translated in geotrek/{app}/translation.py.

If so, you need to add a migration just after the migration generated by django. This migration should rename the old fields generated by modeltranslation my_field_en by new_field_en (example : geotrek/trekking/migrations/0014_auto_20200228_2127.py)

Run tests

Django tests :

ENV variable must be set to run tests:

docker-compose run --rm -e ENV=tests web ./manage.py test

Test without dynamic segmentation:

docker-compose run --rm -e ENV=tests_nds web ./manage.py test

Cypress tests :

Create an empty project with docker :

docker-compose down
docker-compose up -d

Install elements for the cypress tests

make load_data
make load_test_integration
make load_test_integration_workflow

Move in cypress folder and install

cd cypress
npm ci

Launch tests

./node_modules/.bin/cypress run

Pictures of the problem and videos are generated in cypress/videos and cypress/screenshots

Setup to run rando synchronization locally

(sync rando is only relevant for an admin paired with geotrek rando v2)

In your django settings you must set the ‘url’ key of SYNC_RANDO_OPTIONS to use the same domain as defined by SERVER_NAME in your .env.

For instance with SERVER_NAME=geotrek.local (default value)

SYNC_RANDO_OPTIONS = {
        'url': 'http://geotrek.local:8000'
}

Setup to use screamshotter-related features locally

Use the domain defined in SERVER_NAME in your .env to reach your local geotrek admin web instance. By default the address is http://geotrek.local:8000.

Update the /etc/hosts on your machine to resolve geotrek.local on 127.0.0.1.

$ cat /etc/hosts
127.0.0.1       geotrek.local
127.0.0.1       localhost

Database reset

Data only:

docker-compose run web ./manage.py flush

Restore existing Database

Assuming a dump of your database is located in your project directory:

docker-compose run --rm web pg_restore -h $POSTGRES_HOST -U $POSTGRES_USER -d $POSTGRES_DB /opt/geotrek/<path_to_backup>.dump

Restore your ./var/conf/ project files, and data files into ./var/media.

Then run a synchronization.

Mapentity development

TODO

UML diagrams of data model

UML diagrams of Geotrek-admin data models are available in docs/data-model directory. To regenerate them from PostgreSQL, install postgresql-autodoc and graphviz Ubuntu packages and run make uml.

Documentation

A container based on sphinx image is created using docker-compose-dev.yml, documentation is built in watch mode thanks to sphinx-autobuild.

Access to documentation built in html : http://0.0.0.0:8800

Translate documentation

• Generate .pot if needed

docker-compose run --rm sphinx make gettext

• Generate .po files

docker-compose run --rm sphinx sphinx-intl update -p _build/locale -l fr