.. _development-section:

===========
Development
===========

Quickstart
==========

::

    cp .env-dev.dist .env
    # Edit .env if need be
    cp docker-compose-dev.yml docker-compose.yml
    docker-compose build
    docker-compose run --rm web update.sh
    docker-compose run --rm web load_data.sh
    docker-compose run --rm web ./manage.py createsuperuser
    docker-compose up -d

Go to http://geotrek.localhost:8000

PDF generation might not work unless you use this domain and is correctly set to SERVER_NAME variable in your .env file.


Install git hooks
=================

* Several git hooks are available to prevent pushing to master branch or launch quality tests before committing. Install them with following commands:

::

    ln -s -f ../../.githooks/pre-push .git/hooks/pre-push
    ln -s -f ../../.githooks/pre-commit .git/hooks/pre-commit


Adding or upgrade dependencies
==============================

Consider using pip-tools to manage dependencies.

* add your dependency in setup.py for general dependency, dev-requirements.in for dev dependency, then run :

::

   docker-compose run --rm web pip-compile
   docker-compose run --rm web pip-compile dev-requirements.in

or

::

   make deps


Model modification
==================

::

   docker-compose run --rm web ./manage.py makemigrations <appName>
   docker-compose run --rm web ./manage.py migrate

.. note ::

    Add migration file to source control.

Definition of Done for new model fields
---------------------------------------

When updating or adding a new field ``my_field`` to a model ``MyModel``, please proceed with the following changes to ensure this field is included in existing functionalities.

**In** ``MyModel`` **class** :

- If ``my_field`` is a ``ForeignKey``:

    - make sure to override ``related_name`` with an explicit set name

    - if ``my_field`` causes cascading deletion (argument ``on_delete=models.CASCADE``), make sure to log potential deletions (see example ``log_cascade_deletion_from_sector_practice`` in ``geotrek/outdoor/models.py``)

- Make sure to set ``verbose_name`` on the field and add proper translations in ``.po`` files

**Outside of model class** :

- To display ``my_field`` in detail views, add a row in template ``mymodel_detail_attributes.html``

- Look for form class ``MyModelForm(CommonForm)`` :

    - If it exists, and field needs to be included in form, add ``my_field`` to form attributes (``fields`` on the ``Meta`` class, sometimes ``fieldslayout`` as well).

    - If field is added to the form **and is optional**, please add ``my_field`` to the documentation for hideable form fields : in ``docs/advanced-configuration.rst`` look for ``HIDDEN_FORM_FIELDS['mymodel']`` and add your field to the list.

- Look for list view class ``MyModelList(CustomColumnsMixin, MapEntityList)`` :

    - If it exists, please add ``my_field`` to the documentation for custom list view columns : in ``docs/advanced-configuration.rst`` look for ``COLUMNS_LISTS['mymodel_view']`` and add your field to the list.

    - If it exists, and if you wish to display a column for ``my_field`` in the list view for this model by default, simply add ``my_field`` to ``default_extra_colums`` on this class.

- Look for exports view class ``MyModelFormatList(MapEntityFormat, MyModelList)`` :

    - If it exists, please add ``my_field`` to the documentation for custom list exports columns : in ``docs/advanced-configuration.rst`` look for ``COLUMNS_LISTS['mymodel_export']`` and add your field to the list.

    - If it exists, and if you wish to display a column for ``my_field`` in CSV/SHP exports for this model by default, simply add ``my_field`` to ``default_extra_colums`` on this class.

- Follow the documentation you just edited to test that custom columns and hideable fields do work properly with your new field.

- Look for sql file defaults ``geotrek/{app_name}/sql/post_90_defaults.sql`` :

    - If it exists find your modelname in the list and depending on the default value alter column ``my_field`` or add ``-- my_field``

    - If the modelname doesn't exist, create a new section (even if you don't need to alter column)

- Look for sql view file ``geotrek/{app_name}/sql/post_20_views.sql`` and update the view for your model with an alias for the new field


**In API v2** :

If ``MyModel`` is served by APIv2, make sure to add a serializer for the new field in ``geotrek/api/v2/serializers.py`` and if you wish to filter on this field, create a new filter and add it to the right ``ViewSet`` under ``geotrek/api/v2/views``, using attribute ``filter_backends``.


When updating a field ``my_field`` in a model ``MyModel`` for ``new_field``, check if this field is translated in ``geotrek/{app}/translation.py``.

If so, you need to add a migration just after the migration generated by django.
This migration should rename the old fields generated by modeltranslation ``my_field_en`` by ``new_field_en``
(example : ``geotrek/trekking/migrations/0014_auto_20200228_2127.py``)


Check quality
=============

**Flake8**

run:

::

   docker-compose run --rm web flake8 geotrek


or

::

   make flake8


Run tests
=========

**Django tests :**

``ENV`` variable must be set to run tests:

::

   docker-compose run --rm -e ENV=tests web ./manage.py test

Test without dynamic segmentation:

::

   docker-compose run --rm -e ENV=tests_nds web ./manage.py test


**Cypress tests :**

Create an empty project with docker :

::

    docker-compose down
    docker-compose up -d


Install elements for the cypress tests

::

    make load_data
    make load_test_integration
    make load_test_integration_workflow


Move in cypress folder and install

::

    cd cypress
    npm ci


Launch tests

::

    ./node_modules/.bin/cypress run


Pictures of the problem and videos are generated in cypress/videos and cypress/screenshots

Setup to run rando synchronization locally
==========================================

(sync rando is only relevant for an admin paired with geotrek rando v2)

In your django settings you must set the 'url' key of SYNC_RANDO_OPTIONS to use the same domain as defined by SERVER_NAME in your .env.

For instance with SERVER_NAME=geotrek.localhost (default value)

.. code-block :: python

    SYNC_RANDO_OPTIONS = {
        'url': 'http://geotrek.localhost:8000'
    }

Setup to use screamshotter-related features locally
===================================================

Use the domain defined in SERVER_NAME in your .env to reach your local geotrek admin web instance. By default the address is `http://geotrek.localhost:8000`.


Database reset
==============

Data only:

::

   docker-compose run --rm web ./manage.py flush

Restore existing Database
=========================

Assuming a dump of your database is located in your project directory:

::

   docker-compose run --rm web pg_restore --clean --no-owner --no-acl -h $POSTGRES_HOST -U $POSTGRES_USER -d $POSTGRES_DB /opt/geotrek-admin/<path_to_backup>.dump

Restore your ``./var/conf/`` project files, and data files into ``./var/media``.

Then run a synchronization.

Mapentity development
=====================

TODO


UML diagrams of data model
==========================

UML diagrams of Geotrek-admin data models are available in ``docs/data-model`` directory.
To regenerate them from PostgreSQL, install postgresql-autodoc and graphviz Ubuntu packages
and run ``make uml``.

Documentation
=============

A container based on sphinx image is created using docker-compose-dev.yml,
documentation is built in watch mode thanks to sphinx-autobuild.

Access to documentation built in html : http://0.0.0.0:8800


Translate documentation
-----------------------

- Generate .pot if needed

.. code-block :: python

    docker-compose run --rm sphinx make gettext

- Generate .po files

.. code-block :: python

    docker-compose run --rm sphinx sphinx-intl update -p _build/locale -l fr