Defining the Domain Model

The first change we'll make to our stock cookiecutter-generated application will be to define a wiki page domain model.

Note

There is nothing special about the filename user.py or page.py except that they are Python modules. A project may have many models throughout its codebase in arbitrarily named modules. Modules implementing models often have model in their names or they may live in a Python subpackage of your application package named models (as we've done in this tutorial), but this is only a convention and not a requirement.

Declaring dependencies in our setup.py file

The models code in our application will depend on a package which is not a dependency of the original "tutorial" application. The original "tutorial" application was generated by the cookiecutter; it doesn't know about our custom application requirements.

We need to add a dependency, the bcrypt package, to our tutorial package's setup.py file by assigning this dependency to the requires parameter in the setup() function.

Open tutorial/setup.py and edit it to look like the following by adding bcrypt and sorting the packages:

requires = [
    'alembic',
    'bcrypt',
    'plaster_pastedeploy',
    'pyramid',
    'pyramid_debugtoolbar',
    'pyramid_jinja2',
    'pyramid_retry',
    'pyramid_tm',
    'SQLAlchemy',
    'transaction',
    'waitress',
    'zope.sqlalchemy',
]

It is a good practice to sort packages alphabetically to make them easier to find. Our cookiecutter does not have its packages sorted because it merely tacks on additional packages depending on our selections.

Note

We are using the bcrypt package from PyPI to hash our passwords securely. There are other one-way hash algorithms for passwords if bcrypt is an issue on your system. Just make sure that it's an algorithm approved for storing passwords versus a generic one-way hash.

Running pip install -e .

Since a new software dependency was added, you will need to run pip install -e . again inside the root of the tutorial package to obtain and register the newly added dependency distribution.

Make sure your current working directory is the root of the project (the directory in which setup.py lives) and execute the following command.

On Unix:

$VENV/bin/pip install -e .

On Windows:

%VENV%\Scripts\pip install -e .

Success executing this command will end with a line to the console something like the following.

Successfully installed bcrypt-3.2.0 cffi-1.14.4 pycparser-2.20 tutorial

Remove mymodel.py

Let's delete the file tutorial/models/mymodel.py. The MyModel class is only a sample and we're not going to use it.

Add user.py

Create a new file tutorial/models/user.py with the following contents:

import bcrypt
from sqlalchemy import (
    Column,
    Integer,
    Text,
)

from .meta import Base


class User(Base):
    """ The SQLAlchemy declarative model class for a User object. """
    __tablename__ = 'users'
    id = Column(Integer, primary_key=True)
    name = Column(Text, nullable=False, unique=True)
    role = Column(Text, nullable=False)

    password_hash = Column(Text)

    def set_password(self, pw):
        pwhash = bcrypt.hashpw(pw.encode('utf8'), bcrypt.gensalt())
        self.password_hash = pwhash.decode('utf8')

    def check_password(self, pw):
        if self.password_hash is not None:
            expected_hash = self.password_hash.encode('utf8')
            return bcrypt.checkpw(pw.encode('utf8'), expected_hash)
        return False

This is a very basic model for a user who can authenticate with our wiki.

We discussed briefly in the previous chapter that our models will inherit from an SQLAlchemy sqlalchemy.ext.declarative.declarative_base(). This will attach the model to our schema.

As you can see, our User class has a class-level attribute __tablename__ which equals the string users. Our User class will also have class-level attributes named id, name, password_hash, and role (all instances of sqlalchemy.schema.Column). These will map to columns in the users table. The id attribute will be the primary key in the table. The name attribute will be a text column, each value of which needs to be unique within the column. The password_hash is a nullable text attribute that will contain a securely hashed password. Finally, the role text attribute will hold the role of the user.

There are two helper methods that will help us later when using the user objects. The first is set_password which will take a raw password and transform it using bcrypt into an irreversible representation, a process known as "hashing". The second method, check_password, will allow us to compare the hashed value of the submitted password against the hashed value of the password stored in the user's record in the database. If the two hashed values match, then the submitted password is valid, and we can authenticate the user.

We hash passwords so that it is impossible to decrypt them and use them to authenticate in the application. If we stored passwords foolishly in clear text, then anyone with access to the database could retrieve any password to authenticate as any user.

Add page.py

Create a new file tutorial/models/page.py with the following contents:

from sqlalchemy import (
    Column,
    ForeignKey,
    Integer,
    Text,
)
from sqlalchemy.orm import relationship

from .meta import Base


class Page(Base):
    """ The SQLAlchemy declarative model class for a Page object. """
    __tablename__ = 'pages'
    id = Column(Integer, primary_key=True)
    name = Column(Text, nullable=False, unique=True)
    data = Column(Text, nullable=False)

    creator_id = Column(ForeignKey('users.id'), nullable=False)
    creator = relationship('User', backref='created_pages')

As you can see, our Page class is very similar to the User defined above, except with attributes focused on storing information about a wiki page, including id, name, and data. The only new construct introduced here is the creator_id column, which is a foreign key referencing the users table. Foreign keys are very useful at the schema-level, but since we want to relate User objects with Page objects, we also define a creator attribute as an ORM-level mapping between the two tables. SQLAlchemy will automatically populate this value using the foreign key referencing the user. Since the foreign key has nullable=False, we are guaranteed that an instance of page will have a corresponding page.creator, which will be a User instance.

Edit models/__init__.py

Since we are using a package for our models, we also need to update our __init__.py file to ensure that the models are attached to the metadata.

Open the tutorial/models/__init__.py file and edit it to look like the following:

from sqlalchemy import engine_from_config
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm import configure_mappers
import zope.sqlalchemy

# Import or define all models here to ensure they are attached to the
# ``Base.metadata`` prior to any initialization routines.
from .page import Page  # flake8: noqa
from .user import User  # flake8: noqa

# Run ``configure_mappers`` after defining all of the models to ensure
# all relationships can be setup.
configure_mappers()


def get_engine(settings, prefix='sqlalchemy.'):
    return engine_from_config(settings, prefix)


def get_session_factory(engine):
    factory = sessionmaker()
    factory.configure(bind=engine)
    return factory


def get_tm_session(session_factory, transaction_manager, request=None):
    """
    Get a ``sqlalchemy.orm.Session`` instance backed by a transaction.

    This function will hook the session to the transaction manager which
    will take care of committing any changes.

    - When using pyramid_tm it will automatically be committed or aborted
      depending on whether an exception is raised.

    - When using scripts you should wrap the session in a manager yourself.
      For example:

      .. code-block:: python

          import transaction

          engine = get_engine(settings)
          session_factory = get_session_factory(engine)
          with transaction.manager:
              dbsession = get_tm_session(session_factory, transaction.manager)

    This function may be invoked with a ``request`` kwarg, such as when invoked
    by the reified ``.dbsession`` Pyramid request attribute which is configured
    via the ``includeme`` function below. The default value, for backwards
    compatibility, is ``None``.

    The ``request`` kwarg is used to populate the ``sqlalchemy.orm.Session``'s
    "info" dict.  The "info" dict is the official namespace for developers to
    stash session-specific information.  For more information, please see the
    SQLAlchemy docs:
    https://docs.sqlalchemy.org/en/stable/orm/session_api.html#sqlalchemy.orm.session.Session.params.info

    By placing the active ``request`` in the "info" dict, developers will be
    able to access the active Pyramid request from an instance of an SQLAlchemy
    object in one of two ways:

    - Classic SQLAlchemy. This uses the ``Session``'s utility class method:

      .. code-block:: python

          from sqlalchemy.orm.session import Session as sa_Session

          dbsession = sa_Session.object_session(dbObject)
          request = dbsession.info["request"]

    - Modern SQLAlchemy. This uses the "Runtime Inspection API":

      .. code-block:: python

          from sqlalchemy import inspect as sa_inspect

          dbsession = sa_inspect(dbObject).session
          request = dbsession.info["request"]
    """
    dbsession = session_factory(info={"request": request})
    zope.sqlalchemy.register(
        dbsession, transaction_manager=transaction_manager
    )
    return dbsession


def includeme(config):
    """
    Initialize the model for a Pyramid app.

    Activate this setup using ``config.include('tutorial.models')``.

    """
    settings = config.get_settings()
    settings['tm.manager_hook'] = 'pyramid_tm.explicit_manager'

    # Use ``pyramid_tm`` to hook the transaction lifecycle to the request.
    # Note: the packages ``pyramid_tm`` and ``transaction`` work together to
    # automatically close the active database session after every request.
    # If your project migrates away from ``pyramid_tm``, you may need to use a
    # Pyramid callback function to close the database session after each
    # request.
    config.include('pyramid_tm')

    # use pyramid_retry to retry a request when transient exceptions occur
    config.include('pyramid_retry')

    # hook to share the dbengine fixture in testing
    dbengine = settings.get('dbengine')
    if not dbengine:
        dbengine = get_engine(settings)

    session_factory = get_session_factory(dbengine)
    config.registry['dbsession_factory'] = session_factory

    # make request.dbsession available for use in Pyramid
    def dbsession(request):
        # hook to share the dbsession fixture in testing
        dbsession = request.environ.get('app.dbsession')
        if dbsession is None:
            # request.tm is the transaction manager used by pyramid_tm
            dbsession = get_tm_session(
                session_factory, request.tm, request=request
            )
        return dbsession

    config.add_request_method(dbsession, reify=True)

Here we align our imports with the names of the models, Page and User.

Migrate the database with Alembic

Now that we have written our models, we need to modify the database schema to reflect the changes to our code. Let's generate a new revision, then upgrade the database to the latest revision (head).

On Unix:

$VENV/bin/alembic -c development.ini revision --autogenerate \
    -m "use new models Page and User"
$VENV/bin/alembic -c development.ini upgrade head

On Windows:

%VENV%\Scripts\alembic -c development.ini revision \
    --autogenerate -m "use new models Page and User"
%VENV%\Scripts\alembic -c development.ini upgrade head

Success executing these commands will generate output similar to the following.

2021-01-07 08:00:14,550 INFO  [alembic.runtime.migration:155][MainThread] Context impl SQLiteImpl.
2021-01-07 08:00:14,551 INFO  [alembic.runtime.migration:158][MainThread] Will assume non-transactional DDL.
2021-01-07 08:00:14,553 INFO  [alembic.autogenerate.compare:134][MainThread] Detected added table 'users'
2021-01-07 08:00:14,553 INFO  [alembic.autogenerate.compare:134][MainThread] Detected added table 'pages'
2021-01-07 08:00:14,558 INFO  [alembic.autogenerate.compare:622][MainThread] Detected removed index 'my_index' on 'models'
2021-01-07 08:00:14,558 INFO  [alembic.autogenerate.compare:176][MainThread] Detected removed table 'models'
  Generating <somepath>/tutorial/tutorial/alembic/versions/20210107_bc9a3dead43a.py ...  done

2021-01-07 08:00:21,318 INFO  [alembic.runtime.migration:155][MainThread] Context impl SQLiteImpl.
2021-01-07 08:00:21,318 INFO  [alembic.runtime.migration:158][MainThread] Will assume non-transactional DDL.
2021-01-07 08:00:21,320 INFO  [alembic.runtime.migration:517][MainThread] Running upgrade 90658c4a9673 -> bc9a3dead43a, use new models Page and User

Alembic overview

Let's briefly discuss our configuration for Alembic.

In the alchemy cookiecutter's development.ini file, the setting for script_location configures Alembic to look for the migration script in the directory tutorial/alembic. By default Alembic stores the migration files one level deeper in tutorial/alembic/versions. These files are generated by Alembic, then executed when we run upgrade or downgrade migrations. The setting file_template provides the format for each migration's file name. We've configured the file_template setting to make it somewhat easy to find migrations by file name.

At this point in this tutorial, we have two migration files. Examine them to see what Alembic will do when you upgrade or downgrade the database to a specific revision. Notice the revision identifiers and how they relate to one another in a chained sequence.

See also

For further information, see the Alembic documentation.

Edit scripts/initialize_db.py

We haven't looked at the details of this file yet, but within the scripts directory of your tutorial package is a file named initialize_db.py. Code in this file is executed whenever we run the initialize_tutorial_db command, as we did in the installation step of this tutorial.

Note

The command is named initialize_tutorial_db because of the mapping defined in the [console_scripts] entry point of our project's setup.py file.

Since we've changed our model, we need to make changes to our initialize_db.py script. In particular, we'll replace our import of MyModel with those of User and Page. We'll also change the the script to create two User objects (basic and editor) as well as a Page, rather than a MyModel, and add them to our dbsession.

Open tutorial/scripts/initialize_db.py and edit it to look like the following:

import argparse
import sys

from pyramid.paster import bootstrap, setup_logging
from sqlalchemy.exc import OperationalError

from .. import models


def setup_models(dbsession):
    """
    Add or update models / fixtures in the database.

    """
    editor = models.User(name='editor', role='editor')
    editor.set_password('editor')
    dbsession.add(editor)

    basic = models.User(name='basic', role='basic')
    basic.set_password('basic')
    dbsession.add(basic)

    page = models.Page(
        name='FrontPage',
        creator=editor,
        data='This is the front page',
    )
    dbsession.add(page)


def parse_args(argv):
    parser = argparse.ArgumentParser()
    parser.add_argument(
        'config_uri',
        help='Configuration file, e.g., development.ini',
    )
    return parser.parse_args(argv[1:])


def main(argv=sys.argv):
    args = parse_args(argv)
    setup_logging(args.config_uri)
    env = bootstrap(args.config_uri)

    try:
        with env['request'].tm:
            dbsession = env['request'].dbsession
            setup_models(dbsession)
    except OperationalError:
        print('''
Pyramid is having a problem using your SQL database.  The problem
might be caused by one of the following things:

1.  You may need to initialize your database tables with `alembic`.
    Check your README.txt for description and try to run it.

2.  Your database server may not be running.  Check that the
    database server referred to by the "sqlalchemy.url" setting in
    your "development.ini" file is running.
            ''')

Only the highlighted lines need to be changed.

Populating the database

Because our model has changed, and to repopulate the database, we need to rerun the initialize_tutorial_db command to pick up the changes we've made to the initialize_db.py file.

On Unix

$VENV/bin/initialize_tutorial_db development.ini

On Windows

%VENV%\Scripts\initialize_tutorial_db development.ini

There should be no output to your console to indicate success.

View the application in a browser

We can't. At this point, our system is in a "non-runnable" state; we'll need to change view-related files in the next chapter to be able to start the application successfully. If you try to start the application (see Start the application) and visit http://localhost:6543, you'll wind up with a Python traceback on your console that ends with this exception:

AttributeError: module 'tutorial.models' has no attribute 'MyModel'

This will also happen if you attempt to run the tests.