Command-Line Pyramid¶

Your Pyramid application can be controlled and inspected using a variety of command-line utilities. These utilities are documented in this chapter.

We commonly refer to this collection of utilities as "p-scripts", which is short for "Pyramid console scripts".

Each p-script's command line usage details is available in the p* Scripts Documentation.

Running p-scripts¶

All of the Pyramid console scripts may be run either:

by its name
as a Python module

Running by `p*` script name¶

Each of Pyramid's console scripts may be run by its name. For example:

$VENV/bin/pserve development.ini --reload

Note

$VENV/bin/ is a convention we use to simplify Pyramid documentation. It represents the bin directory in a virtual environment, where $VENV is an environment variable representing its path. See Installing Pyramid on a Unix System and Why use $VENV/bin/pip instead of source bin/activate, then pip for more information.

Using Custom Arguments to Python when Running `p*` Scripts¶

New in version 1.5.

Each of Pyramid's console scripts (pserve, pviews, etc.) can be run using python3 -m, allowing custom arguments to be sent to the Python interpreter at runtime. For example:

python3 -m pyramid.scripts.pserve development.ini --reload

`pviews`: Displaying Matching Views for a Given URL¶

See also

`pshell`: The Interactive Shell¶

See also

Extending the Shell¶

It is convenient when using the interactive shell often to have some variables significant to your application already loaded as globals when you start the pshell. To facilitate this, pshell will look for a special [pshell] section in your .ini file and expose the subsequent key/value pairs to the shell. Each key is a variable name that will be global within the pshell session; each value is a dotted Python name. If specified, the special key setup should be a dotted Python name pointing to a callable that accepts the dictionary of globals that will be loaded into the shell. This allows for some custom initializing code to be executed each time the pshell is run. The setup callable can also be specified from the commandline using the --setup option which will override the key in the .ini file.

For example, you want to expose your model to the shell along with the database session so that you can mutate the model on an actual database. Here, we'll assume your model is stored in the myapp.models package and that you're using pyramid_tm to configure a transaction manager on the request as request.tm.

[pshell]
setup = myapp.lib.pshell.setup
models = myapp.models

By defining the setup callable, we will create the module myapp.lib.pshell containing a callable named setup that will receive the global environment before it is exposed to the shell. Here we mutate the environment's request as well as add a new value containing a WebTest version of the application to which we can easily submit requests. The setup callable can also be a generator which can wrap the entire shell lifecycle, executing code when the shell exits.

# myapp/lib/pshell.py
from contextlib import suppress
from transaction.interfaces import NoTransaction
from webtest import TestApp

def setup(env):
    request = env['request']
    request.host = 'www.example.com'
    request.scheme = 'https'

    env['testapp'] = TestApp(env['app'])

    # start a transaction which can be used in the shell
    request.tm.begin()

    # if using the SQLAlchemy backend from our cookiecutter, the dbsession is
    # connected to the transaction manager above
    env['tm'] = request.tm
    env['dbsession'] = request.dbsession
    try:
        yield

    finally:
        with suppress(NoTransaction):
            request.tm.abort()

When this .ini file is loaded, the extra variable models will be available for use immediately. Since a setup callable was also specified, it is executed and new variables testapp, tm, and dbsession are exposed, and the request is configured to generate URLs from the host http://www.example.com. For example:

$VENV/bin/pshell starter/development.ini
Python 2.6.5 (r265:79063, Apr 29 2010, 00:31:32)
[GCC 4.4.3] on linux2
Type "help" for more information.

Environment:
  app          The WSGI application.
  registry     Active Pyramid registry.
  request      Active request object.
  root         Root of the default resource tree.
  root_factory Default root factory used to create `root`.
  testapp      <webtest.TestApp object at ...>

Custom Variables:
  dbsession
  model        myapp.models
  tm

>>> testapp.get('/')
<200 OK text/html body='<!DOCTYPE...l>\n'/3337>
>>> request.route_url('home')
'https://www.example.com/'
>>> user = dbsession.query(models.User).get(1)
>>> user.name = 'Joe'
>>> tm.commit()
>>> tm.begin()
>>> user = dbsession.query(models.User).get(1)
>>> user.name == 'Joe'
'Joe'

Alternative Shells¶

The pshell command can be easily extended with alternate REPLs if the default python REPL is not satisfactory. Assuming you have a binding installed such as pyramid_ipython it will normally be auto-selected and used. You may also specifically invoke your choice with the -p choice or --python-shell choice option.

$VENV/bin/pshell -p ipython development.ini#MyProject

You may use the --list-shells option to see the available shells.

$VENV/bin/pshell --list-shells
Available shells:
  bpython
  ipython
  python

If you want to use a shell that isn't supported out of the box, you can introduce a new shell by registering an entry point in your setup.py:

setup(
    entry_points={
        'pyramid.pshell_runner': [
            'myshell=my_app:ptpython_shell_factory',
        ],
    },
)

And then your shell factory should return a function that accepts two arguments, env and help, which would look like this:

from ptpython.repl import embed

def ptpython_shell_runner(env, help):
    print(help)
    return embed(locals=env)

Changed in version 1.6: User-defined shells may be registered using entry points. Prior to this the only supported shells were ipython, bpython and python.

ipython and bpython have been moved into their respective packages pyramid_ipython and pyramid_bpython.

Setting a Default Shell¶

You may use the default_shell option in your [pshell] ini section to specify a list of preferred shells.

[pshell]
default_shell = ptpython ipython bpython

New in version 1.6.

`proutes`: Displaying All Application Routes¶

See also

`ptweens`: Displaying "Tweens"¶

See also

`prequest`: Invoking a Request¶

See also

`pdistreport`: Showing All Installed Distributions and Their Versions¶

New in version 1.5.

See also

See also the output of pdistreport --help.

You can use the pdistreport command to show the Pyramid version in use, the Python version in use, and all installed versions of Python distributions in your Python environment:

$VENV/bin/pdistreport
Pyramid version: 1.5dev
Platform Linux-3.2.0-51-generic-x86_64-with-debian-wheezy-sid
Packages:
  authapp 0.0
    /home/chrism/projects/foo/src/authapp
  beautifulsoup4 4.1.3
    /home/chrism/projects/foo/lib/python2.7/site-packages/beautifulsoup4-4.1.3-py2.7.egg
# ... more output ...

pdistreport takes no options. Its output is useful to paste into a pastebin when you are having problems and need someone with more familiarity with Python packaging and distribution than you have to look at your environment.

Writing a Script¶

All web applications are, at their hearts, systems which accept a request and return a response. When a request is accepted by a Pyramid application, the system receives state from the request which is later relied on by your application code. For example, one view callable may assume it's working against a request that has a request.matchdict of a particular composition, while another assumes a different composition of the matchdict.

In the meantime, it's convenient to be able to write a Python script that can work "in a Pyramid environment", for instance to update database tables used by your Pyramid application. But a "real" Pyramid environment doesn't have a completely static state independent of a request; your application (and Pyramid itself) is almost always reliant on being able to obtain information from a request. When you run a Python script that simply imports code from your application and tries to run it, there just is no request data, because there isn't any real web request. Therefore some parts of your application and some Pyramid APIs will not work.

For this reason, Pyramid makes it possible to run a script in an environment much like the environment produced when a particular request reaches your Pyramid application. This is achieved by using the pyramid.paster.bootstrap() command in the body of your script.

New in version 1.1: pyramid.paster.bootstrap()

Changed in version 1.8: Added the ability for bootstrap to cleanup automatically via the with statement.

In the simplest case, pyramid.paster.bootstrap() can be used with a single argument, which accepts the PasteDeploy .ini file representing your Pyramid application's configuration as a single argument:

from pyramid.paster import bootstrap

with bootstrap('/path/to/my/development.ini') as env:
    print(env['request'].route_url('home'))

pyramid.paster.bootstrap() returns a dictionary containing framework-related information. This dictionary will always contain a request object as its request key.

The following keys are available in the env dictionary returned by pyramid.paster.bootstrap():

request

A pyramid.request.Request object implying the current request state for your script.

app

The WSGI application object generated by bootstrapping.

root

The resource root of your Pyramid application. This is an object generated by the root factory configured in your application.

registry

The application registry of your Pyramid application.

closer

A parameterless callable that can be used to pop an internal Pyramid threadlocal stack (used by pyramid.threadlocal.get_current_registry() and pyramid.threadlocal.get_current_request()) when your scripting job is finished.

Let's assume that the /path/to/my/development.ini file used in the example above looks like so:

[pipeline:main]
pipeline = translogger
           another

[filter:translogger]
filter_app_factory = egg:Paste#translogger
setup_console_handler = False
logger_name = wsgi

[app:another]
use = egg:MyProject

The configuration loaded by the above bootstrap example will use the configuration implied by the [pipeline:main] section of your configuration file by default. Specifying /path/to/my/development.ini is logically equivalent to specifying /path/to/my/development.ini#main. In this case, we'll be using a configuration that includes an app object which is wrapped in the Paste "translogger" middleware (which logs requests to the console).

You can also specify a particular section of the PasteDeploy .ini file to load instead of main:

from pyramid.paster import bootstrap

with bootstrap('/path/to/my/development.ini#another') as env:
    print(env['request'].route_url('home'))

The above example specifies the another app, pipeline, or composite section of your PasteDeploy configuration file. The app object present in the env dictionary returned by pyramid.paster.bootstrap() will be a Pyramid router.

Changing the Request¶

By default, Pyramid will generate a request object in the env dictionary for the URL http://localhost:80/. This means that any URLs generated by Pyramid during the execution of your script will be anchored here. This is generally not what you want.

So how do we make Pyramid generate the correct URLs?

Assuming that you have a route configured in your application like so:

config.add_route('verify', '/verify/{code}')

You need to inform the Pyramid environment that the WSGI application is handling requests from a certain base. For example, we want to simulate mounting our application at https://example.com/prefix, to ensure that the generated URLs are correct for our deployment. This can be done by either mutating the resulting request object, or more simply by constructing the desired request and passing it into bootstrap():

from pyramid.paster import bootstrap
from pyramid.request import Request

request = Request.blank('/', base_url='https://example.com/prefix')
with bootstrap('/path/to/my/development.ini#another', request=request) as env:
    print(env['request'].application_url)
    # will print 'https://example.com/prefix'

Now you can readily use Pyramid's APIs for generating URLs:

env['request'].route_url('verify', code='1337')
# will return 'https://example.com/prefix/verify/1337'

Cleanup¶

If you're using the with-statement variant then there's nothing to worry about. However if you're using the returned environment directly then when your scripting logic finishes, it's good manners to call the closer callback:

from pyramid.paster import bootstrap
env = bootstrap('/path/to/my/development.ini')

# .. do stuff ...

env['closer']()

Setting Up Logging¶

By default, pyramid.paster.bootstrap() does not configure logging parameters present in the configuration file. If you'd like to configure logging based on [logger] and related sections in the configuration file, use the following command:

import pyramid.paster
pyramid.paster.setup_logging('/path/to/my/development.ini')

See Logging for more information on logging within Pyramid.

Making Your Script into a Console Script¶

A "console script" is Setuptools terminology for a script that gets installed into the bin directory of a Python virtual environment (or "base" Python environment) when a distribution which houses that script is installed. Because it's installed into the bin directory of a virtual environment when the distribution is installed, it's a convenient way to package and distribute functionality that you can call from the command-line. It's often more convenient to create a console script than it is to create a .py script and instruct people to call it with the "right" Python interpreter. A console script generates a file that lives in bin, and when it's invoked it will always use the "right" Python environment, which means it will always be invoked in an environment where all the libraries it needs (such as Pyramid) are available.

In general, you can make your script into a console script by doing the following:

Use an existing distribution (such as one you've already created via cookiecutter) or create a new distribution that possesses at least one package or module. It should, within any module within the distribution, house a callable (usually a function) that takes no arguments and which runs any of the code you wish to run.
Add a [console_scripts] section to the entry_points argument of the distribution which creates a mapping between a script name and a dotted name representing the callable you added to your distribution.
Run pip install -e . or pip install . to get your distribution reinstalled. When you reinstall your distribution, a file representing the script that you named in the last step will be in the bin directory of the virtual environment in which you installed the distribution. It will be executable. Invoking it from a terminal will execute your callable.

As an example, let's create some code that can be invoked by a console script that prints the deployment settings of a Pyramid application. To do so, we'll pretend you have a distribution with a package in it named myproject. Within this package, we'll pretend you've added a scripts.py module which contains the following code:

# myproject.scripts module

import optparse
import sys
import textwrap

from pyramid.paster import bootstrap

def settings_show():
    description = """\
    Print the deployment settings for a Pyramid application.  Example:
    'show_settings deployment.ini'
    """
    usage = "usage: %prog config_uri"
    parser = optparse.OptionParser(
        usage=usage,
        description=textwrap.dedent(description)
        )
    parser.add_option(
        '-o', '--omit',
        dest='omit',
        metavar='PREFIX',
        type='string',
        action='append',
        help=("Omit settings which start with PREFIX (you can use this "
              "option multiple times)")
        )

    options, args = parser.parse_args(sys.argv[1:])
    if not len(args) >= 1:
        print('You must provide at least one argument')
        return 2
    config_uri = args[0]
    omit = options.omit
    if omit is None:
        omit = []
    with bootstrap(config_uri) as env:
        settings = env['registry'].settings
        for k, v in settings.items():
            if any([k.startswith(x) for x in omit]):
                continue
            print('%-40s     %-20s' % (k, v))

This script uses the Python optparse module to allow us to make sense out of extra arguments passed to the script. It uses the pyramid.paster.bootstrap() function to get information about the application defined by a config file, and prints the deployment settings defined in that config file.

After adding this script to the package, you'll need to tell your distribution's setup.py about its existence. Within your distribution's top-level directory, your setup.py file will look something like this:

import os

from setuptools import setup, find_packages

here = os.path.abspath(os.path.dirname(__file__))
with open(os.path.join(here, 'README.txt')) as f:
    README = f.read()
with open(os.path.join(here, 'CHANGES.txt')) as f:
    CHANGES = f.read()

requires = ['pyramid', 'pyramid_debugtoolbar']

tests_require = [
    'WebTest',
    'pytest',
    'pytest-cov',
]

setup(name='MyProject',
    version='0.0',
    description='My project',
    long_description=README + '\n\n' +  CHANGES,
    classifiers=[
        "Programming Language :: Python",
        "Framework :: Pyramid",
        "Topic :: Internet :: WWW/HTTP",
        "Topic :: Internet :: WWW/HTTP :: WSGI :: Application",
    ],
    author='',
    author_email='',
    url='',
    keywords='web pyramid pylons',
    packages=find_packages(exclude=['tests']),
    include_package_data=True,
    zip_safe=False,
    install_requires=requires,
    extras_require={
        'testing': tests_require,
    },
    entry_points = """\
    [paste.app_factory]
    main = myproject:main
    """,
    )

We're going to change the setup.py file to add a [console_scripts] section within the entry_points string. Within this section, you should specify a scriptname = dotted.path.to:yourfunction line. For example:

[console_scripts]
show_settings = myproject.scripts:settings_show

The show_settings name will be the name of the script that is installed into bin. The colon (:) between myproject.scripts and settings_show above indicates that myproject.scripts is a Python module, and settings_show is the function in that module which contains the code you'd like to run as the result of someone invoking the show_settings script from their command line.

The result will be something like:

import os

from setuptools import setup, find_packages

here = os.path.abspath(os.path.dirname(__file__))
with open(os.path.join(here, 'README.txt')) as f:
    README = f.read()
with open(os.path.join(here, 'CHANGES.txt')) as f:
    CHANGES = f.read()

requires = ['pyramid', 'pyramid_debugtoolbar']

tests_require = [
    'WebTest',
    'pytest',
    'pytest-cov',
]

setup(name='MyProject',
    version='0.0',
    description='My project',
    long_description=README + '\n\n' +  CHANGES,
    classifiers=[
        "Programming Language :: Python",
        "Framework :: Pyramid",
        "Topic :: Internet :: WWW/HTTP",
        "Topic :: Internet :: WWW/HTTP :: WSGI :: Application",
    ],
    author='',
    author_email='',
    url='',
    keywords='web pyramid pylons',
    packages=find_packages(),
    include_package_data=True,
    zip_safe=False,
    install_requires=requires,
    extras_require={
        'testing': tests_require,
    },
    entry_points = """\
    [paste.app_factory]
    main = myproject:main
    [console_scripts]
    show_settings = myproject.scripts:settings_show
    """,
)

Once you've done this, invoking $VENV/bin/pip install -e . will install a file named show_settings into the $somevenv/bin directory with a small bit of Python code that points to your entry point. It will be executable. Running it without any arguments will print an error and exit. Running it with a single argument that is the path of a config file will print the settings. Running it with an --omit=foo argument will omit the settings that have keys that start with foo. Running it with two "omit" options (e.g., --omit=foo --omit=bar) will omit all settings that have keys that start with either foo or bar:

$VENV/bin/show_settings development.ini --omit=pyramid --omit=debugtoolbar
debug_routematch                             False
debug_templates                              True
reload_templates                             True
mako.directories                             []
debug_notfound                               False
default_locale_name                          en
reload_resources                             False
debug_authorization                          False
reload_assets                                False
prevent_http_cache                           False

Pyramid's pserve, pshell, prequest, ptweens, and other p* scripts are implemented as console scripts. When you invoke one of those, you are using a console script.

Command-Line Pyramid¶

Running p-scripts¶

Running by `p*` script name¶

Using Custom Arguments to Python when Running `p*` Scripts¶

`pviews`: Displaying Matching Views for a Given URL¶

`pshell`: The Interactive Shell¶

Extending the Shell¶

Alternative Shells¶

Setting a Default Shell¶

`proutes`: Displaying All Application Routes¶

`ptweens`: Displaying "Tweens"¶

`prequest`: Invoking a Request¶

`pdistreport`: Showing All Installed Distributions and Their Versions¶

Writing a Script¶

Changing the Request¶

Cleanup¶

Setting Up Logging¶

Making Your Script into a Console Script¶

Table of Contents

Previous topic

Next topic

This Page

Command-Line Pyramid¶

Running p-scripts¶

Running by p* script name¶

Using Custom Arguments to Python when Running p* Scripts¶

pviews: Displaying Matching Views for a Given URL¶

pshell: The Interactive Shell¶

Extending the Shell¶

Alternative Shells¶

Setting a Default Shell¶

proutes: Displaying All Application Routes¶

ptweens: Displaying "Tweens"¶

prequest: Invoking a Request¶

pdistreport: Showing All Installed Distributions and Their Versions¶

Writing a Script¶

Changing the Request¶

Cleanup¶

Setting Up Logging¶

Making Your Script into a Console Script¶

Running by `p*` script name¶

Using Custom Arguments to Python when Running `p*` Scripts¶

`pviews`: Displaying Matching Views for a Given URL¶

`pshell`: The Interactive Shell¶

`proutes`: Displaying All Application Routes¶

`ptweens`: Displaying "Tweens"¶

`prequest`: Invoking a Request¶

`pdistreport`: Showing All Installed Distributions and Their Versions¶