18: Forms and Validation with Deform

Schema-driven, autogenerated forms with validation.

Background

Modern web applications deal extensively with forms. Developers, though, have a wide range of philosophies about how frameworks should help them with their forms. As such, Pyramid doesn't directly bundle one particular form library. Instead there are a variety of form libraries that are easy to use in Pyramid.

Deform is one such library. In this step, we introduce Deform for our forms. This also gives us Colander for schemas and validation.

Objectives

  • Make a schema using Colander, the companion to Deform.

  • Create a form with Deform and change our views to handle validation.

Steps

  1. First we copy the results of the view_classes step:

    cd ..; cp -r view_classes forms; cd forms

  2. Let's edit forms/setup.py to declare a dependency on Deform, which in turn pulls in Colander as a dependency:

     1from setuptools import setup
 2
 3# List of dependencies installed via `pip install -e .`
 4# by virtue of the Setuptools `install_requires` value below.
 5requires = [
 6    'deform',
 7    'pyramid',
 8    'pyramid_chameleon',
 9    'waitress',
10]
11
12# List of dependencies installed via `pip install -e ".[dev]"`
13# by virtue of the Setuptools `extras_require` value in the Python
14# dictionary below.
15dev_requires = [
16    'pyramid_debugtoolbar',
17    'pytest',
18    'webtest',
19]
20
21setup(
22    name='tutorial',
23    install_requires=requires,
24    extras_require={
25        'dev': dev_requires,
26    },
27    entry_points={
28        'paste.app_factory': [
29            'main = tutorial:main'
30        ],
31    },
32)

  3. We can now install our project in development mode:

    $VENV/bin/pip install -e .

  4. Register a static view in forms/tutorial/__init__.py for Deform's CSS, JavaScript, etc., as well as our demo wiki page's views:

     1from pyramid.config import Configurator
 2
 3
 4def main(global_config, **settings):
 5    config = Configurator(settings=settings)
 6    config.include('pyramid_chameleon')
 7    config.add_route('wiki_view', '/')
 8    config.add_route('wikipage_add', '/add')
 9    config.add_route('wikipage_view', '/{uid}')
10    config.add_route('wikipage_edit', '/{uid}/edit')
11    config.add_static_view('deform_static', 'deform:static/')
12    config.scan('.views')
13    return config.make_wsgi_app()

  5. Implement the new views, as well as the form schemas and some dummy data, in forms/tutorial/views.py:

     1import colander
 2import deform.widget
 3
 4from pyramid.httpexceptions import HTTPFound
 5from pyramid.view import view_config
 6
 7pages = {
 8    '100': dict(uid='100', title='Page 100', body='<em>100</em>'),
 9    '101': dict(uid='101', title='Page 101', body='<em>101</em>'),
10    '102': dict(uid='102', title='Page 102', body='<em>102</em>')
11}
12
13class WikiPage(colander.MappingSchema):
14    title = colander.SchemaNode(colander.String())
15    body = colander.SchemaNode(
16        colander.String(),
17        widget=deform.widget.RichTextWidget()
18    )
19
20
21class WikiViews:
22    def __init__(self, request):
23        self.request = request
24
25    @property
26    def wiki_form(self):
27        schema = WikiPage()
28        return deform.Form(schema, buttons=('submit',))
29
30    @property
31    def reqts(self):
32        return self.wiki_form.get_widget_resources()
33
34    @view_config(route_name='wiki_view', renderer='wiki_view.pt')
35    def wiki_view(self):
36        return dict(pages=pages.values())
37
38    @view_config(route_name='wikipage_add',
39                 renderer='wikipage_addedit.pt')
40    def wikipage_add(self):
41        form = self.wiki_form.render()
42
43        if 'submit' in self.request.params:
44            controls = self.request.POST.items()
45            try:
46                appstruct = self.wiki_form.validate(controls)
47            except deform.ValidationFailure as e:
48                # Form is NOT valid
49                return dict(form=e.render())
50
51            # Form is valid, make a new identifier and add to list
52            last_uid = int(sorted(pages.keys())[-1])
53            new_uid = str(last_uid + 1)
54            pages[new_uid] = dict(
55                uid=new_uid, title=appstruct['title'],
56                body=appstruct['body']
57            )
58
59            # Now visit new page
60            url = self.request.route_url('wikipage_view', uid=new_uid)
61            return HTTPFound(url)
62
63        return dict(form=form)
64
65    @view_config(route_name='wikipage_view', renderer='wikipage_view.pt')
66    def wikipage_view(self):
67        uid = self.request.matchdict['uid']
68        page = pages[uid]
69        return dict(page=page)
70
71    @view_config(route_name='wikipage_edit',
72                 renderer='wikipage_addedit.pt')
73    def wikipage_edit(self):
74        uid = self.request.matchdict['uid']
75        page = pages[uid]
76
77        wiki_form = self.wiki_form
78
79        if 'submit' in self.request.params:
80            controls = self.request.POST.items()
81            try:
82                appstruct = wiki_form.validate(controls)
83            except deform.ValidationFailure as e:
84                return dict(page=page, form=e.render())
85
86            # Change the content and redirect to the view
87            page['title'] = appstruct['title']
88            page['body'] = appstruct['body']
89
90            url = self.request.route_url('wikipage_view',
91                                         uid=page['uid'])
92            return HTTPFound(url)
93
94        form = wiki_form.render(page)
95
96        return dict(page=page, form=form)

  6. A template for the top of the "wiki" in forms/tutorial/wiki_view.pt:

     1<!DOCTYPE html>
 2<html lang="en">
 3<head>
 4    <title>Wiki: View</title>
 5</head>
 6<body>
 7<h1>Wiki</h1>
 8
 9<a href="${request.route_url('wikipage_add')}">Add
10    WikiPage</a>
11<ul>
12    <li tal:repeat="page pages">
13        <a href="${request.route_url('wikipage_view', uid=page.uid)}">
14                ${page.title}
15        </a>
16    </li>
17</ul>
18</body>
19</html>

  7. Another template for adding/editing in forms/tutorial/wikipage_addedit.pt:

     1<!DOCTYPE html>
 2<html lang="en">
 3<head>
 4    <title>WikiPage: Add/Edit</title>
 5    <link rel="stylesheet"
 6          href="${request.static_url('deform:static/css/bootstrap.min.css')}"
 7          type="text/css" media="screen" charset="utf-8"/>
 8    <link rel="stylesheet"
 9          href="${request.static_url('deform:static/css/form.css')}"
10          type="text/css"/>
11    <tal:block tal:repeat="reqt view.reqts['css']">
12        <link rel="stylesheet" type="text/css"
13              href="${request.static_url(reqt)}"/>
14    </tal:block>
15    <script src="${request.static_url('deform:static/scripts/jquery-2.0.3.min.js')}"
16            type="text/javascript"></script>
17    <script src="${request.static_url('deform:static/scripts/bootstrap.min.js')}"
18            type="text/javascript"></script>
19
20    <tal:block tal:repeat="reqt view.reqts['js']">
21        <script src="${request.static_url(reqt)}"
22                type="text/javascript"></script>
23    </tal:block>
24</head>
25<body>
26<h1>Wiki</h1>
27
28<p>${structure: form}</p>
29<script type="text/javascript">
30    deform.load()
31</script>
32</body>
33</html>

  8. Add a template at forms/tutorial/wikipage_view.pt for viewing a wiki page:

     1<!DOCTYPE html>
 2<html lang="en">
 3<head>
 4    <title>WikiPage: View</title>
 5</head>
 6<body>
 7<a href="${request.route_url('wiki_view')}">
 8    Up
 9</a> |
10<a href="${request.route_url('wikipage_edit', uid=page.uid)}">
11    Edit
12</a>
13
14<h1>${page.title}</h1>
15<p>${structure: page.body}</p>
16</body>
17</html>

  9. Our tests in forms/tutorial/tests.py don't run, so let's modify them:

     1import unittest
 2
 3from pyramid import testing
 4
 5
 6class TutorialViewTests(unittest.TestCase):
 7    def setUp(self):
 8        self.config = testing.setUp()
 9
10    def tearDown(self):
11        testing.tearDown()
12
13    def test_home(self):
14        from .views import WikiViews
15
16        request = testing.DummyRequest()
17        inst = WikiViews(request)
18        response = inst.wiki_view()
19        self.assertEqual(len(response['pages']), 3)
20
21
22class TutorialFunctionalTests(unittest.TestCase):
23    def setUp(self):
24        from tutorial import main
25
26        app = main({})
27        from webtest import TestApp
28
29        self.testapp = TestApp(app)
30
31    def tearDown(self):
32        testing.tearDown()
33
34    def test_home(self):
35        res = self.testapp.get('/', status=200)
36        self.assertIn(b'<title>Wiki: View</title>', res.body)
37
38    def test_add_page(self):
39        res = self.testapp.get('/add', status=200)
40        self.assertIn(b'<h1>Wiki</h1>', res.body)
41
42    def test_edit_page(self):
43        res = self.testapp.get('/101/edit', status=200)
44        self.assertIn(b'<h1>Wiki</h1>', res.body)
45
46    def test_post_wiki(self):
47        self.testapp.post('/add', {
48            "title": "New Title",
49            "body": "<p>New Body</p>",
50            "submit": "submit"
51        }, status=302)
52
53        res = self.testapp.get('/103', status=200)
54        self.assertIn(b'<h1>New Title</h1>', res.body)
55        self.assertIn(b'<p>New Body</p>', res.body)
56
57    def test_edit_wiki(self):
58        self.testapp.post('/102/edit', {
59            "title": "New Title",
60            "body": "<p>New Body</p>",
61            "submit": "submit"
62        }, status=302)
63
64        res = self.testapp.get('/102', status=200)
65        self.assertIn(b'<h1>New Title</h1>', res.body)
66        self.assertIn(b'<p>New Body</p>', res.body)

  10. Run the tests:

    $VENV/bin/pytest tutorial/tests.py -q
..
6 passed in 0.81 seconds

  11. Run your Pyramid application with:

    $VENV/bin/pserve development.ini --reload

  12. Open http://localhost:6543/ in a browser.

Analysis

This step helps illustrate the utility of asset specifications for static assets. We have an outside package called Deform with static assets which need to be published. We don't have to know where on disk it is located. We point at the package, then the path inside the package.

We just need to include a call to add_static_view to make that directory available at a URL. For Pyramid-specific packages, Pyramid provides a facility (config.include()) which even makes that unnecessary for consumers of a package. (Deform is not specific to Pyramid.)

Our forms have rich widgets which need the static CSS and JavaScript just mentioned. Deform has a resource registry which allows widgets to specify which JavaScript and CSS are needed. Our wikipage_addedit.pt template shows how we iterated over that data to generate markup that includes the needed resources.

Our add and edit views use a pattern called self-posting forms. Meaning, the same URL is used to GET the form as is used to POST the form. The route, the view, and the template are the same URL whether you are walking up to it for the first time or you clicked a button.

Inside the view we do if 'submit' in self.request.params: to see if this form was a POST where the user clicked on a particular button <input name="submit">.

The form controller then follows a typical pattern:

  • If you are doing a GET, skip over and just return the form.

  • If you are doing a POST, validate the form contents.

  • If the form is invalid, bail out by re-rendering the form with the supplied POST data.

  • If the validation succeeded, perform some action and issue a redirect via HTTPFound.

We are, in essence, writing our own form controller. Other Pyramid-based systems, including pyramid_deform, provide a form-centric view class which automates much of this branching and routing.

Extra credit

  1. Give a try at a button that goes to a delete view for a particular wiki page.