Wiki Example¶
- author:
Ian Bicking <ianb@colorstudy.com>
Introduction¶
This is an example of how to write a WSGI application using WebOb. WebOb isn't itself intended to write applications -- it is not a web framework on its own -- but it is possible to write applications using just WebOb.
The file serving example is a better example of advanced HTTP usage. The comment middleware example is a better example of using middleware. This example provides some completeness by showing an application-focused end point.
This example implements a very simple wiki.
Code¶
The finished code for this is available in docs/wiki-example-code/example.py -- you can run that file as a script to try it out.
Creating an Application¶
A common pattern for creating small WSGI applications is to have a class which is instantiated with the configuration. For our application we'll be storing the pages under a directory.
class WikiApp(object):
def __init__(self, storage_dir):
self.storage_dir = os.path.abspath(os.path.normpath(storage_dir))
WSGI applications are callables like wsgi_app(environ,
start_response). Instances of WikiApp are WSGI applications, so
we'll implement a __call__ method:
class WikiApp(object):
...
def __call__(self, environ, start_response):
# what we'll fill in
To make the script runnable we'll create a simple command-line interface:
if __name__ == '__main__':
import optparse
parser = optparse.OptionParser(
usage='%prog --port=PORT'
)
parser.add_option(
'-p', '--port',
default='8080',
dest='port',
type='int',
help='Port to serve on (default 8080)')
parser.add_option(
'--wiki-data',
default='./wiki',
dest='wiki_data',
help='Place to put wiki data into (default ./wiki/)')
options, args = parser.parse_args()
print 'Writing wiki pages to %s' % options.wiki_data
app = WikiApp(options.wiki_data)
from wsgiref.simple_server import make_server
httpd = make_server('localhost', options.port, app)
print 'Serving on http://localhost:%s' % options.port
try:
httpd.serve_forever()
except KeyboardInterrupt:
print '^C'
There's not much to talk about in this code block. The application is
instantiated and served with the built-in module
wsgiref.simple_server.
The WSGI Application¶
Of course all the interesting stuff is in that __call__ method.
WebOb lets you ignore some of the details of WSGI, like what
start_response really is. environ is a CGI-like dictionary,
but webob.Request gives an object interface to it.
webob.Response represents a response, and is itself a WSGI
application. Here's kind of the hello world of WSGI applications
using these objects:
from webob import Request, Response
class WikiApp(object):
...
def __call__(self, environ, start_response):
req = Request(environ)
resp = Response(
'Hello %s!' % req.params.get('name', 'World'))
return resp(environ, start_response)
req.params.get('name', 'World') gets any query string parameter
(like ?name=Bob), or if it's a POST form request it will look for
a form parameter name. We instantiate the response with the body
of the response. You could also give keyword arguments like
content_type='text/plain' (text/html is the default content
type and 200 OK is the default status).
For the wiki application we'll support a couple different kinds of
screens, and we'll make our __call__ method dispatch to different
methods depending on the request. We'll support an action
parameter like ?action=edit, and also dispatch on the method (GET,
POST, etc, in req.method). We'll pass in the request and expect a
response object back.
Also, WebOb has a series of exceptions in webob.exc, like
webob.exc.HTTPNotFound, webob.exc.HTTPTemporaryRedirect, etc.
We'll also let the method raise one of these exceptions and turn it
into a response.
One last thing we'll do in our __call__ method is create our
Page object, which represents a wiki page.
All this together makes:
from webob import Request, Response
from webob import exc
class WikiApp(object):
...
def __call__(self, environ, start_response):
req = Request(environ)
action = req.params.get('action', 'view')
# Here's where we get the Page domain object:
page = self.get_page(req.path_info)
try:
try:
# The method name is action_{action_param}_{request_method}:
meth = getattr(self, 'action_%s_%s' % (action, req.method))
except AttributeError:
# If the method wasn't found there must be
# something wrong with the request:
raise exc.HTTPBadRequest('No such action %r' % action)
resp = meth(req, page)
except exc.HTTPException, e:
# The exception object itself is a WSGI application/response:
resp = e
return resp(environ, start_response)
The Domain Object¶
The Page domain object isn't really related to the web, but it is
important to implementing this. Each Page is just a file on the
filesystem. Our get_page method figures out the filename given
the path (the path is in req.path_info, which is all the path
after the base path). The Page class handles getting and setting
the title and content.
Here's the method to figure out the filename:
import os
class WikiApp(object):
...
def get_page(self, path):
path = path.lstrip('/')
if not path:
# The path was '/', the home page
path = 'index'
path = os.path.join(self.storage_dir)
path = os.path.normpath(path)
if path.endswith('/'):
path += 'index'
if not path.startswith(self.storage_dir):
raise exc.HTTPBadRequest("Bad path")
path += '.html'
return Page(path)
Mostly this is just the kind of careful path construction you have to
do when mapping a URL to a filename. While the server may normalize
the path (so that a path like /../../ can't be requested), you can
never really be sure. By using os.path.normpath we eliminate
these, and then we make absolutely sure that the resulting path is
under our self.storage_dir with if not
path.startswith(self.storage_dir): raise exc.HTTPBadRequest("Bad
path").
Here's the actual domain object:
class Page(object):
def __init__(self, filename):
self.filename = filename
@property
def exists(self):
return os.path.exists(self.filename)
@property
def title(self):
if not self.exists:
# we need to guess the title
basename = os.path.splitext(os.path.basename(self.filename))[0]
basename = re.sub(r'[_-]', ' ', basename)
return basename.capitalize()
content = self.full_content
match = re.search(r'<title>(.*?)</title>', content, re.I|re.S)
return match.group(1)
@property
def full_content(self):
f = open(self.filename, 'rb')
try:
return f.read()
finally:
f.close()
@property
def content(self):
if not self.exists:
return ''
content = self.full_content
match = re.search(r'<body[^>]*>(.*?)</body>', content, re.I|re.S)
return match.group(1)
@property
def mtime(self):
if not self.exists:
return None
else:
return int(os.stat(self.filename).st_mtime)
def set(self, title, content):
dir = os.path.dirname(self.filename)
if not os.path.exists(dir):
os.makedirs(dir)
new_content = """<html><head><title>%s</title></head><body>%s</body></html>""" % (
title, content)
f = open(self.filename, 'wb')
f.write(new_content)
f.close()
Basically it provides a .title attribute, a .content
attribute, the .mtime (last modified time), and the page can exist
or not (giving appropriate guesses for title and content when the page
does not exist). It encodes these on the filesystem as a simple HTML
page that is parsed by some regular expressions.
None of this really applies much to the web or WebOb, so I'll leave it to you to figure out the details of this.
URLs, PATH_INFO, and SCRIPT_NAME¶
This is an aside for the tutorial, but an important concept. In WSGI, and accordingly with WebOb, the URL is split up into several pieces. Some of these are obvious and some not.
An example:
http://example.com:8080/wiki/article/12?version=10
There are several components here:
req.scheme:
httpreq.host:
example.com:8080req.server_name:
example.comreq.server_port: 8080
req.script_name:
/wikireq.path_info:
/article/12req.query_string:
version=10
One non-obvious part is req.script_name and req.path_info.
These correspond to the CGI environmental variables SCRIPT_NAME
and PATH_INFO. req.script_name points to the application.
You might have several applications in your site at different paths:
one at /wiki, one at /blog, one at /. Each application
doesn't necessarily know about the others, but it has to construct its
URLs properly -- so any internal links to the wiki application should
start with /wiki.
Just as there are pieces to the URL, there are several properties in WebOb to construct URLs based on these:
req.host_url:
http://example.com:8080req.application_url:
http://example.com:8080/wikireq.path_url:
http://example.com:8080/wiki/article/12req.path:
/wiki/article/12req.path_qs:
/wiki/article/12?version=10req.url:
http://example.com:8080/wiki/article/12?version10
You can also create URLs with
req.relative_url('some/other/page'). In this example that would
resolve to http://example.com:8080/wiki/article/some/other/page.
You can also create a relative URL to the application URL
(SCRIPT_NAME) like req.relative_url('some/other/page', True) which
would be http://example.com:8080/wiki/some/other/page.
Back to the Application¶
We have a dispatching function with __call__ and we have a domain
object with Page, but we aren't actually doing anything.
The dispatching goes to action_ACTION_METHOD, where ACTION
defaults to view. So a simple page view will be
action_view_GET. Let's implement that:
class WikiApp(object):
...
def action_view_GET(self, req, page):
if not page.exists:
return exc.HTTPTemporaryRedirect(
location=req.url + '?action=edit')
text = self.view_template.substitute(
page=page, req=req)
resp = Response(text)
resp.last_modified = page.mtime
resp.conditional_response = True
return resp
The first thing we do is redirect the user to the edit screen if the
page doesn't exist. exc.HTTPTemporaryRedirect is a response that
gives a 307 Temporary Redirect response with the given location.
Otherwise we fill in a template. The template language we're going to
use in this example is Tempita, a
very simple template language with a similar interface to
string.Template.
The template actually looks like this:
from tempita import HTMLTemplate
VIEW_TEMPLATE = HTMLTemplate("""\
<html>
<head>
<title>{{page.title}}</title>
</head>
<body>
<h1>{{page.title}}</h1>
<div>{{page.content|html}}</div>
<hr>
<a href="{{req.url}}?action=edit">Edit</a>
</body>
</html>
""")
class WikiApp(object):
view_template = VIEW_TEMPLATE
...
As you can see it's a simple template using the title and the body,
and a link to the edit screen. We copy the template object into a
class method (view_template = VIEW_TEMPLATE) so that potentially a
subclass could override these templates.
tempita.HTMLTemplate is a template that does automatic HTML
escaping. Our wiki will just be written in plain HTML, so we disable
escaping of the content with {{page.content|html}}.
So let's look at the action_view_GET method again:
def action_view_GET(self, req, page):
if not page.exists:
return exc.HTTPTemporaryRedirect(
location=req.url + '?action=edit')
text = self.view_template.substitute(
page=page, req=req)
resp = Response(text)
resp.last_modified = page.mtime
resp.conditional_response = True
return resp
The template should be pretty obvious now. We create a response with
Response(text), which already has a default Content-Type of
text/html.
To allow conditional responses we set resp.last_modified. You can
set this attribute to a date, None (effectively removing the header),
a time tuple (like produced by time.localtime()), or as in this
case to an integer timestamp. If you get the value back it will
always be a datetime object
(or None). With this header we can process requests with
If-Modified-Since headers, and return 304 Not Modified if
appropriate. It won't actually do that unless you set
resp.conditional_response to True.
Note
If you subclass webob.Response you can set the class attribute
default_conditional_response = True and this setting will be
on by default. You can also set other defaults, like the
default_charset ("utf8"), or default_content_type
("text/html").
The Edit Screen¶
The edit screen will be implemented in the method
action_edit_GET. There's a template and a very simple method:
EDIT_TEMPLATE = HTMLTemplate("""\
<html>
<head>
<title>Edit: {{page.title}}</title>
</head>
<body>
{{if page.exists}}
<h1>Edit: {{page.title}}</h1>
{{else}}
<h1>Create: {{page.title}}</h1>
{{endif}}
<form action="{{req.path_url}}" method="POST">
<input type="hidden" name="mtime" value="{{page.mtime}}">
Title: <input type="text" name="title" style="width: 70%" value="{{page.title}}"><br>
Content: <input type="submit" value="Save">
<a href="{{req.path_url}}">Cancel</a>
<br>
<textarea name="content" style="width: 100%; height: 75%" rows="40">{{page.content}}</textarea>
<br>
<input type="submit" value="Save">
<a href="{{req.path_url}}">Cancel</a>
</form>
</body></html>
""")
class WikiApp(object):
...
edit_template = EDIT_TEMPLATE
def action_edit_GET(self, req, page):
text = self.edit_template.substitute(
page=page, req=req)
return Response(text)
As you can see, all the action here is in the template.
In <form action="{{req.path_url}}" method="POST"> we submit to
req.path_url; that's everything but ?action=edit. So we are
POSTing right over the view page. This has the nice side effect of
automatically invalidating any caches of the original page. It also
is vaguely RESTful.
We save the last modified time in a hidden mtime field. This way
we can detect concurrent updates. If start editing the page who's
mtime is 100000, and someone else edits and saves a revision changing
the mtime to 100010, we can use this hidden field to detect that
conflict. Actually resolving the conflict is a little tricky and
outside the scope of this particular tutorial, we'll just note the
conflict to the user in an error.
From there we just have a very straight-forward HTML form. Note that
we don't quote the values because that is done automatically by
HTMLTemplate; if you are using something like string.Template
or a templating language that doesn't do automatic quoting, you have
to be careful to quote all the field values.
We don't have any error conditions in our application, but if there were error conditions we might have to re-display this form with the input values the user already gave. In that case we'd do something like:
<input type="text" name="title"
value="{{req.params.get('title', page.title)}}">
This way we use the value in the request (req.params is both the
query string parameters and any variables in a POST response), but if
there is no value (e.g., first request) then we use the page values.
Processing the Form¶
The form submits to action_view_POST (view is the default
action). So we have to implement that method:
class WikiApp(object):
...
def action_view_POST(self, req, page):
submit_mtime = int(req.params.get('mtime') or '0') or None
if page.mtime != submit_mtime:
return exc.HTTPPreconditionFailed(
"The page has been updated since you started editing it")
page.set(
title=req.params['title'],
content=req.params['content'])
resp = exc.HTTPSeeOther(
location=req.path_url)
return resp
The first thing we do is check the mtime value. It can be an empty
string (when there's no mtime, like when you are creating a page) or
an integer. int(req.params.get('time') or '0') or None basically
makes sure we don't pass "" to int() (which is an error) then
turns 0 into None (0 or None will evaluate to None in Python --
false_value or other_value in Python resolves to other_value).
If it fails we just give a not-very-helpful error message, using 412
Precondition Failed (typically preconditions are HTTP headers like
If-Unmodified-Since, but we can't really get the browser to send
requests like that, so we use the hidden field instead).
Note
Error statuses in HTTP are often under-used because people think they need to either return an error (useful for machines) or an error message or interface (useful for humans). In fact you can do both: you can give any human readable error message with your error response.
One problem is that Internet Explorer will replace error messages
with its own incredibly unhelpful error messages. However, it
will only do this if the error message is short. If it's fairly
large (4Kb is large enough) it will show the error message it was
given. You can load your error with a big HTML comment to
accomplish this, like "<!-- %s -->" % ('x'*4000).
You can change the status of any response with resp.status_int =
412, or you can change the body of an exc.HTTPSomething with
resp.body = new_body. The primary advantage of using the
classes in webob.exc is giving the response a clear name and a
boilerplate error message.
After we check the mtime we get the form parameters from
req.params and issue a redirect back to the original view page.
303 See Other is a good response to give after accepting a POST
form submission, as it gets rid of the POST (no warning messages for the
user if they try to go back).
In this example we've used req.params for all the form values. If
we wanted to be specific about where we get the values from, they
could come from req.GET (the query string, a misnomer since the
query string is present even in POST requests) or req.POST (a POST
form body). While sometimes it's nice to distinguish between these
two locations, for the most part it doesn't matter. If you want to
check the request method (e.g., make sure you can't change a page with
a GET request) there's no reason to do it by accessing these
method-specific getters. It's better to just handle the method
specifically. We do it here by including the request method in our
dispatcher (dispatching to action_view_GET or
action_view_POST).
Conclusion¶
We're done, hurrah!