HTML templating is the process of transforming a template document (one which describes style and structure, but does not itself include any content) into some HTML output which includes information about objects in your application. There are many, many libraries for doing this in Python: to name a few, jinja2 , django templates , and clearsilver . You can easily use any of these libraries in your Twisted Web application, either by running them as WSGI applications or by calling your preferred templating system’s APIs to produce their output as strings, and then writing those strings to Request.write .
Before we begin explaining how to use it, I’d like to stress that you
don’t need to use Twisted’s templating system if you prefer some other
way to generate HTML. Use it if it suits your personal style or your
application, but feel free to use other things. Twisted includes templating for
its own use, because the twisted.web server needs to produce HTML
in various places, and we didn’t want to add another large dependency for that.
Twisted is not in any way incompatible with other systems, so that has
nothing to do with the fact that we use our own.
Twisted includes a templating system, twisted.web.template . This can be convenient for Twisted applications that want to produce some basic HTML for a web interface without an additional dependency.
twisted.web.template also includes
support for Deferred s, so
you can incrementally render the output of a page based on the results of Deferred s that your application
has returned. This feature is fairly unique among templating libraries.
In twisted.web.template , templates are XHTML files which also contain a special namespace for indicating dynamic portions of the document. For example:
<html xmlns:t="http://twistedmatrix.com/ns/twisted.web.template/0.1">
<body>
<div t:render="header" />
<div id="content">
<p>Content goes here.</p>
</div>
<div t:render="footer" />
</body>
</html>
The basic unit of templating is twisted.web.template.Element . An Element is given a way of
loading a bit of markup like the above example, and knows how to
correlate render attributes within that markup to Python methods
exposed with twisted.web.template.renderer :
from twisted.web.template import Element, renderer, XMLFile
from twisted.python.filepath import FilePath
class ExampleElement(Element):
loader = XMLFile(FilePath('template-1.xml'))
@renderer
def header(self, request, tag):
return tag('Header.')
@renderer
def footer(self, request, tag):
return tag('Footer.')
In order to combine the two, we must render the element. For this simple example, we can use the flattenString API, which will convert a single template object - such as an Element - into a Deferred which fires with a single string, the HTML output of the rendering process.
from __future__ import print_function
from twisted.web.template import flattenString
from element_1 import ExampleElement
def renderDone(output):
print(output)
flattenString(None, ExampleElement()).addCallback(renderDone)
This short program cheats a little bit; we know that there are no Deferred s in the template which
require the reactor to eventually fire; therefore, we can simply add a callback
which outputs the result. Also, none of the renderer functions
require the request object, so it’s acceptable to
pass None through here. (The ‘request’ object here is used only to
relay information about the rendering process to each renderer, so you may
always use whatever object makes sense for your application. Note, however,
that renderers from library code may require an IRequest .)
If you run it yourself, you can see that it produces the following output:
<html>
<body>
<div>Header.</div>
<div id="content">
<p>Content goes here.</p>
</div>
<div>Footer.</div>
</body>
</html>
The third parameter to a renderer method is a Tag object which represents the XML element
with the t:render attribute in the template. Calling a Tag adds children to the element
in the DOM, which may be strings, more Tag s, or other renderables such as Element s.
For example, to make the header and footer bold:
from twisted.web.template import Element, renderer, XMLFile, tags
from twisted.python.filepath import FilePath
class ExampleElement(Element):
loader = XMLFile(FilePath('template-1.xml'))
@renderer
def header(self, request, tag):
return tag(tags.b('Header.'))
@renderer
def footer(self, request, tag):
return tag(tags.b('Footer.'))
Rendering this in a similar way to the first example would produce:
<html>
<body>
<div><b>Header.</b></div>
<div id="content">
<p>Content goes here.</p>
</div>
<div><b>Footer.</b></div>
</body>
</html>
In addition to adding children, call syntax can be used to set attributes on a
tag. For example, to change the id on the div while
adding children:
from twisted.web.template import Element, renderer, XMLFile, tags
from twisted.python.filepath import FilePath
class ExampleElement(Element):
loader = XMLFile(FilePath('template-1.xml'))
@renderer
def header(self, request, tag):
return tag(tags.p('Header.'), id='header')
@renderer
def footer(self, request, tag):
return tag(tags.p('Footer.'), id='footer')
And this would produce the following page:
<html>
<body>
<div id="header"><p>Header.</p></div>
<div id="content">
<p>Content goes here.</p>
</div>
<div id="footer"><p>Footer.</p></div>
</body>
</html>
Calling a tag mutates it, it and returns the tag itself, so you can pass it
forward and call it multiple times if you have multiple children or attributes
to add to it. twisted.web.template also exposes some
convenient objects for building more complex markup structures from within
renderer methods in the tags object. In the examples above, we’ve
only used tags.p and tags.b , but there should be a tags.x for each x which is a valid HTML tag. There may be
some omissions, but if you find one, please feel free to file a bug.
t:attr tags allow you to set HTML attributes
(like href in an <a href="... ) on an enclosing
element.
t:slot tags allow you to specify “slots” which you can
conveniently fill with multiple pieces of data straight from your Python
program.
The following example demonstrates both t:attr
and t:slot in action. Here we have a layout which displays a person’s
profile on your snazzy new Twisted-powered social networking site. We use
the t:attr tag to drop in the “src” attribute on the profile picture,
where the actual value of src attribute gets specified by a t:slot
tag within the t:attr tag. Confused? It should make more
sense when you see the code:
<div xmlns:t="http://twistedmatrix.com/ns/twisted.web.template/0.1"
t:render="person_profile"
class="profile">
<img><t:attr name="src"><t:slot name="profile_image_url" /></t:attr></img>
<p><t:slot name="person_name" /></p>
</div>
from twisted.web.template import Element, renderer, XMLFile
from twisted.python.filepath import FilePath
class ExampleElement(Element):
loader = XMLFile(FilePath('slots-attributes-1.xml'))
@renderer
def person_profile(self, request, tag):
# Note how convenient it is to pass these attributes in!
tag.fillSlots(person_name='Luke',
profile_image_url='http://example.com/user.png')
return tag
<div class="profile">
<img src="http://example.com/user.png" />
<p>Luke</p>
</div>
Often, you will have a sequence of things, and want to render each of them,
repeating a part of the template for each one. This can be done by
cloning tag in your renderer:
<ul xmlns:t="http://twistedmatrix.com/ns/twisted.web.template/0.1">
<li t:render="widgets"><t:slot name="widgetName"/></li>
</ul>
from __future__ import print_function
from twisted.web.template import Element, renderer, XMLFile, flattenString
from twisted.python.filepath import FilePath
class WidgetsElement(Element):
loader = XMLFile(FilePath('iteration-1.xml'))
widgetData = ['gadget', 'contraption', 'gizmo', 'doohickey']
@renderer
def widgets(self, request, tag):
for widget in self.widgetData:
yield tag.clone().fillSlots(widgetName=widget)
def printResult(result):
print(result)
flattenString(None, WidgetsElement()).addCallback(printResult)
<ul>
<li>gadget</li><li>contraption</li><li>gizmo</li><li>doohickey</li>
</ul>
This renderer works because a renderer can return anything that can be
rendered, not just tag . In this case, we define a generator, which
returns a thing that is iterable. We also could have returned
a list . Anything that is iterable will be rendered by twisted.web.template rendering each item in it. In
this case, each item is a copy of the tag the renderer received, each filled
with the name of a widget.
Another common pattern is to delegate the rendering logic for a small part of
the page to a separate Element . For example, the widgets from the
iteration example above might be more complicated to render. You can define
an Element subclass which can render a single widget. The renderer
method on the container can then yield instances of this
new Element subclass.
<ul xmlns:t="http://twistedmatrix.com/ns/twisted.web.template/0.1">
<li t:render="widgets"><span t:render="name" /></li>
</ul>
from __future__ import print_function
from twisted.web.template import (
XMLFile, TagLoader, Element, renderer, flattenString)
from twisted.python.filepath import FilePath
class WidgetsElement(Element):
loader = XMLFile(FilePath('subviews-1.xml'))
widgetData = ['gadget', 'contraption', 'gizmo', 'doohickey']
@renderer
def widgets(self, request, tag):
for widget in self.widgetData:
yield WidgetElement(TagLoader(tag), widget)
class WidgetElement(Element):
def __init__(self, loader, name):
Element.__init__(self, loader)
self._name = name
@renderer
def name(self, request, tag):
return tag(self._name)
def printResult(result):
print(result)
flattenString(None, WidgetsElement()).addCallback(printResult)
<ul>
<li><span>gadget</span></li><li><span>contraption</span></li><li><span>gizmo</span></li><li><span>doohickey</span></li>
</ul>
TagLoader lets the portion of the overall template related to
widgets be re-used for WidgetElement , which is otherwise a
normal Element subclass not much different
from WidgetsElement . Notice that the name renderer on
the span tag in this template is satisfied
from WidgetElement , not WidgetsElement .
Note how renderers, slots and attributes require you to specify a renderer on
some outer HTML element. What if you don’t want to be forced to add an element
to your DOM just to drop some content into it? Maybe it messes with your
layout, and you can’t get it to work in IE with that extra div
tag? Perhaps you need t:transparent , which allows you to drop some
content in without any surrounding “container” tag. For example:
<div xmlns:t="http://twistedmatrix.com/ns/twisted.web.template/0.1">
<!-- layout decision - these things need to be *siblings* -->
<t:transparent t:render="renderer1" />
<t:transparent t:render="renderer2" />
</div>
from twisted.web.template import Element, renderer, XMLFile
from twisted.python.filepath import FilePath
class ExampleElement(Element):
loader = XMLFile(FilePath('transparent-1.xml'))
@renderer
def renderer1(self, request, tag):
return tag("hello")
@renderer
def renderer2(self, request, tag):
return tag("world")
<div>
<!-- layout decision - these things need to be *siblings* -->
hello
world
</div>
twisted.web.template will quote any strings that place into the DOM. This provides protection against XSS attacks , in addition to just generally making it easy to put arbitrary strings onto a web page, without worrying about what they might have in them. This can easily be demonstrated with an element using the same template from our earlier examples. Here’s an element that returns some “special” characters in HTML (‘<’, ‘>’, and ‘”’, which is special in attribute values):
from twisted.web.template import Element, renderer, XMLFile
from twisted.python.filepath import FilePath
class ExampleElement(Element):
loader = XMLFile(FilePath('template-1.xml'))
@renderer
def header(self, request, tag):
return tag('<<<Header>>>!')
@renderer
def footer(self, request, tag):
return tag('>>>"Footer!"<<<', id='<"fun">')
Note that they are all safely quoted in the output, and will appear in a web browser just as you returned them from your Python method:
<html>
<body>
<div><<<Header>>>!</div>
<div id="content">
<p>Content goes here.</p>
</div>
<div id="<"fun">">>>>"Footer!"<<<</div>
</body>
</html>
Finally, a simple demonstration of Deferred support, the unique feature of twisted.web.template . Simply put, any renderer may return a Deferred which fires with some template content instead of the template content itself. As shown above, flattenString will return a Deferred that fires with the full content of the string. But if there’s a lot of content, you might not want to wait before starting to send some of it to your HTTP client: for that case, you can use flatten . It’s difficult to demonstrate this directly in a browser-based application; unless you insert very long delays before firing your Deferreds, it just looks like your browser is instantly displaying everything. Here’s an example that just prints out some HTML template, with markers inserted for where certain events happen:
import sys
from twisted.web.template import XMLString, Element, renderer, flatten
from twisted.internet.defer import Deferred
sample = XMLString(
"""
<div xmlns:t="http://twistedmatrix.com/ns/twisted.web.template/0.1">
Before waiting ...
<span t:render="wait"></span>
... after waiting.
</div>
""")
class WaitForIt(Element):
def __init__(self):
Element.__init__(self, loader=sample)
self.deferred = Deferred()
@renderer
def wait(self, request, tag):
return self.deferred.addCallback(
lambda aValue: tag("A value: " + repr(aValue)))
def done(ignore):
print("[[[Deferred fired.]]]")
print('[[[Rendering the template.]]]')
it = WaitForIt()
flatten(None, it, sys.stdout.write).addCallback(done)
print('[[[In progress... now firing the Deferred.]]]')
it.deferred.callback("<value>")
print('[[[All done.]]]')
If you run this example, you should get the following output:
[[[Rendering the template.]]]
<div>
Before waiting ...
[[[In progress... now firing the Deferred.]]]
<span>A value: '<value>'</span>
... after waiting.
</div>[[[Deferred fired.]]]
[[[All done.]]]
This demonstrates that part of the output (everything up to
“[[[In progress... “) is written out immediately as it’s rendered.
But once it hits the Deferred, WaitForIt ‘s rendering needs to pause
until .callback(...) is called on that Deferred. You can see that
no further output is produced until the message indicating that the Deferred is
being fired is complete. By returning Deferreds and using flatten , you can avoid buffering large
amounts of data.
The goal of twisted.web.template is to emit both valid HTML or XHTML .
However, in order to get the maximally standards-compliant output format you
desire, you have to know which one you want, and take a few simple steps to emit
it correctly. Many browsers will probably work with most output if you ignore
this section entirely, but the HTML specification recommends that you specify an appropriate DOCTYPE .
As a DOCTYPE declaration in your template would describe the
template itself, rather than its output, it won’t be included in your output.
If you wish to annotate your template output with a DOCTYPE, you will have to
write it to the browser out of band. One way to do this would be to simply
do request.write('<!DOCTYPE html>\n') when you are ready to
begin emitting your response. The same goes for an XML DOCTYPE
declaration.
twisted.web.template will remove the xmlns attributes
used to declare
the http://twistedmatrix.com/ns/twisted.web.template/0.1 namespace,
but it will not modify other namespace declaration attributes. Therefore if you
wish to serialize in HTML format, you should not use other namespaces; if you
wish to serialize to XML, feel free to insert any namespace declarations that
are appropriate, and they will appear in your output.
Note
This relaxed approach is correct in many cases. However, in certain contexts -
especially <script> and <style> tags - quoting rules differ in
significant ways between HTML and XML, and between different browsers’ parsers
in HTML. If you want to generate dynamic content inside a script or stylesheet,
the best option is to load the resource externally so you don’t have to worry
about quoting rules. The second best option is to strictly configure your
content-types and DOCTYPE declarations for XML, whose quoting rules are simple
and compatible with the approach that twisted.web.template takes.
And, please remember: regardless of how you put it there, any user input placed
inside a <script> or <style> tag is a potential security issue.
Those of you who used Divmod Nevow may notice some
similarities. twisted.web.template is in fact derived from the
latest version of Nevow, but includes only the latest components from Nevow’s
rendering pipeline, and does not have any of the legacy compatibility layers
that Nevow grew over time. This should make
using twisted.web.template a similar experience for many long-time
users of Twisted who have previously used Nevow for its twisted-friendly
templating, but more straightforward for new users.