tool - A TclOO and coroutine based web server
package require Tcl 8.6
package require httpd ?4.1.1?
package require sha1
package require dicttool
package require oo::meta
package require oo::dialect
package require tool
package require coroutine
package require fileutil
package require fileutil::magic::filetype
package require websocket
package require mime
package require cron
package require uri
package require Markdown
constructor ?port ?port?? ?myaddr ?ipaddr?|all? ?server_string
?string?? ?server_name ?string??
method add_uri pattern dict
method connect sock ip port
method Connect uuid sock ip
method counter which
method CheckTimeout
method dispatch header_dict
method log args
method port_listening
method PrefixNormalize prefix
method start
method stop
method template page
method TemplateSearch page
method Validate_Connection sock ip
method ENSEMBLE::add field element
method ENSEMBLE::dump
method ENSEMBLE::get field
method ENSEMBLE::reset
method ENSEMBLE::remove field element
method ENSEMBLE::replace keyvaluelist
method ENSEMBLE::reset
method ENSEMBLE::set field value
method http_info::netstring
method request::parse string
method reply::output
method close
method HttpHeaders sock ?debug?
method dispatch newsock datastate
method error code ?message?
?errorInfo?
method content
method EncodeStatus status
method FormData
method MimeParse mimetext
method output
method DoOutput
method PostData length
method puts string
method reset
method timeOutCheck
method timestamp
method TransferComplete args
method Url_Decode string
method cgi_info
option path
option prefix
method proxy_info
method scgi_info
This module implements a web server, suitable for embedding in an
application. The server is object oriented, and contains all of the
fundamentals needed for a full service website.
Starting a web service requires starting a class of type
httpd::server, and providing that server with one or more URIs to
service, and httpd::reply derived classes to generate them.
tool::define ::reply.hello {
method content {} {
my puts "<HTML><HEAD><TITLE>IRM Dispatch Server</TITLE></HEAD><BODY>"
my puts "<h1>Hello World!</h1>"
my puts </BODY></HTML>
}
}
::docserver::server create HTTPD port 8015 myaddr 127.0.0.1
HTTPD add_uri /* [list mixin reply.hello]
CLASS ::HTTPD::SERVER
This class is the root object of the webserver. It is responsible
for opening the socket and providing the initial connection negotiation.
- constructor
?port ?port?? ?myaddr ?ipaddr?|all? ?server_string ?string?? ?server_name
?string??
- Build a new server object. ?port? is the port to listen on
- method add_uri
pattern dict
- Set the hander for a URI pattern. Information given in the dict is
stored in the data structure the dispatch method uses. If a field
called mixin is given, that class will be mixed into the reply
object immediately after construction.
- method connect
sock ip port
- Reply to an open socket. This method builds a coroutine to manage the
remainder of the connection. The coroutine's operations are driven by the
Connect method.
- method Connect
uuid sock ip
- This method reads HTTP headers, and then consults the dispatch
method to determine if the request is valid, and/or what kind of reply to
generate. Under normal cases, an object of class ::http::reply is
created. Fields the server are looking for in particular are: class: A
class to use instead of the server's own reply_class mixin: A class
to be mixed into the new object after construction. All other fields are
passed along to the http_info structure of the reply object. After
the class is created and the mixin is mixed in, the server invokes the
reply objects dispatch method. This action passes control of the
socket to the reply object. The reply object manages the rest of the
transaction, including closing the socket.
- method counter
which
- Increment an internal counter.
- method
CheckTimeout
- Check open connections for a time out event.
- method dispatch
header_dict
- Given a key/value list of information, return a data structure describing
how the server should reply.
- method log
args
- Log an event. The input for args is free form. This method is intended to
be replaced by the user, and is a noop for a stock http::server
object.
- method
port_listening
- Return the actual port that httpd is listening on.
- method
PrefixNormalize prefix
- For the stock version, trim trailing /'s and *'s from a prefix. This
method can be replaced by the end user to perform any other
transformations needed for the application.
- method
start
- Open the socket listener.
- method
stop
- Shut off the socket listener, and destroy any pending replies.
- method
template page
- Return a template for the string page
- method
TemplateSearch page
- Perform a search for the template that best matches page. This can
include local file searches, in-memory structures, or even database
lookups. The stock implementation simply looks for files with a .tml or
.html extension in the ?doc_root? directory.
- method
Validate_Connection sock ip
- Given a socket and an ip address, return true if this connection should be
terminated, or false if it should be allowed to continue. The stock
implementation always returns 0. This is intended for applications to be
able to implement black lists and/or provide security based on IP
address.
CLASS ::HTTPD::REPLY
A class which shephards a request through the process of
generating a reply. The socket associated with the reply is available at all
times as the chan variable. The process of generating a reply begins
with an httpd::server generating a http::class object, mixing
in a set of behaviors and then invoking the reply object's dispatch
method. In normal operations the dispatch method:
- [1]
- Invokes the reset method for the object to populate default
headers.
- [2]
- Invokes the HttpHeaders method to stream the MIME headers out of
the socket
- [3]
- Invokes the request parse method to convert the stream of MIME
headers into a dict that can be read via the request method.
- [4]
- Stores the raw stream of MIME headers in the rawrequest variable of
the object.
- [5]
- Invokes the content method for the object, generating an call to
the error method if an exception is raised.
- [6]
- Invokes the output method for the object
Manages HTTP headers passed in by the server. Ensemble
Methods:
- method
http_info::netstring
- Return the contents of this data structure as a netstring encoded
block.
Managed data from MIME headers of the request.
- method
request::parse string
- Replace the contents of the data structure with information encoded in a
MIME formatted block of text (string).
Manage the headers sent in the reply.
- method
reply::output
- Return the contents of this data structure as a MIME encoded block
appropriate for an HTTP response.
- method
close
- Terminate the transaction, and close the socket.
- method
HttpHeaders sock ?debug?
- Stream MIME headers from the socket sock, stopping at an empty
line. Returns the stream as a block of text.
- method
dispatch newsock datastate
- Take over control of the socket newsock, and store that as the
chan variable for the object. This method runs through all of the
steps of reading HTTP headers, generating content, and closing the
connection. (See class writetup).
- method error
code ?message? ?errorInfo?
- Generate an error message of the specified code, and display the
message as the reason for the exception. errorInfo is passed
in from calls, but how or if it should be displayed is a prerogative of
the developer.
- method
content
- Generate the content for the reply. This method is intended to be replaced
by the mixin. Developers have the option of streaming output to a buffer
via the puts method of the reply, or simply populating the
reply_body variable of the object. The information returned by the
content method is not interpreted in any way. If an exception is
thrown (via the error command in Tcl, for example) the caller will
auto-generate a 505 {Internal Error} message. A typical implementation of
content look like:
tool::define ::test::content.file {
superclass ::httpd::content.file
# Return a file
# Note: this is using the content.file mixin which looks for the reply_file variable
# and will auto-compute the Content-Type
method content {} {
my reset
set doc_root [my http_info get doc_root]
my variable reply_file
set reply_file [file join $doc_root index.html]
}
}
tool::define ::test::content.time {
# return the current system time
method content {} {
my variable reply_body
my reply set Content-Type text/plain
set reply_body [clock seconds]
}
}
tool::define ::test::content.echo {
method content {} {
my variable reply_body
my reply set Content-Type [my request get Content-Type]
set reply_body [my PostData [my request get Content-Length]]
}
}
tool::define ::test::content.form_handler {
method content {} {
set form [my FormData]
my reply set Content-Type {text/html; charset=UTF-8}
my puts "<HTML><HEADER><TITLE>My Dynamic Page</TITLE></HEADER>"
my puts "<BODY>"
my puts "You Sent<p>"
my puts "<TABLE>"
foreach {f v} $form {
my puts "<TR><TH>$f</TH><TD><verbatim>$v</verbatim></TD>"
}
my puts "</TABLE><p>"
my puts "Send some info:<p>"
my puts "<FORM action=/[my http_info get REQUEST_PATH] method POST>"
my puts "<TABLE>"
foreach field {name rank serial_number} {
set line "<TR><TH>$field</TH><TD><input name=\"$field\" "
if {[dict exists $form $field]} {
append line " value=\"[dict get $form $field]\"""
}
append line " /></TD></TR>"
my puts $line
}
my puts "</TABLE>"
my puts "</BODY></HTML>"
}
}
- method
EncodeStatus status
- Formulate a standard HTTP status header from he string provided.
- method
FormData
- For GET requests, converts the QUERY_DATA header into a key/value list.
For POST requests, reads the Post data and converts that information to a
key/value list for application/x-www-form-urlencoded posts. For multipart
posts, it composites all of the MIME headers of the post to a singular
key/value list, and provides MIME_* information as computed by the
mime package, including the MIME_TOKEN, which can be fed back into
the mime package to read out the contents.
- method MimeParse
mimetext
- Converts a block of mime encoded text to a key/value list. If an exception
is encountered, the method will generate its own call to the error
method, and immediately invoke the output method to produce an
error code and close the connection.
- method
output
- Schedules a call to DoOutput when chan becomes
writeable
- method
DoOutput
- Generates the the HTTP reply, and streams that reply back across
chan.
- method PostData
length
- Stream length bytes from the chan socket, but only of the
request is a POST or PUSH. Returns an empty string otherwise.
- method puts
string
- Appends the value of string to the end of reply_body, as
well as a trailing newline character.
- method
reset
- Clear the contents of the reply_body variable, and reset all
headers in the reply structure back to the defaults for this
object.
- method
timeOutCheck
- Called from the http::server object which spawned this reply.
Checks to see if too much time has elapsed while waiting for data or
generating a reply, and issues a timeout error to the request if it has,
as well as destroy the object and close the chan socket.
- method
timestamp
- Return the current system time in the format:
- method
TransferComplete args
- Intended to be invoked from chan copy as a callback. This closes
every channel fed to it on the command line, and then destroys the
object.
###
# Output the body
###
chan configure $sock -translation binary -blocking 0 -buffering full -buffersize 4096
chan configure $chan -translation binary -blocking 0 -buffering full -buffersize 4096
if {$length} {
###
# Send any POST/PUT/etc content
###
chan copy $sock $chan -command [namespace code [list my TransferComplete $sock]]
} else {
catch {close $sock}
chan flush $chan
my destroy
}
- method
Url_Decode string
- De-httpizes a string.
CLASS ::HTTPD::CONTENT
The httpd module includes several ready to use implementations of
content mixins for common use cases. Options are passed in to the
add_uri method of the server.
CLASS ::HTTPD::CONTENT.CGI
An implementation to relay requests to process which will accept
post data streamed in vie stdin, and sent a reply streamed to stdout.
- method
cgi_info
- Mandatory method to be replaced by the end user. If needed, activates the
process to proxy, and then returns a list of three values: exec -
The arguments to send to exec to fire off the responding process, minus
the stdin/stdout redirection.
CLASS ::HTTPD::CONTENT.FILE
An implementation to deliver files from the local file system.
- option
path
- The root directory on the local file system to be exposed via http.
- option
prefix
- The prefix of the URI portion to ignore when calculating relative file
paths.
CLASS ::HTTPD::CONTENT.PROXY
An implementation to relay requests to another HTTP server, and
relay the results back across the request channel.
- method
proxy_info
- Mandatory method to be replaced by the end user. If needed, activates the
process to proxy, and then returns a list of three values:
proxyhost - The hostname where the proxy is located
proxyport - The port to connect to proxyscript - A pre-amble
block of text to send prior to the mirrored request
CLASS ::HTTPD::CONTENT.SCGI
An implementation to relay requests to a server listening on a
socket expecting SCGI encoded requests, and relay the results back across
the request channel.
- method
scgi_info
- Mandatory method to be replaced by the end user. If needed, activates the
process to proxy, and then returns a list of three values: scgihost
- The hostname where the scgi listener is located scgiport - The
port to connect to scgiscript - The contents of the
SCRIPT_NAME header to be sent
CLASS ::HTTPD::CONTENT.WEBSOCKET
A placeholder for a future implementation to manage requests that
can expect to be promoted to a Websocket. Currently it is an empty
class.
The HTTP module also provides an SCGI server implementation, as
well as an HTTP implementation. To use the SCGI functions, create an object
of the http::server.scgi class instead of the http::server
class.
CLASS ::HTTPD::REPLY.SCGI
An modified http::reply implementation that understands how
to deal with netstring encoded headers.
CLASS ::HTTPD::SERVER.SCGI
A modified http::server which is tailored to replying to
request according to the SCGI standard instead of the HTTP standard.
This document, and the package it describes, will undoubtedly
contain bugs and other problems. Please report such in the category
network of the Tcllib Trackers
[http://core.tcl.tk/tcllib/reportlist]. Please also report any ideas for
enhancements you may have for either package and/or documentation.
When proposing code changes, please provide unified diffs,
i.e the output of diff -u.
Note further that attachments are strongly preferred over
inlined patches. Attachments can be made by going to the Edit form of
the ticket immediately after its creation, and then using the left-most
button in the secondary navigation bar.
TclOO, WWW, http, httpd, httpserver, services
Copyright (c) 2018 Sean Woods <yoda@etoyoc.com>