| http(3tcl) | Tcl Bundled Packages | http(3tcl) |
http - Client-side implementation of the HTTP/1.1 protocol
package require http ?2.9?
::http::config ?-option value ...?
::http::geturl url ?-option value ...?
::http::formatQuery key value ?key value ...?
::http::quoteString value
::http::reset token ?why?
::http::wait token
::http::status token
::http::size token
::http::code token
::http::ncode token
::http::meta token
::http::data token
::http::error token
::http::cleanup token
::http::register proto port command
::http::registerError port ?message?
::http::unregister proto
Namespace http exports the commands config, formatQuery, geturl, quoteString, register, registerError, reset, unregister, and wait.
It does not export the commands cleanup, code, data, error, meta, ncode, size, or status.
The http package provides the client side of the HTTP/1.1 protocol, as defined in RFC 7230 to RFC 7235, which supersede RFC 2616. The package implements the GET, POST, and HEAD operations of HTTP/1.1. It allows configuration of a proxy host to get through firewalls. The package is compatible with the Safesock security policy, so it can be used by untrusted applets to do URL fetching from a restricted set of hosts. This package can be extended to support additional HTTP transport protocols, such as HTTPS, by providing a custom socket command, via ::http::register.
The ::http::geturl procedure does a HTTP transaction. Its options determine whether a GET, POST, or HEAD transaction is performed. The return value of ::http::geturl is a token for the transaction. The value is also the name of an array in the ::http namespace that contains state information about the transaction. The elements of this array are described in the STATE ARRAY section.
If the -command option is specified, then the HTTP operation is done in the background. ::http::geturl returns immediately after generating the HTTP request and the callback is invoked when the transaction completes. For this to work, the Tcl event loop must be active. In Tk applications this is always true. For pure-Tcl applications, the caller can use ::http::wait after calling ::http::geturl to start the event loop.
Note: The event queue is even used without the -command option. As a side effect, arbitrary commands may be processed while http::geturl is running.
The ::http::geturl command runs the -proxyfilter callback inside a catch command. Therefore an error in the callback command does not call the bgerror handler. See the ERRORS section for details.
proc httpCallback {token} {
upvar #0 $token state
# Access state as a Tcl array
}
The ::http::geturl command runs the -command callback inside a catch command. Therefore an error in the callback command does not call the bgerror handler. See the ERRORS section for details.
proc httpHandlerCallback {socket token} {
upvar #0 $token state
# Access socket, and state as a Tcl array
# For example...
...
set data [read $socket 1000]
set nbytes [string length $data]
...
return $nbytes
}
The http::geturl code for the -handler option is not compatible with either compression or chunked transfer-encoding. If -handler is specified, then to work around these issues http::geturl will reduce the HTTP protocol to 1.0, and override the -zip option (i.e. it will not send the header "Accept-Encoding: gzip,deflate,compress").
If options -handler and -channel are used together, the handler is responsible for copying the data from the HTTP socket to the specified channel. The name of the channel is available to the handler as element -channel of the token array.
The ::http::geturl command runs the -handler callback inside a catch command. Therefore an error in the callback command does not call the bgerror handler. See the ERRORS section for details.
Pragma: no-cache
It is the caller's responsibility to ensure that the headers and request body (if any) conform to the requirements of the request method. For example, if using -method POST to send a POST with an empty request body, the caller must also supply the option “-headers {Content-Length 0}”.
proc httpProgress {token total current} {
upvar #0 $token state
}
If -type is not specified, it defaults to application/x-www-form-urlencoded, which requires query to be an x-url-encoding formatted query-string (this -type and query format are used in a POST submitted from an html form). The ::http::formatQuery procedure can be used to do the formatting.
package require http package require tls ::http::register https 443 ::tls::socket set token [::http::geturl https://my.secure.site/]
The ::http::geturl procedure will raise errors in the following cases: invalid command line options, an invalid URL, a URL on a non-existent host, or a URL at a bad port on an existing host. These errors mean that it cannot even start the network transaction. It will also raise an error if it gets an I/O error while writing out the HTTP request header. For synchronous ::http::geturl calls (where -command is not specified), it will raise an error if it gets an I/O error while reading the HTTP reply headers or data. Because ::http::geturl does not return a token in these cases, it does all the required cleanup and there is no issue of your app having to call ::http::cleanup.
For asynchronous ::http::geturl calls, all of the above error situations apply, except that if there is any error while reading the HTTP reply headers or data, no exception is thrown. This is because after writing the HTTP headers, ::http::geturl returns, and the rest of the HTTP transaction occurs in the background. The command callback can check if any error occurred during the read by calling ::http::status to check the status and if its error, calling ::http::error to get the error message.
Alternatively, if the main program flow reaches a point where it needs to know the result of the asynchronous HTTP request, it can call ::http::wait and then check status and error, just as the callback does.
The ::http::geturl command runs the -command, -handler, and -proxyfilter callbacks inside a catch command. Therefore an error in the callback command does not call the bgerror handler. When debugging one of these callbacks, it may be convenient to report errors by using a catch command within the callback command itself, e.g. to write an error message to stdout.
In any case, you must still call ::http::cleanup to delete the state array when you are done.
There are other possible results of the HTTP transaction determined by examining the status from ::http::status. These are described below.
Another error possibility is that ::http::geturl is unable to write all the post query data to the server before the server responds and closes the socket. The error message is saved in the posterror status array element and then ::http::geturl attempts to complete the transaction. If it can read the server's response it will end up with an ok status, otherwise it will have an eof status.
The ::http::geturl procedure returns a token that can be used to get to the state of the HTTP transaction in the form of a Tcl array. Use this construct to create an easy-to-use array variable:
upvar #0 $token state
Once the data associated with the URL is no longer needed, the state array should be unset to free up storage. The ::http::cleanup procedure is provided for that purpose. The following elements of the array are supported:
HTTP/1.1 code string
The code is a three-digit number defined in the HTTP standard. A code of 200 is OK. Codes beginning with 4 or 5 indicate errors. Codes beginning with 3 are redirection errors. In this case the Location meta-data specifies a new URL that contains the requested information.
array set meta $state(meta)
Some of the meta-data keys are listed below, but the HTTP standard defines more, and servers are free to add their own.
See RFC 7230 Sec 6, which supersedes RFC 2616 Sec 8.1.
A persistent connection allows multiple HTTP/1.1 transactions to be carried over the same TCP connection. Pipelining allows a client to make multiple requests over a persistent connection without waiting for each response. The server sends responses in the same order that the requests were received.
If a POST request fails to complete, typically user confirmation is needed before sending the request again. The user may wish to verify whether the server was modified by the failed POST request, before sending the same request again.
A HTTP request will use a persistent socket if the call to http::geturl has the option -keepalive true. It will use pipelining where permitted if the http::config option -pipeline is boolean true (its default value).
The http package maintains no more than one persistent connection to each server (i.e. each value of “domain:port”). If http::geturl is called to make a request over a persistent connection while the connection is busy with another request, the new request will be held in a queue until the connection is free.
The http package does not support HTTP/1.0 persistent connections controlled by the Keep-Alive header.
This subsection discusses issues related to closure of the persistent connection by the server, automatic retry of failed requests, the special treatment necessary for POST requests, and the options for dealing with these cases.
In accordance with RFC 7230, http::geturl does not pipeline requests that use the POST method. If a POST uses a persistent connection and is not the first request on that connection, http::geturl waits until it has received the response for the previous request; or (if http::config option -postfresh is boolean true) it uses a new connection for each POST.
If the server is processing a number of pipelined requests, and sends a response header “Connection: close” with one of the responses (other than the last), then subsequent responses are unfulfilled. http::geturl will send the unfulfilled requests again over a new connection.
A difficulty arises when a HTTP client sends a request over a persistent connection that has been idle for a while. The HTTP server may half-close an apparently idle connection while the client is sending a request, but before the request arrives at the server: in this case (an “asynchronous close event”) the request will fail. The difficulty arises because the client cannot be certain whether the POST modified the state of the server. For HEAD or GET requests, http::geturl opens another connection and retransmits the failed request. However, if the request was a POST, RFC 7230 forbids automatic retry by default, suggesting either user confirmation, or confirmation by user-agent software that has semantic understanding of the application. The http::config option -repost allows for either possibility.
Asynchronous close events can occur only in a short interval of time. The http package monitors each persistent connection for closure by the server. Upon detection, the connection is also closed at the client end, and subsequent requests will use a fresh connection.
If the http::geturl command is called with option -keepalive true, then it will both try to use an existing persistent connection (if one is available), and it will send the server a “Connection: keep-alive” request header asking to keep the connection open for future requests.
The http::config options -pipeline, -postfresh, and -repost relate to persistent connections.
Option -pipeline, if boolean true, will pipeline GET and HEAD requests made over a persistent connection. POST requests will not be pipelined - if the POST is not the first transaction on the connection, its request will not be sent until the previous response has finished. GET and HEAD requests made after a POST will not be sent until the POST response has been delivered, and will not be sent if the POST fails.
Option -postfresh, if boolean true, will override the http::geturl option -keepalive, and always open a fresh connection for a POST request.
Option -repost, if true, permits automatic retry of a POST request that fails because it uses a persistent connection that the server has half-closed (an “asynchronous close event”). Subsequent GET and HEAD requests in a failed pipeline will also be retried. The -repost option should be used only if the application understands that the retry is appropriate - specifically, the application must know that if the failed POST successfully modified the state of the server, a repeat POST would have no adverse effect.
The HTTP/1.1 Connection and Upgrade client headers inform the server that the client wishes to change the protocol used over the existing connection (RFC 7230). This mechanism can be used to request a WebSocket (RFC 6455), a higher version of the HTTP protocol (HTTP 2), or TLS encryption. If the server accepts the upgrade request, its response code will be 101.
To request a protocol upgrade when calling http::geturl, the -headers option must supply appropriate values for Connection and Upgrade, and the -command option must supply a command that implements the requested protocol and can also handle the server response if the server refuses the protocol upgrade. For upgrade requests http::geturl ignores the value of option -keepalive, and always uses the value 0 so that the upgrade request is not made over a connection that is intended for multiple HTTP requests.
The Tcllib library websocket implements WebSockets, and makes the necessary calls to commands in the http package.
There is currently no native Tcl client library for HTTP/2.
The Upgrade mechanism is not used to request TLS in web browsers, because http and https are served over different ports. It is used by protocols such as Internet Printing Protocol (IPP) that are built on top of http(s) and use the same TCP port number for both secure and insecure traffic.
In browsers, opportunistic encryption is instead implemented by the Upgrade-Insecure-Requests client header. If a secure service is available, the server response code is a 307 redirect, and the response header Location specifies the target URL. The browser must call http::geturl again in order to fetch this URL. See https://w3c.github.io/webappsec-upgrade-insecure-requests/
This example creates a procedure to copy a URL to a file while printing a progress meter, and prints the meta-data associated with the URL.
proc httpcopy { url file {chunk 4096} } {
set out [open $file w]
set token [::http::geturl $url -channel $out \
-progress httpCopyProgress -blocksize $chunk]
close $out
# This ends the line started by httpCopyProgress
puts stderr ""
upvar #0 $token state
set max 0
foreach {name value} $state(meta) {
if {[string length $name] > $max} {
set max [string length $name]
}
if {[regexp -nocase ^location$ $name]} {
# Handle URL redirects
puts stderr "Location:$value"
return [httpcopy [string trim $value] $file $chunk]
}
}
incr max
foreach {name value} $state(meta) {
puts [format "%-*s %s" $max $name: $value]
}
return $token
}
proc httpCopyProgress {args} {
puts -nonewline stderr .
flush stderr
}
internet, security policy, socket, www
| 2.9 | http |