Design

This document describes the design of websockets. It assumes familiarity with the specification of the WebSocket protocol in RFC 6455.

It’s primarily intended at maintainers. It may also be useful for users who wish to understand what happens under the hood.

Lifecycle

State

WebSocket connections go through a trivial state machine:

• CONNECTING: initial state,

• OPEN: when the opening handshake is complete,

• CLOSING: when the closing handshake is started,

• CLOSED: when the TCP connection is closed.

Transitions happen in the following places:

• CONNECTING -> OPEN: in connection_open() which runs when the opening handshake completes and the WebSocket connection is established — not to be confused with connection_made() which runs when the TCP connection is established;

• OPEN -> CLOSING: in write_frame() immediately before sending a close frame; since receiving a close frame triggers sending a close frame, this does the right thing regardless of which side started the closing handshake; also in fail_connection() which duplicates a few lines of code from write_close_frame() and write_frame();

• * -> CLOSED: in connection_lost() which is always called exactly once when the TCP connection is closed.

Coroutines

The following diagram shows which coroutines are running at each stage of the connection lifecycle on the client side.

The lifecycle is identical on the server side, except inversion of control makes the equivalent of connect() implicit.

Coroutines shown in green are called by the application. Multiple coroutines may interact with the WebSocket connection concurrently.

Coroutines shown in gray manage the connection. When the opening handshake succeeds, connection_open() starts two tasks:

• transfer_data_task runs transfer_data() which handles incoming data and lets recv() consume it. It may be cancelled to terminate the connection. It never exits with an exception other than CancelledError. See data transfer below.

• close_connection_task runs close_connection() which waits for the data transfer to terminate, then takes care of closing the TCP connection. It must not be cancelled. It never exits with an exception. See connection termination below.

Besides, fail_connection() starts the same close_connection_task when the opening handshake fails, in order to close the TCP connection.

Splitting the responsibilities between two tasks makes it easier to guarantee that websockets can terminate connections:

• within a fixed timeout,

• without leaking pending tasks,

• without leaking open TCP connections,

regardless of whether the connection terminates normally or abnormally.

transfer_data_task completes when no more data will be received on the connection. Under normal circumstances, it exits after exchanging close frames.

close_connection_task completes when the TCP connection is closed.

Opening handshake

websockets performs the opening handshake when establishing a WebSocket connection. On the client side, connect() executes it before returning the protocol to the caller. On the server side, it’s executed before passing the protocol to the ws_handler coroutine handling the connection.

While the opening handshake is asymmetrical — the client sends an HTTP Upgrade request and the server replies with an HTTP Switching Protocols response — websockets aims at keepping the implementation of both sides consistent with one another.

On the client side, handshake():

• builds a HTTP request based on the uri and parameters passed to connect();

• writes the HTTP request to the network;

• reads a HTTP response from the network;

• checks the HTTP response, validates extensions and subprotocol, and configures the protocol accordingly;

• moves to the OPEN state.

On the server side, handshake():

• reads a HTTP request from the network;

• calls process_request() which may abort the WebSocket handshake and return a HTTP response instead; this hook only makes sense on the server side;

• checks the HTTP request, negociates extensions and subprotocol, and configures the protocol accordingly;

• builds a HTTP response based on the above and parameters passed to serve();

• writes the HTTP response to the network;

• moves to the OPEN state;

• returns the path part of the uri.

The most significant assymetry between the two sides of the opening handshake lies in the negociation of extensions and, to a lesser extent, of the subprotocol. The server knows everything about both sides and decides what the parameters should be for the connection. The client merely applies them.

If anything goes wrong during the opening handshake, websockets fails the connection.

Data transfer

Symmetry

Once the opening handshake has completed, the WebSocket protocol enters the data transfer phase. This part is almost symmetrical. There are only two differences between a server and a client:

• client-to-server masking: the client masks outgoing frames; the server unmasks incoming frames;

• closing the TCP connection: the server closes the connection immediately; the client waits for the server to do it.

These differences are so minor that all the logic for data framing, for sending and receiving data and for closing the connection is implemented in the same class, WebSocketCommonProtocol.

The is_client attribute tells which side a protocol instance is managing. This attribute is defined on the WebSocketServerProtocol and WebSocketClientProtocol classes.

Data flow

The following diagram shows how data flows between an application built on top of websockets and a remote endpoint. It applies regardless of which side is the server or the client.

Public methods are shown in green, private methods in yellow, and buffers in orange. Methods related to connection termination are omitted; connection termination is discussed in another section below.

Receiving data

The left side of the diagram shows how websockets receives data.

Incoming data is written to a StreamReader in order to implement flow control and provide backpressure on the TCP connection.

transfer_data_task, which is started when the WebSocket connection is established, processes this data.

When it receives data frames, it reassembles fragments and puts the resulting messages in the messages queue.

When it encounters a control frame:

• if it’s a close frame, it starts the closing handshake;

• if it’s a ping frame, it anwsers with a pong frame;

• if it’s a pong frame, it acknowledges the corresponding ping (unless it’s an unsolicited pong).

Running this process in a task guarantees that control frames are processed promptly. Without such a task, websockets would depend on the application to drive the connection by having exactly one coroutine awaiting recv() at any time. While this happens naturally in many use cases, it cannot be relied upon.

Then recv() fetches the next message from the messages queue, with some complexity added for handling termination correctly.

Sending data

The right side of the diagram shows how websockets sends data.

send() writes a single data frame containing the message. Fragmentation isn’t supported at this time.

ping() writes a ping frame and yields a Future which will be completed when a matching pong frame is received.

pong() writes a pong frame.

close() writes a close frame and waits for the TCP connection to terminate.

Outgoing data is written to a StreamWriter in order to implement flow control and provide backpressure from the TCP connection.

Closing handshake

When the other side of the connection initiates the closing handshake, read_message() receives a close frame while in the OPEN state. It moves to the CLOSING state, sends a close frame, and returns None, causing transfer_data_task to terminate.

When this side of the connection initiates the closing handshake with close(), it moves to the CLOSING state and sends a close frame. When the other side sends a close frame, read_message() receives it in the CLOSING state and returns None, also causing transfer_data_task to terminate.

If the other side doesn’t send a close frame within the connection’s timeout, websockets fails the connection.

The closing handshake can take up to 2 * timeout: one timeout to write a close frame and one timeout to receive a close frame.

Then websockets terminates the TCP connection.

Connection termination

close_connection_task, which is started when the WebSocket connection is established, is responsible for eventually closing the TCP connection.

First close_connection_task waits for transfer_data_task to terminate, which may happen as a result of:

• a successful closing handshake: as explained above, this exits the infinite loop in transfer_data_task;

• a timeout while waiting for the closing handshake to complete: this cancels transfer_data_task;

• a protocol error, including connection errors: depending on the exception, transfer_data_task :ref:`fails the connection <connection-failure>`_ with a suitable code and exits.

close_connection_task is separate from transfer_data_task to make it easier to implement the timeout on the closing handshake. Cancelling transfer_data_task creates no risk of cancelling close_connection_task and failing to close the TCP connection, thus leaking resources.

Terminating the TCP connection can take up to 2 * timeout on the server side and 3 * timeout on the client side. Clients start by waiting for the server to close the connection, hence the extra timeout. Then both sides go through the following steps until the TCP connection is lost: half-closing the connection (only for non-TLS connections), closing the connection, aborting the connection. At this point the connection drops regardless of what happens on the network.

Connection failure

If the opening handshake doesn’t complete successfully, websockets fails the connection by closing the TCP connection.

Once the opening handshake has completed, websockets fails the connection by cancelling transfer_data_task and sending a close frame if appropriate.

transfer_data_task exits, unblocking close_connection_task, which closes the TCP connection.

Cancellation

Most public APIs of websockets are coroutines. They may be cancelled. websockets must handle this situation.

Cancellation during the opening handshake is handled like any other exception: the TCP connection is closed and the exception is re-raised or logged.

Once the WebSocket connection is established, transfer_data_task and close_connection_task mustn’t get accidentally cancelled if a coroutine that awaits them is cancelled. They must be shielded from cancellation.

recv() waits for the next message in the queue or for transfer_data_task to terminate, whichever comes first. It relies on wait() for waiting on two tasks in parallel. As a consequence, even though it’s waiting on the transfer data task, it doesn’t propagate cancellation to that task.

ensure_open() is called by send(), ping(), and pong(). When the connection state is CLOSING, it waits for transfer_data_task but shields it to prevent cancellation.

close() waits for the data transfer task to terminate with wait_for(). If it’s cancelled or if the timout elapses, transfer_data_task is cancelled. transfer_data_task is expected to catch the cancellation and terminate properly. This is the only point where it may be cancelled.

close() then waits for close_connection_task but shields it to prevent cancellation.

close_connnection_task starts by waiting for transfer_data_task. Since transfer_data_task handles CancelledError, cancellation doesn’t propagate to close_connnection_task.

Backpressure

Note

This section discusses backpressure from the perspective of a server but the concept applies to clients symmetrically.

With a naive implementation, if a server receives inputs faster than it can process them, or if it generates outputs faster than it can send them, data accumulates in buffers, eventually causing the server to run out of memory and crash.

The solution to this problem is backpressure. Any part of the server that receives inputs faster than it can it can process them and send the outputs must propagate that information back to the previous part in the chain.

websockets is designed to make it easy to get backpressure right.

For incoming data, websockets builds upon StreamReader which propagates backpressure to its own buffer and to the TCP stream. Frames are parsed from the input stream and added to a bounded queue. If the queue fills up, parsing halts until some the application reads a frame.

For outgoing data, websockets builds upon StreamWriter which implements flow control. If the output buffers grow too large, it waits until they’re drained. That’s why all APIs that write frames are asynchronous.

Of course, it’s still possible for an application to create its own unbounded buffers and break the backpressure. Be careful with queues.

Buffers

Note

This section discusses buffers from the perspective of a server but it applies to clients as well.

An asynchronous systems works best when its buffers are almost always empty.

For example, if a client sends data too fast for a server, the queue of incoming messages will be constantly full. The server will always be 32 messages (by default) behind the client. This consumes memory and increases latency for no good reason. The problem is called bufferbloat.

If buffers are almost always full and that problem cannot be solved by adding capacity — typically because the system is bottlenecked by the output and constantly regulated by backpressure — reducing the size of buffers minimizes negative consequences.

By default websockets has rather high limits. You can decrease them according to your application’s characteristics.

Bufferbloat can happen at every level in the stack where there is a buffer. For each connection, the receiving side contains these buffers:

• OS buffers: tuning them is an advanced optimization.

• StreamReader bytes buffer: the default limit is 64kB. You can set another limit by passing a read_limit keyword argument to connect() or serve().

• Incoming messages Queue: its size depends both on the size and the number of messages it contains. By default the maximum UTF-8 encoded size is 1MB and the maximum number is 32. In the worst case, after UTF-8 decoding, a single message could take up to 4MB of memory and the overall memory consumption could reach 128MB. You should adjust these limits by setting the max_size and max_queue keyword arguments of connect() or serve() according to your application’s requirements.

For each connection, the sending side contains these buffers:

• StreamWriter bytes buffer: the default size is 64kB. You can set another limit by passing a write_limit keyword argument to connect() or serve().

• OS buffers: tuning them is an advanced optimization.

Concurrency

Calling any combination of recv(), send(), close() ping(), or pong() concurrently is safe, including multiple calls to the same method.

As shown above, receiving frames is independent from sending frames. That isolates recv(), which receives frames, from the other methods, which send frames.

While recv() supports being called multiple times concurrently, this is unlikely to be useful: when multiple callers are waiting for the next message, exactly one of them will get it, but there is no guarantee about which one.

Methods that send frames also support concurrent calls. While the connection is open, each frame is sent with a single write. Combined with the concurrency model of asyncio, this enforces serialization. After the connection is closed, sending a frame raises ConnectionClosed.