Beyond supporting streams of data (SOCK_STREAM) or datagrams (SOCK_DGRAM), POSIX sockets have additional features not accessible via send(2) and recv(2). These features include things like scatter/gather I/O, duplicating file descriptors into other processes, and accessing out-of-band data.
Twisted includes a wrapper around the two C APIs which make these things possible, sendmsg and recvmsg . This document covers their usage. It is intended for Twisted maintainers. Application developers looking for this functionality should look for the high-level APIs Twisted provides on top of these wrappers.
sendmsg(2) exposes nearly all sender-side functionality of a socket.
For a SOCK_STREAM socket, it can send bytes that become part of the stream of data being carried over the connection.
For a SOCK_DGRAM socket, it can send bytes that become datagrams sent from the socket.
It can send data from multiple memory locations (gather I/O).
Over AF_UNIX sockets, it can copy file descriptors into whichever process is receiving on the other side.
The wrapper included in Twisted, sendmsg, exposes many (but not all) of these features.
This document covers the usage of the features it does expose.
The primary limitation of this wrapper is that the interface supports sending only one iovec at a time.
Likewise, recvmsg(2) exposes nearly all the receiver-side functionality of a socket.
It can receive stream data over from a SOCK_STREAM socket or datagrams from a SOCK_DGRAM socket.
It can receive that data into multiple memory locations (scatter I/O), and it can receive those copied file descriptors.
The wrapper included in Twisted, recvmsg, exposes many (but not all) of these features.
This document covers the usage of the features it does expose.
The primary limitation of this wrapper is that the interface supports receiving only one iovec at a time.
sendmsg can be used in a way which makes it equivalent to using the send call. The first argument to sendmsg is (in this case and all others) a socket over which to send the data. The second argument is a bytestring giving the data to send.
On the other end, recvmsg can be used to replace a recv call. The first argument to recvmsg is (again, in all cases) a socket over which to receive the data. The second argument is an integer giving the maximum number of bytes of data to receive.
# Copyright (c) Twisted Matrix Laboratories.
# See LICENSE for details.
"""
Demonstration of sending bytes over a TCP connection using sendmsg.
"""
from __future__ import print_function
from socket import socketpair
from twisted.python.sendmsg import sendmsg, recvmsg
def main():
foo, bar = socketpair()
sent = sendmsg(foo, b"Hello, world")
print("Sent", sent, "bytes")
(received, ancillary, flags) = recvmsg(bar, 1024)
print("Received", repr(received))
print("Extra stuff, boring in this case", flags, ancillary)
if __name__ == '__main__':
main()
Used with an AF_UNIX socket, sendmsg send a copy of a file descriptor into whatever process is receiving on the other end of the socket. This is done using the ancillary data argument. Ancillary data consists of a list of three-tuples. A three-tuple constructed with SOL_SOCKET, SCM_RIGHTS, and a platform-endian packed file descriptor number will copy that file descriptor.
File descriptors copied this way must be received using a recvmsg call. No special arguments are required to receive these descriptors. They will appear, encoded as a native-order string, in the ancillary data list returned by recvmsg.
# Copyright (c) Twisted Matrix Laboratories.
# See LICENSE for details.
"""
Demonstration of copying a file descriptor over an AF_UNIX connection using
sendmsg.
"""
from __future__ import print_function
from os import pipe, read, write
from socket import SOL_SOCKET, socketpair
from struct import unpack, pack
from twisted.python.sendmsg import SCM_RIGHTS, sendmsg, recvmsg
def main():
foo, bar = socketpair()
reader, writer = pipe()
# Send a copy of the descriptor. Notice that there must be at least one
# byte of normal data passed in.
sent = sendmsg(
foo, b"\x00", [(SOL_SOCKET, SCM_RIGHTS, pack("i", reader))])
# Receive the copy, including that one byte of normal data.
data, ancillary, flags = recvmsg(bar, 1024)
duplicate = unpack("i", ancillary[0][2])[0]
# Demonstrate that the copy works just like the original
write(writer, b"Hello, world")
print("Read from original (%d): %r" % (reader, read(reader, 6)))
print("Read from duplicate (%d): %r" % (duplicate, read(duplicate, 6)))
if __name__ == '__main__':
main()