Beyond supporting streams of data (SOCK_STREAM) or datagrams (SOCK_DGRAM), POSIX sockets have additional features not accessible via send(2) and recv(2). These features include things like scatter/gather I/O, duplicating file descriptors into other processes, and accessing out-of-band data.
Twisted includes a wrapper around the two C APIs which make these things possible, sendmsg and recvmsg . This document covers their usage. It is intended for Twisted maintainers. Application developers looking for this functionality should look for the high-level APIs Twisted provides on top of these wrappers.
sendmsg(2) exposes nearly all sender-side functionality of a
socket. For a SOCK_STREAM socket, it can send bytes that become part of
the stream of data being carried over the connection. For a SOCK_DGRAM
socket, it can send bytes that become datagrams sent from the socket. It
can send data from multiple memory locations (gather I/O). Over AF_UNIX
sockets, it can copy file descriptors into whichever process is receiving
on the other side. The wrapper included in Twisted,
send1msg , exposes
many (but not all) of these features. This document covers the usage of
the features it does expose. The alternate spelling for the wrapper is
used to indicate the primary limitation, which is it that the interface
supports sending only one iovec at a time.
Likewise, recvmsg(2) exposes nearly all the receiver-side
functionality of a socket. It can receive stream data over from a
SOCK_STREAM socket or datagrams from a SOCK_DGRAM socket. It can receive
that data into multiple memory locations (scatter I/O), and it can receive
those copied file descriptors. The wrapper included in
Twisted, recv1msg ,
exposes many (but not all) of these features. This document covers the
usage of the features it does expose. The alternate spelling for the
wrapper is used to indicate the primary limitation, which is that the
interface supports receiving only one iovec at a time.
sendmsg can be used in a way which makes it equivalent to using the send call. The first argument to sendmsg is (in this case and all others) a file descriptor over which to send the data. The second argument is a string giving the data to send.
On the other end, recvmsg can be used to replace a recv call. The first argument to recvmsg is (again, in all cases) a file descriptor over which to receive the data. The second argument is an integer giving the maximum number of data to receive.
# Copyright (c) Twisted Matrix Laboratories.
# See LICENSE for details.
"""
Demonstration of sending bytes over a TCP connection using sendmsg.
"""
from socket import socketpair
from twisted.python.sendmsg import send1msg, recv1msg
def main():
foo, bar = socketpair()
sent = send1msg(foo.fileno(), "Hello, world")
print "Sent", sent, "bytes"
(received, flags, ancillary) = recv1msg(bar.fileno(), 1024)
print "Received", repr(received)
print "Extra stuff, boring in this case", flags, ancillary
if __name__ == '__main__':
main()
Used with an AF_UNIX socket, sendmsg send a copy of a file descriptor into whatever process is receiving on the other end of the socket. This is done using the ancillary data argument. Ancillary data consists of a list of three-tuples. A three-tuple constructed with SOL_SOCKET, SCM_RIGHTS, and a platform-endian packed file descriptor number will copy that file descriptor.
File descriptors copied this way must be received using a recvmsg call. No special arguments are required to receive these descriptors. They will appear, encoded as a native-order string, in the ancillary data list returned by recvmsg.
# Copyright (c) Twisted Matrix Laboratories.
# See LICENSE for details.
"""
Demonstration of copying a file descriptor over an AF_UNIX connection using
sendmsg.
"""
from os import pipe, read, write
from socket import SOL_SOCKET, socketpair
from struct import unpack, pack
from twisted.python.sendmsg import SCM_RIGHTS, send1msg, recv1msg
def main():
foo, bar = socketpair()
reader, writer = pipe()
# Send a copy of the descriptor. Notice that there must be at least one
# byte of normal data passed in.
sent = send1msg(
foo.fileno(), "\x00", 0,
[(SOL_SOCKET, SCM_RIGHTS, pack("i", reader))])
# Receive the copy, including that one byte of normal data.
data, flags, ancillary = recv1msg(bar.fileno(), 1024)
duplicate = unpack("i", ancillary[0][2])[0]
# Demonstrate that the copy works just like the original
write(writer, "Hello, world")
print "Read from original (%d): %r" % (reader, read(reader, 6))
print "Read from duplicate (%d): %r" % (duplicate, read(duplicate, 6))
if __name__ == '__main__':
main()