Encoding

The LDAP RFCs states that strings sent and received by the LDAP server must be encoded with the utf-8 encoding. Data in ldap3 flow from different sources to different tergets, so they can have different encodings. There are 6 different flows in the ldap3 library:

  • data coming from the server

  • data going to the server

  • data coming from the user

  • data going to the user

  • data going to output logs

  • data coming from flaky servers that return the DN or the schema in not utf-8 encoding.

Server data flow

The LDAP protocol stores strings in a Directory String type that should always be in utf-8 and it’s stored in the DEFAULT_SERVER_ENCODING config parameter. For flaky server this encoding can be changed with:

>>> from ldap3 import set_config_parameter
>>> set_config_parameter('DEFAULT_SERVER_ENCODING', 'latin-1')

DEFAULT_SERVER_ENCODING can be changed multiple times as needed. To know the current setting you can use the following code:

>>> from ldap3 import get_config_parameter
>>> get_config_parameter('DEFAULT_SERVER_ENCODING')
'utf-8'

Some servers don’t completely follow the LDAP RFCs and send data in a different encoding or in a mix of encodings. For example Active Directory can send the DN of entries found in a search in a different encoding than utf-8. In this case you can use the ADDITIONAL_SERVER_ENCODINGS config parameter to decode the DN of the Search operation response. It can be set to one encoding or a list of encodings. If a list of encodings is provided ldap3 tries sequentially each encoding until a valid decode is performed. If any of the specified encodings is able to decode the value then an UnicodeError exception is raised.

ADDITIONAL_SERVER_ENCODINGS defaults to ['latin1, 'koi8-r'] for european and russian encodings

User data flow

ldap3 can receive data from the user in 4 different ways:

  • data input during a program execution

  • data stored in the source files

  • data typed at the Python interpreter (the usual >>>)

  • data input from another program via stdin

When dealing with user input the encoding can be (and probably is) different from utf-8, so ldap3 tries to guess what encoding is used and store it in the DEFAULT_CLIENT_ENCODING parameter. ldap3 uses the stdin.encoding if present, else the sys.getdefaultencoding() and if neither is present sets DEFAULT_CLIENT_ENCODING to utf-8.

You can set a specific DEFAULT_CLIENT_ENCODING with the following code:

>>> from ldap3 import set_config_parameter
>>> set_config_parameter('DEFAULT_CLIENT_ENCODING', 'my_encoding')

DEFAULT_CLIENT_ENCODING can be changed multiple times as needed. To know the current setting you can use the following code:

>>> from ldap3 import get_config_parameter
>>> get_config_parameter('DEFAULT_CLIENT_ENCODING')
'utf-8'

This parameter is used when the auto_encode and auto_escape parameters of the Connection object are set to True.

ldap3 can send data from the user in 3 different ways:

  • data printed at the stdout

  • data printed at the console

  • data printed in the log files

Data printed at stdout and at the console (for example representation of ldap3 Server and Connection objects) are encoded with stdout.encoding if present, else the ascii encoding is used. If ldap3 is unable to properly represents the data with the selected encoding then it uses a hexadecimal representation of the byte value of the data. This ensure that there are no error and no missing data at the REPL or console level. This can lead to some differences in output between Python 3 and Python 2.

Log file data flow

Data printed in the log files are always encoded in ascii with a backslashreplace failback in case of unprintable ascii values. This should assure that the log is written without any decoding error.

Raw data in search response

You can always access the raw byte values of attributes returned in a search accessing the raw_attributes key of a response entry or the raw_values property of an Entry object in the Abstraction Layer.