mlx5dv_wr_set_mkey_crypto(3) | mlx5dv_wr_set_mkey_crypto(3) |
mlx5dv_wr_set_mkey_crypto - Configure a MKey for crypto operation.
#include <infiniband/mlx5dv.h> static inline void mlx5dv_wr_set_mkey_crypto(struct mlx5dv_qp_ex *mqp,
const struct mlx5dv_crypto_attr *attr);
Configure a MKey with crypto properties. With this, the device will encrypt/decrypt data when transmitting data from memory to network and when receiving data from network to memory.
In order to configure MKey with crypto properties, the MKey should be created with MLX5DV_MKEY_INIT_ATTR_FLAGS_CRYPTO. MKey that was created with MLX5DV_MKEY_INIT_ATTR_FLAGS_CRYPTO must have crypto properties configured to it before it can be used, i.e. this setter must be called before the MKey can be used or else traffic will fail, generating a CQE with error. A call to this setter on a MKey that already has crypto properties configured to it will override existing crypto properties.
Configuring crypto properties to a MKey is done by specifying the crypto standard that should be used and its attributes, and also by providing the Data Encryption Key (DEK) to be used for the encryption/decryption itself.
The MKey represents a virtually contiguous memory, by configuring a layout to it. The crypto properties of the MKey describe whether data in this virtually contiguous memory is encrypted or in plaintext, and whether it should be encrypted/decrypted before transmitting it or after receiving it. Depending on the actual operation that happens (TX or RX), the device will do the “right thing” based on the crypto properties configured in the MKey.
MKeys can be configured with both crypto and signature properties at the same time by calling both mlx5dv_wr_set_mkey_crypto()(3) and mlx5dv_wr_set_mkey_sig_block()(3). In this case, both crypto and signature operations will be performed according to the crypto and signature properties configured in the MKey, and the order of operations will be determined by the signature_crypto_order property.
Memory signature domain is not configured, and memory data is encrypted.
Wire signature domain is not configured, and wire data is in plaintext.
encrypt_on_tx is set to false, and because signature is not configured, signature_crypto_order value doesn’t matter.
A SEND is issued using the MKey as a local key.
Result: device will gather the encrypted data from the MKey (using whatever layout configured to the MKey to locate the actual memory), decrypt it using the supplied DEK and transmit the decrypted data to the wire.
Same as above, but a RECV is issued with the same MKey, and RX happens.
Result: device will receive the data from the wire, encrypt it using the supplied DEK and scatter it to the MKey (using whatever layout configured to the MKey to locate the actual memory).
Memory signature domain is configured for no signature, and memory data is in plaintext.
Wire signature domain is configured for T10DIF every 512 Bytes block, and wire data (including the T10DIF) is encrypted.
encrypt_on_tx is set to true and signature_crypto_order is set to be MLX5DV_SIGNATURE_CRYPTO_ORDER_SIGNATURE_BEFORE_CRYPTO_ON_TX. data_unit_size is set to MLX5DV_BLOCK_SIZE_520.
The MKey is sent to a remote node that issues a RDMA_READ to this MKey.
Result: device will gather the data from the MKey (using whatever layout configured to the MKey to locate the actual memory), generate an additional T10DIF field every 512B of data, encrypt the data and the newly generated T10DIF field using the supplied DEK, and transmit it to the wire.
Same as above, but remote node issues a RDMA_WRITE to this MKey.
Result: device will receive the data from the wire, decrypt the data using the supplied DEK, validate each T10DIF field against the previous 512B of data, strip the T10DIF field, and scatter the data alone to the MKey (using whatever layout configured to the MKey to locate the actual memory).
Crypto attributes describe the format (encrypted or plaintext) and layout of the input and output data in memory and wire domains, the crypto standard that should be used and its attributes.
struct mlx5dv_crypto_attr {
enum mlx5dv_crypto_standard crypto_standard;
bool encrypt_on_tx;
enum mlx5dv_signature_crypto_order signature_crypto_order;
enum mlx5dv_block_size data_unit_size;
char initial_tweak[16];
struct mlx5dv_dek *dek;
char keytag[8];
uint64_t comp_mask; };
Table: signature_crypto_order and encrypt_on_tx Meaning.
The table describes the possible data layouts in memory and wire domains, and the order in which crypto and signature operations are performed according to signature_crypto_order, encrypt_on_tx and signature configuration.
Memory column represents the data layout in the memory domain.
Wire column represents the data layout in the wire domain.
There are three possible operations that can be performed by the device on the data when processing it from memory to wire and from wire to memory:
Op1, Op2 and Op3 columns represent these operations. On TX, Op1, Op2 and Op3 are performed on memory data to produce the data layout that is specified in Wire column. On RX, Op3, Op2 and Op1 are performed on wire data to produce the data layout specified in Memory column. “SIG.mem” and “SIG.wire” represent the signature operation that is performed in memory and wire domains respectively. None means no operation is performed. The exact signature operations are determined by the signature attributes configured by mlx5dv_wr_set_mkey_sig_block().
encrypt_on_tx and signature_crypto_order columns represent the values that encrypt_on_tx and signature_crypto_order should have in order to achieve such behavior.
Memory | Op1 | Op2 | Op3 | Wire | encrypt_on_tx | signature_crypto_order | |
A | data | Encrypt on TX | SIG.mem = none | SIG.wire = none | enc(data) | True | Doesn’t matter |
B | data | Encrypt On TX | SIG.mem = none | SIG.wire = SIG | enc(data)+SIG | True | SIGNATURE_AFTER_CRYPTO_ON_TX |
C | data | SIG.mem = none | SIG.wire = SIG | Encrypt on TX | enc(data+SIG) | True | SIGNATURE_BEFORE_CRYPTO_ON_TX |
D | data+SIG | SIG.mem = SIG | SIG.wire = none | Encrypt on TX | enc(data) | True | SIGNATURE_BEFORE_CRYPTO_ON_TX |
E | data+SIG1 | SIG.mem = SIG1 | SIG.wire = SIG2 | Encrypt on TX | enc(data+SIG2) | True | SIGNATURE_BEFORE_CRYPTO_ON_TX |
F | enc(data) | Decrypt on TX | SIG.mem = none | SIG.wire = none | data | False | Doesn’t matter |
G | enc(data) | Decrypt on TX | SIG.mem = none | SIG.wire = SIG | data+SIG | False | SIGNATURE_AFTER_CRYPTO_ON_TX |
H | enc(data+SIG) | Decrypt on TX | SIG.mem = SIG | SIG.wire = none | data | False | SIGNATURE_AFTER_CRYPTO_ON_TX |
I | enc(data+SIG1) | Decrypt on TX | SIG.mem = SIG1 | SIG.wire = SIG2 | data+SIG2 | False | SIGNATURE_AFTER_CRYPTO_ON_TX |
J | enc(data)+SIG | SIG.mem = SIG | SIG.wire = none | Decrypt on TX | data | False | SIGNATURE_BEFORE_CRYPTO_ON_TX |
Notes:
This function does not return a value.
In case of error, user will be notified later when completing the DV WRs chain.
MKey must be created with MLX5DV_MKEY_INIT_ATTR_FLAGS_CRYPTO flag.
The last operation posted on the supplied QP should be mlx5dv_wr_mkey_configure(3), or one of its related setters, and the operation must still be open (no doorbell issued).
In case of ibv_wr_complete() failure or calling to ibv_wr_abort(), the MKey may be left in an unknown state. The next configuration of it should not assume any previous state of the MKey, i.e. signature/crypto should be re-configured or reset, as required. For example, assuming mlx5dv_wr_set_mkey_sig_block() and then ibv_wr_abort() were called, then on the next configuration of the MKey, if signature is not needed, it should be reset using MLX5DV_MKEY_CONF_FLAG_RESET_SIG_ATTR.
When configuring a MKey with AES-XTS crypto offload, and using the former for traffic (send/receive), the amount of data to send/receive must meet one of the following conditions for successful encryption/decryption process (per AES-XTS spec):
Let’s refer to the amount of data to send/receive as `job_size' 1.job_size % data_unit_size == 0 2.(job_size % 16 == 0) && (job_size % data_unit_size <= data_unit_size - 16)
For example: When data_unit_size = 512B: 1. job_size = 512B is valid (1 holds). 2. job_size = 128B is valid (2 holds). 3. job_size = 47B is invalid (neither 1 nor 2 holds).
When data_unit_size = 520B: 1. job_size = 520B is valid (1 holds). 2. job_size = 496B is valid (2 holds). 3. job_size = 512B is invalid (neither 1 nor 2 holds).
mlx5dv_wr_mkey_configure(3), mlx5dv_wr_set_mkey_sig_block(3), mlx5dv_create_mkey(3), mlx5dv_destroy_mkey(3), mlx5dv_crypto_login(3), mlx5dv_crypto_login_create(3), mlx5dv_dek_create(3)
Oren Duer <oren@nvidia.com>
Avihai Horon <avihaih@nvidia.com>
Maher Sanalla <msanalla@nvidia.com>