MPC-Mimicking Neural Network Based on Homomorphic Encryption

This paper showcases the use of homomorphic encryption (HE) scheme for securing process data during the controller evaluation in a simulated untrusted cloud environment. The controller implemented in this work is a neural network (NN) that mimics a model predictive controller (MPC) designed for disturbance rejection. Firstly, an MPC was designed for a process of biochemical reactor. From obtained MPC control data, a neural network (NN-MPC) with fully connected layers was trained. Multiple HE-friendly activation functions were tested during the NN training and testing, and based on the results, a polynomial approximation of hyperbolic tangent was used. Subsequently, the NN-MPC controller was implemented in encrypted control scenario. The measured states of the biochemical reactor were encrypted on the side of the process and sent for the homomorphic evaluation to the simulated cloud (NN-MPC).


I. INTRODUCTION
With control algorithms' growing computational complexity and processing data size, the demand for cloud-based services has become more relevant in recent years. Computing on the cloud can be very beneficial for the data owner since it takes away the computational and data storage demands and outsources them to other parties. However, these benefits come with risks regarding the security and privacy of the data. Most encryption standards, such as the AES and RSA, focus on data security during the transfer. These data are decrypted on the cloud computer before they are processed, raising the concern of data privacy violations. One plausible solution is to use homomorphic encryption (HE), allowing the outsourced algorithms to process the data in encrypted form.
The literature provides numerous examples of HE-enabled control setups [1] with many open challenges [2]. HE is being implemented in various control applications, using polynomial controllers [3], linear feedback [4], and non-linear controllers [5]. Some more advanced control scenarios, like implicit and explicit MPC, were showcased in [6], [7].
In the past decade, several new cryptosystems emerged, primarily based on the ring version of the Learning with Errors (LWE/RLWE) problem [8]. In this paper, we use one of them, the CKKS cryptosystem.
With the security and data-privacy benefits of HE schemes also come some drawbacks. In HE, only a limited set of operations is usually available (mainly addition and multiplication).
Also, the computational complexity and memory demands are much higher than for unencrypted applications. These facts pose a significant challenge in implementing complex controllers like MPC, requiring optimization over encrypted data. These problems can be tackled by approximating control law with more HE-friendly constructs such as neural networks (NN). In [9], a linear MPC was implemented as a nonpolynomial max-out neural network with a single hidden layer.
This paper presents a NN approximation of linear MPC (NN-MPC) designed for state disturbance rejection to control the biochemical reactor model with input constraints. Used NN-MPC consists of 3 fully connected hidden layers (50 neurons each) and an output layer. The inference of NN-MPC is performed over encrypted state measurements of the process, yielding an encrypted control action.
II. CKKS CRYPTOSYSTEM CKKS (Cheon-Kim-Kim-Song) cryptosystem is an RLWEbased scheme that operates with approximated fixed-point arithmetic, originally published under the name "Homomorphic Encryption for Arithmetic of Approximate Numbers" (HEAAN) [10]. CKKS is one of the most implemented schemes in modern HE, included in libraries and frameworks such as OpenFHE 1 , Palisade 2 , HElib 3 , Lattigo 4 , and Microsoft SEAL 5 . It classifies as leveled HE scheme, meaning that additions and multiplication between two encrypted messages are possible, but the number of consecutive multiplications is limited. This limitation comes from a noise (additive error) incorporated into encrypted messages to make them cryptographically secure, which is the main idea behind the hardness assumption of RLWE [8].
In the following sections II-A-II-D, we describe the basic principles of the CKKS cryptosystem. Detailed inner workings of the mathematics behind the scheme can be found in [10], [11].

A. Notation for Message, Plaintext and Ciphertext Spaces
In this paper, we use the following notation and terms. The CKKS operates over polynomial ring where q is a coefficient modulus, and (X N + 1), for N being a power of two, represents irreducible cyclotomic polynomial (basically the equivalent of modulus in polynomial space). N being a polynomial modulus degree defines a maximum size of polynomials in the ring R q to be (N − 1).
The raw input message m ∈ C N 2 is a vector of N 2 complex numbers, of which usually only real parts are used in practical applications. Plaintext m ′ ∈ R q is a polynomial with N coefficients, obtained by encoding procedure E : q is a tuple of two polynomials obtained by encryption procedure E : m ′ ∈ R q → c ∈ R 2 q , described in section II-C.

B. Encoding and Decoding
In the CKKS, the encoding function E(·) is defined as where a complex canonical embedding function π(·) maps the coefficients of polynomial m ′ into the elements of m. Symbol ∆ represents a scaling factor that multiplies polynomial coefficients to move the bits of encoded message to the left, creating a space for addition of cryptographic noise during the encryption without the significant loss of numerical precision. The mathematical details of embedding and encoding are provided in [10, section 2.2 and 3.2]. The decoding procedure D : m ′ ∈ R q → m ∈ C N 2 is just an inverse of (1), defined as It is clear from (1) that rounding function [·] will result in only approximated decoding of original message D (E(m, ∆), ∆) ≈ m, hence the name of the scheme "Homomorphic Encryption for Arithmetic of Approximate Numbers".

C. Key Generation, Encryption and Decryption
CKKS is an asymmetric scheme. One party generates a secret key and a set of public keys for distribution. A secret polynomial s ∈ R with signed binary coefficients {−1, 0, 1} is sampled from a distribution HWT (h) described in [10, section 3.4]. In practical implementation, the secret key is a tuple sk = (1, s). The public key pk = (pk 1 , pk 2 ) is a tuple of two polynomials pk1 = [−a · sk + e]q, where polynomial a is uniformly sampled from R q and e ←֓ X is an error polynomial with coefficients sampled from discrete Gaussian distribution X (see [10, section 2.3]). The encryption of plaintext m ′ is done by using a public key pk and forming a ciphertext c = (c 1 , c 2 ) ∈ R 2 q such that where u ∈ R q is a random polynomial and e 1 , e 2 are error polynomials from X . Decryption of c is performed by evaluating a ciphertext over secret polynomial s to obtain a plaintext with approximationm ′ of originally encrypted m ′ .

D. Homomorphic Operations
CKKS allows operations to be carried out between two ciphertext or between plaintext and ciphertext. In fact, the operations are just a standard polynomial algebra over the ring R q . Considering two ciphertexts c a = (c a,1 , c a,2 ) and c b = (c b,1 , c b,2 ) with the same value of the modulus q, we can calculate the product of their homomorphic addition c add as The homomorphic multiplication between ciphertext c and plaintext k ′ gives a new ciphertext and for two ciphertexts, multiplication between c a and c b is performed as It is clear that by multiplying two ciphertext, the result c cmul will grow in size. Therefore, it is necessary to perform a relinearization where rk is a re-linearization key and p is a big integer. One of the main features of CKKS is message batching, which allows inclusion of multiple message elements into the slots of single plaintext or ciphertext. For polynomial modulus degree N (usually in thousands), N/2 individual numbers can be included in one plaintext/ciphertext. All the slots are evaluated simultaneously during one homomorphic operation, bringing the considerable potential for applications where parallelism is desired. Additionally, CKKS implementations in HE frameworks also provide a vector rotation technique. During this procedure, plaintext/ciphertext slots are shifted and wrapped around (either to the left or right). This technique is used to implement homomorphic multiplication between plaintext matrices and ciphertext vectors, which is especially useful in evaluating layers in NNs. The main drawback of HE vector rotation is that it requires a set of cryptographic keys (Gallois keys) that are large.

A. Process Description
We consider a biochemical reactor with an alcoholic fermentation process. During the reactor operation, microorganisms convert saccharides such as fructose or glucose into ethanol and carbon dioxide. This complex biochemical transformation creates many by-products. However, the overall reaction can be summarized as follows: Besides saccharides concentration, the temperature's influence must be considered to maintain ideal microorganisms' growth. From [12] optimal temperature for the presented yeast growth rate was determined to be between 28 • C − 32 • C. As the aeration inhibits saccharide consumption, the considered model includes oxygen concentration as one of the state variables. The biochemical reactor is modeled as a continuous stirred tank reactor Fig. 1. The tank is fed with a constant glucose solution F r = 25 l/h, feeding the biomass (suspension of yeasts) and producing the continuous outlet flow of ethanol. The processes in the reactor are described with six differential equations obtained along with their parameters from [13].

B. Model Predictive Control for Disturbance Rejection
The presented model predictive control (MPC) takes the form of output steady-state control with discrete time state space model and ∆u penalization: where N is finite prediction horizon, Q ⪰ 0, R ⪰ 0 are penalty matrices. Vectors x k ∈ R nx , u k ∈ R nu , y k ∈ R ny represents system state, input, and controlled output predictions for define step k, respectively. Model presented includes state matrix A d ∈ R nx×nx , input matrix B d ∈ R nx×nu , and output matrix C d ∈ R nx×ny . X ∈ R nx , U ∈ R nu , U du ∈ R nu are polyhedral sets of constraints for states, control input and change of the control input respectively. The linear discrete time model Eq. (18) was derived from the non-linear model of the biochemical reactor presented in [13]. System states include [c x , c p , c s , c O2 , T r , T c ] ⊤ , biomass, ethanol, glucose, dissolved oxygen concentrations, reactor temperature, and the coolant's temperature subsequently. The state-space matrices are formulated as follows:

C. Neural Network Approximation
The main drawback of RLWE-based HE schemes is that they are not usable for evaluating complex algorithms such as MPC. Therefore, we decided to approximate the MPC controller by a neural network (NN-MPC) with an appropriate structure that would allow us to perform a homomorphic inference over encrypted states to compute an encrypted control action. To obtain a good approximation of the MPC controller (17-23), we set up a series of experiments (66 in total) with different initial state conditions, generated by quantizing the state space, and performed MPC control to steady state. Overall 9823 data samples were obtained, containing six state variables and corresponding MPC control action for each sample. Before the NN training, the state variables were normalized. The data samples were then randomly shuffled, and 70% was used as a training set for NN-MPC. The remaining samples were split into two sets of the same size and used as validation and testing data. The NN was trained in Python deep learning API Keras, using an Adam Optimizer. The learning rate was set to 2.5 × 10 −4 , and we used a mean square error as a loss function. The number of training epochs was set to 2500.
With respect to the maximum multiplicative depth of the CKKS setup (Fig. 2), we have chosen a neural network with three fully connected layers, 50 neurons each, and a linear output layer. The main limitation of HE schemes is that  [14]. These are polynomial approximations of hyperbolic tangent (tanh), sigmoid, ReLU, and a commonly used HE-friendly square function. Table I compares four approximated activations along with classical tanh, sigmoid, and ReLU. In this comparison, we focused on the root mean square error (RMSE) value between NN testing data and NN prediction, the number of strict and non-strict constraint violations, the RMSE of those violations, and the HE multiplicative depth of a function. The results show that all NNs with approximated activation functions provide worse prediction performance than NNs with standard activations. From all the activations implementable in HE, the smallest prediction error was achieved by approximated tanh function (p-approx. tanh). Figure 3 shows the prediction performance of trained NN-MPC using a p-approx. tanh as an activation function. It is clear that control action occasionally violates the constraints, however, these violations are just minor with RMSE value of 1.352 for upper and 1.042 for lower constraints, and overall RMSE prediction error only 0.769 (Tab. I).

D. Homomorphic Multiplicative Depth of NN
In RLWE-based HE schemes, one of the most critical aspects of computation is the depth of the arithmetic circuit. Every ciphertext contains cryptographic noise (in the case of CKKS, also an approximation error) that grows in size with the number of operations being carried out over it. This will eventually lead to the corruption of encrypted data if noise reaches a threshold known as noise budget. To avoid the exponential growth of error during the ciphertext multiplication, CKKS employs a modulus-switching technique called rescaling. However, this procedure can be performed a limited number of times and is dictated by the number of inner primes Γ i in coefficient moduli array (Sec. III-E).
The structure of NN used in this work (Fig. 2) contains a chain of consecutive ciphertext multiplications called multiplicative depth. For each hidden NN layer, the plaintext matrix of weights is homomorphically multiplied by an encrypted vector of states. The output layer also requires one ciphertext multiplication. In CKKS, the plaintext polynomial evaluation over ciphertext is of depth n − 1, for n being an order of the polynomial. In the case of p-approx. tanh activation function, the multiplicative depth is two. Overall the depth of NN is 10.

E. Setup of Cryptographic Parameters
Several essential parameters (Table II) are considered when setting up the cryptosystem. First is polynomial modulus degree N . Bigger N increases the security of the scheme but, also increases the computational complexity. Simultaneously, larger N allows for a higher multiplicative depth of the arithmetic circuit. This is done by selecting an array of bit sizes for coefficient moduli The difference between the outer primes Γ o and inner primes Γ i (in bits) defines the bit precision of integer parts of encoded/encrypted numbers (in our case 17 bits). The number of inner primes Γ i (10) defines the maximum multiplicative depth. The scale ∆, usually chosen to be the same value as Γ i , controls numerical precision after the decimal point of encoded/encrypted numbers, such that the precision is roughly the difference between the inner prime bit size and precision before the decimal point (in our case 33 − 17 = 16 bits).

F. Control over Encrypted Data
The control setup consists of three environments. First is MATLAB, where numerical simulation of the biochemical reactor is performed using ode23s solver. MATLAB communicates with an encryption/decryption layer written in Python, where measured states of the process are sent to be encrypted, and encrypted control actions from NN-MPC are decrypted before they are sent back to the process. This layer is considered a part of the trusted environment on the side of the process. The NN-MPC controller is evaluated in a separate server-side Python script, simulating a cloud environment. The encryption/decryption layer and NN-MPC communicate via HTTP. To implement operations over encrypted data, we use Python library TenSEAL 6 [15] based on Microsoft SEAL [16]. The general algorithm of closed-loop control over encrypted data is shown in Alg. 1. Firstly, the data owner (process) generates cryptographic keys. These are public key pk, secret key sk, Galois key gk, and re-linearization key rk. During the key exchange, pk, gk, rk are sent to the controller and later used for homomorphic evaluation. During the control, 6 https://github.com/OpenMined/TenSEAL Process (process side -data owner) 8 Measure states → x k NN-MPC Controller (cloud environmet) 12 Evaluate → u E k = NN(x E k , pk, gk, rk) 13 Process (process side -data owner) 14 Decrypt Apply → u k a vector of state measurements is encoded with the scale ∆ into plaintext and encrypted using sk. Encrypted states x E k are then sent to the NN-MPC controller to be evaluated. Public key pk is used in every homomorphic operation (additions, multiplications), rk is required in multiplication between two ciphertexts (evaluation of activation functions), and gk is used for matrix multiplications. The resulting encrypted control u E k is then sent back to the process, decrypted using sk, decoded, and applied to the process.
The biochemical reactor was controlled on a simulated timespan of 150 hours with a control sampling period of 30 minutes for original MPC, NN-MPC, and NN-MPC over encrypted data. To illustrate the disturbance rejection performance of the controllers, we initiated control in an arbitrary combination of states x init = [0.81, 11.82, 31.50, 6.15, 28.06, 25.12]. Afterward, two additional impulse disturbances on process input (coolant flow) were applied in time 100h (F c = 70) and 125h (F c = 0). The results are shown in figure 4. Both the original MPC (blue line) and NN-MPC (orange line) were able to compensate for disturbances and bring the process states to the desired operating point. MPC provides a faster response and shorter settling time than NN-MPC. As a result of a rather large initial state disturbance, the NN-MPC tent to slightly violate the bottom input constraint. However, this violation did not apply since the inputs are trimmed before being sent to the process. The NN-MPC control over encrypted data (black line) is almost identical to plain NN-MPC, with minor numerical discrepancy caused by cryptographic noise and rounding errors introduced during message encoding.

G. Limitations
While the primary goal of HE-based process control (the ability to compute over secured data) was achieved, several limitations must be considered. The RLWE-based cryptosystems tend to be computationally demanding. In the presented control scenario, a single HE inference of the NN-MPC controller took on average 10.11 seconds (AMD Ryzen 3950X CPU, 128GB DRR4 RAM). The per-layer computational time grows quadratically with the number of neurons (size of weight matrices). While the complexity grows only linearly with an increasing number of layers, more than three hidden layers would require higher multiplicative depth, thus bigger polynomial modulus degree N . This would lead to an impractical cryptosystem setup due to the computational demand and size of the keys. For the CKKS setup used in this work (Table II), the size of the public key was 1.97MB, the secret key 1.01MB, the re-linearization key 21.69MB, and Galois keys 564.81MB.

IV. CONCLUSIONS
This paper shows that even complex controllers like MPC can be implemented in an approximated form on HE frameworks if the implementer is willing to sacrifice some of the original control performance and numerical precision. The main benefit, i.e., the preservation of data privacy, comes with the cost of increased computational and memory demand that can vary based on the setup of the cryptosystem. The practicality of such a setup depends on a specific application. The homomorphically evaluated NN-MPC of the same structure as the one presented in this paper would not be implementable for controlling processes with fast dynamics. However, the presented approach is viable for slow processes like the biochemical reactor. One of the options for reducing both the computational time and size of the transferred data would be decreasing the multiplicative depth of NN. This can be done either by decreasing the number of layers and/or using a square function as activation. This would allow for N to be 4096 or even 2048, reducing the computational overhead significantly at the cost of creating a bigger discrepancy between MPC and NN-MPC. The computational time can also be reduced by decreasing the number of neurons in layers, resulting in homomorphic multiplication between smaller matrices and vectors.