Maximum Entropy Snapshot Sampling for Reduced Basis Modelling

The so-called maximum entropy snapshot sampling method is employed for reducing two nonlinear circuit models. The maximum entropy snapshot sampling directly reduces the number of snapshots by recursively identifying and selecting the snapshots that strictly increase an estimate of the correlation entropy of the considered systems. Reduced bases are then obtained with the orthogonal-triangular decomposition. In the ﬁrst case study, the resulting overdetermined systems are solved in the least squares sense. In the second case study, the basis is incorporated in a reduced order multirate scheme, whilst the reduction parameter is estimated through an optimality requirement. Numerical experiments verify the performance of the advocated approach, in terms of computational costs and accuracy, relative to an established reduction framework that is based on the singular value decomposition.


I. INTRODUCTION
In manufacturing integrated circuits, a range of design explorations that ensure sound functionality of these components need to be performed. To this end, mathematical models of such circuits are simulated numerically. In a discrete setting, the required simulation times may become prohibitively large, in particular, for large-scale problems. Model reduction strategies arose as a remedy to recover computational feasibility for such problems, in particular, when repetitive computations are required. Here, we apply the maximum entropy snapshot sampling (MESS) method [5] to nonlinear circuit problems, as means to reduced basis model reduction. In this paper, two case studies are presented. In the first case study the commonly employed proper orthogonal decomposition (POD) basis is substituted for a MESS obtained basis in a standard Galerkin projection setting for differential algebraic systems. The comparison against the POD demonstrates the overall performance of the advocated MESS framework. In the second case study, the MESS model order reduction framework is incorporated into a reduced order multirate (ROMR) scheme [1], which is applied to a coupled nonlinear thermal-electric circuit. Furthermore, in the second test case, the MESS method is combined with a maximum likelihood estimation of the parameter that controls the degree of reduction.

A. Maximum Entropy Snapshot Sampling
Let X = (x 1 , x 2 , . . . , x n ) be a finite sequence of numerically obtained states x j ∈ R m at time instances t j ∈ R, with j ∈ {1, 2, . . . , n}, of a diode chain model. Provided the probability distribution p of these states, the second-order Rényi entropy of the sample X is where p j ≡ p(x j ) and E p (p) is the expected value of the probability distribution p with respect to p itself. According to the law of large numbers, in the limit n → ∞, the average of p 1 , p 2 , . . ., p n almost surely converges to their expected value, that is, while each p j can be approximated by the sample's relative frequency of occurrence. By considering a norm * on R m , the notion of occurrence can be translated into a proximity condition. In particular, for each x j ∈ R m define the open ball that is centred at x j and whose radius is > 0, and introduce the characteristic function with values Under the aforementioned considerations, the entropy of X can be estimated bŷ Provided that the limit of the evolution ofĤ (2) p exists and measures the sensitivity of the evolution of the system itself [3, §6.6], a reduced sequence X r = (x j1 , x j2 , . . . , x jr ), with r ≤ n, is sampled from X, by requiring that the entropy of X r is a strictly increasing function of the index k ∈ {1, 2, . . . , r} [6]. A reduced basis is then generated from X r with any orthonormalization process. It has been shown [5] that, depending on the recurrence properties of a system, any such basis guarantees that the Euclidean reconstruction error of each snapshot is bounded from above by , while a similar bound holds true for future snapshots, up to a specific time-horizon.
To estimate the parameter , which determines the degree of reduction within the MESS framework, the following optimisation approach is employed [7]. The quantity within the logarithm in the entropy estimate (5) is often referred to as the sample's correlation sum and can be written as with R ∈ {0, 1} n×n being the matrix whose entries are unity, when x i − x j < , and * 2 F being the Frobenius norm. In terms of probability theory, C is a cumulative distribution function, and hence, its derivative dC /d is the associated probability density function. A commonly justified hypothesis is that the correlation sum scales as D [8, Chapter 1], where D ≥ 0 is the so-called correlation dimension of the manifold that is formed in R m by the terms of X. Under this power law assumption, the maximum likelihood estimate [9,Chapter 8] of the correlation dimension is estimated as follows. We find a sample { i }, with i ∈ [0, 1] for all i ∈ {1, 2, . . . , q}, of a random variable E that is sampled according to C . Then, the probability of finding a sample in To calculate the value for which this expression is maximized, we take the logarithm and note that the maximum is attained when This results in the most likely value D * = −1/ ln E , and can be estimated by * = argmin(|D * − ln C / ln |).
Hence, MESS becomes a parameter-free method.

B. Reduced Order Multirate
When using an ROMR scheme, a semi-explicit DAE is decomposed into fast and slow components, with subscripts F and S, respectively; for instance, with f F , f S , and g F being known functions, and zero indexed quantities indicating known Cauchy data. Here, the fast and slow varying differential variables are x F ∈ R nF and x S ∈ R nS , while the algebraic variables z F ∈ R nA is assumed to be fast, since the dynamics of the DAE is considered to be fast in the time interval of interest. The described type of coupling enables the consideration of electrical circuits with a differential index up to unity, coupled to slower ODE systems.
To reduce the computational effort, a reduced basis that is to be used in a Galerkin projection framework is constructed. This reduction approach is then complemented with the gappy POD method [11]. By utilising a direct projection, the reduced system is guaranteed to be again of unity index. To perform the reduction, let V ∈ R nS×r be a non-square matrix whose columns constitute a reduced basis for the range of the slow varying states, with n S r. The full state x S of the slow subsystem is then approximated by x S ∼ = V x S,r using the reduced basis. Then, the reduced model becomeṡ , while the full state is needed for the coupling, and hence, again the gappy approach combined with a MESS basis is used.
The overall index one system (14)-(16) can be integrated with the L-stable implicit Euler scheme, which automatically assures that the algebraic constraints are not violated for all t > 0. To exploit the fast/slow decomposition, a multirate integration scheme has been proposed [1], which is a reduced order extension of a standard multirate scheme for DAEs [4]. The integration of the coupled system (14)-(16) for one macro-step t k → t k+1 = t k + H can be written as where ∈ {0, 1, . . . , m − 1}, h = H/m is the micro-step size, and the coupling variables are denoted byx F ,z F , and x S . Here, the coupling strategy is chosen to be the coupledslowest-first, as this consistent for DAEs of unity order [1].
First the system is solved for the macro-step. The step size H is chosen so that the solution to the slow subsystem remains sufficiently accurate. Then, the fast solutions x * F,k+1 and z * F,k+1 are not accurate enough and can be discarded, as they will be computed in the last micro-step. In the second stage, the fast solutions are computed for the micro-steps ∈ {0, 1, . . . , m − 1}, using linear interpolation for the valuesx S,k+( +1)/m , based on the available information x S,k and x S,k+1 .

A. The Diode Chain Model
First we will perform a case study regarding the sole application of the MESS reduction method to a diode chain. As an instance of an integrated circuit, consider the diode chain model that is depicted in Fig. 1 and described by the differential-algebraic system [10] where i ∈ {2, 3, . . . , m − 2} with integer m > 3, Φ i is the voltage at the i-th node of the circuit and is measured in V, while the time is measured in ns. The current-voltage diode characteristic function I : R × R → R is defined by where I S = 10 −14 A is the saturation current and α is the inverse of the thermal voltage Φ T = 0.0256 V. Additional model parameters are mentioned in Fig. 1. Further, the excitation voltage is To simulate a transient analysis of the diode chain model depicted in Fig. 1, system (23) is integrated numerically. For large m such simulations become prohibitively expensive in terms of computational time. Here, to recover computational feasibility, reduced basis model reduction techniques are exploited. The MESS method is applied to the nonlinear diode chain model, with m = 40002. The transient analysis is performed in the interval [0, 70] ns, using an implicit Euler scheme with time step ∆t = 0.1 ns. Consistent initial conditions are obtained through a direct current simulation using very small time steps and using a linear increasing input voltage from Φ in = 0 to Φ in = 20. The reduced bases are generated from the high-fidelity matrix X ∈ R m×n , with n = 701, see Fig 2. To benchmark the presented MESS based reduction, a comparison with the POD method is made. The Here, the estimated * value is equal to 0.00525. However, in an attempt to maximally reduce the studied system, is manually selected close to a value that turns out to yield a numerically unstable reduced model. In Fig. 3, the case of the MESS reduced system for = 0.0325 is depicted. There, it is shown that the solution to the MESS reduced system converges to the reference solution. To illustrate that some caution is needed if is selected manually, in Fig. 4, a slightly higher value is chosen, when the resulting reduced model becomes unstable. In Table I, the computational times that are required for generating the bases suggest that the MESS has an advantage in the offline stage. Further, for largescale problems, the SVD becomes infeasible due to memory constraints, whereas this is not the case for MESS, since it relies on recursive evaluations.

B. The Thermal-Electric Circuit
As the ROMR case study circuit needs to contain both coupling and different intrinsic time scales, a thermal-electric test circuit is used [2]. This circuit consists of an operational amplifier, two resistors, a diode, and a capacitor. The thermal resistor R(T ) is modelled by a structure of length d = 0.03m and variable diameter a(x) = a 0 /[1 + b(d − x)x], with x ∈ [0, d], while the material parameters are those of a copper wire. The local resistance exhibits quadratic dependence on the temperature. The local resistance per unit cross-section is thus expressed in Ωm.
Using this expression, the total resistance of the wire is Electrical parameters of one-dimensional resistor Material Cu (copper) Specific resistance r 0 = 1.7 µΩ · m Reference temperature T meas = 291 K Length d = 0.03 m Cross section a 0 = 540 m Thermal parameters of the one-dimensional resistor Density d w = 8.98 · 10 3 kg/m 3 Heat conductivity λ w = 390 W/(mK) Specific heat c w = 385 J/(kgK) The amplifier is a heat source and the diode has a temper- Electrical parameters of the zero-dimensional elements The electric behaviour of the circuit is modelled by modified nodal analysis based on Kirchhoff's laws. The thermal model is nonlinear due to the coupling terms, where the local self-heating term, P w , introduces the nonlinear terms. After discretizing in space, the following thermal-electric system is obtained.

Coupling interfaces
Heat equation The computational cost of coupled network simulations are reduced by applying the ROMR scheme. Here, multirate integration and the MESS are employed for solving the equations that govern the thermal-electric circuit that is depicted in Fig.  5. After partitioning the slow and fast varying time-scales, v(t) problem (14)-(16) becomeṡ Here, x F = u 4 , z F = u 3 , and x S,r = V x S , with x S ∈ R m being the discretized temperature in the thermal resistor.
Problem (30) is integrated with a ROMR method, and the parameter is computed by (10), see Figure 7. To verify the performance of (ROMR, MESS, * ), a transient analysis for the output u 4 is performed. A reference solution is obtained with a standard multirate scheme of five finegrid steps for a problem with (m, n) = (10 4 , 500). Then, the ROMR scheme is used, once with (MESS, * ) and once with the proper orthogonal decomposition (POD). In Fig. 7, both the correlation sum (left) and an accuracy plot (right) for (MESS, 0.0816) are depicted. The accuracy result for the POD is indistinguishable from the one depicted in Fig. 7 (right), and hence, it is omitted. The degrees of freedom are reduced from 10 4 to 13, while the optimal * is estimated in 2.9 s and the MESS base is constructed in 0.16 s, in contrast to a total of 6.33 s that is required by the POD.

IV. CONCLUSIONS
The maximum entropy snapshot sampling has been successfully used for reducing two nonlinear circuits; in particular, a diode chain model and a thermal-electric coupled system. In both cases, the maximum entropy snapshot sampling removed unnecessary data, and hence, it reduced the snapshot matrix, before calling the QR basis generation routine. As a result, the triangular-orthogonal decomposition has been called on a reduced snapshot matrix, and the offline stage has been significantly scaled down, in terms of CPU time. Since the maximum entropy snapshot sampling relies on pairwise distance computations, its performance can be further improved through CPU/GPU parallelization, while it enables an accept/reject routine that can be incorporated into the highfidelity solver, in order to immediately decide weather or not a new snapshot needs to be stored. This last feature reduces storage requirements, while, to our knowledge, the maximum entropy snapshot sampling is the only black-boxed method for performing non-homogeneous snapshot sampling, without relying on prior knowledge regarding the application at hand. Through the optimality requirement for selecting the parameter , the maximum entropy snapshot sampling does not require any user input, and hence, it can be seen as a parameterfree method, while a single parameter is required in general. Further research needs to be conducted for selecting an value that guarantees stability and maximally reduced models. The obtained bases have been used in a gappy framework for reduced nonlinear function evaluation, while the corresponding reduced models perform as accurate as the standard reduction framework that relies on the singular value decomposition.
V. ACKNOWLEDGEMENTS