Breaking the Layer Barrier: Remodeling Private Transformer Inference with Hybrid CKKS and MPC

This is the test code of the paper "Breaking the Layer Barrier: Remodeling Private Transformer Inference with Hybrid CKKS and MPC".

We also provide a docker, all the environments are configured, you can directly enter /home/CKKS-MPC/ to run the code:

docker pull crpi-hxmgk7l66ku6bx2x.cn-hangzhou.personal.cr.aliyuncs.com/tianshixu/ppdl:blbv1

docker run --gpus all --name blb_test --cap-add=NET_ADMIN -it [IMAGE_ID] /bin/bash

There is no clear description on how to build the seal and phantom. Please build according to the following command:

cd SCI/extern

git clone https://github.com/microsoft/SEAL.git
cd SEAL
cmake -S . -B build -DSEAL_USE_INTEL_HEXL=ON
cmake --build build

cd SCI/extern

git clone https://github.com/encryptorion-lab/phantom-fhe.git
cd phantom-fhe
git apply 0001-more-flexible.patch
cmake -S . -B build -DCMAKE_CUDA_ARCHITECTURES=native -DCMAKE_INSTALL_PREFIX=build_phantom
cmake --build build --target install --parallel

The 0001-more-flexible.patch is provided in V3.

There is a mistake in the README.md for building and running the code. Enter the "SCI/build" instead of "SCI/tests/build", then build the project and run the code.

The main code we developed is located in the /SCI/tests/bert_large_bolt/ directory. Other related components are found in /SCI/src/, which contains the core privacy-preserving protocol implementations for basic operators. These implementations are primarily based on BOLT and SiRNN.

Within /SCI/tests/bert_large_bolt/:

• The linear_ckks/ folder contains our fused linear operator protocols.

• The ckks_bert.h file includes other key components of the BLB framework, such as the CKKS-to-MPC conversion protocol, and the decomposition of nonlinear functions into secure sub-operators.

• The end-to-end evaluation of a Transformer block is implemented in ckks_bert_large_main.cpp.