Initial release of Dockerfiles and scripts for building and running Mitigator

LICENSE.txt

@@ -0,0 +1,24 @@
+This software is licensed under the MIT License. However, some dependencies
+of this software are licensed under more restrictive licenses.
+
+MIT License
+
+Copyright (c) 2020 Miti Mazmudar and Ian Goldberg
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,28 @@
+# MITIGATOR
+
+This repository contains Docker files for reproducing the implementation of the server-side code for the paper: 
+
+Miti Mazmudar, Ian Goldberg. "Mitigator: Privacy policy compliance using trusted hardware".  
+Proceedings on Privacy Enhancing Technologies. Vol. 2020, No. 3. 18 pages. July 2020.
+
+*Note:* This system requires a machine with Intel Software Guard Extension (SGX) in order to be built and to function correctly. 
+
+This system consists of three enclaves: a decryptor, a verifier and a target enclave. The target enclave is a PHP-enabled apache web server, which obtains user data through forms and we wish to ensure compliance of the webserver with its privacy policy. The Linux SGX driver and Linux SGX SDK are required to run the decryptor enclave. The Linux SGX SDK is also required for the enclaves to attest to each other and to seal secrets to disk. We use the Graphene-SGX library to support running a proof-of-concept verifier and the target application. The target application also contains a PHP extension to interface with the decryptor enclave. The Graphene-SGX driver is required to run any applications within Graphene. 
+
+We use Docker scripts to reproduce our setup for our paper. Two of our scripts set up drivers and the others set up and launch a docker container for running the three enclaves. 
+* `build-driver`: builds specific versions of the Linux SGX and the Graphene-SGX drivers as kernel modules.
+* `install-driver`: installs the kernel modules generated by the build-driver script onto the machine. 
+* `build-mitigator`: builds three docker images, one for each of the Dockerfiles in the sgx-docker/, graphene-docker/, and docker/ folders respectively. The first docker image contains the SGX SDK and PSW setup. The second docker image adds the Graphene-SGX setup onto the first one. The last docker image is formed by downloading the source code for our enclaves, including any dependencies, and generating the three enclaves. 
+* `run-mitigator`: creates a *detached* docker container with the image initialized in the build-mitigator script. Re-run this script after any changes to the `build-mitigator` script to recreate a docker container with the new docker image.
+* `attach-mitigator`: attaches to the container created by the `run-mitigator` script. This lets you interact with the docker container through the terminal.
+* `stop-mitigator`: stops the container created by the `run-mitigator` script. This needs to be run before re-creating a container with a new image through `run-mitigator`. 
+
+## Build steps
+
+1. On an SGX-supporting machine with sudo privileges, run `./build-driver` and then `sudo ./install-driver`. This step requires having sudo privileges in order to insert kernel drivers into the machine.  The Intel SGX and Gtaphene SGX drivers that are built and installed are the standard ones unmodified by us.
+2. On the same machine, run the `build-mitigator` script to build the docker images. Enter the docker container by running `run-mitigator` and then `attach-mitigator`. After entering the docker container, run the `deploy_enclaves` script to create the three enclaves. The decryptor enclave is created first, followed by the verifier and then the Apache server, each in a different tmux session. To restart the Apache server after stopping it, you need to stop the decryptor and then run the `deploy_enclaves` script. This restarts the decryptor, verifier and the Apache server in that order. 
+
+## Testing
+
+We require a client to install [our Firefox browser extension](https://git-crysp.uwaterloo.ca/miti/browser-extension) to test our server-side setup. The last script opens up the port on which the Mitigator server is running within the docker container (port 8001). Obtain the ip address of the docker container by running the `find_docker_ip` script within the container. The server form page can be retrieved on the server machine by accessing `docker_url:8001/index.php`. The browser extension will automatically encrypt all form fields to the decryptor enclave when the submit button is pressed.  
+

attach-mitigator

@@ -0,0 +1,3 @@
+#!/bin/bash
+
+docker exec -it  mitigator bash

build-driver

@@ -0,0 +1,15 @@
+#!/bin/bash -x
+
+rm -rf linux-sgx-driver
+git clone https://github.com/intel/linux-sgx-driver.git
+cd linux-sgx-driver
+git checkout f7dc97c0
+make
+cp -av sgx_user.h ../graphene-docker/
+cd ..
+rm -rf graphene-sgx-driver
+git clone https://github.com/oscarlab/graphene-sgx-driver.git
+cd graphene-sgx-driver
+git checkout 30d4b940
+ISGX_DRIVER_PATH=../linux-sgx-driver ISGX_DRIVER_VERSION=2.5 make
+cp -av isgx_version.h ../graphene-docker/

build-mitigator

@@ -0,0 +1,5 @@
+#!/bin/bash -x
+
+cd sgx-docker && docker build -t sgx . || exit 1
+cd ../graphene-docker && docker build -t graphene . || exit 1
+cd ../docker && docker build -t mitigator . || exit 1

docker/Dockerfile

@@ -0,0 +1,95 @@
+FROM graphene
+ENV SGX_SDK=/opt/intel/sgxsdk/
+
+RUN mkdir source
+WORKDIR source
+## Setting up prereqs for decryptor - Intel SGX SSL
+# Just curling the tar.gz file and then running the build script didn't work as the extracted folder had a different name (dependent on the foldername which was compressed) --- I didnt want to make a new fork of the sgxssl script for that.
+RUN git clone https://github.com/openssl/openssl.git OpenSSL_1.1.1d && \
+	cd OpenSSL_1.1.1d && git checkout tags/OpenSSL_1_1_1d && \
+	cd ../ && tar -cf OpenSSL_1.1.1d.tar.gz OpenSSL_1.1.1d/
+
+#Setting up SGXSSL with the version of OpenSSL that we downloaded in the previous step.
+RUN git clone https://github.com/intel/intel-sgx-ssl.git && \
+	cd intel-sgx-ssl && git checkout tags/lin_2.5_1.1.1d && \
+	cp ../OpenSSL_1.1.1d.tar.gz ./openssl_source/  && \
+	cd Linux &&  make && make install
+
+#Setting up protobuf definitions for exchanging LA and post-LA messages between enclaves.
+RUN git clone https://git-crysp.uwaterloo.ca/miti/dhmsgs_proto_defs.git && \
+	cd dhmsgs_proto_defs/ && protoc --cpp_out=./ ./*.proto
+
+#Setting up the decryptor enclave itself.
+RUN git clone https://git-crysp.uwaterloo.ca/miti/Decryptor.git && \
+	cd Decryptor/ && make
+
+#Setting up common files used in the verifier, PHP extension for LA/post-LA message processing.
+RUN git clone https://git-crysp.uwaterloo.ca/miti/commonVerifierPHPfiles.git
+
+#Setting up a patched version of the linux-sgx repo's SDK for running LA on graphene.
+RUN git clone https://git-crysp.uwaterloo.ca/miti/linux-sgx-trts-modified.git && \
+	cd linux-sgx-trts-modified && git checkout local_attestation_for_graphene && \
+	bash ./download_prebuilt.sh && make USE_OPT_LIBS=0
+
+#Setting up the verifier.
+RUN git clone https://git-crysp.uwaterloo.ca/miti/verifier.git && \
+	cd verifier/ && git checkout recreating_state_for_teeter && \
+	export SGX_SDK_TRTS_MODIFIED=/root/source/linux-sgx-trts-modified/build/linux && make && \
+	cp verifier /root/graphene/LibOS/shim/test/native && \
+	git checkout master && \
+	cp verifier.manifest.template /root/graphene/LibOS/shim/test/native 
+
+#Build the verifier
+WORKDIR /root/graphene/LibOS/shim/test/native
+#Treat the verifier as an executable whose manifest is to be generated from its manifest.template file in this folder.
+RUN sed -i s/ls.manifest/'ls.manifest verifier.manifest'/g Makefile 
+RUN make SGX=1
+
+RUN apt update && apt install -y php7.0-dev tmux libxml2-dev 
+
+WORKDIR /root/source
+RUN git clone --recursive https://github.com/CopernicaMarketingSoftware/PHP-CPP.git && \
+	cd PHP-CPP/ && git checkout tags/v2.1.4 &&  make all && \
+	cp libphpcpp.so.* /usr/lib/ && make install
+
+#Dependencies for Apache
+RUN apt update && apt install -y build-essential flex libapr1-dev libaprutil1-dev libpcre2-dev apache2-utils libssl-dev
+RUN git clone https://git-crysp.uwaterloo.ca/miti/Apache_PHP_extension.git && \
+	cd Apache_PHP_extension && git checkout recreating_state_for_teeter && \
+	make
+
+WORKDIR /root/graphene/LibOS/shim/test/apps/apache
+#Modifying the Makefile to generate the php module.
+RUN sed -i s/'LISTEN_HOST ?= 127.0.0.1'/'LISTEN_HOST ?= 0.0.0.0'/g Makefile
+RUN sed -i 's:testdata ssldata:ssldata $(INSTALL_DIR)/modules/libphp7.so:' Makefile 
+RUN sed -i s/"'-'"/"'-' | tr '+' 'p'"/g Makefile #Fix for stdc++ to be named as a key sgx.trusted_files.stdcpp in the autogenerated manifest. 
+#Copy the PHP 7 archive (PHPCPP requires at least version 7) 
+RUN cp /root/source/Apache_PHP_extension/*.tar.gz ./
+#Copy targets for generating the php module library from the archive to the makefile.
+RUN cat /root/source/Apache_PHP_extension/apacheMakefile >> Makefile 
+#Install dependencies for Apache
+RUN apt update && apt install -y build-essential flex libapr1-dev libaprutil1-dev libpcre2-dev apache2-utils libssl-dev
+#Make the apache/php build first 
+#In the new graphene version, runtime tokens are also generated along with the manifests, and these tokens require the AESMD service. 
+#We start the aesmd service at runtime and generate all targets by running 'make SGX=1' on this folder at runtime. 
+RUN make SGX=1 build-apache-with-php
+
+#Add the updated httpd.manifest.template #Todo update manifest. 
+RUN cp /root/source/Apache_PHP_extension/httpd.manifest.template ./
+#Add the extension to the modules folder. 
+RUN cp /root/source/Apache_PHP_extension/localattestation_decryption.so install/modules/
+#Add the PHP .ini file from the PHP folder to the location accessed by the PHP setup at runtime
+RUN cp ./php-7.0.7/php.ini-development install/lib/php.ini
+#Add the extension path and the extension name to the php.ini 
+RUN echo "extension_dir=/root/graphene/LibOS/shim/test/apps/apache/install/modules" >> install/lib/php.ini && \ 
+	echo "extension=localattestation_decryption.so" >> install/lib/php.ini
+#Add all php source code files to the right place. 
+RUN cp /root/source/Apache_PHP_extension/*.php install/htdocs
+#Fix the configuration file for Apache
+RUN printf '\nAcceptFilter http none\n<IfModule mime_module>\n    AddType application/x-httpd-php .php\n</IfModule>\n' >> install/conf/httpd.conf 
+
+WORKDIR /root
+COPY deploy_enclaves.sh ./
+RUN chmod 755 ./deploy_enclaves.sh
+COPY find_docker_ip.sh ./
+RUN chmod 755 ./find_docker_ip.sh

docker/deploy_enclaves.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+#To exit a tmux session, use Ctrl+B and then press D. 
+#To enter a tmux session, use tmux at -t <session_name> where session_name is decryptor, verifier or phpext.
+#List existing sessions by pressing tmux ls.
+
+cd ~/graphene && make SGX=1 sgx-tokens
+cd ~/source/Decryptor && tmux new-session -d -s dec './app; bash';
+cd ~/graphene/LibOS/shim/test/native && tmux new-session -d -s ver './pal_loader SGX verifier; bash'
+sleep 60; #Should have better logic here to check if the previous command exited successfully (server should only be started after the verifier finishes, or else it will try to do LA with the decryptor and crash' 
+cd ~/graphene/LibOS/shim/test/apps/apache && tmux new-session -d -s php "SGX=1 make start-graphene-server; bash" 
+

docker/find_docker_ip.sh

@@ -0,0 +1,4 @@
+#!/bin/bash 
+
+ip addr | grep "inet" | grep -v "127.0.0.1" |  cut -d'/' -f 1  | tr -d [:alpha:]
+#List all IP addr info | filter out lines with addresses | grab the host part of an IPv4 address (part before '/') | remove all alphabet chars on that line - "inet" etc

graphene-docker/Dockerfile

@@ -0,0 +1,18 @@
+FROM sgx
+RUN apt update
+RUN apt install -y gawk libprotobuf-c-dev protobuf-c-compiler bison python3-pip
+RUN pip3 install protobuf
+
+WORKDIR /root
+RUN git clone https://github.com/oscarlab/graphene
+WORKDIR graphene
+RUN git checkout c0bc728
+RUN git submodule init
+RUN git submodule update
+RUN openssl genrsa -3 -out Pal/src/host/Linux-SGX/signer/enclave-key.pem 3072
+WORKDIR Pal/src/host/Linux-SGX/sgx-driver
+RUN mkdir linux-sgx-driver
+COPY sgx_user.h ./linux-sgx-driver/
+WORKDIR /root/graphene
+RUN make SGX=1 ISGX_DRIVER_PATH=/root/graphene/Pal/src/host/Linux-SGX/sgx-driver/linux-sgx-driver
+WORKDIR /root

install-driver

@@ -0,0 +1,18 @@
+#!/bin/bash -x
+
+service aesmd stop
+/sbin/modprobe -r graphene-sgx
+/sbin/modprobe -r isgx
+mkdir -p "/lib/modules/"`uname -r`"/kernel/drivers/intel/sgx"
+cp linux-sgx-driver/isgx.ko graphene-sgx-driver/graphene-sgx.ko "/lib/modules/"`uname -r`"/kernel/drivers/intel/sgx"
+cat /etc/modules | grep -Fxq isgx || echo isgx >> /etc/modules
+cat /etc/modules | grep -Fxq graphene-sgx || echo graphene-sgx >> /etc/modules
+/sbin/depmod
+/sbin/modprobe isgx
+/sbin/modprobe graphene-sgx
+service aesmd start
+if [ "`sysctl -n vm.mmap_min_addr`" != 0 ]; then
+	cat /etc/sysctl.conf | grep -Fxq 'vm.mmap_min_addr = 0' || \
+		echo -e "\n# For Graphene-SGX\nvm.mmap_min_addr = 0" >> /etc/sysctl.conf
+	sysctl -p
+fi

run-mitigator

@@ -0,0 +1,2 @@
+#!/bin/bash
+docker run --device /dev/isgx --device /dev/gsgx --name mitigator --rm -p 8044:8044 -dt mitigator bash -c "start-aesmd && exec bash"

sgx-docker/Dockerfile

@@ -0,0 +1,15 @@
+FROM ubuntu:16.04
+RUN apt update && apt install -y build-essential sudo ocaml automake autoconf libtool wget python libssl-dev git protobuf-compiler libprotobuf-dev cmake curl libcurl4-openssl-dev lsb-release debhelper unzip nano kmod
+WORKDIR /root
+RUN git clone https://github.com/intel/linux-sgx.git
+WORKDIR linux-sgx
+RUN git checkout sgx_2.1
+RUN ./download_prebuilt.sh #Apparently need to run this even if we use opensource libraries - issue #363
+RUN make USE_OPT_LIBS=0 #For using opensource libraries for sgx library code.
+RUN make sdk_install_pkg
+RUN make psw_install_pkg
+WORKDIR linux/installer/bin
+RUN echo -e "no\n/opt/intel" | ./sgx_linux_x64_sdk_*.bin
+RUN echo -e "no\n/opt/intel" | ./sgx_linux_x64_psw_*.bin
+COPY start-aesmd /usr/local/sbin/
+WORKDIR /root

sgx-docker/start-aesmd

@@ -0,0 +1,3 @@
+#!/bin/bash
+
+su -s /bin/bash aesmd -c "LD_LIBRARY_PATH=/opt/intel/sgxpsw/aesm /opt/intel/sgxpsw/aesm/aesm_service"

stop-mitigator

@@ -0,0 +1,3 @@
+#!/bin/bash
+
+docker stop mitigator