# Circuit ORAM docker experimentation environment
Ian Goldberg, iang@uwaterloo.ca
This repo contains scripts to run [Jarecki and Wei's 3-party Circuit ORAM](https://eprint.iacr.org/2018/347) in docker containers for easy experimentation on varying the ORAM size, and network latency and bandwidth.
These scripts are in support of our paper:
Adithya Vadapalli, Ryan Henry, Ian Goldberg. Duoram: A Bandwidth-Efficient Distributed ORAM for 2- and 3-Party Computation. USENIX Security Symposium 2023. [https://eprint.iacr.org/2022/1747](https://eprint.iacr.org/2022/1747)
It is a fork of [Wei's published code](https://github.com/Boyoung-/circuit-oram-3pc), with two small changes:
- We add an `iters:` field to the config.yaml file to control at runtime the number of iterations of the protocol to run
- We tag the test code output with labels indicating what values it is outputting, where useful.
plus the dockerization and reproduction scripts in the [docker](docker/) folder.
## Reproduction instructions
Follow these instructions to reproduce the Circuit ORAM data points (timings
and bandwidth usage of Circuit ORAM operations for various ORAM sizes and
network settings) for the plots in our paper. See
[below](#manual-instructions) if you want to run experiments of your
choosing.
- `cd docker`
- Build the docker image with `./build-docker`
- Start the dockers with `./start-docker`
- This will start three dockers, each running one of the parties.
- Run the reproduction script `./repro` with one of the following
arguments:
- ./repro test
: Run a short (just about 10 seconds) "kick-the-tires" test.
You should see output like the following:
Running test experiment...
Tue 21 Feb 2023 02:55:51 PM EST: Running 16 1us 100gbit 2 ...
CircuitORAMOnln read 16 1us 100gbit 2 0.172 s
CircuitORAMTotl read 16 1us 100gbit 2 0.54 s
CircuitORAMOnln read 16 1us 100gbit 2 710.625 KiB
CircuitORAMTotl read 16 1us 100gbit 2 4957 KiB
The last four lines are the output data points, telling you that a
Circuit ORAM read test on an ORAM of size 216, with a network
configuration of 1us latency and 100gbit bandwidth, performing 2
read operations, took 0.172 s of online time and 0.54 s of total
(precomputation plus online) time, and 710.625 KiB of
online bandwidth and 4957 KiB of total bandwidth. If you've run
the test before, you will see means and stddevs of all of the output
data points. When you run it, the time of course will depend on
the particulars of your hardware, but the bandwidths used should be
exactly the values quoted above.
- ./repro small _numops_
: Run the "small" tests. These
are the tests up to size 226, and produce all the data
points for most of Figure 9.
_numops_
is the number of operations to run for each
test; we used the default of 128 for the figures in the paper, but
you can use a lower number to make the tests run faster. For the
default of 128, these tests should complete in about 2 to 3 hours,
and require 2 GB of available RAM.
- ./repro large _numops_
: Run the "large" tests. These
are the rightmost 3 data points in Figure 9. They are not
essential to our major claims, so they are optional to run.
For the default _numops_
of 128, these experiments
will require about 2 hours to run and 2 GB of available RAM.
- ./repro all _numops_
: Run both the "small" and
"large" tests.
- ./repro none _numops_
: Run no tests. This command is
nonetheless useful in order to parse the output logs and display
the data points for the graphs (see below).
- ./repro single _size_ _latency_ _bandwidth_ _numops_
:
run a single manually selected test with the given parameters.
- After `small`, `large`, `all`, or `none`, the script will parse
all of the outputs that have been collected with the specified
_numops_
(in this run or previous runs), and output
them as they would appear in each of the subfigures of Figure 9.
- When you're done, `./stop-docker`
## Manual instructions
- `cd docker`
- `./build-docker`
- `./start-docker`
- This will start three dockers, each running one of the parties.
Then to simulate network latency and capacity (optional):
- `./set-networking 30ms 100mbit`
To turn that off again:
- `./unset-networking`
If you have a NUMA machine, you might want to pin each party to one
NUMA node. To do that, set these environment variables before running
`./run-experiment` below:
- `export ORAM_NUMA_C="numactl -N 1 -m 1"`
- `export ORAM_NUMA_D="numactl -N 2 -m 2"`
- `export ORAM_NUMA_E="numactl -N 3 -m 3"`
Adjust the numactl arguments to taste, of course, depending on your
machine's configuration. Alternately, you can use things like `-C 0-7`
instead of `-N 1` to pin to specific cores, even on a non-NUMA machine.
Run experiments:
- ./run-experiment _size_ _numops_ >> _outfile_
- _size_
is the base-2 log of the number of entries in the ORAM (so _size_
= 20 is an ORAM with 1048576 entries, for example). Defaults to 20.
- _numops_
is the number of read operations to perform. Defaults to 128.
- ./parse\_logs _outfile_
- Parses the file output by one or more executions of `./run-experiment` to extract, for each experiment, the average number of bytes sent by each party and the time taken.
When you're all done:
- `./stop-docker`