# Floram docker experimentation environment

Ian Goldberg, iang@uwaterloo.ca  
Adithya Vadapalli, adithya.vadapalli@uwaterloo.ca

This repo contains scripts to run [Doerner and shelat's Floram](https://dl.acm.org/doi/10.1145/3133956.3133967) in docker containers for easy experimentation on varying the ORAM size, and network latency and bandwidth.

These scripts are in support of our paper:

Adithya Vadapalli, Ryan Henry, Ian Goldberg. Duoram: A Bandwidth-Efficient Distributed ORAM for 2- and 3-Party Computation. USENIX Security Symposium 2023. [https://eprint.iacr.org/2022/1747](https://eprint.iacr.org/2022/1747)

It is based on [Doerner and shelat's published code](https://gitlab.com/neucrypt/floram/-/archive/floram-release/floram-floram-release.zip), with two small changes:

  - Their benchmarking code (`bench_oram_read` and `bench_oram_write`) sets up the ORAM, and then does a number of read or a number of write operations.  The _time_ to set up the ORAM is included in the reported time, but the _bandwidth_ to set up the ORAM is not included in the reported bandwith. We have [a patch](bench_oram.patch) to also measure the bandwidth of the setup, and report it separately from the bandwidth of the operations.
  - We also add [a read/write benchmark](bench_oram_readwrite.oc) that does alternating reads and writes. If you ask for 128 iterations, for example, it will do 128 reads and 128 writes, interleaved.

## Instructions:

  - `./build-docker`
  - `./start-docker`
    - This will start two dockers, each running one of the parties

Then to simulate network latency and capacity (optional):

  - `./set-networking 30ms 100mbit`

To turn that off again:

  - `./unset-networking`

If you have a NUMA machine, you might want to pin each party to one
NUMA node.  To do that, set these environment variables before running
`./run-experiment` below:

  - `export FLORAM_NUMA_P0="numactl -N 1 -m 1"`
  - `export FLORAM_NUMA_P1="numactl -N 2 -m 2"`

Adjust the numactl arguments to taste, of course, depending on your
machine's configuration.  Alternately, you can use things like `-C 0-7`
instead of `-N 1` to pin to specific cores, even on a non-NUMA machine.

Run experiments:

  - <code>./run-experiment _mode_ _size_ _iters_ _port_ &gt;&gt; _outfile_</code>
    - <code>_mode_</code> is one of `read`, `write`, `readwrite`, or `init`
      - `init` measures setting up the database with non-zero initial values; the other three modes include setting up the database initialized to 0.  Defaults to `read`.
    - <code>_size_</code> is the base-2 log of the number of entries in the ORAM (so <code>_size_</code> = 20 is an ORAM with 1048576 entries, for example).  Defaults to 20.
    - <code>_iters_</code> is the number of iterations to perform; one setup will be followed by <code>_iters_</code> operations, where each operation is a read, a write, or a read plus a write, depending on the <code>_mode_</code>. Defaults to 128.
    - <code>_port_</code> is the port number to use; if you're running multiple experiments at the same time, they must each be on a different port.  Defaults to 3000.

  - <code>./parse\_sizes _outfile_</code>
    - Parses the file output by one or more executions of `./run-experiment` to extract the number of bytes sent in each experiment.  The output will be, for each experiment, a line with the two numbers <code>_size_</code> and <code>_kib_</code>, which are the size of the experiment and the average number of KiB (kibibytes = 1024 bytes) sent per party, including both the ORAM setup and the operations.

  - <code>./parse\_times _outfile_</code>
    - Parses the file output by one or more executions of `./run-experiment` to extract the runtime of each experiment.  The output will be, for each experiment, a line with the two numbers <code>_size_</code> and <code>_sec_</code>, which are the size of the experiment and the time in seconds, including both the ORAM setup and the operations.

To see an example of how to use `./run-experiment` while varying the experiment size and the network latency and bandwidth, the [`./run-readwrite-experiments`](run-readwrite-experiments) script wraps `./run-experiment`, and is the script we used to generate the interleaved Floram measurements in Figures 7 and 8 of [our paper](https://eprint.iacr.org/2022/1747).

When you're all done:

  - `./stop-docker`