Ian Goldberg iang

iang pushed to master at iang/spiral-spir

  • b49e8544ab Use upstream spiral-rs again instead of our fork They have implemented the gist of our fork, and also parameter and query compression (saves a factor of ~2)

1 month ago

iang pushed to client_mt at iang/spiral-rs-fork

  • d6d546d056 Use interior mutability to allow multithreaded use of Client It would be nice to be able to call client.generate_query from multiple threads in parallel. It currently wants self to be mutable, however, so that can't happen. The only reason it needs mutability of self, though, is for the embedded random number generators. So the strategy is this: make client itself immutable, but wrap the two rngs (the one in DiscreteGaussian and the one in Client itself) with Mutex. Then when we need the rng, we get a lock to the mutable rng from the immutable Mutex using Mutex's interior mutability. But this would still cause contention if lots of threads were trying to use the same rng at the same time, so we put each rng inside a ThreadLocal so that there's actually one instance of each mutex-wrapped rng per thread. The locks will therefore never be in contention, and in fact with ThreadLocal, we can use a simpler RefCell instead of a Mutex. (Note that the ThreadLocal would not be necessary if we insisted that the rng passed to client were ThreadRng, but that would break the existing tests that pass an expicitly seeded rng to check a deterministic output.) This requires a change to the Client::init API: instead of passing a reference to a static rng, you pass a factory function that _outputs_ a new rng. This function will be called once per thread. Also the type of the rng has to be Rng + Send instead of just Rng, since we'll be putting it in a ThreadLocal. This means the factory function can't output a ThreadRng (which wouldn't make sense anyway), but should be something like ChaCha20Rng::from_entropy. With this change, almost all of Client's methods (including generate_query and its callees) can take an immutable &self, and be able to safely run from multiple threads in parallel.
  • 790e70ca06 Use interior mutability to allow multithreaded use of Client It would be nice to be able to call client.generate_query from multiple threads in parallel. It currently wants self to be mutable, however, so that can't happen. The only reason it needs mutability of self, though, is for the embedded random number generators. So the strategy is this: make client itself immutable, but wrap the two rngs (the one in DiscreteGaussian and the one in Client itself) with Mutex. Then when we need the rng, we get a lock to the mutable rng from the immutable Mutex using Mutex's interior mutability. But this would still cause contention if lots of threads were trying to use the same rng at the same time, so we put each rng inside a ThreadLocal so that there's actually one instance of each mutex-wrapped rng per thread. The locks will therefore never be in contention. (Note that the ThreadLocal would not be necessary if we insisted that the rng passed to client were ThreadRng, but that would break the existing tests that pass an expicitly seeded rng to check a deterministic output.) This requires a change to the Client::init API: instead of passing a reference to a static rng, you pass a factory function that _outputs_ a new rng. This function will be called once per thread. Also the type of the rng has to be Rng + Send instead of just Rng, since we'll be putting it in a ThreadLocal. This means the factory function can't output a ThreadRng (which wouldn't make sense anyway), but should be something like ChaCha20Rng::from_entropy. With this change, almost all of Client's methods (including generate_query and its callees) can take an immutable &self, and be able to safely run from multiple threads in parallel.
  • View comparison for these 2 commits »

1 month ago

iang pushed to master at iang/spiral-spir

  • 8232fabb6d Construct PIR preprocess queries in parallel This relies on the client_mt multithreaded client fork of the spiral-rs crate.

1 month ago

iang pushed to client_mt at iang/spiral-rs-fork

  • 790e70ca06 Use interior mutability to allow multithreaded use of Client It would be nice to be able to call client.generate_query from multiple threads in parallel. It currently wants self to be mutable, however, so that can't happen. The only reason it needs mutability of self, though, is for the embedded random number generators. So the strategy is this: make client itself immutable, but wrap the two rngs (the one in DiscreteGaussian and the one in Client itself) with Mutex. Then when we need the rng, we get a lock to the mutable rng from the immutable Mutex using Mutex's interior mutability. But this would still cause contention if lots of threads were trying to use the same rng at the same time, so we put each rng inside a ThreadLocal so that there's actually one instance of each mutex-wrapped rng per thread. The locks will therefore never be in contention. (Note that the ThreadLocal would not be necessary if we insisted that the rng passed to client were ThreadRng, but that would break the existing tests that pass an expicitly seeded rng to check a deterministic output.) This requires a change to the Client::init API: instead of passing a reference to a static rng, you pass a factory function that _outputs_ a new rng. This function will be called once per thread. Also the type of the rng has to be Rng + Send instead of just Rng, since we'll be putting it in a ThreadLocal. This means the factory function can't output a ThreadRng (which wouldn't make sense anyway), but should be something like ChaCha20Rng::from_entropy. With this change, almost all of Client's methods (including generate_query and its callees) can take an immutable &self, and be able to safely run from multiple threads in parallel.
  • 9d3533104d Remove println! calls from server module in library
  • 0f9bdc1570 Update README.md
  • 853ba1df6f Fix e2e test output bug
  • 74f169d8dd Update README

1 month ago

iang created new branch client_mt at iang/spiral-rs-fork

1 month ago

iang pushed to main at iang/spiral-rs-fork

1 month ago

iang created new branch main at iang/spiral-rs-fork

1 month ago

iang created repository iang/spiral-rs-fork

1 month ago

iang pushed to master at iang/spiral-spir

  • 51cf07751d A simple cast can replace the entire need for the AlignedMemoryMT type

1 month ago

iang pushed to master at iang/spiral-spir

1 month ago

iang pushed to master at iang/spiral-spir

1 month ago

iang pushed to master at iang/spiral-spir

1 month ago

iang pushed to master at iang/lox

1 month ago

iang pushed to master at iang/spiral-spir

2 months ago

iang pushed to master at iang/spiral-spir

2 months ago

iang pushed to master at iang/spiral-spir

2 months ago

iang pushed to master at iang/spiral-spir

2 months ago

iang created new branch master at iang/spiral-spir

2 months ago

iang created repository iang/spiral-spir

2 months ago

iang pushed to master at piros/pirserver

4 months ago