tor-parallel-relay-conn/doc/TODO

Legend:
SPEC!!  - Not specified
SPEC    - Spec not finalized
NICK    - nick claims
ARMA    - arma claims
        - Not done
        * Top priority
        . Partially done
        o Done
        D Deferred
        X Abandoned

For scalability:
        - Slightly smarter bandwidth management: use link capacity
          intelligently.
        - Handle full buffers without totally borking

For 0.0.8:

      milestone 2:
        . refer to things by key:
          o extend cells need ip:port:identitykeyhash.
          o Lookup routers and connections by key digest; accept hex
            key digest in place of nicknames.
          o Audit all uses of lookup-by-hostname and lookup-by-addr-port
            to search by digest when appropriate.
          o make sure to use addr/port in cpuworker tasks, because
            OPs don't have keys.
          o and fix the function comments in rephist
          o Rep-hist functions need to store info by keyid
          - also use this in intro points and rendezvous points, and
            hidserv descs.  [XXXX This isn't enough.]
          - figure out what to do about ip:port:differentkey
        o ORs connect on demand. attach circuits to new connections, keep
          create cells around somewhere, send destroy if fail.
        o nickname defaults to first piece of hostname
        o running-routers list refers to nickname if verified, else
          hash-base64'ed.
        o Mark routers as verified or unverified based on whether
          running-routers list includes nickname or id hash.
        o put OR uptime in descriptor
        o name the secret-key directory something to discourage people
          from mailing their identity key to tor-ops

      milestone 3:
        - users can set their bandwidth, or we auto-detect it:
          - advertised bandwidth defaults to 10KB
          o advertised bandwidth is the min of max seen in each direction
            in the past N seconds.
            o calculate this
          o not counting "local" connections
          - round detected bandwidth up to nearest 10KB
        - client software not upload descriptor until:
          - you've been running for an hour
          - it's sufficiently satisfied with its bandwidth
          - it decides it is reachable
          - start counting again if your IP ever changes.
          - never regenerate identity keys, for now.
          - you can set a bit for not-being-an-OR.
NICK    - Reputation info needs to give better weight to recent events than
          very old ones.
        - Have clients and dirservers preserve reputation info over
          reboots.
        - clients choose nodes proportional to advertised bandwidth
        o authdirserver includes descriptor.
        -  and lists as running iff:
          - he can connect to you
          - he has successfully extended to you
          - he has sufficient mean-time-between-failures
        - Don't accept ORs with nicknames same as verified ORs' nicknames.
        - add new "Middleman 1" config variable?
          o if torrc not found, exitpolicy reject *:*
        o change if(options.ORPort) to what we really mean.
        o same with socksport.
        o get contrib/tor_resolve into the tarball and installed
          - and working

      post pre1:
        - Possible to get autoconf to easily install things into ~/.tor?
        - when we sigint tor, the dns/cpuworkers don't intercept sigint?
        - "AcceptOnlyVerifiedRouters" config option?
        - why does common/util.c build-depend on or/or.h ?
        - ORs use uniquer default nicknames
        - Tors deal appropriately when a newly-verified router has the
          same nickname as another router they know about
        X 007 can't extend to unverified 008. they will never be able to.
        - if a begin failed due to exit policy, but we believe the IP
          should have been allowed, switch that router to exitpolicy
          reject *:* until we get our next directory.
        - make advertised_server_mode() ORs fetch dirs more often.
        - should the running-routers list put unverified routers at the
          end?
        - tor-resolve needs a man page
        - tor-resolve should make use of cached answers?
        - defining an ORPort isn't necessary anymore, if you more
          ORAddress:port. Same with DirPort, SocksPort.
        - requiredentrynode vs preferredentrynode
        - per-month byte allowances
        o if using not-socks4a then warn, once.
        o if unverified server then warn, once.
        - add a listener for a ui
          - and a basic gui
        - faq and doc-wiki
          - knoppix distro
          - win32 installer using privoxy's installer
        o win32 problems with pre1
        . asn.1 issues?
        o Switch dirservers entries to config lines:
          - read in and parse each TrustedDir config line.
          - stop reading dirservers file.
          - add some default TrustedDir lines if none defined, or if
            no torrc.
          - remove notion of ->is_trusted_dir from the routerlist. that's
            no longer where you look.
            - clean up router parsing flow, since it's simpler now?
          - when checking signature on a directory, look it up in
            options.TrustedDirs, and make sure there's a descriptor
            with that nickname, whose key hashes to the fingerprint,
            and who correctly signed the directory.
          - when fetching a directory, if you want a trusted one,
            choose from the trusteddir list.
            - which means keeping track of which ones are "up"
          - if you don't need a trusted one, choose from the routerinfo
            list if you have one, else from the trusteddir list.
        - compress the directory. client sends http header
          "accept-transfer-encoding: gzip", server might send http header
          "transfer-encoding: gzip". ta-da.
          - grow a zlib dependency. keep a cached compressed directory.


      ongoing:
        . rename/rearrange functions for what file they're in
        - generalize our transport: add transport.c in preparation for
          http, airhook, etc transport.

For September:
NICK    . Windows port
          o works as client
            - deal with pollhup / reached_eof on all platforms
          . robust as a client
          . works as server
            - can be configured
          - robust as a server
          . Usable as NT service
          - docs for building in win
          - installer

        - Docs
          - FAQ
          o overview of tor. how does it work, what's it do, pros and
            cons of using it, why should I use it, etc.
          - a howto tutorial with examples
          o tutorial: how to set up your own tor network
            - (need to not hardcode dirservers file in config.c)
          . correct, update, polish spec
          - document the exposed function api?
          - document what we mean by socks.

NICK    . packages
          . rpm
          - find a long-term rpm maintainer

        - code
          - better warn/info messages
          o let tor do resolves.
          o extend socks4 to do resolves?
          o make script to ask tor for resolves
          - tsocks
            - gather patches, submit to maintainer
            - intercept gethostbyname and others, do resolve via tor
          - redesign and thorough code revamp, with particular eye toward:
            - support half-open tcp connections
            - conn key rotation
            - other transports -- http, airhook
            - modular introduction mechanism
            - allow non-clique topology

Other details and small and hard things:
        - tor should be able to have a pool of outgoing IP addresses
          that it is able to rotate through. (maybe)
        - tie into squid
        - buffer size pool, to let a few buffers grow huge or many buffers
          grow a bit
        - hidserv offerers shouldn't need to define a SocksPort
        - when the client fails to pick an intro point for a hidserv,
          it should refetch the hidserv desc.
        . should maybe make clients exit(1) when bad things happen?
          e.g. clock skew.
        - should retry exitpolicy end streams even if the end cell didn't
          resolve the address for you
        - Add '[...truncated]' or similar to truncated log entries (like the directory
          in connection_dir_process_inbuf()).
        . Make logs handle it better when writing to them fails.
        o Dirserver shouldn't put you in running-routers list if you haven't
          uploaded a descriptor recently
        . Refactor: add own routerinfo to routerlist.  Right now, only
          router_get_by_nickname knows about 'this router', as a hack to
          get circuit_launch_new to do the right thing.
        . Scrubbing proxies
                - Find an smtp proxy?
                . Get socks4a support into Mozilla
        - Extend by hostname, not by IP.
        - Need a relay teardown cell, separate from one-way ends.
        - Make it harder to circumvent bandwidth caps: look at number of bytes
          sent across sockets, not number sent inside TLS stream.
        - fix router_get_by_* functions so they can get ourselves too,
          and audit everything to make sure rend and intro points are
          just as likely to be us as not.


***************************Future tasks:****************************

Rendezvous and hidden services:
  make it fast:
    - preemptively build and start rendezvous circs.
    - preemptively build n-1 hops of intro circs?
    - cannibalize general circs?
  make it reliable:
    - standby/hotswap/redundant services.
    - store stuff to disk? dirservers forget service descriptors when
      they restart; nodes offering hidden services forget their chosen
      intro points when they restart.
  make it robust:
    - auth mechanisms to let midpoint and bob selectively choose
      connection requests.
  make it scalable:
    - right now the hidserv store/lookup system is run by the dirservers;
      this won't scale.

Tor scalability:
  Relax clique assumptions.
  Redesign how directories are handled.
    - Separate running-routers lookup from descriptor list lookup.
    - Resolve directory agreement somehow.
    - Cache directory on all servers.
  Find and remove bottlenecks
    - Address linear searches on e.g. circuit and connection lists.
  Reputation/memory system, so dirservers can measure people,
    and so other people can verify their measurements.
    - Need to measure via relay, so it's not distinguishable.
  Bandwidth-aware path selection. So people with T3's are picked
    more often than people with DSL.
  Reliability-aware node selection. So people who are stable are
    preferred for long-term circuits such as intro and rend circs,
    and general circs for irc, aim, ssh, etc.
  Let dissidents get to Tor servers via Tor users. ("Backbone model")

Anonymity improvements:
  Is abandoning the circuit the only option when an extend fails, or
    can we do something without impacting anonymity too much?
  Is exiting from the middle of the circuit always a bad idea?
  Helper nodes. Decide how to use them to improve safety.
  DNS resolution: need to make tor support resolve requests. Need to write
    a script and an interface (including an extension to the socks
    protocol) so we can ask it to do resolve requests. Need to patch
    tsocks to intercept gethostbyname, else we'll continue leaking it.
  Improve path selection algorithms based on routing-zones paper. Be sure
    to start and end circuits in different ASs. Ideally, consider AS of
    source and destination -- maybe even enter and exit via nearby AS.
  Intermediate model, with some delays and mixing.
  Add defensive dropping regime?

Make it more correct:
  Handle half-open connections: right now we don't support all TCP
    streams, at least according to the protocol. But we handle all that
    we've seen in the wild.
  Support IPv6.

Efficiency/speed/robustness:
  Congestion control. Is our current design sufficient once we have heavy
    use? Need to measure and tweak, or maybe overhaul.
  Allow small cells and large cells on the same network?
  Cell buffering and resending. This will allow us to handle broken
    circuits as long as the endpoints don't break, plus will allow
    connection (tls session key) rotation.
  Implement Morphmix, so we can compare its behavior, complexity, etc.
  Use cpuworker for more heavy lifting.
    - Signing (and verifying) hidserv descriptors
    - Signing (and verifying) intro/rend requests
    - Signing (and verifying) router descriptors
    - Signing (and verifying) directories
    - Doing TLS handshake (this is very hard to separate out, though)
  Buffer size pool: allocate a maximum size for all buffers, not
    a maximum size for each buffer. So we don't have to give up as
    quickly (and kill the thickpipe!) when there's congestion.
  Exit node caching: tie into squid or other caching web proxy.
  Other transport. HTTP, udp, rdp, airhook, etc. May have to do our own
    link crypto, unless we can bully openssl into it.

P2P Tor:
  Do all the scalability stuff above, first.
  Incentives to relay. Not so hard.
  Incentives to allow exit. Possibly quite hard.
  Sybil defenses without having a human bottleneck.
  How to gather random sample of nodes.
  How to handle nodelist recommendations.
  Consider incremental switches: a p2p tor with only 50 users has
    different anonymity properties than one with 10k users, and should
    be treated differently.