Merge branch 'debian-merge' into debian

* debian-merge: (95 commits)
  New upstream version: 0.2.2.2-alpha
  downgrade a log severity, since this event has been known
  Update to the "September 4 2009" ip-to-country file.
  bump to 0.2.2.2-alpha
  Revert "Teach connection_ap_can_use_exit about Exclude*Nodes"
  fix grammar / add changelog for the torify commit
  Fix compile on Snow Leopard
  Fix build warnings on OSX 10.5.8
  Change the condition on the nonlive timeout counting.
  Add a couple of time helper functions.
  Fix typos and comments, plus two bugs
  Implement and document new network liveness algorithm.
  Fix some precision-related asserts in unit tests.
  replace contrib/auto-naming with a readme saying where it went
  clarify our rules for assigning the Named flag
  disable the end of circuitbuildtimeout units tests
  draw in a lot of 0.2.1.20 changelog items into 0.2.2.2-alpha
  Fix compile on freebsd
  Let our config abbreviations rewrite more than once
  a mish-mash of stuff in my sandbox
  ...

ChangeLog

@@ -1,3 +1,72 @@
+Changes in version 0.2.2.2-alpha - 2009-09-21
+  o Major features:
+    - Tor now tracks how long it takes to build client-side circuits
+      over time, and adapts its timeout to local network performance.
+      Since a circuit that takes a long time to build will also provide
+      bad performance, we get significant latency improvements by
+      discarding the slowest 20% of circuits. Specifically, Tor creates
+      circuits more aggressively than usual until it has enough data
+      points for a good timeout estimate. Implements proposal 151.
+      We are especially looking for reports (good and bad) from users with
+      both EDGE and broadband connections that can move from broadband
+      to EDGE and find out if the build-time data in the .tor/state gets
+      reset without loss of Tor usability. You should also see a notice
+      log message telling you that Tor has reset its timeout.
+    - Directory authorities can now vote on arbitary integer values as
+      part of the consensus process. This is designed to help set
+      network-wide parameters. Implements proposal 167.
+    - Tor now reads the "circwindow" parameter out of the consensus,
+      and uses that value for its circuit package window rather than the
+      default of 1000 cells. Begins the implementation of proposal 168.
+
+  o Major bugfixes:
+    - Fix a remotely triggerable memory leak when a consensus document
+      contains more than one signature from the same voter. Bugfix on
+      0.2.0.3-alpha.
+
+  o Minor bugfixes:
+    - Fix an extremely rare infinite recursion bug that could occur if
+      we tried to log a message after shutting down the log subsystem.
+      Found by Matt Edman. Bugfix on 0.2.0.16-alpha.
+    - Fix parsing for memory or time units given without a space between
+      the number and the unit. Bugfix on 0.2.2.1-alpha; fixes bug 1076.
+    - A networkstatus vote must contain exactly one signature. Spec
+      conformance issue. Bugfix on 0.2.0.3-alpha.
+    - Fix an obscure bug where hidden services on 64-bit big-endian
+      systems might mis-read the timestamp in v3 introduce cells, and
+      refuse to connect back to the client. Discovered by "rotor".
+      Bugfix on 0.2.1.6-alpha.
+    - We were triggering a CLOCK_SKEW controller status event whenever
+      we connect via the v2 connection protocol to any relay that has
+      a wrong clock. Instead, we should only inform the controller when
+      it's a trusted authority that claims our clock is wrong. Bugfix
+      on 0.2.0.20-rc; starts to fix bug 1074. Reported by SwissTorExit.
+    - We were telling the controller about CHECKING_REACHABILITY and
+      REACHABILITY_FAILED status events whenever we launch a testing
+      circuit or notice that one has failed. Instead, only tell the
+      controller when we want to inform the user of overall success or
+      overall failure. Bugfix on 0.1.2.6-alpha. Fixes bug 1075. Reported
+      by SwissTorExit.
+    - Don't warn when we're using a circuit that ends with a node
+      excluded in ExcludeExitNodes, but the circuit is not used to access
+      the outside world. This should help fix bug 1090, but more problems
+      remain. Bugfix on 0.2.1.6-alpha.
+    - Work around a small memory leak in some versions of OpenSSL that
+      stopped the memory used by the hostname TLS extension from being
+      freed.
+    - Make our 'torify' script more portable; if we have only one of
+      'torsocks' or 'tsocks' installed, don't complain to the user;
+      and explain our warning about tsocks better.
+
+  o Minor features:
+    - Add a "getinfo status/accepted-server-descriptor" controller
+      command, which is the recommended way for controllers to learn
+      whether our server descriptor has been successfully received by at
+      least on directory authority. Un-recommend good-server-descriptor
+      getinfo and status events until we have a better design for them.
+    - Update to the "September 4 2009" ip-to-country file.
+
+
 Changes in version 0.2.2.1-alpha - 2009-08-26
   o Security fixes:
     - Start the process of disabling ".exit" address notation, since it
@@ -87,6 +156,22 @@ Changes in version 0.2.2.1-alpha - 2009-08-26
       them, and they provided another avenue for detecting Tor users
       via application-level web tricks.
 
+  o Packaging changes:
+    - Upgrade Vidalia from 0.1.15 to 0.2.3 in the Windows and OS X
+      installer bundles. See
+      https://trac.vidalia-project.net/browser/vidalia/tags/vidalia-0.2.3/CHANGELOG
+      for details of what's new in Vidalia 0.2.3.
+    - Windows Vidalia Bundle: update Privoxy from 3.0.6 to 3.0.14-beta.
+    - OS X Vidalia Bundle: move to Polipo 1.0.4 with Tor specific
+      configuration file, rather than the old Privoxy.
+    - OS X Vidalia Bundle: Vidalia, Tor, and Polipo are compiled as
+      x86-only for better compatibility with OS X 10.6, aka Snow Leopard.
+    - OS X Tor Expert Bundle: Tor is compiled as x86-only for
+      better compatibility with OS X 10.6, aka Snow Leopard.
+    - OS X Vidalia Bundle: The multi-package installer is now replaced
+      by a simple drag and drop to the /Applications folder. This change
+      occurred with the upgrade to Vidalia 0.2.3.
+
 
 Changes in version 0.2.1.20 - 2009-??-??
   o Major bugfixes:
@@ -96,6 +181,9 @@ Changes in version 0.2.1.20 - 2009-??-??
       patch. Bugfix on the 54th commit on Tor -- from July 2002,
       before the release of Tor 0.0.0. This is the new winner of the
       oldest-bug prize.
+    - Fix a remotely triggerable memory leak when a consensus document
+      contains more than one signature from the same voter. Bugfix on
+      0.2.0.3-alpha.
 
   o New directory authorities:
     - Set up urras (run by Jacob Appelbaum) as the seventh v3 directory
@@ -105,6 +193,41 @@ Changes in version 0.2.1.20 - 2009-??-??
     - Fix a signed/unsigned compile warning in 0.2.1.19.
     - Fix possible segmentation fault on directory authorities. Bugfix on
       0.2.1.14-rc.
+    - Fix an extremely rare infinite recursion bug that could occur if
+      we tried to log a message after shutting down the log subsystem.
+      Found by Matt Edman. Bugfix on 0.2.0.16-alpha.
+    - Fix an obscure bug where hidden services on 64-bit big-endian
+      systems might mis-read the timestamp in v3 introduce cells, and
+      refuse to connect back to the client. Discovered by "rotor".
+      Bugfix on 0.2.1.6-alpha.
+    - We were triggering a CLOCK_SKEW controller status event whenever
+      we connect via the v2 connection protocol to any relay that has
+      a wrong clock. Instead, we should only inform the controller when
+      it's a trusted authority that claims our clock is wrong. Bugfix
+      on 0.2.0.20-rc; starts to fix bug 1074. Reported by SwissTorExit.
+    - We were telling the controller about CHECKING_REACHABILITY and
+      REACHABILITY_FAILED status events whenever we launch a testing
+      circuit or notice that one has failed. Instead, only tell the
+      controller when we want to inform the user of overall success or
+      overall failure. Bugfix on 0.1.2.6-alpha. Fixes bug 1075. Reported
+      by SwissTorExit.
+    - Don't warn when we're using a circuit that ends with a node
+      excluded in ExcludeExitNodes, but the circuit is not used to access
+      the outside world. This should help fix bug 1090. Bugfix on
+      0.2.1.6-alpha.
+    - Avoid segfault in rare cases when finishing an introduction circuit
+      as a client and finding out that we don't have an introduction key
+      for it. Fixes bug 1073. Reported by Aaron Swartz.
+    - Work around a small memory leak in some versions of OpenSSL that
+      stopped the memory used by the hostname TLS extension from being
+      freed.
+
+  o Minor features:
+    - Add a "getinfo status/accepted-server-descriptor" controller
+      command, which is the recommended way for controllers to learn
+      whether our server descriptor has been successfully received by at
+      least on directory authority. Un-recommend good-server-descriptor
+      getinfo and status events until we have a better design for them.
 
 
 Changes in version 0.2.1.19 - 2009-07-28
@@ -201,6 +324,37 @@ Changes in version 0.2.1.17-rc - 2009-07-07
       further bugs for relays on dynamic IP addresses.
 
 
+Changes in version 0.2.0.35 - 2009-06-24
+  o Security fix:
+    - Avoid crashing in the presence of certain malformed descriptors.
+      Found by lark, and by automated fuzzing.
+    - Fix an edge case where a malicious exit relay could convince a
+      controller that the client's DNS question resolves to an internal IP
+      address. Bug found and fixed by "optimist"; bugfix on 0.1.2.8-beta.
+
+  o Major bugfixes:
+    - Finally fix the bug where dynamic-IP relays disappear when their
+      IP address changes: directory mirrors were mistakenly telling
+      them their old address if they asked via begin_dir, so they
+      never got an accurate answer about their new address, so they
+      just vanished after a day. For belt-and-suspenders, relays that
+      don't set Address in their config now avoid using begin_dir for
+      all direct connections. Should fix bugs 827, 883, and 900.
+    - Fix a timing-dependent, allocator-dependent, DNS-related crash bug
+      that would occur on some exit nodes when DNS failures and timeouts
+      occurred in certain patterns. Fix for bug 957.
+
+  o Minor bugfixes:
+    - When starting with a cache over a few days old, do not leak
+      memory for the obsolete router descriptors in it. Bugfix on
+      0.2.0.33; fixes bug 672.
+    - Hidden service clients didn't use a cached service descriptor that
+      was older than 15 minutes, but wouldn't fetch a new one either,
+      because there was already one in the cache. Now, fetch a v2
+      descriptor unless the same descriptor was added to the cache within
+      the last 15 minutes. Fixes bug 997; reported by Marcus Griep.
+
+
 Changes in version 0.2.1.16-rc - 2009-06-20
   Tor 0.2.1.16-rc speeds up performance for fast exit relays, and fixes
   a bunch of minor bugs.

configure.in

@@ -4,7 +4,7 @@ dnl Copyright (c) 2007-2008, The Tor Project, Inc.
 dnl See LICENSE for licensing information
 
 AC_INIT
-AM_INIT_AUTOMAKE(tor, 0.2.2.1-alpha)
+AM_INIT_AUTOMAKE(tor, 0.2.2.2-alpha)
 AM_CONFIG_HEADER(orconfig.h)
 
 AC_CANONICAL_HOST

contrib/auto-naming/README

@@ -1,65 +1,6 @@
-=== AUTONAMING FOR TOR ===
-
 Tor directory authorities may maintain a binding of server identities
-(their long term identity key) and nicknames.  In their status documents
-they may for each router they know tell if this is indeed the owner of
-that nickname or not.
-
-This toolset allows automatic maintaining of a binding list of nicknames
-to identity keys, implementing Tor proposal 123[1].
-
-The rules are simple:
- - A router claiming to be Bob is named (i.e. added to the binding list)
-   if there currently does not exist a different binding for that
-   nickname, the router has been around for a bit (2 weeks), no other
-   router has used that nickname in a while (1 month).
- - A binding is removed if the server that owns it has not been seen
-   in a long time (6 months).
-
-
-=== REQUIREMENTS ===
-
- * ruby, and its postgres DBI interface (Debian packages: ruby, ruby1.8, libdbi-ruby1.8, libdbd-pg-ruby1.8)
- * postgres (tested with >= 8.1)
- * cron
-
-=== SETUP ===
-
- * copy this tree some place, like into a 'auto-naming' directory in your Tor's
-   data directory
- * create a database and a user, modifying db-config.rb accordingly
- * initialize the database by executing the sql statements in create-db.sql
- * setup a cronjob that feeds the current consensus to the process-consensus
-   script regularly.
- * once the database is sufficiently populated, maybe a month or so after the
-   previous step, setup a cronjob to regularly build the binding list using
-   the build-approved-routers script.  You probably want to append a manually
-   managed list of rejections to that file and give it to tor as its
-   "approved-routers" file.
-   The Sample-Makefile and Sample-crontab demonstrate the method used at tor26.
-
-
-1. https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/proposals/123-autonaming.txt
-
-
-
-
-Copyright (c) 2007 Peter Palfrader
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
+(their long term identity key) and nicknames.
 
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
+The auto-naming scripts have been moved to svn in
+projects/tor-naming/auto-naming/trunk/
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

contrib/auto-naming/Sample-Makefile

@@ -1,20 +0,0 @@
-
-all: ../approved-routers
-
-update:
-	wget -q -O - http://tor.noreply.org/tor/status-vote/current/consensus | \
-		./process-consensus
-
-.PHONY: approved-routers-auto
-approved-routers-auto:
-	./build-approved-routers > "$@"
-
-.INTERMEDIATE: approved-routers
-approved-routers: approved-routers-auto /etc/tor/approved-routers
-	cat $^ > "$@"
-
-../approved-routers: approved-routers
-	if ! diff -q "$<" "$@"; then \
-		mv "$<" "$@" &&\
-		(! [ -e /var/run/tor/tor.pid ] || kill -HUP `cat /var/run/tor/tor.pid`) ; \
-		fi

contrib/auto-naming/Sample-crontab

@@ -1,3 +0,0 @@
-MAILTO=admin
-# cronjob for tor naming
-23 * * * * make -s -C auto-naming update && make -s -C auto-naming

contrib/auto-naming/build-approved-routers

@@ -1,45 +0,0 @@
-#!/usr/bin/ruby
-
-# build-approved-routers - create a name-binding list for use at a Tor
-#                          directory authority
-#
-# Copyright (c) 2007 Peter Palfrader
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-
-require "yaml"
-
-require 'db'
-require 'db-config'
-
-verbose = ARGV.first == "-v"
-
-db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password'])
-
-db.transaction_begin
-named = db.query2("
-		SELECT fingerprint, router_id, nickname_id, nick, first_seen, last_seen
-		FROM router NATURAL JOIN router_claims_nickname NATURAL JOIN nickname
-		WHERE named")
-while (n=named.next) do
-	puts "# (r##{n['router_id']},n##{n['nickname_id']}); first_seen: #{n['first_seen']}, last_seen: #{n['last_seen']}"
-	fpr = n['fingerprint'].split(/(....)/).delete_if{|x| x=="" }.join(' ')
-	puts "#{n['nick']} #{fpr}"
-end
-db.transaction_commit

contrib/auto-naming/create-db.sql

@@ -1,50 +0,0 @@
-
-CREATE TABLE router (
-	router_id	SERIAL PRIMARY KEY,
-	fingerprint	CHAR(40)		NOT NULL,
-	UNIQUE(fingerprint)
-);
--- already created implicitly due to unique contraint
--- CREATE INDEX router_fingerprint ON router(fingerprint);
-
-CREATE TABLE nickname (
-	nickname_id	SERIAL PRIMARY KEY,
-	nick		VARCHAR(30)		NOT NULL,
-	UNIQUE(nick)
-);
--- already created implicitly due to unique contraint
--- CREATE INDEX nickname_nick ON nickname(nick);
-
-CREATE TABLE router_claims_nickname (
-	router_id	INTEGER		NOT NULL	REFERENCES router(router_id) ON DELETE CASCADE,
-	nickname_id	INTEGER		NOT NULL	REFERENCES nickname(nickname_id) ON DELETE CASCADE,
-	first_seen	TIMESTAMP WITH TIME ZONE	NOT NULL	DEFAULT CURRENT_TIMESTAMP,
-	last_seen	TIMESTAMP WITH TIME ZONE	NOT NULL	DEFAULT CURRENT_TIMESTAMP,
-	named		BOOLEAN				NOT NULL	DEFAULT 'false',
-	UNIQUE(router_id, nickname_id)
-);
-CREATE INDEX router_claims_nickname_router_id ON router_claims_nickname(router_id);
-CREATE INDEX router_claims_nickname_nickname_id ON router_claims_nickname(nickname_id);
-CREATE INDEX router_claims_nickname_first_seen ON router_claims_nickname(first_seen);
-CREATE INDEX router_claims_nickname_last_seen ON router_claims_nickname(last_seen);
-
-
--- Copyright (c) 2007 Peter Palfrader
---
--- Permission is hereby granted, free of charge, to any person obtaining a copy
--- of this software and associated documentation files (the "Software"), to deal
--- in the Software without restriction, including without limitation the rights
--- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
--- copies of the Software, and to permit persons to whom the Software is
--- furnished to do so, subject to the following conditions:
---
--- The above copyright notice and this permission notice shall be included in
--- all copies or substantial portions of the Software.
---
--- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
--- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
--- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
--- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
--- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
--- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
--- SOFTWARE.

contrib/auto-naming/db-config.rb

@@ -1,8 +0,0 @@
-$CONFIG             = {} unless $CONFIG
-$CONFIG['database'] = {} unless $CONFIG['database']
-
-# if you use postgres' "ident sameuser" auth set dbhost to ''
-$CONFIG['database']['dbhost'] = 'localhost';
-$CONFIG['database']['dbname'] = 'tornaming';
-$CONFIG['database']['user'] = 'tornaming';
-$CONFIG['database']['password'] = 'x';

contrib/auto-naming/db.rb

@@ -1,165 +0,0 @@
-#!/usr/bin/ruby
-
-# Copyright (c) 2006, 2007 Peter Palfrader
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-
-require "dbi"
-
-class WeaselDbQueryHandle
-	def initialize(sth)
-		@sth = sth
-	end
-
-	def next()
-		row = @sth.fetch_hash
-		if row
-			return row
-		else
-			@sth.finish
-			return nil
-		end
-	end
-end
-
-class Db
-	def initialize(host, database, user, password)
-		@dbh = DBI.connect("dbi:Pg:#{database}:#{host}", user, password);
-		@dbh['AutoCommit'] = false
-		@transaction = false
-		@pre_initial_transaction=true
-	end
-
-	def do(query,*args)
-		@dbh.do(query,*args)
-	end
-	def transaction_begin()
-		@dbh.do("BEGIN") unless @pre_initial_transaction
-		@transaction = true
-		@pre_initial_transaction=false
-	end
-	def transaction_commit()
-		@dbh.do("COMMIT")
-		@transaction = false
-	end
-	def transaction_rollback()
-		@dbh.do("ROLLBACK")
-	end
-	def get_primarykey_name(table);
-		#return 'ref';
-		return table+'_id';
-	end
-
-	def update(table, values, keys)
-		cols = []
-		vals = []
-		values.each_pair{ |k,v|
-			cols << "#{k}=?"
-			vals << v
-		}
-
-		wheres = []
-		keys.each_pair{ |k,v|
-			wheres << "#{k}=?"
-			vals << v
-		}
-
-		throw "update value set empty" unless cols.size > 0
-		throw "where clause empty" unless wheres.size > 0
-
-		query = "UPDATE #{table} SET #{cols.join(',')} WHERE #{wheres.join(' AND ')}"
-		transaction_begin unless transaction_before=@transaction
-		r = @dbh.do(query, *vals)
-		transaction_commit unless transaction_before
-		return r
-	end
-
-	def update_row(table, values)
-		pk_name = get_primarykey_name(table);
-		throw "Ref not defined" unless values[pk_name]
-		return update(table, values.clone.delete_if{|k,v| k == pk_name}, { pk_name => values[pk_name] });
-	end
-	def insert(table, values)
-		cols = values.keys
-		vals = values.values
-		qmarks = values.values.collect{ '?' }
-
-		query = "INSERT INTO #{table} (#{cols.join(',')}) VALUES (#{qmarks.join(',')})"
-		transaction_begin unless transaction_before=@transaction
-		@dbh.do(query, *vals)
-		transaction_commit unless transaction_before
-	end
-
-	def insert_row(table, values)
-		pk_name = get_primarykey_name(table);
-		if values[pk_name]
-			insert(table, values)
-		else
-			transaction_begin unless transaction_before=@transaction
-			row = query_row("SELECT nextval(pg_get_serial_sequence('#{table}', '#{pk_name}')) AS newref");
-			throw "No newref?" unless row['newref']
-			values[pk_name] = row['newref']
-			insert(table, values);
-			transaction_commit unless transaction_before
-		end
-	end
-	def delete_row(table, ref)
-		pk_name = get_primarykey_name(table);
-		query = "DELETE FROM #{table} WHERE #{pk_name}=?"
-		transaction_begin unless transaction_before=@transaction
-		@dbh.do(query, ref)
-		transaction_commit unless transaction_before
-	end
-	def query(query, *params)
-		sth = @dbh.execute(query, *params)
-		while row = sth.fetch_hash
-			yield row
-		end
-		sth.finish
-	end
-	# nil if no results
-	# hash if one match
-	# throw otherwise
-	def query_row(query, *params)
-		sth = @dbh.execute(query, *params)
-
-		row = sth.fetch_hash
-		if row == nil
-			sth.finish
-			return nil
-		elsif sth.fetch_hash != nil
-			sth.finish
-			throw "More than one result when querying for #{query}"
-		else
-			sth.finish
-			return row
-		end
-	end
-	def query_all(query, *params)
-		sth = @dbh.execute(query, *params)
-
-		rows = sth.fetch_all
-		return nil if rows.size == 0
-		return rows
-	end
-	def query2(query, *params)
-		sth = @dbh.execute(query, *params)
-		return WeaselDbQueryHandle.new(sth)
-	end
-end

contrib/auto-naming/process-consensus

@@ -1,119 +0,0 @@
-#!/usr/bin/ruby
-
-# process-consensus - read a current consensus document, inserting the
-#                     information into a database then calling
-#                     update-named-status.rb to update the name-binding
-#                     flags
-#
-# Copyright (c) 2007 Peter Palfrader
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-
-require "yaml"
-
-require 'db'
-require 'db-config'
-require 'update-named-status'
-
-$db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password'])
-
-$router_cache = {}
-$nickname_cache = {}
-
-def parse_consensus consensus
-	ts = nil
-	routers = []
-	consensus.each do |line|
-		(key, value) = line.split(' ',2)
-		case key
-			when "valid-after", "published": ts = DateTime.parse(value)
-			when "r":
-				(nick, fpr, _) = value.split(' ', 3)
-				nick.downcase!
-				next if nick == 'unnamed'
-				routers << {
-					'nick' => nick,
-					'fingerprint' => (fpr+'=').unpack('m').first.unpack('H*').first
-					}
-		end
-	end
-	throw "Did not find a timestamp" unless ts
-	throw "Did not find any routers" unless routers.size > 0
-	return ts, routers
-end
-
-def insert_routers_into_db(router, table, field, value)
-	pk = table+'_id'
-	row = $db.query_row("SELECT #{pk} FROM #{table} WHERE #{field}=?", value)
-	if row
-		return row[pk]
-	else
-		r = { field => value }
-		$db.insert_row( table, r )
-		return r[pk]
-	end
-end
-
-def handle_one_consensus(c)
-	puts "parsing..." if $verbose
-	timestamp, routers = parse_consensus c
-	puts "storing..." if $verbose
-
-	routers.each do |router|
-		fpr = router['fingerprint']
-		nick = router['nick']
-		$router_cache[fpr] = router_id = ($router_cache[fpr] or insert_routers_into_db(router, 'router', 'fingerprint', router['fingerprint']))
-		$nickname_cache[nick] = nickname_id = ($nickname_cache[nick] or insert_routers_into_db(router, 'nickname', 'nick', router['nick']))
-
-		row = $db.update(
-			'router_claims_nickname',
-			{ 'last_seen' => timestamp.to_s },
-			{ 'router_id' => router_id, 'nickname_id' => nickname_id} )
-		case row
-			when 0:
-				$db.insert('router_claims_nickname',
-					{
-						'first_seen' => timestamp.to_s,
-						'last_seen' => timestamp.to_s,
-						'router_id' => router_id, 'nickname_id' => nickname_id} )
-			when 1:
-			else
-				throw "Update of router_claims_nickname returned unexpected number of affected rows(#{row})"
-		end
-	end
-end
-
-$db.transaction_begin
-if ARGV.first == '-v'
-	$verbose = true
-	ARGV.shift
-end
-
-if ARGV.size == 0
-	handle_one_consensus STDIN.readlines
-	do_update $verbose
-else
-	ARGV.each do |filename|
-		puts filename if $verbose
-		handle_one_consensus File.new(filename).readlines
-		puts "updating..." if $verbose
-		do_update $verbose
-	end
-end
-$db.transaction_commit

contrib/auto-naming/update-named-status.rb

@@ -1,70 +0,0 @@
-#!/usr/bin/ruby
-
-# update-named-status.rb - update the named status of routers in our database
-#
-# Copyright (c) 2007 Peter Palfrader
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-
-require "yaml"
-
-require 'db'
-require 'db-config'
-
-def do_update(verbose)
-	now = $db.query_row("SELECT max(last_seen) AS max FROM router_claims_nickname")['max']
-	unless now
-		STDERR.puts "Could not find the latest last_seen timestamp.  Is the database empty still?"
-		return
-	end
-	now = "TIMESTAMP '" + now.to_s + "'"
-
-	denamed = $db.do("
-			UPDATE router_claims_nickname
-			SET named=false
-			WHERE named
-			  AND last_seen < #{now} - INTERVAL '6 months'")
-	puts "de-named: #{denamed}" if verbose
-
-	named = $db.do("
-			UPDATE router_claims_nickname
-			SET named=true
-			WHERE NOT named
-			  AND first_seen < #{now} - INTERVAL '2 weeks'
-			  AND last_seen  > #{now} - INTERVAL '2 days'
-			  AND NOT EXISTS (SELECT *
-			       FROM router_claims_nickname AS innertable
-			       WHERE named
-			         AND router_claims_nickname.nickname_id=innertable.nickname_id) "+ # if that nickname is already named, we lose.
-			" AND NOT EXISTS (SELECT *
-			       FROM router_claims_nickname AS innertable
-			       WHERE router_claims_nickname.nickname_id=innertable.nickname_id
-			         AND router_claims_nickname.router_id <> innertable.router_id
-			         AND last_seen > #{now} - INTERVAL '1 month') ") # if nobody else wanted that nickname in the last month we are set
-	puts "named: #{named}" if verbose
-end
-
-if __FILE__ == $0
-	$db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password'])
-	verbose = ARGV.first == "-v"
-
-	$db.transaction_begin
-	do_update verbose
-	$db.transaction_commit
-end

contrib/tor-mingw.nsi.in

@@ -9,7 +9,7 @@
 !include "FileFunc.nsh"
 !insertmacro GetParameters
   
-!define VERSION "0.2.2.1-alpha"
+!define VERSION "0.2.2.2-alpha"
 !define INSTALLER "tor-${VERSION}-win32.exe"
 !define WEBSITE "https://www.torproject.org/"
 !define LICENSE "LICENSE"

contrib/torify.in

@@ -3,57 +3,52 @@
 # Wrapper script for use of the tsocks(8) transparent socksification library
 # See the tsocks(1) and torify(1) manpages.
 
-# Copyright (c) 2004, 2006 Peter Palfrader
+# Copyright (c) 2004, 2006, 2009 Peter Palfrader
 # Modified by Jacob Appelbaum <jacob@appelbaum.net> April 16th 2006
 # May be distributed under the same terms as Tor itself
 
-
-# Define and ensure we have tsocks
-# XXX: what if we don't have which?
-TORSOCKS="`which torsocks`"
-TSOCKS="`which tsocks`"
-PROG=""
-if [ ! -x "$TSOCKS" ]
-then
-	echo "$0: Can't find tsocks in PATH. Perhaps you haven't installed it?" >&2
-else
-	PROG=$TSOCKS
-fi
-if [ ! -x "$TORSOCKS" ]
-then
-	echo "$0: Can't find torsocks in PATH. Perhaps you haven't installed it?" >&2
-else
-	PROG=$TORSOCKS
-fi
-
-if [ ! -x "$PROG" ]
-then
-	echo "$0: Can't find the required tor helpers in our PATH. Perhaps you haven't installed them?" >&2
-	exit 1;
-fi
+# taken from Debian's Developer's Reference, 6.4
+pathfind() {
+	OLDIFS="$IFS"
+	IFS=:
+	for p in $PATH; do
+		if [ -x "$p/$*" ]; then
+			IFS="$OLDIFS"
+			return 0
+		fi
+	done
+	IFS="$OLDIFS"
+	return 1
+}
 
 # Check for any argument list
-if [ "$#" = 0 ]
-then
+if [ "$#" = 0 ]; then
 	echo "Usage: $0 [-hv] <command> [<options>...]" >&2
 	exit 1
 fi
-if [ "$#" = 1 ] && ( [ "$1" = "-h" ] || [ "$1" = "--help" ] )
-then
+
+if [ "$#" = 1 ] && ( [ "$1" = "-h" ] || [ "$1" = "--help" ] ); then
 	echo "Usage: $0 [-hv] <command> [<options>...]"
 	exit 0
 fi
 
-if [ "$1" = "-v" ] || [ "$1" = "--verbose" ]
-then
-	echo "We're armed with the following tsocks: $TSOCKS"
-	echo "We're armed with the following torsocks: $TORSOCKS"
-	echo "We're attempting to use $PROG for all tor action."
+if [ "$1" = "-v" ] || [ "$1" = "--verbose" ]; then
+	verbose=1
 	shift 1
+else
+	verbose=0
 fi
 
-if [ "$PROG" = "$TSOCKS" ]
-then
+if pathfind torsocks; then
+	! [ "$verbose" -ge 1 ] || echo "Using torsocks as socksifier." >&2
+
+	exec torsocks "$@"
+	echo "$0: Failed to exec torsocks $@" >&2
+	exit 1
+
+elif pathfind tsocks; then
+	! [ "$verbose" -ge 1 ] || echo "Using tsocks as socksifier." >&2
+
 	# Define our tsocks config file
 	TSOCKS_CONF_FILE="/etc/tor/tor-tsocks.conf"
 	export TSOCKS_CONF_FILE
@@ -61,7 +56,7 @@ then
 	# Check that we've got a tsocks config file
 	if [ -r "$TSOCKS_CONF_FILE" ]
 	then
-		echo "WARNING: tsocks is known to leak DNS and UDP data." >&2
+		echo "WARNING: tsocks is known to leak DNS and UDP data.  If you had torsocks we would use that." >&2
 		exec tsocks "$@"
 		echo "$0: Failed to exec tsocks $@" >&2
 		exit 1
@@ -69,8 +64,8 @@ then
 		echo "$0: Missing tsocks configuration file \"$TSOCKS_CONF_FILE\"." >&2
 		exit 1
 	fi
-fi
-if [ "$PROG" = "$TORSOCKS" ]
-then
-	exec torsocks "$@"
+
+else
+	echo "$0: Can't find either tsocks or torsocks in your PATH. Perhaps you haven't installed either?" >&2
+	exit 1
 fi

debian/changelog

@@ -1,3 +1,9 @@
+tor (0.2.2.2-alpha-1) experimental; urgency=low
+
+  * New upstream version.
+
+ -- Peter Palfrader <weasel@debian.org>  Mon, 21 Sep 2009 13:15:36 +0200
+
 tor (0.2.2.1-alpha-1) experimental; urgency=low
 
   * New upstream version.

doc/HACKING

@@ -3,10 +3,7 @@
 
 0.0 The buildbot.
 
-  http://tor-buildbot.freehaven.net:8010/
-
-  - Down because nickm isn't running services at home any more. ioerror says
-    he will resurrect it.
+  https://buildbot.vidalia-project.net/one_line_per_build
 
 0.1. Useful command-lines that are non-trivial to reproduce but can
 help with tracking bugs or leaks.

doc/spec/control-spec.txt

@@ -557,6 +557,7 @@
     "status/circuit-established"
     "status/enough-dir-info"
     "status/good-server-descriptor"
+    "status/accepted-server-descriptor"
     "status/..."
       These provide the current internal Tor values for various Tor
       states. See Section 4.1.10 for explanations. (Only a few of the
@@ -1254,20 +1255,26 @@
      CLOCK_SKEW
        SKEW="+" / "-" SECONDS
        MIN_SKEW="+" / "-" SECONDS.
-       SOURCE="DIRSERV:IP:Port" / "NETWORKSTATUS:IP:PORT" / "CONSENSUS"
+       SOURCE="DIRSERV:" IP ":" Port /
+              "NETWORKSTATUS:" IP ":" Port /
+              "OR:" IP ":" Port /
+              "CONSENSUS"
          If "SKEW" is present, it's an estimate of how far we are from the
          time declared in the source.  (In other words, if we're an hour in
          the past, the value is -3600.)  "MIN_SKEW" is present, it's a lower
          bound.  If the source is a DIRSERV, we got the current time from a
          connection to a dirserver.  If the source is a NETWORKSTATUS, we
          decided we're skewed because we got a v2 networkstatus from far in
-         the future.  If the source is CONSENSUS, we decided we're skewed
-         because we got a networkstatus consensus from the future.
+         the future.  If the source is OR, the skew comes from a NETINFO
+         cell from a connection to another relay.  If the source is
+         CONSENSUS, we decided we're skewed because we got a networkstatus
+         consensus from the future.
 
-         {Controllers may want to warn the user if the skew is high, or if
-         multiple skew messages appear at severity WARN.  Controllers
-         shouldn't blindly adjust the clock, since the more accurate source
-         of skew info (DIRSERV) is currently unauthenticated.}
+         {Tor should send this message to controllers when it thinks the
+         skew is so high that it will interfere with proper Tor operation.
+         Controllers shouldn't blindly adjust the clock, since the more
+         accurate source of skew info (DIRSERV) is currently
+         unauthenticated.}
 
      BAD_LIBEVENT
      "METHOD=" libevent method
@@ -1481,18 +1488,39 @@
        We successfully uploaded our server descriptor to at least one
        of the directory authorities, with no complaints.
 
-       {This event could affect the controller's idea of server status, but
-       the controller should not interrupt the user to tell them so.}
+       {Originally, the goal of this event was to declare "every authority
+       has accepted the descriptor, so there will be no complaints
+       about it." But since some authorities might be offline, it's
+       harder to get certainty than we had thought. As such, this event
+       is equivalent to ACCEPTED_SERVER_DESCRIPTOR below. Controllers
+       should just look at ACCEPTED_SERVER_DESCRIPTOR and should ignore
+       this event for now.}
+
+     SERVER_DESCRIPTOR_STATUS
+     "STATUS=" "LISTED" / "UNLISTED"
+       We just got a new networkstatus consensus, and whether we're in
+       it or not in it has changed. Specifically, status is "listed"
+       if we're listed in it but previous to this point we didn't know
+       we were listed in a consensus; and status is "unlisted" if we
+       thought we should have been listed in it (e.g. we were listed in
+       the last one), but we're not.
+
+       {Moving from listed to unlisted is not necessarily cause for
+       alarm. The relay might have failed a few reachability tests,
+       or the Internet might have had some routing problems. So this
+       feature is mainly to let relay operators know when their relay
+       has successfully been listed in the consensus.}
+
+       [Not implemented yet. We should do this in 0.2.2.x. -RD]
 
      NAMESERVER_STATUS
      "NS=addr"
      "STATUS=" "UP" / "DOWN"
      "ERR=" message
         One of our nameservers has changed status.
-        // actually notice
 
-       {This event could affect the controller's idea of server status, but
-       the controller should not interrupt the user to tell them so.}
+        {This event could affect the controller's idea of server status, but
+        the controller should not interrupt the user to tell them so.}
 
      NAMESERVER_ALL_DOWN
         All of our nameservers have gone down.

doc/spec/dir-spec.txt

@@ -1098,6 +1098,20 @@
         enough votes were counted for the consensus for an authoritative
         opinion to have been formed about their status.
 
+    "params" SP [Parameters] NL
+
+        [At most once]
+
+        Parameter ::= Keyword '=' Int32
+        Int32 ::= A decimal integer between -2147483648 and 2147483647.
+        Parameters ::= Parameter | Parameters SP Parameter
+
+        The parameters list, if present, contains a space-separated list of
+        key-value pairs, sorted in lexical order by their keyword.  Each
+        parameter has its own meaning.
+
+        (Only included when the vote is generated with consensus-method 7 or
+        later.)
 
    The authority section of a vote contains the following items, followed
    in turn by the authority's current key certificate:
@@ -1406,6 +1420,10 @@
 
      Known-flags is the union of all flags known by any voter.
 
+     Entries are given on the "params" line for every keyword on which any
+     authority voted.  The values given are the low-median of all votes on
+     that keyword.
+
     "client-versions" and "server-versions" are sorted in ascending
      order; A version is recommended in the consensus if it is recommended
      by more than half of the voting authorities that included a
@@ -1473,6 +1491,9 @@
           a router, the authorities produce a consensus containing a 
           Bandwidth= keyword equal to the median of the Measured= votes.
 
+        * If consensus-method 7 or later is in use, the params line is
+          included in the output.
+
      The signatures at the end of a consensus document are sorted in
      ascending order by identity digest.
 

doc/spec/proposals/000-index.txt

@@ -87,6 +87,8 @@ Proposals by number:
 164  Reporting the status of server votes [OPEN]
 165  Easy migration for voting authority sets [OPEN]
 166  Including Network Statistics in Extra-Info Documents [ACCEPTED]
+167  Vote on network parameters in consensus [CLOSED]
+168  Reduce default circuit window [OPEN]
 
 
 Proposals by status:
@@ -114,6 +116,7 @@ Proposals by status:
    163  Detecting whether a connection comes from a client [for 0.2.2]
    164  Reporting the status of server votes [for 0.2.2]
    165  Easy migration for voting authority sets
+   168  Reduce default circuit window [for 0.2.2]
  ACCEPTED:
    110  Avoiding infinite length circuits [for 0.2.1.x] [in 0.2.1.3-alpha]
    117  IPv6 exits [for 0.2.1.x]
@@ -157,6 +160,7 @@ Proposals by status:
    148  Stream end reasons from the client side should be uniform [in 0.2.1.9-alpha]
    150  Exclude Exit Nodes from a circuit [in 0.2.1.3-alpha]
    152  Optionally allow exit from single-hop circuits [in 0.2.1.6-alpha]
+   167  Vote on network parameters in consensus [in 0.2.2]
  SUPERSEDED:
    112  Bring Back Pathlen Coin Weight
    113  Simplifying directory authority administration

doc/spec/proposals/151-path-selection-improvements.txt

@@ -2,13 +2,13 @@ Filename: 151-path-selection-improvements.txt
 Title: Improving Tor Path Selection
 Author: Fallon Chen, Mike Perry
 Created: 5-Jul-2008
-Status: Draft
+Status: Implemented
 
 Overview
 
   The performance of paths selected can be improved by adjusting the
   CircuitBuildTimeout and avoiding failing guard nodes. This proposal
-  describes a method of tracking buildtime statistics at the client, and 
+  describes a method of tracking buildtime statistics at the client, and
   using those statistics to adjust the CircuitBuildTimeout.
 
 Motivation
@@ -20,121 +20,123 @@ Motivation
 
 Implementation
 
-  Storing Build Times
+  Gathering Build Times
 
-    Circuit build times will be stored in the circular array
-    'circuit_build_times' consisting of uint16_t elements as milliseconds.
-    The total size of this array will be based on the number of circuits
+    Circuit build times are stored in the circular array
+    'circuit_build_times' consisting of uint32_t elements as milliseconds.
+    The total size of this array is based on the number of circuits
     it takes to converge on a good fit of the long term distribution of
     the circuit builds for a fixed link. We do not want this value to be
     too large, because it will make it difficult for clients to adapt to
     moving between different links.
 
-    From our initial observations, this value appears to be on the order 
-    of 1000, but will be configurable in a #define NCIRCUITS_TO_OBSERVE.
-    The exact value for this #define will be determined by performing
-    goodness of fit tests using measurments obtained from the shufflebt.py
-    script from TorFlow.
- 
+    From our observations, the minimum value for a reasonable fit appears
+    to be on the order of 500 (MIN_CIRCUITS_TO_OBSERVE). However, to keep
+    a good fit over the long term, we store 5000 most recent circuits in
+    the array (NCIRCUITS_TO_OBSERVE).
+
+    The Tor client will build test circuits at a rate of one per
+    minute (BUILD_TIMES_TEST_FREQUENCY) up to the point of
+    MIN_CIRCUITS_TO_OBSERVE. This allows a fresh Tor to have
+    a CircuitBuildTimeout estimated within 8 hours after install,
+    upgrade, or network change (see below).
+
   Long Term Storage
 
-    The long-term storage representation will be implemented by storing a 
-    histogram with BUILDTIME_BIN_WIDTH millisecond buckets (default 50) when 
-    writing out the statistics to disk. The format of this histogram on disk 
-    is yet to be finalized, but it will likely be of the format 
-    'CircuitBuildTime <bin> <count>', with the total specified as 
-    'TotalBuildTimes <total>'
+    The long-term storage representation is implemented by storing a
+    histogram with BUILDTIME_BIN_WIDTH millisecond buckets (default 50) when
+    writing out the statistics to disk. The format this takes in the
+    state file is 'CircuitBuildTime <bin-ms> <count>', with the total
+    specified as 'TotalBuildTimes <total>'
     Example:
 
     TotalBuildTimes 100
-    CircuitBuildTimeBin 1 50
-    CircuitBuildTimeBin 2 25
-    CircuitBuildTimeBin 3 13
+    CircuitBuildTimeBin 25 50
+    CircuitBuildTimeBin 75 25
+    CircuitBuildTimeBin 125 13
     ...
 
-    Reading the histogram in will entail multiplying each bin by the 
-    BUILDTIME_BIN_WIDTH and then inserting <count> values into the 
-    circuit_build_times array each with the value of
-    <bin>*BUILDTIME_BIN_WIDTH. In order to evenly distribute the 
-    values in the circular array, a form of index skipping must
-    be employed. Values from bin #N with bin count C and total T
-    will occupy indexes specified by N+((T/C)*k)-1, where k is the
-    set of integers ranging from 0 to C-1.
-
-    For example, this would mean that the values from bin 1 would
-    occupy indexes 1+(100/50)*k-1, or 0, 2, 4, 6, 8, 10 and so on.
-    The values for bin 2 would occupy positions 1, 5, 9, 13. Collisions
-    will be inserted at the first empty position in the array greater 
-    than the selected index (which may requiring looping around the 
-    array back to index 0).
+    Reading the histogram in will entail inserting <count> values
+    into the circuit_build_times array each with the value of
+    <bin-ms> milliseconds. In order to evenly distribute the values
+    in the circular array, the Fisher-Yates shuffle will be performed
+    after reading values from the bins.
 
   Learning the CircuitBuildTimeout
 
     Based on studies of build times, we found that the distribution of
-    circuit buildtimes appears to be a Pareto distribution. 
+    circuit buildtimes appears to be a Frechet distribution. However,
+    estimators and quantile functions of the Frechet distribution are
+    difficult to work with and slow to converge. So instead, since we
+    are only interested in the accuracy of the tail, we approximate
+    the tail of the distribution with a Pareto curve starting at
+    the mode of the circuit build time sample set.
 
     We will calculate the parameters for a Pareto distribution
     fitting the data using the estimators at
     http://en.wikipedia.org/wiki/Pareto_distribution#Parameter_estimation.
 
-    The timeout itself will be calculated by solving the CDF for the 
-    a percentile cutoff BUILDTIME_PERCENT_CUTOFF. This value
-    represents the percentage of paths the Tor client will accept out of
-    the total number of paths. We have not yet determined a good
-    cutoff for this mathematically, but 85% seems a good choice for now.
+    The timeout itself is calculated by using the Quartile function (the
+    inverted CDF) to give us the value on the CDF such that
+    BUILDTIME_PERCENT_CUTOFF (80%) of the mass of the distribution is
+    below the timeout value.
+
+    Thus, we expect that the Tor client will accept the fastest 80% of
+    the total number of paths on the network.
+
+  Detecting Changing Network Conditions
 
-    From http://en.wikipedia.org/wiki/Pareto_distribution#Definition,
-    the calculation we need is pow(BUILDTIME_PERCENT_CUTOFF/100.0, k)/Xm. 
+    We attempt to detect both network connectivity loss and drastic
+    changes in the timeout characteristics.
+
+    We assume that we've had network connectivity loss if 3 circuits
+    timeout and we've received no cells or TLS handshakes since those
+    circuits began. We then set the timeout to 60 seconds and stop
+    counting timeouts.
+
+    If 3 more circuits timeout and the network still has not been
+    live within this new 60 second timeout window, we then discard
+    the previous timeouts during this period from our history.
+
+    To detect changing network conditions, we keep a history of
+    the timeout or non-timeout status of the past RECENT_CIRCUITS (20)
+    that successfully completed at least one hop. If more than 75%
+    of these circuits timeout, we discard all buildtimes history,
+    reset the timeout to 60, and then begin recomputing the timeout.
 
   Testing
 
     After circuit build times, storage, and learning are implemented,
     the resulting histogram should be checked for consistency by
-    verifying it persists across successive Tor invocations where 
+    verifying it persists across successive Tor invocations where
     no circuits are built. In addition, we can also use the existing
-    buildtime scripts to record build times, and verify that the histogram 
+    buildtime scripts to record build times, and verify that the histogram
     the python produces matches that which is output to the state file in Tor,
     and verify that the Pareto parameters and cutoff points also match.
-  
-  Soft timeout vs Hard Timeout
-   
-    At some point, it may be desirable to change the cutoff from a 
-    single hard cutoff that destroys the circuit to a soft cutoff and
-    a hard cutoff, where the soft cutoff merely triggers the building
-    of a new circuit, and the hard cutoff triggers destruction of the 
-    circuit.
-
-    Good values for hard and soft cutoffs seem to be 85% and 65% 
-    respectively, but we should eventually justify this with observation.
-
-  When to Begin Calculation
 
-    The number of circuits to observe (NCIRCUITS_TO_CUTOFF) before 
-    changing the CircuitBuildTimeout will be tunable via a #define. From 
-    our measurements, a good value for NCIRCUITS_TO_CUTOFF appears to be 
-    on the order of 100.
+    We will also verify that there are no unexpected large deviations from
+    node selection, such as nodes from distant geographical locations being
+    completely excluded.
 
   Dealing with Timeouts
 
-    Timeouts should be counted as the expectation of the region of 
-    of the Pareto distribution beyond the cutoff. The proposal will
-    be updated with this value soon.
+    Timeouts should be counted as the expectation of the region of
+    of the Pareto distribution beyond the cutoff. This is done by
+    generating a random sample for each timeout at points on the
+    curve beyond the current timeout cutoff.
 
-    Also, in the event of network failure, the observation mechanism 
-    should stop collecting timeout data.
+  Future Work
 
-  Client Hints
+    At some point, it may be desirable to change the cutoff from a
+    single hard cutoff that destroys the circuit to a soft cutoff and
+    a hard cutoff, where the soft cutoff merely triggers the building
+    of a new circuit, and the hard cutoff triggers destruction of the
+    circuit.
 
-    Some research still needs to be done to provide initial values
-    for CircuitBuildTimeout based on values learned from modem
-    users, DSL users, Cable Modem users, and dedicated links. A 
-    radiobutton in Vidalia should eventually be provided that
-    sets CircuitBuildTimeout to one of these values and also 
-    provide the option of purging all learned data, should any exist.
+    It may also be beneficial to learn separate timeouts for each
+    guard node, as they will have slightly different distributions.
+    This will take longer to generate initial values though.
 
-    These values can either be published in the directory, or
-    shipped hardcoded for a particular Tor version.
-    
 Issues
 
   Impact on anonymity

doc/spec/proposals/167-params-in-consensus.txt

@@ -2,8 +2,8 @@ Filename: 167-params-in-consensus.txt
 Title: Vote on network parameters in consensus
 Author: Roger Dingledine
 Created: 18-Aug-2009
-Status: Open
-Target: 0.2.2
+Status: Closed
+Implemented-In: 0.2.2
 
 0. History
 

doc/tor-osx-dmg-creation.txt

@@ -1,31 +1,62 @@
 ## Instructions for building the official dmgs for OSX.
 ##
+## The loose table of contents:
+## Summary
+## Single Architecture Binaries for PPC or X86, not both.
+## Backwards compatible single-architecture binaries for OSX x86 10.4 from newer versions of OS X.
+## Universal Binaries for OSX PPC and X86
+## Each section is delineated by ###.
 
 The following steps are the exact steps used to produce the "official"
 OSX builds of tor.
 
-Summary:
+### Summary:
 1) Compile and install a static version of the latest release of
 libevent.
 2) Acquire and install your preferred version of tor. Extract.
 3) "make dist-osx"
 4) You now have a dmg from which you can install Tor.
 
-## Universal Binaries for OSX PPC and X86
-## This method works in OSX 10.4 (Tiger) and newer OSX versions.
-## See far below if you don't care about cross compiling for PPC and X86.
-## The single architecture process starts with "###"
+### Single Architecture Binaries for PPC or X86, not both.
+### This method works in all versions of OSX 10.3 through 10.6
+
+## Compiling libevent ##
+
+1)  Download the latest stable libevent from
+http://www.monkey.org/~provos/libevent/
+
+2) The first step of compiling libevent is to configure it as
+follows:
+       ./configure --enable-static --disable-shared
+
+3) Complete the "make" and "make install".  You will need to be root,
+or sudo -s, to complete the "make install".
+
+## Compiling Tor ##
+
+4) Get your preferred version of the tor source from https://www.torproject.org.  Extract the
+tarball.
+
+5) In the top level, this means /path/to/tor/, not tor/contrib/osx,
+do a configure with these parameters:
+     CONFDIR=/Library/Tor ./configure --prefix=/Library/Tor \
+     --bindir=/Library/Tor --sysconfdir=/Library
+
+6) In same top level dir, do a "make dist-osx".  There now exists a
+.dmg file in the same directory.  Install from this dmg.
+
+### Backwards compatible single-architecture binaries for OSX x86 10.4 from newer versions of OS X.
 
 1) Install the latest XCode updates available from http://developer.apple.com.
 
-## Compiling libevent
+## Compiling libevent ##
 
 2)  Download latest stable libevent from
 http://www.monkey.org/~provos/libevent/
 
 3) The first step of compiling libevent is to configure it as
 follows:
-CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc" \
+CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386" \
 LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \
 ./configure --enable-static --disable-shared --disable-dependency-tracking
 
@@ -37,16 +68,14 @@ or sudo -s, to complete the "make install".
 	"file /usr/local/lib/libevent.a"
 
 	Your output should be:
-/usr/local/lib/libevent.a: Mach-O fat file with 2 architectures
 /usr/local/lib/libevent.a (for architecture i386):      current ar archive random library
-/usr/local/lib/libevent.a (for architecture ppc):       current ar archive
 
 6) Get your preferred version of the tor source from https://www.torproject.org/download.  
 Extract the tarball.
 
 7) In the top level, this means /path/to/tor/, not tor/contrib/osx,
 do a configure with these parameters:
-CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc" \
+CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386" \
 LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \
 CONFDIR=/Library/Tor \
 ./configure --prefix=/Library/Tor --bindir=/Library/Tor \
@@ -56,39 +85,61 @@ CONFDIR=/Library/Tor \
 
 9) Confirm you have created a universal binary by issuing the follow command:
 "file src/or/tor".  Its output should be as follows:
-src/or/tor: Mach-O fat file with 2 architectures
 src/or/tor (for architecture i386):     Mach-O executable i386
-src/or/tor (for architecture ppc):      Mach-O executable ppc
 
 10) There should exist in the top-level directory a
 Tor-$VERSION-universal-Bundle.dmg
 
-11) Congrats.  You have a universal binary. You are now ready to install Tor.
+11) Congrats.  You have a backwards-compatible binary. You are now ready to install Tor.
 
-### Single Architecture Binaries for PPC or X86, not both.
-### This method works in all versions of OSX 10.3 through 10.5
+### Universal Binaries for OSX PPC and X86
+### This method works in OSX 10.4 (Tiger) and newer OSX versions.
+
+1) Install the latest XCode updates available from http://developer.apple.com.
 
-### Compiling libevent
+## Compiling libevent ##
 
-1)  Download the latest stable libevent from
+2)  Download latest stable libevent from
 http://www.monkey.org/~provos/libevent/
 
-2) The first step of compiling libevent is to configure it as
+3) The first step of compiling libevent is to configure it as
 follows:
-       ./configure --enable-static --disable-shared
+CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc" \
+LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \
+./configure --enable-static --disable-shared --disable-dependency-tracking
 
-3) Complete the "make" and "make install".  You will need to be root,
+4) Complete the "make" and "make install".  You will need to be root,
 or sudo -s, to complete the "make install".
 
-### Compiling Tor
+5) Check for a successful universal binary of libevent.a in, by default,
+/usr/local/lib by using the following command:
+	"file /usr/local/lib/libevent.a"
 
-4) Get your preferred version of the tor source from https://www.torproject.org.  Extract the
-tarball.
+	Your output should be:
+/usr/local/lib/libevent.a: Mach-O fat file with 2 architectures
+/usr/local/lib/libevent.a (for architecture i386):      current ar archive random library
+/usr/local/lib/libevent.a (for architecture ppc):       current ar archive
 
-5) In the top level, this means /path/to/tor/, not tor/contrib/osx,
+6) Get your preferred version of the tor source from https://www.torproject.org/download.  
+Extract the tarball.
+
+7) In the top level, this means /path/to/tor/, not tor/contrib/osx,
 do a configure with these parameters:
-     CONFDIR=/Library/Tor ./configure --prefix=/Library/Tor \
-     --bindir=/Library/Tor --sysconfdir=/Library
+CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc" \
+LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \
+CONFDIR=/Library/Tor \
+./configure --prefix=/Library/Tor --bindir=/Library/Tor \
+--sysconfdir=/Library --disable-dependency-tracking
 
-6) In same top level dir, do a "make dist-osx".  There now exists a
-.dmg file in the same directory.  Install from this dmg.
+8) "make dist-osx"
+
+9) Confirm you have created a universal binary by issuing the follow command:
+"file src/or/tor".  Its output should be as follows:
+src/or/tor: Mach-O fat file with 2 architectures
+src/or/tor (for architecture i386):     Mach-O executable i386
+src/or/tor (for architecture ppc):      Mach-O executable ppc
+
+10) There should exist in the top-level directory a
+Tor-$VERSION-universal-Bundle.dmg
+
+11) Congrats.  You have a universal binary. You are now ready to install Tor.

doc/tor.1.in

@@ -1243,6 +1243,11 @@ When this is set then
 \fBVersioningAuthoritativeDirectory\fP should be set too.
 .LP
 .TP
+\fBConsensusParams \fR\fISTRING\fP
+STRING is a space-separated list of key=value pairs that Tor will
+include in the "params" line of its networkstatus vote.
+.LP
+.TP
 \fBDirAllowPrivateAddresses \fR\fB0\fR|\fB1\fR\fP
 If set to 1, Tor will accept router descriptors with arbitrary "Address"
 elements. Otherwise, if the address is not an IP address or is a private
@@ -1514,7 +1519,7 @@ The most recently downloaded network status document for each authority.  Each f
 .LP
 .TP
 .B \fIDataDirectory\fB/cached-descriptors\fR and \fBcached-descriptors.new\fR
-These files hold downloaded router statuses.  Some routers may appear more than once; if so, the most recently published descriptor is used.    Lines beginning with @-signs are annotations that contain more information about a given router.  The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-routers file.
+These files hold downloaded router statuses.  Some routers may appear more than once; if so, the most recently published descriptor is used.    Lines beginning with @-signs are annotations that contain more information about a given router.  The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-descriptors file.
 .LP
 .TP
 .B \fIDataDirectory\fB/cached-routers\fR and \fBcached-routers.new\fR

src/common/address.c

@@ -373,10 +373,11 @@ tor_addr_parse_reverse_lookup_name(tor_addr_t *result, const char *address,
       return -1; /* malformed. */
 
     /* reverse the bytes */
-    inaddr.s_addr = (((inaddr.s_addr & 0x000000fful) << 24)
-                     |((inaddr.s_addr & 0x0000ff00ul) << 8)
-                     |((inaddr.s_addr & 0x00ff0000ul) >> 8)
-                     |((inaddr.s_addr & 0xff000000ul) >> 24));
+    inaddr.s_addr = (uint32_t)
+      (((inaddr.s_addr & 0x000000ff) << 24)
+       |((inaddr.s_addr & 0x0000ff00) << 8)
+       |((inaddr.s_addr & 0x00ff0000) >> 8)
+       |((inaddr.s_addr & 0xff000000) >> 24));
 
     if (result) {
       tor_addr_from_in(result, &inaddr);

src/common/compat.c

@@ -480,8 +480,8 @@ get_uint32(const char *cp)
   return v;
 }
 /**
- * Read a 32-bit value beginning at <b>cp</b>.  Equivalent to
- * *(uint32_t*)(cp), but will not cause segfaults on platforms that forbid
+ * Read a 64-bit value beginning at <b>cp</b>.  Equivalent to
+ * *(uint64_t*)(cp), but will not cause segfaults on platforms that forbid
  * unaligned memory access.
  */
 uint64_t

src/common/container.c

@@ -1220,6 +1220,7 @@ IMPLEMENT_ORDER_FUNC(find_nth_int, int)
 IMPLEMENT_ORDER_FUNC(find_nth_time, time_t)
 IMPLEMENT_ORDER_FUNC(find_nth_double, double)
 IMPLEMENT_ORDER_FUNC(find_nth_uint32, uint32_t)
+IMPLEMENT_ORDER_FUNC(find_nth_int32, int32_t)
 IMPLEMENT_ORDER_FUNC(find_nth_long, long)
 
 /** Return a newly allocated digestset_t, optimized to hold a total of

src/common/container.h

@@ -627,6 +627,7 @@ void digestset_free(digestset_t* set);
 int find_nth_int(int *array, int n_elements, int nth);
 time_t find_nth_time(time_t *array, int n_elements, int nth);
 double find_nth_double(double *array, int n_elements, int nth);
+int32_t find_nth_int32(int32_t *array, int n_elements, int nth);
 uint32_t find_nth_uint32(uint32_t *array, int n_elements, int nth);
 long find_nth_long(long *array, int n_elements, int nth);
 static INLINE int
@@ -649,6 +650,11 @@ median_uint32(uint32_t *array, int n_elements)
 {
   return find_nth_uint32(array, n_elements, (n_elements-1)/2);
 }
+static INLINE int32_t
+median_int32(int32_t *array, int n_elements)
+{
+  return find_nth_int32(array, n_elements, (n_elements-1)/2);
+}
 static INLINE long
 median_long(long *array, int n_elements)
 {

src/common/log.c

@@ -92,7 +92,8 @@ should_log_function_name(log_domain_mask_t domain, int severity)
 }
 
 /** A mutex to guard changes to logfiles and logging. */
-static tor_mutex_t *log_mutex = NULL;
+static tor_mutex_t log_mutex;
+static int log_mutex_initialized = 0;
 
 /** Linked list of logfile_t. */
 static logfile_t *logfiles = NULL;
@@ -103,9 +104,9 @@ static int syslog_count = 0;
 #endif
 
 #define LOCK_LOGS() STMT_BEGIN                                          \
-  tor_mutex_acquire(log_mutex);                                         \
+  tor_mutex_acquire(&log_mutex);                                        \
   STMT_END
-#define UNLOCK_LOGS() STMT_BEGIN tor_mutex_release(log_mutex); STMT_END
+#define UNLOCK_LOGS() STMT_BEGIN tor_mutex_release(&log_mutex); STMT_END
 
 /** What's the lowest log level anybody cares about?  Checking this lets us
  * bail out early from log_debug if we aren't debugging.  */
@@ -146,8 +147,8 @@ _log_prefix(char *buf, size_t buf_len, int severity)
   t = (time_t)now.tv_sec;
 
   n = strftime(buf, buf_len, "%b %d %H:%M:%S", tor_localtime_r(&t, &tm));
-  r = tor_snprintf(buf+n, buf_len-n, ".%.3ld [%s] ",
-                   (long)now.tv_usec / 1000, sev_to_string(severity));
+  r = tor_snprintf(buf+n, buf_len-n, ".%.3i [%s] ",
+                   (int)now.tv_usec / 1000, sev_to_string(severity));
   if (r<0)
     return buf_len-1;
   else
@@ -446,8 +447,9 @@ logs_free_all(void)
     log_free(victim);
   }
   tor_free(appname);
-  tor_mutex_free(log_mutex);
-  log_mutex = NULL;
+
+  /* We _could_ destroy the log mutex here, but that would screw up any logs
+   * that happened between here and the end of execution. */
 }
 
 /** Remove and free the log entry <b>victim</b> from the linked-list
@@ -543,8 +545,10 @@ add_stream_log(const log_severity_list_t *severity,
 void
 init_logging(void)
 {
-  if (!log_mutex)
-    log_mutex = tor_mutex_new();
+  if (!log_mutex_initialized) {
+    tor_mutex_init(&log_mutex);
+    log_mutex_initialized = 1;
+  }
 }
 
 /** Add a log handler to receive messages during startup (before the real

src/common/torint.h

@@ -117,6 +117,9 @@ typedef unsigned int uint32_t;
 #ifndef INT32_MAX
 #define INT32_MAX 0x7fffffffu
 #endif
+#ifndef INT32_MIN
+#define INT32_MIN (-2147483647-1)
+#endif
 #endif
 
 #if (SIZEOF_LONG == 4)

src/common/tortls.c

@@ -828,6 +828,9 @@ tor_tls_new(int sock, int isServer)
   if (!SSL_set_cipher_list(result->ssl,
                      isServer ? SERVER_CIPHER_LIST : CLIENT_CIPHER_LIST)) {
     tls_log_errors(NULL, LOG_WARN, "setting ciphers");
+#ifdef SSL_set_tlsext_host_name
+    SSL_set_tlsext_host_name(result->ssl, NULL);
+#endif
     SSL_free(result->ssl);
     tor_free(result);
     return NULL;
@@ -838,6 +841,9 @@ tor_tls_new(int sock, int isServer)
   bio = BIO_new_socket(sock, BIO_NOCLOSE);
   if (! bio) {
     tls_log_errors(NULL, LOG_WARN, "opening BIO");
+#ifdef SSL_set_tlsext_host_name
+    SSL_set_tlsext_host_name(result->ssl, NULL);
+#endif
     SSL_free(result->ssl);
     tor_free(result);
     return NULL;
@@ -918,6 +924,9 @@ tor_tls_free(tor_tls_t *tls)
   if (!removed) {
     log_warn(LD_BUG, "Freeing a TLS that was not in the ssl->tls map.");
   }
+#ifdef SSL_set_tlsext_host_name
+  SSL_set_tlsext_host_name(tls->ssl, NULL);
+#endif
   SSL_free(tls->ssl);
   tls->ssl = NULL;
   tls->negotiated_callback = NULL;
@@ -1442,8 +1451,8 @@ tor_tls_used_v1_handshake(tor_tls_t *tls)
  * buffer and *<b>wbuf_bytes</b> to the amount actually used. */
 void
 tor_tls_get_buffer_sizes(tor_tls_t *tls,
-                         int *rbuf_capacity, int *rbuf_bytes,
-                         int *wbuf_capacity, int *wbuf_bytes)
+                         size_t *rbuf_capacity, size_t *rbuf_bytes,
+                         size_t *wbuf_capacity, size_t *wbuf_bytes)
 {
   if (tls->ssl->s3->rbuf.buf)
     *rbuf_capacity = tls->ssl->s3->rbuf.len;

src/common/tortls.h

@@ -73,8 +73,8 @@ void tor_tls_get_n_raw_bytes(tor_tls_t *tls,
                              size_t *n_read, size_t *n_written);
 
 void tor_tls_get_buffer_sizes(tor_tls_t *tls,
-                              int *rbuf_capacity, int *rbuf_bytes,
-                              int *wbuf_capacity, int *wbuf_bytes);
+                              size_t *rbuf_capacity, size_t *rbuf_bytes,
+                              size_t *wbuf_capacity, size_t *wbuf_bytes);
 
 int tor_tls_used_v1_handshake(tor_tls_t *tls);
 

src/common/util.c

@@ -735,7 +735,18 @@ tor_parse_ulong(const char *s, int base, unsigned long min,
   CHECK_STRTOX_RESULT();
 }
 
-/** As tor_parse_log, but return a unit64_t.  Only base 10 is guaranteed to
+/** As tor_parse_long(), but return a double. */
+double
+tor_parse_double(const char *s, double min, double max, int *ok, char **next)
+{
+  char *endptr;
+  double r;
+
+  r = strtod(s, &endptr);
+  CHECK_STRTOX_RESULT();
+}
+
+/** As tor_parse_long, but return a uint64_t.  Only base 10 is guaranteed to
  * work for now. */
 uint64_t
 tor_parse_uint64(const char *s, int base, uint64_t min,
@@ -1023,6 +1034,42 @@ wrap_string(smartlist_t *out, const char *string, size_t width,
  * Time
  * ===== */
 
+/**
+ * Converts struct timeval to a double value.
+ * Preserves microsecond precision, but just barely.
+ * Error is approx +/- 0.1 usec when dealing with epoch values.
+ */
+double
+tv_to_double(const struct timeval *tv)
+{
+  double conv = tv->tv_sec;
+  conv += tv->tv_usec/1000000.0;
+  return conv;
+}
+
+/**
+ * Converts timeval to milliseconds.
+ */
+int64_t
+tv_to_msec(const struct timeval *tv)
+{
+  int64_t conv = ((int64_t)tv->tv_sec)*1000L;
+  /* Round ghetto-style */
+  conv += ((int64_t)tv->tv_usec+500)/1000L;
+  return conv;
+}
+
+/**
+ * Converts timeval to microseconds.
+ */
+int64_t
+tv_to_usec(const struct timeval *tv)
+{
+  int64_t conv = ((int64_t)tv->tv_sec)*1000000L;
+  conv += tv->tv_usec;
+  return conv;
+}
+
 /** Return the number of microseconds elapsed between *start and *end.
  */
 long
@@ -1055,7 +1102,9 @@ tv_mdiff(const struct timeval *start, const struct timeval *end)
     return LONG_MAX;
   }
 
-  mdiff = secdiff*1000L + (end->tv_usec - start->tv_usec) / 1000L;
+  /* Subtract and round */
+  mdiff = secdiff*1000L +
+      ((long)end->tv_usec - (long)start->tv_usec + 500L) / 1000L;
   return mdiff;
 }
 
@@ -1865,7 +1914,8 @@ write_chunks_to_file_impl(const char *fname, const smartlist_t *chunks,
                           int open_flags)
 {
   open_file_t *file = NULL;
-  int fd, result;
+  int fd;
+  ssize_t result;
   fd = start_writing_to_file(fname, open_flags, 0600, &file);
   if (fd<0)
     return -1;
@@ -1950,7 +2000,7 @@ read_file_to_str(const char *filename, int flags, struct stat *stat_out)
   int fd; /* router file */
   struct stat statbuf;
   char *string;
-  int r;
+  ssize_t r;
   int bin = flags & RFTS_BIN;
 
   tor_assert(filename);
@@ -2009,7 +2059,7 @@ read_file_to_str(const char *filename, int flags, struct stat *stat_out)
        * match for size. */
       int save_errno = errno;
       log_warn(LD_FS,"Could read only %d of %ld bytes of file \"%s\".",
-               r, (long)statbuf.st_size,filename);
+               (int)r, (long)statbuf.st_size,filename);
       tor_free(string);
       close(fd);
       errno = save_errno;

src/common/util.h

@@ -182,6 +182,8 @@ long tor_parse_long(const char *s, int base, long min,
                     long max, int *ok, char **next);
 unsigned long tor_parse_ulong(const char *s, int base, unsigned long min,
                               unsigned long max, int *ok, char **next);
+double tor_parse_double(const char *s, double min, double max, int *ok,
+                        char **next);
 uint64_t tor_parse_uint64(const char *s, int base, uint64_t min,
                          uint64_t max, int *ok, char **next);
 const char *hex_str(const char *from, size_t fromlen) ATTR_NONNULL((1));
@@ -210,6 +212,9 @@ void base16_encode(char *dest, size_t destlen, const char *src, size_t srclen);
 int base16_decode(char *dest, size_t destlen, const char *src, size_t srclen);
 
 /* Time helpers */
+double tv_to_double(const struct timeval *tv);
+int64_t tv_to_msec(const struct timeval *tv);
+int64_t tv_to_usec(const struct timeval *tv);
 long tv_udiff(const struct timeval *start, const struct timeval *end);
 long tv_mdiff(const struct timeval *start, const struct timeval *end);
 time_t tor_timegm(struct tm *tm);

src/or/Makefile.am

@@ -41,14 +41,14 @@ AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \
 tor_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ @TOR_LDFLAGS_libevent@
 tor_LDADD = ../common/libor.a ../common/libor-crypto.a \
 	../common/libor-event.a \
-	-lz -levent -lssl -lcrypto @TOR_LIB_WS32@ @TOR_LIB_GDI@
+	-lz -lm -levent -lssl -lcrypto @TOR_LIB_WS32@ @TOR_LIB_GDI@
 test_SOURCES = $(COMMON_SRC) test_data.c test.c
 
 test_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \
         @TOR_LDFLAGS_libevent@
 test_LDADD = ../common/libor.a ../common/libor-crypto.a \
 	../common/libor-event.a \
-	-lz -levent -lssl -lcrypto @TOR_LIB_WS32@ @TOR_LIB_GDI@
+	-lz -lm -levent -lssl -lcrypto @TOR_LIB_WS32@ @TOR_LIB_GDI@
 
 noinst_HEADERS = or.h eventdns.h eventdns_tor.h micro-revision.i
 

src/or/circuitlist.c

@@ -361,14 +361,27 @@ circuit_purpose_to_controller_string(uint8_t purpose)
   }
 }
 
+/** Pick a reasonable package_window to start out for our circuits.
+ * Originally this was hard-coded at 1000, but now the consensus votes
+ * on the answer. See proposal 168. */
+int32_t
+circuit_initial_package_window(void)
+{
+  networkstatus_t *consensus = networkstatus_get_latest_consensus();
+  if (consensus)
+    return networkstatus_get_param(consensus, "circwindow", CIRCWINDOW_START);
+  return CIRCWINDOW_START;
+}
+
 /** Initialize the common elements in a circuit_t, and add it to the global
  * list. */
 static void
 init_circuit_base(circuit_t *circ)
 {
   circ->timestamp_created = time(NULL);
+  tor_gettimeofday(&circ->highres_created);
 
-  circ->package_window = CIRCWINDOW_START;
+  circ->package_window = circuit_initial_package_window();
   circ->deliver_window = CIRCWINDOW_START;
 
   circuit_add(circ);
@@ -395,6 +408,8 @@ origin_circuit_new(void)
 
   init_circuit_base(TO_CIRCUIT(circ));
 
+  circ_times.last_circ_at = approx_time();
+
   return circ;
 }
 

src/or/circuituse.c

@@ -20,6 +20,8 @@ extern circuit_t *global_circuitlist; /* from circuitlist.c */
 static void circuit_expire_old_circuits(time_t now);
 static void circuit_increment_failure_count(void);
 
+long int lround(double x);
+
 /** Return 1 if <b>circ</b> could be returned by circuit_get_best().
  * Else return 0.
  */
@@ -263,16 +265,18 @@ circuit_conforms_to_options(const origin_circuit_t *circ,
 void
 circuit_expire_building(time_t now)
 {
-  circuit_t *victim, *circ = global_circuitlist;
-  time_t general_cutoff = now - get_options()->CircuitBuildTimeout;
-  time_t begindir_cutoff = now - get_options()->CircuitBuildTimeout/2;
+  circuit_t *victim, *next_circ = global_circuitlist;
+  /* circ_times.timeout is BUILD_TIMEOUT_INITIAL_VALUE if we haven't
+   * decided on a customized one yet */
+  time_t general_cutoff = now - lround(circ_times.timeout_ms/1000);
+  time_t begindir_cutoff = now - lround(circ_times.timeout_ms/2000);
   time_t introcirc_cutoff = begindir_cutoff;
   cpath_build_state_t *build_state;
 
-  while (circ) {
+  while (next_circ) {
     time_t cutoff;
-    victim = circ;
-    circ = circ->next;
+    victim = next_circ;
+    next_circ = next_circ->next;
     if (!CIRCUIT_IS_ORIGIN(victim) || /* didn't originate here */
         victim->marked_for_close) /* don't mess with marked circs */
       continue;
@@ -343,6 +347,12 @@ circuit_expire_building(time_t now)
             continue;
           break;
       }
+    } else { /* circuit not open, consider recording failure as timeout */
+      int first_hop_succeeded = TO_ORIGIN_CIRCUIT(victim)->cpath &&
+            TO_ORIGIN_CIRCUIT(victim)->cpath->state == CPATH_STATE_OPEN;
+      if (circuit_build_times_add_timeout(&circ_times, first_hop_succeeded,
+                                          victim->timestamp_created))
+        circuit_build_times_set_timeout(&circ_times);
     }
 
     if (victim->n_conn)
@@ -431,11 +441,11 @@ circuit_stream_is_being_handled(edge_connection_t *conn,
 }
 
 /** Don't keep more than this many unused open circuits around. */
-#define MAX_UNUSED_OPEN_CIRCUITS 12
+#define MAX_UNUSED_OPEN_CIRCUITS 14
 
 /** Figure out how many circuits we have open that are clean. Make
  * sure it's enough for all the upcoming behaviors we predict we'll have.
- * But if we have too many, close the not-so-useful ones.
+ * But put an upper bound on the total number of circuits.
  */
 static void
 circuit_predict_and_launch_new(void)
@@ -517,6 +527,19 @@ circuit_predict_and_launch_new(void)
     circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags);
     return;
   }
+
+  /* Finally, check to see if we still need more circuits to learn
+   * a good build timeout. But if we're close to our max number we
+   * want, don't do another -- we want to leave a few slots open so
+   * we can still build circuits preemptively as needed. */
+  if (num < MAX_UNUSED_OPEN_CIRCUITS-2 &&
+      circuit_build_times_needs_circuits_now(&circ_times)) {
+    flags = CIRCLAUNCH_NEED_CAPACITY;
+    log_info(LD_CIRC,
+             "Have %d clean circs need another buildtime test circ.", num);
+    circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags);
+    return;
+  }
 }
 
 /** Build a new test circuit every 5 minutes */
@@ -624,6 +647,11 @@ circuit_detach_stream(circuit_t *circ, edge_connection_t *conn)
   tor_fragile_assert();
 }
 
+/** If we haven't yet decided on a good timeout value for circuit
+ * building, we close idles circuits aggressively so we can get more
+ * data points. */
+#define IDLE_TIMEOUT_WHILE_LEARNING (10*60)
+
 /** Find each circuit that has been unused for too long, or dirty
  * for too long and has no streams on it: mark it for close.
  */
@@ -631,7 +659,15 @@ static void
 circuit_expire_old_circuits(time_t now)
 {
   circuit_t *circ;
-  time_t cutoff = now - get_options()->CircuitIdleTimeout;
+  time_t cutoff;
+
+  if (circuit_build_times_needs_circuits(&circ_times)) {
+    /* Circuits should be shorter lived if we need more of them
+     * for learning a good build timeout */
+    cutoff = now - IDLE_TIMEOUT_WHILE_LEARNING;
+  } else {
+    cutoff = now - get_options()->CircuitIdleTimeout;
+  }
 
   for (circ = global_circuitlist; circ; circ = circ->next) {
     if (circ->marked_for_close || ! CIRCUIT_IS_ORIGIN(circ))
@@ -724,17 +760,12 @@ circuit_testing_opened(origin_circuit_t *circ)
 static void
 circuit_testing_failed(origin_circuit_t *circ, int at_last_hop)
 {
-  routerinfo_t *me = router_get_my_routerinfo();
   if (server_mode(get_options()) && check_whether_orport_reachable())
     return;
-  if (!me)
-    return;
 
   log_info(LD_GENERAL,
            "Our testing circuit (to see if your ORPort is reachable) "
            "has failed. I'll try again later.");
-  control_event_server_status(LOG_WARN, "REACHABILITY_FAILED ORADDRESS=%s:%d",
-                             me->address, me->or_port);
 
   /* These aren't used yet. */
   (void)circ;
@@ -811,6 +842,9 @@ circuit_build_failed(origin_circuit_t *circ)
                "(%s:%d). I'm going to try to rotate to a better connection.",
                n_conn->_base.address, n_conn->_base.port);
       n_conn->is_bad_for_new_circs = 1;
+    } else {
+      log_info(LD_OR,
+               "Our circuit died before the first hop with no connection");
     }
     if (n_conn_id) {
       entry_guard_register_connect_status(n_conn_id, 0, 1, time(NULL));

src/or/command.c

@@ -575,7 +575,7 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn)
     /* Consider all the other addresses; if any matches, this connection is
      * "canonical." */
     tor_addr_t addr;
-    const char *next = decode_address_from_payload(&addr, cp, end-cp);
+    const char *next = decode_address_from_payload(&addr, cp, (int)(end-cp));
     if (next == NULL) {
       log_fn(LOG_PROTOCOL_WARN,  LD_OR,
              "Bad address in netinfo cell; closing connection.");
@@ -610,9 +610,11 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn)
            conn->_base.address, (int)conn->_base.port,
            apparent_skew>0 ? "ahead" : "behind", dbuf,
            apparent_skew>0 ? "behind" : "ahead");
-    control_event_general_status(LOG_WARN,
-                        "CLOCK_SKEW SKEW=%ld SOURCE=OR:%s:%d",
-                        apparent_skew, conn->_base.address, conn->_base.port);
+    if (severity == LOG_WARN) /* only tell the controller if an authority */
+      control_event_general_status(LOG_WARN,
+                          "CLOCK_SKEW SKEW=%ld SOURCE=OR:%s:%d",
+                          apparent_skew,
+                          conn->_base.address, conn->_base.port);
   }
 
   /* XXX maybe act on my_apparent_addr, if the source is sufficiently

src/or/config.c

@@ -164,10 +164,11 @@ static config_var_t _option_vars[] = {
   V(BridgeRecordUsageByCountry,  BOOL,     "1"),
   V(BridgeRelay,                 BOOL,     "0"),
   V(CellStatistics,              BOOL,     "0"),
-  V(CircuitBuildTimeout,         INTERVAL, "1 minute"),
+  V(CircuitBuildTimeout,         INTERVAL, "0"),
   V(CircuitIdleTimeout,          INTERVAL, "1 hour"),
   V(ClientDNSRejectInternalAddresses, BOOL,"1"),
   V(ClientOnly,                  BOOL,     "0"),
+  V(ConsensusParams,             STRING,   NULL),
   V(ConnLimit,                   UINT,     "1000"),
   V(ConstrainedSockets,          BOOL,     "0"),
   V(ConstrainedSockSize,         MEMUNIT,  "8192"),
@@ -408,6 +409,10 @@ static config_var_t _state_vars[] = {
   V(LastRotatedOnionKey,              ISOTIME,  NULL),
   V(LastWritten,                      ISOTIME,  NULL),
 
+  V(TotalBuildTimes,                  UINT,     NULL),
+  VAR("CircuitBuildTimeBin",          LINELIST_S, BuildtimeHistogram, NULL),
+  VAR("BuildtimeHistogram",           LINELIST_V, BuildtimeHistogram, NULL),
+
   { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL }
 };
 
@@ -596,6 +601,10 @@ static config_var_description_t options_description[] = {
   /* Hidden service options: HiddenService: dir,excludenodes, nodes,
    * options, port.  PublishHidServDescriptor */
 
+  /* Circuit build time histogram options */
+  { "CircuitBuildTimeBin", "Histogram of recent circuit build times"},
+  { "TotalBuildTimes", "Total number of buildtimes in histogram"},
+
   /* Nonpersistent options: __LeaveStreamsUnattached, __AllDirActionsPrivate */
   { NULL, NULL },
 };
@@ -1506,7 +1515,10 @@ expand_abbrev(config_format_t *fmt, const char *option, int command_line,
                  fmt->abbrevs[i].abbreviated,
                  fmt->abbrevs[i].full);
       }
-      return fmt->abbrevs[i].full;
+      /* Keep going through the list in case we want to rewrite it more.
+       * (We could imagine recursing here, but I don't want to get the
+       * user into an infinite loop if we craft our list wrong.) */
+      option = fmt->abbrevs[i].full;
     }
   }
   return option;
@@ -2521,7 +2533,8 @@ is_local_addr(const tor_addr_t *addr)
      * the same /24 as last_resolved_addr will be the same as checking whether
      * it was on net 0, which is already done by is_internal_IP.
      */
-    if ((last_resolved_addr & 0xffffff00ul) == (ip & 0xffffff00ul))
+    if ((last_resolved_addr & (uint32_t)0xffffff00ul)
+        == (ip & (uint32_t)0xffffff00ul))
       return 1;
   }
   return 0;
@@ -2909,11 +2922,6 @@ compute_publishserverdescriptor(or_options_t *options)
 /** Highest allowable value for RendPostPeriod. */
 #define MAX_DIR_PERIOD (MIN_ONION_KEY_LIFETIME/2)
 
-/** Lowest allowable value for CircuitBuildTimeout; values too low will
- * increase network load because of failing connections being retried, and
- * might prevent users from connecting to the network at all. */
-#define MIN_CIRCUIT_BUILD_TIMEOUT 30
-
 /** Lowest allowable value for MaxCircuitDirtiness; if this is too low, Tor
  * will generate too many circuits and potentially overload the network. */
 #define MIN_MAX_CIRCUIT_DIRTINESS 10
@@ -3360,12 +3368,6 @@ options_validate(or_options_t *old_options, or_options_t *options,
     options->RendPostPeriod = MAX_DIR_PERIOD;
   }
 
-  if (options->CircuitBuildTimeout < MIN_CIRCUIT_BUILD_TIMEOUT) {
-    log(LOG_WARN, LD_CONFIG, "CircuitBuildTimeout option is too short; "
-      "raising to %d seconds.", MIN_CIRCUIT_BUILD_TIMEOUT);
-    options->CircuitBuildTimeout = MIN_CIRCUIT_BUILD_TIMEOUT;
-  }
-
   if (options->MaxCircuitDirtiness < MIN_MAX_CIRCUIT_DIRTINESS) {
     log(LOG_WARN, LD_CONFIG, "MaxCircuitDirtiness option is too short; "
       "raising to %d seconds.", MIN_MAX_CIRCUIT_DIRTINESS);
@@ -4280,7 +4282,7 @@ options_init_from_string(const char *cf,
  err:
   config_free(&options_format, newoptions);
   if (*msg) {
-    int len = strlen(*msg)+256;
+    int len = (int)strlen(*msg)+256;
     char *newmsg = tor_malloc(len);
 
     tor_snprintf(newmsg, len, "Failed to parse/validate config: %s", *msg);
@@ -4860,35 +4862,28 @@ config_parse_units(const char *val, struct unit_table_t *u, int *ok)
   uint64_t v = 0;
   double d = 0;
   int use_float = 0;
-
-  smartlist_t *sl;
+  char *cp;
 
   tor_assert(ok);
-  sl = smartlist_create();
-  smartlist_split_string(sl, val, NULL,
-                         SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 3);
 
-  if (smartlist_len(sl) < 1 || smartlist_len(sl) > 2) {
-    *ok = 0;
-    goto done;
-  }
-
-  v = tor_parse_uint64(smartlist_get(sl,0), 10, 0, UINT64_MAX, ok, NULL);
-  if (!*ok) {
-    int r = sscanf(smartlist_get(sl,0), "%lf", &d);
-    if (r == 0 || d < 0)
+  v = tor_parse_uint64(val, 10, 0, UINT64_MAX, ok, &cp);
+  if (!*ok || (cp && *cp == '.')) {
+    d = tor_parse_double(val, 0, UINT64_MAX, ok, &cp);
+    if (!*ok)
       goto done;
     use_float = 1;
   }
 
-  if (smartlist_len(sl) == 1) {
+  if (!cp) {
     *ok = 1;
     v = use_float ? DBL_TO_U64(d) :  v;
     goto done;
   }
 
+  cp = (char*) eat_whitespace(cp);
+
   for ( ;u->unit;++u) {
-    if (!strcasecmp(u->unit, smartlist_get(sl,1))) {
+    if (!strcasecmp(u->unit, cp)) {
       if (use_float)
         v = u->multiplier * d;
       else
@@ -4897,11 +4892,9 @@ config_parse_units(const char *val, struct unit_table_t *u, int *ok)
       goto done;
     }
   }
-  log_warn(LD_CONFIG, "Unknown unit '%s'.", (char*)smartlist_get(sl,1));
+  log_warn(LD_CONFIG, "Unknown unit '%s'.", cp);
   *ok = 0;
  done:
-  SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp));
-  smartlist_free(sl);
 
   if (*ok)
     return v;
@@ -4916,7 +4909,8 @@ config_parse_units(const char *val, struct unit_table_t *u, int *ok)
 static uint64_t
 config_parse_memunit(const char *s, int *ok)
 {
-  return config_parse_units(s, memory_units, ok);
+  uint64_t u = config_parse_units(s, memory_units, ok);
+  return u;
 }
 
 /** Parse a string in the format "number unit", where unit is a unit of time.
@@ -5066,6 +5060,10 @@ or_state_set(or_state_t *new_state)
     log_warn(LD_GENERAL,"Unparseable bandwidth history state: %s",err);
     tor_free(err);
   }
+  if (circuit_build_times_parse_state(&circ_times, global_state, &err) < 0) {
+    log_warn(LD_GENERAL,"%s",err);
+    tor_free(err);
+  }
 }
 
 /** Reload the persistent state from disk, generating a new state as needed.
@@ -5198,6 +5196,7 @@ or_state_save(time_t now)
    * to avoid redundant writes. */
   entry_guards_update_state(global_state);
   rep_hist_update_state(global_state);
+  circuit_build_times_update_state(&circ_times, global_state);
   if (accounting_is_enabled(get_options()))
     accounting_run_housekeeping(now);
 

src/or/connection.c

@@ -2346,7 +2346,7 @@ loop_again:
     return -1;
   }
   if (conn->linked_conn) {
-    /* The other side's handle_write will never actually get called, so
+    /* The other side's handle_write() will never actually get called, so
      * we need to invoke the appropriate callbacks ourself. */
     connection_t *linked = conn->linked_conn;
 
@@ -2363,7 +2363,7 @@ loop_again:
     if (!buf_datalen(linked->outbuf) && conn->active_on_link)
       connection_stop_reading_from_linked_conn(conn);
   }
-  /* If we hit the EOF, call connection_reached_eof. */
+  /* If we hit the EOF, call connection_reached_eof(). */
   if (!conn->marked_for_close &&
       conn->inbuf_reached_eof &&
       connection_reached_eof(conn) < 0) {
@@ -2589,7 +2589,7 @@ connection_handle_write(connection_t *conn, int force)
     return 0; /* do nothing */
 
   if (conn->in_flushed_some) {
-    log_warn(LD_BUG, "called recursively from inside conn->in_flushed_some()");
+    log_warn(LD_BUG, "called recursively from inside conn->in_flushed_some");
     return 0;
   }
 
@@ -2678,8 +2678,8 @@ connection_handle_write(connection_t *conn, int force)
         if (!connection_is_reading(conn)) {
           connection_stop_writing(conn);
           conn->write_blocked_on_bw = 1;
-          /* we'll start reading again when the next second arrives,
-           * and then also start writing again.
+          /* we'll start reading again when we get more tokens in our
+           * read bucket; then we'll start writing again too.
            */
         }
         /* else no problem, we're already reading */
@@ -3067,7 +3067,7 @@ client_check_address_changed(int sock)
     return;
   }
 
-  /* Okay.  If we've used this address previously, we're okay. */
+  /* If we've used this address previously, we're okay. */
   ip_out = ntohl(out_addr.sin_addr.s_addr);
   SMARTLIST_FOREACH(outgoing_addrs, uint32_t*, ip_ptr,
                     if (*ip_ptr == ip_out) return;

src/or/connection_or.c

@@ -1036,6 +1036,8 @@ connection_tls_finish_handshake(or_connection_t *conn)
                                               digest_rcvd) < 0)
     return -1;
 
+  circuit_build_times_network_is_live(&circ_times);
+
   if (tor_tls_used_v1_handshake(conn->tls)) {
     conn->link_proto = 1;
     if (!started_here) {
@@ -1087,6 +1089,7 @@ connection_or_set_state_open(or_connection_t *conn)
   control_event_or_conn_status(conn, OR_CONN_EVENT_CONNECTED, 0);
 
   if (started_here) {
+    circuit_build_times_network_is_live(&circ_times);
     rep_hist_note_connect_succeeded(conn->identity_digest, now);
     if (entry_guard_register_connect_status(conn->identity_digest,
                                             1, 0, now) < 0) {
@@ -1187,6 +1190,7 @@ connection_or_process_cells_from_inbuf(or_connection_t *conn)
     if (connection_fetch_var_cell_from_buf(conn, &var_cell)) {
       if (!var_cell)
         return 0; /* not yet. */
+      circuit_build_times_network_is_live(&circ_times);
       command_process_var_cell(var_cell, conn);
       var_cell_free(var_cell);
     } else {
@@ -1196,6 +1200,7 @@ connection_or_process_cells_from_inbuf(or_connection_t *conn)
                                                                  available? */
         return 0; /* not yet */
 
+      circuit_build_times_network_is_live(&circ_times);
       connection_fetch_from_buf(buf, CELL_NETWORK_SIZE, TO_CONN(conn));
 
       /* retrieve cell info from buf (create the host-order struct from the

src/or/control.c

@@ -1696,7 +1696,11 @@ getinfo_helper_events(control_connection_t *control_conn,
       *answer = tor_strdup(has_completed_circuit ? "1" : "0");
     } else if (!strcmp(question, "status/enough-dir-info")) {
       *answer = tor_strdup(router_have_minimum_dir_info() ? "1" : "0");
-    } else if (!strcmp(question, "status/good-server-descriptor")) {
+    } else if (!strcmp(question, "status/good-server-descriptor") ||
+               !strcmp(question, "status/accepted-server-descriptor")) {
+      /* They're equivalent for now, until we can figure out how to make
+       * good-server-descriptor be what we want. See comment in
+       * control-spec.txt. */
       *answer = tor_strdup(directories_have_accepted_server_descriptor()
                            ? "1" : "0");
     } else if (!strcmp(question, "status/reachability-succeeded/or")) {
@@ -2495,7 +2499,7 @@ handle_control_resolve(control_connection_t *conn, uint32_t len,
   int is_reverse = 0;
   (void) len; /* body is nul-terminated; it's safe to ignore the length */
 
-  if (!(conn->event_mask & (1L<<EVENT_ADDRMAP))) {
+  if (!(conn->event_mask & ((uint32_t)1L<<EVENT_ADDRMAP))) {
     log_warn(LD_CONTROL, "Controller asked us to resolve an address, but "
              "isn't listening for ADDRMAP events.  It probably won't see "
              "the answer.");

src/or/directory.c

@@ -554,11 +554,6 @@ void
 connection_dir_request_failed(dir_connection_t *conn)
 {
   if (directory_conn_is_self_reachability_test(conn)) {
-    routerinfo_t *me = router_get_my_routerinfo();
-    if (me)
-      control_event_server_status(LOG_WARN,
-                                  "REACHABILITY_FAILED DIRADDRESS=%s:%d",
-                                  me->address, me->dir_port);
     return; /* this was a test fetch. don't retry. */
   }
   if (entry_list_can_grow(get_options()))
@@ -886,7 +881,7 @@ static char *
 directory_get_consensus_url(int supports_conditional_consensus)
 {
   char *url;
-  int len;
+  size_t len;
 
   if (supports_conditional_consensus) {
     char *authority_id_list;
@@ -2337,7 +2332,7 @@ client_likes_consensus(networkstatus_t *v, const char *want_url)
   need_at_least = smartlist_len(want_authorities)/2+1;
   SMARTLIST_FOREACH(want_authorities, const char *, d, {
     char want_digest[DIGEST_LEN];
-    int want_len = strlen(d)/2;
+    size_t want_len = strlen(d)/2;
     if (want_len > DIGEST_LEN)
       want_len = DIGEST_LEN;
 

src/or/dirserv.c

@@ -1864,7 +1864,7 @@ version_from_platform(const char *platform)
  *   NS_V2 - Output an entry suitable for a V2 NS opinion document
  *   NS_V3_CONSENSUS - Output the first portion of a V3 NS consensus entry
  *   NS_V3_VOTE - Output a complete V3 NS vote
- *   NS_CONTROL_PORT - Output a NS docunent for the control port
+ *   NS_CONTROL_PORT - Output a NS document for the control port
  */
 int
 routerstatus_format_entry(char *buf, size_t buf_len,
@@ -2324,7 +2324,7 @@ measured_bw_line_apply(measured_bw_line_t *parsed_line,
 
   if (rs) {
     rs->has_measured_bw = 1;
-    rs->measured_bw = parsed_line->bw;
+    rs->measured_bw = (uint32_t)parsed_line->bw;
   } else {
     log_info(LD_DIRSERV, "Node ID %s not found in routerstatus list",
              parsed_line->node_hex);
@@ -2553,6 +2553,13 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key,
   }
   smartlist_sort_strings(v3_out->known_flags);
 
+  if (options->ConsensusParams) {
+    v3_out->net_params = smartlist_create();
+    smartlist_split_string(v3_out->net_params,
+                           options->ConsensusParams, NULL, 0, 0);
+    smartlist_sort_strings(v3_out->net_params);
+  }
+
   voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t));
   voter->nickname = tor_strdup(options->Nickname);
   memcpy(voter->identity_digest, identity_digest, DIGEST_LEN);

src/or/dirvote.c

@@ -24,7 +24,9 @@ static int dirvote_publish_consensus(void);
 static char *make_consensus_method_list(int low, int high);
 
 /** The highest consensus method that we currently support. */
-#define MAX_SUPPORTED_CONSENSUS_METHOD 6
+#define MAX_SUPPORTED_CONSENSUS_METHOD 7
+
+#define MIN_METHOD_FOR_PARAMS 7
 
 /* =====
  * Voting
@@ -97,6 +99,7 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key,
     char fu[ISO_TIME_LEN+1];
     char vu[ISO_TIME_LEN+1];
     char *flags = smartlist_join_strings(v3_ns->known_flags, " ", 0, NULL);
+    char *params;
     authority_cert_t *cert = v3_ns->cert;
     char *methods =
       make_consensus_method_list(1, MAX_SUPPORTED_CONSENSUS_METHOD);
@@ -105,6 +108,11 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key,
     format_iso_time(fu, v3_ns->fresh_until);
     format_iso_time(vu, v3_ns->valid_until);
 
+    if (v3_ns->net_params)
+      params = smartlist_join_strings(v3_ns->net_params, " ", 0, NULL);
+    else
+      params = tor_strdup("");
+
     tor_assert(cert);
     tor_snprintf(status, len,
                  "network-status-version 3\n"
@@ -117,6 +125,7 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key,
                  "voting-delay %d %d\n"
                  "%s" /* versions */
                  "known-flags %s\n"
+                 "params %s\n"
                  "dir-source %s %s %s %s %d %d\n"
                  "contact %s\n",
                  v3_ns->type == NS_TYPE_VOTE ? "vote" : "opinion",
@@ -125,9 +134,11 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key,
                  v3_ns->vote_seconds, v3_ns->dist_seconds,
                  version_lines,
                  flags,
+                 params,
                  voter->nickname, fingerprint, voter->address,
                    ipaddr, voter->dir_port, voter->or_port, voter->contact);
 
+    tor_free(params);
     tor_free(flags);
     tor_free(methods);
     outp = status + strlen(status);
@@ -507,6 +518,89 @@ compute_consensus_versions_list(smartlist_t *lst, int n_versioning)
   return result;
 }
 
+/** Helper: given a list of valid networkstatus_t, return a new string
+ * containing the contents of the consensus network parameter set.
+ */
+/* private */ char *
+dirvote_compute_params(smartlist_t *votes)
+{
+  int i;
+  int32_t *vals;
+
+  int cur_param_len;
+  const char *cur_param;
+  const char *eq;
+  char *result;
+
+  const int n_votes = smartlist_len(votes);
+  smartlist_t *output;
+  smartlist_t *param_list = smartlist_create();
+
+  /* We require that the parameter lists in the votes are well-formed: that
+     is, that their keywords are unique and sorted, and that their values are
+     between INT32_MIN and INT32_MAX inclusive.  This should be guaranteed by
+     the parsing code. */
+
+  vals = tor_malloc(sizeof(int)*n_votes);
+
+  SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) {
+    if (!v->net_params)
+      continue;
+    smartlist_add_all(param_list, v->net_params);
+  } SMARTLIST_FOREACH_END(v);
+
+  if (smartlist_len(param_list) == 0) {
+    tor_free(vals);
+    smartlist_free(param_list);
+    return NULL;
+  }
+
+  smartlist_sort_strings(param_list);
+  i = 0;
+  cur_param = smartlist_get(param_list, 0);
+  eq = strchr(cur_param, '=');
+  tor_assert(eq);
+  cur_param_len = (int)(eq+1 - cur_param);
+
+  output = smartlist_create();
+
+  SMARTLIST_FOREACH_BEGIN(param_list, const char *, param) {
+    const char *next_param;
+    int ok=0;
+    eq = strchr(param, '=');
+    tor_assert(i<n_votes);
+    vals[i++] = (int32_t)
+      tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, NULL);
+    tor_assert(ok);
+
+    if (param_sl_idx+1 == smartlist_len(param_list))
+      next_param = NULL;
+    else
+      next_param = smartlist_get(param_list, param_sl_idx+1);
+    if (!next_param || strncmp(next_param, param, cur_param_len)) {
+      /* We've reached the end of a series. */
+      int32_t median = median_int32(vals, i);
+      char *out_string = tor_malloc(64+cur_param_len);
+      memcpy(out_string, param, cur_param_len);
+      tor_snprintf(out_string+cur_param_len,64, "%ld", (long)median);
+      smartlist_add(output, out_string);
+
+      i = 0;
+      if (next_param) {
+        eq = strchr(next_param, '=');
+        cur_param_len = (int)(eq+1 - next_param);
+      }
+    }
+  } SMARTLIST_FOREACH_END(param);
+
+  result = smartlist_join_strings(output, " ", 0, NULL);
+  SMARTLIST_FOREACH(output, char *, cp, tor_free(cp));
+  smartlist_free(output);
+  smartlist_free(param_list);
+  tor_free(vals);
+  return result;
+}
+
 /** Given a list of vote networkstatus_t in <b>votes</b>, our public
  * authority <b>identity_key</b>, our private authority <b>signing_key</b>,
  * and the number of <b>total_authorities</b> that we believe exist in our
@@ -659,6 +753,15 @@ networkstatus_compute_consensus(smartlist_t *votes,
     tor_free(flaglist);
   }
 
+  if (consensus_method >= MIN_METHOD_FOR_PARAMS) {
+    char *params = dirvote_compute_params(votes);
+    if (params) {
+      smartlist_add(chunks, tor_strdup("params "));
+      smartlist_add(chunks, params);
+      smartlist_add(chunks, tor_strdup("\n"));
+    }
+  }
+
   /* Sort the votes. */
   smartlist_sort(votes, _compare_votes_by_authority_id);
   /* Add the authority sections. */

src/or/eventdns.c

@@ -2332,7 +2332,7 @@ out1:
 
 /* exported function */
 int
-evdns_nameserver_add(unsigned long int address) {
+evdns_nameserver_add(uint32_t address) {
 	struct sockaddr_in sin;
 	memset(&sin, 0, sizeof(sin));
 	sin.sin_family = AF_INET;
@@ -2363,13 +2363,13 @@ evdns_nameserver_ip_add(const char *ip_as_string) {
 
 	cp = strchr(ip_as_string, ':');
 	if (*ip_as_string == '[') {
-		int len;
+		size_t len;
 		if (!(cp = strchr(ip_as_string, ']'))) {
 			log(EVDNS_LOG_DEBUG, "Nameserver missing closing ]");
 			return 4;
 		}
 		len = cp-(ip_as_string + 1);
-		if (len > (int)sizeof(buf)-1) {
+		if (len > sizeof(buf)-1) {
 			log(EVDNS_LOG_DEBUG, "[Nameserver] does not fit in buffer.");
 			return 4;
 		}

src/or/eventdns.h

@@ -112,7 +112,7 @@
  *
  * API reference:
  *
- * int evdns_nameserver_add(unsigned long int address)
+ * int evdns_nameserver_add(uint32_t address)
  *	 Add a nameserver. The address should be an IP address in
  *	 network byte order. The type of address is chosen so that
  *	 it matches in_addr.s_addr.
@@ -258,7 +258,7 @@ typedef void (*evdns_callback_type) (int result, char type, int count, int ttl,
 int evdns_init(void);
 void evdns_shutdown(int fail_requests);
 const char *evdns_err_to_string(int err);
-int evdns_nameserver_add(unsigned long int address);
+int evdns_nameserver_add(uint32_t address);
 int evdns_count_nameservers(void);
 int evdns_clear_nameservers_and_suspend(void);
 int evdns_resume(void);

src/or/geoip.c

@@ -340,7 +340,7 @@ geoip_determine_shares(time_t now)
         ((double) (now - last_time_determined_shares));
     v3_share_times_seconds += v3_share *
         ((double) (now - last_time_determined_shares));
-    share_seconds += now - last_time_determined_shares;
+    share_seconds += (int)(now - last_time_determined_shares);
   }
   last_time_determined_shares = now;
 }
@@ -768,7 +768,7 @@ geoip_get_dirreq_history(geoip_client_action_t action,
         time_diff = 1; /* Avoid DIV/0; "instant" answers are impossible
                         * by law of nature or something, but a milisecond
                         * is a bit greater than "instantly" */
-      bytes_per_second = 1000 * ent->response_size / time_diff;
+      bytes_per_second = (uint32_t)(1000 * ent->response_size / time_diff);
       dltimes[ent_sl_idx] = bytes_per_second;
     } SMARTLIST_FOREACH_END(ent);
     median_uint32(dltimes, complete); /* sorts as a side effect. */

src/or/main.c

@@ -1240,17 +1240,26 @@ second_elapsed_callback(int fd, short event, void *args)
         TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) {
     /* every 20 minutes, check and complain if necessary */
     routerinfo_t *me = router_get_my_routerinfo();
-    if (me && !check_whether_orport_reachable())
+    if (me && !check_whether_orport_reachable()) {
       log_warn(LD_CONFIG,"Your server (%s:%d) has not managed to confirm that "
                "its ORPort is reachable. Please check your firewalls, ports, "
                "address, /etc/hosts file, etc.",
                me->address, me->or_port);
-    if (me && !check_whether_dirport_reachable())
+      control_event_server_status(LOG_WARN,
+                                  "REACHABILITY_FAILED ORADDRESS=%s:%d",
+                                  me->address, me->or_port);
+    }
+
+    if (me && !check_whether_dirport_reachable()) {
       log_warn(LD_CONFIG,
                "Your server (%s:%d) has not managed to confirm that its "
                "DirPort is reachable. Please check your firewalls, ports, "
                "address, /etc/hosts file, etc.",
                me->address, me->dir_port);
+      control_event_server_status(LOG_WARN,
+                                  "REACHABILITY_FAILED DIRADDRESS=%s:%d",
+                                  me->address, me->dir_port);
+    }
   }
 
 /** If more than this many seconds have elapsed, probably the clock
@@ -1653,7 +1662,7 @@ dumpstats(int severity)
 {
   time_t now = time(NULL);
   time_t elapsed;
-  int rbuf_cap, wbuf_cap, rbuf_len, wbuf_len;
+  size_t rbuf_cap, wbuf_cap, rbuf_len, wbuf_len;
 
   log(severity, LD_GENERAL, "Dumping stats:");
 
@@ -1689,7 +1698,7 @@ dumpstats(int severity)
           log(severity, LD_GENERAL,
               "Conn %d: %d/%d bytes used on OpenSSL read buffer; "
               "%d/%d bytes used on write buffer.",
-              i, rbuf_len, rbuf_cap, wbuf_len, wbuf_cap);
+              i, (int)rbuf_len, (int)rbuf_cap, (int)wbuf_len, (int)wbuf_cap);
         }
       }
     }

src/or/networkstatus.c

@@ -286,6 +286,10 @@ networkstatus_vote_free(networkstatus_t *ns)
     SMARTLIST_FOREACH(ns->known_flags, char *, c, tor_free(c));
     smartlist_free(ns->known_flags);
   }
+  if (ns->net_params) {
+    SMARTLIST_FOREACH(ns->net_params, char *, c, tor_free(c));
+    smartlist_free(ns->net_params);
+  }
   if (ns->supported_methods) {
     SMARTLIST_FOREACH(ns->supported_methods, char *, c, tor_free(c));
     smartlist_free(ns->supported_methods);
@@ -1889,6 +1893,33 @@ networkstatus_dump_bridge_status_to_file(time_t now)
   tor_free(status);
 }
 
+/** Return the value of a integer parameter from the networkstatus <b>ns</b>
+ * whose name is <b>param_name</b>.  Return <b>default_val</b> if ns is NULL,
+ * or if it has no parameter called <b>param_name</b>. */
+int32_t
+networkstatus_get_param(networkstatus_t *ns, const char *param_name,
+                        int32_t default_val)
+{
+  size_t name_len;
+
+  if (!ns || !ns->net_params)
+    return default_val;
+
+  name_len = strlen(param_name);
+
+  SMARTLIST_FOREACH_BEGIN(ns->net_params, const char *, p) {
+    if (!strcmpstart(p, param_name) && p[name_len] == '=') {
+      int ok=0;
+      long v = tor_parse_long(p+name_len+1, 10, INT32_MIN,
+                              INT32_MAX, &ok, NULL);
+      if (ok)
+        return (int32_t) v;
+    }
+  } SMARTLIST_FOREACH_END(p);
+
+  return default_val;
+}
+
 /** If <b>question</b> is a string beginning with "ns/" in a format the
  * control interface expects for a GETINFO question, set *<b>answer</b> to a
  * newly-allocated string containing networkstatus lines for the appropriate

src/or/or.h

@@ -1672,6 +1672,10 @@ typedef struct networkstatus_t {
    * not listed here, the voter has no opinion on what its value should be. */
   smartlist_t *known_flags;
 
+  /** List of key=value strings for the parameters in this vote or
+   * consensus, sorted by key. */
+  smartlist_t *net_params;
+
   /** List of networkstatus_voter_info_t.  For a vote, only one element
    * is included.  For a consensus, one element is included for every voter
    * whose vote contributed to the consensus. */
@@ -1866,9 +1870,9 @@ typedef struct crypt_path_t {
   struct crypt_path_t *prev; /**< Link to previous crypt_path_t in the
                               * circuit. */
 
-  int package_window; /**< How many bytes are we allowed to originate ending
+  int package_window; /**< How many cells are we allowed to originate ending
                        * at this step? */
-  int deliver_window; /**< How many bytes are we willing to deliver originating
+  int deliver_window; /**< How many cells are we willing to deliver originating
                        * at this step? */
 } crypt_path_t;
 
@@ -1973,6 +1977,7 @@ typedef struct circuit_t {
   time_t timestamp_created; /**< When was this circuit created? */
   time_t timestamp_dirty; /**< When the circuit was first used, or 0 if the
                            * circuit is clean. */
+  struct timeval highres_created; /**< When exactly was the circuit created? */
 
   uint16_t marked_for_close; /**< Should we close this circuit at the end of
                               * the main loop? (If true, holds the line number
@@ -2583,6 +2588,10 @@ typedef struct {
   /** Location of bandwidth measurement file */
   char *V3BandwidthsFile;
 
+  /** Authority only: key=value pairs that we add to our networkstatus
+   * consensus vote on the 'params' line. */
+  char *ConsensusParams;
+
   /** The length of time that we think an initial consensus should be fresh.
    * Only altered on testing networks. */
   int TestingV3AuthInitialVotingInterval;
@@ -2675,6 +2684,10 @@ typedef struct {
   int         BWHistoryWriteInterval;
   smartlist_t *BWHistoryWriteValues;
 
+  /** Build time histogram */
+  config_line_t * BuildtimeHistogram;
+  uint16_t TotalBuildTimes;
+
   /** What version of Tor wrote this state file? */
   char *TorVersion;
 
@@ -2844,6 +2857,155 @@ void bridges_retry_all(void);
 
 void entry_guards_free_all(void);
 
+/* Circuit Build Timeout "public" functions and structures. */
+
+/** Maximum quantile to use to generate synthetic timeouts.
+ *  We want to stay a bit short of 1.0, because longtail is
+ *  loooooooooooooooooooooooooooooooooooooooooooooooooooong. */
+#define MAX_SYNTHETIC_QUANTILE 0.985
+
+/** Minimum circuits before estimating a timeout */
+#define MIN_CIRCUITS_TO_OBSERVE 500
+
+/** Total size of the circuit timeout history to accumulate.
+ * 5000 is approx 1.5 weeks worth of continual-use circuits. */
+#define NCIRCUITS_TO_OBSERVE 5000
+
+/** Width of the histogram bins in milliseconds */
+#define BUILDTIME_BIN_WIDTH ((build_time_t)50)
+
+/** Cutoff point on the CDF for our timeout estimation.
+ * TODO: This should be moved to the consensus */
+#define BUILDTIMEOUT_QUANTILE_CUTOFF 0.8
+
+/** A build_time_t is milliseconds */
+typedef uint32_t build_time_t;
+#define BUILD_TIME_MAX ((build_time_t)(INT32_MAX))
+
+/** Lowest allowable value for CircuitBuildTimeout in milliseconds */
+#define BUILD_TIMEOUT_MIN_VALUE (3*1000)
+
+/** Initial circuit build timeout in milliseconds */
+#define BUILD_TIMEOUT_INITIAL_VALUE (60*1000)
+
+/** How often in seconds should we build a test circuit */
+#define BUILD_TIMES_TEST_FREQUENCY 60
+
+/** Save state every 10 circuits */
+#define BUILD_TIMES_SAVE_STATE_EVERY 10
+
+/* Circuit Build Timeout network liveness constants */
+
+/**
+ * How many circuits count as recent when considering if the
+ * connection has gone gimpy or changed.
+ */
+#define RECENT_CIRCUITS 20
+
+/**
+ * Have we received a cell in the last N circ attempts?
+ *
+ * This tells us when to temporarily switch back to
+ * BUILD_TIMEOUT_INITIAL_VALUE until we start getting cells,
+ * at which point we switch back to computing the timeout from
+ * our saved history.
+ */
+#define NETWORK_NONLIVE_TIMEOUT_COUNT (lround(RECENT_CIRCUITS*0.15))
+
+/**
+ * This tells us when to toss out the last streak of N timeouts.
+ *
+ * If instead we start getting cells, we switch back to computing the timeout
+ * from our saved history.
+ */
+#define NETWORK_NONLIVE_DISCARD_COUNT (lround(NETWORK_NONLIVE_TIMEOUT_COUNT*2))
+
+/**
+ * Maximum count of timeouts that finish the first hop in the past
+ * RECENT_CIRCUITS before calculating a new timeout.
+ *
+ * This tells us to abandon timeout history and set
+ * the timeout back to BUILD_TIMEOUT_INITIAL_VALUE.
+ */
+#define MAX_RECENT_TIMEOUT_COUNT (lround(RECENT_CIRCUITS*0.75))
+
+/** Information about the state of our local network connection */
+typedef struct {
+  /** The timestamp we last completed a TLS handshake or received a cell */
+  time_t network_last_live;
+  /** If the network is not live, how many timeouts has this caused? */
+  int nonlive_timeouts;
+  /** If the network is not live, have we yet discarded our history? */
+  int nonlive_discarded;
+  /** Circular array of circuits that have made it to the first hop. Slot is
+   * 1 if circuit timed out, 0 if circuit succeeded */
+  int8_t timeouts_after_firsthop[RECENT_CIRCUITS];
+  /** Index into circular array. */
+  int after_firsthop_idx;
+} network_liveness_t;
+
+/** Structure for circuit build times history */
+typedef struct {
+  /** The circular array of recorded build times in milliseconds */
+  build_time_t circuit_build_times[NCIRCUITS_TO_OBSERVE];
+  /** Current index in the circuit_build_times circular array */
+  int build_times_idx;
+  /** Total number of build times accumulated. Maxes at NCIRCUITS_TO_OBSERVE */
+  int total_build_times;
+  /** Information about the state of our local network connection */
+  network_liveness_t liveness;
+  /** Last time we built a circuit. Used to decide to build new test circs */
+  time_t last_circ_at;
+  /** Number of timeouts that have happened before estimating pareto
+   *  parameters */
+  int pre_timeouts;
+  /** "Minimum" value of our pareto distribution (actually mode) */
+  build_time_t Xm;
+  /** alpha exponent for pareto dist. */
+  double alpha;
+  /** Have we computed a timeout? */
+  int have_computed_timeout;
+  /** The exact value for that timeout in milliseconds */
+  double timeout_ms;
+} circuit_build_times_t;
+
+extern circuit_build_times_t circ_times;
+void circuit_build_times_update_state(circuit_build_times_t *cbt,
+                                      or_state_t *state);
+int circuit_build_times_parse_state(circuit_build_times_t *cbt,
+                                    or_state_t *state, char **msg);
+int circuit_build_times_add_timeout(circuit_build_times_t *cbt,
+                                    int did_onehop, time_t start_time);
+void circuit_build_times_set_timeout(circuit_build_times_t *cbt);
+int circuit_build_times_add_time(circuit_build_times_t *cbt,
+                                 build_time_t time);
+int circuit_build_times_needs_circuits(circuit_build_times_t *cbt);
+int circuit_build_times_needs_circuits_now(circuit_build_times_t *cbt);
+void circuit_build_times_init(circuit_build_times_t *cbt);
+
+#ifdef CIRCUIT_PRIVATE
+double circuit_build_times_calculate_timeout(circuit_build_times_t *cbt,
+                                             double quantile);
+build_time_t circuit_build_times_generate_sample(circuit_build_times_t *cbt,
+                                                 double q_lo, double q_hi);
+void circuit_build_times_initial_alpha(circuit_build_times_t *cbt,
+                                       double quantile, double time_ms);
+void circuit_build_times_update_alpha(circuit_build_times_t *cbt);
+double circuit_build_times_cdf(circuit_build_times_t *cbt, double x);
+void circuit_build_times_add_timeout_worker(circuit_build_times_t *cbt,
+                                       double quantile_cutoff);
+void circuitbuild_running_unit_tests(void);
+void circuit_build_times_reset(circuit_build_times_t *cbt);
+
+/* Network liveness functions */
+int circuit_build_times_network_check_changed(circuit_build_times_t *cbt);
+#endif
+
+/* Network liveness functions */
+void circuit_build_times_network_is_live(circuit_build_times_t *cbt);
+int circuit_build_times_network_check_live(circuit_build_times_t *cbt);
+void circuit_build_times_network_circ_success(circuit_build_times_t *cbt);
+
 /********************************* circuitlist.c ***********************/
 
 circuit_t * _circuit_get_global_list(void);
@@ -2856,6 +3018,7 @@ void circuit_set_n_circid_orconn(circuit_t *circ, circid_t id,
                                  or_connection_t *conn);
 void circuit_set_state(circuit_t *circ, uint8_t state);
 void circuit_close_all_marked(void);
+int32_t circuit_initial_package_window(void);
 origin_circuit_t *origin_circuit_new(void);
 or_circuit_t *or_circuit_new(circid_t p_circ_id, or_connection_t *p_conn);
 circuit_t *circuit_get_by_circid_orconn(circid_t circ_id,
@@ -3661,9 +3824,9 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key,
                                         authority_cert_t *cert);
 
 #ifdef DIRVOTE_PRIVATE
-char *
-format_networkstatus_vote(crypto_pk_env_t *private_key,
-                          networkstatus_t *v3_ns);
+char *format_networkstatus_vote(crypto_pk_env_t *private_key,
+                                 networkstatus_t *v3_ns);
+char *dirvote_compute_params(smartlist_t *votes);
 #endif
 
 /********************************* dns.c ***************************/
@@ -3956,6 +4119,8 @@ void signed_descs_update_status_from_consensus_networkstatus(
 char *networkstatus_getinfo_helper_single(routerstatus_t *rs);
 char *networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now);
 void networkstatus_dump_bridge_status_to_file(time_t now);
+int32_t networkstatus_get_param(networkstatus_t *ns, const char *param_name,
+                                int32_t default_val);
 int getinfo_helper_networkstatus(control_connection_t *conn,
                                  const char *question, char **answer);
 void networkstatus_free_all(void);

src/or/relay.c

@@ -1635,7 +1635,8 @@ cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell)
       it_pool = mp_pool_new(sizeof(insertion_time_elem_t), 1024);
     tor_gettimeofday(&now);
 #define SECONDS_IN_A_DAY 86400L
-    added = (now.tv_sec % SECONDS_IN_A_DAY) * 100L + now.tv_usec / 10000L;
+    added = (uint32_t)(((now.tv_sec % SECONDS_IN_A_DAY) * 100L)
+            + ((uint32_t)now.tv_usec / (uint32_t)10000L));
     if (!it_queue) {
       it_queue = tor_malloc_zero(sizeof(insertion_time_queue_t));
       queue->insertion_times = it_queue;
@@ -1879,15 +1880,17 @@ connection_or_flush_from_first_active_circuit(or_connection_t *conn, int max,
       uint32_t cell_waiting_time;
       insertion_time_queue_t *it_queue = queue->insertion_times;
       tor_gettimeofday(&now);
-      flushed = (now.tv_sec % SECONDS_IN_A_DAY) * 100L +
-                 now.tv_usec / 10000L;
+      flushed = (uint32_t)((now.tv_sec % SECONDS_IN_A_DAY) * 100L +
+                 (uint32_t)now.tv_usec / (uint32_t)10000L);
       if (!it_queue || !it_queue->first) {
         log_warn(LD_BUG, "Cannot determine insertion time of cell.");
       } else {
         or_circuit_t *orcirc = TO_OR_CIRCUIT(circ);
         insertion_time_elem_t *elem = it_queue->first;
-        cell_waiting_time = (flushed * 10L + SECONDS_IN_A_DAY * 1000L -
-            elem->insertion_time * 10L) % (SECONDS_IN_A_DAY * 1000L);
+        cell_waiting_time =
+            (uint32_t)((flushed * 10L + SECONDS_IN_A_DAY * 1000L -
+                        elem->insertion_time * 10L) %
+                       (SECONDS_IN_A_DAY * 1000L));
 #undef SECONDS_IN_A_DAY
         elem->counter--;
         if (elem->counter < 1) {

src/or/rendclient.c

@@ -91,8 +91,9 @@ rend_client_send_introduction(origin_circuit_t *introcirc,
     }
   });
   if (!intro_key) {
-    log_warn(LD_BUG, "Internal error: could not find intro key; we "
-             "only have a v2 rend desc with %d intro points.",
+    log_info(LD_REND, "Our introduction point knowledge changed in "
+             "mid-connect! Could not find intro key; we only have a "
+             "v2 rend desc with %d intro points. Giving up.",
              smartlist_len(entry->parsed->intro_nodes));
     goto err;
   }
@@ -128,7 +129,7 @@ rend_client_send_introduction(origin_circuit_t *introcirc,
              REND_DESC_COOKIE_LEN);
       v3_shift += 2+REND_DESC_COOKIE_LEN;
     }
-    set_uint32(tmp+v3_shift+1, htonl(time(NULL)));
+    set_uint32(tmp+v3_shift+1, htonl((uint32_t)time(NULL)));
     v3_shift += 4;
   } /* if version 2 only write version number */
   else if (entry->parsed->protocols & (1<<2)) {
@@ -644,7 +645,7 @@ rend_client_receive_rendezvous(origin_circuit_t *circ, const char *request,
   /* set the windows to default. these are the windows
    * that alice thinks bob has.
    */
-  hop->package_window = CIRCWINDOW_START;
+  hop->package_window = circuit_initial_package_window();
   hop->deliver_window = CIRCWINDOW_START;
 
   onion_append_to_cpath(&circ->cpath, hop);

src/or/rendservice.c

@@ -264,7 +264,7 @@ rend_config_services(or_options_t *options, int validate_only)
 
   for (line = options->RendConfigLines; line; line = line->next) {
     if (!strcasecmp(line->key, "HiddenServiceDir")) {
-      if (service) {
+      if (service) { /* register the one we just finished parsing */
         if (validate_only)
           rend_service_free(service);
         else
@@ -921,7 +921,7 @@ rend_service_introduce(origin_circuit_t *circuit, const char *request,
   len = r;
   if (*buf == 3) {
     /* Version 3 INTRODUCE2 cell. */
-    time_t ts = 0, now = time(NULL);
+    time_t ts = 0;
     v3_shift = 1;
     auth_type = buf[1];
     switch (auth_type) {
@@ -944,13 +944,12 @@ rend_service_introduce(origin_circuit_t *circuit, const char *request,
     }
 
     /* Check timestamp. */
-    memcpy((char*)&ts, buf+1+v3_shift, sizeof(uint32_t));
+    ts = ntohl(get_uint32(buf+1+v3_shift));
     v3_shift += 4;
-    ts = ntohl(ts);
     if ((now - ts) < -1 * REND_REPLAY_TIME_INTERVAL / 2 ||
         (now - ts) > REND_REPLAY_TIME_INTERVAL / 2) {
       log_warn(LD_REND, "INTRODUCE2 cell is too %s. Discarding.",
-          (now - ts) < 0 ? "old" : "new");
+               (now - ts) < 0 ? "old" : "new");
       return -1;
     }
   }
@@ -1101,7 +1100,7 @@ rend_service_introduce(origin_circuit_t *circuit, const char *request,
   circ_needs_uptime = rend_service_requires_uptime(service);
 
   /* help predict this next time */
-  rep_hist_note_used_internal(time(NULL), circ_needs_uptime, 1);
+  rep_hist_note_used_internal(now, circ_needs_uptime, 1);
 
   /* Launch a circuit to alice's chosen rendezvous point.
    */
@@ -1137,7 +1136,7 @@ rend_service_introduce(origin_circuit_t *circuit, const char *request,
   launched->build_state->pending_final_cpath = cpath =
     tor_malloc_zero(sizeof(crypt_path_t));
   cpath->magic = CRYPT_PATH_MAGIC;
-  launched->build_state->expiry_time = time(NULL) + MAX_REND_TIMEOUT;
+  launched->build_state->expiry_time = now + MAX_REND_TIMEOUT;
 
   cpath->dh_handshake_state = dh;
   dh = NULL;
@@ -1477,7 +1476,7 @@ rend_service_rendezvous_has_opened(origin_circuit_t *circuit)
   /* set the windows to default. these are the windows
    * that bob thinks alice has.
    */
-  hop->package_window = CIRCWINDOW_START;
+  hop->package_window = circuit_initial_package_window();
   hop->deliver_window = CIRCWINDOW_START;
 
   onion_append_to_cpath(&circuit->cpath, hop);

src/or/router.c

@@ -772,9 +772,6 @@ consider_testing_reachability(int test_or, int test_dir)
              me->address, me->or_port);
     circuit_launch_by_router(CIRCUIT_PURPOSE_TESTING, me,
                              CIRCLAUNCH_NEED_CAPACITY|CIRCLAUNCH_IS_INTERNAL);
-    control_event_server_status(LOG_NOTICE,
-                                "CHECKING_REACHABILITY ORADDRESS=%s:%d",
-                                me->address, me->or_port);
   }
 
   tor_addr_from_ipv4h(&addr, me->addr);
@@ -790,10 +787,6 @@ consider_testing_reachability(int test_or, int test_dir)
                                DIR_PURPOSE_FETCH_SERVERDESC,
                                ROUTER_PURPOSE_GENERAL,
                                1, "authority.z", NULL, 0, 0);
-
-    control_event_server_status(LOG_NOTICE,
-                                "CHECKING_REACHABILITY DIRADDRESS=%s:%d",
-                                me->address, me->dir_port);
   }
 }
 
@@ -809,8 +802,11 @@ router_orport_found_reachable(void)
                  " Publishing server descriptor." : "");
     can_reach_or_port = 1;
     mark_my_descriptor_dirty();
-    if (!me)
+    if (!me) { /* should never happen */
+      log_warn(LD_BUG, "ORPort found reachable, but I have no routerinfo "
+               "yet. Failing to inform controller of success.");
       return;
+    }
     control_event_server_status(LOG_NOTICE,
                                 "REACHABILITY_SUCCEEDED ORADDRESS=%s:%d",
                                 me->address, me->or_port);
@@ -828,8 +824,11 @@ router_dirport_found_reachable(void)
     can_reach_dir_port = 1;
     if (!me || decide_to_advertise_dirport(get_options(), me->dir_port))
       mark_my_descriptor_dirty();
-    if (!me)
+    if (!me) { /* should never happen */
+      log_warn(LD_BUG, "DirPort found reachable, but I have no routerinfo "
+               "yet. Failing to inform controller of success.");
       return;
+    }
     control_event_server_status(LOG_NOTICE,
                                 "REACHABILITY_SUCCEEDED DIRADDRESS=%s:%d",
                                 me->address, me->dir_port);
@@ -1909,7 +1908,7 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo,
     if (options->DirReqStatistics &&
         load_stats_file("stats"PATH_SEPARATOR"dirreq-stats",
                         "dirreq-stats-end", since, &contents) > 0) {
-      int pos = strlen(s);
+      size_t pos = strlen(s);
       if (strlcpy(s + pos, contents, maxlen - strlen(s)) !=
           strlen(contents)) {
         log_warn(LD_DIR, "Could not write dirreq-stats to extra-info "
@@ -1921,7 +1920,7 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo,
     if (options->EntryStatistics &&
         load_stats_file("stats"PATH_SEPARATOR"entry-stats",
                         "entry-stats-end", since, &contents) > 0) {
-      int pos = strlen(s);
+      size_t pos = strlen(s);
       if (strlcpy(s + pos, contents, maxlen - strlen(s)) !=
           strlen(contents)) {
         log_warn(LD_DIR, "Could not write entry-stats to extra-info "
@@ -1933,7 +1932,7 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo,
     if (options->CellStatistics &&
         load_stats_file("stats"PATH_SEPARATOR"buffer-stats",
                         "cell-stats-end", since, &contents) > 0) {
-      int pos = strlen(s);
+      size_t pos = strlen(s);
       if (strlcpy(s + pos, contents, maxlen - strlen(s)) !=
           strlen(contents)) {
         log_warn(LD_DIR, "Could not write buffer-stats to extra-info "
@@ -1945,7 +1944,7 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo,
     if (options->ExitPortStatistics &&
         load_stats_file("stats"PATH_SEPARATOR"exit-stats",
                         "exit-stats-end", since, &contents) > 0) {
-      int pos = strlen(s);
+      size_t pos = strlen(s);
       if (strlcpy(s + pos, contents, maxlen - strlen(s)) !=
           strlen(contents)) {
         log_warn(LD_DIR, "Could not write exit-stats to extra-info "

src/or/routerparse.c

@@ -102,6 +102,7 @@ typedef enum {
   K_VOTING_DELAY,
 
   K_KNOWN_FLAGS,
+  K_PARAMS,
   K_VOTE_DIGEST,
   K_CONSENSUS_DIGEST,
   K_CONSENSUS_METHODS,
@@ -433,6 +434,7 @@ static token_rule_t networkstatus_token_table[] = {
   T1("valid-until",            K_VALID_UNTIL,      CONCAT_ARGS, NO_OBJ ),
   T1("voting-delay",           K_VOTING_DELAY,     GE(2),       NO_OBJ ),
   T1("known-flags",            K_KNOWN_FLAGS,      ARGS,        NO_OBJ ),
+  T01("params",                K_PARAMS,           ARGS,        NO_OBJ ),
   T( "fingerprint",            K_FINGERPRINT,      CONCAT_ARGS, NO_OBJ ),
 
   CERTIFICATE_MEMBERS
@@ -470,6 +472,7 @@ static token_rule_t networkstatus_consensus_token_table[] = {
   T01("client-versions",     K_CLIENT_VERSIONS, CONCAT_ARGS, NO_OBJ ),
   T01("server-versions",     K_SERVER_VERSIONS, CONCAT_ARGS, NO_OBJ ),
   T01("consensus-method",    K_CONSENSUS_METHOD,    EQ(1),   NO_OBJ),
+  T01("params",                K_PARAMS,           ARGS,        NO_OBJ ),
 
   END_OF_TABLE
 };
@@ -2002,8 +2005,9 @@ routerstatus_parse_entry_from_string(memarea_t *area,
     for (i=0; i < tok->n_args; ++i) {
       if (!strcmpstart(tok->args[i], "Bandwidth=")) {
         int ok;
-        rs->bandwidth = tor_parse_ulong(strchr(tok->args[i], '=')+1, 10,
-                                        0, UINT32_MAX, &ok, NULL);
+        rs->bandwidth = (uint32_t)tor_parse_ulong(strchr(tok->args[i], '=')+1,
+                                                  10, 0, UINT32_MAX,
+                                                  &ok, NULL);
         if (!ok) {
           log_warn(LD_DIR, "Invalid Bandwidth %s", escaped(tok->args[i]));
           goto err;
@@ -2011,8 +2015,9 @@ routerstatus_parse_entry_from_string(memarea_t *area,
         rs->has_bandwidth = 1;
       } else if (!strcmpstart(tok->args[i], "Measured=")) {
         int ok;
-        rs->measured_bw = tor_parse_ulong(strchr(tok->args[i], '=')+1, 10,
-                                          0, UINT32_MAX, &ok, NULL);
+        rs->measured_bw =
+            (uint32_t)tor_parse_ulong(strchr(tok->args[i], '=')+1,
+                                      10, 0, UINT32_MAX, &ok, NULL);
         if (!ok) {
           log_warn(LD_DIR, "Invalid Measured Bandwidth %s",
                    escaped(tok->args[i]));
@@ -2406,6 +2411,34 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out,
     goto err;
   }
 
+  tok = find_opt_by_keyword(tokens, K_PARAMS);
+  if (tok) {
+    inorder = 1;
+    ns->net_params = smartlist_create();
+    for (i = 0; i < tok->n_args; ++i) {
+      int ok=0;
+      char *eq = strchr(tok->args[i], '=');
+      if (!eq) {
+        log_warn(LD_DIR, "Bad element '%s' in params", escaped(tok->args[i]));
+        goto err;
+      }
+      tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, NULL);
+      if (!ok) {
+        log_warn(LD_DIR, "Bad element '%s' in params", escaped(tok->args[i]));
+        goto err;
+      }
+      if (i > 0 && strcmp(tok->args[i-1], tok->args[i]) >= 0) {
+        log_warn(LD_DIR, "%s >= %s", tok->args[i-1], tok->args[i]);
+        inorder = 0;
+      }
+      smartlist_add(ns->net_params, tor_strdup(tok->args[i]));
+    }
+    if (!inorder) {
+      log_warn(LD_DIR, "params not in order");
+      goto err;
+    }
+  }
+
   ns->voters = smartlist_create();
 
   SMARTLIST_FOREACH_BEGIN(tokens, directory_token_t *, _tok) {
@@ -2605,6 +2638,14 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out,
     } else {
       if (tok->object_size >= INT_MAX)
         goto err;
+      /* We already parsed a vote from this voter. Use the first one. */
+      if (v->signature) {
+        log_fn(LOG_PROTOCOL_WARN, LD_DIR, "We received a networkstatus "
+                   "that contains two votes from the same voter. Ignoring "
+                   "the second vote.");
+        continue;
+      }
+
       v->signature = tor_memdup(tok->object_body, tok->object_size);
       v->signature_len = (int) tok->object_size;
     }
@@ -2614,6 +2655,10 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out,
   if (! n_signatures) {
     log_warn(LD_DIR, "No signatures on networkstatus vote.");
     goto err;
+  } else if (ns->type == NS_TYPE_VOTE && n_signatures != 1) {
+    log_warn(LD_DIR, "Received more than one signature on a "
+             "network-status vote.");
+    goto err;
   }
 
   if (eos_out)
@@ -3516,9 +3561,11 @@ tor_version_parse(const char *s, tor_version_t *out)
     if (! close_paren)
       return -1;
     cp += 5;
-    hexlen = (close_paren-cp);
+    if (close_paren-cp > HEX_DIGEST_LEN)
+      return -1;
+    hexlen = (int)(close_paren-cp);
     memset(digest, 0, sizeof(digest));
-    if (hexlen > HEX_DIGEST_LEN || hexlen == 0 || (hexlen % 2) == 1)
+    if ( hexlen == 0 || (hexlen % 2) == 1)
       return -1;
     if (base16_decode(digest, hexlen/2, cp, hexlen))
       return -1;

src/or/test.c

@@ -37,6 +37,15 @@ const char tor_git_revision[] = "";
 #define GEOIP_PRIVATE
 #define MEMPOOL_PRIVATE
 #define ROUTER_PRIVATE
+#define CIRCUIT_PRIVATE
+
+/*
+ * Linux doesn't provide lround in math.h by default, but mac os does...
+ * It's best just to leave math.h out of the picture entirely.
+ */
+//#include <math.h>
+long int lround(double x);
+double fabs(double x);
 
 #include "or.h"
 #include "test.h"
@@ -410,7 +419,7 @@ test_crypto_dh(void)
   char p2[DH_BYTES];
   char s1[DH_BYTES];
   char s2[DH_BYTES];
-  int s1len, s2len;
+  ssize_t s1len, s2len;
 
   test_eq(crypto_dh_get_bytes(dh1), DH_BYTES);
   test_eq(crypto_dh_get_bytes(dh2), DH_BYTES);
@@ -1114,6 +1123,24 @@ test_util(void)
               tor_parse_uint64("12345678901",10,500,INT32_MAX, &i, &cp));
   test_assert(i == 0);
 
+  {
+  /* Test tor_parse_double. */
+  double d = tor_parse_double("10", 0, UINT64_MAX,&i,NULL);
+  test_assert(i == 1);
+  test_assert(DBL_TO_U64(d) == 10);
+  d = tor_parse_double("0", 0, UINT64_MAX,&i,NULL);
+  test_assert(i == 1);
+  test_assert(DBL_TO_U64(d) == 0);
+  d = tor_parse_double(" ", 0, UINT64_MAX,&i,NULL);
+  test_assert(i == 0);
+  d = tor_parse_double(".0a", 0, UINT64_MAX,&i,NULL);
+  test_assert(i == 0);
+  d = tor_parse_double(".0a", 0, UINT64_MAX,&i,&cp);
+  test_assert(i == 1);
+  d = tor_parse_double("-.0", 0, UINT64_MAX,&i,NULL);
+  test_assert(i == 1);
+  }
+
   /* Test failing snprintf cases */
   test_eq(-1, tor_snprintf(buf, 0, "Foo"));
   test_eq(-1, tor_snprintf(buf, 2, "Foo"));
@@ -3337,6 +3364,220 @@ done:
   return;
 }
 
+static void
+test_dirutil_param_voting(void)
+{
+  networkstatus_t vote1, vote2, vote3, vote4;
+  smartlist_t *votes = smartlist_create();
+  char *res = NULL;
+
+  /* dirvote_compute_params only looks at the net_params field of the votes,
+     so that's all we need to set.
+   */
+  memset(&vote1, 0, sizeof(vote1));
+  memset(&vote2, 0, sizeof(vote2));
+  memset(&vote3, 0, sizeof(vote3));
+  memset(&vote4, 0, sizeof(vote4));
+  vote1.net_params = smartlist_create();
+  vote2.net_params = smartlist_create();
+  vote3.net_params = smartlist_create();
+  vote4.net_params = smartlist_create();
+  smartlist_split_string(vote1.net_params,
+                         "ab=90 abcd=20 cw=50 x-yz=-99", NULL, 0, 0);
+  smartlist_split_string(vote2.net_params,
+                         "ab=27 cw=5 x-yz=88", NULL, 0, 0);
+  smartlist_split_string(vote3.net_params,
+                         "abcd=20 c=60 cw=500 x-yz=-9 zzzzz=101", NULL, 0, 0);
+  smartlist_split_string(vote4.net_params,
+                         "ab=900 abcd=200 c=1 cw=51 x-yz=100", NULL, 0, 0);
+  test_eq(100, networkstatus_get_param(&vote4, "x-yz", 50));
+  test_eq(222, networkstatus_get_param(&vote4, "foobar", 222));
+
+  smartlist_add(votes, &vote1);
+  smartlist_add(votes, &vote2);
+  smartlist_add(votes, &vote3);
+  smartlist_add(votes, &vote4);
+
+  res = dirvote_compute_params(votes);
+  test_streq(res,
+             "ab=90 abcd=20 c=1 cw=50 x-yz=-9 zzzzz=101");
+
+ done:
+  tor_free(res);
+  SMARTLIST_FOREACH(vote1.net_params, char *, cp, tor_free(cp));
+  SMARTLIST_FOREACH(vote2.net_params, char *, cp, tor_free(cp));
+  SMARTLIST_FOREACH(vote3.net_params, char *, cp, tor_free(cp));
+  SMARTLIST_FOREACH(vote4.net_params, char *, cp, tor_free(cp));
+  smartlist_free(vote1.net_params);
+  smartlist_free(vote2.net_params);
+  smartlist_free(vote3.net_params);
+  smartlist_free(vote4.net_params);
+
+  return;
+}
+
+static void
+test_circuit_timeout(void)
+{
+  /* Plan:
+   *  1. Generate 1000 samples
+   *  2. Estimate parameters
+   *  3. If difference, repeat
+   *  4. Save state
+   *  5. load state
+   *  6. Estimate parameters
+   *  7. compare differences
+   */
+  circuit_build_times_t initial;
+  circuit_build_times_t estimate;
+  circuit_build_times_t final;
+  double timeout1, timeout2;
+  or_state_t state;
+  char *msg;
+  int i, runs;
+  circuit_build_times_init(&initial);
+  circuit_build_times_init(&estimate);
+  circuit_build_times_init(&final);
+
+  memset(&state, 0, sizeof(or_state_t));
+
+  circuitbuild_running_unit_tests();
+#define timeout0 (build_time_t)(30*1000.0)
+  initial.Xm = 750;
+  circuit_build_times_initial_alpha(&initial, BUILDTIMEOUT_QUANTILE_CUTOFF,
+                                    timeout0);
+  do {
+    int n = 0;
+    for (i=0; i < MIN_CIRCUITS_TO_OBSERVE; i++) {
+      if (circuit_build_times_add_time(&estimate,
+              circuit_build_times_generate_sample(&initial, 0, 1)) == 0) {
+        n++;
+      }
+    }
+    circuit_build_times_update_alpha(&estimate);
+    timeout1 = circuit_build_times_calculate_timeout(&estimate,
+                                  BUILDTIMEOUT_QUANTILE_CUTOFF);
+    circuit_build_times_set_timeout(&estimate);
+    log_warn(LD_CIRC, "Timeout is %lf, Xm is %d", timeout1, estimate.Xm);
+    /* XXX: 5% distribution error may not be the right metric */
+  } while (fabs(circuit_build_times_cdf(&initial, timeout0) -
+                circuit_build_times_cdf(&initial, timeout1)) > 0.05
+                /* 5% error */
+           && estimate.total_build_times < NCIRCUITS_TO_OBSERVE);
+
+  test_assert(estimate.total_build_times < NCIRCUITS_TO_OBSERVE);
+
+  circuit_build_times_update_state(&estimate, &state);
+  test_assert(circuit_build_times_parse_state(&final, &state, &msg) == 0);
+
+  circuit_build_times_update_alpha(&final);
+  timeout2 = circuit_build_times_calculate_timeout(&final,
+                                 BUILDTIMEOUT_QUANTILE_CUTOFF);
+
+  circuit_build_times_set_timeout(&final);
+  log_warn(LD_CIRC, "Timeout is %lf, Xm is %d", timeout2, final.Xm);
+
+  test_assert(fabs(circuit_build_times_cdf(&initial, timeout0) -
+                   circuit_build_times_cdf(&initial, timeout2)) < 0.05);
+
+  for (runs = 0; runs < 50; runs++) {
+    int build_times_idx = 0;
+    int total_build_times = 0;
+
+    final.timeout_ms = BUILD_TIMEOUT_INITIAL_VALUE;
+    estimate.timeout_ms = BUILD_TIMEOUT_INITIAL_VALUE;
+
+    for (i = 0; i < RECENT_CIRCUITS*2; i++) {
+      circuit_build_times_network_circ_success(&estimate);
+      circuit_build_times_add_time(&estimate,
+            circuit_build_times_generate_sample(&estimate, 0,
+                BUILDTIMEOUT_QUANTILE_CUTOFF));
+      estimate.have_computed_timeout = 1;
+      circuit_build_times_network_circ_success(&estimate);
+      circuit_build_times_add_time(&final,
+            circuit_build_times_generate_sample(&final, 0,
+                BUILDTIMEOUT_QUANTILE_CUTOFF));
+      final.have_computed_timeout = 1;
+    }
+
+    test_assert(!circuit_build_times_network_check_changed(&estimate));
+    test_assert(!circuit_build_times_network_check_changed(&final));
+
+    /* Reset liveness to be non-live */
+    final.liveness.network_last_live = 0;
+    estimate.liveness.network_last_live = 0;
+
+    build_times_idx = estimate.build_times_idx;
+    total_build_times = estimate.total_build_times;
+    for (i = 0; i < NETWORK_NONLIVE_TIMEOUT_COUNT; i++) {
+      test_assert(circuit_build_times_network_check_live(&estimate));
+      test_assert(circuit_build_times_network_check_live(&final));
+
+      if (circuit_build_times_add_timeout(&estimate, 0,
+                 (time_t)(approx_time()-estimate.timeout_ms/1000.0-1)))
+        estimate.have_computed_timeout = 1;
+      if (circuit_build_times_add_timeout(&final, 0,
+                 (time_t)(approx_time()-final.timeout_ms/1000.0-1)))
+        final.have_computed_timeout = 1;
+    }
+
+    test_assert(!circuit_build_times_network_check_live(&estimate));
+    test_assert(!circuit_build_times_network_check_live(&final));
+
+    for ( ; i < NETWORK_NONLIVE_DISCARD_COUNT; i++) {
+      if (circuit_build_times_add_timeout(&estimate, 0,
+                (time_t)(approx_time()-estimate.timeout_ms/1000.0-1)))
+        estimate.have_computed_timeout = 1;
+
+      if (i < NETWORK_NONLIVE_DISCARD_COUNT-1) {
+        if (circuit_build_times_add_timeout(&final, 0,
+                (time_t)(approx_time()-final.timeout_ms/1000.0-1)))
+          final.have_computed_timeout = 1;
+      }
+    }
+
+    test_assert(!circuit_build_times_network_check_live(&estimate));
+    test_assert(!circuit_build_times_network_check_live(&final));
+
+    log_info(LD_CIRC, "idx: %d %d, tot: %d %d",
+             build_times_idx, estimate.build_times_idx,
+             total_build_times, estimate.total_build_times);
+
+    /* Check rollback index. Should match top of loop. */
+    test_assert(build_times_idx == estimate.build_times_idx);
+    test_assert(total_build_times == estimate.total_build_times);
+
+    /* Now simulate that the network has become live and we need
+     * a change */
+    circuit_build_times_network_is_live(&estimate);
+    circuit_build_times_network_is_live(&final);
+
+    for (i = 0; i < MAX_RECENT_TIMEOUT_COUNT; i++) {
+      if (circuit_build_times_add_timeout(&estimate, 1, approx_time()-1))
+        estimate.have_computed_timeout = 1;
+
+      if (i < MAX_RECENT_TIMEOUT_COUNT-1) {
+        if (circuit_build_times_add_timeout(&final, 1, approx_time()-1))
+          final.have_computed_timeout = 1;
+      }
+    }
+
+    test_assert(estimate.liveness.after_firsthop_idx == 0);
+    test_assert(final.liveness.after_firsthop_idx ==
+                MAX_RECENT_TIMEOUT_COUNT-1);
+
+    test_assert(circuit_build_times_network_check_live(&estimate));
+    test_assert(circuit_build_times_network_check_live(&final));
+
+    if (circuit_build_times_add_timeout(&final, 1, approx_time()-1))
+      final.have_computed_timeout = 1;
+
+  }
+
+done:
+  return;
+}
+
 extern const char AUTHORITY_CERT_1[];
 extern const char AUTHORITY_SIGNKEY_1[];
 extern const char AUTHORITY_CERT_2[];
@@ -3494,6 +3735,9 @@ test_v3_networkstatus(void)
   crypto_pk_get_digest(cert1->identity_key, voter->identity_digest);
   smartlist_add(vote->voters, voter);
   vote->cert = authority_cert_dup(cert1);
+  vote->net_params = smartlist_create();
+  smartlist_split_string(vote->net_params, "circuitwindow=101 foo=990",
+                         NULL, 0, 0);
   vote->routerstatus_list = smartlist_create();
   /* add the first routerstatus. */
   vrs = tor_malloc_zero(sizeof(vote_routerstatus_t));
@@ -3635,6 +3879,9 @@ test_v3_networkstatus(void)
   vote->dist_seconds = 300;
   authority_cert_free(vote->cert);
   vote->cert = authority_cert_dup(cert2);
+  vote->net_params = smartlist_create();
+  smartlist_split_string(vote->net_params, "bar=2000000000 circuitwindow=20",
+                         NULL, 0, 0);
   tor_free(vote->client_versions);
   tor_free(vote->server_versions);
   voter = smartlist_get(vote->voters, 0);
@@ -3673,6 +3920,9 @@ test_v3_networkstatus(void)
   vote->dist_seconds = 250;
   authority_cert_free(vote->cert);
   vote->cert = authority_cert_dup(cert3);
+  vote->net_params = smartlist_create();
+  smartlist_split_string(vote->net_params, "circuitwindow=80 foo=660",
+                         NULL, 0, 0);
   smartlist_add(vote->supported_methods, tor_strdup("4"));
   vote->client_versions = tor_strdup("0.1.2.14,0.1.2.17");
   vote->server_versions = tor_strdup("0.1.2.10,0.1.2.15,0.1.2.16");
@@ -3729,6 +3979,10 @@ test_v3_networkstatus(void)
   test_streq(cp, "Authority:Exit:Fast:Guard:MadeOfCheese:MadeOfTin:"
              "Running:Stable:V2Dir:Valid");
   tor_free(cp);
+  cp = smartlist_join_strings(con->net_params, ":", 0, NULL);
+  test_streq(cp, "bar=2000000000:circuitwindow=80:foo=660");
+  tor_free(cp);
+
   test_eq(4, smartlist_len(con->voters)); /*3 voters, 1 legacy key.*/
   /* The voter id digests should be in this order. */
   test_assert(memcmp(cert2->cache_info.identity_digest,
@@ -4848,6 +5102,8 @@ static struct {
   ENT(dir_format),
   ENT(dirutil),
   SUBENT(dirutil, measured_bw),
+  SUBENT(dirutil, param_voting),
+  ENT(circuit_timeout),
   ENT(v3_networkstatus),
   ENT(policies),
   ENT(rend_fns),

src/tools/tor-gencert.c

@@ -70,7 +70,7 @@ show_help(void)
 static void
 crypto_log_errors(int severity, const char *doing)
 {
-  unsigned int err;
+  unsigned long err;
   const char *msg, *lib, *func;
   while ((err = ERR_get_error()) != 0) {
     msg = (const char*)ERR_reason_error_string(err);
@@ -94,7 +94,7 @@ load_passphrase(void)
 {
   char *cp;
   char buf[1024]; /* "Ought to be enough for anybody." */
-  int n = read_all(passphrase_fd, buf, sizeof(buf), 0);
+  ssize_t n = read_all(passphrase_fd, buf, sizeof(buf), 0);
   if (n < 0) {
     log_err(LD_GENERAL, "Couldn't read from passphrase fd: %s",
             strerror(errno));

src/tools/tor-resolve.c

@@ -51,7 +51,7 @@ static void usage(void) ATTR_NORETURN;
 /** Set *<b>out</b> to a newly allocated SOCKS4a resolve request with
  * <b>username</b> and <b>hostname</b> as provided.  Return the number
  * of bytes in the request. */
-static int
+static ssize_t
 build_socks_resolve_request(char **out,
                             const char *username,
                             const char *hostname,
@@ -184,7 +184,7 @@ do_resolve(const char *hostname, uint32_t sockshost, uint16_t socksport,
   int s;
   struct sockaddr_in socksaddr;
   char *req = NULL;
-  int len = 0;
+  ssize_t len = 0;
 
   tor_assert(hostname);
   tor_assert(result_addr);

src/win32/orconfig.h

@@ -226,5 +226,5 @@
 #define USING_TWOS_COMPLEMENT
 
 /* Version number of package */
-#define VERSION "0.2.2.1-alpha"
+#define VERSION "0.2.2.2-alpha"