Filename: 154-automatic-updates.txt Title: Automatic Software Update Protocol Author: Matt Edman Created: 30-July-2008 Status: Superseded Target: 0.2.1.x Superseded by thandy-spec.txt Scope This proposal specifies the method by which an automatic update client can determine the most recent recommended Tor installation package for the user's platform, download the package, and then verify that the package was downloaded successfully. While this proposal focuses on only the Tor software, the protocol defined is sufficiently extensible such that other components of the Tor bundles, like Vidalia, Polipo, and Torbutton, can be managed and updated by the automatic update client as well. The initial target platform for the automatic update framework is Windows, given that's the platform used by a majority of our users and that it lacks a sane package management system that many Linux distributions already have. Our second target platform will be Mac OS X, and so the protocol will be designed with this near-future direction in mind. Other client-side aspects of the automatic update process, such as user interaction, the interface presented, and actual package installation procedure, are outside the scope of this proposal. Motivation Tor releases new versions frequently, often with important security, anonymity, and stability fixes. Thus, it is important for users to be able to promptly recognize when new versions are available and to easily download, authenticate, and install updated Tor and Tor-related software packages. Tor's control protocol [2] provides a method by which controllers can identify when the user's Tor software is obsolete or otherwise no longer recommended. Currently, however, no mechanism exists for clients to automatically download and install updated Tor and Tor-related software for the user. Design Overview The core of the automatic update framework is a well-defined file called a "recommended-packages" file. The recommended-packages file is accessible via HTTP[S] at one or more well-defined URLs. An example recommended-packages URL may be: https://updates.torproject.org/recommended-packages The recommended-packages document is formatted according to Section 1.2 below and specifies the most recent recommended installation package versions for Tor or Tor-related software, as well as URLs at which the packages and their signatures can be downloaded. An automatic update client process runs on the Tor user's computer and periodically retrieves the recommended-packages file according to the method described in Section 2.0. As described further in Section 1.2, the recommended-packages file is signed and can be verified by the automatic update client with one or more public keys included in the client software. Since it is signed, the recommended-packages file can be mirrored by multiple hosts (e.g., Tor directory authorities), whose URLs are included in the automatic update client's configuration. After retrieving and verifying the recommended-packages file, the automatic update client compares the versions of the recommended software packages listed in the file with those currently installed on the end-user's computer. If one or more of the installed packages is determined to be out of date, an updated package and its signature will be downloaded from one of the package URLs listed in the recommended-packages file as described in Section 2.2. The automatic update system uses a multilevel signing key scheme for package signatures. There are a small number of entities we call "packaging authorities" that each have their own signing key. A packaging authority is responsible for signing and publishing the recommended-packages file. Additionally, each individual packager responsible for producing an installation package for one or more platforms has their own signing key. Every packager's signing key must be signed by at least one of the packaging authority keys. Specification 1. recommended-packages Specification In this section we formally specify the format of the published recommended-packages file. 1.1. Document Meta-format The recommended-packages document follows the lightweight extensible information format defined in Tor's directory protocol specification [1]. In the interest of self-containment, we have reproduced the relevant portions of that format's specification in this Section. (Credits to Nick Mathewson for much of the original format definition language.) The highest level object is a Document, which consists of one or more Items. Every Item begins with a KeywordLine, followed by zero or more Objects. A KeywordLine begins with a Keyword, optionally followed by whitespace and more non-newline characters, and ends with a newline. A Keyword is a sequence of one or more characters in the set [A-Za-z0-9-]. An Object is a block of encoded data in pseudo-Open-PGP-style armor. (cf. RFC 2440) More formally: Document ::= (Item | NL)+ Item ::= KeywordLine Object* KeywordLine ::= Keyword NL | Keyword WS ArgumentChar+ NL Keyword ::= KeywordChar+ KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-' ArgumentChar ::= any printing ASCII character except NL. WS ::= (SP | TAB)+ Object ::= BeginLine Base-64-encoded-data EndLine BeginLine ::= "-----BEGIN " Keyword "-----" NL EndLine ::= "-----END " Keyword "-----" NL The BeginLine and EndLine of an Object must use the same keyword. In our Document description below, we also tag Items with a multiplicity in brackets. Possible tags are: "At start, exactly once": These items MUST occur in every instance of the document type, and MUST appear exactly once, and MUST be the first item in their documents. "Exactly once": These items MUST occur exactly one time in every instance of the document type. "Once or more": These items MUST occur at least once in any instance of the document type, and MAY occur more than once. "At end, exactly once": These items MUST occur in every instance of the document type, and MUST appear exactly once, and MUST be the last item in their documents. 1.2. recommended-packages Document Format When interpreting a recommended-packages Document, software MUST ignore any KeywordLine that starts with a keyword it doesn't recognize; future implementations MUST NOT require current automatic update clients to understand any KeywordLine not currently described. In lines that take multiple arguments, extra arguments SHOULD be accepted and ignored. The currently defined Items contained in a recommended-packages document are: "recommended-packages-format" SP number NL [Exactly once] This Item specifies the version of the recommended-packages format that is contained in the subsequent document. The version defined in this proposal is version "1". Subsequent iterations of this protocol MUST increment this value if they introduce incompatible changes to the document format and MAY increment this value if they only introduce additional Keywords. "published" SP YYYY-MM-DD SP HH:MM:SS NL [Exactly once] The time, in GMT, when this recommended-packages document was generated. Automatic update clients SHOULD ignore Documents over 60 days old. "tor-stable-win32-version" SP TorVersion NL [Exactly once] This keyword specifies the latest recommended release of Tor's "stable" branch for the Windows platform that has an installation package available. Note that this version does not necessarily correspond to the most recently tagged stable Tor version, since that version may not yet have an installer package available, or may have known issues on Windows. The TorVersion field is formatted according to Section 2 of Tor's version specification [3]. "tor-stable-win32-package" SP Url NL [Once or more] This Item specifies the location from which the most recent recommended Windows installation package for Tor's stable branch can be downloaded. When this Item appears multiple times within the Document, automatic update clients SHOULD select randomly from the available package mirrors. "tor-dev-win32-version" SP TorVersion NL [Exactly once] This Item specifies the latest recommended release of Tor's "development" branch for the Windows platform that has an installation package available. The same caveats from the description of "tor-stable-win32-version" also apply to this keyword. The TorVersion field is formatted according to Section 2 of Tor's version specification [3]. "tor-dev-win32-package" SP Url NL [Once or more] This Item specifies the location from which the most recent recommended Windows installation package and its signature for Tor's development branch can be downloaded. When this Keyword appears multiple times within the Document, automatic update clients SHOULD select randomly from the available package mirrors. "signature" NL SIGNATURE NL [At end, exactly once] The "SIGNATURE" Object contains a PGP signature (using a packaging authority signing key) of the entire document, taken from the beginning of the "recommended-packages-format" keyword, through the newline after the "signature" Keyword. 2. Automatic Update Client Behavior The client-side component of the automatic update framework is an application that runs on the end-user's machine. It is responsible for fetching and verifying a recommended-packages document, as well as downloading, verifying, and subsequently installing any necessary updated software packages. 2.1. Download and verify a recommended-packages document The first step in the automatic update process is for the client to download a copy of the recommended-packages file. The automatic update client contains a (hardcoded and/or user-configurable) list of URLs from which it will attempt to retrieve a recommended-packages file. Connections to each of the recommended-packages URLs SHOULD be attempted in the following order: 1) HTTPS over Tor 2) HTTP over Tor 3) Direct HTTPS 4) Direct HTTP If the client fails to retrieve a recommended-packages document via any of the above connection methods from any of the configured URLs, the client SHOULD retry its download attempts following an exponential back-off algorithm. After the first failed attempt, the client SHOULD delay one hour before attempting again, up to a maximum of 24 hours delay between retry attempts. After successfully downloading a recommended-packages file, the automatic update client will verify the signature using one of the public keys distributed with the client software. If more than one recommended-packages file is downloaded and verified, the file with the most recent "published" date that is verified will be retained and the rest discarded. 2.2. Download and verify the updated packages The automatic update client next compares the latest recommended package version from the recommended-packages document with the currently installed Tor version. If the user currently has installed a Tor version from Tor's "development" branch, then the version specified in "tor-dev-*-version" Item is used for comparison. Similarly, if the user currently has installed a Tor version from Tor's "stable" branch, then the version specified in the "tor-stable-*version" Item is used for comparison. Version comparisons are done according to Tor's version specification [3]. If the automatic update client determines an installation package newer than the user's currently installed version is available, it will attempt to download a package appropriate for the user's platform and Tor branch from a URL specified by a "tor-[branch]-[platform]-package" Item. If more than one mirror for the selected package is available, a mirror will be chosen at random from all those available. The automatic update client must also download a ".asc" signature file for the retrieved package. The URL for the package signature is the same as that for the package itself, except with the extension ".asc" appended to the package URL. Connections to download the updated package and its signature SHOULD be attempted in the same order described in Section 2.1. After completing the steps described in Sections 2.1 and 2.2, the automatic update client will have downloaded and verified a copy of the latest Tor installation package. It can then take whatever subsequent platform-specific steps are necessary to install the downloaded software updates. 2.3. Periodic checking for updates The automatic update client SHOULD maintain a local state file in which it records (at a minimum) the timestamp at which it last retrieved a recommended-packages file and the timestamp at which the client last successfully downloaded and installed a software update. Automatic update clients SHOULD check for an updated recommended-packages document at most once per day but at least once every 30 days. 3. Future Extensions There are several possible areas for future extensions of this framework. The extensions below are merely suggestions and should be the subject of their own proposal before being implemented. 3.1. Additional Software Updates There are several software packages often included in Tor bundles besides Tor, such as Vidalia, Privoxy or Polipo, and Torbutton. The versions and download locations of updated installation packages for these bundle components can be easily added to the recommended-packages document specification above. 3.2. Including ChangeLog Information It may be useful for automatic update clients to be able to display for users a summary of the changes made in the latest Tor or Tor-related software release, before the user chooses to install the update. In the future, we can add keywords to the specification in Section 1.2 that specify the location of a ChangeLog file for the latest recommended package versions. It may also be desirable to allow localized ChangeLog information, so that the automatic update client can fetch release notes in the end-user's preferred language. 3.3. Weighted Package Mirror Selection We defined in Section 1.2 a method by which automatic update clients can select from multiple available package mirrors. We may want to add a Weight argument to the "*-package" Items that allows the recommended-packages file to suggest to clients the probability with which a package mirror should be chosen. This will allow clients to more appropriately distribute package downloads across available mirrors proportional to their approximate bandwidth. Implementation Implementation of this proposal will consist of two separate components. The first component is a small "au-publish" tool that takes as input a configuration file specifying the information described in Section 1.2 and a private key. The tool is run by a "packaging authority" (someone responsible for publishing updated installation packages), who will be prompted to enter the passphrase for the private key used to sign the recommended-packages document. The output of the tool is a document formatted according to Section 1.2, with a signature appended at the end. The resulting document can then be published to any of the update mirrors. The second component is an "au-client" tool that is run on the end-user's machine. It periodically checks for updated installation packages according to Section 2 and fetches the packages if necessary. The public keys used to sign the recommended-packages file and any of the published packages are included in the "au-client" tool. References [1] Tor directory protocol (version 3), https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/dir-spec.txt [2] Tor control protocol (version 2), https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/control-spec.txt [3] Tor version specification, https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/version-spec.txt