tor/doc/spec/proposals/154-automatic-updates.txt

Filename: 154-automatic-updates.txt
Title: Automatic Software Update Protocol
Author: Matt Edman
Created: 30-July-2008
Status: Superseded
Target: 0.2.1.x

Superseded by thandy-spec.txt

Scope

  This proposal specifies the method by which an automatic update client can
  determine the most recent recommended Tor installation package for the
  user's platform, download the package, and then verify that the package was
  downloaded successfully. While this proposal focuses on only the Tor
  software, the protocol defined is sufficiently extensible such that other
  components of the Tor bundles, like Vidalia, Polipo, and Torbutton, can be
  managed and updated by the automatic update client as well.

  The initial target platform for the automatic update framework is Windows,
  given that's the platform used by a majority of our users and that it lacks
  a sane package management system that many Linux distributions already have.
  Our second target platform will be Mac OS X, and so the protocol will be
  designed with this near-future direction in mind.

  Other client-side aspects of the automatic update process, such as user
  interaction, the interface presented, and actual package installation
  procedure, are outside the scope of this proposal.


Motivation

  Tor releases new versions frequently, often with important security,
  anonymity, and stability fixes. Thus, it is important for users to be able
  to promptly recognize when new versions are available and to easily
  download, authenticate, and install updated Tor and Tor-related software
  packages.

  Tor's control protocol [2] provides a method by which controllers can
  identify when the user's Tor software is obsolete or otherwise no longer
  recommended. Currently, however, no mechanism exists for clients to
  automatically download and install updated Tor and Tor-related software for
  the user.


Design Overview

  The core of the automatic update framework is a well-defined file called a
  "recommended-packages" file. The recommended-packages file is accessible via
  HTTP[S] at one or more well-defined URLs. An example recommended-packages
  URL may be:

    https://updates.torproject.org/recommended-packages

  The recommended-packages document is formatted according to Section 1.2
  below and specifies the most recent recommended installation package
  versions for Tor or Tor-related software, as well as URLs at which the
  packages and their signatures can be downloaded.

  An automatic update client process runs on the Tor user's computer and
  periodically retrieves the recommended-packages file according to the method
  described in Section 2.0. As described further in Section 1.2, the
  recommended-packages file is signed and can be verified by the automatic
  update client with one or more public keys included in the client software.
  Since it is signed, the recommended-packages file can be mirrored by
  multiple hosts (e.g., Tor directory authorities), whose URLs are included in
  the automatic update client's configuration.

  After retrieving and verifying the recommended-packages file, the automatic
  update client compares the versions of the recommended software packages
  listed in the file with those currently installed on the end-user's
  computer. If one or more of the installed packages is determined to be out
  of date, an updated package and its signature will be downloaded from one of
  the package URLs listed in the recommended-packages file as described in
  Section 2.2.

  The automatic update system uses a multilevel signing key scheme for package
  signatures. There are a small number of entities we call "packaging
  authorities" that each have their own signing key. A packaging authority is
  responsible for signing and publishing the recommended-packages file.
  Additionally, each individual packager responsible for producing an
  installation package for one or more platforms has their own signing key.
  Every packager's signing key must be signed by at least one of the packaging
  authority keys.


Specification

  1. recommended-packages Specification

  In this section we formally specify the format of the published
  recommended-packages file.

  1.1. Document Meta-format

  The recommended-packages document follows the lightweight extensible
  information format defined in Tor's directory protocol specification [1]. In
  the interest of self-containment, we have reproduced the relevant portions
  of that format's specification in this Section. (Credits to Nick Mathewson
  for much of the original format definition language.)

  The highest level object is a Document, which consists of one or more
  Items.  Every Item begins with a KeywordLine, followed by zero or more
  Objects. A KeywordLine begins with a Keyword, optionally followed by
  whitespace and more non-newline characters, and ends with a newline.  A
  Keyword is a sequence of one or more characters in the set [A-Za-z0-9-].
  An Object is a block of encoded data in pseudo-Open-PGP-style
  armor. (cf. RFC 2440)

  More formally:

    Document     ::= (Item | NL)+
    Item         ::= KeywordLine Object*
    KeywordLine  ::= Keyword NL | Keyword WS ArgumentChar+ NL
    Keyword      ::= KeywordChar+
    KeywordChar  ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-'
    ArgumentChar ::= any printing ASCII character except NL.
    WS           ::= (SP | TAB)+
    Object       ::= BeginLine Base-64-encoded-data EndLine
    BeginLine    ::= "-----BEGIN " Keyword "-----" NL
    EndLine      ::= "-----END " Keyword "-----" NL

    The BeginLine and EndLine of an Object must use the same keyword.

  In our Document description below, we also tag Items with a multiplicity in
  brackets. Possible tags are:

    "At start, exactly once": These items MUST occur in every instance of the
    document type, and MUST appear exactly once, and MUST be the first item in
    their documents.

    "Exactly once": These items MUST occur exactly one time in every
    instance of the document type.

    "Once or more": These items MUST occur at least once in any instance
    of the document type, and MAY occur more than once.

    "At end, exactly once": These items MUST occur in every instance of
    the document type, and MUST appear exactly once, and MUST be the
    last item in their documents.

  1.2. recommended-packages Document Format

  When interpreting a recommended-packages Document, software MUST ignore
  any KeywordLine that starts with a keyword it doesn't recognize; future
  implementations MUST NOT require current automatic update clients to
  understand any KeywordLine not currently described.

  In lines that take multiple arguments, extra arguments SHOULD be
  accepted and ignored.

  The currently defined Items contained in a recommended-packages document
  are:

    "recommended-packages-format" SP number NL

      [Exactly once]

      This Item specifies the version of the recommended-packages format that
      is contained in the subsequent document. The version defined in this
      proposal is version "1". Subsequent iterations of this protocol MUST
      increment this value if they introduce incompatible changes to the
      document format and MAY increment this value if they only introduce
      additional Keywords.

    "published" SP YYYY-MM-DD SP HH:MM:SS NL

      [Exactly once]

      The time, in GMT, when this recommended-packages document was generated.
      Automatic update clients SHOULD ignore Documents over 60 days old.

    "tor-stable-win32-version" SP TorVersion NL

      [Exactly once]

      This keyword specifies the latest recommended release of Tor's "stable"
      branch for the Windows platform that has an installation package
      available. Note that this version does not necessarily correspond to the
      most recently tagged stable Tor version, since that version may not yet
      have an installer package available, or may have known issues on
      Windows.

      The TorVersion field is formatted according to Section 2 of Tor's
      version specification [3].

    "tor-stable-win32-package" SP Url NL

      [Once or more]

      This Item specifies the location from which the most recent
      recommended Windows installation package for Tor's stable branch can be
      downloaded.

      When this Item appears multiple times within the Document, automatic
      update clients SHOULD select randomly from the available package
      mirrors.

    "tor-dev-win32-version" SP TorVersion NL

      [Exactly once]

      This Item specifies the latest recommended release of Tor's
      "development" branch for the Windows platform that has an installation
      package available. The same caveats from the description of
      "tor-stable-win32-version" also apply to this keyword.

      The TorVersion field is formatted according to Section 2 of Tor's
      version specification [3].

    "tor-dev-win32-package" SP Url NL

      [Once or more]

      This Item specifies the location from which the most recent recommended
      Windows installation package and its signature for Tor's development
      branch can be downloaded.

      When this Keyword appears multiple times within the Document, automatic
      update clients SHOULD select randomly from the available package
      mirrors.

    "signature" NL SIGNATURE NL

      [At end, exactly once]

      The "SIGNATURE" Object contains a PGP signature (using a packaging
      authority signing key) of the entire document, taken from the beginning
      of the "recommended-packages-format" keyword, through the newline after
      the "signature" Keyword.


  2. Automatic Update Client Behavior

  The client-side component of the automatic update framework is an
  application that runs on the end-user's machine. It is responsible for
  fetching and verifying a recommended-packages document, as well as
  downloading, verifying, and subsequently installing any necessary updated
  software packages.

  2.1. Download and verify a recommended-packages document

  The first step in the automatic update process is for the client to download
  a copy of the recommended-packages file. The automatic update client
  contains a (hardcoded and/or user-configurable) list of URLs from which it
  will attempt to retrieve a recommended-packages file.

  Connections to each of the recommended-packages URLs SHOULD be attempted in
  the following order:

    1) HTTPS over Tor
    2) HTTP over Tor
    3) Direct HTTPS
    4) Direct HTTP

  If the client fails to retrieve a recommended-packages document via any of
  the above connection methods from any of the configured URLs, the client
  SHOULD retry its download attempts following an exponential back-off
  algorithm. After the first failed attempt, the client SHOULD delay one hour
  before attempting again, up to a maximum of 24 hours delay between retry
  attempts.

  After successfully downloading a recommended-packages file, the automatic
  update client will verify the signature using one of the public keys
  distributed with the client software. If more than one recommended-packages
  file is downloaded and verified, the file with the most recent "published"
  date that is verified will be retained and the rest discarded.

  2.2. Download and verify the updated packages

  The automatic update client next compares the latest recommended package
  version from the recommended-packages document with the currently installed
  Tor version. If the user currently has installed a Tor version from Tor's
  "development" branch, then the version specified in "tor-dev-*-version" Item
  is used for comparison. Similarly, if the user currently has installed a Tor
  version from Tor's "stable" branch, then the version specified in the
  "tor-stable-*version" Item is used for comparison. Version comparisons are
  done according to Tor's version specification [3].

  If the automatic update client determines an installation package newer than
  the user's currently installed version is available, it will attempt to
  download a package appropriate for the user's platform and Tor branch from a
  URL specified by a "tor-[branch]-[platform]-package" Item. If more than one
  mirror for the selected package is available, a mirror will be chosen at
  random from all those available.

  The automatic update client must also download a ".asc" signature file for
  the retrieved package. The URL for the package signature is the same as that
  for the package itself, except with the extension ".asc" appended to the
  package URL.

  Connections to download the updated package and its signature SHOULD be
  attempted in the same order described in Section 2.1.

  After completing the steps described in Sections 2.1 and 2.2, the automatic
  update client will have downloaded and verified a copy of the latest Tor
  installation package. It can then take whatever subsequent platform-specific
  steps are necessary to install the downloaded software updates.

  2.3. Periodic checking for updates

  The automatic update client SHOULD maintain a local state file in which it
  records (at a minimum) the timestamp at which it last retrieved a
  recommended-packages file and the timestamp at which the client last
  successfully downloaded and installed a software update.

  Automatic update clients SHOULD check for an updated recommended-packages
  document at most once per day but at least once every 30 days.


  3. Future Extensions

  There are several possible areas for future extensions of this framework.
  The extensions below are merely suggestions and should be the subject of
  their own proposal before being implemented.

  3.1. Additional Software Updates

  There are several software packages often included in Tor bundles besides
  Tor, such as Vidalia, Privoxy or Polipo, and Torbutton. The versions and
  download locations of updated installation packages for these bundle
  components can be easily added to the recommended-packages document
  specification above.

  3.2. Including ChangeLog Information

  It may be useful for automatic update clients to be able to display for
  users a summary of the changes made in the latest Tor or Tor-related
  software release, before the user chooses to install the update. In the
  future, we can add keywords to the specification in Section 1.2 that specify
  the location of a ChangeLog file for the latest recommended package
  versions. It may also be desirable to allow localized ChangeLog information,
  so that the automatic update client can fetch release notes in the
  end-user's preferred language.

  3.3. Weighted Package Mirror Selection

  We defined in Section 1.2 a method by which automatic update clients can
  select from multiple available package mirrors. We may want to add a Weight
  argument to the "*-package" Items that allows the recommended-packages file
  to suggest to clients the probability with which a package mirror should be
  chosen. This will allow clients to more appropriately distribute package
  downloads across available mirrors proportional to their approximate
  bandwidth.


Implementation

  Implementation of this proposal will consist of two separate components.

  The first component is a small "au-publish" tool that takes as input a
  configuration file specifying the information described in Section 1.2 and a
  private key. The tool is run by a "packaging authority" (someone responsible
  for publishing updated installation packages), who will be prompted to enter
  the passphrase for the private key used to sign the recommended-packages
  document. The output of the tool is a document formatted according to
  Section 1.2, with a signature appended at the end. The resulting document
  can then be published to any of the update mirrors.

  The second component is an "au-client" tool that is run on the end-user's
  machine. It periodically checks for updated installation packages according
  to Section 2 and fetches the packages if necessary. The public keys used
  to sign the recommended-packages file and any of the published packages are
  included in the "au-client" tool.


References

  [1] Tor directory protocol (version 3),
  https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/dir-spec.txt

  [2] Tor control protocol (version 2),
  https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/control-spec.txt

  [3] Tor version specification,
  https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/version-spec.txt