More pass overhaul based on making contextfinding explicit within documentation.

1 files changed, 213 insertions, 183 deletions

diff --git a/docs/narr/traversal.rst b/docs/narr/traversal.rst
index 95df3d6b9..bb773c195 100644
--- a/docs/narr/traversal.rst
+++ b/docs/narr/traversal.rst
@@ -3,47 +3,136 @@
 Traversal
 =========
 
-:term:`traversal` is a :term:`context finding` mechanism that is used
-by :mod:`repoze.bfg`. :term:`traversal` is the act of finding a
-:term:`context` and a :term:`view name` by walking over an *object
-graph*, starting from a :term:`root` object, using a :term:`request`
-object as a source of path information.
+:term:`Traversal` is a :term:`context finding` mechanism. It is the
+act of finding a :term:`context` and a :term:`view name` by walking
+over an *object graph*, starting from a :term:`root` object, using a
+:term:`request` object as a source of path information.
 
 In this chapter, we'll provide a high-level overview of traversal,
 we'll explain the concept of an *object graph*, and we'll show how
 traversal might be used within an application.
 
 .. index::
+   single: traversal analogy
+
+A Traversal Analogy
+-------------------
+
+We use an analogy to provide an introduction to :term:`traversal`.
+Imagine an inexperienced UNIX computer user, wishing only to use the
+command line to find a file and to invoke the ``cat`` command against
+that file.  Because he is inexperienced, the only commands he knows
+how to use are ``cd``, which changes the current directory and
+``cat``, which prints the contents of a file.  And because he is
+inexperienced, he doesn't understand that ``cat`` can take an absolute
+path specification as an argument, so he doesn't know that you can
+issue a single command command ``cat /an/absolute/path`` to get the
+desired result.  Instead, this user believes he must issue the ``cd``
+command, starting from the root, for each intermediate path segment,
+*even the path segment that represents the file itself*.  Once he gets
+an error (because you cannot successfully ``cd`` into a file), he
+knows he has reached the file he wants, and he will be able to execute
+``cat`` against the resulting path segment.
+
+This inexperienced user's attempt to execute ``cat`` against the file
+named ``/fiz/buz/myfile`` might be to issue the following set of UNIX
+commands:
+
+.. code-block::  text
+
+   cd /
+   cd fiz
+   cd buz
+   cd myfile
+
+The user now know he has found a *file*, because the ``cd`` command
+issues an error when he executed ``cd myfile``.  Now he knows that he
+can run the ``cat`` command:
+
+.. code-block::  text
+
+   cat myfile
+
+The contents of ``myfile`` are now printed on the user's behalf.
+
+:mod:`repoze.bfg` is very much like this inexperienced UNIX user as it
+uses :term:`traversal` against an object graph.  In this analogy, we
+can map the ``cat`` program to the :mod:`repoze.bfg` concept of a
+:term:`view callable`: it is a program that can be run against some
+:term:`context` as the result of :term:`view lookup`.  The file being
+operated on in this analogy is the :term:`context` object; the context
+is the "last node found" in a traversal.  The directory structure is
+the object graph being traversed.  The act of progressively changing
+directories to find the file as well as the handling of a ``cd`` error
+as a stop condition is analogous to :term:`traversal`.
+
+The analogy we've used is not *exactly* correct, because, while the
+naive user already knows which command he wants to invoke before he
+starts "traversing" (``cat``), :mod:`repoze.bfg` needs to obtain that
+information from the path being traversed itself.  In
+:term:`traversal`, the "command" meant to be invoked is a :term:`view
+callable`.  A view callable is derived via :term:`view lookup` from
+the combination of the :term:`view name` and the :term:`context`.
+Traversal is the act of obtaining these two items.
+
+.. index::
    pair: traversal; high-level overview
 
 A High-Level Overview of Traversal
 ----------------------------------
 
 :term:`Traversal` is dependent on information in a :term:`request`
-object.  The :term:`request` object contains URL path information in
+object.  Every :term:`request` object contains URL path information in
 the ``PATH_INFO`` portion of the :term:`WSGI` environment.  The
-``PATH_INFO`` portion of the WSGI environment is the URL data in a
-request following the hostname and port number, but before any query
-string elements or fragments, for example the ``/a/b/c`` portion of
-the URL ``http://example.com/a/b/c?foo=1``.
+``PATH_INFO`` portion of the WSGI environment is the portion of a
+request's URL following the hostname and port number, but before any
+query string elements or fragment element.  For example the
+``PATH_INFO`` portion of the the URL
+``http://example.com:8080/a/b/c?foo=1`` is ``/a/b/c``.
 
 Traversal treats the ``PATH_INFO`` segment of a URL as a sequence of
 path segments.  For example, the ``PATH_INFO`` string ``/a/b/c`` is
-treated as the sequence ``['a', 'b', 'c']``.  Traversal pops the first
-element (``a``) from the path segment sequence and attempts to use it
-as a lookup key into an object graph supplied by an application.  If
-that succeeeds, the :term:`context` temporarily becomes the object
-found via that lookup.  Then the next segment (``b``) is popped from
-the sequence, and the object graph is queried for that segment; if
-that lookup succeeds, the :term:`context` becomes that object.  This
-process continues until the path segment sequence is exhausted or any
-lookup for a name in the sequence fails.  In either case, a
-:term:`context` is found.
+coverted to the sequence ``['a', 'b', 'c']``.
+
+After the path info is converted, a lookup is performed against the
+object graph for each path segment.  Each lookup uses the
+``__getitem__`` method of an object in the graph.
+
+For example, if the path info sequence is ``['a', 'b', 'c']``:
+
+- :term:`Traversal` pops the first element (``a``) from the path
+  segment sequence and attempts to call the root object's
+  ``__getitem__`` method using that value (``a``) as an argument;
+  we'll presume it succeeds.
+
+- When the root object's ``__getitem__`` succeeeds it will return an
+  object, which we'll call "A".  The :term:`context` temporarily
+  becomes the "A" object.
+
+- The next segment (``b``) is popped from the path sequence, and the
+  "A" object's ``__getitem__`` is called with that value (``b``) as an
+  argument; we'll presume it succeeds.
+
+- When the "A" object's ``__getitem__`` succeeeds it will return an
+  object, which we'll call "B".  The :term:`context` temporarily
+  becomes the "B" object.
+
+This process continues until the path segment sequence is exhausted or
+a lookup for a path element fails.  In either case, a :term:`context`
+is found.
+
+Traversal "stops" when it either reaches a leaf level model instance
+in your object graph or when the path segments implied by the URL "run
+out".  The object that traversal "stops on" becomes the
+:term:`context`.  If at any point during traversal any node in the
+graph doesn't have a ``__getitem__`` method, or if the ``__getitem__``
+method of a node raises a :exc:`KeyError`, traversal ends immediately,
+and that node becomes the :term:`context`.
 
 The results of a :term:`traversal` also include a :term:`view name`.
 The :term:`view name` is the *first* URL path segment in the set of
 ``PATH_INFO`` segments "left over" in the path segment list popped by
-the traversal process.
+the traversal process *after* traversal finds a context object.
 
 The combination of the :term:`context` object and the :term:`view
 name` found via traversal is used later in the same request by a
@@ -62,11 +151,13 @@ The Object Graph
 
 When your application uses :term:`traversal` to resolve URLs to code,
 your application must supply an *object graph* to :mod:`repoze.bfg`.
+This graph is represented by a :term:`root` object.
 
-At system startup time, the :mod:`repoze.bfg` :term:`Router` is
-configured with a callback known as a :term:`root factory`, supplied
-by the application developer as the ``root_factory`` argument to a
-:term:`Configurator`.
+In order to supply a root object for an application, at system startup
+time, the :mod:`repoze.bfg` :term:`Router` is configured with a
+callback known as a :term:`root factory`.  The root factory is
+supplied by the application developer as the ``root_factory`` argument
+to the application's :term:`Configurator`.
 
 Here's an example of a simple root factory:
 
@@ -86,18 +177,19 @@ named ``config``:
 
    config = Configurator(root_factory=Root)
 
-Making a declaration like this at startup means that your
-:mod:`repoze.bfg` application will call the root factory (in this
-case, the class ``Root``) to generate a root object whenever a request
-enters the application.  Usually a root factory for a traversal-based
-application will be more complicated than the above ``Root`` object;
-in particular it may be associated with a database connection or
-another persistence mechanism.
+Using the ``root_factory`` argument to a
+:class:`repoze.bfg.configuration.Configurator` constructor tells your
+:mod:`repoze.bfg` application to call this root factory to generate a
+root object whenever a request enters the application.
 
 A root factory is passed a :term:`request` object and it is expected
 to return an object which represents the root of the object graph.
-All :term:`traversal` will begin at this root object.  The root object
-is often an instance of a class which has a ``__getitem__`` method.
+All :term:`traversal` will begin at this root object.  Usually a root
+factory for a traversal-based application will be more complicated
+than the above ``Root`` object; in particular it may be associated
+with a database connection or another persistence mechanism.  A root
+object is often an instance of a class which has a ``__getitem__``
+method.
 
 .. warning:: In :mod:`repoze.bfg` 1.0 and prior versions, the root
    factory was passed a term WSGI *environment* object (a dictionary)
@@ -107,9 +199,10 @@ is often an instance of a class which has a ``__getitem__`` method.
    emulates the WSGI environment, so code expecting the argument to be
    a dictionary will continue to work.
 
-If a :term:`root factory` is passed to the :mod:`repoze.bfg`
-:term:`Configurator` constructor as the value ``None``, a *default*
-root factory is used.
+If no :term:`root factory` is passed to the :mod:`repoze.bfg`
+:term:`Configurator` constructor, or the ``root_factory`` is specified
+as the value ``None``, a *default* root factory is used.  The default
+root factory always returns an object that has no child nodes.
 
 .. sidebar:: Emulating the Default Root Factory
 
@@ -138,96 +231,6 @@ concept of :term:`model` objects used by many other frameworks (and
 :mod:`repoze.bfg` APIs often refers to them as "models", as well).
 They are typically instances of Python classes.
 
-.. index::
-   single: traversal; algorithm
-
-.. _traversal_behavior:
-
-The :mod:`repoze.bfg` Traversal Algorithm
------------------------------------------
-
-This section will attempt to explain the :mod:`repoze.bfg` traversal
-algorithm.  We'll provide an an analogy, a diagram of how the
-traversal algorithm works, and some example traversal scenarios that
-might aid in understanding how the traversal algorithm operates
-against a specific object graph.
-
-The :ref:`views_chapter` chapter discusses :term:`view lookup` in
-detail, and it is the canonical source for information about views.
-Technically, :term:`traversal` is a :mod:`repoze.bfg` subsystem that
-is separated from traversal entirely.  However, we'll describe the
-fundamental behavior of view lookup in the examples in the next few
-sections to give you an idea of how traversal and view lookup
-cooperate, because they are always used cooperatively.
-
-.. index::
-   single: traversal analogy
-
-An Analogy
-~~~~~~~~~~
-
-We need to use an analogy to clarify how :mod:`repoze.bfg` traversal
-works against an arbitrary object graph.
-
-Let's imagine an inexperienced UNIX computer user, wishing only to use
-the command line to find a file and to invoke the ``cat`` command
-against that file.  Because he is inexperienced, the only commands he
-knows how to use are ``cd``, which changes the current directory and
-``cat``, which prints the contents of a file.  And because he is
-inexperienced, he doesn't understand that ``cat`` can take an absolute
-path specification as an argument, so he doesn't know that you can
-issue a single command command ``cat /an/absolute/path`` to get the
-desired result.  Instead, this user believes he must issue the ``cd``
-command, starting from the root, for each intermediate path segment,
-*even the path segment that represents the file itself*.  Once he gets
-an error (because you cannot successfully ``cd`` into a file), he knows
-he has reached the file he wants, and he will be able to execute
-``cat`` against the resulting path segment.
-
-This inexperienced user's attempt to execute ``cat`` against the file
-named ``/fiz/buz/myfile`` might be to issue the following set of UNIX
-commands:
-
-.. code-block::  text
-
-   cd /
-   cd fiz
-   cd buz
-   cd myfile
-
-The user now know he has found a *file*, because the ``cd`` command
-issues an error when he executed ``cd myfile``.  Now he knows that he
-can run the ``cat`` command:
-
-.. code-block::  text
-
-   cat myfile
-
-The contents of ``myfile`` are now printed on the user's behalf.
-
-:mod:`repoze.bfg` is very much like this inexperienced UNIX user as it
-uses :term:`traversal` against an object graph.  In this analogy, we
-can map the ``cat`` program to the :mod:`repoze.bfg` concept of a
-:term:`view callable`: it is a program that can be run against some
-:term:`context` as the result of :term:`view lookup`.  The file being
-operated on in this analogy is the :term:`context` object; the context
-is the "last node found" in a traversal.  The directory structure is
-the object graph being traversed.  The act of progressively changing
-directories to find the file as well as the handling of a ``cd`` error
-as a stop condition is analogous to :term:`traversal`.
-
-The object graph is traversed, beginning at a root object, represented
-by the root URL (``/``); if there are further path segments in the
-path info of the request being processed, the root object's
-``__getitem__`` is called with the next path segment, and it is
-expected to return another graph object.  The resulting object's
-``__getitem__`` is called with the very next path segment, and it is
-expected to return another graph object.  This happens *ad infinitum*
-until all path segments are exhausted.  If at any point during
-traversal any node in the graph doesn't *have* a ``__getitem__``
-method, or if the ``__getitem__`` of a node raises a :exc:`KeyError`,
-traversal ends immediately, and the node becomes the :term:`context`.
-
 The object graph consists of *container* nodes and *leaf* nodes.
 There is only one difference between a *container* node and a *leaf*
 node: *container* nodes possess a ``__getitem__`` method while *leaf*
@@ -249,23 +252,47 @@ Usually, the traversal root is a *container* node, and as such it
 contains other nodes.  However, it doesn't *need* to be a container.
 Your object graph can be as shallow or as deep as you require.
 
-Traversal "stops" when :mod:`repoze.bfg` either reaches a leaf level
-model instance in your object graph or when the path segments implied
-by the URL "run out".  The object that traversal "stops on" becomes
-the :term:`context`.
+In general, the object graph is traversed beginning at its root object
+using a sequence of path elements described by the ``PATH_INFO`` of
+the current request; if there are path segments, the root object's
+``__getitem__`` is called with the next path segment, and it is
+expected to return another graph object.  The resulting object's
+``__getitem__`` is called with the very next path segment, and it is
+expected to return another graph object.  This happens *ad infinitum*
+until all path segments are exhausted.
+
+.. index::
+   single: traversal algorithm
+
+.. _traversal_algorithm:
+
+The Traversal Algorithm
+-----------------------
+
+This section will attempt to explain the :mod:`repoze.bfg` traversal
+algorithm.  We'll provide a description of the algorithm, a diagram of
+how the algorithm works, and some example traversal scenarios that
+might help you understand how the algorithm operates against a
+specific object graph.
+
+We'll also talk a bit about :term:`view lookup` .  The
+:ref:`views_chapter` chapter discusses :term:`view lookup` in detail,
+and it is the canonical source for information about views.
+Technically, :term:`traversal` is a :mod:`repoze.bfg` subsystem that
+is separated from traversal entirely.  However, we'll describe the
+fundamental behavior of view lookup in the examples in the next few
+sections to give you an idea of how traversal and view lookup
+cooperate, because they are almost always used together.
 
 .. index::
-   pair: traversal; unicode
    pair: traversal; algorithm
 
-.. _how_bfg_traverses:
-
-The Algorithm
-~~~~~~~~~~~~~
+A Descrption of The Traversal Algorithm
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 When a user requests a page from your :mod:`traversal` -powered
-application, the system uses this algorithm to determine which Python
-code to execute:
+application, the system uses this algorithm to find a :term:`context`
+and a :term:`view name`.
 
 #.  The request for the page is presented to the :mod:`repoze.bfg`
     :term:`router` in terms of a standard :term:`WSGI` request, which
@@ -281,12 +308,22 @@ code to execute:
 #.  The router uses the WSGI environment's ``PATH_INFO`` information
     to determine the path segments to traverse.  The leading slash is
     stripped off ``PATH_INFO``, and the remaining path segments are
-    split on the slash character to form a traversal sequence, so a
-    request with a ``PATH_INFO`` variable of ``/a/b/c`` maps to the
-    traversal sequence ``[u'a', u'b', u'c']``.  Note that each of the
-    path segments in the sequence is converted to Unicode using the
-    UTF-8 decoding; if the decoding fails, a :exc:`TypeError` is
-    raised.
+    split on the slash character to form a traversal sequence.
+
+    The traversal algorithm by default attempts to first URL-unquote
+    and then Unicode-decode each path segment derived from
+    ``PATH_INFO`` from its natural byte string (``str`` type)
+    representation.  URL unquoting is performed using the Python
+    standard library ``urllib.unquote`` function.  Conversion from a
+    URL-decoded string into Unicode is attempted using the UTF-8
+    encoding.  If any URL-unquoted path segment in ``PATH_INFO`` is
+    not decodeable using the UTF-8 decoding, a :exc:`TypeError` is
+    raised.  A segment will be fully URL-unquoted and UTF8-decoded
+    before it is passed it to the ``__getitem__`` of any model object
+    during traversal.
+
+    Thus, a request with a ``PATH_INFO`` variable of ``/a/b/c`` maps
+    to the traversal sequence ``[u'a', u'b', u'c']``.
 
 #.  :term:`Traversal` begins at the root object returned by the root
     factory.  For the traversal sequence ``[u'a', u'b', u'c']``, the
@@ -311,21 +348,13 @@ code to execute:
     :term:`context`.  If the path has been exhausted when traversal
     ends, the :term:`view name` is deemed to be the empty string
     (``''``).  However, if the path was *not* exhausted before
-    traversal terminated, the first remaining path element is treated
+    traversal terminated, the first remaining path segment is treated
     as the view name.
 
-    Any subsequent path elements after the view name are deemed the
-    :term:`subpath`.  The subpath is always a sequence of path
-    segments that come from ``PATH_INFO`` that are "left over" after
-    traversal has completed. For instance, if ``PATH_INFO`` was
-    ``/a/b`` and the root returned an "object ``a``", and "object
-    ``a``" subsequently returned an "object ``b``", the router deems
-    that the context is "object ``b``", the view name is the empty
-    string, and the subpath is the empty sequence.  On the other hand,
-    if ``PATH_INFO`` was ``/a/b/c`` and "object ``a``" was found but
-    raised a ``KeyError`` for the name ``b``, the router deems that
-    the context is "object ``a``", the view name is ``b`` and the
-    subpath is ``('c',)``.
+#.  Any subsequent path elements after the :term:`view name` is found
+    are deemed the :term:`subpath`.  The subpath is always a sequence
+    of path segments that come from ``PATH_INFO`` that are "left over"
+    after traversal has completed.
 
 Once :term:`context` and :term:`view name` and associated attributes
 such as the :term:`subpath` are located, the job of :term:`traversal`
@@ -334,18 +363,7 @@ caller, the :mod:`repoze.bfg` :term:`Router`, which subsequently
 invokes :term:`view lookup` with the context and view name
 information.
 
-Note well that the traversal machinery by default attempts to first
-URL-unquote and then Unicode-decode each path element in ``PATH_INFO``
-from its natural byte string (``str`` type) representation.  URL
-unquoting is performed using the Python standard library
-``urllib.unquote`` function.  Conversion from a URL-decoded string
-into Unicode is attempted using the UTF-8 encoding.  If any
-URL-unquoted path segment in ``PATH_INFO`` is not decodeable using the
-UTF-8 decoding, a :exc:`TypeError` is raised.  A segment will be fully
-URL-unquoted and UTF8-decoded before it is passed it to the
-``__getitem__`` of any model object during traversal.
-
-The standard traversal algorithm exposes two special cases:
+The traversal algorithm exposes two special cases:
 
 - You will often end up with a :term:`view name` that is the empty
   string as the result of a particular traversal.  This indicates that
@@ -360,23 +378,28 @@ The standard traversal algorithm exposes two special cases:
   views that may have the same names as model instance names in the
   graph unambiguously.
 
+Finally, traversal is responsible for locating a :term:`virtual root`.
+A virtual root is used during "virtual hosting"; see the
+:ref:`vhosting_chapter` chapter for information.  We won't speak more
+about it in this chapter.
+
 .. image:: modelgraphtraverser.png
 
 .. index::
-   pair: traversal; example
+   pair: traversal; examples
 
-Traversal Examples
-~~~~~~~~~~~~~~~~~~
+Traversal Algorithm Examples
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 No one can be expected to understand the traversal algorithm by
 analogy and description alone, so let's examine some traversal
 scenarios that use concrete URLs and object graph compositions.
 
 Let's pretend the user asks for
-``http://example.com/foo/bar/baz/biz/buz.txt``. Let's pretend that the
-request's ``PATH_INFO`` in that case is ``/foo/bar/baz/biz/buz.txt``.
-Let's further pretend that when this request comes in that we're
-traversing the following graph::
+``http://example.com/foo/bar/baz/biz/buz.txt``. The request's
+``PATH_INFO`` in that case is ``/foo/bar/baz/biz/buz.txt``.  Let's
+further pretend that when this request comes in that we're traversing
+the following object graph::
 
   /--
      |
@@ -469,12 +492,11 @@ view lookup asks the :term:`application registry` this question:
   configuration` with the name ``buz.txt`` that can be used for class
   ``Biz``.
 
-Let's say that question is answered by the application registry with
-the equivalent of "here you go, here's a bit of code that is willing
-to deal with that case"; the application registry returns a
-:term:`view callable`.  The view callable is then called with the
-current :term:`WebOb` :term:`request` as the sole argument:
-``request``; it is expected to return a response.
+Let's say that question is answered by the application registry; in
+such a situation, the application registry returns a :term:`view
+callable`.  The view callable is then called with the current
+:term:`WebOb` :term:`request` as the sole argument: ``request``; it is
+expected to return a response.
 
 .. sidebar:: The Example View Callables Accept Only a Request; How Do I Access the Context?
 
@@ -495,8 +517,16 @@ current :term:`WebOb` :term:`request` as the sole argument:
 References
 ----------
 
-A tutorial showing how :term:`traversal` can be used to create a
+A tutorial showing how :term:`traversal` can be used within a
 :mod:`repoze.bfg` application exists in :ref:`bfg_wiki_tutorial`.
 
 See the :ref:`views_chapter` chapter for detailed information about
 :term:`view lookup`.
+
+The :mod:`repoze.bfg.traversal` module contains API functions that
+deal with traversal, such as traversal invocation from within
+application code.
+
+The :func:`repoze.bfg.url.model_url` function generates a URL when
+given an object retrived from an object graph.
+