Features

--------

- A ``request.matched_route`` attribute is now added to the request
  when a route has matched.  Its value is the "route" object that
  matched (see the ``IRoute`` interface within
  ``repoze.bfg.interfaces`` API documentation for the API of a route
  object).

- The ``exception`` attribute of the request is now set slightly
  earlier and in a slightly different set of scenarios, for benefit of
  "finished callbacks" and "response callbacks".  In previous
  versions, the ``exception`` attribute of the request was not set at
  all if an exception view was not found.  In this version, the
  ``request.exception`` attribute is set immediately when an exception
  is caught by the router, even if an exception view could not be
  found.

Backwards Incompatibilities
---------------------------

- The router no longer sets the value ``wsgiorg.routing_args`` into
  the environ when a route matches. The value used to be something
  like ``((), matchdict)``.  This functionality was only ever
  obliquely referred to in change logs; it was never documented as an
  API.

- The ``exception`` attribute of the request now defaults to ``None``.
  In prior versions, the ``request.exception`` attribute did not exist
  if an exception was not raised by user code during request
  processing; it only began existence once an exception view was
  found.

Deprecations
------------

- References to the WSGI environment values ``bfg.routes.matchdict``
  and ``bfg.routes.route`` were removed from documentation.  These
  will stick around internally for several more releases, but it is
  ``request.matchdict`` and ``request.matched_route`` are now the
  "official" way to obtain the matchdict and the route object which
  resulted in the match.

Documentation
-------------

- Added two sections to the "Hooks" chapter of the documentation:
  "Using Response Callbacks" and "Using Finished Callbacks".

- Added documentation of the ``request.exception`` attribute to the
  ``repoze.bfg.request.Request`` API documentation.

- Added glossary entries for "response callback" and "finished
  callback".

- The "Request Processing" narrative chapter has been updated to note
  finished and response callback steps.

9 files changed, 297 insertions, 51 deletions

diff --git a/CHANGES.txt b/CHANGES.txt
index 848fccfc3..748080683 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -26,7 +26,39 @@ Features
 
 - Add a new request API: ``request.add_finished_callback``.  Finished
   callbacks are called by the router unconditionally near the very end
-  of request processing.
+  of request processing.  See the "Using Finished Callbacks" section
+  of the "Hooks" narrative chapter of the documentation for more
+  information.
+
+- A ``request.matched_route`` attribute is now added to the request
+  when a route has matched.  Its value is the "route" object that
+  matched (see the ``IRoute`` interface within
+  ``repoze.bfg.interfaces`` API documentation for the API of a route
+  object).
+
+- The ``exception`` attribute of the request is now set slightly
+  earlier and in a slightly different set of scenarios, for benefit of
+  "finished callbacks" and "response callbacks".  In previous
+  versions, the ``exception`` attribute of the request was not set at
+  all if an exception view was not found.  In this version, the
+  ``request.exception`` attribute is set immediately when an exception
+  is caught by the router, even if an exception view could not be
+  found.
+
+Backwards Incompatibilities
+---------------------------
+
+- The router no longer sets the value ``wsgiorg.routing_args`` into
+  the environ when a route matches. The value used to be something
+  like ``((), matchdict)``.  This functionality was only ever
+  obliquely referred to in change logs; it was never documented as an
+  API.
+
+- The ``exception`` attribute of the request now defaults to ``None``.
+  In prior versions, the ``request.exception`` attribute did not exist
+  if an exception was not raised by user code during request
+  processing; it only began existence once an exception view was
+  found.
 
 Deprecations
 ------------
@@ -44,6 +76,13 @@ Deprecations
   ``repoze.bfg.events.ContextFound``.  The older aliases will continue
   to work indefinitely.
 
+- References to the WSGI environment values ``bfg.routes.matchdict``
+  and ``bfg.routes.route`` were removed from documentation.  These
+  will stick around internally for several more releases, but it is
+  ``request.matchdict`` and ``request.matched_route`` are now the
+  "official" way to obtain the matchdict and the route object which
+  resulted in the match.
+
 Documentation
 -------------
 
@@ -59,6 +98,18 @@ Documentation
 - Document ``renderer_globals_factory`` and ``request_factory``
   arguments to Configurator constructor.
 
+- Added two sections to the "Hooks" chapter of the documentation:
+  "Using Response Callbacks" and "Using Finished Callbacks".
+
+- Added documentation of the ``request.exception`` attribute to the
+  ``repoze.bfg.request.Request`` API documentation.
+
+- Added glossary entries for "response callback" and "finished
+  callback".
+
+- The "Request Processing" narrative chapter has been updated to note
+  finished and response callback steps.
+
 1.3a12 (2010-09-08)
 ===================
 
diff --git a/docs/api/request.rst b/docs/api/request.rst
index 86202b830..08d8d76e1 100644
--- a/docs/api/request.rst
+++ b/docs/api/request.rst
@@ -73,6 +73,21 @@
       :ref:`vhosting_chapter`  for  more  information  about  virtual
       roots.
 
+   .. attribute:: exception
+
+     If an exception was raised by a :term:`root factory` or a
+     :term:`view callable`, or at various other points where
+     :mod:`repoze.bfg` executes user-defined code during the
+     processing of a request, the exception object which was caught
+     will be available as the ``exception`` attribute of the request
+     within a :term:`exception view`, a :term:`response callback` or a
+     :term:`finished callback`.  If no exception occurred, the value
+     of ``request.exception`` will be ``None`` within response and
+     finished callbacks.
+
+     .. note:: The exception attribute is new in :mod:`repoze.bfg`
+        1.3.
+
 .. autofunction:: make_request_ascii
 
 
diff --git a/docs/glossary.rst b/docs/glossary.rst
index a703f187c..158af2230 100644
--- a/docs/glossary.rst
+++ b/docs/glossary.rst
@@ -774,3 +774,13 @@ Glossary
       Values injected as names into a renderer based on application
       policy.  See :ref:`adding_renderer_globals` for more
       information.
+
+   response callback
+      A user-defined callback executed by the :term:`router` at a
+      point after a :term:`response` object is successfully created.
+      See :ref:`using_response_callbacks`.
+
+   finished callback
+      A user-defined callback executed by the :term:`router`
+      unconditionally at the very end of request processing .  See
+      :ref:`using_finished_callbacks`.
diff --git a/docs/narr/hooks.rst b/docs/narr/hooks.rst
index b00fc6727..965a4c77f 100644
--- a/docs/narr/hooks.rst
+++ b/docs/narr/hooks.rst
@@ -461,6 +461,111 @@ method:
    config = Configurator()
    config.set_renderer_globals_factory(renderer_globals_factory)
 
+.. _using_response_callbacks:
+
+Using Response Callbacks
+------------------------
+
+Unlike many other web frameworks, :mod:`repoze.bfg` does not eagerly
+create a global response object.  Adding a :term:`response callback`
+allows an application to register an action to be performed against a
+response object once it is created, usually in order to mutate it.
+
+The :meth:`repoze.bfg.request.Request.add_response_callback` method is
+used to register a response callback.  
+
+A response callback is a callable which accepts two positional
+parameters: ``request`` and ``response``.  For example:
+
+.. code-block:: python
+   :linenos:
+
+   def cache_callback(request, response):
+       """Set the cache_control max_age for the response"""
+       if request.exception is not None:
+           response.cache_control.max_age = 360
+   request.add_response_callback(cache_callback)
+
+No response callback is called if an unhandled exception happens in
+application code, or if the response object returned by a :term:`view
+callable` is invalid.  Response callbacks *are*, however, invoked when
+a :term:`exception view` is rendered successfully: in such a case, the
+:attr:`request.exception` attribute of the request when it enters a
+response callback will be an exception object instead of its default
+value of ``None``.
+
+Response callbacks are called in the order they're added
+(first-to-most-recently-added).  All response callbacks are called
+*before* the :class:`repoze.bfg.interfaces.INewResponse` event is
+sent.  Errors raised by response callbacks are not handled specially.
+They will be propagated to the caller of the :mod:`repoze.bfg` router
+application.
+
+A response callback has a lifetime of a *single* request.  If you want
+a response callback to happen as the result of *every* request, you
+must re-register the callback into every new request (perhaps within a
+subscriber of a :class:`repoze.bfg.interfaces.INewRequest` event).
+
+.. _using_finished_callbacks:
+
+Using Finished Callbacks
+------------------------
+
+A :term:`finished callback` is a function that will be called
+unconditionally by the :mod:`repoze.bfg` :term:`router` at the very
+end of request processing.  A finished callback can be used to perform
+an action at the end of a request unconditionally.
+
+The :meth:`repoze.bfg.request.Request.add_finished_callback` method is
+used to register a finished callback.
+
+A finished callback is a callable which accepts a single positional
+parameter: ``request``.  For example:
+
+.. code-block:: python
+   :linenos:
+
+   import transaction
+
+   def commit_callback(request):
+       '''commit or abort the transaction associated with request'''
+       if request.exception is not None:
+           transaction.abort()
+       else:
+           transaction.commit()
+   request.add_finished_callback(commit_callback)
+
+Finished callbacks are called in the order they're added ( first- to
+most-recently- added).  Finished callbacks (unlike a :term:`response
+callback`) are *always* called, even if an exception happens in
+application code that prevents a response from being generated.
+
+The set of finished callbacks associated with a request are called
+*very late* in the processing of that request; they are essentially
+the very last thing called by the :term:`router` before a request
+"ends". They are called after response processing has already occurred
+in a top-level ``finally:`` block within the router request processing
+code.  As a result, mutations performed to the ``request`` provided to
+a finished callback will have no meaningful effect, because response
+processing will have already occurred, and the request's scope will
+expire almost immediately after all finished callbacks have been
+processed.
+
+It is often necessary to tell whether an exception occurred within
+:term:`view callable` code from within a finished callback: in such a
+case, the :attr:`request.exception` attribute of the request when it
+enters a response callback will be an exception object instead of its
+default value of ``None``.
+
+Errors raised by finished callbacks are not handled specially.  They
+will be propagated to the caller of the :mod:`repoze.bfg` router
+application.
+
+A finished callback has a lifetime of a *single* request.  If you want
+a finished callback to happen as the result of *every* request, you
+must re-register the callback into every new request (perhaps within a
+subscriber of a :class:`repoze.bfg.interfaces.INewRequest` event).
+
 .. _registering_configuration_decorators:
 
 Registering Configuration Decorators
diff --git a/docs/narr/router.rst b/docs/narr/router.rst
index af8f057b0..8bb76b366 100644
--- a/docs/narr/router.rst
+++ b/docs/narr/router.rst
@@ -26,30 +26,37 @@ processing?
 
 #. A :term:`request` object is created based on the WSGI environment.
 
+#. The :term:`registry` associated with the application and the
+   :term:`request` object created in the last step are pushed on to the
+   :term:`thread local` stack that :mod:`repoze.bfg` uses to allow the
+   functions named :func:`repoze.bfg.threadlocal.get_current_request`
+   and :func:`repoze.bfg.threadlocal.get_current_registry` to work.
+
 #. A :class:`repoze.bfg.interfaces.INewRequest` :term:`event` is sent
    to any subscribers.
 
 #. If any :term:`route` has been defined within application
    configuration, the :mod:`repoze.bfg` :term:`router` calls a
    :term:`URL dispatch` "route mapper."  The job of the mapper is to
-   examine the ``PATH_INFO`` implied by the request to determine
-   whether any user-defined :term:`route` pattern matches the current
-   WSGI environment.  The :term:`router` passes the request as an
-   argument to the mapper.
-
-#. If any route matches, the WSGI environment is mutated; a
-   ``bfg.routes.route`` key and a ``bfg.routes.matchdict`` are added
-   to the WSGI environment, and an attribute named ``matchdict`` is
-   added to the request.  A root object associated with the route
-   found is also generated.  If the :term:`route configuration`
-   which matched has an associated a ``factory`` argument, this
-   factory is used to generate the root object, otherwise a default
-   :term:`root factory` is used.
+   examine the request to determine whether any user-defined
+   :term:`route` matches the current WSGI environment.  The
+   :term:`router` passes the request as an argument to the mapper.
+
+#. If any route matches, the request is mutated; a ``matchdict`` and
+   ``matched_route`` attributes are added to the request object; the
+   former contains a dictionary representign the matched dynamic
+   elements of the request's ``PATH_INFO`` value, the latter contains
+   the :class:`repoze.bfg.interfaces.IRoute` object representing the
+   route which matched.  The root object associated with the route
+   found is also generated: if the :term:`route configuration` which
+   matched has an associated a ``factory`` argument, this factory is
+   used to generate the root object, otherwise a default :term:`root
+   factory` is used.
 
 #. If a route match was *not* found, and a ``root_factory`` argument
    was passed to the :term:`Configurator` constructor, that callable
    is used to generate the root object.  If the ``root_factory``
-   argument passed to the Configurator constructor is ``None``, a
+   argument passed to the Configurator constructor was ``None``, a
    default root factory is used to generate a root object.
 
 #. The :mod:`repoze.bfg` router calls a "traverser" function with the
@@ -75,11 +82,12 @@ processing?
    doesn't exist for this combination of objects (based on the type of
    the context, the type of the request, and the value of the view
    name, and any :term:`predicate` attributes applied to the view
-   configuration), :mod:`repoze.bfg` uses a "not found" view callable
-   to generate a response, and returns that response.
+   configuration), :mod:`repoze.bfg` raises a
+   :class:`repoze.bfg.exceptions.NotFound` exception, which is meant
+   to be caught by a surrounding exception handler.
 
-#. If a view callable was found, :mod:`repoze.bfg` calls the view
-   function.
+#. If a view callable was found, :mod:`repoze.bfg` attempts to call
+   the view function.
 
 #. If an :term:`authorization policy` is in use, and the view was
    protected by a :term:`permission`, :mod:`repoze.bfg` passes the
@@ -88,15 +96,38 @@ processing?
    requesting user, based on credential information in the request and
    security information attached to the context.  If it returns
    ``True``, :mod:`repoze.bfg` calls the view callable to obtain a
-   response.  If it returns ``False``, it uses a :term:`forbidden
-   view` callable to generate a response.
-
-#. A :class:`repoze.bfg.interfaces.INewResponse` :term:`event` is sent
-   to any subscribers.
-
-#. The response object's ``app_iter``, ``status``, and ``headerlist``
-   attributes are used to generate a WSGI response.  The response is
-   sent back to the upstream WSGI server.
+   response.  If it returns ``False``, it raises a
+   :class:`repoze.bfg.exceptions.Forbidden` exception, which is meant
+   to be called by a surrounding exception handler.
+
+#. If any exception was raised within a :term:`root factory`, by
+   :term:`traversal`, by a :term:`view callable` or by
+   :mod:`repoze.bfg` itself (such as when it raises
+   :class:`repoze.bfg.exceptions.NotFound` or
+   :class:`repoze.bfg.exceptions.Forbidden`), the router catches the
+   exception, and attaches it to the request as the ``exception``
+   attribute.  It then attempts to find a :term:`exception view` for
+   the exception that was caught.  If it finds an exception view
+   callable, that callable is called, and is presumed to generate a
+   response.  If an :term:`exception view` that matches the exception
+   cannot be found, the exception is reraised.
+
+#. The following steps occur only when a :term:`response` could be
+   successfully generated by a normal :term:`view callable` or an
+   :term:`exception view` callable.  :mod:`repoze.bfg` will attempt to
+   execute any :term:`response callback` functions attached via
+   :meth:`repoze.bfg.request.Request.add_response_callback`.  A
+   :class:`repoze.bfg.interfaces.INewResponse` :term:`event` is then
+   sent to any subscribers.  The response object's ``app_iter``,
+   ``status``, and ``headerlist`` attributes are then used to generate
+   a WSGI response.  The response is sent back to the upstream WSGI
+   server.
+
+#. :mod:`repoze.bfg` will attempt to execute any :term:`finished
+   callback` functions attached via
+   :meth:`repoze.bfg.request.Request.add_finished_callback`.
+
+#. The :term:`thread local` stack is popped.
 
 .. image:: router.png
 
diff --git a/repoze/bfg/request.py b/repoze/bfg/request.py
index 6e5417d9a..84a5660ef 100644
--- a/repoze/bfg/request.py
+++ b/repoze/bfg/request.py
@@ -40,6 +40,7 @@ class Request(WebobRequest):
     implements(IRequest)
     response_callbacks = ()
     finished_callbacks = ()
+    exception = None
     default_charset = 'utf-8'
 
     def add_response_callback(self, callback):
@@ -72,7 +73,13 @@ class Request(WebobRequest):
 
         Errors raised by callbacks are not handled specially.  They
         will be propagated to the caller of the :mod:`repoze.bfg`
-        router application.  """
+        router application.
+
+        .. note: ``add_response_callback`` is new in :mod:`repoze.bfg`
+           1.3.
+
+        See also: :ref:`using_response_callbacks`.
+        """
 
         callbacks = self.response_callbacks
         if not callbacks:
@@ -101,7 +108,7 @@ class Request(WebobRequest):
 
            def commit_callback(request):
                '''commit or abort the transaction associated with request'''
-               if hasattr(request, 'exception'):
+               if request.exception is not None:
                    transaction.abort()
                else:
                    transaction.commit()
@@ -126,7 +133,13 @@ class Request(WebobRequest):
 
         Errors raised by finished callbacks are not handled specially.
         They will be propagated to the caller of the :mod:`repoze.bfg`
-        router application.  """
+        router application.
+
+        .. note: ``add_finished_callback`` is new in :mod:`repoze.bfg`
+           1.3.
+
+        See also: :ref:`using_finished_callbacks`.
+        """
 
         callbacks = self.finished_callbacks
         if not callbacks:
diff --git a/repoze/bfg/router.py b/repoze/bfg/router.py
index 6532beec4..e6de4fdd7 100644
--- a/repoze/bfg/router.py
+++ b/repoze/bfg/router.py
@@ -57,9 +57,9 @@ class Router(object):
         has_listeners = registry.has_listeners
         logger = self.logger
         manager = self.threadlocal_manager
-        threadlocals = {'registry':registry, 'request':None}
-        manager.push(threadlocals)
         request = None
+        threadlocals = {'registry':registry, 'request':request}
+        manager.push(threadlocals)
 
         try:
             # create the request
@@ -72,16 +72,19 @@ class Router(object):
             request_iface = IRequest
 
             try:
-                # find the root
+                # find the root object
                 root_factory = self.root_factory
                 if self.routes_mapper is not None:
                     info = self.routes_mapper(request)
                     match, route = info['match'], info['route']
                     if route is not None:
-                        environ['wsgiorg.routing_args'] = ((), match)
-                        environ['bfg.routes.route'] = route
+                        # TODO: kill off bfg.routes.* environ keys
+                        # when traverser requires request arg, and
+                        # cant cope with environ anymore (likely 1.4+)
+                        environ['bfg.routes.route'] = route 
                         environ['bfg.routes.matchdict'] = match
                         attrs['matchdict'] = match
+                        attrs['matched_route'] = route
                         request_iface = registry.queryUtility(
                             IRouteRequest,
                             name=route.name,
@@ -91,7 +94,7 @@ class Router(object):
                 root = root_factory(request)
                 attrs['root'] = root
 
-                # find a view callable
+                # find a context
                 traverser = adapters.queryAdapter(root, ITraverser)
                 if traverser is None:
                     traverser = ModelGraphTraverser(root)
@@ -102,6 +105,8 @@ class Router(object):
                     tdict['virtual_root_path'])
                 attrs.update(tdict)
                 has_listeners and registry.notify(ContextFound(request))
+
+                # find a view callable
                 context_iface = providedBy(context)
                 view_callable = adapters.lookup(
                     (IViewClassifier, request_iface, context_iface),
@@ -121,27 +126,31 @@ class Router(object):
                         logger and logger.debug(msg)
                     else:
                         msg = request.path_info
+                    # repoze.bfg.message should die
                     environ['repoze.bfg.message'] = msg
                     raise NotFound(msg)
                 else:
                     response = view_callable(context, request)
 
-            # handle exceptions raised during root finding and view lookup
+            # handle exceptions raised during root finding and view execution
             except Exception, why:
+                attrs['exception'] = why
+
                 for_ = (IExceptionViewClassifier,
                         request_iface.combined,
                         providedBy(why))
                 view_callable = adapters.lookup(for_, IView, default=None)
+
                 if view_callable is None:
                     raise
 
-                try:
+                # r.b.message should be deprecated
+                try: 
                     msg = why[0]
-                except Exception:
+                except:
                     msg = ''
                 environ['repoze.bfg.message'] = msg
 
-                attrs['exception'] = why
                 response = view_callable(why, request)
 
             # process the response
@@ -164,6 +173,7 @@ class Router(object):
             return app_iter
 
         finally:
+            # post-response cleanup
             try:
                 if request is not None and request.finished_callbacks:
                     request._process_finished_callbacks()
diff --git a/repoze/bfg/tests/test_request.py b/repoze/bfg/tests/test_request.py
index d8a3c09fb..ae52f8420 100644
--- a/repoze/bfg/tests/test_request.py
+++ b/repoze/bfg/tests/test_request.py
@@ -23,6 +23,10 @@ class TestRequest(unittest.TestCase):
         r = self._makeOne({'PATH_INFO':'/'})
         self.assertEqual(r.charset, 'utf-8')
 
+    def test_exception_defaults_to_None(self):
+        r = self._makeOne({'PATH_INFO':'/'})
+        self.assertEqual(r.exception, None)
+
     def test_params_decoded_from_utf_8_by_default(self):
         environ = {
             'PATH_INFO':'/',
diff --git a/repoze/bfg/tests/test_router.py b/repoze/bfg/tests/test_router.py
index 199602a96..fade0679b 100644
--- a/repoze/bfg/tests/test_router.py
+++ b/repoze/bfg/tests/test_router.py
@@ -518,12 +518,11 @@ class TestRouter(unittest.TestCase):
         self.assertEqual(request.subpath, [])
         self.assertEqual(request.context, context)
         self.assertEqual(request.root, root)
-        routing_args = environ['wsgiorg.routing_args'][1]
-        self.assertEqual(routing_args['action'], 'action1')
-        self.assertEqual(routing_args['article'], 'article1')
-        self.assertEqual(environ['bfg.routes.matchdict'], routing_args)
+        matchdict = {'action':'action1', 'article':'article1'}
+        self.assertEqual(environ['bfg.routes.matchdict'], matchdict)
         self.assertEqual(environ['bfg.routes.route'].name, 'foo')
-        self.assertEqual(request.matchdict, routing_args)
+        self.assertEqual(request.matchdict, matchdict)
+        self.assertEqual(request.matched_route.name, 'foo')
 
     def test_call_route_matches_doesnt_overwrite_subscriber_iface(self):
         from repoze.bfg.interfaces import INewRequest
@@ -559,12 +558,11 @@ class TestRouter(unittest.TestCase):
         self.assertEqual(request.subpath, [])
         self.assertEqual(request.context, context)
         self.assertEqual(request.root, root)
-        routing_args = environ['wsgiorg.routing_args'][1]
-        self.assertEqual(routing_args['action'], 'action1')
-        self.assertEqual(routing_args['article'], 'article1')
-        self.assertEqual(environ['bfg.routes.matchdict'], routing_args)
+        matchdict = {'action':'action1', 'article':'article1'}
+        self.assertEqual(environ['bfg.routes.matchdict'], matchdict)
         self.assertEqual(environ['bfg.routes.route'].name, 'foo')
-        self.assertEqual(request.matchdict, routing_args)
+        self.assertEqual(request.matchdict, matchdict)
+        self.assertEqual(request.matched_route.name, 'foo')
         self.failUnless(IFoo.providedBy(request))
 
     def test_root_factory_raises_notfound(self):
@@ -634,6 +632,12 @@ class TestRouter(unittest.TestCase):
             pass
         from repoze.bfg.interfaces import IRequest
         from repoze.bfg.interfaces import IViewClassifier
+        from repoze.bfg.interfaces import IRequestFactory
+        def rfactory(environ):
+            return request
+        self.registry.registerUtility(rfactory, IRequestFactory)
+        from repoze.bfg.request import Request
+        request = Request.blank('/')
         context = DummyContext()
         directlyProvides(context, IContext)
         self._registerTraverserFactory(context, subpath=[''])
@@ -644,6 +648,9 @@ class TestRouter(unittest.TestCase):
         router = self._makeOne()
         start_response = DummyStartResponse()
         self.assertRaises(RuntimeError, router, environ, start_response)
+        # ``exception`` must be attached to request even if a suitable
+        # exception view cannot be found
+        self.assertEqual(request.exception.__class__, RuntimeError)
 
     def test_call_view_raises_exception_view(self):
         from repoze.bfg.interfaces import IViewClassifier