diff options
| -rw-r--r-- | CHANGES.txt | 2 | ||||
| -rw-r--r-- | docs/designdefense.rst | 364 | ||||
| -rw-r--r-- | docs/index.rst | 8 |
3 files changed, 374 insertions, 0 deletions
diff --git a/CHANGES.txt b/CHANGES.txt index 1dc78a37e..efbbe9dbc 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -15,6 +15,8 @@ Documentation - Add examples to ``repoze.bfg.url.model_url`` docstring. +- Add "Defending BFG Design" chapter. + Templates --------- diff --git a/docs/designdefense.rst b/docs/designdefense.rst new file mode 100644 index 000000000..4dbadae26 --- /dev/null +++ b/docs/designdefense.rst @@ -0,0 +1,364 @@ +Defending BFG's Design +====================== + +From time to time, challenges to various aspects of :mod:`repoze.bfg` +design are lodged. To give context to the discussions that follow, we +detail some of the design decisions and tradeoffs here. + +BFG Uses The Zope Component Architecture ("ZCA") +------------------------------------------------ + +BFG uses the :term:`Zope Component Architecture` (ZCA) "under the +hood". This is a point of some contention. :mod:`repoze.bfg` is of a +:term:`Zope` pedigree, so it was natural for it developers to use the +ZCA at its inception. However, :mod:`repoze.bfg` allegiance to its +Zope pedigree is not blind. We understand that using the ZCA has +issues and consequences, which we've attempted to address as best we +can. Here'a a full disclosure about BFG's use of the ZCA, and the +tradeoffs its usage involves. + +Problems +++++++++ + +The API that is commonly used to access data in a ZCA "component +registry" is not particularly pretty or intuitive, and sometimes it's +just plain obtuse. Likewise, the conceptual load on a casual source +code reader of code that uses the component architecture is somewhat +high. Consider a ZCA neophyte reading the code that performs a +typical "unnamed utility" lookup: + +.. code-block:: python + :linenos: + + from repoze.bfg.interfaces import ISettings + from zope.component import getUtility + settings = getUtility(ISettings) + +After this code runs, ``settings`` will be a Python dictionary. But +it's unlikely that any "civilian" would know that just by reading the +code. There are a number of comprehension issues with the bit of code +above that are pretty obvious. + +First, what's a "utility"? Well, for the purposes of this dicussion, +and for the purpose of the code above, it's just not very important. +If you really care, you can go read `this +<http://www.muthukadan.net/docs/zca.html#utility>`. However, still +folks need to understand the concept in order to parse the code while +reading it. This is problem number one. + +Second, what's this ``ISettings`` thing? Well, it's an +:term:`interface`. Is that important here? Not really, we're just +using it as a "key" for some lookup based on its identity as a marker: +it represents an object that has the dictionary API, but that's not +really very important. That's problem number two. + +Third of all, what does ``getUtility`` do? It's performing a lookup +for the ``ISettings`` "utility" that should return.. well, a utility. +Note how we've already built up a dependency on the understanding of +an :term:`interface` and the concept of "utility" to answer this +question: a bad sign so far. Note also that the answer is circular, a +*really* bad sign. + +Fourth, where does ``getUtility`` look to get the data? Well, the +"component registry" of course. What's component registry? Right. +Problem number four. + +Fifth, assuming you buy that there's some magical registry hanging +around, where *is* this registry? Homina homina... "around"? That's +sort of the best answer in this context (a more specific answer would +require knowledge of internals). Can there be more than one? Yes. +So *which* registry does it find the registration in? Well, the +"current" registry of course. In terms of :mod:`repoze.bfg`, the +current registry is a thread local variable. Using an API that +consults a thread local makes understanding how it works nonlocal. + +Sixth, fine, you've bought in to the fact that there's a registry that +is just "hanging around". But how does the registry get populated? +Why, :term:`ZCML` of course. Sometimes. In this particular case, +however, the registration of ``ISettings`` is made by the framework +itself "under the hood": it's not present in any ZCML. This is +extremely hard to comprehend. + +Clearly there's some amount of cognitive load here that needs to be +borne by a reader of code that extends the :mod:`repoze.bfg` framework +due to its use of the ZCA, even if he or she is already an expert +Python programmer and whom is an expert in the domain of web +applications. This is suboptimal. + +Ameliorations ++++++++++++++ + +First, the biggest amelioration: :mod:`repoze.bfg` *does not expect +application developers to understand ZCA concepts or its API*. If an +*application* developer needs to understand a ZCA concept or API +during the creation of a :mod:`repoze.bfg` application, we've failed +on some axis. + +Instead, the framework hides the presence of the ZCA behind +special-purpose API functions that *do* use the ZCA API. Take for +example the ``repoze.bfg.security.authenticated_userid`` function, +which returns the userid present in the current request or ``None`` if +no userid is present in the current request. The application +developer calls it like so: + +.. code-block:: python + :linenos: + + from repoze.bfg.security import authenticated_userid + userid = authenticated_userid(request) + +He now has the current user id. + +Under its hood however, the implementation of ``authenticated_userid`` +is this: + +.. code-block: python + :linenos: + + def authenticated_userid(request): + """ Return the userid of the currently authenticated user or + ``None`` if there is no authentication policy in effect or there + is no currently authenticated user. """ + + policy = queryUtility(IAuthenticationPolicy) + if policy is None: + return None + return policy.authenticated_userid(request) + +Using such wrappers, we strive to always hide the ZCA this way from +application developers: application developers should just never know +about the ZCA. They call a function with some object germane to the +domain, it returns a result; they needn't understand components at +all. A corollary that follows is that any reader of an application +that has been written using :mod:`repoze.bfg` needn't understand the +ZCA either. + +Hiding the ZCA from application developers and code readers in this +way a form of enhancing "domain specificity". No end user wants to +understand the minutiae of the mechanics of how a web framework does +its thing. People want to deal in concepts that are closer to the +domain they're working in: for example, web developers want to know +about *users*, not *utilities*. :mod:`repoze.bfg` uses the ZCA as an +implementation detail, not as a feature which is exposed to end users. + +However, unlike application developers, BFG *framework developers*, +including people who want to override :mod:`repoze.bfg` functionality +via preordained framework plugpoints like traversal or view lookup +*must* understand the ZCA. + +:mod:`repoze.bfg` framework developers were so concerned about +conceptual load issues of the ZCA API for framework developers that a +`replacement <http://svn.repoze.org/repoze.component/trunk>`_ was +actually developed. Though this package is fully functional and +well-tested, and its API is much nicer than the ZCA API, work on it +was largely abandoned and it is not used in :mod:`repoze.bfg`. We +continued to use the ZCA within :mod:`repoze.bfg`. + +Making framework developers and extenders understand the ZCA is a +tradeoff. We (the :mod:`repoze.bfg` developers) like the features +that the ZCA gives us, and we have long-ago borne the weight of +understanding what it does and how it works. The authors of +:mod:`repoze.bfg` understand the ZCA deeply and can read code that +uses it as easily as any other code. + +However, we do recognize that other developers who my want to extend +the framework are not as comfortable with ZCA we are with it. So, for +the purposes of being kind to framework developers who may be dismayed +by some of the more flagrant uses of the ZCA API in :mod:`repoze.bfg`, +we've turned the component registry used in BFG into something that is +accessible using the plain old dictionary API (like the +:mod:`repoze.component` API). Our example in the problem section +above was: + +.. code-block:: python + :linenos: + + from repoze.bfg.interfaces import ISettings + from zope.component import getUtility + settings = getUtility(ISettings) + +In a better world, we might be able to spell this as: + +.. code-block:: python + :linenos: + + from repoze.bfg.threadlocal import get_registry + + registry = get_registry() + settings = registry['settings'] + +In this world, we've removed the need to understand utilities and +interfaces. We *haven't* removed the need to understand the concept +of a *registry*, but for the purposes of this example, it's simply a +dictionary. We haven't killed off the concept of a thread local +either. Let's kill off thread locals, pretending to want to do this +in some code that has access to the :term:`request`: + +.. code-block:: python + :linenos: + + registry = request.registry + settings = registry['settings'] + +In *this* world, we've reduced the conceptual problem to understanding +attributes and the dictionary API. Every Python programmer knows +these things, even framework programmers! + +We continued using ZCA rather than ditching it for +:mod:`repoze.component` largely because the ZCA concept of interfaces +provides for use of an interface hierarchy, which is useful in a lot +of scenarios (such as context type inheritance). Coming up with a +marker type that was something like an interface that allowed for this +functionality seemed like it was just reinventing the wheel. + +While :mod:`repoze.bfg` still uses some suboptimal unnamed utility +registrations, future versions of it will (where possible) disuse +these things in favor of straight dictionary assignments and lookups, +as demonstrated above, to be kinder to new developers and extenders. +We'll continue to seek ways to reduce framework extender cognitive +load. + +Rationale ++++++++++ + +Here are the main rationales for BFG's design decision to use the ZCA: + +- Pedigree. A nontrivial part of the answer to this question is + "pedigree". Much of the design of :mod:`repoze.bfg` is stolen + directly from :term:`Zope`. Zope uses the ZCA to do a number of + tricks. :mod:`repoze.bfg` mimics these tricks apeishly, and, + because the ZCA works pretty well for that set of tricks, the + :mod:`repoze.bfg` mimicry uses it for the same purposes. For + example, the way that BFG maps a :term:`request` to a :term:`view + callable` is lifted almost entirely from Zope. The ZCA plays an + important role in the particulars of how this request to view + mapping is done. + +- Features. The ZCA essentially provides what can be considered + something like a "superdictionary", which allows for more complex + lookups than retrieving a value based on a single key. Some of this + lookup capability is very useful for end users, such as being able + to register a view that is only found when the context is some class + of object, or when the context implements some :term:`interface`. + +- Singularity. There's only one "place" where "application + configuration" lives in a BFG application: in a component registry. + The component registry answers questions made to it by the framework + at runtime based on the configuration of *an application*. Note: + "an application" is not the same as "a process", multiple + independently configured copies of the same BFG application are + capable of running in the same process space. + +- Composability. A ZCA registry can be populated imperatively, or + there's an existing mechanism to populate a registry via the use of + a configuration file (ZCML). We didn't need to write a frontend + from scratch to make use of configuration-file-driven registry + population. + +- Pluggability. Use of the ZCA allows for framework extensibility via + a well-defined and widely understood plugin architecture. As long + as framework developers and extenders understand the ZCA, it's + possible to extend BFG almost arbitrarily. For example, it's + relatively easy to build a ZCML directive that registers several + views "all at once", allowing app developers to use that ZCML + directive as a "macro" in code that they write. This is somewhat of + a differentiating feature from other (non-Zope) frameworks. + +- Testability. Judicious use of the ZCA in framework code makes + testing that code slightly easier. Instead of using monkeypatching + or other facilities to register mock objects for testing, we inject + dependencies via ZCA registrations and then use lookups in the code + find our mock objects. + +- Speed. The ZCA is very fast for a specific set of complex lookup + scenarios that BFG uses, having been optimized through the years for + just these purposes. The ZCA contains optional C code for this + purpose which demonstrably has no (or very few) bugs. + +- Ecosystem. Many existing Zope packages can be used in + :mod:`repoze.bfg` with few (or no) changes due to our use of the ZCA + and :term:`ZCML`. + +Conclusion +++++++++++ + +If you only *develop applications* using :mod:`repoze.bfg`, there's +just basically nothing to think about here. You just should never +need to understand the ZCA or even know about its presence: use +documented APIs instead. If you're an application developer who +doesn't read API documentation because its unmanly, but instead uses +raw source code, and considers everything an API, and you've pained +yourself into a conceptual corner as a result of needing to wrestle +with some ZCA-using internals, it's hard to have a lot of sympathy for +you. You'll either need to get familiar with how we're using the ZCA +or you'll need to use only the documented APIs; that's why we document +'em. + +If you *extend* or *develop* :mod:`repoze.bfg` (create new ZCML +directives, use some of the more obscure "ZCML hooks" as described in +:ref:`hooks_chapter`, or work on the :mod:`repoze.bfg` core code), you +will be faced with needing to understand at least some ZCA concepts. +The ZCA API is pretty quirky: we've tried to make it at least slightly +nicer by disusing it for common registrations and lookups such as +unnamed utilities. Some places it's used unabashedly, and will be +forever. We know it's a bit quirky, but it's also useful and +fundamentally understandable if you take the time to do some reading +about it. + +BFG "Encourages Use of ZCML" +---------------------------- + +:term:`ZCML` is a configuration language that can be used to configure +the :term:`Zope Component Architecture` registry that BFG uses as its +application configuration. + +Quick answer: well, no, it doesn't.. not really. You can use the +``bfg_view`` decorator for the most common form of configuration. +But, yes, your application currently does need to possess a ZCML file +for it to begin executing successfully even if its only contents are a +``<scan>`` directive that kicks off the location of decorated views. + +In any case, in the interest of completeness and in the spirit of , +BFG 1.2 will include a completely imperative mode for all +configuration. You will be able to make "single file" apps in this +mode, which should help people who need to see everything done +completely imperatively. For example, the very most basic +:mod:`repoze.bfg` "helloworld" program will become something like:: + + from webob import Response + from wsgiref import simple_server + from repoze.bfg.registry import Registry + from repoze.bfg.router import Router + + def helloworld_view(request): + return Response(hello') + + if __name__ == '__main__': + reg = Registry() + reg.view(helloworld_view) + app = Router(reg) + simple_server.make_server('', 8080, app).serve_forever() + +In this mode, no ZCML will be required. Hopefully this mode will +allow people who are used to doing everything imperatively feel more +comfortable. + +Other Topics +------------ + +We'll be trying to cover the following in this document as time allows: + +- BFG View Lookup and Registration Is "Complex" + +- BFG Template Lookup Is "Complex" + +- BFG Views Do Not Accept Arbitrary Keyword Arguments + +- BFG Does Traversal, And I Don't Like Traversal + +- BFG Does URL Dispatch, And I Don't Like URL Dispatch + +Other challenges are encouraged to be sent to the `Repoze-Dev +<http://lists.repoze.org/listinfo/repoze-dev>`_ maillist. We'll try +to address them by considering a design change, or at very least via +exposition here. diff --git a/docs/index.rst b/docs/index.rst index efed839c9..a1a2d377b 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -145,6 +145,14 @@ Detailed Change History changes +Design Documentation +==================== + +.. toctree:: + :maxdepth: 2 + + designdefense + Sample Applications =================== |