diff options
Diffstat (limited to 'docs/tutorials')
109 files changed, 2707 insertions, 1258 deletions
diff --git a/docs/tutorials/wiki2/authentication.rst b/docs/tutorials/wiki2/authentication.rst index 3f2fcec83..a4937d93e 100644 --- a/docs/tutorials/wiki2/authentication.rst +++ b/docs/tutorials/wiki2/authentication.rst @@ -10,7 +10,7 @@ APIs to add login and logout functionality to our wiki. We will implement authentication with the following steps: -* Add an :term:`authentication policy` and a ``request.user`` computed property +* Add a :term:`security policy` and a ``request.user`` computed property (``security.py``). * Add routes for ``/login`` and ``/logout`` (``routes.py``). * Add login and logout views (``views/auth.py``). @@ -18,25 +18,24 @@ We will implement authentication with the following steps: * Add "Login" and "Logout" links to every page based on the user's authenticated state (``layout.jinja2``). * Make the existing views verify user state (``views/default.py``). -* Redirect to ``/login`` when a user is denied access to any of the views that - require permission, instead of a default "403 Forbidden" page - (``views/auth.py``). +* Redirect to ``/login`` when a user is not logged in and is denied access to any of the views that require permission (``views/auth.py``). +* Show a custom "403 Forbidden" page if a logged in user is denied access to any views that require permission (``views/auth.py``). Authenticating requests ----------------------- -The core of :app:`Pyramid` authentication is an :term:`authentication policy` +The core of :app:`Pyramid` authentication is a :term:`security policy` which is used to identify authentication information from a ``request``, as well as handling the low-level login and logout operations required to track users across requests (via cookies, headers, or whatever else you can imagine). -Add the authentication policy -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Add the security policy +~~~~~~~~~~~~~~~~~~~~~~~ -Create a new file ``tutorial/security.py`` with the following content: +Update ``tutorial/security.py`` with the following content: .. literalinclude:: src/authentication/tutorial/security.py :linenos: @@ -44,49 +43,26 @@ Create a new file ``tutorial/security.py`` with the following content: Here we've defined: -* A new authentication policy named ``MyAuthenticationPolicy``, which is - subclassed from Pyramid's - :class:`pyramid.authentication.AuthTktAuthenticationPolicy`, which tracks the - :term:`userid` using a signed cookie (lines 7-11). -* A ``get_user`` function, which can convert the ``unauthenticated_userid`` - from the policy into a ``User`` object from our database (lines 13-17). -* The ``get_user`` is registered on the request as ``request.user`` to be used - throughout our application as the authenticated ``User`` object for the - logged-in user (line 27). - -The logic in this file is a little bit interesting, so we'll go into detail -about what's happening here: - -First, the default authentication policies all provide a method named -``unauthenticated_userid`` which is responsible for the low-level parsing -of the information in the request (cookies, headers, etc.). If a ``userid`` -is found, then it is returned from this method. This is named -``unauthenticated_userid`` because, at the lowest level, it knows the value of -the userid in the cookie, but it doesn't know if it's actually a user in our -system (remember, anything the user sends to our app is untrusted). - -Second, our application should only care about ``authenticated_userid`` and -``request.user``, which have gone through our application-specific process of -validating that the user is logged in. - -In order to provide an ``authenticated_userid`` we need a verification step. -That can happen anywhere, so we've elected to do it inside of the cached -``request.user`` computed property. This is a convenience that makes -``request.user`` the source of truth in our system. It is either ``None`` or -a ``User`` object from our database. This is why the ``get_user`` function -uses the ``unauthenticated_userid`` to check the database. +* A new security policy named ``MySecurityPolicy``, which is implementing most of the :class:`pyramid.interfaces.ISecurityPolicy` interface by tracking a :term:`identity` using a signed cookie implemented by :class:`pyramid.authentication.AuthTktCookieHelper` (lines 7-29). +* The ``request.user`` computed property is registered for use throughout our application as the authenticated ``tutorial.models.User`` object for the logged-in user (line 38-39). +Our new :term:`security policy` defines how our application will remember, forget, and identify users. +It also handles authorization, which we'll cover in the next chapter (if you're wondering why we didn't implement the ``permits`` method yet). -Configure the app -~~~~~~~~~~~~~~~~~ +Identifying the current user is done in a couple steps: -Since we've added a new ``tutorial/security.py`` module, we need to include it. -Open the file ``tutorial/__init__.py`` and edit the following lines: +1. The ``MySecurityPolicy.authenticated_identity`` method asks the cookie helper to pull the identity from the request. + This value is ``None`` if the cookie is missing or the content cannot be verified. +2. We then translate the identity into a ``tutorial.models.User`` object by looking for a record in the database. + +This is a good spot to confirm that the user is actually allowed to access our application. +For example, maybe they were marked deleted or banned and we should return ``None`` instead of the ``user`` object. + +Finally, :attr:`pyramid.request.Request.authenticated_identity` contains either ``None`` or a ``tutorial.models.User`` instance and that value is aliased to ``request.user`` for convenience in our application. -.. literalinclude:: src/authentication/tutorial/__init__.py - :linenos: - :emphasize-lines: 11 - :language: python + +Configure the app +~~~~~~~~~~~~~~~~~ Our authentication policy is expecting a new setting, ``auth.secret``. Open the file ``development.ini`` and add the highlighted line below: @@ -97,7 +73,7 @@ the file ``development.ini`` and add the highlighted line below: :lineno-match: :language: ini -Finally, best practices tell us to use a different secret for production, so +Finally, best practices tell us to use a different secret in each environment, so open ``production.ini`` and add a different secret: .. literalinclude:: src/authentication/production.ini @@ -106,6 +82,14 @@ open ``production.ini`` and add a different secret: :lineno-match: :language: ini +And ``testing.ini``: + +.. literalinclude:: src/authentication/testing.ini + :lines: 17-19 + :emphasize-lines: 3 + :lineno-match: + :language: ini + Add permission checks ~~~~~~~~~~~~~~~~~~~~~ @@ -125,7 +109,7 @@ Remember our goals: Open the file ``tutorial/views/default.py`` and fix the following import: .. literalinclude:: src/authentication/tutorial/views/default.py - :lines: 5-9 + :lines: 3-7 :lineno-match: :emphasize-lines: 2 :language: python @@ -135,7 +119,7 @@ Change the highlighted line. In the same file, now edit the ``edit_page`` view function: .. literalinclude:: src/authentication/tutorial/views/default.py - :lines: 45-60 + :lines: 44-59 :lineno-match: :emphasize-lines: 5-7 :language: python @@ -148,18 +132,16 @@ If the user either is not logged in or the user is not the page's creator In the same file, now edit the ``add_page`` view function: .. literalinclude:: src/authentication/tutorial/views/default.py - :lines: 62-76 + :lines: 61- :lineno-match: :emphasize-lines: 3-5,13 :language: python Only the highlighted lines need to be changed. -If the user either is not logged in or is not in the ``basic`` or ``editor`` -roles, then we raise ``HTTPForbidden``, which will return a "403 Forbidden" -response to the user. However, we will hook this later to redirect to the login -page. Also, now that we have ``request.user``, we no longer have to hard-code -the creator as the ``editor`` user, so we can finally drop that hack. +If the user either is not logged in or is not in the ``basic`` or ``editor`` roles, then we raise ``HTTPForbidden``, which will trigger our forbidden view to compute a response. +However, we will hook this later to redirect to the login page. +Also, now that we have ``request.user``, we no longer have to hard-code the creator as the ``editor`` user, so we can finally drop that hack. These simple checks should protect our views. @@ -215,6 +197,9 @@ This code adds three new views to the application: The check is done by first finding a ``User`` record in the database, then using our ``user.check_password`` method to compare the hashed passwords. + At a privilege boundary we are sure to reset the CSRF token using :meth:`pyramid.csrf.new_csrf_token`. + If we were using sessions we would want to invalidate that as well. + If the credentials are valid, then we use our authentication policy to store the user's id in the response using :meth:`pyramid.security.remember`. @@ -227,16 +212,19 @@ This code adds three new views to the application: credentials using :meth:`pyramid.security.forget`, then redirecting them to the front page. + At a privilege boundary we are sure to reset the CSRF token using :meth:`pyramid.csrf.new_csrf_token`. + If we were using sessions we would want to invalidate that as well. + - The ``forbidden_view`` is registered using the :class:`pyramid.view.forbidden_view_config` decorator. This is a special :term:`exception view`, which is invoked when a :class:`pyramid.httpexceptions.HTTPForbidden` exception is raised. - This view will handle a forbidden error by redirecting the user to - ``/login``. As a convenience, it also sets the ``next=`` query string to the - current URL (the one that is forbidding access). This way, if the user - successfully logs in, they will be sent back to the page which they had been - trying to access. + By default, the view will return a "403 Forbidden" response and display our ``403.jinja2`` template (added below). + + However, if the user is not logged in, this view will handle a forbidden error by redirecting the user to ``/login``. + As a convenience, it also sets the ``next=`` query string to the current URL (the one that is forbidding access). + This way, if the user successfully logs in, they will be sent back to the page which they had been trying to access. Add the ``login.jinja2`` template @@ -258,9 +246,9 @@ Open ``tutorial/templates/layout.jinja2`` and add the following code as indicated by the highlighted lines. .. literalinclude:: src/authentication/tutorial/templates/layout.jinja2 - :lines: 35-46 + :lines: 35-48 :lineno-match: - :emphasize-lines: 2-10 + :emphasize-lines: 2-12 :language: html The ``request.user`` will be ``None`` if the user is not authenticated, or a @@ -269,6 +257,17 @@ make the logout link shown only when the user is logged in, and conversely the login link is only shown when the user is logged out. +Add the ``403.jinja2`` template +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Create ``tutorial/templates/403.jinja2`` with the following content: + +.. literalinclude:: src/authentication/tutorial/templates/403.jinja2 + :language: html + +The above template is referenced in the forbidden view that we just added in ``tutorial/views/auth.py``. + + Viewing the application in a browser ------------------------------------ @@ -287,15 +286,16 @@ following URLs, checking that the result is as expected: - http://localhost:6543/FrontPage/edit_page invokes the ``edit_page`` view for the ``FrontPage`` page object. It is executable by only the ``editor`` user. - If a different user (or the anonymous user) invokes it, then a login form - will be displayed. Supplying the credentials with the username ``editor`` and - password ``editor`` will display the edit page form. + If a different user invokes it, then the "403 Forbidden" page will be displayed. + If an anonymous user invokes it, then a login form will be displayed. + Supplying the credentials with the username ``editor`` and password ``editor`` will display the edit page form. - http://localhost:6543/add_page/SomePageName invokes the ``add_page`` view for a page. If the page already exists, then it redirects the user to the ``edit_page`` view for the page object. It is executable by either the - ``editor`` or ``basic`` user. If a different user (or the anonymous user) - invokes it, then a login form will be displayed. Supplying the credentials + ``editor`` or ``basic`` user. + If an anonymous user invokes it, then a login form will be displayed. + Supplying the credentials with either the username ``editor`` and password ``editor``, or username ``basic`` and password ``basic``, will display the edit page form. diff --git a/docs/tutorials/wiki2/authorization.rst b/docs/tutorials/wiki2/authorization.rst index 234f40e3b..e8f95f8cf 100644 --- a/docs/tutorials/wiki2/authorization.rst +++ b/docs/tutorials/wiki2/authorization.rst @@ -12,10 +12,8 @@ the constraints from the view function itself. We will implement access control with the following steps: -* Update the :term:`authentication policy` to break down the :term:`userid` - into a list of :term:`principals <principal>` (``security.py``). -* Define an :term:`authorization policy` for mapping users, resources and - permissions (``security.py``). +* Update the :term:`security policy` to break down the :term:`identity` into a list of :term:`principals <principal>` (``security.py``). +* Utilize the :class:`pyramid.authorization.ACLHelper` to support a per-context mapping of principals to permissions (``security.py``). * Add new :term:`resource` definitions that will be used as the :term:`context` for the wiki pages (``routes.py``). * Add an :term:`ACL` to each resource (``routes.py``). @@ -23,8 +21,8 @@ We will implement access control with the following steps: (``views/default.py``). -Add user principals -------------------- +Add ACL support +--------------- A :term:`principal` is a level of abstraction on top of the raw :term:`userid` that describes the user in terms of its capabilities, roles, or other @@ -42,7 +40,7 @@ Open the file ``tutorial/security.py`` and edit it as follows: .. literalinclude:: src/authorization/tutorial/security.py :linenos: - :emphasize-lines: 3-6,17-24 + :emphasize-lines: 2,4-7,15,37-48 :language: python Only the highlighted lines need to be added. @@ -51,33 +49,16 @@ Note that the role comes from the ``User`` object. We also add the ``user.id`` as a principal for when we want to allow that exact user to edit pages which they have created. +We're using the :class:`pyramid.authorization.ACLHelper`, which will suffice for most applications. +It uses the :term:`context` to define the mapping between a :term:`principal` and :term:`permission` for the current request via the ``__acl__`` method or attribute. -Add the authorization policy ----------------------------- - -We already added the :term:`authorization policy` in the previous chapter -because :app:`Pyramid` requires one when adding an -:term:`authentication policy`. However, it was not used anywhere, so we'll -mention it now. - -In the file ``tutorial/security.py``, notice the following lines: - -.. literalinclude:: src/authorization/tutorial/security.py - :lines: 38-40 - :lineno-match: - :emphasize-lines: 2 - :language: python - -We're using the :class:`pyramid.authorization.ACLAuthorizationPolicy`, which -will suffice for most applications. It uses the :term:`context` to define the -mapping between a :term:`principal` and :term:`permission` for the current -request via the ``__acl__``. +The ``permits`` method completes our implementation of the :class:`pyramid.interfaces.ISecurityPolicy` interface and enables our application to use :attr:`pyramid.request.Request.has_permission` and the ``permission=`` constraint on views. Add resources and ACLs ---------------------- -Resources are the hidden gem of :app:`Pyramid`. You've made it! +Resources and context are the hidden gems of :app:`Pyramid`. You've made it! Every URL in a web application represents a :term:`resource` (the "R" in Uniform Resource Locator). Often the resource is something in your data model, @@ -108,7 +89,7 @@ Open the file ``tutorial/routes.py`` and edit the following lines: .. literalinclude:: src/authorization/tutorial/routes.py :linenos: - :emphasize-lines: 1-11,17- + :emphasize-lines: 1-11,18- :language: python The highlighted lines need to be edited or added. @@ -120,7 +101,7 @@ the principals of either ``role:editor`` or ``role:basic`` to have the ``create`` permission: .. literalinclude:: src/authorization/tutorial/routes.py - :lines: 30-38 + :lines: 31-39 :lineno-match: :emphasize-lines: 5-9 :language: python @@ -129,7 +110,7 @@ The ``NewPage`` is loaded as the :term:`context` of the ``add_page`` route by declaring a ``factory`` on the route: .. literalinclude:: src/authorization/tutorial/routes.py - :lines: 18-19 + :lines: 19-20 :lineno-match: :emphasize-lines: 1-2 :language: python @@ -138,7 +119,7 @@ The ``PageResource`` class defines the :term:`ACL` for a ``Page``. It uses an actual ``Page`` object to determine *who* can do *what* to the page. .. literalinclude:: src/authorization/tutorial/routes.py - :lines: 47- + :lines: 48- :lineno-match: :emphasize-lines: 5-10 :language: python @@ -147,7 +128,7 @@ The ``PageResource`` is loaded as the :term:`context` of the ``view_page`` and ``edit_page`` routes by declaring a ``factory`` on the routes: .. literalinclude:: src/authorization/tutorial/routes.py - :lines: 17-21 + :lines: 18-22 :lineno-match: :emphasize-lines: 1,4-5 :language: python @@ -167,7 +148,7 @@ Open the file ``tutorial/views/default.py``. First, you can drop a few imports that are no longer necessary: .. literalinclude:: src/authorization/tutorial/views/default.py - :lines: 5-7 + :lines: 3-5 :lineno-match: :emphasize-lines: 1 :language: python @@ -207,7 +188,7 @@ Note the ``pagename`` here is pulled off of the context instead of ``request.matchdict``. The factory has done a lot of work for us to hide the actual route pattern. -The ACLs defined on each :term:`resource` are used by the :term:`authorization +The ACLs defined on each :term:`resource` are used by the :term:`security policy` to determine if any :term:`principal` is allowed to have some :term:`permission`. If this check fails (for example, the user is not logged in) then an ``HTTPForbidden`` exception will be raised automatically. Thus @@ -238,14 +219,14 @@ following URLs, checking that the result is as expected: - http://localhost:6543/FrontPage/edit_page invokes the ``edit_page`` view for the ``FrontPage`` page object. It is executable by only the ``editor`` user. - If a different user (or the anonymous user) invokes it, then a login form + If an anonymous user invokes it, then a login form will be displayed. Supplying the credentials with the username ``editor`` and password ``editor`` will display the edit page form. - http://localhost:6543/add_page/SomePageName invokes the ``add_page`` view for a page. If the page already exists, then it redirects the user to the ``edit_page`` view for the page object. It is executable by either the - ``editor`` or ``basic`` user. If a different user (or the anonymous user) + ``editor`` or ``basic`` user. If an anonymous user invokes it, then a login form will be displayed. Supplying the credentials with either the username ``editor`` and password ``editor``, or username ``basic`` and password ``basic``, will display the edit page form. diff --git a/docs/tutorials/wiki2/basiclayout.rst b/docs/tutorials/wiki2/basiclayout.rst index ae58d80a5..ef78e052b 100644 --- a/docs/tutorials/wiki2/basiclayout.rst +++ b/docs/tutorials/wiki2/basiclayout.rst @@ -58,24 +58,24 @@ dictionary of settings parsed from the ``.ini`` file, which contains deployment-related values, such as ``pyramid.reload_templates``, ``sqlalchemy.url``, and so on. -Next include the package ``models`` using a dotted Python path. The exact -setup of the models will be covered later. +Next include :term:`Jinja2` templating bindings so that we can use renderers +with the ``.jinja2`` extension within our project. .. literalinclude:: src/basiclayout/tutorial/__init__.py :lines: 8 :lineno-match: :language: py -Next include :term:`Jinja2` templating bindings so that we can use renderers -with the ``.jinja2`` extension within our project. +Next include the ``routes`` module using a dotted Python path. This module will +be explained in the next section. .. literalinclude:: src/basiclayout/tutorial/__init__.py :lines: 9 :lineno-match: :language: py -Next include the ``routes`` module using a dotted Python path. This module will -be explained in the next section. +Next include the package ``models`` using a dotted Python path. The exact +setup of the models will be covered later. .. literalinclude:: src/basiclayout/tutorial/__init__.py :lines: 10 @@ -207,7 +207,7 @@ Without repeating ourselves, we will point out the differences between this view Content models with the ``models`` package ------------------------------------------ -In an SQLAlchemy-based application, a *model* object is an object composed by +In a SQLAlchemy-based application, a *model* object is an object composed by querying the SQL database. The ``models`` package is where the ``alchemy`` cookiecutter put the classes that implement our models. @@ -348,3 +348,9 @@ code in our stock application. The ``Index`` import and the ``Index`` object creation in ``mymodel.py`` is not required for this tutorial, and will be removed in the next step. + +Tests +----- + +The project contains a basic structure for a test suite using ``pytest``. +The structure is covered later in :ref:`wiki2_adding_tests`. diff --git a/docs/tutorials/wiki2/definingmodels.rst b/docs/tutorials/wiki2/definingmodels.rst index 4b80e09ac..f84ca6588 100644 --- a/docs/tutorials/wiki2/definingmodels.rst +++ b/docs/tutorials/wiki2/definingmodels.rst @@ -32,8 +32,10 @@ parameter in the ``setup()`` function. Open ``tutorial/setup.py`` and edit it to look like the following: .. literalinclude:: src/models/setup.py + :lines: 11-30 :linenos: - :emphasize-lines: 11-24 + :lineno-match: + :emphasize-lines: 3 :language: python It is a good practice to sort packages alphabetically to make them easier to find. @@ -42,7 +44,9 @@ After adding ``bcrypt`` and sorting packages, we should have the above ``require .. note:: - We are using the ``bcrypt`` package from PyPI to hash our passwords securely. There are other one-way hash algorithms for passwords if ``bcrypt`` is an issue on your system. Just make sure that it's an algorithm approved for storing passwords versus a generic one-way hash. + We are using the ``bcrypt`` package from PyPI to hash our passwords securely. + There are other one-way hash algorithms for passwords if ``bcrypt`` is an issue on your system. + Just make sure that it's an algorithm approved for storing passwords versus a generic one-way hash. Running ``pip install -e .`` @@ -245,7 +249,7 @@ following: .. literalinclude:: src/models/tutorial/scripts/initialize_db.py :linenos: :language: python - :emphasize-lines: 11-24 + :emphasize-lines: 15-28 Only the highlighted lines need to be changed. diff --git a/docs/tutorials/wiki2/definingviews.rst b/docs/tutorials/wiki2/definingviews.rst index a434039ca..122164083 100644 --- a/docs/tutorials/wiki2/definingviews.rst +++ b/docs/tutorials/wiki2/definingviews.rst @@ -26,14 +26,15 @@ is not a dependency of the original "tutorial" application. We need to add a dependency on the ``docutils`` package to our ``tutorial`` package's ``setup.py`` file by assigning this dependency to the ``requires`` -parameter in the ``setup()`` function. +list. Open ``tutorial/setup.py`` and edit it to look like the following: .. literalinclude:: src/views/setup.py - :linenos: - :emphasize-lines: 14 - :language: python + :lines: 11-31 + :lineno-match: + :emphasize-lines: 4 + :language: python Only the highlighted line needs to be added. @@ -50,7 +51,7 @@ were provided at the time we created the project. As an example, the CSS file will be accessed via ``http://localhost:6543/static/theme.css`` by virtue of the call to the -``add_static_view`` directive we've made in the ``routes.py`` file. Any number +``add_static_view`` directive we've made in the ``tutorial/routes.py`` file. Any number and type of static assets can be placed in this directory (or subdirectories) and are just referred to by URL or by using the convenience method ``static_url``, e.g., ``request.static_url('<package>:static/foo.css')`` within @@ -63,7 +64,7 @@ Adding routes to ``routes.py`` This is the `URL Dispatch` tutorial, so let's start by adding some URL patterns to our app. Later we'll attach views to handle the URLs. -The ``routes.py`` file contains :meth:`pyramid.config.Configurator.add_route` +The ``tutorial/routes.py`` file contains :meth:`pyramid.config.Configurator.add_route` calls which serve to add routes to our application. First we'll get rid of the existing route created by the template using the name ``'home'``. It's only an example and isn't relevant to our application. @@ -96,13 +97,13 @@ order they're registered. decorator attached to the ``edit_page`` view function, which in turn will be indicated by ``route_name='edit_page'``. -As a result of our edits, the ``routes.py`` file should look like the +As a result of our edits, the ``tutorial/routes.py`` file should look like the following: .. literalinclude:: src/views/tutorial/routes.py - :linenos: - :emphasize-lines: 3-6 - :language: python + :linenos: + :emphasize-lines: 3-6 + :language: python The highlighted lines are the ones that need to be added or edited. @@ -117,18 +118,41 @@ The highlighted lines are the ones that need to be added or edited. behavior in your own apps. +CSRF protection +=============== + +When handling HTML forms that mutate data in our database we need to verify that the form submission is legitimate and not from a URL embedded in a third-party website. +This is done by adding a unique token to each form that a third-party could not easily guess. +Read more about CSRF at :ref:`csrf_protection`. +For this tutorial, we'll store the active CSRF token in a cookie. + +Let's add a new ``tutorial/security.py`` file: + +.. literalinclude:: src/views/tutorial/security.py + :linenos: + :emphasize-lines: 5-6 + :language: python + +Since we've added a new ``tutorial/security.py`` module, we need to include it. +Open the file ``tutorial/__init__.py`` and edit the following lines: + +.. literalinclude:: src/views/tutorial/__init__.py + :linenos: + :emphasize-lines: 9 + :language: python + +On forms that mutate data, we'll be sure to add the CSRF token to the form, using :func:`pyramid.csrf.get_csrf_token`. + + Adding view functions in ``views/default.py`` ============================================= It's time for a major change. Open ``tutorial/views/default.py`` and -edit it to look like the following: +replace it with the following: .. literalinclude:: src/views/tutorial/views/default.py - :linenos: - :language: python - :emphasize-lines: 1-9,14- - -The highlighted lines need to be added or edited. + :linenos: + :language: python We added some imports, and created a regular expression to find "WikiWords". @@ -137,7 +161,7 @@ when originally rendered after we selected the ``sqlalchemy`` backend option in the cookiecutter. It was only an example and isn't relevant to our application. We also deleted the ``db_err_msg`` string. -Then we added four :term:`view callable` functions to our ``views/default.py`` +Then we added four :term:`view callable` functions to our ``tutorial/views/default.py`` module, as mentioned in the previous step: * ``view_wiki()`` - Displays the wiki itself. It will answer on the root URL. @@ -163,10 +187,10 @@ The ``view_wiki`` view function Following is the code for the ``view_wiki`` view function and its decorator: .. literalinclude:: src/views/tutorial/views/default.py - :lines: 17-20 - :lineno-match: - :linenos: - :language: python + :lines: 16-19 + :lineno-match: + :linenos: + :language: python ``view_wiki()`` is the :term:`default view` that gets called when a request is made to the root URL of our wiki. It always redirects to a URL which @@ -174,12 +198,12 @@ represents the path to our "FrontPage". The ``view_wiki`` view callable always redirects to the URL of a Page resource named "FrontPage". To do so, it returns an instance of the -:class:`pyramid.httpexceptions.HTTPFound` class (instances of which implement +:class:`pyramid.httpexceptions.HTTPSeeOther` class (instances of which implement the :class:`pyramid.interfaces.IResponse` interface, like :class:`pyramid.response.Response`). It uses the :meth:`pyramid.request.Request.route_url` API to construct a URL to the ``FrontPage`` page (i.e., ``http://localhost:6543/FrontPage``), and uses it as -the "location" of the ``HTTPFound`` response, forming an HTTP redirect. +the "location" of the ``HTTPSeeOther`` response, forming an HTTP redirect. The ``view_page`` view function @@ -188,10 +212,10 @@ The ``view_page`` view function Here is the code for the ``view_page`` view function and its decorator: .. literalinclude:: src/views/tutorial/views/default.py - :lines: 22-42 - :lineno-match: - :linenos: - :language: python + :lines: 21-41 + :lineno-match: + :linenos: + :language: python ``view_page()`` is used to display a single page of our wiki. It renders the :term:`reStructuredText` body of a page (stored as the ``data`` attribute of a @@ -241,10 +265,10 @@ The ``edit_page`` view function Here is the code for the ``edit_page`` view function and its decorator: .. literalinclude:: src/views/tutorial/views/default.py - :lines: 44-56 - :lineno-match: - :linenos: - :language: python + :lines: 43-55 + :lineno-match: + :linenos: + :language: python ``edit_page()`` is invoked when a user clicks the "Edit this Page" button on the view form. It renders an edit form, but it also acts as the handler for the @@ -252,14 +276,13 @@ form which it renders. The ``matchdict`` attribute of the request passed to the ``edit_page`` view will have a ``'pagename'`` key matching the name of the page that the user wants to edit. -If the view execution *is* a result of a form submission (i.e., the expression -``'form.submitted' in request.params`` is ``True``), the view grabs the +If the view execution *is* a result of a form submission (i.e., ``request.method == 'POST'``), the view grabs the ``body`` element of the request parameters and sets it as the ``data`` attribute of the page object. It then redirects to the ``view_page`` view of the wiki page. If the view execution is *not* a result of a form submission (i.e., the -expression ``'form.submitted' in request.params`` is ``False``), the view +expression ``request.method != 'POST'``), the view simply renders the edit form, passing the page object and a ``save_url`` which will be used as the action of the generated form. @@ -279,10 +302,10 @@ The ``add_page`` view function Here is the code for the ``add_page`` view function and its decorator: .. literalinclude:: src/views/tutorial/views/default.py - :lines: 58- - :lineno-match: - :linenos: - :language: python + :lines: 57- + :lineno-match: + :linenos: + :language: python ``add_page()`` is invoked when a user clicks on a *WikiWord* which isn't yet represented as a page in the system. The ``add_link`` function within the @@ -301,7 +324,7 @@ the database. If it already exists, then the client is redirected to the ``edit_page`` view, else we continue to the next check. If the view execution *is* a result of a form submission (i.e., the expression -``'form.submitted' in request.params`` is ``True``), we grab the page body from +``request.method == 'POST'``), we grab the page body from the form data, create a Page object with this page body and the name taken from ``matchdict['pagename']``, and save it into the database using ``request.dbession.add``. Since we have not yet covered authentication, we @@ -312,7 +335,7 @@ Finally, we redirect the client back to the ``view_page`` view for the newly created page. If the view execution is *not* a result of a form submission (i.e., the -expression ``'form.submitted' in request.params`` is ``False``), the view +expression ``request.method != 'POST'`` is ``False``), the view callable renders a template. To do so, it generates a ``save_url`` which the template uses as the form post URL during rendering. We're lazy here, so we're going to use the same template (``templates/edit.jinja2``) for the add @@ -339,9 +362,9 @@ Update ``tutorial/templates/layout.jinja2`` with the following content, as indicated by the emphasized lines: .. literalinclude:: src/views/tutorial/templates/layout.jinja2 - :linenos: - :emphasize-lines: 11,35-37 - :language: html + :linenos: + :emphasize-lines: 11,35-37 + :language: html Since we're using a templating engine, we can factor common boilerplate out of our page templates into reusable components. One method for doing this is @@ -350,8 +373,7 @@ template inheritance via blocks. - We have defined two placeholders in the layout template where a child template can override the content. These blocks are named ``subtitle`` (line 11) and ``content`` (line 36). -- Please refer to the `Jinja2 documentation <https://palletsprojects.com/p/jinja/>`_ for more information about template - inheritance. +- Please refer to the `Jinja2 documentation <https://palletsprojects.com/p/jinja/>`_ for more information about template inheritance. The ``view.jinja2`` template @@ -360,8 +382,8 @@ The ``view.jinja2`` template Create ``tutorial/templates/view.jinja2`` and add the following content: .. literalinclude:: src/views/tutorial/templates/view.jinja2 - :linenos: - :language: html + :linenos: + :language: html This template is used by ``view_page()`` for displaying a single wiki page. @@ -384,9 +406,9 @@ The ``edit.jinja2`` template Create ``tutorial/templates/edit.jinja2`` and add the following content: .. literalinclude:: src/views/tutorial/templates/edit.jinja2 - :linenos: - :emphasize-lines: 1,3,12,14,17 - :language: html + :linenos: + :emphasize-lines: 1,3,12,13,15,18 + :language: html This template serves two use cases. It is used by ``add_page()`` and ``edit_page()`` for adding and editing a wiki page. It displays a page @@ -396,11 +418,13 @@ containing a form and which provides the following: of the page (line 1). - Override the ``subtitle`` block to affect the ``<title>`` tag in the ``head`` of the page (line 3). +- Add the CSRF token to the form (line 13). + Without this line, attempts to edit the page would result in a ``400 Bad Request`` error. - A 10-row by 60-column ``textarea`` field named ``body`` that is filled with - any existing page data when it is rendered (line 14). -- A submit button that has the name ``form.submitted`` (line 17). + any existing page data when it is rendered (line 15). +- A submit button (line 18). - The form POSTs back to the ``save_url`` argument supplied by the view (line - 12). The view will use the ``body`` and ``form.submitted`` values. + 12). The view will use the ``body`` value. The ``404.jinja2`` template @@ -409,16 +433,16 @@ The ``404.jinja2`` template Replace ``tutorial/templates/404.jinja2`` with the following content: .. literalinclude:: src/views/tutorial/templates/404.jinja2 - :linenos: - :language: html + :linenos: + :language: html This template is linked from the ``notfound_view`` defined in ``tutorial/views/notfound.py`` as shown here: .. literalinclude:: src/views/tutorial/views/notfound.py - :linenos: - :emphasize-lines: 6 - :language: python + :linenos: + :emphasize-lines: 6 + :language: python There are several important things to note about this configuration: diff --git a/docs/tutorials/wiki2/installation.rst b/docs/tutorials/wiki2/installation.rst index 55fca15a1..b144fc4e0 100644 --- a/docs/tutorials/wiki2/installation.rst +++ b/docs/tutorials/wiki2/installation.rst @@ -182,8 +182,8 @@ The console will show ``pip`` checking for packages and installing missing packa alembic-1.3.2 attrs-19.3.0 beautifulsoup4-4.8.2 coverage-5.0.1 \ hupper-1.9.1 importlib-metadata-1.3.0 more-itertools-8.0.2 packaging-19.2 \ plaster-1.0 plaster-pastedeploy-0.7 pluggy-0.13.1 py-1.8.1 \ - pyparsing-2.4.6 pyramid-1.10.4 pyramid-debugtoolbar-4.5.1 \ - pyramid-jinja2-2.8 pyramid-mako-1.1.0 pyramid-retry-2.1 pyramid-tm-2.3 \ + pyparsing-2.4.6 pyramid-1.10.4 pyramid-debugtoolbar-4.5.2 \ + pyramid-jinja2-2.8 pyramid-mako-1.1.0 pyramid-retry-2.1 pyramid-tm-2.4 \ pytest-5.3.2 pytest-cov-2.8.1 python-dateutil-2.8.1 python-editor-1.0.4 \ repoze.lru-0.7 six-1.13.0 soupsieve-1.9.5 transaction-3.0.0 \ translationstring-1.3 tutorial venusian-3.0.0 waitress-1.4.1 \ @@ -350,30 +350,33 @@ If successful, you will see output something like this: ======================== test session starts ======================== platform -- Python 3.7.3, pytest-5.3.2, py-1.8.1, pluggy-0.13.1 - rootdir: <somepath>/tutorial, inifile: pytest.ini, testpaths: tutorial + rootdir: <somepath>/tutorial, inifile: pytest.ini, testpaths: tutorial, tests plugins: cov-2.8.1 - collected 2 items - - tutorial/tests.py .. - - ------------------ coverage: platform Python 3.7.3 ------------------ - Name Stmts Miss Cover Missing - ----------------------------------------------------------------- - tutorial/__init__.py 8 6 25% 7-12 - tutorial/models/__init__.py 24 0 100% - tutorial/models/meta.py 5 0 100% - tutorial/models/mymodel.py 8 0 100% - tutorial/pshell.py 7 7 0% 1-13 - tutorial/routes.py 3 3 0% 1-3 - tutorial/scripts/__init__.py 0 0 100% - tutorial/scripts/initialize_db.py 22 22 0% 1-38 - tutorial/views/__init__.py 0 0 100% - tutorial/views/default.py 12 0 100% - tutorial/views/notfound.py 4 4 0% 1-7 - ----------------------------------------------------------------- - TOTAL 93 42 55% - - ===================== 2 passed in 0.64 seconds ====================== + collected 5 items + + tests/test_functional.py .. + tests/test_views.py ... + + ---------- coverage: platform darwin, python 3.7.4-final-0 ----------- + Name Stmts Miss Cover Missing + ---------------------------------------------------------------------------------- + tutorial/__init__.py 8 0 100% + tutorial/alembic/env.py 23 4 83% 28-30, 56 + tutorial/alembic/versions/20200106_8c274fe5f3c4.py 12 2 83% 31-32 + tutorial/models/__init__.py 32 2 94% 71, 82 + tutorial/models/meta.py 5 0 100% + tutorial/models/mymodel.py 8 0 100% + tutorial/pshell.py 7 5 29% 5-13 + tutorial/routes.py 3 0 100% + tutorial/scripts/__init__.py 0 0 100% + tutorial/scripts/initialize_db.py 22 14 36% 15-16, 20-25, 29-38 + tutorial/views/__init__.py 0 0 100% + tutorial/views/default.py 12 0 100% + tutorial/views/notfound.py 4 0 100% + ---------------------------------------------------------------------------------- + TOTAL 136 27 80% + + ===================== 5 passed in 0.77 seconds ====================== Our package doesn't quite have 100% test coverage. diff --git a/docs/tutorials/wiki2/src/authentication/.gitignore b/docs/tutorials/wiki2/src/authentication/.gitignore index 1853d983c..c612e59f2 100644 --- a/docs/tutorials/wiki2/src/authentication/.gitignore +++ b/docs/tutorials/wiki2/src/authentication/.gitignore @@ -19,3 +19,4 @@ Data.fs* .DS_Store coverage test +*.sqlite diff --git a/docs/tutorials/wiki2/src/authentication/setup.py b/docs/tutorials/wiki2/src/authentication/setup.py index 500c5e599..12eabaff2 100644 --- a/docs/tutorials/wiki2/src/authentication/setup.py +++ b/docs/tutorials/wiki2/src/authentication/setup.py @@ -20,8 +20,8 @@ requires = [ 'pyramid_tm', 'SQLAlchemy', 'transaction', - 'zope.sqlalchemy', 'waitress', + 'zope.sqlalchemy', ] tests_require = [ diff --git a/docs/tutorials/wiki2/src/authentication/testing.ini b/docs/tutorials/wiki2/src/authentication/testing.ini new file mode 100644 index 000000000..d3c601f16 --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/testing.ini @@ -0,0 +1,81 @@ +### +# app configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/environment.html +### + +[app:main] +use = egg:tutorial + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en + +sqlalchemy.url = sqlite:///%(here)s/testing.sqlite + +retry.attempts = 3 + +auth.secret = test-seekrit + +[pshell] +setup = tutorial.pshell.setup + +### +# wsgi server configuration +### + +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + +[server:main] +use = egg:waitress#main +listen = localhost:6543 + +### +# logging configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html +### + +[loggers] +keys = root, tutorial, sqlalchemy, alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_tutorial] +level = DEBUG +handlers = +qualname = tutorial + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[logger_alembic] +level = WARN +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s diff --git a/docs/tutorials/wiki2/src/authentication/tests/conftest.py b/docs/tutorials/wiki2/src/authentication/tests/conftest.py new file mode 100644 index 000000000..2db65f887 --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/tests/conftest.py @@ -0,0 +1,125 @@ +import alembic +import alembic.config +import alembic.command +import os +from pyramid.paster import get_appsettings +from pyramid.scripting import prepare +from pyramid.testing import DummyRequest +import pytest +import transaction +from webob.cookies import Cookie +import webtest + +from tutorial import main +from tutorial import models +from tutorial.models.meta import Base + + +def pytest_addoption(parser): + parser.addoption('--ini', action='store', metavar='INI_FILE') + +@pytest.fixture(scope='session') +def ini_file(request): + # potentially grab this path from a pytest option + return os.path.abspath(request.config.option.ini or 'testing.ini') + +@pytest.fixture(scope='session') +def app_settings(ini_file): + return get_appsettings(ini_file) + +@pytest.fixture(scope='session') +def dbengine(app_settings, ini_file): + engine = models.get_engine(app_settings) + + alembic_cfg = alembic.config.Config(ini_file) + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + + # run migrations to initialize the database + # depending on how we want to initialize the database from scratch + # we could alternatively call: + # Base.metadata.create_all(bind=engine) + # alembic.command.stamp(alembic_cfg, "head") + alembic.command.upgrade(alembic_cfg, "head") + + yield engine + + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + +@pytest.fixture(scope='session') +def app(app_settings, dbengine): + return main({}, dbengine=dbengine, **app_settings) + +@pytest.fixture +def tm(): + tm = transaction.TransactionManager(explicit=True) + tm.begin() + tm.doom() + + yield tm + + tm.abort() + +@pytest.fixture +def dbsession(app, tm): + session_factory = app.registry['dbsession_factory'] + return models.get_tm_session(session_factory, tm) + +@pytest.fixture +def testapp(app, tm, dbsession): + # override request.dbsession and request.tm with our own + # externally-controlled values that are shared across requests but aborted + # at the end + testapp = webtest.TestApp(app, extra_environ={ + 'HTTP_HOST': 'example.com', + 'tm.active': True, + 'tm.manager': tm, + 'app.dbsession': dbsession, + }) + + return testapp + +@pytest.fixture +def app_request(app, tm, dbsession): + """ + A real request. + + This request is almost identical to a real request but it has some + drawbacks in tests as it's harder to mock data and is heavier. + + """ + env = prepare(registry=app.registry) + request = env['request'] + request.host = 'example.com' + + # without this, request.dbsession will be joined to the same transaction + # manager but it will be using a different sqlalchemy.orm.Session using + # a separate database transaction + request.dbsession = dbsession + request.tm = tm + + yield request + env['closer']() + +@pytest.fixture +def dummy_request(app, tm, dbsession): + """ + A lightweight dummy request. + + This request is ultra-lightweight and should be used only when the + request itself is not a large focus in the call-stack. + + It is way easier to mock and control side-effects using this object. + + - It does not have request extensions applied. + - Threadlocals are not properly pushed. + + """ + request = DummyRequest() + request.registry = app.registry + request.host = 'example.com' + request.dbsession = dbsession + request.tm = tm + + return request diff --git a/docs/tutorials/wiki2/src/authentication/tests/test_functional.py b/docs/tutorials/wiki2/src/authentication/tests/test_functional.py new file mode 100644 index 000000000..dbcd8aec7 --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/tests/test_functional.py @@ -0,0 +1,13 @@ +from tutorial import models + +def test_my_view_success(testapp, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + res = testapp.get('/', status=200) + assert res.body + +def test_notfound(testapp): + res = testapp.get('/badurl', status=404) + assert res.status_code == 404 diff --git a/docs/tutorials/wiki2/src/authentication/tests/test_it.py b/docs/tutorials/wiki2/src/authentication/tests/test_it.py deleted file mode 100644 index ea16534fc..000000000 --- a/docs/tutorials/wiki2/src/authentication/tests/test_it.py +++ /dev/null @@ -1,66 +0,0 @@ -import unittest - -from pyramid import testing - -import transaction - - -def dummy_request(dbsession): - return testing.DummyRequest(dbsession=dbsession) - - -class BaseTest(unittest.TestCase): - def setUp(self): - self.config = testing.setUp(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) - self.config.include('tutorial.models') - settings = self.config.get_settings() - - from tutorial.models import ( - get_engine, - get_session_factory, - get_tm_session, - ) - - self.engine = get_engine(settings) - session_factory = get_session_factory(self.engine) - - self.session = get_tm_session(session_factory, transaction.manager) - - def init_database(self): - from tutorial.models.meta import Base - Base.metadata.create_all(self.engine) - - def tearDown(self): - from tutorial.models.meta import Base - - testing.tearDown() - transaction.abort() - Base.metadata.drop_all(self.engine) - - -class TestMyViewSuccessCondition(BaseTest): - - def setUp(self): - super(TestMyViewSuccessCondition, self).setUp() - self.init_database() - - from tutorial.models import MyModel - - model = MyModel(name='one', value=55) - self.session.add(model) - - def test_passing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info['one'].name, 'one') - self.assertEqual(info['project'], 'myproj') - - -class TestMyViewFailureCondition(BaseTest): - - def test_failing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info.status_int, 500) diff --git a/docs/tutorials/wiki2/src/authentication/tests/test_views.py b/docs/tutorials/wiki2/src/authentication/tests/test_views.py new file mode 100644 index 000000000..8ae464d03 --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/tests/test_views.py @@ -0,0 +1,23 @@ +from tutorial import models +from tutorial.views.default import my_view +from tutorial.views.notfound import notfound_view + + +def test_my_view_failure(app_request): + info = my_view(app_request) + assert info.status_int == 500 + +def test_my_view_success(app_request, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + info = my_view(app_request) + assert app_request.response.status_int == 200 + assert info['one'].name == 'one' + assert info['project'] == 'myproj' + +def test_notfound_view(app_request): + info = notfound_view(app_request) + assert app_request.response.status_int == 404 + assert info == {} diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/__init__.py b/docs/tutorials/wiki2/src/authentication/tutorial/__init__.py index ce2e9f12a..81a22c68c 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/__init__.py +++ b/docs/tutorials/wiki2/src/authentication/tutorial/__init__.py @@ -5,9 +5,9 @@ def main(global_config, **settings): """ This function returns a Pyramid WSGI application. """ with Configurator(settings=settings) as config: - config.include('.models') config.include('pyramid_jinja2') - config.include('.routes') config.include('.security') + config.include('.routes') + config.include('.models') config.scan() return config.make_wsgi_app() diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/models/__init__.py b/docs/tutorials/wiki2/src/authentication/tutorial/models/__init__.py index a4209a6e9..47d77ef01 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/models/__init__.py +++ b/docs/tutorials/wiki2/src/authentication/tutorial/models/__init__.py @@ -66,13 +66,21 @@ def includeme(config): # use pyramid_retry to retry a request when transient exceptions occur config.include('pyramid_retry') - session_factory = get_session_factory(get_engine(settings)) + # hook to share the dbengine fixture in testing + dbengine = settings.get('dbengine') + if not dbengine: + dbengine = get_engine(settings) + + session_factory = get_session_factory(dbengine) config.registry['dbsession_factory'] = session_factory # make request.dbsession available for use in Pyramid - config.add_request_method( - # r.tm is the transaction manager used by pyramid_tm - lambda r: get_tm_session(session_factory, r.tm), - 'dbsession', - reify=True - ) + def dbsession(request): + # hook to share the dbsession fixture in testing + dbsession = request.environ.get('app.dbsession') + if dbsession is None: + # request.tm is the transaction manager used by pyramid_tm + dbsession = get_tm_session(session_factory, request.tm) + return dbsession + + config.add_request_method(dbsession, reify=True) diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initialize_db.py b/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initialize_db.py index e6350fb36..c8034e5a5 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initialize_db.py +++ b/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initialize_db.py @@ -8,6 +8,10 @@ from .. import models def setup_models(dbsession): + """ + Add or update models / fixtures in the database. + + """ editor = models.User(name='editor', role='editor') editor.set_password('editor') dbsession.add(editor) diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/security.py b/docs/tutorials/wiki2/src/authentication/tutorial/security.py index 8ea3858d2..48149d6e5 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/security.py +++ b/docs/tutorials/wiki2/src/authentication/tutorial/security.py @@ -1,27 +1,39 @@ -from pyramid.authentication import AuthTktAuthenticationPolicy -from pyramid.authorization import ACLAuthorizationPolicy +from pyramid.authentication import AuthTktCookieHelper +from pyramid.csrf import CookieCSRFStoragePolicy -from .models import User +from . import models -class MyAuthenticationPolicy(AuthTktAuthenticationPolicy): +class MySecurityPolicy: + def __init__(self, secret): + self.authtkt = AuthTktCookieHelper(secret) + + def authenticated_identity(self, request): + identity = self.authtkt.identify(request) + if identity is None: + return None + + userid = identity['userid'] + user = request.dbsession.query(models.User).get(userid) + return user + def authenticated_userid(self, request): - user = request.user + user = self.authenticated_identity(request) if user is not None: return user.id -def get_user(request): - user_id = request.unauthenticated_userid - if user_id is not None: - user = request.dbsession.query(User).get(user_id) - return user + def remember(self, request, userid, **kw): + return self.authtkt.remember(request, userid, **kw) + + def forget(self, request, **kw): + return self.authtkt.forget(request, **kw) def includeme(config): settings = config.get_settings() - authn_policy = MyAuthenticationPolicy( - settings['auth.secret'], - hashalg='sha512', - ) - config.set_authentication_policy(authn_policy) - config.set_authorization_policy(ACLAuthorizationPolicy()) - config.add_request_method(get_user, 'user', reify=True) + + config.set_csrf_storage_policy(CookieCSRFStoragePolicy()) + config.set_default_csrf_options(require_csrf=True) + + config.set_security_policy(MySecurityPolicy(settings['auth.secret'])) + config.add_request_method( + lambda request: request.authenticated_identity, 'user', property=True) diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/templates/403.jinja2 b/docs/tutorials/wiki2/src/authentication/tutorial/templates/403.jinja2 new file mode 100644 index 000000000..7a6f523bc --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/tutorial/templates/403.jinja2 @@ -0,0 +1,6 @@ +{% extends "layout.jinja2" %} + +{% block content %} +<h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> +<p class="lead"><span class="font-semi-bold">403</span> Forbidden</p> +{% endblock content %} diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/templates/404.jinja2 b/docs/tutorials/wiki2/src/authentication/tutorial/templates/404.jinja2 index aaf12413f..5edb15285 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/templates/404.jinja2 +++ b/docs/tutorials/wiki2/src/authentication/tutorial/templates/404.jinja2 @@ -1,8 +1,6 @@ {% extends "layout.jinja2" %} {% block content %} -<div class="content"> - <h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> - <p class="lead"><span class="font-semi-bold">404</span> Page Not Found</p> -</div> +<h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> +<p class="lead"><span class="font-semi-bold">404</span> Page Not Found</p> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/templates/edit.jinja2 b/docs/tutorials/wiki2/src/authentication/tutorial/templates/edit.jinja2 index 7db25c674..27b545054 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/templates/edit.jinja2 +++ b/docs/tutorials/wiki2/src/authentication/tutorial/templates/edit.jinja2 @@ -10,11 +10,12 @@ Editing <strong>{{pagename}}</strong> <a href="{{request.route_url('view_page', pagename='FrontPage')}}">FrontPage</a>. </p> <form action="{{ save_url }}" method="post"> +<input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> <div class="form-group"> <textarea class="form-control" name="body" rows="10" cols="60">{{ pagedata }}</textarea> </div> <div class="form-group"> - <button type="submit" name="form.submitted" value="Save" class="btn btn-default">Save</button> + <button type="submit" class="btn btn-default">Save</button> </div> </form> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/templates/layout.jinja2 b/docs/tutorials/wiki2/src/authentication/tutorial/templates/layout.jinja2 index 4016b26c9..64a1db0c5 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/templates/layout.jinja2 +++ b/docs/tutorials/wiki2/src/authentication/tutorial/templates/layout.jinja2 @@ -35,18 +35,29 @@ <div class="content"> {% if request.user is none %} <p class="pull-right"> - <a href="{{ request.route_url('login') }}">Login</a> + <a href="{{ request.route_url('login') }}">Login</a> </p> {% else %} - <p class="pull-right"> - {{request.user.name}} <a href="{{request.route_url('logout')}}">Logout</a> - </p> + <form class="pull-right" action="{{ request.route_url('logout') }}" method="post"> + {{request.user.name}} + <input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> + <button class="btn btn-link" type="submit">Logout</button> + </form> {% endif %} {% block content %}{% endblock %} </div> </div> </div> <div class="row"> + <div class="links"> + <ul> + <li><i class="glyphicon glyphicon-cog icon-muted"></i><a href="https://github.com/Pylons/pyramid">Github Project</a></li> + <li><i class="glyphicon glyphicon-globe icon-muted"></i><a href="https://webchat.freenode.net/?channels=pyramid">IRC Channel</a></li> + <li><i class="glyphicon glyphicon-home icon-muted"></i><a href="https://pylonsproject.org">Pylons Project</a></li> + </ul> + </div> + </div> + <div class="row"> <div class="copyright"> Copyright © Pylons Project </div> diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/templates/login.jinja2 b/docs/tutorials/wiki2/src/authentication/tutorial/templates/login.jinja2 index 1806de0ff..058b7254b 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/templates/login.jinja2 +++ b/docs/tutorials/wiki2/src/authentication/tutorial/templates/login.jinja2 @@ -10,6 +10,7 @@ {{ message }} </p> <form action="{{ url }}" method="post"> +<input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> <input type="hidden" name="next" value="{{ next_url }}"> <div class="form-group"> <label for="login">Username</label> @@ -20,7 +21,7 @@ <input type="password" name="password"> </div> <div class="form-group"> - <button type="submit" name="form.submitted" value="Log In" class="btn btn-default">Log In</button> + <button type="submit" class="btn btn-default">Log In</button> </div> </form> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/views/auth.py b/docs/tutorials/wiki2/src/authentication/tutorial/views/auth.py index 16fa616e5..e1a564415 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/views/auth.py +++ b/docs/tutorials/wiki2/src/authentication/tutorial/views/auth.py @@ -1,14 +1,15 @@ -from pyramid.httpexceptions import HTTPFound +from pyramid.csrf import new_csrf_token +from pyramid.httpexceptions import HTTPSeeOther from pyramid.security import ( remember, forget, - ) +) from pyramid.view import ( forbidden_view_config, view_config, ) -from ..models import User +from .. import models @view_config(route_name='login', renderer='tutorial:templates/login.jinja2') @@ -18,29 +19,43 @@ def login(request): next_url = request.route_url('view_wiki') message = '' login = '' - if 'form.submitted' in request.params: + if request.method == 'POST': login = request.params['login'] password = request.params['password'] - user = request.dbsession.query(User).filter_by(name=login).first() + user = ( + request.dbsession.query(models.User) + .filter_by(name=login) + .first() + ) if user is not None and user.check_password(password): + new_csrf_token(request) headers = remember(request, user.id) - return HTTPFound(location=next_url, headers=headers) + return HTTPSeeOther(location=next_url, headers=headers) message = 'Failed login' + request.response.status = 400 return dict( message=message, url=request.route_url('login'), next_url=next_url, login=login, - ) + ) @view_config(route_name='logout') def logout(request): - headers = forget(request) next_url = request.route_url('view_wiki') - return HTTPFound(location=next_url, headers=headers) + if request.method == 'POST': + new_csrf_token(request) + headers = forget(request) + return HTTPSeeOther(location=next_url, headers=headers) + + return HTTPSeeOther(location=next_url) + +@forbidden_view_config(renderer='tutorial:templates/403.jinja2') +def forbidden_view(exc, request): + if request.user is None: + next_url = request.route_url('login', _query={'next': request.url}) + return HTTPSeeOther(location=next_url) -@forbidden_view_config() -def forbidden_view(request): - next_url = request.route_url('login', _query={'next': request.url}) - return HTTPFound(location=next_url) + request.response.status = 403 + return {} diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/views/default.py b/docs/tutorials/wiki2/src/authentication/tutorial/views/default.py index d1c429950..378ce0ae9 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/views/default.py +++ b/docs/tutorials/wiki2/src/authentication/tutorial/views/default.py @@ -1,24 +1,23 @@ -from html import escape -import re from docutils.core import publish_parts - +from html import escape from pyramid.httpexceptions import ( HTTPForbidden, - HTTPFound, HTTPNotFound, - ) - + HTTPSeeOther, +) from pyramid.view import view_config +import re from .. import models + # regular expression used to find WikiWords wikiwords = re.compile(r"\b([A-Z]\w+[A-Z]+\w+)") @view_config(route_name='view_wiki') def view_wiki(request): next_url = request.route_url('view_page', pagename='FrontPage') - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) @view_config(route_name='view_page', renderer='tutorial:templates/view.jinja2') def view_page(request): @@ -49,15 +48,15 @@ def edit_page(request): user = request.user if user is None or (user.role != 'editor' and page.creator != user): raise HTTPForbidden - if 'form.submitted' in request.params: + if request.method == 'POST': page.data = request.params['body'] next_url = request.route_url('view_page', pagename=page.name) - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) return dict( pagename=page.name, pagedata=page.data, save_url=request.route_url('edit_page', pagename=page.name), - ) + ) @view_config(route_name='add_page', renderer='tutorial:templates/edit.jinja2') def add_page(request): @@ -67,13 +66,13 @@ def add_page(request): pagename = request.matchdict['pagename'] if request.dbsession.query(models.Page).filter_by(name=pagename).count() > 0: next_url = request.route_url('edit_page', pagename=pagename) - return HTTPFound(location=next_url) - if 'form.submitted' in request.params: + return HTTPSeeOther(location=next_url) + if request.method == 'POST': body = request.params['body'] page = models.Page(name=pagename, data=body) page.creator = request.user request.dbsession.add(page) next_url = request.route_url('view_page', pagename=pagename) - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) save_url = request.route_url('add_page', pagename=pagename) return dict(pagename=pagename, pagedata='', save_url=save_url) diff --git a/docs/tutorials/wiki2/src/authorization/.gitignore b/docs/tutorials/wiki2/src/authorization/.gitignore index 1853d983c..c612e59f2 100644 --- a/docs/tutorials/wiki2/src/authorization/.gitignore +++ b/docs/tutorials/wiki2/src/authorization/.gitignore @@ -19,3 +19,4 @@ Data.fs* .DS_Store coverage test +*.sqlite diff --git a/docs/tutorials/wiki2/src/authorization/setup.py b/docs/tutorials/wiki2/src/authorization/setup.py index 500c5e599..12eabaff2 100644 --- a/docs/tutorials/wiki2/src/authorization/setup.py +++ b/docs/tutorials/wiki2/src/authorization/setup.py @@ -20,8 +20,8 @@ requires = [ 'pyramid_tm', 'SQLAlchemy', 'transaction', - 'zope.sqlalchemy', 'waitress', + 'zope.sqlalchemy', ] tests_require = [ diff --git a/docs/tutorials/wiki2/src/authorization/testing.ini b/docs/tutorials/wiki2/src/authorization/testing.ini new file mode 100644 index 000000000..d3c601f16 --- /dev/null +++ b/docs/tutorials/wiki2/src/authorization/testing.ini @@ -0,0 +1,81 @@ +### +# app configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/environment.html +### + +[app:main] +use = egg:tutorial + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en + +sqlalchemy.url = sqlite:///%(here)s/testing.sqlite + +retry.attempts = 3 + +auth.secret = test-seekrit + +[pshell] +setup = tutorial.pshell.setup + +### +# wsgi server configuration +### + +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + +[server:main] +use = egg:waitress#main +listen = localhost:6543 + +### +# logging configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html +### + +[loggers] +keys = root, tutorial, sqlalchemy, alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_tutorial] +level = DEBUG +handlers = +qualname = tutorial + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[logger_alembic] +level = WARN +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s diff --git a/docs/tutorials/wiki2/src/authorization/tests/conftest.py b/docs/tutorials/wiki2/src/authorization/tests/conftest.py new file mode 100644 index 000000000..2db65f887 --- /dev/null +++ b/docs/tutorials/wiki2/src/authorization/tests/conftest.py @@ -0,0 +1,125 @@ +import alembic +import alembic.config +import alembic.command +import os +from pyramid.paster import get_appsettings +from pyramid.scripting import prepare +from pyramid.testing import DummyRequest +import pytest +import transaction +from webob.cookies import Cookie +import webtest + +from tutorial import main +from tutorial import models +from tutorial.models.meta import Base + + +def pytest_addoption(parser): + parser.addoption('--ini', action='store', metavar='INI_FILE') + +@pytest.fixture(scope='session') +def ini_file(request): + # potentially grab this path from a pytest option + return os.path.abspath(request.config.option.ini or 'testing.ini') + +@pytest.fixture(scope='session') +def app_settings(ini_file): + return get_appsettings(ini_file) + +@pytest.fixture(scope='session') +def dbengine(app_settings, ini_file): + engine = models.get_engine(app_settings) + + alembic_cfg = alembic.config.Config(ini_file) + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + + # run migrations to initialize the database + # depending on how we want to initialize the database from scratch + # we could alternatively call: + # Base.metadata.create_all(bind=engine) + # alembic.command.stamp(alembic_cfg, "head") + alembic.command.upgrade(alembic_cfg, "head") + + yield engine + + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + +@pytest.fixture(scope='session') +def app(app_settings, dbengine): + return main({}, dbengine=dbengine, **app_settings) + +@pytest.fixture +def tm(): + tm = transaction.TransactionManager(explicit=True) + tm.begin() + tm.doom() + + yield tm + + tm.abort() + +@pytest.fixture +def dbsession(app, tm): + session_factory = app.registry['dbsession_factory'] + return models.get_tm_session(session_factory, tm) + +@pytest.fixture +def testapp(app, tm, dbsession): + # override request.dbsession and request.tm with our own + # externally-controlled values that are shared across requests but aborted + # at the end + testapp = webtest.TestApp(app, extra_environ={ + 'HTTP_HOST': 'example.com', + 'tm.active': True, + 'tm.manager': tm, + 'app.dbsession': dbsession, + }) + + return testapp + +@pytest.fixture +def app_request(app, tm, dbsession): + """ + A real request. + + This request is almost identical to a real request but it has some + drawbacks in tests as it's harder to mock data and is heavier. + + """ + env = prepare(registry=app.registry) + request = env['request'] + request.host = 'example.com' + + # without this, request.dbsession will be joined to the same transaction + # manager but it will be using a different sqlalchemy.orm.Session using + # a separate database transaction + request.dbsession = dbsession + request.tm = tm + + yield request + env['closer']() + +@pytest.fixture +def dummy_request(app, tm, dbsession): + """ + A lightweight dummy request. + + This request is ultra-lightweight and should be used only when the + request itself is not a large focus in the call-stack. + + It is way easier to mock and control side-effects using this object. + + - It does not have request extensions applied. + - Threadlocals are not properly pushed. + + """ + request = DummyRequest() + request.registry = app.registry + request.host = 'example.com' + request.dbsession = dbsession + request.tm = tm + + return request diff --git a/docs/tutorials/wiki2/src/authorization/tests/test_functional.py b/docs/tutorials/wiki2/src/authorization/tests/test_functional.py new file mode 100644 index 000000000..dbcd8aec7 --- /dev/null +++ b/docs/tutorials/wiki2/src/authorization/tests/test_functional.py @@ -0,0 +1,13 @@ +from tutorial import models + +def test_my_view_success(testapp, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + res = testapp.get('/', status=200) + assert res.body + +def test_notfound(testapp): + res = testapp.get('/badurl', status=404) + assert res.status_code == 404 diff --git a/docs/tutorials/wiki2/src/authorization/tests/test_it.py b/docs/tutorials/wiki2/src/authorization/tests/test_it.py deleted file mode 100644 index ea16534fc..000000000 --- a/docs/tutorials/wiki2/src/authorization/tests/test_it.py +++ /dev/null @@ -1,66 +0,0 @@ -import unittest - -from pyramid import testing - -import transaction - - -def dummy_request(dbsession): - return testing.DummyRequest(dbsession=dbsession) - - -class BaseTest(unittest.TestCase): - def setUp(self): - self.config = testing.setUp(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) - self.config.include('tutorial.models') - settings = self.config.get_settings() - - from tutorial.models import ( - get_engine, - get_session_factory, - get_tm_session, - ) - - self.engine = get_engine(settings) - session_factory = get_session_factory(self.engine) - - self.session = get_tm_session(session_factory, transaction.manager) - - def init_database(self): - from tutorial.models.meta import Base - Base.metadata.create_all(self.engine) - - def tearDown(self): - from tutorial.models.meta import Base - - testing.tearDown() - transaction.abort() - Base.metadata.drop_all(self.engine) - - -class TestMyViewSuccessCondition(BaseTest): - - def setUp(self): - super(TestMyViewSuccessCondition, self).setUp() - self.init_database() - - from tutorial.models import MyModel - - model = MyModel(name='one', value=55) - self.session.add(model) - - def test_passing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info['one'].name, 'one') - self.assertEqual(info['project'], 'myproj') - - -class TestMyViewFailureCondition(BaseTest): - - def test_failing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info.status_int, 500) diff --git a/docs/tutorials/wiki2/src/authorization/tests/test_views.py b/docs/tutorials/wiki2/src/authorization/tests/test_views.py new file mode 100644 index 000000000..8ae464d03 --- /dev/null +++ b/docs/tutorials/wiki2/src/authorization/tests/test_views.py @@ -0,0 +1,23 @@ +from tutorial import models +from tutorial.views.default import my_view +from tutorial.views.notfound import notfound_view + + +def test_my_view_failure(app_request): + info = my_view(app_request) + assert info.status_int == 500 + +def test_my_view_success(app_request, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + info = my_view(app_request) + assert app_request.response.status_int == 200 + assert info['one'].name == 'one' + assert info['project'] == 'myproj' + +def test_notfound_view(app_request): + info = notfound_view(app_request) + assert app_request.response.status_int == 404 + assert info == {} diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/__init__.py b/docs/tutorials/wiki2/src/authorization/tutorial/__init__.py index ce2e9f12a..81a22c68c 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/__init__.py +++ b/docs/tutorials/wiki2/src/authorization/tutorial/__init__.py @@ -5,9 +5,9 @@ def main(global_config, **settings): """ This function returns a Pyramid WSGI application. """ with Configurator(settings=settings) as config: - config.include('.models') config.include('pyramid_jinja2') - config.include('.routes') config.include('.security') + config.include('.routes') + config.include('.models') config.scan() return config.make_wsgi_app() diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/models/__init__.py b/docs/tutorials/wiki2/src/authorization/tutorial/models/__init__.py index a4209a6e9..47d77ef01 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/models/__init__.py +++ b/docs/tutorials/wiki2/src/authorization/tutorial/models/__init__.py @@ -66,13 +66,21 @@ def includeme(config): # use pyramid_retry to retry a request when transient exceptions occur config.include('pyramid_retry') - session_factory = get_session_factory(get_engine(settings)) + # hook to share the dbengine fixture in testing + dbengine = settings.get('dbengine') + if not dbengine: + dbengine = get_engine(settings) + + session_factory = get_session_factory(dbengine) config.registry['dbsession_factory'] = session_factory # make request.dbsession available for use in Pyramid - config.add_request_method( - # r.tm is the transaction manager used by pyramid_tm - lambda r: get_tm_session(session_factory, r.tm), - 'dbsession', - reify=True - ) + def dbsession(request): + # hook to share the dbsession fixture in testing + dbsession = request.environ.get('app.dbsession') + if dbsession is None: + # request.tm is the transaction manager used by pyramid_tm + dbsession = get_tm_session(session_factory, request.tm) + return dbsession + + config.add_request_method(dbsession, reify=True) diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/routes.py b/docs/tutorials/wiki2/src/authorization/tutorial/routes.py index 1fd45a994..f016d7541 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/routes.py +++ b/docs/tutorials/wiki2/src/authorization/tutorial/routes.py @@ -1,6 +1,6 @@ from pyramid.httpexceptions import ( HTTPNotFound, - HTTPFound, + HTTPSeeOther, ) from pyramid.security import ( Allow, @@ -9,6 +9,7 @@ from pyramid.security import ( from . import models + def includeme(config): config.add_static_view('static', 'static', cache_max_age=3600) config.add_route('view_wiki', '/') @@ -24,7 +25,7 @@ def new_page_factory(request): pagename = request.matchdict['pagename'] if request.dbsession.query(models.Page).filter_by(name=pagename).count() > 0: next_url = request.route_url('edit_page', pagename=pagename) - raise HTTPFound(location=next_url) + raise HTTPSeeOther(location=next_url) return NewPage(pagename) class NewPage(object): @@ -52,5 +53,5 @@ class PageResource(object): return [ (Allow, Everyone, 'view'), (Allow, 'role:editor', 'edit'), - (Allow, str(self.page.creator_id), 'edit'), + (Allow, 'u:' + str(self.page.creator_id), 'edit'), ] diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/scripts/initialize_db.py b/docs/tutorials/wiki2/src/authorization/tutorial/scripts/initialize_db.py index e6350fb36..c8034e5a5 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/scripts/initialize_db.py +++ b/docs/tutorials/wiki2/src/authorization/tutorial/scripts/initialize_db.py @@ -8,6 +8,10 @@ from .. import models def setup_models(dbsession): + """ + Add or update models / fixtures in the database. + + """ editor = models.User(name='editor', role='editor') editor.set_password('editor') dbsession.add(editor) diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/security.py b/docs/tutorials/wiki2/src/authorization/tutorial/security.py index 1ce1c8753..448183c95 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/security.py +++ b/docs/tutorials/wiki2/src/authorization/tutorial/security.py @@ -1,5 +1,6 @@ -from pyramid.authentication import AuthTktAuthenticationPolicy -from pyramid.authorization import ACLAuthorizationPolicy +from pyramid.authentication import AuthTktCookieHelper +from pyramid.authorization import ACLHelper +from pyramid.csrf import CookieCSRFStoragePolicy from pyramid.security import ( Authenticated, Everyone, @@ -8,33 +9,50 @@ from pyramid.security import ( from . import models -class MyAuthenticationPolicy(AuthTktAuthenticationPolicy): +class MySecurityPolicy: + def __init__(self, secret): + self.authtkt = AuthTktCookieHelper(secret) + self.acl = ACLHelper() + + def authenticated_identity(self, request): + identity = self.authtkt.identify(request) + if identity is None: + return None + + userid = identity['userid'] + user = request.dbsession.query(models.User).get(userid) + return user + def authenticated_userid(self, request): - user = request.user + user = self.authenticated_identity(request) if user is not None: return user.id + def remember(self, request, userid, **kw): + return self.authtkt.remember(request, userid, **kw) + + def forget(self, request, **kw): + return self.authtkt.forget(request, **kw) + + def permits(self, request, context, permission): + principals = self.effective_principals(request) + return self.acl.permits(context, principals, permission) + def effective_principals(self, request): principals = [Everyone] - user = request.user + user = self.authenticated_identity(request) if user is not None: principals.append(Authenticated) - principals.append(str(user.id)) + principals.append('u:' + str(user.id)) principals.append('role:' + user.role) return principals -def get_user(request): - user_id = request.unauthenticated_userid - if user_id is not None: - user = request.dbsession.query(models.User).get(user_id) - return user - def includeme(config): settings = config.get_settings() - authn_policy = MyAuthenticationPolicy( - settings['auth.secret'], - hashalg='sha512', - ) - config.set_authentication_policy(authn_policy) - config.set_authorization_policy(ACLAuthorizationPolicy()) - config.add_request_method(get_user, 'user', reify=True) + + config.set_csrf_storage_policy(CookieCSRFStoragePolicy()) + config.set_default_csrf_options(require_csrf=True) + + config.set_security_policy(MySecurityPolicy(settings['auth.secret'])) + config.add_request_method( + lambda request: request.authenticated_identity, 'user', property=True) diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/templates/403.jinja2 b/docs/tutorials/wiki2/src/authorization/tutorial/templates/403.jinja2 new file mode 100644 index 000000000..7a6f523bc --- /dev/null +++ b/docs/tutorials/wiki2/src/authorization/tutorial/templates/403.jinja2 @@ -0,0 +1,6 @@ +{% extends "layout.jinja2" %} + +{% block content %} +<h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> +<p class="lead"><span class="font-semi-bold">403</span> Forbidden</p> +{% endblock content %} diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/templates/404.jinja2 b/docs/tutorials/wiki2/src/authorization/tutorial/templates/404.jinja2 index aaf12413f..5edb15285 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/templates/404.jinja2 +++ b/docs/tutorials/wiki2/src/authorization/tutorial/templates/404.jinja2 @@ -1,8 +1,6 @@ {% extends "layout.jinja2" %} {% block content %} -<div class="content"> - <h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> - <p class="lead"><span class="font-semi-bold">404</span> Page Not Found</p> -</div> +<h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> +<p class="lead"><span class="font-semi-bold">404</span> Page Not Found</p> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/templates/edit.jinja2 b/docs/tutorials/wiki2/src/authorization/tutorial/templates/edit.jinja2 index 7db25c674..27b545054 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/templates/edit.jinja2 +++ b/docs/tutorials/wiki2/src/authorization/tutorial/templates/edit.jinja2 @@ -10,11 +10,12 @@ Editing <strong>{{pagename}}</strong> <a href="{{request.route_url('view_page', pagename='FrontPage')}}">FrontPage</a>. </p> <form action="{{ save_url }}" method="post"> +<input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> <div class="form-group"> <textarea class="form-control" name="body" rows="10" cols="60">{{ pagedata }}</textarea> </div> <div class="form-group"> - <button type="submit" name="form.submitted" value="Save" class="btn btn-default">Save</button> + <button type="submit" class="btn btn-default">Save</button> </div> </form> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/templates/layout.jinja2 b/docs/tutorials/wiki2/src/authorization/tutorial/templates/layout.jinja2 index 4016b26c9..64a1db0c5 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/templates/layout.jinja2 +++ b/docs/tutorials/wiki2/src/authorization/tutorial/templates/layout.jinja2 @@ -35,18 +35,29 @@ <div class="content"> {% if request.user is none %} <p class="pull-right"> - <a href="{{ request.route_url('login') }}">Login</a> + <a href="{{ request.route_url('login') }}">Login</a> </p> {% else %} - <p class="pull-right"> - {{request.user.name}} <a href="{{request.route_url('logout')}}">Logout</a> - </p> + <form class="pull-right" action="{{ request.route_url('logout') }}" method="post"> + {{request.user.name}} + <input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> + <button class="btn btn-link" type="submit">Logout</button> + </form> {% endif %} {% block content %}{% endblock %} </div> </div> </div> <div class="row"> + <div class="links"> + <ul> + <li><i class="glyphicon glyphicon-cog icon-muted"></i><a href="https://github.com/Pylons/pyramid">Github Project</a></li> + <li><i class="glyphicon glyphicon-globe icon-muted"></i><a href="https://webchat.freenode.net/?channels=pyramid">IRC Channel</a></li> + <li><i class="glyphicon glyphicon-home icon-muted"></i><a href="https://pylonsproject.org">Pylons Project</a></li> + </ul> + </div> + </div> + <div class="row"> <div class="copyright"> Copyright © Pylons Project </div> diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/templates/login.jinja2 b/docs/tutorials/wiki2/src/authorization/tutorial/templates/login.jinja2 index 1806de0ff..058b7254b 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/templates/login.jinja2 +++ b/docs/tutorials/wiki2/src/authorization/tutorial/templates/login.jinja2 @@ -10,6 +10,7 @@ {{ message }} </p> <form action="{{ url }}" method="post"> +<input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> <input type="hidden" name="next" value="{{ next_url }}"> <div class="form-group"> <label for="login">Username</label> @@ -20,7 +21,7 @@ <input type="password" name="password"> </div> <div class="form-group"> - <button type="submit" name="form.submitted" value="Log In" class="btn btn-default">Log In</button> + <button type="submit" class="btn btn-default">Log In</button> </div> </form> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/views/auth.py b/docs/tutorials/wiki2/src/authorization/tutorial/views/auth.py index 16fa616e5..e1a564415 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/views/auth.py +++ b/docs/tutorials/wiki2/src/authorization/tutorial/views/auth.py @@ -1,14 +1,15 @@ -from pyramid.httpexceptions import HTTPFound +from pyramid.csrf import new_csrf_token +from pyramid.httpexceptions import HTTPSeeOther from pyramid.security import ( remember, forget, - ) +) from pyramid.view import ( forbidden_view_config, view_config, ) -from ..models import User +from .. import models @view_config(route_name='login', renderer='tutorial:templates/login.jinja2') @@ -18,29 +19,43 @@ def login(request): next_url = request.route_url('view_wiki') message = '' login = '' - if 'form.submitted' in request.params: + if request.method == 'POST': login = request.params['login'] password = request.params['password'] - user = request.dbsession.query(User).filter_by(name=login).first() + user = ( + request.dbsession.query(models.User) + .filter_by(name=login) + .first() + ) if user is not None and user.check_password(password): + new_csrf_token(request) headers = remember(request, user.id) - return HTTPFound(location=next_url, headers=headers) + return HTTPSeeOther(location=next_url, headers=headers) message = 'Failed login' + request.response.status = 400 return dict( message=message, url=request.route_url('login'), next_url=next_url, login=login, - ) + ) @view_config(route_name='logout') def logout(request): - headers = forget(request) next_url = request.route_url('view_wiki') - return HTTPFound(location=next_url, headers=headers) + if request.method == 'POST': + new_csrf_token(request) + headers = forget(request) + return HTTPSeeOther(location=next_url, headers=headers) + + return HTTPSeeOther(location=next_url) + +@forbidden_view_config(renderer='tutorial:templates/403.jinja2') +def forbidden_view(exc, request): + if request.user is None: + next_url = request.route_url('login', _query={'next': request.url}) + return HTTPSeeOther(location=next_url) -@forbidden_view_config() -def forbidden_view(request): - next_url = request.route_url('login', _query={'next': request.url}) - return HTTPFound(location=next_url) + request.response.status = 403 + return {} diff --git a/docs/tutorials/wiki2/src/authorization/tutorial/views/default.py b/docs/tutorials/wiki2/src/authorization/tutorial/views/default.py index de0bcd816..214788357 100644 --- a/docs/tutorials/wiki2/src/authorization/tutorial/views/default.py +++ b/docs/tutorials/wiki2/src/authorization/tutorial/views/default.py @@ -1,19 +1,19 @@ -from html import escape -import re from docutils.core import publish_parts - -from pyramid.httpexceptions import HTTPFound +from html import escape +from pyramid.httpexceptions import HTTPSeeOther from pyramid.view import view_config +import re from .. import models + # regular expression used to find WikiWords wikiwords = re.compile(r"\b([A-Z]\w+[A-Z]+\w+)") @view_config(route_name='view_wiki') def view_wiki(request): next_url = request.route_url('view_page', pagename='FrontPage') - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) @view_config(route_name='view_page', renderer='tutorial:templates/view.jinja2', permission='view') @@ -39,26 +39,26 @@ def view_page(request): permission='edit') def edit_page(request): page = request.context.page - if 'form.submitted' in request.params: + if request.method == 'POST': page.data = request.params['body'] next_url = request.route_url('view_page', pagename=page.name) - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) return dict( pagename=page.name, pagedata=page.data, save_url=request.route_url('edit_page', pagename=page.name), - ) + ) @view_config(route_name='add_page', renderer='tutorial:templates/edit.jinja2', permission='create') def add_page(request): pagename = request.context.pagename - if 'form.submitted' in request.params: + if request.method == 'POST': body = request.params['body'] page = models.Page(name=pagename, data=body) page.creator = request.user request.dbsession.add(page) next_url = request.route_url('view_page', pagename=pagename) - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) save_url = request.route_url('add_page', pagename=pagename) return dict(pagename=pagename, pagedata='', save_url=save_url) diff --git a/docs/tutorials/wiki2/src/basiclayout/.gitignore b/docs/tutorials/wiki2/src/basiclayout/.gitignore index 1853d983c..c612e59f2 100644 --- a/docs/tutorials/wiki2/src/basiclayout/.gitignore +++ b/docs/tutorials/wiki2/src/basiclayout/.gitignore @@ -19,3 +19,4 @@ Data.fs* .DS_Store coverage test +*.sqlite diff --git a/docs/tutorials/wiki2/src/basiclayout/testing.ini b/docs/tutorials/wiki2/src/basiclayout/testing.ini new file mode 100644 index 000000000..5caa1a8dc --- /dev/null +++ b/docs/tutorials/wiki2/src/basiclayout/testing.ini @@ -0,0 +1,79 @@ +### +# app configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/environment.html +### + +[app:main] +use = egg:tutorial + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en + +sqlalchemy.url = sqlite:///%(here)s/testing.sqlite + +retry.attempts = 3 + +[pshell] +setup = tutorial.pshell.setup + +### +# wsgi server configuration +### + +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + +[server:main] +use = egg:waitress#main +listen = localhost:6543 + +### +# logging configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html +### + +[loggers] +keys = root, tutorial, sqlalchemy, alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_tutorial] +level = DEBUG +handlers = +qualname = tutorial + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[logger_alembic] +level = WARN +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s diff --git a/docs/tutorials/wiki2/src/basiclayout/tests/conftest.py b/docs/tutorials/wiki2/src/basiclayout/tests/conftest.py new file mode 100644 index 000000000..2db65f887 --- /dev/null +++ b/docs/tutorials/wiki2/src/basiclayout/tests/conftest.py @@ -0,0 +1,125 @@ +import alembic +import alembic.config +import alembic.command +import os +from pyramid.paster import get_appsettings +from pyramid.scripting import prepare +from pyramid.testing import DummyRequest +import pytest +import transaction +from webob.cookies import Cookie +import webtest + +from tutorial import main +from tutorial import models +from tutorial.models.meta import Base + + +def pytest_addoption(parser): + parser.addoption('--ini', action='store', metavar='INI_FILE') + +@pytest.fixture(scope='session') +def ini_file(request): + # potentially grab this path from a pytest option + return os.path.abspath(request.config.option.ini or 'testing.ini') + +@pytest.fixture(scope='session') +def app_settings(ini_file): + return get_appsettings(ini_file) + +@pytest.fixture(scope='session') +def dbengine(app_settings, ini_file): + engine = models.get_engine(app_settings) + + alembic_cfg = alembic.config.Config(ini_file) + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + + # run migrations to initialize the database + # depending on how we want to initialize the database from scratch + # we could alternatively call: + # Base.metadata.create_all(bind=engine) + # alembic.command.stamp(alembic_cfg, "head") + alembic.command.upgrade(alembic_cfg, "head") + + yield engine + + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + +@pytest.fixture(scope='session') +def app(app_settings, dbengine): + return main({}, dbengine=dbengine, **app_settings) + +@pytest.fixture +def tm(): + tm = transaction.TransactionManager(explicit=True) + tm.begin() + tm.doom() + + yield tm + + tm.abort() + +@pytest.fixture +def dbsession(app, tm): + session_factory = app.registry['dbsession_factory'] + return models.get_tm_session(session_factory, tm) + +@pytest.fixture +def testapp(app, tm, dbsession): + # override request.dbsession and request.tm with our own + # externally-controlled values that are shared across requests but aborted + # at the end + testapp = webtest.TestApp(app, extra_environ={ + 'HTTP_HOST': 'example.com', + 'tm.active': True, + 'tm.manager': tm, + 'app.dbsession': dbsession, + }) + + return testapp + +@pytest.fixture +def app_request(app, tm, dbsession): + """ + A real request. + + This request is almost identical to a real request but it has some + drawbacks in tests as it's harder to mock data and is heavier. + + """ + env = prepare(registry=app.registry) + request = env['request'] + request.host = 'example.com' + + # without this, request.dbsession will be joined to the same transaction + # manager but it will be using a different sqlalchemy.orm.Session using + # a separate database transaction + request.dbsession = dbsession + request.tm = tm + + yield request + env['closer']() + +@pytest.fixture +def dummy_request(app, tm, dbsession): + """ + A lightweight dummy request. + + This request is ultra-lightweight and should be used only when the + request itself is not a large focus in the call-stack. + + It is way easier to mock and control side-effects using this object. + + - It does not have request extensions applied. + - Threadlocals are not properly pushed. + + """ + request = DummyRequest() + request.registry = app.registry + request.host = 'example.com' + request.dbsession = dbsession + request.tm = tm + + return request diff --git a/docs/tutorials/wiki2/src/basiclayout/tests/test_functional.py b/docs/tutorials/wiki2/src/basiclayout/tests/test_functional.py new file mode 100644 index 000000000..dbcd8aec7 --- /dev/null +++ b/docs/tutorials/wiki2/src/basiclayout/tests/test_functional.py @@ -0,0 +1,13 @@ +from tutorial import models + +def test_my_view_success(testapp, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + res = testapp.get('/', status=200) + assert res.body + +def test_notfound(testapp): + res = testapp.get('/badurl', status=404) + assert res.status_code == 404 diff --git a/docs/tutorials/wiki2/src/basiclayout/tests/test_it.py b/docs/tutorials/wiki2/src/basiclayout/tests/test_it.py deleted file mode 100644 index ea16534fc..000000000 --- a/docs/tutorials/wiki2/src/basiclayout/tests/test_it.py +++ /dev/null @@ -1,66 +0,0 @@ -import unittest - -from pyramid import testing - -import transaction - - -def dummy_request(dbsession): - return testing.DummyRequest(dbsession=dbsession) - - -class BaseTest(unittest.TestCase): - def setUp(self): - self.config = testing.setUp(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) - self.config.include('tutorial.models') - settings = self.config.get_settings() - - from tutorial.models import ( - get_engine, - get_session_factory, - get_tm_session, - ) - - self.engine = get_engine(settings) - session_factory = get_session_factory(self.engine) - - self.session = get_tm_session(session_factory, transaction.manager) - - def init_database(self): - from tutorial.models.meta import Base - Base.metadata.create_all(self.engine) - - def tearDown(self): - from tutorial.models.meta import Base - - testing.tearDown() - transaction.abort() - Base.metadata.drop_all(self.engine) - - -class TestMyViewSuccessCondition(BaseTest): - - def setUp(self): - super(TestMyViewSuccessCondition, self).setUp() - self.init_database() - - from tutorial.models import MyModel - - model = MyModel(name='one', value=55) - self.session.add(model) - - def test_passing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info['one'].name, 'one') - self.assertEqual(info['project'], 'myproj') - - -class TestMyViewFailureCondition(BaseTest): - - def test_failing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info.status_int, 500) diff --git a/docs/tutorials/wiki2/src/basiclayout/tests/test_views.py b/docs/tutorials/wiki2/src/basiclayout/tests/test_views.py new file mode 100644 index 000000000..8ae464d03 --- /dev/null +++ b/docs/tutorials/wiki2/src/basiclayout/tests/test_views.py @@ -0,0 +1,23 @@ +from tutorial import models +from tutorial.views.default import my_view +from tutorial.views.notfound import notfound_view + + +def test_my_view_failure(app_request): + info = my_view(app_request) + assert info.status_int == 500 + +def test_my_view_success(app_request, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + info = my_view(app_request) + assert app_request.response.status_int == 200 + assert info['one'].name == 'one' + assert info['project'] == 'myproj' + +def test_notfound_view(app_request): + info = notfound_view(app_request) + assert app_request.response.status_int == 404 + assert info == {} diff --git a/docs/tutorials/wiki2/src/basiclayout/tutorial/__init__.py b/docs/tutorials/wiki2/src/basiclayout/tutorial/__init__.py index 5c2ba5cc0..7edc0957d 100644 --- a/docs/tutorials/wiki2/src/basiclayout/tutorial/__init__.py +++ b/docs/tutorials/wiki2/src/basiclayout/tutorial/__init__.py @@ -5,8 +5,8 @@ def main(global_config, **settings): """ This function returns a Pyramid WSGI application. """ with Configurator(settings=settings) as config: - config.include('.models') config.include('pyramid_jinja2') config.include('.routes') + config.include('.models') config.scan() return config.make_wsgi_app() diff --git a/docs/tutorials/wiki2/src/basiclayout/tutorial/models/__init__.py b/docs/tutorials/wiki2/src/basiclayout/tutorial/models/__init__.py index d8a273e9e..1c3ec5ee8 100644 --- a/docs/tutorials/wiki2/src/basiclayout/tutorial/models/__init__.py +++ b/docs/tutorials/wiki2/src/basiclayout/tutorial/models/__init__.py @@ -65,13 +65,21 @@ def includeme(config): # use pyramid_retry to retry a request when transient exceptions occur config.include('pyramid_retry') - session_factory = get_session_factory(get_engine(settings)) + # hook to share the dbengine fixture in testing + dbengine = settings.get('dbengine') + if not dbengine: + dbengine = get_engine(settings) + + session_factory = get_session_factory(dbengine) config.registry['dbsession_factory'] = session_factory # make request.dbsession available for use in Pyramid - config.add_request_method( - # r.tm is the transaction manager used by pyramid_tm - lambda r: get_tm_session(session_factory, r.tm), - 'dbsession', - reify=True - ) + def dbsession(request): + # hook to share the dbsession fixture in testing + dbsession = request.environ.get('app.dbsession') + if dbsession is None: + # request.tm is the transaction manager used by pyramid_tm + dbsession = get_tm_session(session_factory, request.tm) + return dbsession + + config.add_request_method(dbsession, reify=True) diff --git a/docs/tutorials/wiki2/src/basiclayout/tutorial/views/default.py b/docs/tutorials/wiki2/src/basiclayout/tutorial/views/default.py index 094b2f303..a0f654d38 100644 --- a/docs/tutorials/wiki2/src/basiclayout/tutorial/views/default.py +++ b/docs/tutorials/wiki2/src/basiclayout/tutorial/views/default.py @@ -1,7 +1,6 @@ from pyramid.view import view_config from pyramid.response import Response - -from sqlalchemy.exc import DBAPIError +from sqlalchemy.exc import SQLAlchemyError from .. import models @@ -10,8 +9,8 @@ from .. import models def my_view(request): try: query = request.dbsession.query(models.MyModel) - one = query.filter(models.MyModel.name == 'one').first() - except DBAPIError: + one = query.filter(models.MyModel.name == 'one').one() + except SQLAlchemyError: return Response(db_err_msg, content_type='text/plain', status=500) return {'one': one, 'project': 'myproj'} diff --git a/docs/tutorials/wiki2/src/installation/.gitignore b/docs/tutorials/wiki2/src/installation/.gitignore index 1853d983c..c612e59f2 100644 --- a/docs/tutorials/wiki2/src/installation/.gitignore +++ b/docs/tutorials/wiki2/src/installation/.gitignore @@ -19,3 +19,4 @@ Data.fs* .DS_Store coverage test +*.sqlite diff --git a/docs/tutorials/wiki2/src/installation/testing.ini b/docs/tutorials/wiki2/src/installation/testing.ini new file mode 100644 index 000000000..5caa1a8dc --- /dev/null +++ b/docs/tutorials/wiki2/src/installation/testing.ini @@ -0,0 +1,79 @@ +### +# app configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/environment.html +### + +[app:main] +use = egg:tutorial + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en + +sqlalchemy.url = sqlite:///%(here)s/testing.sqlite + +retry.attempts = 3 + +[pshell] +setup = tutorial.pshell.setup + +### +# wsgi server configuration +### + +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + +[server:main] +use = egg:waitress#main +listen = localhost:6543 + +### +# logging configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html +### + +[loggers] +keys = root, tutorial, sqlalchemy, alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_tutorial] +level = DEBUG +handlers = +qualname = tutorial + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[logger_alembic] +level = WARN +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s diff --git a/docs/tutorials/wiki2/src/installation/tests/conftest.py b/docs/tutorials/wiki2/src/installation/tests/conftest.py new file mode 100644 index 000000000..2db65f887 --- /dev/null +++ b/docs/tutorials/wiki2/src/installation/tests/conftest.py @@ -0,0 +1,125 @@ +import alembic +import alembic.config +import alembic.command +import os +from pyramid.paster import get_appsettings +from pyramid.scripting import prepare +from pyramid.testing import DummyRequest +import pytest +import transaction +from webob.cookies import Cookie +import webtest + +from tutorial import main +from tutorial import models +from tutorial.models.meta import Base + + +def pytest_addoption(parser): + parser.addoption('--ini', action='store', metavar='INI_FILE') + +@pytest.fixture(scope='session') +def ini_file(request): + # potentially grab this path from a pytest option + return os.path.abspath(request.config.option.ini or 'testing.ini') + +@pytest.fixture(scope='session') +def app_settings(ini_file): + return get_appsettings(ini_file) + +@pytest.fixture(scope='session') +def dbengine(app_settings, ini_file): + engine = models.get_engine(app_settings) + + alembic_cfg = alembic.config.Config(ini_file) + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + + # run migrations to initialize the database + # depending on how we want to initialize the database from scratch + # we could alternatively call: + # Base.metadata.create_all(bind=engine) + # alembic.command.stamp(alembic_cfg, "head") + alembic.command.upgrade(alembic_cfg, "head") + + yield engine + + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + +@pytest.fixture(scope='session') +def app(app_settings, dbengine): + return main({}, dbengine=dbengine, **app_settings) + +@pytest.fixture +def tm(): + tm = transaction.TransactionManager(explicit=True) + tm.begin() + tm.doom() + + yield tm + + tm.abort() + +@pytest.fixture +def dbsession(app, tm): + session_factory = app.registry['dbsession_factory'] + return models.get_tm_session(session_factory, tm) + +@pytest.fixture +def testapp(app, tm, dbsession): + # override request.dbsession and request.tm with our own + # externally-controlled values that are shared across requests but aborted + # at the end + testapp = webtest.TestApp(app, extra_environ={ + 'HTTP_HOST': 'example.com', + 'tm.active': True, + 'tm.manager': tm, + 'app.dbsession': dbsession, + }) + + return testapp + +@pytest.fixture +def app_request(app, tm, dbsession): + """ + A real request. + + This request is almost identical to a real request but it has some + drawbacks in tests as it's harder to mock data and is heavier. + + """ + env = prepare(registry=app.registry) + request = env['request'] + request.host = 'example.com' + + # without this, request.dbsession will be joined to the same transaction + # manager but it will be using a different sqlalchemy.orm.Session using + # a separate database transaction + request.dbsession = dbsession + request.tm = tm + + yield request + env['closer']() + +@pytest.fixture +def dummy_request(app, tm, dbsession): + """ + A lightweight dummy request. + + This request is ultra-lightweight and should be used only when the + request itself is not a large focus in the call-stack. + + It is way easier to mock and control side-effects using this object. + + - It does not have request extensions applied. + - Threadlocals are not properly pushed. + + """ + request = DummyRequest() + request.registry = app.registry + request.host = 'example.com' + request.dbsession = dbsession + request.tm = tm + + return request diff --git a/docs/tutorials/wiki2/src/installation/tests/test_functional.py b/docs/tutorials/wiki2/src/installation/tests/test_functional.py new file mode 100644 index 000000000..dbcd8aec7 --- /dev/null +++ b/docs/tutorials/wiki2/src/installation/tests/test_functional.py @@ -0,0 +1,13 @@ +from tutorial import models + +def test_my_view_success(testapp, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + res = testapp.get('/', status=200) + assert res.body + +def test_notfound(testapp): + res = testapp.get('/badurl', status=404) + assert res.status_code == 404 diff --git a/docs/tutorials/wiki2/src/installation/tests/test_it.py b/docs/tutorials/wiki2/src/installation/tests/test_it.py deleted file mode 100644 index ea16534fc..000000000 --- a/docs/tutorials/wiki2/src/installation/tests/test_it.py +++ /dev/null @@ -1,66 +0,0 @@ -import unittest - -from pyramid import testing - -import transaction - - -def dummy_request(dbsession): - return testing.DummyRequest(dbsession=dbsession) - - -class BaseTest(unittest.TestCase): - def setUp(self): - self.config = testing.setUp(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) - self.config.include('tutorial.models') - settings = self.config.get_settings() - - from tutorial.models import ( - get_engine, - get_session_factory, - get_tm_session, - ) - - self.engine = get_engine(settings) - session_factory = get_session_factory(self.engine) - - self.session = get_tm_session(session_factory, transaction.manager) - - def init_database(self): - from tutorial.models.meta import Base - Base.metadata.create_all(self.engine) - - def tearDown(self): - from tutorial.models.meta import Base - - testing.tearDown() - transaction.abort() - Base.metadata.drop_all(self.engine) - - -class TestMyViewSuccessCondition(BaseTest): - - def setUp(self): - super(TestMyViewSuccessCondition, self).setUp() - self.init_database() - - from tutorial.models import MyModel - - model = MyModel(name='one', value=55) - self.session.add(model) - - def test_passing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info['one'].name, 'one') - self.assertEqual(info['project'], 'myproj') - - -class TestMyViewFailureCondition(BaseTest): - - def test_failing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info.status_int, 500) diff --git a/docs/tutorials/wiki2/src/installation/tests/test_views.py b/docs/tutorials/wiki2/src/installation/tests/test_views.py new file mode 100644 index 000000000..8ae464d03 --- /dev/null +++ b/docs/tutorials/wiki2/src/installation/tests/test_views.py @@ -0,0 +1,23 @@ +from tutorial import models +from tutorial.views.default import my_view +from tutorial.views.notfound import notfound_view + + +def test_my_view_failure(app_request): + info = my_view(app_request) + assert info.status_int == 500 + +def test_my_view_success(app_request, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + info = my_view(app_request) + assert app_request.response.status_int == 200 + assert info['one'].name == 'one' + assert info['project'] == 'myproj' + +def test_notfound_view(app_request): + info = notfound_view(app_request) + assert app_request.response.status_int == 404 + assert info == {} diff --git a/docs/tutorials/wiki2/src/installation/tutorial/__init__.py b/docs/tutorials/wiki2/src/installation/tutorial/__init__.py index 5c2ba5cc0..7edc0957d 100644 --- a/docs/tutorials/wiki2/src/installation/tutorial/__init__.py +++ b/docs/tutorials/wiki2/src/installation/tutorial/__init__.py @@ -5,8 +5,8 @@ def main(global_config, **settings): """ This function returns a Pyramid WSGI application. """ with Configurator(settings=settings) as config: - config.include('.models') config.include('pyramid_jinja2') config.include('.routes') + config.include('.models') config.scan() return config.make_wsgi_app() diff --git a/docs/tutorials/wiki2/src/installation/tutorial/models/__init__.py b/docs/tutorials/wiki2/src/installation/tutorial/models/__init__.py index d8a273e9e..1c3ec5ee8 100644 --- a/docs/tutorials/wiki2/src/installation/tutorial/models/__init__.py +++ b/docs/tutorials/wiki2/src/installation/tutorial/models/__init__.py @@ -65,13 +65,21 @@ def includeme(config): # use pyramid_retry to retry a request when transient exceptions occur config.include('pyramid_retry') - session_factory = get_session_factory(get_engine(settings)) + # hook to share the dbengine fixture in testing + dbengine = settings.get('dbengine') + if not dbengine: + dbengine = get_engine(settings) + + session_factory = get_session_factory(dbengine) config.registry['dbsession_factory'] = session_factory # make request.dbsession available for use in Pyramid - config.add_request_method( - # r.tm is the transaction manager used by pyramid_tm - lambda r: get_tm_session(session_factory, r.tm), - 'dbsession', - reify=True - ) + def dbsession(request): + # hook to share the dbsession fixture in testing + dbsession = request.environ.get('app.dbsession') + if dbsession is None: + # request.tm is the transaction manager used by pyramid_tm + dbsession = get_tm_session(session_factory, request.tm) + return dbsession + + config.add_request_method(dbsession, reify=True) diff --git a/docs/tutorials/wiki2/src/installation/tutorial/views/default.py b/docs/tutorials/wiki2/src/installation/tutorial/views/default.py index 094b2f303..a0f654d38 100644 --- a/docs/tutorials/wiki2/src/installation/tutorial/views/default.py +++ b/docs/tutorials/wiki2/src/installation/tutorial/views/default.py @@ -1,7 +1,6 @@ from pyramid.view import view_config from pyramid.response import Response - -from sqlalchemy.exc import DBAPIError +from sqlalchemy.exc import SQLAlchemyError from .. import models @@ -10,8 +9,8 @@ from .. import models def my_view(request): try: query = request.dbsession.query(models.MyModel) - one = query.filter(models.MyModel.name == 'one').first() - except DBAPIError: + one = query.filter(models.MyModel.name == 'one').one() + except SQLAlchemyError: return Response(db_err_msg, content_type='text/plain', status=500) return {'one': one, 'project': 'myproj'} diff --git a/docs/tutorials/wiki2/src/models/.gitignore b/docs/tutorials/wiki2/src/models/.gitignore index 1853d983c..c612e59f2 100644 --- a/docs/tutorials/wiki2/src/models/.gitignore +++ b/docs/tutorials/wiki2/src/models/.gitignore @@ -19,3 +19,4 @@ Data.fs* .DS_Store coverage test +*.sqlite diff --git a/docs/tutorials/wiki2/src/models/setup.py b/docs/tutorials/wiki2/src/models/setup.py index 60234751a..fbd848136 100644 --- a/docs/tutorials/wiki2/src/models/setup.py +++ b/docs/tutorials/wiki2/src/models/setup.py @@ -19,8 +19,8 @@ requires = [ 'pyramid_tm', 'SQLAlchemy', 'transaction', - 'zope.sqlalchemy', 'waitress', + 'zope.sqlalchemy', ] tests_require = [ diff --git a/docs/tutorials/wiki2/src/models/testing.ini b/docs/tutorials/wiki2/src/models/testing.ini new file mode 100644 index 000000000..5caa1a8dc --- /dev/null +++ b/docs/tutorials/wiki2/src/models/testing.ini @@ -0,0 +1,79 @@ +### +# app configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/environment.html +### + +[app:main] +use = egg:tutorial + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en + +sqlalchemy.url = sqlite:///%(here)s/testing.sqlite + +retry.attempts = 3 + +[pshell] +setup = tutorial.pshell.setup + +### +# wsgi server configuration +### + +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + +[server:main] +use = egg:waitress#main +listen = localhost:6543 + +### +# logging configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html +### + +[loggers] +keys = root, tutorial, sqlalchemy, alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_tutorial] +level = DEBUG +handlers = +qualname = tutorial + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[logger_alembic] +level = WARN +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s diff --git a/docs/tutorials/wiki2/src/models/tests/conftest.py b/docs/tutorials/wiki2/src/models/tests/conftest.py new file mode 100644 index 000000000..2db65f887 --- /dev/null +++ b/docs/tutorials/wiki2/src/models/tests/conftest.py @@ -0,0 +1,125 @@ +import alembic +import alembic.config +import alembic.command +import os +from pyramid.paster import get_appsettings +from pyramid.scripting import prepare +from pyramid.testing import DummyRequest +import pytest +import transaction +from webob.cookies import Cookie +import webtest + +from tutorial import main +from tutorial import models +from tutorial.models.meta import Base + + +def pytest_addoption(parser): + parser.addoption('--ini', action='store', metavar='INI_FILE') + +@pytest.fixture(scope='session') +def ini_file(request): + # potentially grab this path from a pytest option + return os.path.abspath(request.config.option.ini or 'testing.ini') + +@pytest.fixture(scope='session') +def app_settings(ini_file): + return get_appsettings(ini_file) + +@pytest.fixture(scope='session') +def dbengine(app_settings, ini_file): + engine = models.get_engine(app_settings) + + alembic_cfg = alembic.config.Config(ini_file) + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + + # run migrations to initialize the database + # depending on how we want to initialize the database from scratch + # we could alternatively call: + # Base.metadata.create_all(bind=engine) + # alembic.command.stamp(alembic_cfg, "head") + alembic.command.upgrade(alembic_cfg, "head") + + yield engine + + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + +@pytest.fixture(scope='session') +def app(app_settings, dbengine): + return main({}, dbengine=dbengine, **app_settings) + +@pytest.fixture +def tm(): + tm = transaction.TransactionManager(explicit=True) + tm.begin() + tm.doom() + + yield tm + + tm.abort() + +@pytest.fixture +def dbsession(app, tm): + session_factory = app.registry['dbsession_factory'] + return models.get_tm_session(session_factory, tm) + +@pytest.fixture +def testapp(app, tm, dbsession): + # override request.dbsession and request.tm with our own + # externally-controlled values that are shared across requests but aborted + # at the end + testapp = webtest.TestApp(app, extra_environ={ + 'HTTP_HOST': 'example.com', + 'tm.active': True, + 'tm.manager': tm, + 'app.dbsession': dbsession, + }) + + return testapp + +@pytest.fixture +def app_request(app, tm, dbsession): + """ + A real request. + + This request is almost identical to a real request but it has some + drawbacks in tests as it's harder to mock data and is heavier. + + """ + env = prepare(registry=app.registry) + request = env['request'] + request.host = 'example.com' + + # without this, request.dbsession will be joined to the same transaction + # manager but it will be using a different sqlalchemy.orm.Session using + # a separate database transaction + request.dbsession = dbsession + request.tm = tm + + yield request + env['closer']() + +@pytest.fixture +def dummy_request(app, tm, dbsession): + """ + A lightweight dummy request. + + This request is ultra-lightweight and should be used only when the + request itself is not a large focus in the call-stack. + + It is way easier to mock and control side-effects using this object. + + - It does not have request extensions applied. + - Threadlocals are not properly pushed. + + """ + request = DummyRequest() + request.registry = app.registry + request.host = 'example.com' + request.dbsession = dbsession + request.tm = tm + + return request diff --git a/docs/tutorials/wiki2/src/models/tests/test_functional.py b/docs/tutorials/wiki2/src/models/tests/test_functional.py new file mode 100644 index 000000000..dbcd8aec7 --- /dev/null +++ b/docs/tutorials/wiki2/src/models/tests/test_functional.py @@ -0,0 +1,13 @@ +from tutorial import models + +def test_my_view_success(testapp, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + res = testapp.get('/', status=200) + assert res.body + +def test_notfound(testapp): + res = testapp.get('/badurl', status=404) + assert res.status_code == 404 diff --git a/docs/tutorials/wiki2/src/models/tests/test_it.py b/docs/tutorials/wiki2/src/models/tests/test_it.py deleted file mode 100644 index ea16534fc..000000000 --- a/docs/tutorials/wiki2/src/models/tests/test_it.py +++ /dev/null @@ -1,66 +0,0 @@ -import unittest - -from pyramid import testing - -import transaction - - -def dummy_request(dbsession): - return testing.DummyRequest(dbsession=dbsession) - - -class BaseTest(unittest.TestCase): - def setUp(self): - self.config = testing.setUp(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) - self.config.include('tutorial.models') - settings = self.config.get_settings() - - from tutorial.models import ( - get_engine, - get_session_factory, - get_tm_session, - ) - - self.engine = get_engine(settings) - session_factory = get_session_factory(self.engine) - - self.session = get_tm_session(session_factory, transaction.manager) - - def init_database(self): - from tutorial.models.meta import Base - Base.metadata.create_all(self.engine) - - def tearDown(self): - from tutorial.models.meta import Base - - testing.tearDown() - transaction.abort() - Base.metadata.drop_all(self.engine) - - -class TestMyViewSuccessCondition(BaseTest): - - def setUp(self): - super(TestMyViewSuccessCondition, self).setUp() - self.init_database() - - from tutorial.models import MyModel - - model = MyModel(name='one', value=55) - self.session.add(model) - - def test_passing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info['one'].name, 'one') - self.assertEqual(info['project'], 'myproj') - - -class TestMyViewFailureCondition(BaseTest): - - def test_failing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info.status_int, 500) diff --git a/docs/tutorials/wiki2/src/models/tests/test_views.py b/docs/tutorials/wiki2/src/models/tests/test_views.py new file mode 100644 index 000000000..8ae464d03 --- /dev/null +++ b/docs/tutorials/wiki2/src/models/tests/test_views.py @@ -0,0 +1,23 @@ +from tutorial import models +from tutorial.views.default import my_view +from tutorial.views.notfound import notfound_view + + +def test_my_view_failure(app_request): + info = my_view(app_request) + assert info.status_int == 500 + +def test_my_view_success(app_request, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + info = my_view(app_request) + assert app_request.response.status_int == 200 + assert info['one'].name == 'one' + assert info['project'] == 'myproj' + +def test_notfound_view(app_request): + info = notfound_view(app_request) + assert app_request.response.status_int == 404 + assert info == {} diff --git a/docs/tutorials/wiki2/src/models/tutorial/__init__.py b/docs/tutorials/wiki2/src/models/tutorial/__init__.py index 5c2ba5cc0..7edc0957d 100644 --- a/docs/tutorials/wiki2/src/models/tutorial/__init__.py +++ b/docs/tutorials/wiki2/src/models/tutorial/__init__.py @@ -5,8 +5,8 @@ def main(global_config, **settings): """ This function returns a Pyramid WSGI application. """ with Configurator(settings=settings) as config: - config.include('.models') config.include('pyramid_jinja2') config.include('.routes') + config.include('.models') config.scan() return config.make_wsgi_app() diff --git a/docs/tutorials/wiki2/src/models/tutorial/models/__init__.py b/docs/tutorials/wiki2/src/models/tutorial/models/__init__.py index a4209a6e9..47d77ef01 100644 --- a/docs/tutorials/wiki2/src/models/tutorial/models/__init__.py +++ b/docs/tutorials/wiki2/src/models/tutorial/models/__init__.py @@ -66,13 +66,21 @@ def includeme(config): # use pyramid_retry to retry a request when transient exceptions occur config.include('pyramid_retry') - session_factory = get_session_factory(get_engine(settings)) + # hook to share the dbengine fixture in testing + dbengine = settings.get('dbengine') + if not dbengine: + dbengine = get_engine(settings) + + session_factory = get_session_factory(dbengine) config.registry['dbsession_factory'] = session_factory # make request.dbsession available for use in Pyramid - config.add_request_method( - # r.tm is the transaction manager used by pyramid_tm - lambda r: get_tm_session(session_factory, r.tm), - 'dbsession', - reify=True - ) + def dbsession(request): + # hook to share the dbsession fixture in testing + dbsession = request.environ.get('app.dbsession') + if dbsession is None: + # request.tm is the transaction manager used by pyramid_tm + dbsession = get_tm_session(session_factory, request.tm) + return dbsession + + config.add_request_method(dbsession, reify=True) diff --git a/docs/tutorials/wiki2/src/models/tutorial/scripts/initialize_db.py b/docs/tutorials/wiki2/src/models/tutorial/scripts/initialize_db.py index e6350fb36..c8034e5a5 100644 --- a/docs/tutorials/wiki2/src/models/tutorial/scripts/initialize_db.py +++ b/docs/tutorials/wiki2/src/models/tutorial/scripts/initialize_db.py @@ -8,6 +8,10 @@ from .. import models def setup_models(dbsession): + """ + Add or update models / fixtures in the database. + + """ editor = models.User(name='editor', role='editor') editor.set_password('editor') dbsession.add(editor) diff --git a/docs/tutorials/wiki2/src/models/tutorial/views/default.py b/docs/tutorials/wiki2/src/models/tutorial/views/default.py index 094b2f303..a0f654d38 100644 --- a/docs/tutorials/wiki2/src/models/tutorial/views/default.py +++ b/docs/tutorials/wiki2/src/models/tutorial/views/default.py @@ -1,7 +1,6 @@ from pyramid.view import view_config from pyramid.response import Response - -from sqlalchemy.exc import DBAPIError +from sqlalchemy.exc import SQLAlchemyError from .. import models @@ -10,8 +9,8 @@ from .. import models def my_view(request): try: query = request.dbsession.query(models.MyModel) - one = query.filter(models.MyModel.name == 'one').first() - except DBAPIError: + one = query.filter(models.MyModel.name == 'one').one() + except SQLAlchemyError: return Response(db_err_msg, content_type='text/plain', status=500) return {'one': one, 'project': 'myproj'} diff --git a/docs/tutorials/wiki2/src/tests/.gitignore b/docs/tutorials/wiki2/src/tests/.gitignore index 1853d983c..c612e59f2 100644 --- a/docs/tutorials/wiki2/src/tests/.gitignore +++ b/docs/tutorials/wiki2/src/tests/.gitignore @@ -19,3 +19,4 @@ Data.fs* .DS_Store coverage test +*.sqlite diff --git a/docs/tutorials/wiki2/src/tests/setup.py b/docs/tutorials/wiki2/src/tests/setup.py index 500c5e599..12eabaff2 100644 --- a/docs/tutorials/wiki2/src/tests/setup.py +++ b/docs/tutorials/wiki2/src/tests/setup.py @@ -20,8 +20,8 @@ requires = [ 'pyramid_tm', 'SQLAlchemy', 'transaction', - 'zope.sqlalchemy', 'waitress', + 'zope.sqlalchemy', ] tests_require = [ diff --git a/docs/tutorials/wiki2/src/tests/testing.ini b/docs/tutorials/wiki2/src/tests/testing.ini new file mode 100644 index 000000000..d3c601f16 --- /dev/null +++ b/docs/tutorials/wiki2/src/tests/testing.ini @@ -0,0 +1,81 @@ +### +# app configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/environment.html +### + +[app:main] +use = egg:tutorial + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en + +sqlalchemy.url = sqlite:///%(here)s/testing.sqlite + +retry.attempts = 3 + +auth.secret = test-seekrit + +[pshell] +setup = tutorial.pshell.setup + +### +# wsgi server configuration +### + +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + +[server:main] +use = egg:waitress#main +listen = localhost:6543 + +### +# logging configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html +### + +[loggers] +keys = root, tutorial, sqlalchemy, alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_tutorial] +level = DEBUG +handlers = +qualname = tutorial + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[logger_alembic] +level = WARN +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s diff --git a/docs/tutorials/wiki2/src/tests/tests/conftest.py b/docs/tutorials/wiki2/src/tests/tests/conftest.py new file mode 100644 index 000000000..094bc06f1 --- /dev/null +++ b/docs/tutorials/wiki2/src/tests/tests/conftest.py @@ -0,0 +1,165 @@ +import alembic +import alembic.config +import alembic.command +import os +from pyramid.paster import get_appsettings +from pyramid.scripting import prepare +from pyramid.testing import DummyRequest +import pytest +import transaction +from webob.cookies import Cookie +import webtest + +from tutorial import main +from tutorial import models +from tutorial.models.meta import Base + + +def pytest_addoption(parser): + parser.addoption('--ini', action='store', metavar='INI_FILE') + +@pytest.fixture(scope='session') +def ini_file(request): + # potentially grab this path from a pytest option + return os.path.abspath(request.config.option.ini or 'testing.ini') + +@pytest.fixture(scope='session') +def app_settings(ini_file): + return get_appsettings(ini_file) + +@pytest.fixture(scope='session') +def dbengine(app_settings, ini_file): + engine = models.get_engine(app_settings) + + alembic_cfg = alembic.config.Config(ini_file) + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + + # run migrations to initialize the database + # depending on how we want to initialize the database from scratch + # we could alternatively call: + # Base.metadata.create_all(bind=engine) + alembic.command.upgrade(alembic_cfg, "head") + + yield engine + + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + +@pytest.fixture(scope='session') +def app(app_settings, dbengine): + return main({}, dbengine=dbengine, **app_settings) + +@pytest.fixture +def tm(): + tm = transaction.TransactionManager(explicit=True) + tm.begin() + tm.doom() + + yield tm + + tm.abort() + +@pytest.fixture +def dbsession(app, tm): + session_factory = app.registry['dbsession_factory'] + return models.get_tm_session(session_factory, tm) + +class TestApp(webtest.TestApp): + def get_cookie(self, name, default=None): + # webtest currently doesn't expose the unescaped cookie values + # so we're using webob to parse them for us + # see https://github.com/Pylons/webtest/issues/171 + cookie = Cookie(' '.join( + '%s=%s' % (c.name, c.value) + for c in self.cookiejar + if c.name == name + )) + return next( + (m.value.decode('latin-1') for m in cookie.values()), + default, + ) + + def get_csrf_token(self): + """ + Convenience method to get the current CSRF token. + + This value must be passed to POST/PUT/DELETE requests in either the + "X-CSRF-Token" header or the "csrf_token" form value. + + testapp.post(..., headers={'X-CSRF-Token': testapp.get_csrf_token()}) + + or + + testapp.post(..., {'csrf_token': testapp.get_csrf_token()}) + + """ + return self.get_cookie('csrf_token') + + def login(self, params, status=303, **kw): + """ Convenience method to login the client.""" + body = dict(csrf_token=self.get_csrf_token()) + body.update(params) + return self.post('/login', body, **kw) + +@pytest.fixture +def testapp(app, tm, dbsession): + # override request.dbsession and request.tm with our own + # externally-controlled values that are shared across requests but aborted + # at the end + testapp = TestApp(app, extra_environ={ + 'HTTP_HOST': 'example.com', + 'tm.active': True, + 'tm.manager': tm, + 'app.dbsession': dbsession, + }) + + # initialize a csrf token instead of running an initial request to get one + # from the actual app - this only works using the CookieCSRFStoragePolicy + testapp.set_cookie('csrf_token', 'dummy_csrf_token') + + return testapp + +@pytest.fixture +def app_request(app, tm, dbsession): + """ + A real request. + + This request is almost identical to a real request but it has some + drawbacks in tests as it's harder to mock data and is heavier. + + """ + env = prepare(registry=app.registry) + request = env['request'] + request.host = 'example.com' + + # without this, request.dbsession will be joined to the same transaction + # manager but it will be using a different sqlalchemy.orm.Session using + # a separate database transaction + request.dbsession = dbsession + request.tm = tm + + yield request + env['closer']() + +@pytest.fixture +def dummy_request(app, tm, dbsession): + """ + A lightweight dummy request. + + This request is ultra-lightweight and should be used only when the + request itself is not a large focus in the call-stack. + + It is way easier to mock and control side-effects using this object. + + - It does not have request extensions applied. + - Threadlocals are not properly pushed. + + """ + request = DummyRequest() + request.registry = app.registry + request.host = 'example.com' + request.dbsession = dbsession + request.tm = tm + + return request diff --git a/docs/tutorials/wiki2/src/tests/tests/test_functional.py b/docs/tutorials/wiki2/src/tests/tests/test_functional.py index 0250e71c9..c6bbd3d5a 100644 --- a/docs/tutorials/wiki2/src/tests/tests/test_functional.py +++ b/docs/tutorials/wiki2/src/tests/tests/test_functional.py @@ -1,134 +1,127 @@ +import pytest import transaction -import unittest -import webtest - - -class FunctionalTests(unittest.TestCase): - - basic_login = ( - '/login?login=basic&password=basic' - '&next=FrontPage&form.submitted=Login') - basic_wrong_login = ( - '/login?login=basic&password=incorrect' - '&next=FrontPage&form.submitted=Login') - basic_login_no_next = ( - '/login?login=basic&password=basic' - '&form.submitted=Login') - editor_login = ( - '/login?login=editor&password=editor' - '&next=FrontPage&form.submitted=Login') - - @classmethod - def setUpClass(cls): - from tutorial.models.meta import Base - from tutorial.models import ( - User, - Page, - get_tm_session, - ) - from tutorial import main - - settings = { - 'sqlalchemy.url': 'sqlite://', - 'auth.secret': 'seekrit', - } - app = main({}, **settings) - cls.testapp = webtest.TestApp(app) - - session_factory = app.registry['dbsession_factory'] - cls.engine = session_factory.kw['bind'] - Base.metadata.create_all(bind=cls.engine) - - with transaction.manager: - dbsession = get_tm_session(session_factory, transaction.manager) - editor = User(name='editor', role='editor') - editor.set_password('editor') - basic = User(name='basic', role='basic') - basic.set_password('basic') - page1 = Page(name='FrontPage', data='This is the front page') - page1.creator = editor - page2 = Page(name='BackPage', data='This is the back page') - page2.creator = basic - dbsession.add_all([basic, editor, page1, page2]) - - @classmethod - def tearDownClass(cls): - from tutorial.models.meta import Base - Base.metadata.drop_all(bind=cls.engine) - - def test_root(self): - res = self.testapp.get('/', status=302) - self.assertEqual(res.location, 'http://localhost/FrontPage') - - def test_FrontPage(self): - res = self.testapp.get('/FrontPage', status=200) - self.assertTrue(b'FrontPage' in res.body) - - def test_unexisting_page(self): - self.testapp.get('/SomePage', status=404) - - def test_successful_log_in(self): - res = self.testapp.get(self.basic_login, status=302) - self.assertEqual(res.location, 'http://localhost/FrontPage') - - def test_successful_log_in_no_next(self): - res = self.testapp.get(self.basic_login_no_next, status=302) - self.assertEqual(res.location, 'http://localhost/') - - def test_failed_log_in(self): - res = self.testapp.get(self.basic_wrong_login, status=200) - self.assertTrue(b'login' in res.body) - - def test_logout_link_present_when_logged_in(self): - self.testapp.get(self.basic_login, status=302) - res = self.testapp.get('/FrontPage', status=200) - self.assertTrue(b'Logout' in res.body) - - def test_logout_link_not_present_after_logged_out(self): - self.testapp.get(self.basic_login, status=302) - self.testapp.get('/FrontPage', status=200) - res = self.testapp.get('/logout', status=302) - self.assertTrue(b'Logout' not in res.body) - - def test_anonymous_user_cannot_edit(self): - res = self.testapp.get('/FrontPage/edit_page', status=302).follow() - self.assertTrue(b'Login' in res.body) - - def test_anonymous_user_cannot_add(self): - res = self.testapp.get('/add_page/NewPage', status=302).follow() - self.assertTrue(b'Login' in res.body) - - def test_basic_user_cannot_edit_front(self): - self.testapp.get(self.basic_login, status=302) - res = self.testapp.get('/FrontPage/edit_page', status=302).follow() - self.assertTrue(b'Login' in res.body) - - def test_basic_user_can_edit_back(self): - self.testapp.get(self.basic_login, status=302) - res = self.testapp.get('/BackPage/edit_page', status=200) - self.assertTrue(b'Editing' in res.body) - - def test_basic_user_can_add(self): - self.testapp.get(self.basic_login, status=302) - res = self.testapp.get('/add_page/NewPage', status=200) - self.assertTrue(b'Editing' in res.body) - - def test_editors_member_user_can_edit(self): - self.testapp.get(self.editor_login, status=302) - res = self.testapp.get('/FrontPage/edit_page', status=200) - self.assertTrue(b'Editing' in res.body) - - def test_editors_member_user_can_add(self): - self.testapp.get(self.editor_login, status=302) - res = self.testapp.get('/add_page/NewPage', status=200) - self.assertTrue(b'Editing' in res.body) - - def test_editors_member_user_can_view(self): - self.testapp.get(self.editor_login, status=302) - res = self.testapp.get('/FrontPage', status=200) - self.assertTrue(b'FrontPage' in res.body) - - def test_redirect_to_edit_for_existing_page(self): - self.testapp.get(self.editor_login, status=302) - res = self.testapp.get('/add_page/FrontPage', status=302) - self.assertTrue(b'FrontPage' in res.body) + +from tutorial import models + + +basic_login = dict(login='basic', password='basic') +editor_login = dict(login='editor', password='editor') + +@pytest.fixture(scope='session', autouse=True) +def dummy_data(app): + """ + Add some dummy data to the database. + + Note that this is a session fixture that commits data to the database. + Think about it similarly to running the ``initialize_db`` script at the + start of the test suite. + + This data should not conflict with any other data added throughout the + test suite or there will be issues - so be careful with this pattern! + + """ + tm = transaction.TransactionManager(explicit=True) + with tm: + dbsession = models.get_tm_session(app.registry['dbsession_factory'], tm) + editor = models.User(name='editor', role='editor') + editor.set_password('editor') + basic = models.User(name='basic', role='basic') + basic.set_password('basic') + page1 = models.Page(name='FrontPage', data='This is the front page') + page1.creator = editor + page2 = models.Page(name='BackPage', data='This is the back page') + page2.creator = basic + dbsession.add_all([basic, editor, page1, page2]) + +def test_root(testapp): + res = testapp.get('/', status=303) + assert res.location == 'http://example.com/FrontPage' + +def test_FrontPage(testapp): + res = testapp.get('/FrontPage', status=200) + assert b'FrontPage' in res.body + +def test_missing_page(testapp): + res = testapp.get('/SomePage', status=404) + assert b'404' in res.body + +def test_successful_log_in(testapp): + params = dict( + **basic_login, + csrf_token=testapp.get_csrf_token(), + ) + res = testapp.post('/login', params, status=303) + assert res.location == 'http://example.com/' + +def test_successful_log_with_next(testapp): + params = dict( + **basic_login, + next='WikiPage', + csrf_token=testapp.get_csrf_token(), + ) + res = testapp.post('/login', params, status=303) + assert res.location == 'http://example.com/WikiPage' + +def test_failed_log_in(testapp): + params = dict( + login='basic', + password='incorrect', + csrf_token=testapp.get_csrf_token(), + ) + res = testapp.post('/login', params, status=400) + assert b'login' in res.body + +def test_logout_link_present_when_logged_in(testapp): + testapp.login(basic_login) + res = testapp.get('/FrontPage', status=200) + assert b'Logout' in res.body + +def test_logout_link_not_present_after_logged_out(testapp): + testapp.login(basic_login) + testapp.get('/FrontPage', status=200) + params = dict(csrf_token=testapp.get_csrf_token()) + res = testapp.post('/logout', params, status=303) + assert b'Logout' not in res.body + +def test_anonymous_user_cannot_edit(testapp): + res = testapp.get('/FrontPage/edit_page', status=303).follow() + assert b'Login' in res.body + +def test_anonymous_user_cannot_add(testapp): + res = testapp.get('/add_page/NewPage', status=303).follow() + assert b'Login' in res.body + +def test_basic_user_cannot_edit_front(testapp): + testapp.login(basic_login) + res = testapp.get('/FrontPage/edit_page', status=403) + assert b'403' in res.body + +def test_basic_user_can_edit_back(testapp): + testapp.login(basic_login) + res = testapp.get('/BackPage/edit_page', status=200) + assert b'Editing' in res.body + +def test_basic_user_can_add(testapp): + testapp.login(basic_login) + res = testapp.get('/add_page/NewPage', status=200) + assert b'Editing' in res.body + +def test_editors_member_user_can_edit(testapp): + testapp.login(editor_login) + res = testapp.get('/FrontPage/edit_page', status=200) + assert b'Editing' in res.body + +def test_editors_member_user_can_add(testapp): + testapp.login(editor_login) + res = testapp.get('/add_page/NewPage', status=200) + assert b'Editing' in res.body + +def test_editors_member_user_can_view(testapp): + testapp.login(editor_login) + res = testapp.get('/FrontPage', status=200) + assert b'FrontPage' in res.body + +def test_redirect_to_edit_for_existing_page(testapp): + testapp.login(editor_login) + res = testapp.get('/add_page/FrontPage', status=303) + assert b'FrontPage' in res.body diff --git a/docs/tutorials/wiki2/src/tests/tests/test_initdb.py b/docs/tutorials/wiki2/src/tests/tests/test_initdb.py deleted file mode 100644 index a66945ccc..000000000 --- a/docs/tutorials/wiki2/src/tests/tests/test_initdb.py +++ /dev/null @@ -1,10 +0,0 @@ -import os -import unittest - - -class TestInitializeDB(unittest.TestCase): - - def test_usage(self): - from tutorial.scripts.initialize_db import main - with self.assertRaises(SystemExit): - main(argv=['foo']) diff --git a/docs/tutorials/wiki2/src/tests/tests/test_security.py b/docs/tutorials/wiki2/src/tests/tests/test_security.py deleted file mode 100644 index 9a1455ef9..000000000 --- a/docs/tutorials/wiki2/src/tests/tests/test_security.py +++ /dev/null @@ -1,23 +0,0 @@ -import unittest -from pyramid.testing import DummyRequest - - -class TestMyAuthenticationPolicy(unittest.TestCase): - - def test_no_user(self): - request = DummyRequest() - request.user = None - - from tutorial.security import MyAuthenticationPolicy - policy = MyAuthenticationPolicy(None) - self.assertEqual(policy.authenticated_userid(request), None) - - def test_authenticated_user(self): - from tutorial.models import User - request = DummyRequest() - request.user = User() - request.user.id = 'foo' - - from tutorial.security import MyAuthenticationPolicy - policy = MyAuthenticationPolicy(None) - self.assertEqual(policy.authenticated_userid(request), 'foo') diff --git a/docs/tutorials/wiki2/src/tests/tests/test_user_model.py b/docs/tutorials/wiki2/src/tests/tests/test_user_model.py index 21904da6b..f91116360 100644 --- a/docs/tutorials/wiki2/src/tests/tests/test_user_model.py +++ b/docs/tutorials/wiki2/src/tests/tests/test_user_model.py @@ -1,67 +1,23 @@ -import unittest -import transaction +from tutorial import models -from pyramid import testing +def test_password_hash_saved(): + user = models.User(name='foo', role='bar') + assert user.password_hash is None -class BaseTest(unittest.TestCase): + user.set_password('secret') + assert user.password_hash is not None - def setUp(self): - from tutorial.models import get_tm_session - self.config = testing.setUp(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) - self.config.include('tutorial.models') - self.config.include('tutorial.routes') +def test_password_hash_not_set(): + user = models.User(name='foo', role='bar') + assert not user.check_password('secret') - session_factory = self.config.registry['dbsession_factory'] - self.session = get_tm_session(session_factory, transaction.manager) +def test_correct_password(): + user = models.User(name='foo', role='bar') + user.set_password('secret') + assert user.check_password('secret') - self.init_database() - - def init_database(self): - from tutorial.models.meta import Base - session_factory = self.config.registry['dbsession_factory'] - engine = session_factory.kw['bind'] - Base.metadata.create_all(engine) - - def tearDown(self): - testing.tearDown() - transaction.abort() - - def makeUser(self, name, role): - from tutorial.models import User - return User(name=name, role=role) - - -class TestSetPassword(BaseTest): - - def test_password_hash_saved(self): - user = self.makeUser(name='foo', role='bar') - self.assertFalse(user.password_hash) - - user.set_password('secret') - self.assertTrue(user.password_hash) - - -class TestCheckPassword(BaseTest): - - def test_password_hash_not_set(self): - user = self.makeUser(name='foo', role='bar') - self.assertFalse(user.password_hash) - - self.assertFalse(user.check_password('secret')) - - def test_correct_password(self): - user = self.makeUser(name='foo', role='bar') - user.set_password('secret') - self.assertTrue(user.password_hash) - - self.assertTrue(user.check_password('secret')) - - def test_incorrect_password(self): - user = self.makeUser(name='foo', role='bar') - user.set_password('secret') - self.assertTrue(user.password_hash) - - self.assertFalse(user.check_password('incorrect')) +def test_incorrect_password(): + user = models.User(name='foo', role='bar') + user.set_password('secret') + assert not user.check_password('incorrect') diff --git a/docs/tutorials/wiki2/src/tests/tests/test_views.py b/docs/tutorials/wiki2/src/tests/tests/test_views.py index 5c17457dd..007184af8 100644 --- a/docs/tutorials/wiki2/src/tests/tests/test_views.py +++ b/docs/tutorials/wiki2/src/tests/tests/test_views.py @@ -1,168 +1,109 @@ -import unittest -import transaction +from tutorial import models -from pyramid import testing +def makeUser(name, role): + return models.User(name=name, role=role) -def dummy_request(dbsession): - return testing.DummyRequest(dbsession=dbsession) - - -class BaseTest(unittest.TestCase): - def setUp(self): - from tutorial.models import get_tm_session - self.config = testing.setUp(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) - self.config.include('tutorial.models') - self.config.include('tutorial.routes') - - session_factory = self.config.registry['dbsession_factory'] - self.session = get_tm_session(session_factory, transaction.manager) - - self.init_database() - - def init_database(self): - from tutorial.models.meta import Base - session_factory = self.config.registry['dbsession_factory'] - engine = session_factory.kw['bind'] - Base.metadata.create_all(engine) - - def tearDown(self): - testing.tearDown() - transaction.abort() - - def makeUser(self, name, role, password='dummy'): - from tutorial.models import User - user = User(name=name, role=role) - user.set_password(password) - return user - - def makePage(self, name, data, creator): - from tutorial.models import Page - return Page(name=name, data=data, creator=creator) - - -class ViewWikiTests(unittest.TestCase): - def setUp(self): - self.config = testing.setUp() - self.config.include('tutorial.routes') - - def tearDown(self): - testing.tearDown() +def makePage(name, data, creator): + return models.Page(name=name, data=data, creator=creator) +class Test_view_wiki: def _callFUT(self, request): from tutorial.views.default import view_wiki return view_wiki(request) - def test_it(self): - request = testing.DummyRequest() - response = self._callFUT(request) - self.assertEqual(response.location, 'http://example.com/FrontPage') - + def test_it(self, dummy_request): + response = self._callFUT(dummy_request) + assert response.location == 'http://example.com/FrontPage' -class ViewPageTests(BaseTest): +class Test_view_page: def _callFUT(self, request): from tutorial.views.default import view_page return view_page(request) - def test_it(self): + def _makeContext(self, page): from tutorial.routes import PageResource + return PageResource(page) + def test_it(self, dummy_request, dbsession): # add a page to the db - user = self.makeUser('foo', 'editor') - page = self.makePage('IDoExist', 'Hello CruelWorld IDoExist', user) - self.session.add_all([page, user]) + user = makeUser('foo', 'editor') + page = makePage('IDoExist', 'Hello CruelWorld IDoExist', user) + dbsession.add_all([page, user]) # create a request asking for the page we've created - request = dummy_request(self.session) - request.context = PageResource(page) + dummy_request.context = self._makeContext(page) # call the view we're testing and check its behavior - info = self._callFUT(request) - self.assertEqual(info['page'], page) - self.assertEqual( - info['content'], + info = self._callFUT(dummy_request) + assert info['page'] is page + assert info['content'] == ( '<div class="document">\n' '<p>Hello <a href="http://example.com/add_page/CruelWorld">' 'CruelWorld</a> ' '<a href="http://example.com/IDoExist">' 'IDoExist</a>' - '</p>\n</div>\n') - self.assertEqual(info['edit_url'], - 'http://example.com/IDoExist/edit_page') - + '</p>\n</div>\n' + ) + assert info['edit_url'] == 'http://example.com/IDoExist/edit_page' -class AddPageTests(BaseTest): +class Test_add_page: def _callFUT(self, request): from tutorial.views.default import add_page return add_page(request) - def test_it_pageexists(self): - from tutorial.models import Page + def _makeContext(self, pagename): from tutorial.routes import NewPage - request = testing.DummyRequest({'form.submitted': True, - 'body': 'Hello yo!'}, - dbsession=self.session) - request.user = self.makeUser('foo', 'editor') - request.context = NewPage('AnotherPage') - self._callFUT(request) - pagecount = self.session.query(Page).filter_by(name='AnotherPage').count() - self.assertGreater(pagecount, 0) - - def test_it_notsubmitted(self): - from tutorial.routes import NewPage - request = dummy_request(self.session) - request.user = self.makeUser('foo', 'editor') - request.context = NewPage('AnotherPage') - info = self._callFUT(request) - self.assertEqual(info['pagedata'], '') - self.assertEqual(info['save_url'], - 'http://example.com/add_page/AnotherPage') - - def test_it_submitted(self): - from tutorial.models import Page - from tutorial.routes import NewPage - request = testing.DummyRequest({'form.submitted': True, - 'body': 'Hello yo!'}, - dbsession=self.session) - request.user = self.makeUser('foo', 'editor') - request.context = NewPage('AnotherPage') - self._callFUT(request) - page = self.session.query(Page).filter_by(name='AnotherPage').one() - self.assertEqual(page.data, 'Hello yo!') - - -class EditPageTests(BaseTest): + return NewPage(pagename) + + def test_get(self, dummy_request, dbsession): + dummy_request.user = makeUser('foo', 'editor') + dummy_request.context = self._makeContext('AnotherPage') + info = self._callFUT(dummy_request) + assert info['pagedata'] == '' + assert info['save_url'] == 'http://example.com/add_page/AnotherPage' + + def test_submit_works(self, dummy_request, dbsession): + dummy_request.method = 'POST' + dummy_request.POST['body'] = 'Hello yo!' + dummy_request.context = self._makeContext('AnotherPage') + dummy_request.user = makeUser('foo', 'editor') + self._callFUT(dummy_request) + page = ( + dbsession.query(models.Page) + .filter_by(name='AnotherPage') + .one() + ) + assert page.data == 'Hello yo!' + +class Test_edit_page: def _callFUT(self, request): from tutorial.views.default import edit_page return edit_page(request) - def makeContext(self, page): + def _makeContext(self, page): from tutorial.routes import PageResource return PageResource(page) - def test_it_notsubmitted(self): - user = self.makeUser('foo', 'editor') - page = self.makePage('abc', 'hello', user) - self.session.add_all([page, user]) - - request = dummy_request(self.session) - request.context = self.makeContext(page) - info = self._callFUT(request) - self.assertEqual(info['pagename'], 'abc') - self.assertEqual(info['save_url'], - 'http://example.com/abc/edit_page') - - def test_it_submitted(self): - user = self.makeUser('foo', 'editor') - page = self.makePage('abc', 'hello', user) - self.session.add_all([page, user]) - - request = testing.DummyRequest({'form.submitted': True, - 'body': 'Hello yo!'}, - dbsession=self.session) - request.context = self.makeContext(page) - response = self._callFUT(request) - self.assertEqual(response.location, 'http://example.com/abc') - self.assertEqual(page.data, 'Hello yo!') + def test_get(self, dummy_request, dbsession): + user = makeUser('foo', 'editor') + page = makePage('abc', 'hello', user) + dbsession.add_all([page, user]) + + dummy_request.context = self._makeContext(page) + info = self._callFUT(dummy_request) + assert info['pagename'] == 'abc' + assert info['save_url'] == 'http://example.com/abc/edit_page' + + def test_submit_works(self, dummy_request, dbsession): + user = makeUser('foo', 'editor') + page = makePage('abc', 'hello', user) + dbsession.add_all([page, user]) + + dummy_request.method = 'POST' + dummy_request.POST['body'] = 'Hello yo!' + dummy_request.user = user + dummy_request.context = self._makeContext(page) + response = self._callFUT(dummy_request) + assert response.location == 'http://example.com/abc' + assert page.data == 'Hello yo!' diff --git a/docs/tutorials/wiki2/src/tests/tutorial/__init__.py b/docs/tutorials/wiki2/src/tests/tutorial/__init__.py index ce2e9f12a..81a22c68c 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/__init__.py +++ b/docs/tutorials/wiki2/src/tests/tutorial/__init__.py @@ -5,9 +5,9 @@ def main(global_config, **settings): """ This function returns a Pyramid WSGI application. """ with Configurator(settings=settings) as config: - config.include('.models') config.include('pyramid_jinja2') - config.include('.routes') config.include('.security') + config.include('.routes') + config.include('.models') config.scan() return config.make_wsgi_app() diff --git a/docs/tutorials/wiki2/src/tests/tutorial/models/__init__.py b/docs/tutorials/wiki2/src/tests/tutorial/models/__init__.py index a4209a6e9..47d77ef01 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/models/__init__.py +++ b/docs/tutorials/wiki2/src/tests/tutorial/models/__init__.py @@ -66,13 +66,21 @@ def includeme(config): # use pyramid_retry to retry a request when transient exceptions occur config.include('pyramid_retry') - session_factory = get_session_factory(get_engine(settings)) + # hook to share the dbengine fixture in testing + dbengine = settings.get('dbengine') + if not dbengine: + dbengine = get_engine(settings) + + session_factory = get_session_factory(dbengine) config.registry['dbsession_factory'] = session_factory # make request.dbsession available for use in Pyramid - config.add_request_method( - # r.tm is the transaction manager used by pyramid_tm - lambda r: get_tm_session(session_factory, r.tm), - 'dbsession', - reify=True - ) + def dbsession(request): + # hook to share the dbsession fixture in testing + dbsession = request.environ.get('app.dbsession') + if dbsession is None: + # request.tm is the transaction manager used by pyramid_tm + dbsession = get_tm_session(session_factory, request.tm) + return dbsession + + config.add_request_method(dbsession, reify=True) diff --git a/docs/tutorials/wiki2/src/tests/tutorial/routes.py b/docs/tutorials/wiki2/src/tests/tutorial/routes.py index 1fd45a994..f016d7541 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/routes.py +++ b/docs/tutorials/wiki2/src/tests/tutorial/routes.py @@ -1,6 +1,6 @@ from pyramid.httpexceptions import ( HTTPNotFound, - HTTPFound, + HTTPSeeOther, ) from pyramid.security import ( Allow, @@ -9,6 +9,7 @@ from pyramid.security import ( from . import models + def includeme(config): config.add_static_view('static', 'static', cache_max_age=3600) config.add_route('view_wiki', '/') @@ -24,7 +25,7 @@ def new_page_factory(request): pagename = request.matchdict['pagename'] if request.dbsession.query(models.Page).filter_by(name=pagename).count() > 0: next_url = request.route_url('edit_page', pagename=pagename) - raise HTTPFound(location=next_url) + raise HTTPSeeOther(location=next_url) return NewPage(pagename) class NewPage(object): @@ -52,5 +53,5 @@ class PageResource(object): return [ (Allow, Everyone, 'view'), (Allow, 'role:editor', 'edit'), - (Allow, str(self.page.creator_id), 'edit'), + (Allow, 'u:' + str(self.page.creator_id), 'edit'), ] diff --git a/docs/tutorials/wiki2/src/tests/tutorial/scripts/initialize_db.py b/docs/tutorials/wiki2/src/tests/tutorial/scripts/initialize_db.py index e6350fb36..c8034e5a5 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/scripts/initialize_db.py +++ b/docs/tutorials/wiki2/src/tests/tutorial/scripts/initialize_db.py @@ -8,6 +8,10 @@ from .. import models def setup_models(dbsession): + """ + Add or update models / fixtures in the database. + + """ editor = models.User(name='editor', role='editor') editor.set_password('editor') dbsession.add(editor) diff --git a/docs/tutorials/wiki2/src/tests/tutorial/security.py b/docs/tutorials/wiki2/src/tests/tutorial/security.py index 1ce1c8753..448183c95 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/security.py +++ b/docs/tutorials/wiki2/src/tests/tutorial/security.py @@ -1,5 +1,6 @@ -from pyramid.authentication import AuthTktAuthenticationPolicy -from pyramid.authorization import ACLAuthorizationPolicy +from pyramid.authentication import AuthTktCookieHelper +from pyramid.authorization import ACLHelper +from pyramid.csrf import CookieCSRFStoragePolicy from pyramid.security import ( Authenticated, Everyone, @@ -8,33 +9,50 @@ from pyramid.security import ( from . import models -class MyAuthenticationPolicy(AuthTktAuthenticationPolicy): +class MySecurityPolicy: + def __init__(self, secret): + self.authtkt = AuthTktCookieHelper(secret) + self.acl = ACLHelper() + + def authenticated_identity(self, request): + identity = self.authtkt.identify(request) + if identity is None: + return None + + userid = identity['userid'] + user = request.dbsession.query(models.User).get(userid) + return user + def authenticated_userid(self, request): - user = request.user + user = self.authenticated_identity(request) if user is not None: return user.id + def remember(self, request, userid, **kw): + return self.authtkt.remember(request, userid, **kw) + + def forget(self, request, **kw): + return self.authtkt.forget(request, **kw) + + def permits(self, request, context, permission): + principals = self.effective_principals(request) + return self.acl.permits(context, principals, permission) + def effective_principals(self, request): principals = [Everyone] - user = request.user + user = self.authenticated_identity(request) if user is not None: principals.append(Authenticated) - principals.append(str(user.id)) + principals.append('u:' + str(user.id)) principals.append('role:' + user.role) return principals -def get_user(request): - user_id = request.unauthenticated_userid - if user_id is not None: - user = request.dbsession.query(models.User).get(user_id) - return user - def includeme(config): settings = config.get_settings() - authn_policy = MyAuthenticationPolicy( - settings['auth.secret'], - hashalg='sha512', - ) - config.set_authentication_policy(authn_policy) - config.set_authorization_policy(ACLAuthorizationPolicy()) - config.add_request_method(get_user, 'user', reify=True) + + config.set_csrf_storage_policy(CookieCSRFStoragePolicy()) + config.set_default_csrf_options(require_csrf=True) + + config.set_security_policy(MySecurityPolicy(settings['auth.secret'])) + config.add_request_method( + lambda request: request.authenticated_identity, 'user', property=True) diff --git a/docs/tutorials/wiki2/src/tests/tutorial/templates/403.jinja2 b/docs/tutorials/wiki2/src/tests/tutorial/templates/403.jinja2 new file mode 100644 index 000000000..7a6f523bc --- /dev/null +++ b/docs/tutorials/wiki2/src/tests/tutorial/templates/403.jinja2 @@ -0,0 +1,6 @@ +{% extends "layout.jinja2" %} + +{% block content %} +<h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> +<p class="lead"><span class="font-semi-bold">403</span> Forbidden</p> +{% endblock content %} diff --git a/docs/tutorials/wiki2/src/tests/tutorial/templates/404.jinja2 b/docs/tutorials/wiki2/src/tests/tutorial/templates/404.jinja2 index aaf12413f..5edb15285 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/templates/404.jinja2 +++ b/docs/tutorials/wiki2/src/tests/tutorial/templates/404.jinja2 @@ -1,8 +1,6 @@ {% extends "layout.jinja2" %} {% block content %} -<div class="content"> - <h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> - <p class="lead"><span class="font-semi-bold">404</span> Page Not Found</p> -</div> +<h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> +<p class="lead"><span class="font-semi-bold">404</span> Page Not Found</p> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/tests/tutorial/templates/edit.jinja2 b/docs/tutorials/wiki2/src/tests/tutorial/templates/edit.jinja2 index 7db25c674..27b545054 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/templates/edit.jinja2 +++ b/docs/tutorials/wiki2/src/tests/tutorial/templates/edit.jinja2 @@ -10,11 +10,12 @@ Editing <strong>{{pagename}}</strong> <a href="{{request.route_url('view_page', pagename='FrontPage')}}">FrontPage</a>. </p> <form action="{{ save_url }}" method="post"> +<input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> <div class="form-group"> <textarea class="form-control" name="body" rows="10" cols="60">{{ pagedata }}</textarea> </div> <div class="form-group"> - <button type="submit" name="form.submitted" value="Save" class="btn btn-default">Save</button> + <button type="submit" class="btn btn-default">Save</button> </div> </form> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/tests/tutorial/templates/layout.jinja2 b/docs/tutorials/wiki2/src/tests/tutorial/templates/layout.jinja2 index 4016b26c9..64a1db0c5 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/templates/layout.jinja2 +++ b/docs/tutorials/wiki2/src/tests/tutorial/templates/layout.jinja2 @@ -35,18 +35,29 @@ <div class="content"> {% if request.user is none %} <p class="pull-right"> - <a href="{{ request.route_url('login') }}">Login</a> + <a href="{{ request.route_url('login') }}">Login</a> </p> {% else %} - <p class="pull-right"> - {{request.user.name}} <a href="{{request.route_url('logout')}}">Logout</a> - </p> + <form class="pull-right" action="{{ request.route_url('logout') }}" method="post"> + {{request.user.name}} + <input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> + <button class="btn btn-link" type="submit">Logout</button> + </form> {% endif %} {% block content %}{% endblock %} </div> </div> </div> <div class="row"> + <div class="links"> + <ul> + <li><i class="glyphicon glyphicon-cog icon-muted"></i><a href="https://github.com/Pylons/pyramid">Github Project</a></li> + <li><i class="glyphicon glyphicon-globe icon-muted"></i><a href="https://webchat.freenode.net/?channels=pyramid">IRC Channel</a></li> + <li><i class="glyphicon glyphicon-home icon-muted"></i><a href="https://pylonsproject.org">Pylons Project</a></li> + </ul> + </div> + </div> + <div class="row"> <div class="copyright"> Copyright © Pylons Project </div> diff --git a/docs/tutorials/wiki2/src/tests/tutorial/templates/login.jinja2 b/docs/tutorials/wiki2/src/tests/tutorial/templates/login.jinja2 index 1806de0ff..058b7254b 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/templates/login.jinja2 +++ b/docs/tutorials/wiki2/src/tests/tutorial/templates/login.jinja2 @@ -10,6 +10,7 @@ {{ message }} </p> <form action="{{ url }}" method="post"> +<input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> <input type="hidden" name="next" value="{{ next_url }}"> <div class="form-group"> <label for="login">Username</label> @@ -20,7 +21,7 @@ <input type="password" name="password"> </div> <div class="form-group"> - <button type="submit" name="form.submitted" value="Log In" class="btn btn-default">Log In</button> + <button type="submit" class="btn btn-default">Log In</button> </div> </form> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/tests/tutorial/views/auth.py b/docs/tutorials/wiki2/src/tests/tutorial/views/auth.py index 16fa616e5..e1a564415 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/views/auth.py +++ b/docs/tutorials/wiki2/src/tests/tutorial/views/auth.py @@ -1,14 +1,15 @@ -from pyramid.httpexceptions import HTTPFound +from pyramid.csrf import new_csrf_token +from pyramid.httpexceptions import HTTPSeeOther from pyramid.security import ( remember, forget, - ) +) from pyramid.view import ( forbidden_view_config, view_config, ) -from ..models import User +from .. import models @view_config(route_name='login', renderer='tutorial:templates/login.jinja2') @@ -18,29 +19,43 @@ def login(request): next_url = request.route_url('view_wiki') message = '' login = '' - if 'form.submitted' in request.params: + if request.method == 'POST': login = request.params['login'] password = request.params['password'] - user = request.dbsession.query(User).filter_by(name=login).first() + user = ( + request.dbsession.query(models.User) + .filter_by(name=login) + .first() + ) if user is not None and user.check_password(password): + new_csrf_token(request) headers = remember(request, user.id) - return HTTPFound(location=next_url, headers=headers) + return HTTPSeeOther(location=next_url, headers=headers) message = 'Failed login' + request.response.status = 400 return dict( message=message, url=request.route_url('login'), next_url=next_url, login=login, - ) + ) @view_config(route_name='logout') def logout(request): - headers = forget(request) next_url = request.route_url('view_wiki') - return HTTPFound(location=next_url, headers=headers) + if request.method == 'POST': + new_csrf_token(request) + headers = forget(request) + return HTTPSeeOther(location=next_url, headers=headers) + + return HTTPSeeOther(location=next_url) + +@forbidden_view_config(renderer='tutorial:templates/403.jinja2') +def forbidden_view(exc, request): + if request.user is None: + next_url = request.route_url('login', _query={'next': request.url}) + return HTTPSeeOther(location=next_url) -@forbidden_view_config() -def forbidden_view(request): - next_url = request.route_url('login', _query={'next': request.url}) - return HTTPFound(location=next_url) + request.response.status = 403 + return {} diff --git a/docs/tutorials/wiki2/src/tests/tutorial/views/default.py b/docs/tutorials/wiki2/src/tests/tutorial/views/default.py index de0bcd816..214788357 100644 --- a/docs/tutorials/wiki2/src/tests/tutorial/views/default.py +++ b/docs/tutorials/wiki2/src/tests/tutorial/views/default.py @@ -1,19 +1,19 @@ -from html import escape -import re from docutils.core import publish_parts - -from pyramid.httpexceptions import HTTPFound +from html import escape +from pyramid.httpexceptions import HTTPSeeOther from pyramid.view import view_config +import re from .. import models + # regular expression used to find WikiWords wikiwords = re.compile(r"\b([A-Z]\w+[A-Z]+\w+)") @view_config(route_name='view_wiki') def view_wiki(request): next_url = request.route_url('view_page', pagename='FrontPage') - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) @view_config(route_name='view_page', renderer='tutorial:templates/view.jinja2', permission='view') @@ -39,26 +39,26 @@ def view_page(request): permission='edit') def edit_page(request): page = request.context.page - if 'form.submitted' in request.params: + if request.method == 'POST': page.data = request.params['body'] next_url = request.route_url('view_page', pagename=page.name) - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) return dict( pagename=page.name, pagedata=page.data, save_url=request.route_url('edit_page', pagename=page.name), - ) + ) @view_config(route_name='add_page', renderer='tutorial:templates/edit.jinja2', permission='create') def add_page(request): pagename = request.context.pagename - if 'form.submitted' in request.params: + if request.method == 'POST': body = request.params['body'] page = models.Page(name=pagename, data=body) page.creator = request.user request.dbsession.add(page) next_url = request.route_url('view_page', pagename=pagename) - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) save_url = request.route_url('add_page', pagename=pagename) return dict(pagename=pagename, pagedata='', save_url=save_url) diff --git a/docs/tutorials/wiki2/src/views/.gitignore b/docs/tutorials/wiki2/src/views/.gitignore index 1853d983c..c612e59f2 100644 --- a/docs/tutorials/wiki2/src/views/.gitignore +++ b/docs/tutorials/wiki2/src/views/.gitignore @@ -19,3 +19,4 @@ Data.fs* .DS_Store coverage test +*.sqlite diff --git a/docs/tutorials/wiki2/src/views/setup.py b/docs/tutorials/wiki2/src/views/setup.py index 500c5e599..12eabaff2 100644 --- a/docs/tutorials/wiki2/src/views/setup.py +++ b/docs/tutorials/wiki2/src/views/setup.py @@ -20,8 +20,8 @@ requires = [ 'pyramid_tm', 'SQLAlchemy', 'transaction', - 'zope.sqlalchemy', 'waitress', + 'zope.sqlalchemy', ] tests_require = [ diff --git a/docs/tutorials/wiki2/src/views/testing.ini b/docs/tutorials/wiki2/src/views/testing.ini new file mode 100644 index 000000000..5caa1a8dc --- /dev/null +++ b/docs/tutorials/wiki2/src/views/testing.ini @@ -0,0 +1,79 @@ +### +# app configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/environment.html +### + +[app:main] +use = egg:tutorial + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en + +sqlalchemy.url = sqlite:///%(here)s/testing.sqlite + +retry.attempts = 3 + +[pshell] +setup = tutorial.pshell.setup + +### +# wsgi server configuration +### + +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + +[server:main] +use = egg:waitress#main +listen = localhost:6543 + +### +# logging configuration +# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html +### + +[loggers] +keys = root, tutorial, sqlalchemy, alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_tutorial] +level = DEBUG +handlers = +qualname = tutorial + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[logger_alembic] +level = WARN +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s diff --git a/docs/tutorials/wiki2/src/views/tests/conftest.py b/docs/tutorials/wiki2/src/views/tests/conftest.py new file mode 100644 index 000000000..2db65f887 --- /dev/null +++ b/docs/tutorials/wiki2/src/views/tests/conftest.py @@ -0,0 +1,125 @@ +import alembic +import alembic.config +import alembic.command +import os +from pyramid.paster import get_appsettings +from pyramid.scripting import prepare +from pyramid.testing import DummyRequest +import pytest +import transaction +from webob.cookies import Cookie +import webtest + +from tutorial import main +from tutorial import models +from tutorial.models.meta import Base + + +def pytest_addoption(parser): + parser.addoption('--ini', action='store', metavar='INI_FILE') + +@pytest.fixture(scope='session') +def ini_file(request): + # potentially grab this path from a pytest option + return os.path.abspath(request.config.option.ini or 'testing.ini') + +@pytest.fixture(scope='session') +def app_settings(ini_file): + return get_appsettings(ini_file) + +@pytest.fixture(scope='session') +def dbengine(app_settings, ini_file): + engine = models.get_engine(app_settings) + + alembic_cfg = alembic.config.Config(ini_file) + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + + # run migrations to initialize the database + # depending on how we want to initialize the database from scratch + # we could alternatively call: + # Base.metadata.create_all(bind=engine) + # alembic.command.stamp(alembic_cfg, "head") + alembic.command.upgrade(alembic_cfg, "head") + + yield engine + + Base.metadata.drop_all(bind=engine) + alembic.command.stamp(alembic_cfg, None, purge=True) + +@pytest.fixture(scope='session') +def app(app_settings, dbengine): + return main({}, dbengine=dbengine, **app_settings) + +@pytest.fixture +def tm(): + tm = transaction.TransactionManager(explicit=True) + tm.begin() + tm.doom() + + yield tm + + tm.abort() + +@pytest.fixture +def dbsession(app, tm): + session_factory = app.registry['dbsession_factory'] + return models.get_tm_session(session_factory, tm) + +@pytest.fixture +def testapp(app, tm, dbsession): + # override request.dbsession and request.tm with our own + # externally-controlled values that are shared across requests but aborted + # at the end + testapp = webtest.TestApp(app, extra_environ={ + 'HTTP_HOST': 'example.com', + 'tm.active': True, + 'tm.manager': tm, + 'app.dbsession': dbsession, + }) + + return testapp + +@pytest.fixture +def app_request(app, tm, dbsession): + """ + A real request. + + This request is almost identical to a real request but it has some + drawbacks in tests as it's harder to mock data and is heavier. + + """ + env = prepare(registry=app.registry) + request = env['request'] + request.host = 'example.com' + + # without this, request.dbsession will be joined to the same transaction + # manager but it will be using a different sqlalchemy.orm.Session using + # a separate database transaction + request.dbsession = dbsession + request.tm = tm + + yield request + env['closer']() + +@pytest.fixture +def dummy_request(app, tm, dbsession): + """ + A lightweight dummy request. + + This request is ultra-lightweight and should be used only when the + request itself is not a large focus in the call-stack. + + It is way easier to mock and control side-effects using this object. + + - It does not have request extensions applied. + - Threadlocals are not properly pushed. + + """ + request = DummyRequest() + request.registry = app.registry + request.host = 'example.com' + request.dbsession = dbsession + request.tm = tm + + return request diff --git a/docs/tutorials/wiki2/src/views/tests/test_functional.py b/docs/tutorials/wiki2/src/views/tests/test_functional.py new file mode 100644 index 000000000..dbcd8aec7 --- /dev/null +++ b/docs/tutorials/wiki2/src/views/tests/test_functional.py @@ -0,0 +1,13 @@ +from tutorial import models + +def test_my_view_success(testapp, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + res = testapp.get('/', status=200) + assert res.body + +def test_notfound(testapp): + res = testapp.get('/badurl', status=404) + assert res.status_code == 404 diff --git a/docs/tutorials/wiki2/src/views/tests/test_it.py b/docs/tutorials/wiki2/src/views/tests/test_it.py deleted file mode 100644 index ea16534fc..000000000 --- a/docs/tutorials/wiki2/src/views/tests/test_it.py +++ /dev/null @@ -1,66 +0,0 @@ -import unittest - -from pyramid import testing - -import transaction - - -def dummy_request(dbsession): - return testing.DummyRequest(dbsession=dbsession) - - -class BaseTest(unittest.TestCase): - def setUp(self): - self.config = testing.setUp(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) - self.config.include('tutorial.models') - settings = self.config.get_settings() - - from tutorial.models import ( - get_engine, - get_session_factory, - get_tm_session, - ) - - self.engine = get_engine(settings) - session_factory = get_session_factory(self.engine) - - self.session = get_tm_session(session_factory, transaction.manager) - - def init_database(self): - from tutorial.models.meta import Base - Base.metadata.create_all(self.engine) - - def tearDown(self): - from tutorial.models.meta import Base - - testing.tearDown() - transaction.abort() - Base.metadata.drop_all(self.engine) - - -class TestMyViewSuccessCondition(BaseTest): - - def setUp(self): - super(TestMyViewSuccessCondition, self).setUp() - self.init_database() - - from tutorial.models import MyModel - - model = MyModel(name='one', value=55) - self.session.add(model) - - def test_passing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info['one'].name, 'one') - self.assertEqual(info['project'], 'myproj') - - -class TestMyViewFailureCondition(BaseTest): - - def test_failing_view(self): - from tutorial.views.default import my_view - info = my_view(dummy_request(self.session)) - self.assertEqual(info.status_int, 500) diff --git a/docs/tutorials/wiki2/src/views/tests/test_views.py b/docs/tutorials/wiki2/src/views/tests/test_views.py new file mode 100644 index 000000000..8ae464d03 --- /dev/null +++ b/docs/tutorials/wiki2/src/views/tests/test_views.py @@ -0,0 +1,23 @@ +from tutorial import models +from tutorial.views.default import my_view +from tutorial.views.notfound import notfound_view + + +def test_my_view_failure(app_request): + info = my_view(app_request) + assert info.status_int == 500 + +def test_my_view_success(app_request, dbsession): + model = models.MyModel(name='one', value=55) + dbsession.add(model) + dbsession.flush() + + info = my_view(app_request) + assert app_request.response.status_int == 200 + assert info['one'].name == 'one' + assert info['project'] == 'myproj' + +def test_notfound_view(app_request): + info = notfound_view(app_request) + assert app_request.response.status_int == 404 + assert info == {} diff --git a/docs/tutorials/wiki2/src/views/tutorial/__init__.py b/docs/tutorials/wiki2/src/views/tutorial/__init__.py index 5c2ba5cc0..81a22c68c 100644 --- a/docs/tutorials/wiki2/src/views/tutorial/__init__.py +++ b/docs/tutorials/wiki2/src/views/tutorial/__init__.py @@ -5,8 +5,9 @@ def main(global_config, **settings): """ This function returns a Pyramid WSGI application. """ with Configurator(settings=settings) as config: - config.include('.models') config.include('pyramid_jinja2') + config.include('.security') config.include('.routes') + config.include('.models') config.scan() return config.make_wsgi_app() diff --git a/docs/tutorials/wiki2/src/views/tutorial/models/__init__.py b/docs/tutorials/wiki2/src/views/tutorial/models/__init__.py index a4209a6e9..47d77ef01 100644 --- a/docs/tutorials/wiki2/src/views/tutorial/models/__init__.py +++ b/docs/tutorials/wiki2/src/views/tutorial/models/__init__.py @@ -66,13 +66,21 @@ def includeme(config): # use pyramid_retry to retry a request when transient exceptions occur config.include('pyramid_retry') - session_factory = get_session_factory(get_engine(settings)) + # hook to share the dbengine fixture in testing + dbengine = settings.get('dbengine') + if not dbengine: + dbengine = get_engine(settings) + + session_factory = get_session_factory(dbengine) config.registry['dbsession_factory'] = session_factory # make request.dbsession available for use in Pyramid - config.add_request_method( - # r.tm is the transaction manager used by pyramid_tm - lambda r: get_tm_session(session_factory, r.tm), - 'dbsession', - reify=True - ) + def dbsession(request): + # hook to share the dbsession fixture in testing + dbsession = request.environ.get('app.dbsession') + if dbsession is None: + # request.tm is the transaction manager used by pyramid_tm + dbsession = get_tm_session(session_factory, request.tm) + return dbsession + + config.add_request_method(dbsession, reify=True) diff --git a/docs/tutorials/wiki2/src/views/tutorial/scripts/initialize_db.py b/docs/tutorials/wiki2/src/views/tutorial/scripts/initialize_db.py index e6350fb36..c8034e5a5 100644 --- a/docs/tutorials/wiki2/src/views/tutorial/scripts/initialize_db.py +++ b/docs/tutorials/wiki2/src/views/tutorial/scripts/initialize_db.py @@ -8,6 +8,10 @@ from .. import models def setup_models(dbsession): + """ + Add or update models / fixtures in the database. + + """ editor = models.User(name='editor', role='editor') editor.set_password('editor') dbsession.add(editor) diff --git a/docs/tutorials/wiki2/src/views/tutorial/security.py b/docs/tutorials/wiki2/src/views/tutorial/security.py new file mode 100644 index 000000000..216894e07 --- /dev/null +++ b/docs/tutorials/wiki2/src/views/tutorial/security.py @@ -0,0 +1,6 @@ +from pyramid.csrf import CookieCSRFStoragePolicy + + +def includeme(config): + config.set_csrf_storage_policy(CookieCSRFStoragePolicy()) + config.set_default_csrf_options(require_csrf=True) diff --git a/docs/tutorials/wiki2/src/views/tutorial/templates/404.jinja2 b/docs/tutorials/wiki2/src/views/tutorial/templates/404.jinja2 index aaf12413f..5edb15285 100644 --- a/docs/tutorials/wiki2/src/views/tutorial/templates/404.jinja2 +++ b/docs/tutorials/wiki2/src/views/tutorial/templates/404.jinja2 @@ -1,8 +1,6 @@ {% extends "layout.jinja2" %} {% block content %} -<div class="content"> - <h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> - <p class="lead"><span class="font-semi-bold">404</span> Page Not Found</p> -</div> +<h1><span class="font-semi-bold">Pyramid</span> <span class="smaller">Starter project</span></h1> +<p class="lead"><span class="font-semi-bold">404</span> Page Not Found</p> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/views/tutorial/templates/edit.jinja2 b/docs/tutorials/wiki2/src/views/tutorial/templates/edit.jinja2 index 7db25c674..27b545054 100644 --- a/docs/tutorials/wiki2/src/views/tutorial/templates/edit.jinja2 +++ b/docs/tutorials/wiki2/src/views/tutorial/templates/edit.jinja2 @@ -10,11 +10,12 @@ Editing <strong>{{pagename}}</strong> <a href="{{request.route_url('view_page', pagename='FrontPage')}}">FrontPage</a>. </p> <form action="{{ save_url }}" method="post"> +<input type="hidden" name="csrf_token" value="{{ get_csrf_token() }}"> <div class="form-group"> <textarea class="form-control" name="body" rows="10" cols="60">{{ pagedata }}</textarea> </div> <div class="form-group"> - <button type="submit" name="form.submitted" value="Save" class="btn btn-default">Save</button> + <button type="submit" class="btn btn-default">Save</button> </div> </form> {% endblock content %} diff --git a/docs/tutorials/wiki2/src/views/tutorial/templates/layout.jinja2 b/docs/tutorials/wiki2/src/views/tutorial/templates/layout.jinja2 index 80062cbff..17e8f7688 100644 --- a/docs/tutorials/wiki2/src/views/tutorial/templates/layout.jinja2 +++ b/docs/tutorials/wiki2/src/views/tutorial/templates/layout.jinja2 @@ -38,6 +38,15 @@ </div> </div> <div class="row"> + <div class="links"> + <ul> + <li><i class="glyphicon glyphicon-cog icon-muted"></i><a href="https://github.com/Pylons/pyramid">Github Project</a></li> + <li><i class="glyphicon glyphicon-globe icon-muted"></i><a href="https://webchat.freenode.net/?channels=pyramid">IRC Channel</a></li> + <li><i class="glyphicon glyphicon-home icon-muted"></i><a href="https://pylonsproject.org">Pylons Project</a></li> + </ul> + </div> + </div> + <div class="row"> <div class="copyright"> Copyright © Pylons Project </div> diff --git a/docs/tutorials/wiki2/src/views/tutorial/views/default.py b/docs/tutorials/wiki2/src/views/tutorial/views/default.py index 867ba3f6c..df0e4cb9e 100644 --- a/docs/tutorials/wiki2/src/views/tutorial/views/default.py +++ b/docs/tutorials/wiki2/src/views/tutorial/views/default.py @@ -1,23 +1,22 @@ -from html import escape -import re from docutils.core import publish_parts - +from html import escape from pyramid.httpexceptions import ( - HTTPFound, HTTPNotFound, - ) - + HTTPSeeOther, +) from pyramid.view import view_config +import re from .. import models + # regular expression used to find WikiWords wikiwords = re.compile(r"\b([A-Z]\w+[A-Z]+\w+)") @view_config(route_name='view_wiki') def view_wiki(request): next_url = request.route_url('view_page', pagename='FrontPage') - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) @view_config(route_name='view_page', renderer='tutorial:templates/view.jinja2') def view_page(request): @@ -45,29 +44,29 @@ def view_page(request): def edit_page(request): pagename = request.matchdict['pagename'] page = request.dbsession.query(models.Page).filter_by(name=pagename).one() - if 'form.submitted' in request.params: + if request.method == 'POST': page.data = request.params['body'] next_url = request.route_url('view_page', pagename=page.name) - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) return dict( pagename=page.name, pagedata=page.data, save_url=request.route_url('edit_page', pagename=page.name), - ) + ) @view_config(route_name='add_page', renderer='tutorial:templates/edit.jinja2') def add_page(request): pagename = request.matchdict['pagename'] if request.dbsession.query(models.Page).filter_by(name=pagename).count() > 0: next_url = request.route_url('edit_page', pagename=pagename) - return HTTPFound(location=next_url) - if 'form.submitted' in request.params: + return HTTPSeeOther(location=next_url) + if request.method == 'POST': body = request.params['body'] page = models.Page(name=pagename, data=body) page.creator = ( request.dbsession.query(models.User).filter_by(name='editor').one()) request.dbsession.add(page) next_url = request.route_url('view_page', pagename=pagename) - return HTTPFound(location=next_url) + return HTTPSeeOther(location=next_url) save_url = request.route_url('add_page', pagename=pagename) return dict(pagename=pagename, pagedata='', save_url=save_url) diff --git a/docs/tutorials/wiki2/tests.rst b/docs/tutorials/wiki2/tests.rst index c7d1a0f31..8a3e79363 100644 --- a/docs/tutorials/wiki2/tests.rst +++ b/docs/tutorials/wiki2/tests.rst @@ -8,101 +8,142 @@ We will now add tests for the models and views as well as a few functional tests in a new ``tests`` package. Tests ensure that an application works, and that it continues to work when changes are made in the future. -The file ``tests/test_it.py`` at the root of our project directory was generated from choosing the ``sqlalchemy`` backend option. + +Test harness +============ + +The project came bootstrapped with some tests and a basic harness. +These are located in the ``tests`` package at the top-level of the project. It is a common practice to put tests into a ``tests`` package alongside the application package, especially as projects grow in size and complexity. -Each module in the test package should contain tests for its corresponding module in our application. -Each corresponding pair of modules should have the same names, except the test module should have the prefix ``test_``. +A useful convention is for each module in the application to contain a corresponding module in the ``tests`` package. +The test module would have the same name with the prefix ``test_``. -Start by deleting ``tests/test_it.py``. +The harness consists of the following setup: -.. warning:: +- ``pytest.ini`` - controls basic ``pytest`` config including where to find the tests. + We have configured ``pytest`` to search for tests in the application package and in the ``tests`` package. - It is very important when refactoring a Python module into a package to be - sure to delete the cache files (``.pyc`` files or ``__pycache__`` folders) - sitting around! Python will prioritize the cache files before traversing - into folders, using the old code, and you will wonder why none of your - changes are working! +- ``.coveragerc`` - controls coverage config. + In our setup, it works with the ``pytest-cov`` plugin that we use via the ``--cov`` options to the ``pytest`` command. +- ``testing.ini`` - a mirror of ``development.ini`` and ``production.ini`` that contains settings used for executing the test suite. + Most importantly, it contains the database connection information used by tests that require the database. -Test the views -============== +- ``tests_require`` in ``setup.py`` - controls the dependencies installed when testing. + When the list is changed, it's necessary to re-run ``$VENV/bin/pip install -e ".[testing]"`` to ensure the new dependencies are installed. -We'll create a new ``tests/test_views.py`` file, adding a ``BaseTest`` class -used as the base for other test classes. Next we'll add tests for each view -function we previously added to our application. We'll add four test classes: -``ViewWikiTests``, ``ViewPageTests``, ``AddPageTests``, and ``EditPageTests``. -These test the ``view_wiki``, ``view_page``, ``add_page``, and ``edit_page`` -views. +- ``tests/conftest.py`` - the core fixtures available throughout our tests. + The fixtures are explained in more detail below. -Functional tests -================ +Session-scoped test fixtures +---------------------------- -We'll test the whole application, covering security aspects that are not tested -in the unit tests, like logging in, logging out, checking that the ``basic`` -user cannot edit pages that it didn't create but the ``editor`` user can, and -so on. +- ``app_settings`` - the settings ``dict`` parsed from the ``testing.ini`` file that would normally be passed by ``pserve`` into your app's ``main`` function. +- ``dbengine`` - initializes the database. + It's important to start each run of the test suite from a known state, and this fixture is responsible for preparing the database appropriately. + This includes deleting any existing tables, running migrations, and potentially even loading some fixture data into the tables for use within the tests. -View the results of all our edits to ``tests`` package -====================================================== +- ``app`` - the :app:`Pyramid` WSGI application, implementing the :class:`pyramid.interfaces.IRouter` interface. + Most commonly this would be used for functional tests. -Create ``tests/test_views.py`` such that it appears as follows: -.. literalinclude:: src/tests/tests/test_views.py - :linenos: - :language: python +Per-test fixtures +----------------- -Create ``tests/test_functional.py`` such that it appears as follows: +- ``tm`` - a :class:`transaction.TransactionManager` object controlling a transaction lifecycle. + Generally other fixtures would join to the ``tm`` fixture to control their lifecycle and ensure they are aborted at the end of the test. -.. literalinclude:: src/tests/tests/test_functional.py - :linenos: - :language: python +- ``dbsession`` - a :class:`sqlalchemy.orm.session.Session` object connected to the database. + The session is scoped to the ``tm`` fixture. + Any changes made will be aborted at the end of the test. -Create ``tests/test_initdb.py`` such that it appears as follows: +- ``testapp`` - a :class:`webtest.TestApp` instance wrapping the ``app`` and is used to sending requests into the application and return full response objects that can be inspected. + The ``testapp`` is able to mutate the request environ such that the ``dbsession`` and ``tm`` fixtures are injected and used by any code that's touching ``request.dbsession`` and ``request.tm``. + The ``testapp`` maintains a cookiejar, so it can be used to share state across requests, as well as the transaction database connection. -.. literalinclude:: src/tests/tests/test_initdb.py - :linenos: - :language: python +- ``app_request`` - a :class:`pyramid.request.Request` object that can be used for more lightweight tests versus the full ``testapp``. + The ``app_request`` can be passed to view functions and other code that need a fully functional request object. + +- ``dummy_request`` - a :class:`pyramid.testing.DummyRequest` object that is very lightweight. + This is a great object to pass to view functions that have minimal side-effects as it'll be fast and simple. + + +Modifying the fixtures +---------------------- -Create ``tests/test_security.py`` such that it appears as follows: +We're going to make a few application-specific changes to the test harness. +It's always good to come up with patterns for things that are done often to avoid lots of boilerplate. -.. literalinclude:: src/tests/tests/test_security.py +- Initialize the cookiejar with a CSRF token. + Remember our application is using :class:`pyramid.csrf.CookieCSRFStoragePolicy`. + +- ``testapp.get_csrf_token()`` - every POST/PUT/DELETE/PATCH request must contain the current CSRF token to prove to our app that the client isn't a third-party. + So we want an easy way to grab the current CSRF token and add it to the request. + +- ``testapp.login(params)`` - many pages are only accessible by logged in users so we want a simple way to login a user at the start of a test. + +Update ``tests/conftest.py`` to look like the following, adding the highlighted lines: + +.. literalinclude:: src/tests/tests/conftest.py :linenos: + :emphasize-lines: 10,68-103,110,117-119 :language: python + +Unit tests +========== + +We can test individual APIs within our codebase to ensure they fulfill the expected contract that the rest of the application expects. +For example, we'll test the password hashing features we added to the ``tutorial.models.User`` object. + Create ``tests/test_user_model.py`` such that it appears as follows: .. literalinclude:: src/tests/tests/test_user_model.py :linenos: :language: python -.. note:: - We're utilizing the excellent WebTest_ package to do functional testing of - the application. This is defined in the ``tests_require`` section of our - ``setup.py``. Any other dependencies needed only for testing purposes can be - added there and will be installed automatically when running - ``setup.py test``. +Integration tests +================= + +We can directly execute the view code, bypassing :app:`Pyramid` and testing just the code that we've written. +These tests use dummy requests that we'll prepare appropriately to set the conditions each view expects. +For example, setting ``request.user``, or adding some dummy data to the session. + +Update ``tests/test_views.py`` such that it appears as follows: + +.. literalinclude:: src/tests/tests/test_views.py + :linenos: + :language: python + + +Functional tests +================ + +We'll test the whole application, covering security aspects that are not tested in the unit and integration tests, like logging in, logging out, checking that the ``basic`` user cannot edit pages that it didn't create but the ``editor`` user can, and so on. + +Update ``tests/test_functional.py`` such that it appears as follows: + +.. literalinclude:: src/tests/tests/test_functional.py + :linenos: + :language: python Running the tests ================= -We can run these tests similarly to how we did in :ref:`running_tests`, but first delete the SQLite database ``tutorial.sqlite``. If you do not delete the database, then you will see an integrity error when running the tests. - On Unix: .. code-block:: bash - rm tutorial.sqlite $VENV/bin/pytest -q On Windows: .. code-block:: doscon - del tutorial.sqlite %VENV%\Scripts\pytest -q The expected result should look like the following: |