Merge pull request #3421 from mmerickel/drop-py2

remove py2 from the codebase

11 files changed, 61 insertions, 91 deletions

diff --git a/docs/narr/commandline.rst b/docs/narr/commandline.rst
index b571a7d7b..21b2a0839 100644
--- a/docs/narr/commandline.rst
+++ b/docs/narr/commandline.rst
@@ -1006,8 +1006,8 @@ top-level directory, your ``setup.py`` file will look something like this:
     requires = ['pyramid', 'pyramid_debugtoolbar']
 
     tests_require = [
-        'WebTest >= 1.3.1',  # py3 compat
-        'pytest',  # includes virtualenv
+        'WebTest',
+        'pytest',
         'pytest-cov',
     ]
 
@@ -1073,8 +1073,8 @@ The result will be something like:
     requires = ['pyramid', 'pyramid_debugtoolbar']
 
     tests_require = [
-        'WebTest >= 1.3.1',  # py3 compat
-        'pytest',  # includes virtualenv
+        'WebTest',
+        'pytest',
         'pytest-cov',
     ]
 
diff --git a/docs/narr/hooks.rst b/docs/narr/hooks.rst
index 9f405c336..0dac8d426 100644
--- a/docs/narr/hooks.rst
+++ b/docs/narr/hooks.rst
@@ -659,15 +659,15 @@ that implements the following interface:
             ``virtual_root``, and ``virtual_root_path``.  These values are
             typically the result of a resource tree traversal.  ``root``
             is the physical root object, ``context`` will be a resource
-            object, ``view_name`` will be the view name used (a Unicode
-            name), ``subpath`` will be a sequence of Unicode names that
+            object, ``view_name`` will be the view name used (a string),
+            ``subpath`` will be a sequence of strings that
             followed the view name but were not traversed, ``traversed``
-            will be a sequence of Unicode names that were traversed
+            will be a sequence of strings that were traversed
             (including the virtual root path, if any) ``virtual_root``
             will be a resource object representing the virtual root (or the
             physical root if traversal was not performed), and
             ``virtual_root_path`` will be a sequence representing the
-            virtual root path (a sequence of Unicode names) or None if
+            virtual root path (a sequence of strings) or ``None`` if
             traversal was not performed.
 
             Extra keys for special purpose functionality can be added as
diff --git a/docs/narr/i18n.rst b/docs/narr/i18n.rst
index 9b838c7f4..b8cd396c0 100644
--- a/docs/narr/i18n.rst
+++ b/docs/narr/i18n.rst
@@ -33,7 +33,7 @@ While you write your software, you can insert specialized markup into your
 Python code that makes it possible for the system to translate text values into
 the languages used by your application's users.  This markup creates a
 :term:`translation string`.  A translation string is an object that behaves
-mostly like a normal Unicode object, except that it also carries around extra
+mostly like a normal Unicode string, except that it also carries around extra
 information related to its job as part of the :app:`Pyramid` translation
 machinery.
 
@@ -49,7 +49,7 @@ The most primitive way to create a translation string is to use the
     from pyramid.i18n import TranslationString
     ts = TranslationString('Add')
 
-This creates a Unicode-like object that is a TranslationString.
+This creates a ``str``-like object that is a TranslationString.
 
 .. note::
 
@@ -61,9 +61,8 @@ This creates a Unicode-like object that is a TranslationString.
 
 The first argument to :class:`~pyramid.i18n.TranslationString` is the
 ``msgid``; it is required.  It represents the key into the translation mappings
-provided by a particular localization. The ``msgid`` argument must be a Unicode
-object or an ASCII string.  The msgid may optionally contain *replacement
-markers*.  For instance:
+provided by a particular localization. The ``msgid`` argument must be a string.
+The ``msgid`` may optionally contain *replacement markers*.  For instance:
 
 .. code-block:: python
     :linenos:
@@ -81,14 +80,14 @@ may be supplied at the same time as the replacement marker itself:
     from pyramid.i18n import TranslationString
     ts = TranslationString('Add ${number}', mapping={'number':1})
 
-Any number of replacement markers can be present in the msgid value, any number
+Any number of replacement markers can be present in the ``msgid`` value, any number
 of times.  Only markers which can be replaced by the values in the *mapping*
 will be replaced at translation time.  The others will not be interpolated and
 will be output literally.
 
 A translation string should also usually carry a *domain*.  The domain
 represents a translation category to disambiguate it from other translations of
-the same msgid, in case they conflict.
+the same ``msgid``, in case they conflict.
 
 .. code-block:: python
     :linenos:
@@ -100,7 +99,7 @@ the same msgid, in case they conflict.
 The above translation string named a domain of ``form``.  A :term:`translator`
 function will often use the domain to locate the right translator file on the
 filesystem which contains translations for a given domain.  In this case, if it
-were trying to translate our msgid to German, it might try to find a
+were trying to translate our ``msgid`` to German, it might try to find a
 translation from a :term:`gettext` file within a :term:`translation directory`
 like this one:
 
@@ -429,7 +428,7 @@ Performing a Translation
 
 A :term:`localizer` has a ``translate`` method which accepts either a
 :term:`translation string` or a Unicode string and which returns a Unicode
-object representing the translation.  Generating a translation in a view
+string representing the translation.  Generating a translation in a view
 component of an application might look like so:
 
 .. code-block:: python
diff --git a/docs/narr/install.rst b/docs/narr/install.rst
index 248b432d3..268ae5f8d 100644
--- a/docs/narr/install.rst
+++ b/docs/narr/install.rst
@@ -21,8 +21,7 @@ the following sections.
 
 .. sidebar:: Python Versions
 
-    As of this writing, :app:`Pyramid` is tested against Python 2.7,
-    Python 3.4, Python 3.5, Python 3.6, Python 3.7, and PyPy.
+    As of this writing, :app:`Pyramid` is tested against Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 (with allowed failures), and PyPy3.
 
 :app:`Pyramid` is known to run on all popular Unix-like systems such as Linux,
 macOS, and FreeBSD, as well as on Windows platforms.  It is also known to
diff --git a/docs/narr/myproject/setup.py b/docs/narr/myproject/setup.py
index cf626880f..1ee272270 100644
--- a/docs/narr/myproject/setup.py
+++ b/docs/narr/myproject/setup.py
@@ -17,8 +17,8 @@ requires = [
 ]
 
 tests_require = [
-    'WebTest >= 1.3.1',  # py3 compat
-    'pytest>=3.7.4',
+    'WebTest',
+    'pytest',
     'pytest-cov',
 ]
 
diff --git a/docs/narr/renderers.rst b/docs/narr/renderers.rst
index 493f808d5..6b4982e4b 100644
--- a/docs/narr/renderers.rst
+++ b/docs/narr/renderers.rst
@@ -145,8 +145,7 @@ used in the ``renderer`` attribute of view configurations.
 The ``string`` renderer renders a view callable result to a string.  If a view
 callable returns a non-Response object, and the ``string`` renderer is
 associated in that view's configuration, the result will be to run the object
-through the Python ``str`` function to generate a string.  Note that if a
-Unicode object is returned by the view callable, it is not ``str()``-ified.
+through the Python ``str`` function to generate a string.
 
 Here's an example of a view that returns a dictionary.  If the ``string``
 renderer is specified in the configuration for this view, the view will render
@@ -496,7 +495,7 @@ interface. A typical class that follows this setup is as follows:
         def __call__(self, value, system):
             """ Call the renderer implementation with the value
             and the system value passed in as arguments and return
-            the result (a string or unicode object).  The value is
+            the result (a bytes or string object).  The value is
             the return value of a view.  The system value is a
             dictionary containing available system values
             (e.g., view, context, and request). """
diff --git a/docs/narr/traversal.rst b/docs/narr/traversal.rst
index 9b91e21ba..0282c6096 100644
--- a/docs/narr/traversal.rst
+++ b/docs/narr/traversal.rst
@@ -237,19 +237,19 @@ uses this algorithm to find a :term:`context` resource and a :term:`view name`.
 
     The traversal algorithm by default attempts to first URL-unquote and then
     Unicode-decode each path segment derived from ``PATH_INFO`` from its
-    natural byte string (``str`` type) representation.  URL unquoting is
+    natural string representation.  URL unquoting is
     performed using the Python standard library ``urllib.unquote`` function.
     Conversion from a URL-decoded string into Unicode is attempted using the
     UTF-8 encoding.  If any URL-unquoted path segment in ``PATH_INFO`` is not
     decodeable using the UTF-8 decoding, a :exc:`TypeError` is raised.  A
-    segment will be fully URL-unquoted and UTF8-decoded before it is passed in
+    segment will be fully URL-unquoted and UTF-8-decoded before it is passed in
     to the ``__getitem__`` of any resource during traversal.
 
     Thus a request with a ``PATH_INFO`` variable of ``/a/b/c`` maps to the
-    traversal sequence ``[u'a', u'b', u'c']``.
+    traversal sequence ``['a', 'b', 'c']``.
 
 #.  :term:`Traversal` begins at the root resource returned by the root factory.
-    For the traversal sequence ``[u'a', u'b', u'c']``, the root resource's
+    For the traversal sequence ``['a', 'b', 'c']``, the root resource's
     ``__getitem__`` is called with the name ``'a'``.  Traversal continues
     through the sequence.  In our example, if the root resource's
     ``__getitem__`` called with the name ``a`` returns a resource (a.k.a. 
diff --git a/docs/narr/upgrading.rst b/docs/narr/upgrading.rst
index 12e146cf1..87e4647c3 100644
--- a/docs/narr/upgrading.rst
+++ b/docs/narr/upgrading.rst
@@ -86,7 +86,6 @@ At the time of a Pyramid version release, each supports all versions of Python
 through the end of their lifespans. The end-of-life for a given version of
 Python is when security updates are no longer released.
 
-- `Python 2.7 Lifespan <https://devguide.python.org/#status-of-python-branches>`_ 2020-01-01.
 - `Python 3.4 Lifespan <https://devguide.python.org/#status-of-python-branches>`_ 2019-03-16	.
 - `Python 3.5 Lifespan <https://devguide.python.org/#status-of-python-branches>`_ 2020-09-13	.
 - `Python 3.6 Lifespan <https://devguide.python.org/#status-of-python-branches>`_ 2021-12-23.
diff --git a/docs/narr/urldispatch.rst b/docs/narr/urldispatch.rst
index 3b737b46d..9372163e8 100644
--- a/docs/narr/urldispatch.rst
+++ b/docs/narr/urldispatch.rst
@@ -165,8 +165,8 @@ The above pattern will match these URLs, generating the following matchdicts:
 
 .. code-block:: text
 
-    foo/1/2        -> {'baz':u'1', 'bar':u'2'}
-    foo/abc/def    -> {'baz':u'abc', 'bar':u'def'}
+    foo/1/2        -> {'baz': '1', 'bar': '2'}
+    foo/abc/def    -> {'baz': 'abc', 'bar': 'def'}
 
 It will not match the following patterns however:
 
@@ -184,7 +184,7 @@ instance, if this route pattern was used:
     foo/{name}.html
 
 The literal path ``/foo/biz.html`` will match the above route pattern, and the
-match result will be ``{'name':u'biz'}``.  However, the literal path
+match result will be ``{'name': 'biz'}``.  However, the literal path
 ``/foo/biz`` will not match, because it does not contain a literal ``.html`` at
 the end of the segment represented by ``{name}.html`` (it only contains
 ``biz``, not ``biz.html``).
@@ -242,7 +242,7 @@ The matchdict will look like so (the value is URL-decoded / UTF-8 decoded):
 
 .. code-block:: text
 
-    {'bar':u'La Pe\xf1a'}
+    {'bar': 'La Pe\xf1a'}
 
 Literal strings in the path segment should represent the *decoded* value of the
 ``PATH_INFO`` provided to Pyramid.  You don't want to use a URL-encoded value
@@ -303,10 +303,10 @@ The above pattern will match these URLs, generating the following matchdicts:
 .. code-block:: text
 
     foo/1/2/           ->
-             {'baz':u'1', 'bar':u'2', 'fizzle':()}
+             {'baz': '1', 'bar': '2', 'fizzle': ()}
 
     foo/abc/def/a/b/c  ->
-             {'baz':u'abc', 'bar':u'def', 'fizzle':(u'a', u'b', u'c')}
+             {'baz': 'abc', 'bar': 'def', 'fizzle': ('a', 'b', 'c')}
 
 Note that when a ``*stararg`` remainder match is matched, the value put into
 the matchdict is turned into a tuple of path segments representing the
@@ -327,7 +327,7 @@ Will generate the following matchdict:
 
 .. code-block:: text
 
-    {'fizzle':(u'La Pe\xf1a', u'a', u'b', u'c')}
+    {'fizzle': ('La Pe\xf1a', 'a', 'b', 'c')}
 
 By default, the ``*stararg`` will parse the remainder sections into a tuple
 split by segment. Changing the regular expression used to match a marker can
@@ -341,8 +341,8 @@ The above pattern will match these URLs, generating the following matchdicts:
 
 .. code-block:: text
 
-    foo/1/2/           -> {'baz':u'1', 'bar':u'2', 'fizzle':u''}
-    foo/abc/def/a/b/c  -> {'baz':u'abc', 'bar':u'def', 'fizzle': u'a/b/c'}
+    foo/1/2/           -> {'baz': '1', 'bar': '2', 'fizzle': ''}
+    foo/abc/def/a/b/c  -> {'baz': 'abc', 'bar': 'def', 'fizzle': 'a/b/c'}
 
 This occurs because the default regular expression for a marker is ``[^/]+``
 which will match everything up to the first ``/``, while ``{fizzle:.*}`` will
@@ -513,7 +513,7 @@ When the ``/site/{id}`` route pattern matches during a request, the
 When this route matches, a ``matchdict`` will be generated and attached to the
 request as ``request.matchdict``.  If the specific URL matched is ``/site/1``,
 the ``matchdict`` will be a dictionary with a single key, ``id``; the value
-will be the string ``'1'``, ex.: ``{'id':'1'}``.
+will be the string ``'1'``, ex.: ``{'id': '1'}``.
 
 The ``mypackage.views`` module referred to above might look like so:
 
@@ -562,12 +562,12 @@ Here is an example of a corresponding ``mypackage.views`` module:
     @view_config(route_name='user')
     def user_view(request):
         user = request.matchdict['user']
-        return Response(u'The user is {}.'.format(user))
+        return Response('The user is {}.'.format(user))
 
     @view_config(route_name='tag')
     def tag_view(request):
         tag = request.matchdict['tag']
-        return Response(u'The tag is {}.'.format(tag))
+        return Response('The tag is {}.'.format(tag))
     
 The above configuration will allow :app:`Pyramid` to service URLs in these
 forms:
@@ -581,17 +581,17 @@ forms:
 - When a URL matches the pattern ``/ideas/{idea}``, the view callable
   available at the dotted Python pathname ``mypackage.views.idea_view`` will
   be called.  For the specific URL ``/ideas/1``, the ``matchdict`` generated
-  and attached to the :term:`request` will consist of ``{'idea':'1'}``.
+  and attached to the :term:`request` will consist of ``{'idea': '1'}``.
 
 - When a URL matches the pattern ``/users/{user}``, the view callable
   available at the dotted Python pathname ``mypackage.views.user_view`` will be
   called.  For the specific URL ``/users/1``, the ``matchdict`` generated and
-  attached to the :term:`request` will consist of ``{'user':'1'}``.
+  attached to the :term:`request` will consist of ``{'user': '1'}``.
 
 - When a URL matches the pattern ``/tags/{tag}``, the view callable available
   at the dotted Python pathname ``mypackage.views.tag_view`` will be called.
   For the specific URL ``/tags/1``, the ``matchdict`` generated and attached to
-  the :term:`request` will consist of ``{'tag':'1'}``.
+  the :term:`request` will consist of ``{'tag': '1'}``.
 
 In this example we've again associated each of our routes with a :term:`view
 callable` directly.  In all cases, the request, which will have a ``matchdict``
@@ -714,13 +714,13 @@ Therefore, if you've added a route like so:
 
 .. code-block:: python
 
-    config.add_route('la', u'/La Peña/{city}')
+    config.add_route('la', '/La Peña/{city}')
 
 And you later generate a URL using ``route_path`` or ``route_url`` like so:
 
 .. code-block:: python
 
-    url = request.route_path('la', city=u'Québec')
+    url = request.route_path('la', city='Québec')
 
 You will wind up with the path encoded to UTF-8 and URL-quoted like so:
 
@@ -739,7 +739,7 @@ And you later generate a URL using ``route_path`` or ``route_url`` using a
 
 .. code-block:: python
 
-    url = request.route_path('abc', foo=u'Québec/biz')
+    url = request.route_path('abc', foo='Québec/biz')
 
 The value you pass will be URL-quoted except for embedded slashes in the
 result:
@@ -752,7 +752,7 @@ You can get a similar result by passing a tuple composed of path elements:
 
 .. code-block:: python
 
-    url = request.route_path('abc', foo=(u'Québec', u'biz'))
+    url = request.route_path('abc', foo=('Québec', 'biz'))
 
 Each value in the tuple will be URL-quoted and joined by slashes in this case:
 
diff --git a/docs/narr/views.rst b/docs/narr/views.rst
index a53063f78..1b4118b85 100644
--- a/docs/narr/views.rst
+++ b/docs/narr/views.rst
@@ -442,10 +442,7 @@ browser client, and its ``action`` points at some :app:`Pyramid` view code:
     :linenos:
 
     <html xmlns="http://www.w3.org/1999/xhtml">
-      <head>
-        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
-      </head>
-      <form method="POST" action="myview">
+      <form method="POST" action="myview" accept-charset="UTF-8">
         <div>
           <input type="text" name="firstname"/>
         </div>
@@ -457,8 +454,8 @@ browser client, and its ``action`` points at some :app:`Pyramid` view code:
     </html>
 
 The ``myview`` view code in the :app:`Pyramid` application *must* expect that
-the values returned by ``request.params`` will be of type ``unicode``, as
-opposed to type ``str``. The following will work to accept a form post from the
+the values returned by ``request.params`` will be of type ``str``, as opposed
+to type ``bytes``. The following will work to accept a form post from the
 above form:
 
 .. code-block:: python
@@ -468,24 +465,12 @@ above form:
         firstname = request.params['firstname']
         lastname = request.params['lastname']
 
-But the following ``myview`` view code *may not* work, as it tries to decode
-already-decoded (``unicode``) values obtained from ``request.params``:
-
-.. code-block:: python
-    :linenos:
-
-    def myview(request):
-        # the .decode('utf-8') will break below if there are any high-order
-        # characters in the firstname or lastname
-        firstname = request.params['firstname'].decode('utf-8')
-        lastname = request.params['lastname'].decode('utf-8')
-
 For implicit decoding to work reliably, you should ensure that every form you
 render that posts to a :app:`Pyramid` view explicitly defines a charset
 encoding of UTF-8. This can be done via a response that has a
 ``;charset=UTF-8`` in its ``Content-Type`` header; or, as in the form above,
-with a ``meta http-equiv`` tag that implies that the charset is UTF-8 within
-the HTML ``head`` of the page containing the form.  This must be done
+with an ``accept-charset`` attribute, informing the browser that the
+server expects the form content to be encoded using UTF-8.  This must be done
 explicitly because all known browser clients assume that they should encode
 form data in the same character set implied by the ``Content-Type`` value of
 the response containing the form when subsequently submitting that form. There
@@ -499,21 +484,15 @@ when it can't decode some high-order character encoded in another character set
 within form data, e.g., when ``request.params['somename']`` is accessed.
 
 If you are using the :class:`~pyramid.response.Response` class to generate a
-response, or if you use the ``render_template_*`` templating APIs, the UTF-8
-``charset`` is set automatically as the default via the ``Content-Type``
-header. If you return a ``Content-Type`` header without an explicit
-``charset``, a request will add a ``;charset=utf-8`` trailer to the
+response, or if you use the ``pyramid.renderers.render_*`` templating APIs,
+the UTF-8 ``charset`` is set automatically as the default via the
+``Content-Type`` header. If you return a ``Content-Type`` header without an
+explicit ``charset``, a request will add a ``;charset=utf-8`` trailer to the
 ``Content-Type`` header value for you for response content types that are
 textual (e.g., ``text/html`` or ``application/xml``) as it is rendered.  If you
 are using your own response object, you will need to ensure you do this
 yourself.
 
-.. note:: Only the *values* of request params obtained via ``request.params``,
-   ``request.GET`` or ``request.POST`` are decoded to Unicode objects
-   implicitly in the :app:`Pyramid` default configuration.  The keys are still
-   (byte) strings.
-
-
 .. index::
    single: view calling convention
 
diff --git a/docs/narr/webob.rst b/docs/narr/webob.rst
index 89dc6e0f1..665bbddc9 100644
--- a/docs/narr/webob.rst
+++ b/docs/narr/webob.rst
@@ -188,15 +188,10 @@ of them.  Here are a couple that might be useful:
 Text (Unicode)
 ++++++++++++++
 
-Many of the properties of the request object will be text values (``unicode``
-under Python 2 or ``str`` under Python 3) if the request encoding/charset is
-provided.  If it is provided, the values in ``req.POST``, ``req.GET``,
-``req.params``, and ``req.cookies`` will contain text.  The client *can*
-indicate the charset with something like ``Content-Type:
-application/x-www-form-urlencoded; charset=utf8``, but browsers seldom set
-this.  You can reset the charset of an existing request with ``newreq =
-req.decode('utf-8')``, or during instantiation with ``Request(environ,
-charset='utf8')``.
+Most of the properties of the request object will be text values.
+The values in ``req.POST``, ``req.GET``, ``req.params``, and ``req.cookies`` will contain text and are generated assuming a UTF-8 charset.
+The client *can* indicate the charset with something like ``Content-Type: application/x-www-form-urlencoded; charset=utf8``, but browsers seldom set this.
+You can reset the charset of an existing request with ``newreq = req.decode('utf-8')``, or during instantiation with ``Request(environ, charset='utf8')``.
 
 .. index::
    single: multidict (WebOb)
@@ -264,7 +259,7 @@ to a :app:`Pyramid` application:
     jQuery.ajax({type:'POST',
                  url: 'http://localhost:6543/', // the pyramid server
                  data: JSON.stringify({'a':1}),
-                 contentType: 'application/json; charset=utf-8'});
+                 contentType: 'application/json'});
 
 When such a request reaches a view in your application, the
 ``request.json_body`` attribute will be available in the view callable body.
@@ -280,7 +275,7 @@ For the above view, printed to the console will be:
 
 .. code-block:: python
 
-    {u'a': 1}
+    {'a': 1}
 
 For bonus points, here's a bit of client-side code that will produce a request
 that has a body suitable for reading via ``request.json_body`` using Python's
@@ -386,8 +381,8 @@ A response object has three fundamental parts:
 
 ``response.app_iter``
     An iterable (such as a list or generator) that will produce the content of
-    the response.  This is also accessible as ``response.body`` (a string),
-    ``response.text`` (a unicode object, informed by ``response.charset``), and
+    the response.  This is also accessible as ``response.body`` (bytes),
+    ``response.text`` (a Unicode string, informed by ``response.charset``), and
     ``response.body_file`` (a file-like object; writing to it appends to
     ``app_iter``).