.. _appengine_tutorial: Running :mod:`repoze.bfg` on Google's App Engine ================================================ As of :mod:`repoze.bfg` version 0.8, it is possible to run a :mod:`repoze.bfg` application on Google's `App Engine `_. Content from this tutorial was contributed by YoungKing, based on the `"appengine-monkey" tutorial for Pylons `_. This tutorial is written in terms of using the command line on a UNIX system; it should be possible to perform similar actions on a Windows system. .. note:: :term:`chameleon.zpt` cannot easily be used on Google App Engine due to GAE environment limitations, so the tutorial is presented in terms of using :term:`Jinja2` as the templating language in the generated BFG application. It is possible to *use* Chameleon templates on GAE, it's just not possible to *compile* them under GAE. #. Download Google's `App Engine SDK `_ and install it on your system. #. Use Subversion to check out the source code for ``appengine-monkey``. .. code-block:: bash :linenos: $ svn co http://appengine-monkey.googlecode.com/svn/trunk/ appengine-monkey #. Use ``appengine_homedir.py`` script in ``appengine-monkey`` to create a :term:`virtualenv` for your application. .. code-block:: bash :linenos: $ export GAE_PATH=/usr/local/google_appengine $ python2.5 /path/to/appengine-monkey/appengine-homedir.py --gae $GAE_PATH bfgapp Note that ``$GAE_PATH`` should be the path where you have unpacked the App Engine SDK. (On Mac OS X at least, ``/usr/local/google_appengine`` is indeed where the installer puts it). This will set up an environment in ``bfgapp/``, with some tools installed in ``bfgapp/bin``. There will also be a directory ``bfgapp/app/`` which is the directory you will upload to appengine. #. Install :mod:`repoze.bfg.jinja2` into the virtualenv .. code-block:: bash :linenos: $ cd bfgapp/ $ bin/easy_install -i http://dist.repoze.org/bfg/current/simple/ repoze.bfg.jinja2 This will install :mod:`repoze.bfg` in the environment. #. Create your application We'll use the standard way to create a :mod:`repoze.bfg` application, but we'll have to move some files around when we are done. The below commands assume your current working directory is the ``bfgapp`` virtualenv director you created in the third step above: .. code-block:: bash :linenos: $ cd app $ rm -rf bfgapp $ bin/paster create -t bfg_jinja2_starter bfgapp $ mv bfgapp aside $ mv aside/bfgapp . $ rm -rf aside #. Edit ``config.py`` Edit the ``APP_NAME`` and ``APP_ARGS`` settings within ``config.py``. The APP_NAME must be ``bfgapp.run:make_app``, and the APP_ARGS must be ``({},)``. Any other settings in ``config.py`` should remain the same. .. code-block:: python :linenos: APP_NAME = 'bfgapp.run:app' APP_ARGS = ({},) .. warning:: It's very important that the ``APP_NAME`` is ``bfgapp.run:app`` instead of ``bfgapp.run.make_app``. If you use the latter by mistake, you'll be confused for at least an hour when debugging an error like this: ``IOError: [Errno 2] No such file or directory: '/Users/chrism/projects/bfg_gae/bfgapp2/app/configure.zcml'``. #. Edit ``runner.py`` To prevent errors for ``import site``, add this code stanza before ``import site`` in app/runner.py: .. code-block:: python :linenos: import sys sys.path = [path for path in sys.path if 'site-packages' not in path] import site You will also need to comment out the line that starts with ``assert sys.path`` in the file. .. code-block:: python :linenos: # comment the sys.path assertion out # assert sys.path[:len(cur_sys_path)] == cur_sys_path, ( # "addsitedir() caused entries to be prepended to sys.path") #. Run the application. ``dev_appserver.py`` is typically installed by the SDK in the global path but you need to be sure to run it with Python 2.5 (or whatever version of Python your GAE SDK expects). .. code-block:: python $ cd ../.. $ python2.5 /usr/local/bin/dev_appserver.py bfgapp/app/ Startup success looks something like this:: [chrism@vitaminf bfg_gae]$ python2.5 /usr/local/bin/dev_appserver.py bfgapp/app/INFO 2009-05-03 22:23:13,887 appengine_rpc.py:157] Server: appengine.google.com INFO 2009-05-03 22:23:13,898 appcfg.py:320] Checking for updates to the SDK. INFO 2009-05-03 22:23:14,034 appcfg.py:334] The SDK is up to date. WARNING 2009-05-03 22:23:14,035 datastore_file_stub.py:368] Could not read datastore data from /var/folders/dB/dByJ-qkiE6igZD4Yrm+nMk+++TI/-Tmp-/dev_appserver.datastore WARNING 2009-05-03 22:23:14,035 datastore_file_stub.py:368] Could not read datastore data from /var/folders/dB/dByJ-qkiE6igZD4Yrm+nMk+++TI/-Tmp-/dev_appserver.datastore.history WARNING 2009-05-03 22:23:14,045 dev_appserver.py:3240] Could not initialize images API; you are likely missing the Python "PIL" module. ImportError: No module named _imaging INFO 2009-05-03 22:23:14,050 dev_appserver_main.py:463] Running application bfgapp on port 8080: http://localhost:8080 You may need to run "Make Symlinks" from the Google App Engine Launcher GUI application if your system doesn't already have the ``dev_appserver.py`` script sitting around somewhere. #. Hack on your bfg application, using a normal run, debug, restart process. For tips on how to use the ``pdb`` module within Google App Engine, `see this blog post `_. In particular, you can create a function like so and call it to drop your console into a pdb trace: .. code-block:: python :linenos: def set_trace(): import pdb, sys debugger = pdb.Pdb(stdin=sys.__stdin__, stdout=sys.__stdout__) debugger.set_trace(sys._getframe().f_back) #. `Sign up for a GAE account `_ and create an application. You'll need a mobile phone to accept an SMS in order to receive authorization. #. Edit the application's ID in ``app.yaml`` to match the application name you created during GAE account setup. .. code-block:: python application: mycoolbfgapp #. Upload the application .. code-block:: python $ python2.5 /usr/local/bin/appcfg.py update bfgapp/app You almost certainly won't hit the 3000-file GAE file number limit when invoking this command. If you do, however, it will look like so: .. code-block:: python HTTPError: HTTP Error 400: Bad Request Rolling back the update. Error 400: --- begin server output --- Max number of files and blobs is 1000. --- end server output --- If you do experience this error, you will be able to get around this by zipping libraries. You can use ``pip`` to create zipfiles from packages. See :ref:`pip_zip` for more information about this. A successful upload looks like so:: [chrism@vitaminf bfgapp]$ python2.5 /usr/local/bin/appcfg.py update ../bfgapp/app/ Scanning files on local disk. Scanned 500 files. Scanned 1000 files. Initiating update. Cloning 761 application files. Cloned 100 files. Cloned 200 files. Cloned 300 files. Cloned 400 files. Cloned 500 files. Cloned 600 files. Cloned 700 files. Uploading 12 files. Deploying new version. Checking if new version is ready to serve. Will check again in 1 seconds. Checking if new version is ready to serve. Will check again in 2 seconds. Checking if new version is ready to serve. Will check again in 4 seconds. Checking if new version is ready to serve. Will check again in 8 seconds. Checking if new version is ready to serve. Will check again in 16 seconds. Checking if new version is ready to serve. Closing update: new version is ready to start serving. Uploading index definitions. #. Visit ".appspot.com" in a browser. .. _pip_zip: Zipping Files Via Pip --------------------- If you hit the Google App Engine 3000-file limit, you may need to create zipfile archives out of some distributions installed in your application's virtualenv. First, see which packages are available for zipping: .. code-block:: python $ bin/pip zip -l This shows your zipped packages (by default, none) and your unzipped packages. You can zip a package like so: .. code-block:: python $ bin/pip zip pytz-2009g-py2.5.egg Note that it requires the whole egg file name. For a BFG app, the following packages are good candidates to be zipped. - pytz - chameleon.core - chameleon.zpt - zope.i18n - zope.testing Once the zipping procedure is finished you can try uploading again.