A Summer in the Wild

Alex Gaynor
DjangoCon.us 2011

6 September 2011

Alex Gaynor

A Summer in the Wild

Alex Gaynor

Me

Django/CPython/PyPy core developer Mostly been building websites for the past 4 years or so. I was getting kind of sick of websites.

Alex Gaynor

PyPy

Implementation of Python, written in [a restricted subset of] Python Advanced tracing just-in-time compiler. Fast: http://speed.pypy.org/

Alex Gaynor

Quora

Question and answer, knowledge database site All Python Wanted it to go faster.

Alex Gaynor

This Summer

Internship at Quora, to port the site to run on PyPy. Ultimately successful.

Alex Gaynor

This Talk

Everything I learned in 4 months... in 30 minutes. Stuff I missed terribly when I wasnt using Django Stuff you need to be doing, on all your projects.

Alex Gaynor

First Mover

Quora was (to my knowledge) the largest website to migrate to PyPy Not all packages worked out of the box

Alex Gaynor

MySQLdb

Wrote a fresh implementation of MySQLdb using ctypes Knew it was going to be needed going in. Still a pain, next guy doesnt have to do that.

Alex Gaynor

MYSQL RAGE

32-bit vs. 64-bit Client has been blocked from this server

Alex Gaynor

Paster

Under CPython used pasters HTTP server It had obscure bugs under PyPy with chunked encoding Switched to gunicorn: Just Worked (tm)

Alex Gaynor

Cython

Division between .py and .pxd les Better support hopefully coming soon.

Alex Gaynor

On being rst

Any major infrastructure change carries inherent risk. Know what that risk is going in. Tests are an absolute must (more on this later).

Alex Gaynor

Proling

It sucks. siege for very basic comparison. Editing gunicorn itself for actual proling with cProle (wtf?)

Alex Gaynor

Things Django is Doing Very Very Right

URLs The ORM Escaping Conventions

Alex Gaynor

URLs

With 100% ease, I can start from the root URL conf, and track down what view serves a given request. DO NOT VIOLATE THIS WITH MIDDLEWARE HACKS.

Alex Gaynor

The ORM

You forget how awesome it is, until you dont have it. 80-20 solution, more like 95-5 solution.

Alex Gaynor

Escaping

Security as a top priority. You have to actively work at it to give yourself an XSS hole.

Alex Gaynor

Conventions

You always know where your models, views, urls, etc. are within an app.
Doesnt require Django itself to do anything.

We need to keep going with this, organize projects in the same way. Conventions for using WSGI middleware with Django (coming soon)

Alex Gaynor

Seperation of concerns

Practically un-workaround-able Fundamental syntactic difference means you never forget.

Alex Gaynor

Tools everyone must use

virtualenv, pip No discussions.

Alex Gaynor

Project startup
cvs checkout mkvirtualenv <project> pip install -r <path/to/requirements.txt> Edit some settings.py
We need better conventions for settings.

python manage.py syncdb python manage.py runserver

And if those last two arent enough, your README damn well better say so.
Alex Gaynor

We do not

Check python binaries into our VCS Check compiled modules into our VCS Vendor our dependencies. Run your own PyPI server if you feel the need, dont check crap into the repository. Have a process that isnt a good process. Just a process isnt enough.

Alex Gaynor

Automate Everything

Content on Quora automatically updates if its changed by anyone on the site. General framework for doing this, not done manually. (Cool idea, someone should write it for Django) To a rst approximation anything done manually will break.

Alex Gaynor

Automate EVERYTHING

DRY applies to the human side of things as well. Write tests so you dont need to test everything by hand when you make a change Have a buildserver to run tests One-click deploys If you run the same 3 commands for occasional maintenance, PUT THEM IN A SHELL SCRIPT

Alex Gaynor

Caching

At a minimum organized. Better is automated (easier for simple object-caches). No caching in your views.

Alex Gaynor

Tests

How do you know what youre writing works? How do you know you didnt just break it? How do you know your big refactor still does the right thing?

Alex Gaynor

Build server

Because I dont trust myself, much less everyone else, to run all the tests every time

Alex Gaynor

One click deploy

{fabric, gondor, ep.io, chef, puppet} deploy git push deploy WHATEVER Deployment is hard enough, without having to re-invent it each time.

Alex Gaynor

In Summary

Infrastructure changes can be dangerous, mitigate risk by doing research up front Django is lovely, take advantage. Automate everything, or you and your colleagues will slowly go mad.

Alex Gaynor

Questions?

@alex_gaynor

http://alexgaynor.net/

Alex Gaynor