Psyco and Django
Psyco is a module that optimizes Python applications on the fly. Numerous resources online describe how to use psyco in a Django-powered application to speed it up.
My experiences with this has been less than wonderful. I wrote a simple middleware class to import psyco as suggested here:
# Be sure to only load on the proper architecture from platform import architecture if not settings.DEBUG and architecture()[0] == '32bit': if architecture()[0] == '32bit': try: import psyco except ImportError: pass class PsycoMiddleware(object): def process_request(self, request): # Do not waste time trying to optimize the re module psyco.cannotcompile(re.compile) # Limit memory usage psyco.profile(memory=2048) return None
In development this appeared to work fine (we excluded the if not settings.DEBUG portion while testing). Once we were up and running on the production server, however, mysterious exceptions began to surface.
In particular, we seemed to be “missing” on some page hits. Specifically, we would get TemplateDoesNotExist exceptions when the templates did, in fact, exist. We were never able to sort that out. They immediately stopped when we commented out the psyco-related stuff. We tried changing the memory usage, excluding more items from compilation (including the Django loader classes), all to no avail.
My recommendation is avoidance of psyco in a Django setting. If someone has an idea what might have caused this, I’d love to hear an explanation.
It’s been over a year since your original post — has the state of psyco+django changed at all? It seems like over the last year, both projects have matured greatly, and was wondering if some of these mysterious issues have been solved…
Not that I am aware of.
I know my following statement is somewhat naive, since that is the first thing I thought and probably you did too. But since I was wondering to try this out myself, so I decided to ask first, so I won’t waste time if you know it does not work reliably:
Was the mysterious errors of
TemplateDoesNotExistonly errors? I mean, template non-existance might be that TEMPLATE_DIRS setting is not set using absolute paths, by default the TEMPLATE_DIRS has relative paths to “/templates/”, while this works normally, maybe psyco was changing dirs somehow and that got mixed up sometimes?Hi,
I have had a similar experience with Django+Psyco, but on a dev server too.
I managed to sort this out by using psyco.profile() instead of psyco.full()
And, there was a type error very common with psyco and django, I removed it by adding faulty functions to the blacklist with the psyco.cannotcompile(class/function/etc)
So far so good, but my pages are fast and psyco does not seem to make their generation faster.
class PsycoMiddleware(object):
def process_request(self, request):
try:
import psyco
psyco.profile()
except ImportError:
pass
return None
Enables the psyco extension module which can massively
speed up the execution of any Python code.