Wednesday, September 17, 2008

Pycon UK, Javascript and the GIL

Just got back from Pycon UK 2008 - here are some impressions.

Both the keynote speakers Mark Shuttleworth (Canonical) and Ted Leung (Sun Microsystems) expressed their concerns about Javascript becoming so fast and prominent that it could displace Python in the future. They also highlighted the fact that Multi-core systems get cheaper and more popular also on desktop computers or notebooks. They challenged the community to advance Python implementations to exploit it. Question was up what PyPy can do here. As it stands, PyPy still uses the good old Global Interpreter Lock (GIL) but our approaches should indeed lend itself well to do experimentation with free threading.

During the 2-day conference we met many interesting people, most notably the guys from Resolver, among them William Reade who is working on IronClad -- which implements a fake python25.dll on top of IronPython. He presented some good results for Numpy in his lightning talk. This approach is surely something to follow closely and potentially use for PyPy.

We also had lunch and a couple of chats with Jacob Kaplan-Moss from Django fame - he is apparently up to try use PyPy's sandboxing features for one of his projects, cool!

Conference itself was well organized for the 230 attending people - although the venue might be a bit small for next year's EuroPython. Ah, and we gave three well attended talks, find the slides here:

cheers,
Holger, Maciej, Anto (associated through merlinux, btw)

6 comments:

Lucian said...

Is the work done on this (http://code.google.com/p/python-safethread/) useful conceptually?

René Leonhardt said...

Is the new multiprocessing module going to offer improved multi-core performance?

Colin Walters said...

Jython? About to hit 2.5, has a threading model for free from the JVM.

Luis said...

I wonder how the new javascript improvements compare to pypy technically. For example, the tracing techniques of Mozilla's Tracemonkey look impressive, but I don't know if these techniques are conceptually close or not to pypy's. Is there anything you can learn from them (tracemonkey, chrome's v8, etc).

Luis

Miguel Filipe said...

ReneL:
Yes, the new multiprocessing module will improve multi-core performance if you use it.
That module allows a easy way to use multiple processes cooperatively in python. It tries to mimic the threading API.
If you use multiple processes instead of threads you will avoid the Global Interpreter Lock.

About new chalenges for PyPy, multicore isn't the major problem.. the absense of a powerfull JIT and GC is. Please keep working on a super-fast "VM+JIT" for python! (super linux performance is a must)

holger krekel said...

colin: true, jython and also ironpython can make use of multiple threads. should have mentioned it. Doesn't mean that pypy-c shouldn't go for it, rather the opposite i'd think :)

renel/multiprocessing module: i can imagine it helps with multi-core cpus. are there practical experiences using it yet?

luis, miguel: there are considerable efforts on the PyPy/JIT front - particularly from Armin, Carl Friedrich and Antonio - would be worth a dedicated blog post to relate this to tracing JITs, V8, squirrelfish, etc. One thing i know is that we probably want to apply for funding to help completing the JIT.

Miguel: We do have advanced GCs and are working on on improving them, currently.

to all: thanks for your feedback!