Well, the python lightning talks are over, and I've officially and publicly conceded the contest. In the end we only managed four of the seven tests (and I found that Leo's version didn't do import, so you actually need to do some cut-n-paste to get one of the tests running) and part of a fifth. Guido was quite gracious and didn't pitch a pie at me, and while he disappointed many in the crowd (Perl, Python, and the random spectacle seekers) I do appreciate it.
Anyway, here's what we do have for results.
| Parrot | Python | |
|---|---|---|
| b1 | 1.40 | 1.65 |
| b2: | 0.15 | 0.45 |
| b3: | 5.00 | 2.40 |
| b6: | 1.62 | 2.40 |
How do you duplicate them?
First, you need an optimized build of Parrot. By default we don't build parrot with optimizations. Why? Well, we're still deep in development. It's really a pain to debug an optimized build (or a core dump from an optimized build) because of some of the things the optimizer does, so we don't. Sensible for development, less sensible for performance checks. (On the other hand, if your code is snappy without optimizations, that's a cool thing too) So, when you configure pass in the --optimize flag.
Second, you need to do the runs on an x86 linux system. This is especially true when running with the JIT--each architecture has a different set of ops that get fast JITting, with the x86 doing best.
Third, you need to run the tests at least twice for Parrot. The driver harness in CVS compiles the python to bytecode and translates the python bytecode to parrot bytecode the first time things are run. Second and subsequent times use the cached bytecode and skip that step, for a pretty significant speedup.
Finally you do need a mildly thumped version of test b6. One of the things the translator doesn't do right now is python's import, so the bits imported from b5 need to be cut-and-pasted into the source for b6. From there... run the tests a few times. I found there was a fair amount of flux, so the times above are the best of five runs for both parrot and python.
I'm pretty sure the bad times on b3 are from the hack job done to implement python classes. That's one of the things that's going to get fixed up. And things will get fixed up. Finishing the translator is high on the list 'o things to do, one I'm hoping to get done by the end of August.
Posted by Dan at August 1, 2004 06:41 AM | TrackBack (0)What are the units for those numbers?
I assume from your b3 comment that higher is better.
Posted by: Brad Fitzpatrick at August 1, 2004 02:21 PMThose numbers are seconds--lower is better. Our b3 times aren't at all good, in part because how we handled classes for the converter was sub-optimal, and b3 is the class and object test. (The rest of the tests are pretty non-OO) We're going to fix that up, as hand-rolled OO tests show us significantly faster, which means we need to work a bunch on the translation.
Posted by: Dan at August 1, 2004 05:23 PMCongrats on those number, Dan! Maybe this competition can come up again next year so you can have your chance at revenge.
Now I am just curious to see once this is finished if the Python community will put more effort in a Parrot port or if people will move over to IronPython and thus the .NET CLR.
Posted by: Brett at August 2, 2004 11:45 PMWhile revenge is a dish best served cold (with a nice white wine and fresh fruit) I think I'll pass on it. The bet was kind of silly in the first place and while I have no problem looking silly in public, there are limits to a joke. :) Besides, if we pull this off in the timeframe I'm shooting for (August this year) it'll be a bit anti-climactic.
Whether the community moves at all, and to where, is up in the air. We've never set out to be the VM for anything but Perl 6, and I'm not entirely sure I'd be happy with Parrot marginalizing another VM. (Well, OK, I'd be really unhappy with it) If IronPython gets finished I'd bet it ends up in about the same state as Jython, and I'd guess that Parrot'll be in the same position. Which, honestly, is fine with me.
Posted by: Dan at August 3, 2004 08:42 AMNote that there's also a project by L. Peter Deutsch to get Python running on top of the smalltalk VM.
Hopefully implementing Python on top of Parrot has been/will be also useful for Parrot's development. It certainly seems so, from what I've read...
Getting Python going on Parrot has definitely been very useful. Besides shaking out some implementation issues (we may well be able to speed up our sub/method calls a lot because of this) it's been really useful in pointing out some areas we were weak in (like slices) and got us a good chunk of experience doing bytecode translation.
All in all I'm glad we did it, and I think we'll have something pretty useful when we're done.
Posted by: Dan at August 6, 2004 01:25 PM