I know it's an act of supreme, if likely wrongheaded, laziness, but rather than trying to figure out how to build libraries on Windows, or get libraries loading via relative paths right on OS X, I just hacked up the makefiles for the perl module and the main tornado distribution to directly reference the objects. That means you get the whole wad just linked into your perl module, and at some point I'll get library building done in a proper cross-platform way so you can have your .so / .dll / .dylib again.
In the mean time, I get to leverage perl's wad 'o platform-specific knowledge and just bodge all the objects together into whatever perl needs, and if someone twiddles with a ruby or python version they can do the same thing.
I'd feel guilty, except the module library still weighs in at under 140K. I think I can live with that, 'specially as the library built for Devel::Size is 57K and there's not nearly so much code in it.
Windows still doesn't quite build right, but if all goes well it will in the next day or so. Assuming you've got the same development environment I do (which'd be activestate perl and the .NET 2003 dev kit), and if you don't then I probably ought to make PPMs to go with the source tarball. 'Cause, what the heck, having the perl module makes testing this stuff a lot easier. Yay for dynamism and all.
(No, there's no particularly interesting insights here. This is all grunt work, though I am so looking forward to learning Windows threading model enough to get its thread stuff into Tornado...)
Just cut a release of Tornado 0.04 -- http://www.sidhe.org/~dan/things/tornado-0.04.tar.gz. More fixed bugs and whatnot, but the fun new thing now is that the perl module (in the perl subdirectory) builds and tests, and that includes assembling a sample program, invoking it, and getting the results back. This is, I think, a pretty cool thing.
Unfortunately there's still some manual bits, like you have to copy the tornado shared library into the lib/ directory in the perl module's build directory when you're building and testing, but that's only because I've forgotten the Weird Magic needed to get MakeMaker to add in library paths and so I cheat. It's probably dead-simple, but for now this is easy enough, and I can write test driver programs in perl rather than having to abuse the main.c in the main tornado source directory. (Which is terribly inflexible)
That does work out, in its own way. Use the flexible language to do flexible things, the fast (I hope) engine to do the fast things, and let the glue code sort out the interfacing. The way things really ought to be, I think.
I should know by now -- whenever I'm facing large masses of mostly-identical code it's a sign that it's time to do some templating.
I did this in tornado with the opcodes, in large part because I fully planned on having a simple oploop to start with, and more complex oploops later. I also knew I was going to need to yank out opcode info for the assembler and whatnot, so pre-processing the source made a lot of sense. (This was something that we did quite successfully with Parrot)
I didn't do it with the various MMD math ops. This is something I'm coming to regret.
Firstly, there's just a whole wad of cut'n'paste code in because of it, and that's always really error prone. Yech for errors. Fixing bugs in the boilerplate is always a pain when it's boiled over, and you know that you always find the bug in the code that's been copied everywhere but miss one or two spots in the cut'n'paste fix.
Second it really sucks when you realize that you want to roll out a functionality change, but you've got a half-zillion places you have to wedge it in because it's in the boilerplate.
Feh. Should'a done it in the first place. Now I have to go back and rip out the guts of math.c and replace it with code that handles shorter vectors on the RHS of an op for looping. And possibly the LHS too, I'm not sure. (There is the question of what you should do -- loop back to the beginning, or pad with zeroes, or just run the length of the shorter vector)
Ah, well, it's only code. No reason to not rip it out and make it better if there's an actual win to doing so, and it'll make the source code smaller too. Smaller is usually more comprehensible, and I like comprehensible.
(I really ought to cut an 0.04 release -- fixed a bug on the train and added in vector/scalar versions of the math ops. Maybe tomorrow)
Yeah, so I blew it. A required file was missing, one I hadn't added to the repository. Ouch.
Anyway, it's in, and as a bonus there are some (very) minimal docs in the docs/ directory that I've cleaned up a little, the README file is mildly useful, and the garbage collector won't collect up and kill global variables. I regenerated the ops table too, so opcodes will have constant numbers. Not a huge deal, but stability of opcode numbers is nice. I'll probably toss and redo the table and bump the version number before release, since there's a lot of crud in there that I think I'll redo when I have time. (Like there are _vv _ss and _sv versions of opcodes, which made sense until I built the MMD system into tornado, and are now mostly just annoying fiddly things)
I probably ought to dig out some subversion plugin or other that'll automatically generate a changes file.
New version at http://www.sidhe.org/~dan/things/tornado-0.03.tar.gz. Feel free to have at it, and I think the comments on the blog are even fixed. (A mixed blessing, since that means blogspam again)
(And if you snagged the tarball before 10:30PM Eastern (I think we're GMT -500) re-snag -- it now builds on linux, and I tossed the damn redundant ops and went full MMD for dispatch. We have it, so we might as well use it, and I can add in static analysis later if I want)
So, I added in rudimentary garbage collection to Tornado. Nothing fancy, but enough to clean up after itself as its running. (Not as it exits yet, though -- still leaks then)
It only builds as packaged on OS X, and requires perl (but, then, if you're on OS X you have perl) and the docs are abysmal to non-existant, but if you want to play it's at http://www.sidhe.org/~dan/things/tornado-0.01.tar.gz. It's under the BSD license, so it's pretty much a go for it, good luck sort of deal.
It ought to work on Linux if someone kicks the config system a little. I'm working next on getting it to build on Windows, and that should be interesting only because the threading's so different there.
Don't expect much threading, though. This is, right now, entirely vector oriented, with the sole exception that calling into the interpreter to execute a function spawns off a thread and that function runs in the background. The cool tornado threading stuff comes next, but one thing at a time. Good chunks of vector functionality are still missing, and It's also not too friendly to program in either, but that's OK too. It does enough for me to hang more stuff onto it, and I'm fine with that.
If you play with it, expect to make friends with gdb, though. :)
And before anyone decides to mention it, yes, it does have a big function pointer table and do indirect function calls for opcode dispatch. There's enough indirection in place to fix that later, but opcode dispatch time is, bluntly, noise for Tornado right now if it's operating on any real data. Adding together two 100K element vectors (which can be done with a single op) eats more time in memory wait time than all the opcode dispatch overhead in your entire program.
This may change as the threading ops get added in and I start doing more work with threads rather than vector data, but then I go and massage the opcode function code generator to emit a switch statement or computed goto core or something. Or bite the bullet and write a JIT. Something like that.
Yeah, yeah, I know, it's been ages. That's OK, that's what RSS feeds are for -- to remind you that those of us who've gone missing have occasionally resurfaced.
Anyway, in my Copious Free Time (which is to say barely ever) I've been abusing Tornado, trying to get the basics of the thing to work. And... now it does. Woo! I'll grant, that's the sort of Woo! that comes from spending an order of magnitude more time on a dead-simple problem than the problem actually warrants (which means, of course, I was doing threads work) but I don't care. Woo! it is.
Okay, so Tornado doesn't as yet have garbage collection, and while it has most of the vector ops in there's no internal threading yet (merely external threading; Sending a function request to an instantiated Tornado instance spawns a new thread to run it), and some of the error checking's a bit dodgy. Which means it leaks like a sieve and falls over if you stare at it funny, but it works anyway. Besides, adding a GC's only an hour or two of work, so that's no big deal.
I've gotta admit, it's kind of cool to write a program that does fairly complex work on megabytes of data, when that program's all of twenty or thirty likes of assembly code and has no explicit loops in it.
I think this beastie's nearly ready for a release for folks to fiddle with, if they're so inclined. Or not, that's OK too.
One thing I have noticed that's really odd is that printf and its ilk don't work from inside Tornado on my OS X box. This is not a huge problem (I've made friends with GDB) but it is puzzling. They work fine from the driver program but not inside the engine code that lives in the .dylib. I have no idea why, and in practice it doesn't matter (since the engine can't do any IO itself by definition) but it's weird. I expect there's some odd flag or library I need to link in, or I should've waved the chicken widdershins around my laptop. Something technical like that.
The nice thing is that there's more than just a skeleton in place. Now there's a good chunk of meat on the bones, and a framework that makes it straightforward and pretty easy (courtesy of some of the lessons I learned from getting Parrot up and going) to extend things. This makes me... happy.
Of course, I expect getting this running on Win32 will make me Unhappy again, but that's OK, I can live with that. The code I need's in example form inside the perl 5 sources (and yeah, I know, that is a scary thought). Just gotta handle thread creation and joining, mutex creation, locking, unlocking, and condition waiting/signalling/broadcasting. (Which, I realize, is done a little differently under windows, but I've already abstracted it away a lot, so I'm fine there, I think)