SRFI-32 for Bigloo

I needed some fast vector sorting routines for my n-body integrator. To that end, I've packaged up the withdrawn SRFI 32 for Bigloo. The SRFI is technically withdrawn, but even half of a half-standard is better than none (and this way I don't have to delve into sorting myself---I'm sure Olin Shivers is way better at that than I am).

Wolfram Tones

Here's an idea sure to appeal to nerds everywhere (including me): cellphone ring tones generated by cellular automata. Though it's a bit hard to stomach how much Wolfram exaggerates his contributions to the science of complexity and the study of cellular automata and related computational algorithms, it's still an interesting idea. Discovered through this New York Times article.

In addition to the unpleasantness of enriching someone who seems---shall we say---reluctant to acknowledge other contributions to the study of cellular automata, I am too cheap to just drop $2 on a cellphone ring tone. However, I do think my custom tone is pretty neat.

Srfi-4 Vectors in Bigloo

I just thought I'd drop a quick note about srfi-4-like vectors in Bigloo. For those who don't know, srfi-4 specifies an interface for homogeneous numeric vector types. For those of us who do numerical computing, this is a big win---boxing a double, for example, every time you reference or store into a vector is just impossibly inefficient if you want to get anything numerical done. But, Bigloo doesn't have an implementation of srfi-4---so how is it that I'm such an enthusiastic user?

Here's how: Bigloo allows for the definition of typed vectors. Typed vectors are vectors that only hold one of the native Bigloo types. (Unfortunately, this cannot be done for user-created types like classes.) For example, the module clause

(module f64vector
   (type (tvector f64vector (double)))
   (eval (export-all)))

Creates homogeneous vectors of doubles labeled by the type ::f64vector and functions to access these vectors. Nearly the entire srfi-4 interface is created; the only warts are

1. The make-f64vector procedure requires two arguments; the initializing value is not optional.
2. The f64vector procedure is not created.

Note that the created ref and set! procedures are typed correctly, which helps immensely in the type analysis of the remainder of your program (I hardly ever end up putting in typing information except at procedure arguments).

In fact, the Bigloo source distribution contains an example of using tvectors, but it's not a widely known fact, so I thought I would mention it here for posterity's sake.

By the way, supposedly the type system in Bigloo can identify when vectors are used homogeneously, and automatically convert them to tvectors. That's nice in theory, but in practice it only works on vectors that are used in only a few procedures within the same module, and even then it has trouble if you use the vectors in some strange way or forget to explicitly initialize them in the creation statement. For things like numerical computing, where the f64vectors can get passed around through the entire program it's hopeless---just use a tvector.

Economic Integration

Check this out (from an article in the New York Times):

A school board election will take place in October. While the board has continued to endorse economic integration, some supporters worry that that could change one day.

"It's not easy and it can be very contentious in the community," said Walter C. Sherlin, who retired two years ago as an associate superintendent. "Is it worth doing? Look at 91 percent at or above grade level. Look at 139 schools, all of them successful. I think the answer is obvious."

This is great! A public initiative backed by research into student performance that seems to be wildly successful (40%->80% grade-level scoring among black students over the last decade, 79%->91% overall). And, unlike race-based busing, this doesn't suppose that your performance is the result of your race (which is an internal factor that you have no control over) but rather your economic situation (which is an external factor over which you have some control). I hope that the parents of the 2.5% of children who are involuntarily bussed don't become such a vocal minority that this stops happening.

Unemployment

Just read an interesting paragraph in an essay about the internet bubble:

The prospect of technological leverage will of course raise the specter of unemployment. I'm surprised people still worry about this. After centuries of supposedly job-killing innovations, the number of jobs is within ten percent of the number of people who want them. This can't be a coincidence. There must be some kind of balancing mechanism.

From Paul Graham. He's probably right, but it seems possible that the mechanism is people bitching about being unemployed and holding society effectively at ransom to find jobs for them to do. (i.e. a country that has 50% unemployment usually has social unrest, too, and some sort of reforms are proposed so that the rich people and companies can get back to making money without that pesky mob at the front gate.) Probably there's someone who studies this; it would be interesting to know whether my idea is right.

The Flu

Not that I have it (or usually get it, being young and in good shape). However, this article in the New York Times today points out that vaccines are "apparently ineffective" in preventing the flu for elderly people:

The fact that the vaccine study showed that inoculations have had only a modest effect in the elderly is particularly worrisome, because this is a group that tends to suffer high rates of complications and deaths from the disease and vaccination is the standard practice. In people over 65, the vaccines "are apparently ineffective" in the prevention of influenza, pneumonia and hospital admissions, although they did reduce deaths from pneumonia a bit, by "up to 30 percent," the study says.

"What you see is that marketing rules the response to influenza, and scientific evidence comes fourth or fifth," Dr. Jefferson said. "Vaccines may have a role, but they appear to have a modest effect. The best strategy to prevent the illness is to wash your hands."

Being young, I would like to see mention of the effectiveness of the vaccine in younger people (i.e. whether it's worth it for me to get one).

On a related note, how do people who don't believe in evolution at all (i.e. something made the world exactly as it is now) cope with the idea of viruses sharing genetic material and becoming resistant?

Bigloo Finalization Library

I've just posted a library which allows finalization of Bigloo objects before they're reclaimed by the GC. You can view the documentation and download the library here.

Manuel provided the hint to look at doc/README.finalizer, but I didn't like the way finalization was done in the example there, and it seemed a bit out of date. So, I wrapped up my own finalization library. I worry that I may have gone overboard (both in the documentation and in the care I've taken with license), but I have had a lot of trouble using poorly-documented libraries in the past, so hopefully this will be helpful to more people because I've spent some time writing out good documentation.

Bigloo 2.7a Beta Mac OS X Package

I've just posted a Mac OS X installer for the latest (Sept 20) beta of Bigloo. You can read more about it here. It's been quite a lot of work to compile Bigloo properly on OS X (which you can read about on the Bigloo mailing list), and Manuel has been a huge help. Many of these things are happening automatically now, but I figured that, since I've already put in the effort, I might as well make things as easy as possible for those following. Instead of putting a few #defines into some header files, typing ./configure --lots-of-options and then compiling, you can now just grab the disk image and Bigloo is only 4 clicks away. Enjoy!

Bigloo Internals, More Dynamic Linking and Shark...

without much Scheme-y stuff in sight.

Over the past few days Manuel Serrano has been helping me improve the speed of my n-body integrator by changing the behavior of Bigloo's threading when run in single-threaded mode on OS X. There is a function in runtime/Clib/cthread.c called denv_get which, I assume, gets some representation of the current dynamic environment. Since the cthread.c functions are really stubs for a more complete thread implementation denv_get is really short---it compiles to about 5 instructions (including prelude and post-lude) on my machine. Unfortunately, in the beta version of 2.7a from 18Aug denv_get was installed as the function to get the dynamic environment through a function pointer, preventing its inlining by the C compiler. My n-body code was spending upwards of 50% of its time in denv_get (remember: 5 instructions!). Manuel had me insert #define BGL_DYNAMIC_ENV_ALTERNATE 1 in runtime/Include/bigloo.h from the newest beta of 2.7a (12Sept)¹. This apparently allowed for inlining of denv_get in single-threaded mode---presto: faster by a factor of two.

I'm mentioning this for two reasons: 1. Manuel is really cool to take so much time to help me with my performance issue. 2. If you use OS X and Bigloo 2.7a beta, you should definitely get the 12Sept version and insert #define BGL_DYNAMIC_ENV_ALTERNATE 1 into runtime/Include/bigloo.h!

In other news, Maunel has used my patch fixing a problem with the build script which was preventing the building of the compiler with shared libraries even when the --sharedcompiler=yes option was provided to configure. This makes it much simpler to build a bigloo linked against dynamic libraries on OS X. All you have to do is make sure to provide --sharedcompiler=yes (and, if you like, --sharedbde=yes) to configure. Then edit the Makefile.config, changing the following variables:

• LD=gcc
• LDFLAGS=-dynamiclib -single_module
• LDLIBS=-lbigloogc-2.7a
• SHAREDSUFFIX=dylib

That's it---no restarting the make in the middle after changing options or anything. (I can't remember whether this works for building the Development Environment, but I do remember that if it fails it's pretty clear what to fix.)

Finally, I'd just like to put in a plug for SHARK, Apple's profiler that comes with OS X. You can profile a running process, obtaining a call graph with who-calls-who and elapsed time in each call path, see the memory usage pattern (and page faults), look at the disassembled object code with color-coded hot spots and pipeline stalls, and get helpful advice like using the reciprocal square root estimate assembly instruction rather than the math library sqrt when you don't need super accuracy. I can't imagine the amount of effort it would have been to do the profiling I've done today and yesterday with gprof! Bigloo works quite well with SHARK, since it outputs relatively idiomatic C code (though you might have to de-mangle function names with bigloo-demangle); other compilers like Gambit-C (C used as assembler) or Chicken (C used in CPS) wouldn't give you code that SHARK would help with much.

Eventually I'll get back to Scheme-y things (I haven't written a good macro in days!), but I thought it would be good to get down some of the things I've been doing lately.

1: You can get the 12Sept beta of 2.7a from INRIA's FTP Site or its mirror at U. Nice.

Hybrid Cars and Pork

In today's New York Times:

"I would like to get more hybrids out of our system because I do think it's something that is here to stay," William Clay Ford Jr., chairman and chief executive of Ford Motor, told reporters in Detroit on Aug. 23. Last fall, Ford introduced a hybrid version of its Escape sport utility vehicle, the first of several hybrids planned.

At the same time, the energy bill signed by President Bush on Aug. 8 effectively gave a break to American manufacturers by extending what could be a tax credit of as much as $3,400 per car to purchasers of the first 60,000 hybrids sold by a company. The credit phases out after that. Toyota sold more than 60,000 hybrids in the first six months of this year, so the tax law seems intended to help General Motors and Ford.

[...]

The new tax law "seems to benefit those who haven't done anything in the area of hybrids until now," Christopher Richter, a CLSA Asia-Pacific Markets auto analyst, said in Tokyo. "And it seems to penalize those who have been pioneers."

No kidding! And note that Ford has leased its hybrid technology from Toyota---no R&D costs for them. The only good news on the pork-ridden government front is that California is going to make a distinction between hybrids based on their environmental quality:

In early August, California regulators started to distinguish between fuel-efficient hybrids and "muscle" hybrids, the high-powered versions that save little gasoline. Of the seven hybrid models now on sale in the United States, owners of the Honda Civic, Honda Insight and Toyota Prius can qualify for decals allowing them to drive alone, rather than with two or more passengers, in highway commuter lanes. That reward is not extended to four hybrids not rated as exceptional energy savers: Honda Accord, Ford Escape, Toyota Highlander and Lexus RX 400h.

Bigloo Tracing

I just saw that I'm being syndicated at planet scheme. Looks like it hasn't quite made it to planet lisp-like number of contributors (or probably readers). But, in honor of the nevertheless auspicious occasion I plan to say something scheme-y after the past few days of political posts (which, by the way, have probably alienated my one reader---sorry dude, but I call it the way I see it :) ).

I just noticed a very useful thing for debugging Bigloo: tracing macros! The way this works is you put (with-tracing n 'foo ...) declarations around your code and if the debug level (set by -g[2..4]) when compiling is greater than n you get a trace which outputs the symbol 'foo indented according to how many previous traces are active. You can also issue (trace-item . items) commands which display the items given at the appropriate level (though this only seems to work when the debug level is greater than 2). Want the debugging info to go away? Just lower the debug level (or---even easier---increase n) when you compile that module. It doesn't sound like much, but the indentation and (trace-item ...) are leaps and bounds better than the (print ...) statements I was constantly un-commenting and commenting out. I now have (with-tracing ...) scattered through my code---it's great.

Two warts if you use Bee in Emacs and dynamically load your compiled modules:

1. If an error occurs within the dynamic extent of the (with-tracing ...) form, the indentation doesn't always reset, so sometimes after several errors your traces are walking off the edge of the screen. (I don't know why a simple dynamic-wind doesn't cut it here:

   (dynamic-wind
      (lambda () (increase-trace-indentation!))
      (lambda () 'do something)
      (lambda () (decrease-trace-indentation!)))
   

   but it sometimes doesn't work.)
2. Sometimes the prefix of the trace data contains shell formatting characters that kind of mess up the output:

     |  |  |  |  |  |  |--+make-advancer-lambda
     |  |  |  |  |  |  |  |--+ calculate-new-dt
   

   Sometimes calling (trace-margin-set! "") clears this stuff, but sometimes it doesn't.

Anyways, even with the warts it's a pretty useful feature. Enjoy.

Update: In fact, you can get rid of all the cruft at the beginning of these lines by setting the parameter bigloo-trace-color to #f: (bigloo-trace-color-set! #f). Then everything works great within emacs.

What are they doing?

Yet more incompetence from our lovely executive branch: FEMA ignores an offer of help from Chicago.

Even before the storm hit the Gulf Coast on Monday, he [Chicago Mayor Daly] said, the city's Office of Emergency Management and Communications had contacted emergency response agencies in Illinois and Washington.

In the event of a disaster, the city offered to send 44 Chicago Fire Department rescue and medical personnel and their gear, more than 100 Chicago police officers, 140 Streets and Sanitation, 146 Public Health and 8 Human Services workers, and a fleet of vehicles including 29 trucks, two boats and a mobile clinic.

"So far FEMA has requested only one piece of equipment {ndash} [sic] a tank truck to support the Illinois Emergency Response Team, which is already down there," Daley said. "The tank truck is on its way. We are awaiting further instructions from FEMA."

Even before the storm hit the Gulf, Chicago was offering to help. Too bad FEMA doesn't believe in planning; they just lock the door after the horse has left the barn. You can read more about their obstructionism in the name of coordination from a list here. (Not that I'm opposed to coordination by one agency, but I think that you should have to demonstrate that you aren't a total fuck-up before being allowed to tell other helpful people what not to do. In any case, I'd have a lot more confidence in a press-release asking First Responders not to help unless requested by state and local authorities if it actually came from the state and local authorities, not from an ex-horse-show overseer and political supporter of the President.)

Labor Day

From a NY Times Op-Ed:

...But because the French work 35 hours a week, Americans sneer, forgetting that in many years French workers have a higher productivity rate than their American counterparts - proof that you can work better by working less.

Americans also forget that going to work every day is often more a chore than a pleasure. You seem more and more disillusioned about work: only a third of you say that you love your jobs. In such conditions, it's not surprising that you spend on average two hours of your workday ... not working. Answering personal e-mail messages, shopping online, playing computer games or chatting with co-workers ... it's so much more pleasant than working, really.

Only two hours a day? Hell, I might play two hours of foosball on a good (bad?) day. Then there are the days when I work 18 hours straight. It's just the nature of an intellectual job---when you're hot, you're hot, and when you're not you play around. Fortunately, I have an employer who realizes the capricious nature of Physics research---I wish everyone was so lucky in their "boss".

P.S.---I love my job! Guess I'm part of the lucky 1/3.

Science and Politics in the NY Times Magazine

There's an interesting article in the New York Times Magazine today about politics and science. The article discusses possible reasons for why most (nearly all?) scientists think the Bush administration has done an abysmal job listening to and properly reacting to scientific developments and studies. It's probably a good factual review of several instances where science was suppressed/changed for political ends in the administration, and also a good description of John Marburger's role and why he thinks he's doing good for science.

Unfortunately, I think the writer missed the point. He focused nearly entirely on the way that scientific results are incorporated into reports and policy decisions. This is surely an important component of how the administration treats science, but it's not the whole story. What I think really upsets scientists---me included---about this administration is not the specific suppression of a study in an EPA report (though this is pretty bad); it's their seeming disregard for facts and reason and their willingness to take public positions in support of pseudo-scientific, ideologically driven theories. This complaint would be there even if they had never, ever, pressured an employee to change the mention of scientific results.

Several examples of what, generally, pisses scientists off:

1. The president's stance on Creationism (aka Intelligent Design). I know that most of the country believes this, but I would hope that the president would want to lead the country, not follow it in its ignorance. (By the way, the article doesn't mention Creationism at all.)
2. The president's position on climate change, and the response of the administration to the overwhelming scientific consensus that CO2 is causing global warming. (This is discussed in the article, but more from a point of view of suppressing the relevant information in EPA reports and whatnot.)
3. The rationale for tax cuts in the face of growing deficits and no significant spending cuts. You can't have your missile-defense cake and eat it, too, people. This isn't science, but it's such a glaring stupidity that scientifically minded people can't help but notice it.

The common theme here is: the article focused on the details of policy---reports, committee selections, etc---but I think scientists are more interested in the projected public attitude of the administration. I think that most scientists would be willing to forgive some pretty big omissions and commissions on the details front if you didn't get the feeling that the people in change had wooden heads. I mean, if a guy really believes in Creationism and tells the public that it's on an equal scientific footing with evolution, does it really matter if he gets all the facts right in an EPA report? Admittedly, the public pronouncements of the President don't have as much of an immediate policy effect as the committee work and rules published by the agencies, but it's the public persona that people follow; Bush and the upper levels of the administration are basically telling the country that there's two worlds: the scientific, rational one and the one that does whatever you want. And you don't have to listen to the scientists because you can just live in the other world. That attitude does more damage than the specific examples in the article, and that's what really pisses scientists (well, at least me) off.

Update: I totally forgot about issues like AIDS in Africa, but it's another one. Again the message: it doesn't matter what you scientists discover about the best way to prevent AIDS---we don't have to listen to it. It's not getting the facts wrong; it's ignoring the facts you know to be true.

Update II: I didn't realize that this article was still available, or I would have linked to it above. Favorite quote (the article is by Pulitzer Prize winner Ron Suskind):

In the summer of 2002, after I had written an article in Esquire that the White House didn't like about Bush's former communications director, Karen Hughes, I had a meeting with a senior adviser to Bush. He expressed the White House's displeasure, and then he told me something that at the time I didn't fully comprehend -- but which I now believe gets to the very heart of the Bush presidency. The aide said that guys like me were ''in what we call the reality-based community,'' which he defined as people who ''believe that solutions emerge from your judicious study of discernible reality.'' I nodded and murmured something about enlightenment principles and empiricism. He cut me off. ''That's not the way the world really works anymore,'' he continued. ''We're an empire now, and when we act, we create our own reality. And while you're studying that reality -- judiciously, as you will -- we'll act again, creating other new realities, which you can study too, and that's how things will sort out. We're history's actors . . . and you, all of you, will be left to just study what we do.''

Really! This kind of shit goes over like a lead balloon with physicists (and probably most other scientists, too). Again, the outrage isn't in the details---it's in the fundamental assumptions of the people in power.

Google Helper for Mac

Google Mail Helper is available for the Mac! I'm now using it, but I'm wracked with guilt over my desertion of the wonderful soul who created Gee. Poor Lloyd---and because it never rains but when it pours, Lloyd looks to have lost a bunch of photos in a Linux drive crash. Bummer.

I guess people may still use Gee for other Atom feeds, but mail helper has much better features for Gmail (not least that it registers Gmail as the default mail handler so that accidentally clicking on those mailto: tags doesn't open up Mail.app but instead takes you to Gmail).

Chlorine Tidbits

Some sites regarding chlorine exposure in swimmers are interesting, since I just got back into swimming. The information is somewhat sketchy on the web (and most sites are trying to sell you something---as in http://www.karinya.com/chlorine.htm), but there are some seemingly-authoritative pages. This review points out that there isn't a very good understanding of the effects of chlorine and related compounds in the air above a pool. (It does also point to the well-known beneficial effect of swimming on asthmatic children, which is reassuring.) Excessive chlorination may lower the pH of the water, leading to problems with dental enamel. A good summary of papers is here.

In any case, I think swimming is pretty fun, and it keeps me in great shape. I'm not worried overly about the chlorination, since it seems like the health benefits far outweigh the risks, but I'm not saying that I wouldn't be happy if my pool used something else.

Sean Carroll's Blog

I just discovered Sean Carroll's Blog (he blogs along with several other physicists at cosmicvariance). Great stuff! I particularly enjoyed the political cartoons #1 and #2. Of course, the physics is neat, too.

I haven't hung out much with Sean (just a bit when I was visiting Chicago as a prospective grad student), but now I can follow him vicariously and wonder what Chicago would have been like. (Not that I'm unhappy here at MIT; just that it's interesting how different Sean and Ed are.)