real bio pointed us to Juggernautsearch which actually looks interesting. Its GPLd. It can index 800 million pages every 3 months and deliver 10 million pages a day on a Pentium II. So I guess if you want to run your own Altavista, you can.
The world is moving so fast these days that the man who says it can't be done is generally interrupted by someone doing it. -- E. Hubbard