Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
Books Software Media United States Linux

Linux and OSS to Aid the Library of Congress 63

Posted by Zonk from the go-little-patriotic-penguin-go dept.
flakeman2 writes with a link to Linux.com article about Linux's new role at the Library of Congress. The national archive of books is looking to begin an ambitious digitization project, aimed at getting some rare and crumbling documents into the public record online. These will include "Civil War and genealogical documents, technical and artistic works concerning photography, scores of books, and the 850 titles written, printed, edited, or published by Benjamin Franklin. According to Brewster Kahle of the Internet Archive, which developed the digitizing technology, open source software will play an 'absolutely critical' role in getting the job done. The main component is Scribe, a combination of hardware and free software. 'Scribe is a book-scanning system that takes high-quality images of books and then does a set of manipulations, gets them in optical character recognition and compressed, so you can get beautiful, printable versions of the book that are also searchable,' says Kahle." Linux.com and Slashdot.org are both owned by OSTG.
This discussion has been archived. No new comments can be posted.

Linux and OSS to Aid the Library of Congress More

Linux and OSS to Aid the Library of Congress

Comments Filter:
  • For the past few months Microsoft has been dispatching crack teams of special operatives into the past to alter the course of American History for their benefit, in hopes of eventually transforming the United States into the New Microsoft Empire. But little do they know, a world-weary Librarian and Ex-Marine at the Library of Congress won't stand for that shit. He's put together a team of agents in hopes of reversing the damage to the timestream before it becomes irreversable. Together with Agents Linus Torvalds (Technology Specialist - Special power: x-ray glasses), Donald Trump (Logistics Specialist - Special power: nuclear fusion comb-over) and Stephen Hawking (Quantum Physics Specialist - Special power: medusa glare), he just may be the only hope for American History's future.

  • and got "Dear Blair, let's set so double the killer delete select all ..."

    Suffice to say, they settled with Linux. The Microsoft version had psychic powers, apparently!

  • It's only natural (Score:3, Funny)

    by donglekey ( 124433 ) on Monday April 02, 2007 @03:42AM (#18571287) Homepage
    So many technologies have been made specifically to hold libraries of congress.

  • No matter how you look at it (Score:3, Interesting)

    by zappepcs ( 820751 ) on Monday April 02, 2007 @03:50AM (#18571313) Journal
    this has got to cause some flying chairs in Redmond.

    Arguably one of the most important repositories of information in the U.S. is about to be available via OSS software and not MS products. For all the efforts that MS put out in Mass. this has got to be a kick in the face! Just wow!

    • The most important part is not free software (Score:5, Interesting)

      by Ed Avis ( 5917 ) <ed@membled.com> on Monday April 02, 2007 @04:15AM (#18571371) Homepage
      As the article says, the OCR itself is still done with proprietary software. I wonder if Google is using Tesseract [google.com] for their digitization efforts. It would be cool if the original raw scanned images could also be archived and available for download - then you could print your own copy of the book, check the OCR for errors, or even do some weird genetic algorithm thing to make a LaTeX style that typesets the text in the same format as the original book.

      • Oh, it will be. (Score:3, Interesting)

        by Grendel Drago ( 41496 )
        Project Gutenberg uses plenty of scans from American Memory to make their etexts--they do pretty much what you describe. At the lowest level, they make a plaintext copy, but they also do formatting and in-text hyperlinking: for instance, linking footnotes to their references, or index page numbers to anchors in the text. (See the HTML version of this etext to see what I mean [gutenberg.org].) Browse to a random book from this random collection [loc.gov], and you'll see what the LoC provides for their collections currently. As Brewst
      • Huh. I was going to look into it, as I didn't know there was any good open source OCR software.

        Good thing I decided to comment before reading the article.

  • All copyrighted works should be held (Score:4, Interesting)

    by rtb61 ( 674572 ) on Monday April 02, 2007 @03:56AM (#18571325) Homepage
    They should expand the role so that all copyrighted works are held at the Library of Congress. It would certainly save the confusion over who holds what rights to what content. At the same time, unless congress wants to hold and distribute material of questionable moral quality, the copyright law could be amended to limit the protections of copyright to those works that do actually further the arts and the sciences as defined in the constitution.

    The revisions to the law would not be infringing freedom of speech, in fact by allowing the free copying of works that did not further the arts or the sciences it would be limiting copyrights impact upon the freedom of speech. If people are really concerned about the quality of content, they should remember that eliminating the profit motive will have a substantial impact upon the amount of questionable content that is out there including movies, music, pictures and literature. Most of the members of the RIAA and the MPAA have a total disregard for the harm their content cause to society, let them feel some of the pain, wipe out the copyright protections on some of their more divisive content ;).

    • in fact by allowing the free copying of works that did not further the arts or the sciences

      Way to kill the entire non-fiction genre.

    • Re: (Score:3, Funny)

      by forkazoo ( 138186 )

      At the same time, unless congress wants to hold and distribute material of questionable moral quality, the copyright law could be amended to limit the protections of copyright to those works that do actually further the arts and the sciences as defined in the constitution.

      Sir, you pre-suppose that morally questionable articles do not serve to further the arts or the sciences. I protest most heartily. Every modern technology is served by serving pornography, as we all know. Let us firstly ponder the case

      • Re: (Score:2)

        by Aladrin ( 926209 )
        He got modded interesting, and you got modded funny... It should be the opposite. What you say is absolutely correct. Maybe just not exactly how you said it.

        When Shakespeare was writing his work, do you think he thought 'I'll improve the arts and be known throughout the ages as a great writer' or do you think he merely enjoyed his work and liked the money? At the time, I'm sure nobody thought his work even a fraction as important as we now think it is.

        So how are we to judge works of today? We obviously

        • Re: (Score:2)

          by rtb61 ( 674572 )
          Of course you can selectively apply copyright. it says so in the constitution, there is absolutely nothing to stop you from expressing yourself, but should other people be spending their tax money to protect your works, especially when your work specifically attacks their values. Should law enforcement or the courts be protecting work that attacks the society that allows it to be produced.

          Should work that attacks family values be protected by the tax dollars that are taken from families. Copyright protect

          • Perhaps a minor copyright tax could be levied to offset the cost, as well as to limit the duration of copyright for works that do not need it and stop orphaning works so they cannot be licensed.

    • Re: (Score:3, Insightful)

      by vivaoporto ( 1064484 )
      "At the same time, unless congress wants to hold and distribute material of questionable moral quality, the copyright law could be amended to limit the protections of copyright to those works that do actually further the arts and the sciences as defined in the constitution."

      Uh-uh. Let's repeat the same errors from the past, keeping what the current generation deems "of excelent moral quality", and censoring everything else, just like some works of Michelangelo [wikipedia.org] were. People must to remember, what is of q

    • Re: (Score:2)

      by rasilon ( 18267 )
      I would suggest that holding material of questionable moral content is an important function, even if only to further the historical record so that future scholars can see where we drew the line. Leaving an accurate record of our times does future generations more good than than trying to erase those parts of our culture that some find objectionable.

    • Re:All copyrighted works should be held (Score:5, Insightful)

      by jimicus ( 737525 ) on Monday April 02, 2007 @06:26AM (#18571711)
      At the same time, unless congress wants to hold and distribute material of questionable moral quality,

      Stop right there.

      When the purpose of your organisation is, to put it in very simple terms, "catalogue everything", you can't start making exceptions on moral grounds on the simple basis that what constitutes "questionable moral quality" today may be totally different tomorrow. Furthermore, who gets to define "questionable moral quality"? The closest anyone's ever come to creating such a definition is to say "Well, I can't actually come up with a concrete definition but I knows it when I sees it".

      • Re: (Score:2)

        by rtb61 ( 674572 )
        Same as always, the majority, whatever it may be, after all it is not about blocking it, it is all about society as a whole not providing an opportunity for questionable product to profit at societies expense. Why should any be fined or imprisoned, why should children be threatened by the courts either civil or criminal, for content that the majority would deem not worth protect.

        I support free speech, 'FREE' as in 'FREE', you want society to allow your to generate a profit at society expense, then you are

        • Re: (Score:2)

          by jimicus ( 737525 )
          Same as always, the majority, whatever it may be, after all it is not about blocking it, it is all about society as a whole not providing an opportunity for questionable product to profit at societies expense.

          The one copy that goes to a nation's library hardly constitutes a great profit at societies expense.

          And the whole point of my post was that society changes. What may be considered perfectly acceptable today may not have been 100 years ago. Pre-marital sex immediately springs to mind, but I'm sure the

          • Re: (Score:2)

            by rtb61 ( 674572 )
            So what about the 'immorality' of copying, copyrighted immoral content. Your same argument defeats itself. Copyright infringement is a morality and greed question, as such all the conditions of copyright should be subject to a morality review and greed should not be the basis for acceptance for any particular condition of law or content.
    • Comment removed based on user account deletion
      • IANAL, but this doesn't sound correct. Don't you have to provide proof of creation in order to hold the copyright on something? If you don't claim it as yours, for example, by putting "Copyright 2007 Some Dude" or something like that, someone can copy it and use it until you ask them to stop. And if I'm not mistaken, just SAYING "Copyright" doesn't mean it is - there's more to do.

        • Re: (Score:2)

          by Scaba ( 183684 )

          You don't have to be a lawyer, and you are mistaken - just look at the Copyright Office [copyright.gov] website. Simply creating a work in fixed form copyrights it [copyright.gov]. If you want to be able to prove it in court later that you are the creator of said work, however, it's best to register your copyrights with the Library of Congress. It used to be that you were required to put a copyright notice on your works lest you could lose the copyright, but that's no longer true.

          • Comment removed based on user account deletion

            • Re: (Score:2)

              by Scaba ( 183684 )

              There's nothing in copyright law that provides for the "poor man's copyright registration" you're talking about, and I don't think there are any cases in which it proved someone the true owner of a copyright. All it proves it that the work was made into a fixed form at some time. It's an easily forged method - you can just mail yourself an unsealed empty envelope, fill it with whatever you want when you receive it, then seal it - and any attorney you're up against will certainly bring that up in the court

  • Does anyone have more information on this 'Scribe' system? Google fails me.

  • Excellent project!!! (Score:3, Interesting)

    by 2Bits ( 167227 ) on Monday April 02, 2007 @04:41AM (#18571433)
    This is absolutely cool. There are a lot more places where "brittle books" are laying around, waiting to be digitalized and distributed to the whole world. And as the technologies used in this project are going to be refined and improved, and eventually released, everyone will benefit.

    The question now is: would they accept technical contributions from the public (I mean, OS geek communities), just like other open source projects? I know a lot of people would be eager to join. How about a SETI-like system to harness the power of desktop computers around the world to help with image processing and OCR? Hey, I got 4 decent desktop computers that can contribute at least 8 hours/day each.

  • How much data is needed. Of course, it would be necessary to have this number in a useful units system. Perhaps, the number of Libaries of Congresses of Data?
  • It would be neat if they did some other type of scanning, such as laser scanning the exterior, so that the book's heft and presence can be reproduced in the future. I've printed out crisp, high-resolution PDFs of user manuals, and having three hundred pages of printer paper binder clipped together just isn't the same as a nice, perfect-bound manual with a glossy cover.

    Sure, I imagine most of the consumption in the future will be done in a digital environment, but it would be nice if future generations had t

  • The sad part of digitization. (Score:5, Interesting)

    by Lethyos ( 408045 ) on Monday April 02, 2007 @05:28AM (#18571543) Journal

    Eventually we will have no physical record of these writings and may someday learn from the digital copies that Benjamin Franklin, George Washington, and others had offered enthusiastic support for wiretapping and other forms of electronic surveillance [huffingtonpost.com].


  • The OCR software from Scribe is still closed source.
  • What OCR-Engine do they use?
  • Yeah, but how many Libraries of Congress will... eh... never mind.
  • ...has been done before. The Domesday Book [wikipedia.org] was digitized by the BBC Domesday Project [wikipedia.org]. Unfortunately, it ended up being a comic rather than a technical triumph. People must remember that, no matter how low-tech a method of physical data storage seems, it's more reliable in the long run than data storage relying on complex technology. I'm not against digitization by any means, of course: it could be useful as a research tool, and as an alternate method of access. It shouldn't be viewed as a long-term ar

  • Quality as well as quantity, please (Score:4, Informative)

    by Ankh ( 19084 ) * on Monday April 02, 2007 @09:59AM (#18573927) Homepage
    The books I've looked at have been scanned at a resolution that's more or less adequate for OCR, but isn't really adequate for reproducing fine woodcuts, and is hopeless at metal engravings. I've found from my work on fromoldbooks.org [fromoldbooks.org] that anything less than 1200 dpi generally produces pretty poor results for images, so that, for example, you can't read the signatures of the artist and engraver, still less compare engraving styles. It would be sort of like having a paraphrase of the text instead of the actual words.

    It does, of course, vary a lot depending on the style of image. Bold illustrations for children's books, for example, do better at, say, 800dpi greyscale or colour. Fine steel engravings with lines at, say, less than a tenth of a degree from horizontal (they were done by hand after all) and that come out only a couple of pixels wide even at 1200dpi just turn into gray mush with weird banding artefacts until you go to a higher resolution (I use 2400dpi). There's a widely-cited study indicating that an "ultra-high" scan resolution of 400dpi is more than sufficient, based on an extremely small sample of images.

    The damage that's done by poor quality digitization is that it makes it harder to justify doing a better job in the future.
  • Poor Benjamin Franklin is about to be deprived of the legitimate compensation owed him. How is the fellow supposed to make a living? He's only been dead 217 years. Surely copyright on his works ought to be retained for awhile yet. Now those communist pinko linux f@9$ have really gone over the line.

      Just in case you're immeasurably thick.
  • Is that these documents will be made available on iTunes for $0.99 at roughly 80% digital accuracy and for $1.29 at around 92% digital accuracy. If you want 100% digital accuracy, just get a Library of Congress card, check it out, and copy it yourself.

Slashdot Top Deals

"...a most excellent barbarian ... Genghis Kahn!" -- _Bill And Ted's Excellent Adventure_

Close