Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Open Source Linux IT

Open Source OCR That Makes Searchable PDFs 133

Posted by timothy from the word-of-advice dept.
An anonymous reader writes "In my job all of our multifunction copiers scan to PDF but many of our users want and expect those PDFs to be text searchable. I looked around for software that would create text searchable pdfs but most are very expensive and I couldn't find any that were open source (free). I did find some open source packages like CuneiForm and Exactimage that could in theory do the job, but they were hard to install and difficult to set up and use over a network. Then I stumbled upon WatchOCR. This is a Live CD distro that can easily create a server on your network that provides an OCR service using watched folders. Now all my scanners scan to a watched folder, WatchOCR picks up those files and OCRs them, and then spits them out into another folder. It uses CuneiForm and ExactImage but it is all configured and ready to deploy. It can even be remotely managed via the Web interface. Hope this proves helpful to someone else who has this same situation."
This discussion has been archived. No new comments can be posted.

Open Source OCR That Makes Searchable PDFs More

Open Source OCR That Makes Searchable PDFs

Comments Filter:

  • Thanks! (Score:5, Insightful)

    by Fast Thick Pants ( 1081517 ) <fastthickpants@gma i l . c om> on Thursday July 22, 2010 @02:23PM (#32994454)
    Wow, it's a "Tell Slashdot" segment! I've been looking for something similar myself, so thanks, I'll give this a spin!

    • Re: (Score:3, Informative)

      by godrik ( 1287354 )

      Same here. Thank you too!

      (I know this post is very redundant and useless. But thanks are always welcome, aren't they ?)

    • Re:Thanks! (Score:4, Interesting)

      by MikeBabcock ( 65886 ) <mtb-slashdot@mikebabcock.ca> on Thursday July 22, 2010 @03:22PM (#32995328) Homepage Journal

      I only wish I could find a source download on their site. Even a "what we're doing" guide. Downloading the ISO and reverse-engineering what they're doing with cuneiform and exactimage doesn't seem nearly as productive, especially when I'd rather implement this on an existing server than boot a special piece of hardware with it.

      • Re: (Score:3, Insightful)

        by tsstahl ( 812393 )

        Virtual machine?

        • Virtual machine?

          Only a solution to "How do I get this running" and not "What is this thing doing?" The lack of source is a bit offputting to me. I will look at it, but I may wait to roll it out.

          • "Only a solution to "How do I get this running" and not "What is this thing doing?" The lack of source is a bit offputting to me. I will look at it, but I may wait to roll it out.

            I would tend to agree, only because I'm extremely paranoid when it comes to security; I'd do some site analysis and make sure unexpected connections to foreign hosts aren't going out over the wire. If they were I'd want to do some code analysis to see what exactly is going on. Or if I wanted to add some customization: extremely important. But; in a pinch it sounds like a really worthwhile solution.

      • Setup a VM; not only can you monitor / limit its communication but it's a cinch to back up. In my environment this is the easiest way to test something also. I use ntop for monitoring and it works ok; it would probably be a good fit in this case.
        • Many systems are better dedicated to a single problem, i.e., just because you have a server doesn't mean to say that you have to serve everything. VMs are a great solution to this allowing you to partition up your server so that each service that you provide runs in its own little virtual box without having to worry so much about unwanted interactions.

          • If I have a working server that already hosts PDF content, why would I want to virtualize this one when I could integrate its features into the existing one?

            That said, the one server per service concept is a mentality I do not subscribe to.

            • That said, the one server per service concept is a mentality I do not subscribe to.

              This is where Microsoft came apart. Due to their pricing model, there was always pressure to stick as much on one box as possible. This in turn led to interesting side effects.

              Linux always made it easier to have many boxes, which tended to simplify problems. VMs meant you no longer had to worry about physical machines and you can still limit resources - useful if the OCR turns out to be a CPU pig.

      • It's all GPL so there has to be source somewhere. Their site says

        The source code of the standard packages on the CD are available from their respective original providers (for example on the FTP servers at Debian). Special components such as the WatchOCR program and scripts are available on the CD.

        so it's probably on the disk.

    • Another "thanks"; I have been looking for something along these lines on a personal basis. Looking forward to checking it out.

  • Wait a sec (Score:5, Funny)

    by inKubus ( 199753 ) on Thursday July 22, 2010 @02:30PM (#32994542) Homepage Journal

    There's something wrong with this Slashvertisement--it's for a free product!

    • I guess that's where

      step 3: ????

      Comes into play.

    • Re:Wait a sec (Score:4, Insightful)

      by ushering05401 ( 1086795 ) on Thursday July 22, 2010 @02:51PM (#32994870) Journal

      Seriously, I'm conflicted. I'm not any sort of web search guru, but it looks like that site just got put up. Is submitter an early adopter (v0.2) or a social engineer?

      • Version 0.2 has been out for at least a month by the looks of their forum, and version numbers are a very imprecise way of telling how useful the software is for your needs, or even how stable it is. What's wrong with being an "early adopter" if it's the only working and free solution to your problem?

  • Thanks for the info... (Score:3)

    by TiggertheMad ( 556308 ) on Thursday July 22, 2010 @02:30PM (#32994544) Journal
    Wow, very cool. I have been looking around for something similar myself.

    While we are on the topic, anyone seen a good solution to scan, OCR, and reconvert existing crappy pdfs to improve them?

    • Re: (Score:2, Informative)

      by It's the tripnaut! ( 687402 )

      While we are on the topic, anyone seen a good solution to scan, OCR, and reconvert existing crappy pdfs to improve them?

      I've tried quite a few free and proprietary OCR's and the best available right now, imho, is ABBYY Finereader [abbyy.com]. Other than fonts, it also easily recognizes tables, diagrams and illustrations. But most of all, it can read and render 189 languages (including Chinese and Cyrillic) accurately. A free trial version is available.

      • It's true. I teach Translation Studies and one of the main pieces of software that's needed is OCR. I use OOo, Poedit, Lokalize, Jubler and OmegaT in my class, I teach Creative Common's Licensing, I promote sites like http://pootle.locamotion.org/ [locamotion.org] and http://www.transifex.net/ [transifex.net] I *really* *really* wish I could give my students a best of the bunch free OCR link. But the reality is, ABBYY Finereader is the best that's available. And since it's relatively cheap (compared to some of the translation software lik

    • While we are on the topic, anyone seen a good solution to scan, OCR, and reconvert existing crappy pdfs to improve them?

      I think they are called interns. Photoshoop's Content-Aware Fill isn't very good with charts or handwriting.

      ...wait, you actually kept the original document after PDFing? Troglodyte.

  • Saw this on facebook. While I don't personally have a need for this, I know that down the line, I'll be glad I knew about it. Good post.

  • Run on a VM (Score:3, Insightful)

    by ChuckDriver ( 1276092 ) on Thursday July 22, 2010 @02:32PM (#32994568)
    Ultimately, it would be nice to figure out what script or daemon is running in this and put it on an existing server. In the mean time, I could see just creating a VM for this thing to get started.
  • I agree with above posters, it's amazing to see a useful Slashvertisement. This one, however, has some quality behind it. I had not seen this program and OCR is one area where it's been difficult to find quality OSS solutions. Thanks for the post.
    • You would be amazed at how much the people over at Groklaw could use something like this; since most US court documents are recorded as scanned PDF's and TIFF files. I'm saving this link.
  • Nice, thanks for sharing. Currently we use Acrobat to OCR scanned documents, it seems to work well but doesn't keep up to our high-speed scanners. Having it automated sounds great. How does the speed/accuracy of WatchOCR compare to commercial products?

    • Re: (Score:2)

      by IICV ( 652597 )

      Who gives a shit? My cheapass "free" workflow for OCR-ing PDF documents on Windows was basically what's described here [imagemagick.org]. With this, all I need is to run a virtual server on my computer! That's significantly better.

      • MODI just leaves you with the text pulled out of context. ExactImage's hocr2pdf can merge the OCR'd text back into the original scanned pages to produce a PDF with searchable text and all the original formatting and images.

    • Re:ocr (Score:4, Funny)

      by 0100010001010011 ( 652467 ) on Thursday July 22, 2010 @02:45PM (#32994780)

      Now it just needs to incorporate a Recaptcha Lite to improve accuracy.

      Maybe something on the web interface when it doesn't recognize a word you can correct it.

      [Given the success of the Cow Clicker on Facebook, maybe turn it into a facebook game. Tell people they're only allowed to correct words every 6 hours. If they want to correct more words, they'll have to pay for it. Add friends and correct more words to level up!]

  • Anyone got error rates? (Score:4, Insightful)

    by savanik ( 1090193 ) on Thursday July 22, 2010 @02:39PM (#32994680)

    I was looking for something like this last year - it looks like this just got released last month, so I don't feel too bad about not finding it.

    It looks really interesting, but how accurate is it? I've got some old books that are falling apart I'd like to scan in and textify, but I'd like to know how much time I'm going to have to budget ahead of time fixing problems and proofing.

    • This is tesseract without training so the error rates are going to be high. It doesn't say if it is specifically using the development version, but if it's not, there is no layout analysis. That doesn't stop you from doing the scanning, and then do the OCR sometime in the future. Consider Diybookscanner.org [diybookscanner.org] for a much faster, cheaper, etc. way to scan your books.
  • Is there something similar available commercially anyone can recommend? We may end up needing to scan large amounts of pdf's to a shared drive somewhere and need the whole thing to be searchable for keywords, but a requirement for that would be a commercial product that has 24x7 support.

    • Re: (Score:2, Informative)

      by Anonymous Coward

      After doing a similar search recently, your two major choices are ABBY FineReader (they have Enterprise/Server level editions) or OmniReader (again at the Server/Enterprise level). They're priced pretty closely and have pretty well matched features, plus high accuracy. We're in the process of moving from a solution originally based on Adobe Acrobat's built-in OCR, which is okay but not great. Initial testing with ABBY showed a demonstrably lower error rate on documents from scanned in legal files.

    • Re:commercial? (Score:4, Informative)

      by ganjadude ( 952775 ) on Thursday July 22, 2010 @02:46PM (#32994786) Homepage
      there is! I happen to work for a company (shameless plug) called DocuWare. Its document management software that does all of that., we are not in 24/7 we are in 8 AM-8 PM eastern m-f for support (I am the support) at the corporate level, however we sell through a dealer network that provides support on a contract basis (many Toshiba business solutions are resellers for us, I know they are 24X7) www.docuware.com

      • Re: (Score:1, Offtopic)

        by h4rm0ny ( 722443 )

        You work for the company and you couldn't be bothered making it a proper link? Don't you know that making people have to copy and paste a URL will actually halve the number of referrals? No joke!

      • Re: (Score:3, Funny)

        by FelixNZ ( 1426093 )
        Sole support staff's user name in 'ganjadude' I am a little wary :)
        • sorry, should have been more clear, I am not the only person in support, we have a very well rounded team

    • Re: (Score:2)

      by h4rr4r ( 612664 )

      Why?
      You like giving away money?

      I suggest you install this on your own machine, find a quote for a "commercial 24x7" support solution, then tell your boss your company does the same thing for 1/2 the price.

    • All of the major OCR packages will have a Pro version that will have a drop folder server type setup that will do that. OmniPage and FineReader are the standard options with OmniPage being a little more accurate, but if you want to go budget busting (not kidding even a little) for extreme accuracy go for PrimeOCR [primerec.com]. They also seem to have the consulting and support services you may want. But for searchable PDFs you don't actually need extreme accuracy, just moderate depending on the task, so perhaps support o
  • If this works well, I have a bunch of use for this. Thank you for the heads-up.

  • Your copier providers probably already include this in the package you have. It just hasn't been enabled.

    Our direct-to-pdf document scanners include copies of Acrobat Pro (both Windows and OSX), automatically do OCR, and were less than $400 each.

  • Funny, I was just looking for something to do this the other day.

    But isn't there some middle grown betweeen (a) making users do complicated setup work, vs. (b) making an entire OS out of it?

    How about just making a tarball or Ubuntu/Debian/RPM package that installs and sensibly configures those two tools?

  • I settled on an expensive propriety solution some months ago at work(I am the IT guy, Dishwasher, and Business...something) to do our orgs scan and ocr. Admittedly its end to end including the scanner as well. But $15K and does a good job.

    I did searches online(a dozen hours) and they all funneled back to "FOSS less good, proprietry for best results)

    I am afraid to look at this one, because I did make final decision with pressure from the General Manager.

    I dunno what google uses actually, but their in-
    • Well, since this apparently was just put online recently you probably made the right decision. But since there seems to be so much interest in this thread, hopefully it may becomed polished rather quickly for future needs.
    • In my role it's always better to be aware and try it out than pretend it doesn't exist. You can't thoroughly research every solution in advance every time. We call it due diligence even after the fact because you might find a better way of doing something and it's always good to have options. If nothing else it may also give you some negotiating room with the proprietary vendor at renewal time.

  • Stupid (Score:2, Insightful)

    by Archangel Michael ( 180766 )

    Most, if not ALL of the documents being scanned into PDF format, are generated on computers already, so why go through the whole OCR process, and not get the actual document from the original source in a PDF version that is already text searchable?

    THIS is exactly the problem with document management and processing today! Doing things the hard way because we can't be bothered changing processes that will save tons of money, be more effective, and accurate.

    I know people who type a document in WORD and then pr

    • Re: (Score:2)

      by Big Boss ( 7354 )

      Just about anyone can read a PDF. If you send a MS Word doc, you have to wonder what version of Word the other person has. And these days, Macs are popular enough that they might not have Word at all! PDF works, and works for everyone. It would be far simpler to print to PDF, but not everyone has a print driver that can do that. ODF is supposed to fix that, but it probably won't.

      • You clearly missed the point.

        -CF

      • Print to PDF, ever heard of that?

        OpenOffice Export to PDF, ever heard of that?

        Acrobat Professional, ever heard of that?

        How about copy/paste into email?

        There are plenty of alternatives to take your WORD (or whatever) doc and get it into a searchable PDF without scanning the damn thing into a TIFF and then OCRing it back to text later.

      • Re: (Score:2)

        by Bert64 ( 520050 )

        The "print to pdf" function often creates very poor pdf files, a proper pdf export function in the program is a lot better...
        Get a relatively complex document and compare the output from the native pdf export of openoffice and printing to a pdf file.

        • What is true for OpenOffice may not, and probably is not true for all applications. CorelDraw, yes, I'm looking at you. That purple is supposed to be blue.

        • No matter how bad the print-to-PDF function is, it's is not going to be worse than the print-to-paper-then-scan-and-OCR function.

    • Re:Stupid ... maybe (Score:1, Insightful)

      by Anonymous Coward

      Not everyone wanting to do this does in fact have access to the electronic source. I know I would like to try it for some my old crumbling books, as someone else mentioned above, no longer in print (or otherwise only available in DRM-encumbered ebook formats that I cannot read on Linux or Windows Mobile).

      RO

    • You obviously live in a utopia somewhere. Most of the documents I've seen scanned in to document management may have had their origins on a computer, but they've had signatures, comments, and other stuff penned in by hand, and you can't always get the originals sent to you.

      The poster is addressing a real need, as evidenced by the number of comments proclaiming the usefulness of the post.

      • All of the "exceptions" you listed (signatures, comments penned by hand) are NOT OCRed, making it text searchable as needed by the ORIGINAL concept.

        Changing processes would solve the need to OCR documents that already exist as searchable text elsewhere. EVEN if you have need to document signatures and other hand written notes.

        It is a real need (searchable text), I never said that it wasn't. I'm just quibbling over the process to attain the goal.

    • ... and fax machines still exist because some processes require signatures.

    • That is 100% correct, in theory.

      In the real world, however, process changes in any large organization tend to be slow, expensive, and messy.

      It's often a NECESSITY to look for incremental optimizations and workarounds.

      I saw a situation very similar to the one you are describing. A large organization with 1000+ points of sale, having to snailmail paper documents (i.e. contracts) to the headquarters - everyday.

      Such contracts were produced/printed with a proprietary software solution, in which Cobol (!) code w

  • I have tried twice to download it, and it 'finishes' at about 150mb both times, while the file size on their web page shows over 600mb. As a double-check, (suspecting a file size reporting error on their page), it fails MD5 sum as well. Has anyone successfully downloaded it?

    • Did you try wget? See what error it reports, or try with wget --continue (shorthand -c).

      • Re: (Score:1)

        by Kiralan ( 765796 ) *
        I had the 'short' download error using the link on the home page, which leads to an FTP-like directory page. I am trying the link in the forums, and it appears to be working. Thanks for the advice, though!

  • Tesseract OCR (Score:3, Informative)

    by TheSync ( 5291 ) on Thursday July 22, 2010 @04:35PM (#32996454) Journal

    I found tesseract [google.com] to work very well to do OCR tasks. Doesn't generate PDF though.

  • I can't seem to be able to download this file, it keeps giving up after a couple of hundred megs... probably slashdotted.

  • Can you go one step further with this and get it read the text (only) out of Microsoft formatted files? Maybe it could even read words out of Word files, Powerpoint, etc.

  • I haven't tried it yet, but this looks promising. It isn't free, but it also doesn't seem as pricey as Adobe.

    Qoppa Software [ http://www.qoppa.com/index.html [qoppa.com] ]

  • I wrote a bash script a few months back which, in a little over 130 lines (it has a few command line options), can convert any old PDF to a text searcheable PDF. I really wonder whether a distro is a bit overkill for this? But it is such an important tool to have that I commend the authors for making it available... I just wish they'd put up the actual script that they used so I could compare it to my own!

  • I must be missing something. Why would you want OCR on a server and not as part of the program that interfaces with the scanner?

    • Because you want an internal infrastructure that allows you to replace the scanner easily. Those thing brake down, or more importantly gets replaced with faster new scanners.

      One of my clients has a scanner farm that scans documents, then the images are sent to a OCR server farm. It's way easier to replace any part if it that way (we're actually trying out a new OCR suite right now so I'll test this one).

      Even if you just have one scanner and does OCR on the same machine, braking it up this way makes it e

    • Re: (Score:1, Insightful)

      by Anonymous Coward

      If you are running a high speed scanner that scans 100ppm/200ipm, the computer would not be able to OCR the pages fast enough to keep up with the scanner throughput. Since you are paying good money for that scanner (and the operator running it), you want to get every possible image through that scanner per day. The OCR can be done after the fact on a server that only needs to be periodically monitored by IT.

    • Because not all scanners are directly plugged into user computers with a software interface.
      I have a large office MFP that scans into network shares and with this little server running I can have it watch that share and fix up my PDF's real nice for the users, rather than installing something on everyones computers.

  • I use gscan2pdf for my Linux desktop. I find it's incredibly simple and convenient.
    It just does *everything* I need. It takes scans from scanner, it processes it with OCR, it allows me to delete or insert pages.....it's just very simple and does the job well.

    For OCR, gscan2pdf works with 4 OCR programs currently:
    • GOCR
    • Tesseract
    • Ocropus
    • Cuneiform

    Ocropus is developed with funding/support from Google. It uses tesseract as a backend to do a lot of the work. In simple terms, Ocropus is awesome. I

Slashdot Top Deals

If you steal from one author it's plagiarism; if you steal from many it's research. -- Wilson Mizner

Close