Jiàoyù zīliào yǔ túshūguǎn xué (Mar 2003)
中文OCR文件檢索測試集之製作與應用 Construction and Application of an Chinese OCR Test Collection for Information Retrieval
Abstract
本文描述一套中文OCR檢索測試集的建構過程及其實際的檢索應用。我們克服回溯性資訊需求難以獲得的困難，擬定出30道模擬使用者需求的查詢主題。為獲得真實的OCR文件，我們以OCR軟體將8439篇全文影像轉換成數位檔案，並評估其辨識率在7成上下。為了求得每一道查詢主題的相關文件，我們邀請三位人員分別檢視並判斷每一篇文件是否跟查詢主題相關。經由Kendall和諧係數的統計驗證，這三位判斷者在20道查詢主題上，相關判斷的結果非常一致，顯示標準答案(即相關文件)有足夠的共識。最後，以12種檢索策略來比較OCR文件的檢索成效，我們發現辨識率降低到7成的情況下，檢索成效差不多也降低到7成左右。This article describes the process of constructing a Chinese OCR test collection and the application of this collection in an retrieval experiment. We have overcome the difficulty of obtaining past information need for retrospective data and created 30 query topics that simulate real user needs. To obtain real OCR documents instead of simulated ones, we have converted 8439 full-text images into 8439 OCR text files.An evaluation of the OCR documents reveals an average of 70% of recognition accuracy.To obtain the relevant documents for each query, we invited 3 judges to examine each of 8439 images and give relevance score to each document for each topic.According to Kendall s statistical coefficient, highly consistent judgments are obtained in 20 query topics. Finally in our experiment with 12 search strategies,our results show that the retrieval effectiveness of OCR documents decrease to 70% when the recognition accuracy is about 70%.
