A Deep Learning Model for Automatic Citation Document Recommendation in Non-Obviousness Judgment: Using BERT-for-patents and Contrastive Learning

Abstract

Patent laws in various countries stipulate that inventions identical to or easily derivable from prior art lack novelty and non-obviousness, rendering them ineligible for registration. To assess these criteria, prior art searches are conducted. The evaluation of non-obviousness is challenging because of the difficulty in assessing obviousness and the possibility of utilizing multiple citation documents. Therefore, an artificial intelligence (AI) model that can preliminarily filter prior art references relevant to non-obviousness determination would enhance the efficiency and speed of prior art searches. To address this need, this study proposes a deep learning model that automatically recommends additional citation documents corresponding to the remaining elements of an invention when provided with some elements and the corresponding citation documents. The United States Patent and Trademark Office (USPTO) patent data rejected because of a lack of non-obviousness were preprocessed. Six models were trained based on the bidirectional encoder representations from transformers (BERT), and the performances were compared. The model TRP-Pat, trained using a contrastive learning approach with BERT-for-patents, demonstrated significantly superior performance. These results suggest that TRP-Pat can contribute to more efficient prior art searches by expediting the process. An example of applying the TRP-Pat model to prior art search tasks is also presented.

Keywords

• non-obviousness
• citation documents
• prior art search
• deep learning
• contrastive learning
• bert-for-patents