A scientific-article key-insight extraction system based on multi-actor of fine-tuned open-source large language models

Abstract

Abstract The exponential growth of scientific articles has presented challenges in information organization and extraction. Automation is urgently needed to streamline literature reviews and enhance insight extraction. We explore the potential of Large Language Models (LLMs) in key-insights extraction from scientific articles, including OpenAI’s GPT-4.0, MistralAI’s Mixtral 8 × 7B, 01AI’s Yi, and InternLM’s InternLM2. We have developed an article-level key-insight extraction system based on LLMs, calling it ArticleLLM. After evaluating the LLMs against manual benchmarks, we have enhanced their performance through fine-tuning. We propose a multi-actor LLM approach, merging the strengths of multiple fine-tuned LLMs to improve overall key-insight extraction performance. This work demonstrates not only the feasibility of LLMs in key-insight extraction, but also the effectiveness of cooperation of multiple fine-tuned LLMs, leading to efficient academic literature survey and knowledge discovery.

Keywords

• Key-insight extraction
• Large Language Model fine-tuning
• Automatic information extraction