XCompress: LLM assisted Python-based text compression toolkit

Abstract

This study introduces XCompress, a Python-based tool for effectively utilizing various compression algorithms. XCompress offers manual, brute force, and Large Language Model (LLM) methods to determine the most suitable algorithm based on the type of text data. Its modular structure allows easy addition of new algorithms and includes functions for benchmarking and result comparison. Tests on diverse text types demonstrate the efficacy of the LLM-assisted Compression Selection Model (CSM). With XCompress, users can determine the most suitable method for their files. Additionally, in academic research, they can easily compare different methods without needing any scripting or programming language.

Keywords

• Benchmarking
• Large language models
• Text compression
• Python