Hadoop Processing Methods for Large Scale Video Data

Abstract

As the continuous advancement of the We-media era, there are more and more short video users. However, there are still problems with low data processing efficiency and serious video damage in the current processing framework for large-scale video data. To address this issue, research is conducted to improve the artificial fish swarm algorithm using particle swarm optimization algorithm, and the improved algorithm is applied to the distributed system architecture Hadoop to enhance Hadoop’s computation speed and accuracy of data. The study first conducts comparative experiments on the improved algorithm. The outcomes denoted that the accuracy of the improved algorithm was as high as 98.7%, while the error rate was only 0.9%. The recall and precision of video data were 92.1% and 96.5%, respectively, and the computational performance was significantly better than other comparative algorithms. The Hadoop data processing framework optimized after the improved algorithm was further tested, and the outcomes denoted that the response accuracy of the optimized data processing framework was as high as 98.6%, and the response time was only 0.32 seconds. From the above outcomes, the proposed improved algorithm can optimize the processing capability of the data processing framework for large-scale video data. Moreover, the optimized data processing framework can not only be used for large-scale video data processing, but also widely used in other fields to raise computational accuracy.

Keywords

• Video data
• particle swarm optimization algorithm
• artificial fish swarm algorithm
• Hadoop framework
• data processing