A web scraping app for smart literature search of the keywords

Abstract

Detailed literature search and writing is very important for the success of long research projects, publications and theses. Search engines provide significant convenience in research processes. However, conducting a comprehensive and systematic research on the web requires a long working process. In order to make literature searches effective, simple and comprehensive, various libraries and development tools have been created and made available. By using these development tools, research processes that may take days can be reduced to hours or even minutes. Literature review is not only necessary for academic studies, but it is a process that should be used and performed in every field where new approaches are adopted. Literature review is a process that gives us important ideas about whether similar studies have been conducted before, which methods have been used before and what has not been addressed in previous studies. It is also of great importance in terms of preventing possible copyright problems in future studies. The main purpose of this study is to propose an application that will facilitate, speed up and increase the efficiency of literature searches. In existing systems, literature searches are performed by browsing search sites or various article sites one by one and using the search tools provided by these sites. It is simple to use, allows the entire World Wide Web environment to be searched, and provides the user with the search findings. In this study, we have implemented an application that allows the crawling of the entire World Wide Web environment, is very simple to use, and quickly presents the crawl findings to the user.

Keywords

• Literature search framework
• Web crawling
• Web scraping
• Smart literature search
• Scraping and crawling bots
• Knowledge discovery