Implementation of Data Backup and Synchronization Based on Identity Column Real Time Data Warehouse

Abstract

Failure in the process of loading data from the Online Transactional Processing(OLTP) system to the Normalized Data Store (NDS) database can occur. This caused by a disruption in the network so that the OLTP system is unable to save data to the OLTP and NDS databases. Backup and synchronization data scenarios are needed to maintain data consistency and data availability. In this research, the process of data backup and synchronization is done by providing an identity column for the table in the OLTP database. An identity column is used to give status data, value '0' if the inserting process fails, and value '1' if successful. Data backup is done by storing temporary data into a CSV file format, then the CSV file is read, and an insert process is carried out into the OLTP database. After the insertion process into the OLTP database is successful, it continues with the synchronization process between the OLTP database and the NDS. Data synchronization between OLTP and NDS databases is done by checking the value of the Identity Column in each table in the OLTP database.