In milling processes, chatter is a kind of sudden relative vibration appearing between the cutter and the workpiece, which results in poor part quality, accelerated tool wear, and shortened spindle life. In this paper, an open modular architecture controller (OMAC) of machine tool which integrates the algorithms including chatter recognition, compensation command generation, and execution is proposed with the aim of providing an integrated solution for milling chatter suppression in CNC kernel. To effectively identify chatter, experiments are designed to determine the optimal installation place of accelerometer and then triaxial cutting forces and acceleration signals are compared to see which are more sensitive to chatter onset. In terms of data processing, 16 sampling points in time domain are chosen to perform online fast Fourier transform (FFT) in consideration of signal effectiveness and computational efficiency. To implement real-time chatter suppression in CNC kernel, a simplified dynamic model of milling system is used to obtain the relationship between chatter frequency and spindle speed. Finally, an adaptive control module which completes force signal extraction and processing by FFT and has the ability to modify related cutting parameters is designed to interact with other modules in OMAC where data acquisition thread and interpolation thread are synchronized. The proposed system is experimentally validated.