Fourier ptychographic microscopy (FPM), an emerging computational imaging technology, can realize wide-field and high-resolution simultaneously and thus achieves great success in biomedical fields. The computational imaging process, however, also accumulates various but inevitable noises into measurements. Conventional deep based reconstruction methods simply ignore the existence of noise and integrate all features from measurements for prediction, which severely degrades the quality of reconstructed images. To mitigate this issue, we present a novel network, dubbed DeUnet, that adaptively filters out noise during reconstruction through a new cross-level channel attention mechanism. Specifically, DeUnet comprises of three key stages: encoding, denoising and decoding. The encoding stage abstracts hierarchical features from input measurements. The denoising stage learns to re-calibrate channel-wise feature responses from multiple levels in a collaborative manner. It selectively highlights informative channels while suppressing noisy ones by exploiting long-range dependencies among features across multiple resolutions. The decoding stage integrates denoised features to reconstruct final intensity and phase images. Extensive experiments on both the simulated and real data well demonstrate the effectiveness of DeUnet in suppressing noise. In particular, DeUnet can produce reconstruction results of higher quality and more valuable details, especially for phase images, compared to previous state-of-the-arts.