This might be because data are not in the proper unit, because a mask is needed, or because of a global shift of the interferogram (ie. data with no displacements are not close to null). The global shift typically shows with quadtree subsampling. You should also mask unwanted data.