In optical scanning holography, one pupil produces a spherical wave and another produces a plane wave. They interfere with each other and result in a fringe pattern for scanning a three-dimensional object. The resolution of the hologram reconstruction is affected by the point spread function (PSF) of the optical system. In this paper, we modulate the PSF by a spiral phase plate, which significantly enhances the lateral and depth resolution. We explain the theory for such resolution enhancement and show simulation results to verify the efficacy of the approach.