When I had a 709 it was the same as you are describing. I took my rear sight off as well (to do a slide melt) and there was no burr or debris under the sight. I came to the conclusion that it was the result of the elevation adjustment screw. The 709 is known to shoot low and or left out of the box. When I adjusted the rear sight, the little cam on the screw held the right side a little higher. If it bothers you, consider replacing them with some after market sights. Maybe night sights?