You are correct, using the supplied allen wrench loosen the set screw in the rear sight, I wouldn't remove it as it could get lost real easy. Using a small plastic mallet or plastic drift pin (an old cut off toothbrush handle works good for this) drift the rear sight in the direction you want to bullet to go and retighten the set screw.
Des
Des