I doubt you can remove enough material to get it to work without destroying its ability to hold the cartridges.
It's just another case of too small of a revolver.
I believe you are stuck using speed strips or loading them with your fingers,

 

Speed strips are pretty fast and are FAR easier to carry. If you don't try to shove the loader all the way into the cylinder, you can release the cartridges and they'll load, but it's gotta pretty much be muzzle down.

I prefer the speed strips.

 

yes, i have ground HKS speedloaders to fit around different grips... for example, a Charter Arms revolver:

note that 5 Star may make a speedloader that will work for you (but they cost 3X as much); also note that i simply rounded between each chamber on the *top* of the loader - that may be enough to slip the loader/rounds into your cylinder... here's another pic for a Pathfinder:

again, note that only the top of the loader has been sanded - i used a small Dremel drum - go slow, and hope this helps

btw, i got a chuckle out of this little tidbit on the back of the HKS speedloader package:

hmm, modify an $8 plastic speedloader or a pair of $50+ grips? lemme think on that for a minute...

willie
on the Gulf of Mexico

 

You might look into the 5 Star speedloader. It has about as thin of material as you're going to get.

 

WOW!
ur speedy loader thangs are lot different and fatter than the ones fur my guns!
mine go in great.

 

Try Safariland® Comp 1 speedloaders, much thinner footprint than HKS®.