I do that too and never worry about it. When I was a kid, the neighbors garage caught fire. It was totally engulfed and I could feel the heat in my yard as I watched the firemen put out the blaze. Shotgun shells began exploding. After the fire was out, I went over there the next day and found several shells that did not explode, but they had been burnt pretty bad. That tells me it would have to be pretty hot - fire burning hot - to get them to explode.