So the SPEED of the detonation (hence high and low velocity) is what causes the damage.
Right, basically this gets into momentum vs force (1st derivation, change in momentum over time) vs impulse (2nd derivation, force over time). Another maybe more intuitive example is the operation of firearms. If one takes a moment to think about it, obviously both the shooter and the target being shot must experience near identical (the target gets less but it's not significant within effective range) changes in momentum since momentum is conserved, yet of course the effect on the target is rather different, that's the whole point after all. Guns fundamentally work by ensuring the operator experiences the force spread out over time and area relative to the target, aiming to keep it within an elastic regime whereas the target experiences inelastic effects. You can easily exceed the force of a bullet just by pressing down firmly on your arm with a finger for a bit. It's the compression of time and area that makes all the difference.