I’ve got a vague idea of what might be happening, but apologies if this is a red herring.
It sounds like Box2D requires multi-threading, meaning you therefore have to export with multi-threading enabled. For web exports that means using a Web feature called SharedBufferArrays, which technically works fine, but requires the hosting website to support specific HTML headers to function correctly.
As quite a few hosting sites don’t support the correct headers, the Godot team recently updated web exports so they don’t require multi-threading. While that is fine in general, it clearly won’t help you if you’re using a physics engine that requires multiple threads.
Itchi.io has a good breakdown of the headers needed to support SharedArrayBuffers, so check if your chosen hosting sites can support those headers, otherwise you’ll need to try hosting it somewhere else.
Having been through this for my own game, I ended up disabling my HTML demo, although I’m going to retry in a week or so. Therefore keen to know how you get on.