Go to the Paint tab beside it, where you can specify some attributes, like physics layer (with shape of course), and apply (that’s why we call it “paint”) to any tile in the tile set.
That solves it, though I kind of want to copy/paste one tile to another, meaning starting my painting with an already setup tile’s value. Any way to do that? I don’t see a simple eyedropper tool anywhere, for example. It would be easy that way.