The difference is where the check happens. @export_range is validated by the GDScript parser, and that annotation is registered as expecting a numeric type, so a Vector3 gets rejected before the hint ever reaches the inspector. PROPERTY_HINT_RANGE itself has no issue with vectors, the editor applies the range to each component.
@export_custom skips that validation step. The docs say GDScript performs no validation on the syntax there, it just passes your hint string through to the property info, which is why yours works. The tradeoff is that a typo in that string won’t error either, it’ll just behave oddly in the inspector.