This is not really a trivial problem. Determining how much two shapes overlap is more complex than simply checking if they overlap. The degree to which two shapes overlap can be described by the shape resulting from their intersection.
Generally, you should define the maximum potential overlap as the area of the smallest shape (max_overlap = min(A, B)).
However, depending on your shapes, it may be impossible to achieve complete overlap. Take this into account when making gameplay systems that rely on overlapping shapes (e.g. by padding the overlap value or calibrating the maximum overlap beforehand).
You mention that you’re using triangles as the shape for your area(s). However, I don’t think the shape you’re using matters all that much. Unless you’re using a circle, the configuration of the two shapes is too varied to implement a specialized implementation. You need a generic approach that can compute the intersection between two arbitrary shapes.
From my quick investigation into this issue I found the topic of clipping algorithms: the type of algorithm you’re looking for. There seems to be a few proven algorithms out there, each with their own benefits and drawbacks. The approach I’ve found that looks the most promising is the Weiler–Atherton clipping algorithm (see Wiki).
I don’t have the energy to implement the technique for you at the moment. You should be able to find videos or papers on this (or other) algorithms though.
Once you have computed the intersection between your two shapes, you have to compute the area of the intersection such that you can arrive at the percentage metric you’re looking for.
Computing the area can be done with the Shoelace formula (see Wiki). Again, I don’t have the energy right now but the steps seem pretty straightforward.
After all of this hard work acquiring the intersection area, you should just be able to divide the area of the intersection with the smallest area from the two shapes you’re using.
var A, B # Your shapes
var overlap_percentage = intersection_area / min(A, B)
I haven’t looked at Godot’s Area2D, CollisionShape2D, or CollisionPolygon2D class references for any native utility functions. Please check the documentation before implementing the discussed algorithms above. It would be a shame if you later find out that Godot is capable of computing these things for you.
If you need help with the algorithms, I’ll be happy to help with any questions you may have.