OpenCV Godot Extension

Godot Version

Godot V4.2.1

Question

I am trying to create a GDExtension that connects to OpenCV using C++. The idea end result is to have a user draw an image in game, have the same save the file, and then send the image through the extension, analyze it with OpenCV, and spit out what it matches to from my dataset. I’ve proven out this functionality in a standalone C++ script that utilizes the same libraries and directories as my Godot project. I’m currently running into a problem that I believe stems from sending a Godot string to my C++ extension, changing that to a C++ standard string, then having OpenCV analyze it, and sending the results back. When I attempt to send the file path of the photo I’d like to analyze through to my C++ script and OpenCV, I receive an error marker indicating the file is not readable by OpenCV. I’ve tried hardcoding the file reference through my system path as well as using the relative path from my project. I can also confirm the test.png can be analyzed by OpenCV because I used the same exact file in the same exact location in my standalone script. Here is the section of my C++ script that is erroring when given the file path from my GDScript in Godot:

std::pair<std::string, double> PatternMatcher::recognize_pattern(const String& input_image_path) {

std::string path = input_image_path.utf8().get_data();

//std::cout << "Converted path: " << path << std::endl;

cv::Mat input_image = cv::imread(path);

if (input_image.empty()) {
    path = path.c_str();
    throw std::runtime_error("Failed to open image file piss and poop: " + path);
}

cv::Mat input_image_gray;
cv::cvtColor(input_image, input_image_gray, cv::COLOR_BGR2GRAY);

std::string best_match;
double best_match_percentage = 0;

auto fire_templates = load_templates("Fire");
auto air_templates = load_templates("Air");
auto earth_templates = load_templates("Earth");
auto water_templates = load_templates("Water");

auto process_templates = [&](const std::vector<cv::Mat>& templates, const std::string& element) {
    for (const auto& template_img : templates) {
        cv::Mat result;
        cv::matchTemplate(input_image_gray, template_img, result, cv::TM_CCOEFF_NORMED);
        double min_val, max_val;
        cv::minMaxLoc(result, &min_val, &max_val);

        if (max_val > best_match_percentage) {
            best_match_percentage = max_val;
            best_match = element;
        }
    }
    };

process_templates(fire_templates, "Fire");
process_templates(air_templates, "Air");
process_templates(earth_templates, "Earth");
process_templates(water_templates, "Water");

return { best_match, best_match_percentage };

}

It keeps throwing the input_image.empty() error, but it gives me the correct path in the print line. I assume this problem has something to do with the conversion of a Godot String to an acceptable string by OpenCV for the filepath, but I can’t see why my code doesn’t already account for that. Any suggestions?

Loads an image from a file.

The function imread loads an image from the specified file and returns it. If the image cannot be read (because of missing file, improper permissions, unsupported or invalid format), the function returns an empty matrix ( Mat::data==NULL ).

I would check these things.

Is the path absolute?
Is the path correct?
Does process have permission?
Wrong format?

I believe the solution is one of these. I got OpenCV running with Godot for this. The code works and is returning an image that isn’t empty.