I just figured out how to integrate Firebase Cloud Messaging Push Notification with Godot Game engine for android from a backend server(Node.JS) as well as from Firebase console without any third party library. Check the video here.
If you like more videos on other Firebase products do let me know in the replies