So, I figured out the solution.
Since I was using a tween to move the player, I just created another tween and ran in parallel with it.
I added the calculation for the offset of the object in relation to player and added that to the target destination.
No more sprite shakes or causing the player to get moved a few extra pixels.
Now to detect collisions for the new object that I have attached to my player!