I wanted to use real hand gestures as game input without VR, so I built it myself

Showcase In Development
,
1

Okay so i thought of this really cool idea:
what if a game made you make irl gestures to do actions!!!(So cooooll)

I knoww thats already a game but i wanted to make my own and do it INSIDE GODOT and WITHOUT VR(most of em are vr i think)

Heres a clip of how it works!

(i only showed the hand skeleton since i didn’t wanna show my face but this is live webcam input)

Now this is WIP,
and i only have two actions right now grab and punch

But i will add more, later on i might dive into how i did it and how yall can do it!!
(MAYBE even open-source the code)

Soo yeah gimme your thoughts, i spent likee 70% my day on this lmao.
image
image

(as you can see the functions are most of the code)

7 Likes
2

UPDATE: I GOT A IDEA
I think ill make a beat saber like game like you have to make different gestures and stuff infront of the camera while holding the correct keyboard button and stuff, yk?

2 Likes
3

That’s cool! are you using python?

1 Like
4

yeah!!, im js connecting godot and python, doing all the webcam stuff in python and connecting that to godot so it can act on it.

2 Likes
5


i made a cutscene : D

3 Likes
6


Main menu prototype

1 Like
7

HELL YEAH
(im trying to model everything)
also i used CC0 texures for the 4k texures

3 Likes
8


tuff?

1 Like
9

Super cool! Could something like this be used for full body tracking, like a simplified Kinect?

Either way, I think you could sell this solution, it can be used not only for games but for other interactive works (like interactive digital art installations).

Great work

1 Like
10

hey! thank you!
also, im using MEDIAPIPE so i didint invent the tech,i dont know about full body tracking but i am building a small bridge beetween godot and python and python is detecting the gestures using custom functions i made like detect_fist()
image

2 Likes
11

Impressive, keep on updating us on your progress this is some cool gamedev hacking!
Cheers !

1 Like
12

i will!
Progress is going fast cause i REALLY want to make a cool released project before continuing my game
As i dont have very good knowledge of level design : P

4 Likes
13

pressure plates are so tuff

1 Like
14

This is DOPE!

EDIT:Wait,why is the floor like this? OMG it looks scary! (I am talking about the fear of void)

1 Like
15

TYSMM!! : D
(balh blah blahs extra letters cause it wont let me post)
(also about the flooorrr… yeah idk :sob: i js think it looks cool )

1 Like
16

i added cool camera effects but this sh is lagging on a RX 570 and i want it to run on a gtx 750 TI at LEAST
any tips?

1 Like
17

Doesn’t seem like it’s lagging? Also it doesn’t exactly help you recording it an that is using some pc resources. Also lag isn’t all from just graphics card.

18

Your right!
This lwk made me realise that I think it’s cause the hand tracking is very heavy on the cpu side
And I don’t have a really gud cpu(I’ll upgrade soon) so yeah.

19

image
image790×633 39 KB

new rock model

1 Like
20

Rockest rock that was ever modeled…

3 Likes