# Computer Vision Gamepad Hack

**URL:** <https://forum.makecode.com/t/computer-vision-gamepad-hack/3557>\
**Category:** Arcade\
**Created:** [September 21, 2020, 12:22am UTC](https://forum.makecode.com/t/computer-vision-gamepad-hack/3557 "2020-09-21T00:22:08Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![FlyingFox](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/flyingfox/32/1048_2.png) [@FlyingFox](https://forum.makecode.com/u/FlyingFox)\
**Post date:** [September 21, 2020, 12:22am UTC](https://forum.makecode.com/t/computer-vision-gamepad-hack/3557/1 "2020-09-21T00:22:09Z")

</div>

Hey friends!  
I was jealous of Scratch’s “Video Sensing”, so I wanted to see what it would take to do it with MakeCode Arcade. I cobbled together a bit of Python script that measures the optical flow of the entire camera view (the direction of most of the motion) and sends keypresses to whichever app has the operating system focus. You can see the window from the script on the right, labeling the direction detected.

[![](https://img.youtube.com/vi/88rcq-tsggg/hqdefault.jpg "Rough Hack MakeCode Arcade Computer Vision Controller") ](https://www.youtube.com/watch?v=88rcq-tsggg)

My repo is here:

> **[GitHub - UnicycleDumpTruck/cvdpad: Computer Vision Direction Pad Game Controller](https://github.com/UnicycleDumpTruck/cvdpad)**
>
> Computer Vision Direction Pad Game Controller. Contribute to UnicycleDumpTruck/cvdpad development by creating an account on GitHub.

I don’t expect anyone to use this, I just wanted to share the fun. Would something like this be possible as an extension? Can anyone say if Microsoft is considering adding video sensing?