# A camera extension?

**URL:** https://forum.makecode.com/t/a-camera-extension/5950
**Category:** Game Design
**Created:** [February 4, 2021, 4:17pm UTC](https://forum.makecode.com/t/a-camera-extension/5950 "2021-02-04T16:17:32Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![Agent\_14](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/agent_14/32/17567_2.png) [@Agent\_14](https://forum.makecode.com/u/Agent_14)
#### Post date: [February 4, 2021, 4:17pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/1 "2021-02-04T16:17:32Z")

</div>

Is it possible to have an extension that supports camera input? For example, `on movement detection [Dropdown: Small, Medium, Large]` and `on sprite collides color`. This would be compatible on a computer with a webcam, and maybe on hardware where a camera can be connected (like a Raspberry Pi 4).

---

<div class="post-metadata">

### Author: ![UnsignedArduino](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/unsignedarduino/32/592_2.png) [@UnsignedArduino](https://forum.makecode.com/u/UnsignedArduino)
#### Post date: [February 4, 2021, 5:05pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/2 "2021-02-04T17:05:07Z")

</div>

Probably not, because Arcade doesn’t expose anything about peripherals like keyboard or mice or cameras. 😕

---

<div class="post-metadata">

### Author: ![Agent\_14](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/agent_14/32/17567_2.png) [@Agent\_14](https://forum.makecode.com/u/Agent_14)
#### Post date: [February 4, 2021, 5:17pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/3 "2021-02-04T17:17:57Z")

</div>

😭 😭 😭

---

<div class="post-metadata">

### Author: ![eanders](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/eanders/32/3134_2.png) [@eanders](https://forum.makecode.com/u/eanders)
#### Post date: [February 4, 2021, 6:00pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/4 "2021-02-04T18:00:54Z")

</div>

So… I made this. [https://eanders-ms.github.io/arcade-webcam/](https://eanders-ms.github.io/arcade-webcam/). Might be a good point to start from to add gesture detection, color cutout, etc.

---

<div class="post-metadata">

### Author: ![eanders](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/eanders/32/3134_2.png) [@eanders](https://forum.makecode.com/u/eanders)
#### Post date: [February 4, 2021, 6:01pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/5 "2021-02-04T18:01:31Z")

</div>

Only works in the simulator.

---

<div class="post-metadata">

### Author: ![jwunderl](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/jwunderl/32/5308_2.png) [@jwunderl](https://forum.makecode.com/u/jwunderl)
#### Post date: [February 4, 2021, 6:31pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/6 "2021-02-04T18:31:16Z")

</div>

For anyone else that comes across this looking for ways to extend arcade with their own special features / controls / simulators, a lot can be done in the same way as this one using the simmessages – this is where the makecode project says to listen for the messages / how to handle them:

> <https://github.com/eanders-MS/arcade-webcam/blob/e9fb61f120fb1d251ed445f7e55cdeb10299a9c8/main.ts#L30>

and this is where the outer website sends the messages to the frame:

> <https://github.com/eanders-MS/arcade-webcam/blob/e9fb61f120fb1d251ed445f7e55cdeb10299a9c8/assets/js/custom.js#L105>

and here’s where all the stuff related to this is defined in common packages:

> <https://github.com/microsoft/pxt-common-packages/blob/master/libs/base/sim/controlmessage.ts#L3>

Again, worth noting this is simulator only, and only works when hosted from your own website – but it’s fairly easy to make your own site if you want to play around with this thanks to the github integration creating a github pages page like Eric’s automatically 🙂 ~

---

<div class="post-metadata">

### Author: ![GameGod](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/gamegod/32/2756_2.png) [@GameGod](https://forum.makecode.com/u/GameGod)
#### Post date: [February 4, 2021, 6:31pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/7 "2021-02-04T18:31:45Z")

</div>

That is so cool!

---

<div class="post-metadata">

### Author: ![eanders](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/eanders/32/3134_2.png) [@eanders](https://forum.makecode.com/u/eanders)
#### Post date: [February 4, 2021, 6:47pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/8 "2021-02-04T18:47:50Z")

</div>

Thank you for explaining this @jwunderl! It was fairly straight forward to extend Arcade like this, once I knew what the key parts were. My first iteration of this project was ridiculously complicated because I didn’t know about these extension points :).

One limitation I kept hitting was the fact that this kind of extension can’t run in the editor. I wonder if these extension points could be packaged in a way that allowed simulator extensibility.

---

<div class="post-metadata">

### Author: ![Agent\_14](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/agent_14/32/17567_2.png) [@Agent\_14](https://forum.makecode.com/u/Agent_14)
#### Post date: [February 4, 2021, 7:06pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/9 "2021-02-04T19:06:31Z")

</div>

The camera feature was what I was asking for exactly, just that know I need a on color overlap extension like scratch’s. Like `on sprite touches color` or `color touches color`

---

<div class="post-metadata">

### Author: ![GameGod](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/gamegod/32/2756_2.png) [@GameGod](https://forum.makecode.com/u/GameGod)
#### Post date: [February 4, 2021, 7:33pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/10 "2021-02-04T19:33:58Z")

</div>

Wait, so can you add keyboard/mouse inputs, instead of just A, B, and the d-pad?

---

<div class="post-metadata">

### Author: ![GameGod](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/gamegod/32/2756_2.png) [@GameGod](https://forum.makecode.com/u/GameGod)
#### Post date: [February 4, 2021, 7:34pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/11 "2021-02-04T19:34:01Z")

</div>

Can you add this as an extension, where we can make the sprite’s image the camera?

---

<div class="post-metadata">

### Author: ![eanders](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/eanders/32/3134_2.png) [@eanders](https://forum.makecode.com/u/eanders)
#### Post date: [February 4, 2021, 9:07pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/13 "2021-02-04T21:07:02Z")

</div>

@GameGod Yep it should be possible to use the webcam image as a sprite. Would be neat to do a green screen type background removal. That might be better achieved on the image capture side, rather than in Arcade.

---

<div class="post-metadata">

### Author: ![Agent\_14](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/agent_14/32/17567_2.png) [@Agent\_14](https://forum.makecode.com/u/Agent_14)
#### Post date: [February 4, 2021, 10:29pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/14 "2021-02-04T22:29:05Z")

</div>

Apple with camera

 ![SmartSelect_20210204-145843_Samsung Internet](https://us1.discourse-cdn.com/flex020/uploads/makecode/original/2X/2/29f486df44e761ed0b1fd583bcdeeaae2f514538.jpeg)

---

<div class="post-metadata">

### Author: ![omnisImperium](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/omnisimperium/32/2728_2.png) [@omnisImperium](https://forum.makecode.com/u/omnisImperium)
#### Post date: [February 4, 2021, 11:43pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/15 "2021-02-04T23:43:58Z")

</div>

Hello. Could you possible use the same code with some modifications to allow for mouse inputs or other keyboard inputs. I, and maybe many others, would be grateful for such an extension

---

<div class="post-metadata">

### Author: ![eanders](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/eanders/32/3134_2.png) [@eanders](https://forum.makecode.com/u/eanders)
#### Post date: [February 5, 2021, 3:37am UTC](https://forum.makecode.com/t/a-camera-extension/5950/16 "2021-02-05T03:37:45Z")

</div>

Yes this code would serve as a good base for projects that want to support other kinds of input like mouse or keyboard.

---

<div class="post-metadata">

### Author: ![omnisImperium](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/omnisimperium/32/2728_2.png) [@omnisImperium](https://forum.makecode.com/u/omnisImperium)
#### Post date: [February 5, 2021, 4:28pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/17 "2021-02-05T16:28:20Z")

</div>

If you have time and wouldn’t mind doing so, could you possible code such an extension. I have no real JS / TS coding experience.

---

<div class="post-metadata">

### Author: ![GameGod](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/gamegod/32/2756_2.png) [@GameGod](https://forum.makecode.com/u/GameGod)
#### Post date: [February 5, 2021, 4:29pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/18 "2021-02-05T16:29:12Z")

</div>

Can someone make an extension for it?

---

<div class="post-metadata">

### Author: ![eanders](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/eanders/32/3134_2.png) [@eanders](https://forum.makecode.com/u/eanders)
#### Post date: [February 5, 2021, 7:50pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/19 "2021-02-05T19:50:12Z")

</div>

Sure I’ll take a stab at it this weekend. I’ll start a new topic for it!

---

<div class="post-metadata">

### Author: ![Agent\_14](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/agent_14/32/17567_2.png) [@Agent\_14](https://forum.makecode.com/u/Agent_14)
#### Post date: [February 5, 2021, 8:06pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/20 "2021-02-05T20:06:27Z")

</div>

I think a new category called extension on the forums would be nice.

---

<div class="post-metadata">

### Author: ![UnsignedArduino](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.makecode.com/unsignedarduino/32/592_2.png) [@UnsignedArduino](https://forum.makecode.com/u/UnsignedArduino)
#### Post date: [February 5, 2021, 8:58pm UTC](https://forum.makecode.com/t/a-camera-extension/5950/21 "2021-02-05T20:58:08Z")

</div>

[https://forum.makecode.com/tag/extension](https://forum.makecode.com/tag/extension)

[Next page](https://forum.makecode.com/t/a-camera-extension/5950.md?page=2)
