A camera extension?

Is it possible to have an extension that supports camera input? For example, on movement detection [Dropdown: Small, Medium, Large] and on sprite collides color. This would be compatible on a computer with a webcam, and maybe on hardware where a camera can be connected (like a Raspberry Pi 4).

1 Like

Probably not, because Arcade doesn’t expose anything about peripherals like keyboard or mice or cameras. :confused:

4 Likes

:sob: :sob: :sob:

1 Like

So… I made this. https://eanders-ms.github.io/arcade-webcam/. Might be a good point to start from to add gesture detection, color cutout, etc.

6 Likes

Only works in the simulator.

4 Likes

For anyone else that comes across this looking for ways to extend arcade with their own special features / controls / simulators, a lot can be done in the same way as this one using the simmessages – this is where the makecode project says to listen for the messages / how to handle them:

and this is where the outer website sends the messages to the frame:

and here’s where all the stuff related to this is defined in common packages:

Again, worth noting this is simulator only, and only works when hosted from your own website – but it’s fairly easy to make your own site if you want to play around with this thanks to the github integration creating a github pages page like Eric’s automatically :slight_smile: ~

6 Likes

That is so cool!

1 Like

Thank you for explaining this @jwunderl! It was fairly straight forward to extend Arcade like this, once I knew what the key parts were. My first iteration of this project was ridiculously complicated because I didn’t know about these extension points :).

One limitation I kept hitting was the fact that this kind of extension can’t run in the editor. I wonder if these extension points could be packaged in a way that allowed simulator extensibility.

2 Likes

The camera feature was what I was asking for exactly, just that know I need a on color overlap extension like scratch’s. Like on sprite touches color or color touches color

Wait, so can you add keyboard/mouse inputs, instead of just A, B, and the d-pad?

2 Likes

Can you add this as an extension, where we can make the sprite’s image the camera?

2 Likes

@GameGod Yep it should be possible to use the webcam image as a sprite. Would be neat to do a green screen type background removal. That might be better achieved on the image capture side, rather than in Arcade.

Apple with camera

2 Likes

Hello. Could you possible use the same code with some modifications to allow for mouse inputs or other keyboard inputs. I, and maybe many others, would be grateful for such an extension

Yes this code would serve as a good base for projects that want to support other kinds of input like mouse or keyboard.

1 Like

If you have time and wouldn’t mind doing so, could you possible code such an extension. I have no real JS / TS coding experience.

Can someone make an extension for it?

2 Likes

Sure I’ll take a stab at it this weekend. I’ll start a new topic for it!

3 Likes

I think a new category called extension on the forums would be nice.

https://forum.makecode.com/tag/extension