I’m poking around a little bit at the internals of MakeCode Arcade. I like the ability to play melodies, but I feel like we’re missing the “pew pew” and “crash bang” types of sound effects that are also typical in arcade games.
Specifically, I’m looking for a way to play WAV file data like what is done in MakeCode for Lego Mindstorms EV3. In Arcade, it looks like music.playInstructions() plays sound from a buffer. Does it work the same way as sounds.play() works in Mindstorms, or is there some other way to hook into the sound system in Arcade to play WAV data?
We still have some work to do around sound. Eventually we would like to have a sound designer similar to the sprite image editor so that kids can “shape” their own sounds.
Brilliant; thanks for the update, @peli. A sound designer would be super cool! A set of stock sounds, though, would be perfectly sufficient to begin with, similar to the stock melodies list.
The engine already has support for creating basic sound effects from waveforms with frequency and volume sweeps using music.playInstructions(), but this didn’t have a Blocks interface, and was a bit clunky to use from JavaScript since it involves modifying buffers.
I’ve recently added engine code for new waveforms for more sound effect variety, including tunable noise and distorted square waves. This is available in the beta editor but not yet on the main site. There’s more information in this rather long thread. You can try a demo using this link to the beta editor: https://arcade.makecode.com/beta#pub:_JL6P9m5fYg9u
This uses the pxt-sound-effects extension which adds a Blocks interface to make this a bit more accessible:
If you want to use it in your own project, click on Advanced / Extensions in the editor’s blocks panel, and copy the extension’s address into the search field: https://github.com/klausw3/pxt-sound-effects
The extension works for the current version of MakeCode Arcade, but the new waveforms (tunable noise and cycle 16/32/64) will just produce silence if you’re not using the beta editor.
Just to clarify, my previous answer wasn’t quite what the original poster was asking for. music.playInstructions() uses small buffers of instructions for controlling synthesized waveforms. There’s no support currently for playing samples or WAV data directly, and I’m not convinced that adding that functionality would be a good idea since these use a lot of memory and storage space.
However, I think you can go a long way with synthesized sound effects if there’s enough variety in waveforms. The code I added was intended to sound similar to old 8-bit computers such as the Atari POKEY sound chip, here’s an example what it sounds like. (Warning: loud. Watch your speaker volume.)
I think this isn’t quite comparable. As far as I know MIT Scratch runs on modern computers, while MakeCode Arcade targets comparatively low-powered standalone hardware or emulated versions of that. Sampled audio requires about 22 kBytes for a single second of mono 8-bit audio at 22 kHz, and that’s a lot when some target platforms may only have 96 kByte of RAM available.
Last I checked, I think that LEGO Mindstorms cheats a bit and plays sound effects on the tablet which is controlling the robot, so the samples don’t need to be downloaded onto the hardware platform. The downside of that is that the programs don’t work standalone. (I may be out of date for this, that’s what I saw last time I used LEGO Boost and WeDo.)
Is this related to the micropython 1.0.1 Audio API? That does appear to be quite powerful, though running user-provided waveform-generating callbacks reliably at 244 Hz seems challenging in a cooperative-multitasking environment.
If you go to https://makecode.microbit.org/beta, scroll down to the bottom of music drawer and try “play giggle”. This is still a recorded WAV file (from piping sound through serial on a microbit v2) until we have a web audio engine.
If I’m understanding it right this looks like a powerful way to control frequency and volume for effects, but it’s still controlling an underlying synthesizer with sine/sawtooth/triangle/square/noise waveform choices? If that’s the case, I think it would potentially combine very nicely with the cycle noise waveforms I had added which could add more variety in the underlying sounds.
This is awesome, @peli! It’ll make building melodies a lot easier for those of us who know musical notation. Plus, being able to store melodies in variables will give advanced coders a lot more flexibility. Bravo!
P.S. I love your GitHub avatar. It cracks me up every time I see it.
FYI, you can do that already, MakeCode has its own string-based notation. peli@'s extension converts RTTL format into the MakeCode syntax which makes it possible to use pre-existing melodies in that format, but if you’re making melodies from scratch I think the existing MakeCode format may be a bit more powerful since it’s more closely tied to the way the underlying sound engine works.
Here’s an example from a test I was working on. FWIW, it doesn’t currently work great since the different instruments quickly get out of sync, it would need a different approach and not just simple repeated loops.
const hihat = new music.Melody("@0,50,0,0 ~5 c8-240 c8 c c c c c @0,250,0,0 c")
const snare = new music.Melody("@10,75,0,0 ~5 r-240 r g5 r r r g r")
const bass = new music.Melody("@10,120,80,0 ~15 c2-120 d# f f# g f# f d#")
const guitar = new music.Melody("@100,100,160,0 ~15 c4-240 g r r c5 g r r c5 b4 g4 f# e d#-120 e-240")
Something I hadn’t realized with the built-in melody block is that you can store the melodies as strings and then play them back. @peli 's extension makes that capability much more obvious, albeit using the RTTL notation.
