@richard, music.playInstructions’s Buffer is arranged something like this: (copied from Gemini)
Offset Size Type Parameter Name Value Range
----------------------------------------------------------------------
0 2 UInt16LE Sound Waveform Waveform ID
2 2 UInt16LE Frequency Start 0 to 65,535
4 2 UInt16LE Duration Milliseconds
6 2 UInt16LE Volume Start 0 to 1023
8 2 UInt16LE Volume End 0 to 1023
10 2 UInt16LE Frequency End 0 to 65,535 Hz
But there is no place I can enter phases, which is when the waveform starts. for example, is a sine wave had a phase of 0, the vallue at 0 would be 0. Having a phases of pi/2 would make it 1. This could be helpful for additive reconstructive synthesis, and we could in theory play any audio file in makecode. The issue is, right now playInstructions has no phase value to do it. that means the audio would be up to 50% worse! Can you add it some how, or is it supported somewhere?
i don’t think adding a phase to the instruction would get you significantly closer to playing audio files (i like to think that i’m pretty familiar with this particular subject).
if i were to change the play instructions buffer format it would break a lot of stuff so i’d need a pretty strong reason to do so.
one thing i might consider doing is adding some more sine wave options to the waveform field, similar to what we do for duty values on square waves. but i don’t think that would give you the fidelity you’re looking for and it would only actually make a difference if you were rapidly switching phase or playing multiple notes simultaneously to get some phase cancellation effects… it would be pretty niche overall. unless you can demonstrate a compelling use case i’m not sure it would be worth doing.
@richard, you could probably design a playSineBuffer, which would be mostly the same thing but only sine (but with phase). I want to try to use ARS (Additive Reconstructive Synthesis) to play a normal sound file. I think this could work, since playInstructions talks to AudioContext, which is a function made in the browser. That means that AudioContext plays at whatever sample rate the device is (e.g. 44,100 or 48000 hz), rather than at 1000 hz. playInstructions I believe can only be given a waveform to play at a 1000 hz rate. The are around 48 samples (depending on the system) between each frame. By using ARS, I could play a couple sine waves every frame, and I would 100% replicate some frames. But to accurately use ARS, I need access to phase. This is why I am asking for it.
even if i surfaced something like that, just for the browser and not for hardware, the reality of javascript running in the browser means that it wouldn’t be executing at audio rate. there is an API for generating low-level audio samples, but at that point we might as well just do the work to add sample playback.
also, makecode projects aren’t meant for storing large binary files like music or video. for one thing, there isn’t enough flash space on microcontrollers to hold that and the user’s code. and our sharing system wasn’t meant to hold large files anyhow. as soon as we added support for any sort of audio playback, i guarantee you that people would start attempting to to make projects with entire songs and run into all sorts of bugs/issues.
overall, while i appreciate that people like to experiment with audio playback, i wouldn’t expect it to be something that we support or add APIs for anytime soon (or ever)