# Custom audio clips, possibly using queuePlayInstructions
**URL:** https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168
**Category:** Arcade
**Created:** [April 10, 2024, 12:19am UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168 "2024-04-10T00:19:11Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@chembot](https://forum.makecode.com/u/chembot)
#### Post date: [April 10, 2024, 12:19am UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/1 "2024-04-10T00:19:11Z")
Trying to play custom audio in makecode arcade. So far I’ve had luck layering waves using the method in this thread: [Sound effect demo (no extensions)](https://forum.makecode.com/t/sound-effect-demo-no-extensions/5833) to make simple sounds, and I’ve seen piano sounds made from scratch here: [Various Instruments Recreated in Makecode Arcade](https://forum.makecode.com/t/various-instruments-recreated-in-makecode-arcade/24040), but I want to go further.
I tried using similar syntax to `melody.addNote` and calling `queuePlayInstructions` to play several sine waves (generated using a Fourier analysis tool, and a few others) but the results are a little shaky. I’m sure it can be refined further but it seems like a very roundabout way of getting things done. [Voice Test Project](https://arcade.makecode.com/S29802-07562-50646-68084)
In another project I tried setting the frequency every 0.25ms, matching a 8khz wav file I put into makecode, but anything less than 1ms broke. I started looking through the pxt git and found `queuePlayInstructions`, so I’m thinking maybe `U.delay(when)` is the culprit. Found `playInstructionAsync` as well, but I couldn’t find any kind of definition for `OscillatorNode`, which seems like my best bet, since playing audio seems to be done with `osc.frequency.setValueAtTime` and `gain.gain.setValueAtTime`.
Not sure if this is even going to be possible or if sine wave layering is the limit for now, and wanted to ask about it before going any further. Almost all of my information is from testing or inference, so please correct me if I’m wrong. Thanks!
---
### Author:  [@KIKIvsIT](https://forum.makecode.com/u/KIKIvsIT)
#### Post date: [April 10, 2024, 12:19am UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/2 "2024-04-10T00:19:29Z")
Woah!
---
### Author:  [@Brohann](https://forum.makecode.com/u/Brohann)
#### Post date: [April 10, 2024, 4:03pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/3 "2024-04-10T16:03:54Z")
Wow, just wow.
If i recall even @richard himself said it wasn’t possible
please, please, please, keep working on this and share your findings and methods. This is a huge breakthrough!
---
### Author:  [@Sarge](https://forum.makecode.com/u/Sarge)
#### Post date: [April 10, 2024, 4:06pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/4 "2024-04-10T16:06:08Z")
This is huge 🤯
---
### Author:  [@danger\_kitty](https://forum.makecode.com/u/danger_kitty)
#### Post date: [April 10, 2024, 4:18pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/5 "2024-04-10T16:18:39Z")
thats really cool
---
### Author:  [@richard](https://forum.makecode.com/u/richard)
#### Post date: [April 10, 2024, 4:26pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/6 "2024-04-10T16:26:56Z")
hahaha anything’s possible! I just meant we didn’t have support for it built in
---
### Author:  [@richard](https://forum.makecode.com/u/richard)
#### Post date: [April 10, 2024, 5:00pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/7 "2024-04-10T17:00:52Z")
@chembot anything less than 1ms is going to be running up against limitations of the browser itself. iirc that’s the range where browsers start messing with things to avoid fingerprinting via high precision timestamps. I know firefox limits precision to 2ms
that being said, i’ve been playing around with adding some new experimental audio features that might help you out with this; specifically, wavetable synthesis which could probably be expanded to playing PCM buffers.
below is a demo link, but this is made with a beta version of the editor so please be sure to open it in an incognito window to avoid messing with your projects. also, this might not ship anytime soon so don’t do too much with it! very much subject to change
> **Open in incognito window**
>
> [demo](https://arcade.makecode.com/app/49ed0b302c31cf28a6d43e34d287f6f9e7587f88-902128047a?ws=memory#pub:59500-47780-41674-91494)
---
### Author:  [@Vegz78](https://forum.makecode.com/u/Vegz78)
#### Post date: [April 10, 2024, 11:10pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/8 "2024-04-10T23:10:04Z")
Well, “hello”, there, @chembot! -This was mighty impressive!!!
---
### Author:  [@randomuser](https://forum.makecode.com/u/randomuser)
#### Post date: [April 11, 2024, 4:41pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/9 "2024-04-11T16:41:58Z")
GASP!?
I’m flabbergasted. Have you HEARD my [‘Hello’?](https://forum.makecode.com/t/a-new-way-to-create-music-in-makecode/17563/4) I mean it was literally just a whistle. This is _stunning_, simply. You have no idea how long I’ve researched using Fourier analysis for this!!
Please, you must tell me. What tool did you use!? I’ve been using PRAAT, but its data output rarely works.
Your research of queuePlayInstructions and melody.addNote are exactly what I was missing- [my research](https://forum.makecode.com/t/small-update-on-the-mp3-speech-in-makecode-idea/23243) stopped because I just don’t know as much about JS.
(By the way, have you seen my work on this? Did I inspire you at all?)
This is AMAZING!! And @richard , wavetable synthesis is exactly what I’ve been asking for… my dreams are coming true here!! : D
---
### Author:  [@chembot](https://forum.makecode.com/u/chembot)
#### Post date: [April 12, 2024, 9:24pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/10 "2024-04-12T21:24:36Z")
Thanks for all the kind words, I wasn’t expecting this kind of response 😅
@richard that demo is interesting… Looking forwards to the eventual release! I also didn’t think about browser limits, I had assumed it was makecode. Thanks for the info.
@randomuser I glad to find out someone else was working on the same thing as me! I used a few programs to get this result:
First processed the recordings to .wav files with some online tool, so I could open them in [SPEAR](https://www.klingbeil.com/spear/). I found I got decent results with a frequency resolution of around 30-90, and a minimum amplitude threshold of around -40dB. I also deleted anything above ~4000Hz, and any very small lines.
Exported it to a .txt, and then processed that with a C# console app I threw together. If you want to try something similar the data has a 5 line header, then each line starts with `time`, `number of waves`, then repeating wave data in groups of three: `channel`, `pitch`, `volume`. Not sure if volume is amplitude or actual volume in dB though.
The console app spits out a `number[][][][]` array, where
`number[n]` represents a sine wave that changes frequency and amplitude over time,
`number[n][0]=[[start tick, end tick]]` (SPEAR gave me 10ms/tick) and
`number[n][1]` is an array of `[frequency, amplitude]` pairs.
Kept the start and end tick an extra array deep so the type stayed `number[][][][]`.
I’ll post the code for the converter github tonight once I clean it up a little bit. I’m tempted to change it from an array to buffers, since 3 lines per ms of audio is a bit much, but that can wait.
I’m glad to know there’s so many people pushing the bounds of makecode! So many interesting ideas to try
---
### Author:  [@randomuser](https://forum.makecode.com/u/randomuser)
#### Post date: [April 12, 2024, 9:29pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/11 "2024-04-12T21:29:30Z")
Wow, yeah, that is complicated! I kept up with you until the C#, but I think I get it- you ran the audio through SPEAR which gave you the individual sine waves, and you used the C# to sort it out / get it into a workable format. I’m excited for that converter!!
This is SOO cool!! : D
---
### Author:  [@chembot](https://forum.makecode.com/u/chembot)
#### Post date: [April 13, 2024, 4:43pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/12 "2024-04-13T16:43:05Z")
Sorry, I realize now I was a bit vague with how SPEAR works. It outputs a bunch of sine waves that change in frequency and amplitude over time, which start and end at different times. It’s a bit more advanced than normal Fourier analysis, and works better than overlaying unchanging sine waves, although I have no idea how it actually works. But it is why the “Hello” is so audible despite only like 30 waves being overlayed.
---
### Author:  [@randomuser](https://forum.makecode.com/u/randomuser)
#### Post date: [April 14, 2024, 1:30am UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/13 "2024-04-14T01:30:08Z")
Wow, only 30! That’s much more optimized- I think the first time I tried to synthesize speech it was asking for hundreds!
---
### Author:  [@chembot](https://forum.makecode.com/u/chembot)
#### Post date: [April 14, 2024, 7:37pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/14 "2024-04-14T19:37:44Z")
In the test program going past around 30 waves sounded worse and worse. At a min length of 12 with 30 waves its mostly audible, but at a min length of 8 with 38 waves it starts to get muddy. Even going as far as min 25 and playing only 12 waves is still somewhat passable.
I’m not sure if it’s the data from spear being interpreted wrong, higher frequencies being played too loudly, or just a browser limitation, I’m going to do some testing and see what I can find out why more waves sounds worse.
I put the [SPEAR to MakeCode](https://github.com/ch3mbot/SPEARtoMakeCodeConverter) program on git. It’s commented a little but feel free to message if anything is unclear!
---
### Author:  [@Brohann](https://forum.makecode.com/u/Brohann)
#### Post date: [August 31, 2024, 6:51pm UTC](https://forum.makecode.com/t/custom-audio-clips-possibly-using-queueplayinstructions/28168/15 "2024-08-31T18:51:47Z")
Fun!
> **[mario](https://arcade.makecode.com/06657-18841-58253-04104)**
>
> Made with ❤️ in Microsoft MakeCode Arcade.