I’m new to the PXT-Microbit MakeCode project and would greatly appreciate your help in understanding the compilation and linking process for a project created using blocks on the local MakeCode server.
Specifically, I’m curious about the process followed to link and compile the block-based code or the JavaScript code generated from the blocks. Does the project convert JavaScript to TypeScript and then transform the TypeScript code into .cpp files? Or does the compilation process work differently, directly converting JavaScript into the final .hex file without a TypeScript to C++ transformation?
If the process involves converting the JavaScript file into TypeScript and then into C++ (potentially using the auto-generated shim.d.ts for each block), how is this done? Additionally, which files and methods are responsible for generating the final .hex file from the C++ code?
Lastly, if the JavaScript code is indeed converted into TypeScript and then into C++, I would like to know how to extract the C++ file for the entire project. Any guidance on how to do this would be extremely helpful!
Thank you so much for your support and assistance!
basically, we first take the C++ and assembly code in a project and send it off to our cloud compilation service. this compiler produces a hexfile which is downloaded by the browser and stored in the browser’s localstorage so that makecode can work offline after it has been fetched at least once. we also download a lot of information about the hex file in this step which is used later by our linker.
next, when the user attempts to compile their blocks or python project, we first convert that code into typescript. then our browser compiler takes that typescript and compiles it down into an intermediate representation (IR). note that we do not use the standard typescript compiler to convert the typescript code into javascript first; it goes straight from typescript to the IR.
this IR is then sent to one of two emitters. if this compilation is being done to run the user code in the simulator in the browser, we send the IR to our javascript emitter which produces a string of javascript that can be run inside of the simulator iframe to simulate the program. this javascript is not human readable, it’s machine generated code that accurately mimics the behavior of our native code (e.g. it supports multithreading which javascript does not)
if this is a compilation for a binary file, the IR is instead sent to a native emitter which outputs machine code. this code is then linked with the binary hex file that was compiled by the cloud compiler to create the final hex/uf2 file that you download.
this entire process happens offline unless the project contains C++ code; in that case we have to ping the cloud compiler once to get the compiled binary for that source.
so, to answer your question, we do not convert typescript files into C++ files.
Thanks a lot for your detailed reply yesterday; it really clarified the overall MakeCode process for me!
I have a few more questions to better understand the backend:
C++ Compilation on Local Host
I’ve cloned the MakeCode Microbit repository and have it running on my local server and webpage. If I wanted to compile the C++ files directly on my PC without relying on the cloud compiler, could you share any insights on how this would be handled in MakeCode?
2.Toolchain Used in Browser Linker for Final Output
In the final step where the native emitter outputs machine code, I understand the browser linker links it with the binary hex file to produce the Hex/UF2 file. Could you provide details on the specific toolchain the browser linker uses for this final linkage?
Adding JavaScript (TypeScript) to C Conversion
I’m curious if it’s feasible to add functionality to directly convert JavaScript (TypeScript) to C within the MakeCode Microbit environment. Would this be possible, or would it require substantial modification?
@Monarch_Gohil can you give me some details on what exactly you’re trying to do? i might be able to give better advice if i had a clearer picture of what your endgoal is here.
converting javascript to c is not feasible within makecode. it would require pretty substantial effort.
Thanks for your reply! Let me share a bit more about my goal here and ask a few more questions:
Local C++ Block Coding Project
I’m aiming to create a project that enables block coding directly in C++ within the MakeCode Microbit editor and to run the full project environment on my local PC. Given this setup, could you advise if there’s any specific path or setup that would allow me to locally compile C++ files within MakeCode?
Cloud Compilation Process for C++
In your response, you mentioned that the C++ and assembly code in a project is sent to a cloud compilation service, which then produces a hex file downloaded by the browser and stored for offline use. In this cloud compilation process for the C++ files, is there any containerization technology or specific setup involved? If so, could you clarify when and how it’s used within the project setup or during cloud compilation?
Code Flow Documentation
Lastly, is there any official documentation available to understand the code flow for the project? This would be very helpful for diving deeper into the compilation and linkage processes.
@Monarch_Gohil i don’t think makecode is a good fit for what you’re trying to do; it’s not meant to be a general C++ coding environment. our toolchain is very specific to typescript and would be difficult to extract and use for any other purpose.
if you want to get set up with compiling C++ for the microbit, check out this article which has links to sample projects for each version of the micro:bit
As part of my exploration, I have a few specific questions regarding the Yotta build system and certain files within the MakeCode repository.
C++ Compilation in Yotta Environment:
Could you please provide a detailed explanation of how the C++ compilation process works in the Yotta environment? Specifically, I would like to understand:
Which files or configuration settings are involved in this process?
How the compilation workflow is structured within the context of MakeCode.
main.cpp File Usage:
I came across the main.cpp file located at:
makecode/pxt-microbit/libs/core/built/codal/libraries/codal-microbit-v2/samples/main.cpp.
Could you clarify its purpose and how it integrates with the overall build and execution? If I intend to modify this file to accommodate custom changes for my project, what steps should I follow to ensure these changes are applied effectively?
pxt.h File:
Additionally, I noticed the pxt.h file at the path:
makecode/pxt-microbit/libs/core/pxt.h.
Could you elaborate on its role within the MakeCode framework and how it interacts with other components in the project?
I aim to gain a deeper understanding of these components to effectively incorporate modifications and develop custom functionality for my project. Any guidance, documentation references, or best practices you could share would be greatly appreciated.
Hi,
I had a similar question based on the above explanation now i am having a clear view about the compilation process, now @richard what i want to know is, when i edit the blocks on my local-host in the editor, it definitely might be taking some of the files (either ts,js or cpp) to generate the hex file,
so now as an independent project, in a docker, which files shall i include so that it can generate a hexfile for my board?
if you are trying to extract the makecode compiler for use in some other project, i would not recommend doing so. it isn’t meant for that, and it’s pretty tied up in the rest of the codebase
if you are trying to compile C++ to the micro:bit, you should not use makecode. we are a typescript compiler, not a C++ one.
thanks @richard, got your point!, i just now want to know that when i modify some blocks on the make-code editor, then does the final code gets into the any of the files? which i can include in my docker compilation process which i can use to generate hex file for another processor?
@Jake_Peralta do you mean the source code? we do package the source code inside of hex files, though the i can’t remember the exact file format at the moment. we do have a command to extract the source code from a hex file in the pxt cli (not the makecode one), but it’s kind of a pain to use. i should probably add it to the makecode cli…
out of curiosity, what processor are you compiling for? can you give me some more details on what your use case is?
@richard i am trying to build the exact same project but for NXP’s IMX-8 ULP, that’s what my end goal is to achieve here is . So as a approach,i thought to take is the front-end would work exactly the same and on clicking the download button in the back-end a docker would have the yocto setup to generate the final binary which i can flash. That’s why i asked, if any final cpp or js file is being created somewhere i can take that and generate the output file.
yeah, unfortunately we don’t compile down to c++ or js, we compile straight from the typescript to arm thumb machine code. our stack isn’t really modular enough right now to slot in a different board to our micro:bit editor.
are you aware of https://maker.makecode.com/? it’s our more experimental editor for targeting various dev boards. there are instructions for adding another board here and the repo is below:
@richard hi there, through the above conversation, I had a curiosity to know the offline compilation process. I think you mentioned above that you explained the online compilation process,
What I am curious to know is that when the download button is clicked, and all the process is taking place offline, then how exactly is the hex-file is being generated?
yes @richard i am aware of maker, but i think it would be working the exact same way and after referring the repo i think it supports the boards with UF2 flashing format only, i am using the following board :
this one’s a heterogeneous board that means it has a processor and a controller both on board. Do you have any reference or idea that the maker’s approach will work with this? I mean what i want to conclude through this is, i am going in right direction with this and will it be worth putting extra time and effort through this approach?..Also, compiling the code from direct typescript means the hexfile is being generated from the internally written code only right, like is there any way i can generate the hex file locally on my x86 linux system?
@Monarch_Gohil the process is the same, but obviously we won’t be able to hit our cloud. when we compile a project that references extensions with C++ (or contains C++ itself) we take a hash over all of the C++ files and use that as a key to cache the hexfile that is received from our cloud locally in the browser. subsequent compiles are much faster because we just fetch the hexfile from the cache and link that with the user code instead of hitting our cloud. when we ship a new version of the editor, we always cache the hexfile for the default project (i.e. a project with no extensions loaded) so that it works offline after one load even if you don’t hit download while online.
in other words, we do not have an offline way to compile C++ files. you either hit the local browser cache or the compilation fails and you get an error
@Jake_Peralta as mentioned above, you can use the MakeCode CLI to compile MakeCode projects locally. the CLI does need to be online in order to download the correct source files for whatever target you are compiling against (e.g. micro:bit, arcade, etc.) but they are cached locally after one compile. the MakeCode CLI runs on Node.js, so it works well cross-platform (including on Linux)
and you are correct, maker only supports UF2 boards. there is currently no way to flash a board with an interface chip from that editor, so sorry my suggestion was off-base there!
when i say we compile directly from typescript to machine code, what i mean is that we do not translate the code into any other language before compiling it. we have a compiler that takes in TypeScript source files and spits out compiled code for ARM processors. this differs from the usual way TypeScript is compiled since TypeScript is usually emitted as JavaScript. we never emit any JavaScript code for user projects (or C++ code for that matter).
i think that taking our compiled hex files and converting them to work with another board is going to be a very difficult process. unfortunately, our stack also isn’t really modular enough right now to add support for another board to an existing target; you’d have to create a whole new target which is not a very well documented process. if that’s a path you want to go down, i’d suggest reaching out to our team through our public email (at the bottom of this page) with more information about your organization, hardware, etc. so that we can better advise you
alright!!, and could you please let me know, @richard that what typescript files or which exact typescript we include for the hexfile generation? It will be really helpful to know this one.
Also, thanks a lot man your replies are really helpful!
