Lost main.ts code, is there a way to recover?

I was working on an experiment in JavaScript, and when I loaded the project today I got greeted by an empty Blocks screen, and the main.ts file is now empty :frowning:

This looks a bit similar to previous issues such as https://github.com/microsoft/pxt-arcade/issues/1855 and https://github.com/microsoft/pxt-arcade/issues/702, is there any current known bug that could trigger this? I’m using the live version, and it’s possible that I had never opened the Blocks view while working on it previously until I closed the editor.

Separately, is there any way to recover the lost source code? I was poking around a bit in Chrome Dev Tools in Application / Storage, and I found the IndexedDB storage (_pouch_pxt-arcade / by-sequence (go to last page), but that also has the empty main.ts.

I still have UF2 files, and https://github.com/microsoft/uf2#embedding-sources mentions that the file format supports embedding sources - does Arcade do that, and if yes is there a tool to extract it?

Note to self: make sure to export or save as image regularly, I was trusting the editor to keep my work-in-progress safe…

1 Like

Edit: See UnsignedArduino’s response, simply drag&drop the UF2 file into arcade.makecode.com, same as loading a saved image. I’m keeping the unnecessarily complicated method below for curiosity’s sake.

The UF2 file does indeed contain a copy of the source :slight_smile:

There’s likely a cleaner way to do that, but here’s how I got the data out using command line tools.

First confirm that the “source” magic number is present - this part isn’t part of recovering, but I wanted to check if it has source at all:

$ fgrep -ba $'\x41\x14\x0E\x2F\xB8\x2F\xA2\xBB' FILE.uf2
495620:[binary stuff]{"compression":"LZMA"}

Use the uf2conv tool to get a binary file: https://github.com/microsoft/uf2/blob/master/utils/uf2conv.md

$./uf2conv.py ~/FILE.uf2 --convert --output FILE.bin

Load the file into a binary-clean editor, search for “LZMA”, delete everything up to and including the final } of the JSON header, and save to FILE_SOURCE.lzma. In my case, the first bytes of the compressed data was ]^@^@ or 5d 00 00 80 00 in hex.

Then use the lzma and jq tools to get the source back:

$ lzma -d < FILE_SOURCE.lzma > source.json
$ jq '.["main.ts"]' < source.json > recovered-main.ts

(The source actually contains an extra JSON header, but the extraction worked anyway.)

It should be possible to wrap this in a recovery tool, but I wanted to save the steps in case this helps someone else, or future me if I get bitten by such a bug again…

2 Likes

You can load UF2 files just like images.

1 Like

Ah, that would have been a lot easier :slight_smile: I guess I learned something along the way though.

1 Like

We had a bug like that before where it would switch to blocks without decompiling in some cases and delete main.ts, but to my knowledge that was fixed ~ half a year ago and I haven’t seen it since then; any more details you have would be super helpful if you have them in case there’s a sequence that regressed.

in case it’s helpful, one of the old ways to repro it if I recall correctly was to go from main.ts to another file (pxt.json or a locked file in an extension), then click the blocks toggle at the top; it wouldn’t notice there were edits made in ts and would just load up whatever was in the block file at that point.

And yes, dragging a uf2 onto the homescreen or using the import → import file button on the homescreen should both work :slight_smile:

1 Like

I don’t remember the exact sequence, but I think it was something like this:

  • create a new project
  • switch to JavaScript
  • add a second source file with “+” in explorer, use that for an added class
  • switch back between main.js and the second source js a few times. Don’t open Blocks mode.
  • close the editor
  • (later, after a computer restart) open the editor

At this point, the editor opened into an empty Blocks view.

Interesting, thanks! I’ll play around and see if I can find a way to repro, this is a very bad bug if it’s happening again!

1 Like

Thanks for investigating! I didn’t end up losing anything since I had the UF2 file due to testing on hardware, but this could be nasty for someone who’s only working with the emulator.

This has happened to me before - and it’s one way to circumvent the code too large for blocks error!

You can always ‘skip and go back’ if you put a syntax error into the ts and go to blocks (it will pop a dialog with “Discard and go to blocks or stay in javascript?”) but if it goes directly to blocks and throws stuff away without an option… that’s bad.

I think this exact thing just happened to me while playing around in the Turtle extension;

Went from main.blocks to main.ts, then (for some unknown reason)investigated pxt.json and some of the files in the extension. Then when switching back to block view, mostly everything was gone…

Looked like I was back to the file as it was when I loaded it into the editor an hour or so ago, all the last hour’s progress lost, and could not find a way to get it back.

Another useful reminder to always save/backup often, I guess… :wink:

PS: Thanks by the way, to @kwx, for all the useful information about the uf2conv tool above! I’ve been thinking about how to do this for some while now without luck, until I saw your post here, which I believe was the answer I was looking for. “Ingenting er så ille at det ikke er godt for noe…”, as we say here in Norway.

1 Like

Once I was coding a huuuuge game. sometimes later, I opened it, but it never loaded.
How can you fix this?