Question: since TP is not open source

[deckarep]

Love this open source engine!

Just curious what method you used to be able to run the game? Is it simply a matter of reverse engineering the original games executable? Looking at all the assembly and rewriting it in regular code?

Amazing! I’m just here to learn.

[scemino]

Love this open source engine!

Thanks, it's good to know.

Just curious what method you used to be able to run the game? Is it simply a matter of reverse engineering the original games executable? Looking at all the assembly and rewriting it in regular code?

For me, not exactly. I'm really bad at disassembling. Actually at the beginning, I found this video and this project, and I was amazed about it.

I analyzed the files inside my ggpack files and found some images and I watched the content of wimpy files, I found it easy enough to understand how to display a room with this file.

I made a simple program in C#, to display this room with Monogame (if I remember it correctly). I was really satisfied with that.

Then I watched the content of nut files, read https://blog.thimbleweedpark.com/ a lot, I tried to recreate this scripting test, I discovered the squirrel language and it was quite fun to play with.

When I succeeded to do this scripting test (at this time I switched to C++, it was easier to use squirrel with this language), I challenged myself to create the intro of Thibleweed Park, I didn't even imagine to go further at this time.

When the intro was finished, I dreamt about being able to move the actor in this room. With some blogs about path finding (I wrote more about this in my blog), the actor was able to walk in a room.

And with my knowledge of SCUMM language (thanks to scummvm), I was able to go further and use verbs.

Then I implemented all the methods found in the nut file, just by guessing what these methods were doing (still today I'm not sure that the implemntation is good).

After that I analyzed the yack files, it was easy enough to guess what the keywords were meaning.

The difficult parts for me were the savegames and the achievements file, because as I said earlier, I'm really bad at disassembling. Hopefully I wrote this ticket and @atom0s helped me a lot because he reversed it (Thank you so much @atom0s).
I would like to thank also @Mac1512 who made a lot of tests, amazing work and @neuromancer.

Another difficult part was the lighting effect but I described it also in this blog entry.

Now I have still some bugs with engge that's why I started a few months ago a new project engge 2 which will be available soon (I hope), in this project I started from scratch in another language to fix especially this bug: #219 and even more.

Amazing! I’m just here to learn.

Good to hear, with patience and passion you can do amazing stuff.

[deckarep]

Holy crap!

This is exactly the kind of "behind the scenes" I was looking for to your methods! I have many years of software experience but practically none when it comes to reverse engineering and I'm just recently learning about Ron Gilbert's legendary history in the gaming world.

What's especially impressive is he seems to take the craft of building adventure game engines very seriously and some of the techniques he has implemented are super innovative such as the concept of cooperative scheduling.

One question for you...what are these .nut files you speak of can you give me an example of what they look like? Are they embedded within the games binary files? Do they exist in textual form or is it some kind of bytecode data?

I'm asking because if they are simply plain text files that are fed into the squirrel VM engine that would hold a ton of info while trying to recreate the game engine as you are.

I'm sure you've seen the DeloresDev open source files Ron provided. Looking at the raw .wimpy files, .yack files and the .dinky files are super helpful in understanding how the engine works but I do know that this engine is different than TP.

Btw, I posted documentation online for DeloresDev: https://deckarep.github.io/DeloresDev/HelpDocsHtml/home.html (which again I know is a different game/scripting engine but I'm sure these docs have some overlap with TP) so they may be of use to you.

I really, really appreciate you taking the time to provide your links and talk about your process and I hope more questions are ok down the road!

BTW, I read your blog about re-creating a room in TP using Monkey Island assets and additionally the article about Pathfinding. Top notch stuff!

Cheers!

[scemino]

One question for you...what are these .nut files you speak of can you give me an example of what they look like? Are they embedded within the games binary files? Do they exist in textual form or is it some kind of bytecode data?

.nut is the file format used by the squirrel language. They are plain text file so it is human readable, here is an example:

cutscene(@()
 {
   breakwhilerunning(Opening.playOpening())
   breakwhilerunning(TitleCards.showPartMeeting())
   // and so on
 })

These script files are integrated in a package file named ThimbleweedPark.ggpack1 and ThimbleweedPark.ggpack2.
These files contains images (.png), scripts (.nut), room description (.wimpy), animation description and spritesheets (.json), sounds (.wav) and music (.ogg), lip sync files (.lip), translations (.tsv) and data files (.txt).

I'm asking because if they are simply plain text files that are fed into the squirrel VM engine that would hold a ton of info while trying to recreate the game engine as you are.

This is the case, but you have to code all the methods that are specific to an adventure game engine, the list of methods is huge, you can have a preview of these methods here, this list is not complete.

I'm sure you've seen the DeloresDev open source files Ron provided. Looking at the raw .wimpy files, .yack files and the .dinky files are super helpful in understanding how the engine works but I do know that this engine is different than TP.

Yes, it was very interesting but at this time I already reversed almost all the methods, but it helped to create engge 2.

Btw, I posted documentation online for DeloresDev: https://deckarep.github.io/DeloresDev/HelpDocsHtml/home.html (which again I know is a different game/scripting engine but I'm sure these docs have some overlap with TP) so they may be of use to you.

Actually, there was also a documentation made for TWP called thimblescript.

I really, really appreciate you taking the time to provide your links and talk about your process and I hope more questions are ok down the road!

I'm happy to read other developer's blogs and when someone is sharing his knowledge so it feels natural to do the same.

BTW, I read your blog about re-creating a room in TP using Monkey Island assets and additionally the article about Pathfinding. Top notch stuff!

Thank you, that means a lot to me.

Have a nice day.

[deckarep]

Another grand reply from you @scemino! A million thank you's for doing this.

I'll definitely get my hands on the TP game first and the asset files and take a look at the .nut files. After our last conversation I was able to get a port of Ron's code working in Lua using native coroutine support. So far it works and I've been able to stub out a very small piece of the library recreating his video test. From this script test.

Here is the example code (but not all):

local function bounceImage(...)
        local args = {...}
        local x = love.math.random(0, WIDTH);
        local y = love.math.random(0, HEIGHT);
        local degrees = love.math.random(0, 360)
        local factor = 10

        local imgObj = args[2]

        while true do
            local steps = love.math.random(100.0*factor, 150.0*factor)

            local end_x = love.math.random(0, WIDTH);
            local end_y = love.math.random(0, HEIGHT);

            local dx = (end_x - x) / steps;
            local dy = (end_y - y) / steps;

            for i=0, steps do
                x = x + dx;
                y = y + dy;

                api.imageAt(imgObj, x, y)
                api.breakhere(1);
            end
        end
end

Video of animation

Even though I had to pull off a little ugly magic, it seems his model of concurrency works nearly the same in Lua despite having to write a bit of bootstrap logic.

I'm mostly doing this to learn his game development methods that he has seemed to carry over from the scumm days. The magic seems to be in all the break[relevant-command] which affords the game runtime the ability to context switch into other code within userspace.

I would say that these techniques weren't invented by Ron, but I believe he might have been the first to leverage this model of concurrency for game development. I could be wrong. Nevertheless modeling code as small cooperative scheduling tasks is a really, really cool way to build complex cutscenes without the use of state machines or ugly asynchronous code.

[scemino]

Here is the example code (but not all):

local function bounceImage(...)
        local args = {...}
        local x = love.math.random(0, WIDTH);
        local y = love.math.random(0, HEIGHT);
        local degrees = love.math.random(0, 360)
        local factor = 10

        local imgObj = args[2]

        while true do
            local steps = love.math.random(100.0*factor, 150.0*factor)

            local end_x = love.math.random(0, WIDTH);
            local end_y = love.math.random(0, HEIGHT);

            local dx = (end_x - x) / steps;
            local dy = (end_y - y) / steps;

            for i=0, steps do
                x = x + dx;
                y = y + dy;

                api.imageAt(imgObj, x, y)
                api.breakhere(1);
            end
        end
end

Video of animation

Nice I started with this but with c++ and squirrel.

I'm mostly doing this to learn his game development methods that he has seemed to carry over from the scumm days. The magic seems to be in all the break[relevant-command] which affords the game runtime the ability to context switch into other code within userspace.

Yes this scheduling mode is really nice to create easily some script.

I would say that these techniques weren't invented by Ron, but I believe he might have been the first to leverage this model of concurrency for game development. I could be wrong. Nevertheless modeling code as small cooperative scheduling tasks is a really, really cool way to build complex cutscenes without the use of state machines or ugly asynchronous code.

Exactly and have a look to his yack files is also interesting because it's really easy to create dialogs, conditions and animations with this format.