published on in gpt LLM gptaoc2022

Day 10 – The computer can't compute – ChatGPT vs Advent of Code

Ok, after a few harder ones, we’re back to something which looks like right up the alley of GPT. A simulated computer. Although it isn’t what large language models (LLM) are made to do, previous examples has shown that GPT does quite well at it.

So let’s give it a try and hope for the best. As usual, we start with the full input.

It produces something that works on the first try. Promising start. But this does not consider that the addx takes two cycles. But it seems on this “thread” it is not possible to get GPT to understand this. GPT just tries to skip cycles instead of delaying the update.

I also tried a reset with the same instructions, still no luck at getting it to properly handle the delay.

I tried multiple variations of prompts like

You need take into account that the addx takes two cycles before x is updated, there is a delay between the addx and when x is actually modified. During this delay, the execution continues as normal. After the second cycle, the value of X is updated.
You also need to take into account that there can be multiple addx functions in-flight which has been executed but the x value has yet to be updated.

It’s interesting that GPT is struggling with this because to me the solutions seems quite straightforward, just keep the pending addx in a list and keep track of when they were executed, and apply them when it’s time.

Let’s see if we can just tell GPT this.

No, with trying to add

As a hint, as there is a delay between addx and the value being applied, we probably need to track pending addx and at which cycle they were executed, and after each cycle we check if there is any pending addx updates we need to apply.

To the instructions. For some reason, the solutions are even worse. Let’s give the simplify-solution a try.

Simply no-go

After a few iterations, and then a few iterations to hash out coding errors, we end up with this, which looks intriguing. But alas, it produces the wrong result, and I’m out of patience with a barely responding ChatGPT.