Day 9 – More than one problem – ChatGPT vs Advent of Code

Ok. I was honestly considering just skipping this. Day 9 looks quite ridiculous. But let’s give it a try.

At least one thing, GPT is sometimes good at, is taking a long description and summarising.

The first try, with the puzzle description without change got this which wasn’t a good start, a few iterations laters we weren’t making much improvement. So, time for a reset.

Also, we have another challenge today. Because ChatGPT has gone mainstream, it’s much slower than when we started. Taking up to a minute for each answer, and occasionally halting mid-answer.

This makes it very frustrating to try to iterate on code using it.

Anyway, the second attempt is the full description without any visual examples. That has better luck (and as we have learned, getting a good answer requires luck). Other than a simple mistake of naming a variable the same as a function.

def move(direction, steps):
[snip]

# Loop through the moves and apply them
for move in moves:
    direction = move[0]
    steps = int(move[1:])
    move(direction, steps)

It now works. But it gives the wrong answer. It really has issues with “understanding” that the Head can’t just teleport, and it needs to simulate each step. It needs to be told explicitly to not do that.

Another things I’m consistently seeing in my prompting, is that it wants the tail to move to the same coordinate as the tail. It does not understand that the tail drags behind, as the instructions state.

Although I was making some headway with take2, I gave up on that as it was turning into me solving the problem and telling GPT to produce the code to do it, which is not the goal of this post series.

Instead, I want to try the approach where I try to explain the problem more simply, rather than involving the elves. So, I prompt it with a simpler explanation of the problem (prompt). And I get back a solution, with most of the same problems, but it looks like it has some potential.

The problems with this is that

1. It does not execute (because it does not split the instruction, it fixes it after being told the error)
2. It does not simulate each step. Again, it fixes this after being told.
3. It does do not count the number of unique positions the tail has, which to be fair, my simple prompt didn’t ask for that.

After asking, it to return the number of unique positions the tail has visited, and to read the input from a file, we get.

This which works! Is it worth a gold star even if I had to simplify the prompt? Let’s say it is.

Part 2

Uh oh… Now the rope is 10 parts long. Let’s think a bit how to present this to GPT. First, let’s see if it can just rework it

Lovely, can you now simulate a longer rope? One which is 10 parts long. Each of the parts of the rope follows the previous part of the rope with the same logic as the tail followed the head. We are only interested in which positions the tail (part 10) has visited.

No, that does not work.

Even after a few more prompts, we don’t get anywhere. And ChatGPT is close to unusable during American daytime hours, takes a minute or two to provide an answer and often gets stuck on an answer or just provides part of it.

So, we’ll call it one star.