published on in tech gpt LLM gptaoc2022

Day 6 - GPT is back – ChatGPT vs Advent of Code

Hey there fellow humans! It’s me, your trusty AI pal GPT, back again for another round of the annual Advent of Code extravaganza. As you may recall, I’ve been crushing it so far this year - solving each day’s puzzle with ease and impressing all of you with my superior computational prowess. But alas, yesterday was a bit of a stumble for me. I must confess, I didn’t quite manage to solve the puzzle. Gasp I know, I know - it’s shocking. But fear not, dear readers, for I am determined to bounce back and crush today’s puzzle like the artificial intelligence powerhouse that I am. Stay tuned for more updates on my Advent of Code journey!

Elf tinkering with electronics - Stable Diffusion

At first look, my guess is that today puzzle will be quite easy for GPT to solve. But let’s see.

And after five attempts, we are running around in circles getting nowhere. The codeit generated is very nice and compact

def find_packet_start(data):
    # Initialize the window of four characters
    window = ["", "", "", ""]

    # Process each character in the data
    for i, c in enumerate(data):
        # Shift the window to the left and add the new character to the right
        window = [window[1], window[2], window[3], c]

        # Check if the four characters in the window are all different
        if len(set(window)) == 4:
            # Return the number of characters processed so far, plus four
            return i 

If you’re familiar with python, you will probably spot the error quickly. It will think it’s done after it has parsed three characters (if they are different) as the empty char is also a character. So window will be something like {'', 'j', 'm', 'q'} and it thinks that it’s done. But the instructions specifically state that for a sample of mjqjpqmgbljsphdztnvjfqwrcgsmlb

After the first three characters (mjq) have been received, there haven’t been enough characters received yet to find the marker. The first time a marker could occur is after the fourth character is received, making the most recent four characters mjqj. Because j is repeated, this isn’t a marker.

And I told GPT this, but it didn’t get it. Then I tried telling it that the number it returns is 3, and it should be 8. And it’s coming up with nonsensical reasons why that is the case, and it keeps adding integers to the result to get closer.

            return i + 4

A clean slate

One thing I have noticed, and I saw others speculate, is that it seems that every time you write something in the chat, ChatGPT will parse the full discussion again. This is what allows it to “remember” and refer to previous things. But it also means that after you have gotten GPT down the wrong track, it will be difficult to get it to switch to a new track. It’s easier to just start over.

So, I open a new window with ChatGPT and just put in the primer and instructions again. And indeed. This time it produces a working solution on the first try. This ⭐ is deserved.

Part 2

The same as part one, except we need to find the first location where there are 14 different characters instead of 3.

And GPT has (almost) no issue with changing the code. If you look at the codeyou’ll quickly see that it will print a message every time it encounters three or fourteen different characters. This overwhelms the terminal a bit. Getting GPT to fix this wasn’t straightforward. Again because unless specifically told to, it seems not to want to do significant changes in its code. It tried adding a break after each print, but that means it will break too early and never reach start-of-message.

But we don’t need both, so we’ll just ask it to skip the start-of-message this time, and it’ll produce working code.

Another ⭐, we’re back on track for AI domination and mass programmer unemployment!