So, it’s that time of year again. Advent of Code is released, and I eagerly decided it’s a good time to learn a new programming language. This time the idea was to learn Clojure. But not being familiar with any other lisp style languages, I quickly got very stuck and was just about to give up.
But then I saw all these examples of ChatGPT going around, and I thought, “I wonder if ChatGPT could help me learn Clojure”. And it turns out, yes, it can!
Almost. Turns out, ChatGPT isn’t very good at Clojure. None of the solutions it provided worked without tweaking and debugging.
However, what I saw was so impressive, so instead of learning Clojure, I have decided to see how far ChatGPT can get with Advent of Code.
To begin with, as with most AI solutions out there, getting the prompt right is key. I did try to just copy the Advent of Code instructions into ChatGPT to see what happened, but that did not produce the right output. Instead, it took a few (sometimes a lot) of tweaks to the prompt to get a working result.
But eventually, we succeeded. ChatGPT (with a bit of help from me to combine outputs) produced a working solution for the first day of Advent of Code. So follow along with what will hopefully be 24 posts about how ChatGPT solves Advent of Code, using any means necessary. Get ready for some AI-powered fun!
ChatGPT tries to write some Clojure
I won’t repeat the original instructions here, instead you can find them here. Let’s try with a simpler version.
But that won’t do because Advent Of Code will provide me with a very long text file in the format, and this solution requires that I enter it in the above format. Also, we get the first indication that ChatGPT might not be that great at Clojure. When I tried running the code out of curiosity, I got an error.
; Execution error (ClassCastException) at java.lang.Class/cast (Class.java:3889).
; Cannot cast clojure.lang.PersistentVector to java.lang.Number
But no worries, I’ll just ask it to provide me a solution which reads and parses the input from a text file.
Sadly, it seems ChatGPT isn’t very good at Clojure. None of the examples it provided worked outright. There was always some issue. And even if I explained the error and asked ChatGPT to provide a solution, it rarely succeeded.
If not Clojure, then what?
After a while of trying to get it to produce correct Clojure, and helping me debug and explain it (which it is quite good at) I decided to just see what else it could do.
There is always a debate about which language is the “most efficient” or “concise” so let’s see what happens if we let ChatGPT choose.
awk of course. The second most popular choice for job-security (after perl). But this time it almost works. Awk throws a syntax error. For some reason, it does not like the ternary operating in the END{}. But if we just assume the last one isn’t the biggest one, we can change it to awk '{if($0!=""){s+=$0}else{m=s>m?s:m;s=0}}END{print m}' and. TADA! That works.
ChatGPT3 has (with a tiny bit of help) solved the first puzzle in the Advent of Code 2022.
Second part of the puzzle
Let’s see what the second step in the problem is. Ok, apparently we need to find the sum of the top three now. Sadly, I seem to have misplaced the prompt, but either I am getting better at writing prompts, ChatGPT is better at producing solutions, or we just got lucky but the solution it provided worked on the first try!
The code it created was
awk '
BEGIN {
sum=0
num_groups=0
top_groups[3]=0
}
{
if ($0 != "") {
sum += $0
} else {
if (sum > top_groups[1]) {
top_groups[3] = top_groups[2]
top_groups[2] = top_groups[1]
top_groups[1] = sum
} else if (sum > top_groups[2]) {
top_groups[3] = top_groups[2]
top_groups[2] = sum
} else if (sum > top_groups[3]) {
top_groups[3] = sum
}
sum = 0
num_groups++
}
}
END {
if (num_groups < 3) {
print "Not enough groups"
} else {
print top_groups[1] + top_groups[2] + top_groups[3]
}
}
' INPUT
The future?
Tomorrow we will try Day 2, or rather I have already tried and ChatGPT3 did succeed with the first part, I just haven’t had time to do the second yet. But already the first part took a lot of tweaking the prompt, and it highlighted one of the core issues with these Lange Language Models (LLM).
Tune in tomorrow to find out what the issue was and how it did.