Vibe coding is a slot machine. I agree.

You pull the lever and out comes code. Most of the time it’s close but not right, so you pull again.

Meanwhile you watch the funny little loading messages they added to give a bit of extra dopamine kick.

But it’s the most useful slot machine I’ve ever seen.

Why I’m not worried about my job part.

Evidence 17: Google forcibly rolling Gemini everywhere before anyone has had a chance to actually consider the security implications.

www.youtube.com/watch

From the risky biz newsletter

#blaugust

This is related to my previous post on LLM’s. Feel free to skip it if had too much LLM.

There have been several studies on how LLM’s seem to make us dumber. And there is probably truth in that. It’s quite logical if we look at the biology. The brain requires more energy when it’s working, and probably even more when forming new memories.

So if there is an easier way, it will prefer that way, as for most of humanity’s existence wasted energy was not good for survival. And biologically, like your muscles will become smaller unless you exercise them, the brain probably does the same.

So I think “brain exercise” will become more important as LLMs become more useful, just like intentional exercise like going to the gym is an important part of our lifestyle because of powered transport like cars.

I don’t know what will it look like. But I’m quite sure it won’t be like the “brain training” applications. They claim to make you smarter, but based on what I’ve understood they are a bit too limited. From what I’ve read, you become better at doing their brain games but it does not translate well effectively to becoming better other things.

So what kind of brain gym should you do? I guess whatever you want as long as it’s closely related to what you want to train and requires thinking, but intentionally leave the LLM out of it.

Although you might actually want the LLM before and after, because an important part of becoming better at something is that the difficulty is right, so it challenges you. Creating customised challenges, and evaluating them afterwards is probably something a LLM is good at.

blaugust

This is another blaugust post, very little editing so it’s marked as draft.

I’m a happy user of micro.blog since forever, but I must say when I saw the Ghost 6.0 release I was tempted to try, so shiny.

Then I saw the system requirements and feature list and remembered that it’s not what I need or want.

(Also a micro blog post is still a blog post)

Word of warning: This is mostly a rant or reflection, I’m not sure there is anything useful here so feel free to skip this one :-)

The problem

If you tried solving complex problems with any of the state of the art models, you have probably noticed how the LLM has a tendency work fine up until a point, and then they break down completely. And after that, they don’t seem to be able to recover.

Even if you tell them something is wrong, unless you give it the solution it will just state it’s typical “You’re absolutely right! I see the issue now” but then it usually breaks things even more. And there is no going back, or at least I have not ever gotten it back. The only thing is to back up enough in the discussion and fork it, or reset the context if you’re coding.

It makes sense because of how a LLM works, but it’s still limiting and wastes a lot of time because it’s hard to know where it took the wrong turn.

I’ve found this happens much more when dealing with more obscure things, my most recent example is logcheck, which is old but not very popular, and claude code got stuck multiple times when I was making my logcheck simulator.

This is an interesting effect, and the Illusion of Thinking paper discusses this also.

Note that this model also achieves near-perfect accuracy when solving the Tower of Hanoi with (N=5), which requires 31 moves, while it fails to solve the River Crossing puzzle when (N=3), which has a solution of 11 moves. This likely suggests that examples of River Crossing with N>2 are scarce on the web, meaning LRMs may not have frequently encountered or memorized such instances during training.

Possible solutions

One solution was mentioned above, reset the context and try to prompt it another way. Unless the problem is actually too complex, it might work.

If the problem is “too complex” you can try a bigger model if you can. Otherwise you have to figure it out yourself so you can break it down for the model.

Another, if the problem is medium-complex is to just let it spend more cycles on it. To allow it to do that, you probably need to do something where it can iterate on the problem. For me, when I was having issues with getting the regex converted between POSIX and javascript, I told it to create a js-test script that it could run with node. That allowed it to take me out of the loop, and it ran for a few minutes trying to brute force the problem until it happened to come up with a working solution.

Do you care?

I don’t consider myself an expert on LLM supported coding but I’ve played around with it enough to have gotten some experience. And one of the takeaways for me is that I would never use it for anything actually important.

Not because I don’t think it could do it, it probably could in many cases. But I’ve noticed that, for some reason, because the LLM produces so much so fast I very quickly become disconnected from the solution. Previously when I was writing code myself (which I’ll admit was quite rare nowadays) I cared about it being correct.

Maybe this is a luxury problem, looking at the state of software, many or most developers nowadays don’t care.

Which brings me back to the, if it’s important, and you care about it, don’t outsource it to a stochastic parrot.

logcheck, is a really old collection of bash scripts that are surprisingly great for monitoring a *nix server.

It’s great because it’s really lightweight and easy to set up compared to most modern logging and alerting stacks.

It can do this because it works in reverse to how most logging tools work. Instead of trying to find the important stuff and alert on that, it just filters out everything “standard” and alert on everything else.

On a normal, low activity server like my personal one, the standard logs (excluding noisy stuff like web logs) are generally very uniform and boring. And the maintainers and contributors of logcheck have spent quite some time to pre-write filters for all the standard noise which applications put into their logs as part of daily functioning.

I’d recommend everyone who runs their own servers to give it a try. The only annoying part is writing the ignore rules for the stuff that is not yet ignored, but I’m vibe coding a solution for that, for another blog post.

I will now explain how to install on openwrt, which is interesting and useful if you can’t dnf or apt-get it, if you can, do that instead. I’ll use the Turris Omni as example because I have one, but instructions should work for all openwrt and similar. You just need bash, msmtp (or similar) and some cron.

Turris

I have a Turris Omnia, which is a nice router running a variant of openwrt. A long time ago the USB dongle I had in it started throwing lots of errors and I was none the wiser until I happened to login to it by coincidence and saw the errors in the log.

Nov 11 09:40:04 turris kernel: [525532.234506] BTRFS error (device sda): bdev /dev/sda errs: wr 709, rd 1629, flush 0, corrupt 0, gen 0

So wanting get some kind of generic alerting, and having good previous experience with logcheck I thought I would try to get it installed. But logcheck is not in the normal openwrt package repository, so I had to go look until I found a blog post by Glen Pitt-Pladdy, who had made a simple logcheck in bash, which works on OpenWRT back in 2011. Here is what I did.

Installing and configuring

Start with downloading mini_logcheck.sh

If you have SMTP, great, if not then you’ll need to create one. I prefer mailgun.com but there are a lot of providers that have free low or restricted tiers.

Modify /etc/msmtp so it looks something like this. Replace the host with your smtp host.

# Example for a system wide configuration file

# A system wide configuration file is optional.
# If it exists, it usually defines a default account.
# This allows msmtp to be used like /usr/sbin/sendmail.
account default

# The SMTP smarthost.
host smtp.eu.mailgun.org
tls on
tls_trust_file /etc/ssl/cert.pem
port 587
from turris@YOUR-DOMAIN
auth on
user <YOUR-SMTP-LOGIN>
password <YOUR-SMTP-PASSWORD>

# Construct envelope-from addresses of the form "user@oursite.example".
#auto_from on
#maildomain oursite.example

# Use TLS.
#tls on
#tls_trust_file /etc/ssl/certs/ca-certificates.crt

# Syslog logging with facility LOG_MAIL instead of the default LOG_USER.
syslog LOG_MAIL

Create

mkdir /etc/logcheck.d.ignore/

edit a file inside that to create a rule, historically, to organise rule files they are split per process/daemon, but unless you have lots of rules I prefer to keep them in one like so

here is a sample to get your started

/etc/logcheck.d.ignore/rules:

odhcpd[[0-9]+]: DHCPV6
cron[[0-9]+]: \(root\) CMD
kresd[[0-9]+\]: $
kresd[[0-9]+\]: > hints
kresd[[0-9]+\]: \[result\] => true$
99-dhcp_host_domain_ng.py

Then edit/create

/etc/cron.d/logcheck with the following

MAILTO=""
36     *       *       *       *       root     /root/mini_logcheck > /dev/null

Testing rules in logcheck

The hardest thing in my opinion is writing correct rules, the regex grep uses is basic and I generally have to use quite a bit of trial and error to get more complex rules to match.

mini_logcheck does not support the test mode that the normal logcheck does, so I quickly hacked together a test_logcheck script based on mini_logcheck. It can be found here https://gist.github.com/gnyman/a4d7ad7c13113dd9c3fa74442e42c17c

It will test the rules and display any matching lines. So you can modify your rules and re-run the script to see if it matches.

Blaugust

This is another blaugust post. It’s a draft I had lying around but it has not received enough editing or spell checking to graduate from that, so it’s a #draft still.

Many years ago, Mikko Hyppönen posted a thread on twitter[xcancel.com] on machine readable codes like QR codes.

It was interesting and I went and made this one. I dare you to scan it. If you haven’t figured out what it is, try singing it. You can find the music for it here.

Either way. A while later while reading the chapter on machine-readable codes in If it’s secure it’s vulnerable by the same Mikko, I went down the machine-readable code rabbit hole again.

First, QR codes has an encoding called ALPHANUMERIC. That allows 4296 characters from a limited character set.

So I was curious, would the whole chapter on codes in the book fit into a QR code.

The answer is no, the chapter is ~4700 characters. ~400 too many. Also <alphanumeric>ALPHANUMERIC IS/NT VERY READABLE. NO NEWLINES AND ONLY UPPERCASE AND $%*+-./: ALLOWED</alphanumeric>

But wait

What about compression?

Yes! Even DEFLATE can do it, and there is a BASE54 encoding specifically for ALPHANUMERIC QR codes.

Now the whole chapter fits in ~3500 chars (or 3400 with bzip2).

And actually… the BASE54 is unnecessary. We can store binary directly in QR. A whopping 23648 bytes (~23 KiB) if we use the lowest error correction.

So I wonder if we could compress the whole book into one code?

Spoiler: The answer is no, and if you’re an expert on QR codes you know why and was already writing me an angry email to correct me. It’s actually not 23648 bytes, it’s bits. So a binary QR code can fit around ~3 KiB and the text content of the book compresses to 111 KiB, so it will not fit. But if I knew that I wouldn’t have continued down the rabbit hole so let’s just continue seeing how much compressed text we can fit into a QR code.

The plaintext of “If it’s secure it’s vulnerable” compresses to roughly 111 KiB of bzip2, which is ~̶𝟻̶𝚡̶ too much. (To what I believed a QR code could store).

How about more modern ones? Let’s try zstd, and brotli. No… actually turned out bigger! 123 Kib and 129 KiB respectively. Is there anything else out there?

Turns out yes, there is at least two long long running competitions for compressing pure text as much as possible with little regard for speed or resource usage.

mattmahoney.net/dc/text.html and https://prize.hutter1.net

So let’s try the second best one from mattmahoney’s competition, cmix.

Ok, wow… that was slow, it took 10 minutes (vs <1s for bzip2 –best). But it got us down to 88 KiB!

That’s nice but not enough. We’d need ~30 QR codes (with 2.9 KiB per QR). Which is actually not that bad. A whole book in 30 QR codes.

Thinking outside the box

So if we leave the limitation of QR codes aside and look for any machine readable code format that we can print and then later scan back into data.

Then we will have no problem getting one book onto a A4.

Martin Monperrus had a great overview at monperrus.net/martin/store-data-paper (the link is dead but I linked to the archive).

We could use OPTAR which can apparently store ~200 KiB of data per a4 page, so the whole compressed “If it’s smart, It’s vulnerable” (~120 to 88 KiB) would fit just fine on one page.

or JABCode which I’m not sure, it seems 4,6 KiB per “symbol” (square) but you can have more than one symbol. AB Code seems interesting. Seems it was developed by Fraunhofer Institute for Secure Information Technology and is nowadays a ISO Standard 23634:2022

If you want to read the details without paying, the BSI doesn’t paywall their standards .

Blaugust note

This is day four of blaugust. Again, while this is mostly based on a old twitter thread of mine from 2022, little to no spell-checking has been done so marking as #draft

(Let’s skip the discussion about if LLM’s are a net positive or negative. Let’s just look at what is happening.)

LLM’s are increasingly being used to write code, lots of code. And according to a recent veracode paper they have a tendency to write insecure code.

The tl.dr. of that paper is that (just like humans), unless told and trained to write secure code they won’t.

No surprise there really, as a lot of example code out there is insecure. The old adage of bad data in, bad data out still holds.

This is probably bad news for the society, but good news for anyone working in incident response and security overall.

And even if they could be made not to write obviously insecure code, they write a lot of code, and humans are lazy and won’t spend the time to understand it. As long as it works it will be shipped.

So we’re going to se a lot of new vulnerable code being pushed to production in the coming years. I think we’re going to see a whole methodology popping up around finding what kind of mistakes are systemic for the AI’s and then abusing them.

Isn’t there anything we can do about it? Not really. After all we have been trying to get humans to write secure code since the start. The incentives are just not there.

This is one of those things where, as much as we techies would want, it’s not a technical problem it’s a political one.

Blaugust note

This post is the third one from me for the Blaugust festival. It was conceived and written in less than an hour and not edited or proofed, so I’ll mark it as a #draft

Why did I join Blaugfest to encourage myself to write more? I’m not sure.

Sometimes I think it’s become of some subconscious hope it in the hopes that my post would go viral, but why? I assume it’s some built in social drive to be “popular” because that was once important for survival.

But I’ve seen many examples of being popular on the internet is not a good thing. I would like to specifically thank Marcus Hutchins aka. Malwaretech for regularly sharing examples of the downsides.

Which means even if I on some level want it, I don’t on another level. Ugh, internal conflicts are the second worst kind of conflicts.

This definitely something I will need to explore during the month. And luckily, or unluckily if you were here for the previous somewhat serious tech blogging, I will need to write more on this.

I’ll just end with a old screenshot from something Marcus posted a long time ago, with a good example of how having a “hit post” will not be roses and sunshine.

Hi there! Yeah, you! My one trusty reader (honestly I don’t have any stats but I don’t think I have that many readers). Sorry if I surprised you by popping up in your feed again, I bet you assumed this was another dead blog.

But no, it is apparently Blaugust.

What? Yeah I know, it sounds a bit like the sound the dog makes when it ate too much too fast. But it’s this great thing where people join and commit to blogging (more) during August. I’d like to thank Juha-Matti “Juhis” for introducing this to me, he also wrote the great explanation linked above. If you want to learn more check it out.

Either way, because as I’m a late starter and I have not read any other posts than his, I’ll assume he knows that he is right when he says its customary to start with an introductory post.

Who am I?

My name is Gabriel. I’m not sure what I am but I work in infosec and have done so for many years. I’m old enough to have first gotten on the internet when you had to pay per minute, but I’ve been called an old man shouting at clouds since I was young.

Blaugust and I

My goal with Blaugust is just to write more, because practice makes perfect better. It’s something I’d like to do, but I have a tendency to put too high expectations on myself and end up doing nothing. This website’s subtitle even says “don’t let perfect be the enemy of bad” because I need to remind myself that practice makes perfect.

Things I hope to write about, in no particular order

• information security
• books which were important enough that I remember them
• rants about the state of technology
• technology tips and guidance
• life philosophy

And probably some other things, this list was made up half an hour before the deadline of the first post

If any of these seem interesting, then you can follow this years attempts using RSS, on micro.blog, by following me on mastodon or the old-fashioned way by typing in this address every day in the hopes that there will be a fresh new post waiting for you.