Coding Bits
Bits about code, or working with code.  Not so much about what's being worked on.
Subscribe via RSS here

Zerolog's API Mistake

I'll be honest, I was expecting a lot more moan-routine posts than I've written to date. Guess I've been in a positive mood. That is, until I started using Zerolog again this morning. Zerolog is a Go logging package that we use at work. It's pretty successful, and all in all a good logger. But they made a fundamental mistake in their API which trips me up from time to time: they're not consistent with their return types. When you want to create a new logger, you get a `zerolog.Logger` instance. This is also returned when you're creating a new...

Packaging Services With Systemd

This is going to be of a rambling post as I try to find my way to best configure a long-running service that's managed with Systemd. The motivation here is to install the service on a Linux machine, then configure it to run using Systemd such that: • It launches on startup • It restarts when the launch fails with a non-zero error code • It runs as a non-root user • It can be configured via environment variables loaded from an external file We'll go through each step piece-by-piece, as each one builds on the next, and a typical...

AWS Secrets Manager Cached Credentials Error

Spent around two hours trying to diagnose why one of our containers couldn't read secrets from AWS Secrets Manager. I was seeing errors of the following form when I was trying to call `GetSecretValue`: operation error Secrets Manager: GetSecretValue, get identity: get credentials: failed to refresh cached credentials, not found, SigningTurned out to be a bug in version `v1.29.2` of the secrets manager client: `github.com/aws/aws-sdk-go-v2/service/secretsmanager`. Downgrading the client version to `v1.25.4` seems to have resolved the problem. Could also be a version miss-match with `github.com/aws/aws-sdk-go-v2 v1.27.1`. Might be if I were to change the version of the AWS SDK Go...

Disabling Parallel Test Runs In Go

Go has had parallel tests enabled by default for a while now. But if you can't run your tests in parallel, what should you do? Most people probably know of the `-p 1` option to disable parallel execution of tests: go test -p 1 . But this only works if you have access to the command itself. If you've got the `go test` command in a Makefile running in a CI/CD environment that's difficult to change, this may not be available to you (it wasn't available to me when I faced this problem yesterday). A hint to an alternative approach...

PostgreSQL, pgx, sqlc and bytea

Subtitle: if you know what these four words mean, then this post is for you This is a quick one  as I'm not really in a blogging mood. But I couldn't for the life of me find a way to decode a PostgreSQL `bytea` value (which is what PostgreSQL uses for BLOB values) using pgx and sqlc so that it would match what I actually stored. The documentation of pgx and sqlc, along with various web searches, yielded nothing. I did eventually find an answer, which I'm posting here for anyone else who happens to run into this problem. That...

Obsidian PDF Styling Improvements

I've been writing a bunch of technical documents in Obsidian recently: complete with code-blocks and Mermaid diagrams. And I must say, it's been a pretty good writing experience. Certaintly much nicer than writing in Confluence. But when I made PDF exports of these documents, I found a few things that could be improved. Namely: • Moving page-breaks to the next page if doing so will avoided introducing a page-break. • Mermaid.JS images being too wide, and running off the right side of the page. Fortunately Obsidian makes it easy to fix this with CSS. And after a few tests, I...

PostgreSQL BIGSERIAL "Type"

The same person that taught me about LATERIAL SQL queries also showed me the `BIGSERIAL` type, which is an automatically incrementing integer column. I don't know why I didn't see this before. The `AUTOINCREMENT` option in MySQL was one of the things I missed when I started using PostgreSQL. I guess I just assumed that one had to explicitly create a sequence and include calls to `nextval()` when inserting things into a PostgreSQL table, and that was "just how it was done." After reading the docs on this type, it looks like this is still case here too, but PostgreSQL...

Counting In DynamoDB

Does getting a count in DynamoDB return the total across the entire table, or just the total for the current page? While a brief search online didn't give any conclusive result, it did show that it's possible to get a count for a scan or a query without fetching the items themselves, which is a good thing (honestly, I was not expecting this to be possible). This is done by setting the `select` parameter to `count`: out, err := client.Scan(&dynamodb.ScanInput{ TableName: aws.String("my-table"), ExpressionAttributeNames: expr.Names(), ExpressionAttributeValues: expr.Values(), FilterExpression: expr.Filter(), Select: types.SelectCount, }) if err != nil { return 0, err }...

Custom Import Paths In Go

One of the craziest ideas I had recently was to move all my code from Github to a self-hosted SCM system. The impetus for this was to have import paths with a custom domain name for all my Go packages, rather than have all them start with `github.com/lmika/(something)`. Fortunately, this proved to be unnecessary, as Go does allow one to customise the import path of packages hosted elsewhere. This area of the docs has all the details, but here's the process in short. Lets say you have a package hosted in Github at `github.com/example/mypkg` and you want to use the...

PostgreSQL LATERIAL Joins

Someone shared with me the `LATERIAL` join type supported by PostgreSQL. He described it as a "for each" built into SQL: “When a FROM item contains LATERAL cross-references, evaluation proceeds as follows: for each row of the FROM item providing the cross-referenced column(s), or set of rows of multiple FROM items providing the columns, the LATERAL item is evaluated using that row or row set's values of the columns. The resulting row(s) are joined as usual with the rows they were computed from. This is repeated for each row or set of rows from the column source table(s).” Here's an...

Changing gRPC Schemas

What changes can you make to a gRPC schema that's already in use? Honestly, I always forget these rules, or get paranoid when I'm touching messages that are already deployed, so it was time to find out what these rules are. I came across this Medium post by Toan Hoang which listed them, and I thought I'd make a note of them here. Here they are in brief. Non-Breaking ChangesThe following changes are completely safe, and will not break existing clients or servers using an earlier version of the schema: • Adding a new service • Adding a new method...

Go Unit Test Naming Conventions

You're working on a Go project (yay! 🎉) and you need to write a unit test. You decide to go with a table-driven approach. What names should you use for your variables? For a long while, I was writing tests using names like these: func TestSomething(t *testing.T) { scenarios := []struct { description string arg string expected string }{ { description: "Thing 1", arg: "this", expected: "that" } // more cases go here } for _, scenario := range scenarios { t.Run(scenario.description, func(t *testing.T) { // ... } } }The cases are "scenarios" and they are to produce "expected" values....

Moan-routine: Stripe Prices

I love coding and anything computers. I've spent, and continue to spend, a significant amount of my life writing code. And on the whole, it's been a magical experience. But not always. Sometimes I encounter something that makes me wonder why? Why was that designed that way? Why doesn't it work? Why couldn't this be easier? You encounter something that blocks you or puzzles you, maybe even questions how anything in computers can work at all. You've got things to do, and you try your best to work around the problem. Sometimes you succeed. Most times you get blocked and...

Rubber-ducking: On Context

I'm torn between extracting auth credentials in the handler from a Go Context and passing them as arguments to service methods, or just passing the context and having the service methods get it from the Context themselves. Previously, when the auth credentials just had a user ID, we were doing the former. But we're now using more information about what the user has access to and if we were to continue doing this, we'll need to pass more parameters through to the service layer. Not only does this make things a little less neater, it'll mean the next time we...