Skip navigation

Category Archives: Golang

If you’ve got 👀 on this blog (directly, or via syndication) you’d have to have been living under a rock to not know about the libwebp supply chain disaster. An unfortunate casualty of inept programming just happened to be any app in the Electron ecosystem that doesn’t undergo bleeding-edge updates.

Former cow-orker Tom Sellers (one of the best humans in cyber) did a great service to the macOS user community with tips on how to stay safe on macOS. His find + strings + grep combo was superbly helpful and I hope many macOS users did the command line dance to see how negligent their app providers were/are.

But, you still have to know what versions are OK and which ones are not to do that dance. And, having had yet-another immune system invasion (thankfully, not COVID, again) on top of still working through long COVID (#protip: you may be over the pandemic, but I guarantee it’s not done with you/us for a while) which re-sapped mobility energy, I put my sedentary time to less woesome use by hacking together a small, Golang macOS CLI to help ferret out bad Electron-based apps you may have installed.

I named it positron, since that’s kind of the opposite of Electron, and I was pretty creativity-challenged today.

It does virtually the same thing as Tom’s strings and grep does, just in a single, lightweight, universal, signed macOS binary.

When I ran it after the final build, all my Electron-based apps were 🔴. After deleting some, and updating others, this is my current status:

$ find /Applications -type f -name "*Electron Framework*" -exec ./positron "{}" \;
/Applications/Signal.app: Chrome/114.0.5735.289 Electron/25.8.4 🟢
/Applications/Keybase.app: Chrome/87.0.4280.141 Electron/11.5.0 🔴
/Applications/Raindrop.io.app: Chrome/102.0.5005.167 Electron/19.0.17 🔴
/Applications/1Password.app: Chrome/114.0.5735.289 Electron/25.8.1 🟢
/Applications/Replit.app: Chrome/116.0.5845.188 Electron/26.2.1 🟢
/Applications/lghub.app: Chrome/104.0.5112.65 Electron/20.0.0 🔴

It’s still on you to do the find (cooler folks run fd) since I’m not about to write a program that’ll rummage across your SSDs or disc drives, but it does all the MachO inspection internally, and then also does the SemVer comparison to let you know which apps still suck at keeping you safe.

FWIW, the Keybase folks did accept a PR for the libwebp thing, but darned if I will spend any time building it (I don’t run it anymore, anyway, so I should just delete it).

The aforementioned signed, universal, macOS binary is in the GitLab releases.

Stay safe out there!

I’ve been (slowly) making my way through FOSDEM `23 presentations and caught up to Peter Lowe‘s “Bizarre and Unusual Uses of DNS • Rule 53: If you can think of it, someone’s done it in the DNS” talk. DNS oddities are items I collect whenever I see them, and while I knew about a good number of the ones in Peter’s presentation, the ones where DNS is used to retrieve your external IP address were oddly missing from my collection.

His presentation mentioned both a Google DNS hack and OpenDNS DNS hack, and I learned of a similar DNS hack from Akamai from John Payne. I keep saying “hack” because these folks are most certainly abusing the original intentions and design of DNS. “Hack” is not being used pejoratively, as this is a pretty cool and efficient way of discovering your external IP address vs setting up a full-blown HTTP TLS session, making a GET request and retrieving the payload.

I’ve been Down and Out on COVID Street for the past few weeks (#4 brought it home from high school, making multiple years of being overly cautious and careful outside the house quite moot), and had a bit of a level drain relapse over the weekend, so I decided to get my mind directed away from malicious spike proteins and build a client for the existing services and then a server anyone could run to host the same type of service.

I’ve been nerding out on Rust for the past few years, but chose Go (also calling it “Golang” in this parenthetical for the sake of SEO) since I really wanted a small binary, and DNS ops are part of Go’s “batteries included” libraries (and, I’ve worked with them before).

dig-ging The Hacks

Shaft Silhouette with Can You Dig It below

You don’t need a special client for these hacks. dig can do all the hard work for you, and it is (for the most part) on every modern system (or easily installed).

Here are three shell executable statements that will return your external IP address into a shell variable (just remove the VAR= and outer $() to see the result vs store it):

MY_OPENDNS_IP="$(dig myip.opendns.com @resolver1.opendns.com +short)"

MY_GOOGLE_IP="$(dig o-o.myaddr.1.google.com @ns1.google.com TXT +short | tr -d '"')"

MY_AKAMAI_IP="$(dig +short TXT whoami.ds.akahelp.net @$(dig +short +answer NS akamai.com | head -1) | grep ns | sed -e 's/[^0-9\.\:]//g')"

The Akamai one is a bit longer since I didn’t want to lock it in to a pre-specified Akamai resolver (you never know when orgs are going to change things). So, it looks up the nameserver first, then does the IP check.

Remove the pipes to see the “raw” output.

[Client] Hacks In Go

Go's mascot in a hacker hoodie

I’ll eventually set up a GitHub Action to build out binaries for various platforms (and setup a Homebrew tap for it) but you can get started using the nascent Go CLI via:

go install -ldflags "-s -w" github.com/hrbrmstr/extip@latest

the extra flags are there to make the binary size smaller than it otherwise would be (Go and Rust both make larger binaries than I care for, but they do that for good reasons).

At present, there are no command line options, so when you run extip, the executable makes the DNS calls to all three services and will return just your IP address if they all agree (if you’re being service intercepted in a really nasty way, that might not be the case). If any fail, the discrepancies are shown.

Serving Up Your Own Hack

Another reason to use Go is that building a DNS server in it is super straightforward, thanks to Miek Gieben‘s battle tested DNS library.

Now, thanks to this tiny, hacky DNS server I whipped up, you can:

go install -ldflags "-s -w" github.com/hrbrmstr/extip-svr@latest

and run it anywhere you’d like to have the same type of service.

It supports A, AAAA and TXT queries, though I’d use the TXT one if I were you, since you don’t need to know what type of network you’re on or interface the request is coming from. I’ve got it running on one of my random internet nodes, so you can try it out before running it:

dig myip.is TXT @ip.rudis.net

(any FQDN ending in .is can be used)

FIN

Peter’s talk was super fun and informative, so you should 100% watch it. It was great being able to have something to focus on whilst getting better, and also cool to stretch some Golang muscles.

If you have any opines on the CLI argument parser I should use, drop a comment or issue on the repos. I’ll be tweaking both the client and server quite a bit over the coming weeks.

This is a re-post from today’s newsletter. I generally avoid doing this but the content here is def more “bloggy” than “newslettery”.

You can now receive these blog posts in your activity stream. Just follow @hrbrmstr@rud.is and the new posts from here will slide right into your timeline.

So, you’ve committed to abandoning the bird site, joined a 🐘 instance, or three, and are now a citizen of the Fediverse. This is 👍🏽! But, what if you want to dig into this brave new universe a bit and see how it works? Or, perhaps you would like to engage with a specific set of other folks without committing to a particular BBSnode?

Running a full-on Mastodon instance means dealing with PostgreSQL, Redis, Ruby (ugh), and NodeJS. Sure, Docker is an option, but this is still a big project, and it’s more than likely that you’re not a Ruby programmer (which makes it difficult to poke at the code to bend it to your will).

What if I told you there’s a way to run your own ActivityPub (et al.) server that:

  • is built with Golang (requires libsqlite3)
  • uses SQLite for persistence
  • compiles in seconds
  • sets up in minutes
  • takes almost no system resources
  • supports custom emojis
  • allows markdown in posts (including source code block syntax highlighting)
  • features location check-ins (like Foursquare back in the old days)
  • enables filtering and censorship (for abuse prevention)
  • sports a tiny but quite useful API
  • lets folks consume your activity stream as an RSS feed

If that sounds more to your liking, let me introduce you to Honk by Ted Unangst (@tedu@honk.tedunangst.com), and walk you through my Honk (@bob@honk.hrbrmstr.de) setup.

Prepare To Honk

You can either use Mercurial and:

$ hg clone https://humungus.tedunangst.com/r/honk

or grab a tarball and expand it (do either of these things on the box you will be running Honk). Then, just:

$ cd honk
$ make

and in a few seconds you’ll have a honk server binary ready to use.

Now, you’re also going to need a “TLS terminating reverse proxy”. We’ll be using Nginx since it is nigh ubiquitous and straightforward to setup. If you’ve never set up an HTTPS Nginx instance. Nginx drops in nicely almost everywhere, and this isn’t a terrible certbot/nginx ‘splainer. The rest of this drop assumes you have an Nginx instance ready to configure for honking.

I’m hosting my actual Honk instance on an overkill home data science server (you can use a Raspberry Pi to run Honk), which is exposed to one of my internet-facing Nginx reverse proxy servers via Tailscale. Using a setup like this means you can go super-cheap ($5/mo) on a VPS. You can also 100% just run Honk on the same internet-facing box as you do Nginx, just make sure to follow the specific guidance for that setup below, and mebbe spring for a slightly bigger server. FWIW I use SSD Nodes (full-disclosure: that is an affiliate link).

Finally, you’ll need a FQDN configured that points to your reverse proxy. Mine is honk.hrbrmstr.de which has an A record pointing to my internet-facing VPS. Your ActivityPub handle will be @user@ThisFQDNyouChose, so pick one you can live with.

Make sure all three of those things are ready for the remaining steps.

Configure + Run Honk

This part is pretty straightforward. Run:

$ ./honk init

on the box you’re running Honk from. It’s going to ask you for four pieces of information:

  • username: the username you want. Again, this will be your @username@TheHonkFQDNyouChose identity, so pick one you can live with.
  • password: the password you want; pick a long passphrase from a password manager generator that you’ll keep in said password manager. Honk does not support MFA. Attackers will find you. You cannot hide. Just make it hard for them.
  • listenaddr: host + port Honk will listen on. If running Honk on the same system as Nginx make it something like 127.0.0.1:31337 so Honk itself is only accessible locally. I used my VPS’ Tailscale IP address.
  • servername: the FQDN you configured in the previous section.

If you mess up, just remove the SQLite databases Honk just made and start over.

Feel free to get all fancy with whatever system service runner you like, but I just run Honk from a custom application directory with:

$ nohup ./honk &

You leave out the nohup and &, or tail -f nohup.out, if you want to see the log as you configure Nginx in the next section.

Configure Nginx

Remember that this bit assumes you’ve installed Nginx and set it up with a certbot TLS certificate. See above for links to resources to help you do that.

Now you need to tell Nginx where to serve up your honk instance. Modify your base config to look something like this:

server {

  server_name honk.example.com;

  location / {
    proxy_pass http://127.0.0.1:31337;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme; 
  }

  listen 443 ssl; # managed by Certbot

  ssl_certificate /etc/letsencrypt/live/honk.example.com/fullchain.pem; # managed by Certbot
  ssl_certificate_key /etc/letsencrypt/live/honk.example.com/privkey.pem; # managed by Certbot
  include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
  ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {

  if ($host = honk.example.com) {
    return 301 https://$host$request_uri;
  } # managed by Certbot

  listen 80;

  server_name honk.example.com;

  return 404; # managed by Certbot

}

Replace the proxy_pass, honk.example.com, and 127.0.0.1:31337 values with your specific config.

Restart Nginx and go to honk.example.com. If you don’t see the Honk main page, check the Nginx logs and the Honk logs and give it a go again.

Go Honkin’ Crazy

The Honk docs are useful and quite fun reads.

  • The user manual should be required reading so, at the very least, you can grok the honking terminology.
  • The server manual shows you some options you can use when starting Honk, explains how to customize Honk’s (it supports some fun customizations), explains user management, and some additional care and feeding tips. Read this thoroughly.
  • The composition manual is a must-read since it shows off all the post composition features.
  • The filtering and censorship system manual will help you deal with any abuse.
  • The ActivityPub manual explains what Honk does and does not support in that protocol.
  • The API manual has all the information you need to use your Honk instance via some programming language or just curl.

Stuff To Try!

  • Follow folks on other servers! Hit me up at @hrbrmstr@mastodon.social or @bob@honk.hrbrmstr.de if you want to test following out (and get a reply).
  • Customize your site CSS! Make it yours. The manuals provide all the information you need to do this.
  • Add custom emoji and other components (again, the manuals are great).
  • Write an API wrapper package so you can use your instance programmatically (this is a good way to make an ActivityPub bot).
  • Look at the toys/ subdirectory. It has some fine example programs you can riff from (or just use):
    • autobonker.go – repeats mentioned posts
    • gettoken.go – obtains an authorization token
    • saytheday.go – posts a new honk that’s a date based look and say sequence
    • sprayandpray.go – send an activity with no error checking and hope it works
    • youvegothonks.go – polls for new messages
  • Import your toots or Twitter archive
  • Start a Honk instance for one of the communities you’re in. Honk really cannot support a large community, but small clubs can use Honk vs deal with a full-on Mastodon instance.
  • Poke around the SQLite databases Honk uses.
  • Help someone else setup a Honk instance
  • Customize the Honk codebase and show off your additions

Get Familiar With The Protocols

WebFinger (mentioned yesterday) is the on-ramp to poking at things, and I prefer playing with instances I own vs annoy folks trying to run “real” Mastodon servers. Honk lets you play without being a bad netizen.

$ webfinger acct:bob@honk.hrbrmstr.de

drops the following to the terminal:

{
  "aliases": [
    "https://honk.hrbrmstr.de/u/bob"
  ],
  "links": [
    {
      "href": "https://honk.hrbrmstr.de/u/bob",
      "rel": "self",
      "type": "application/activity+json"
    }
  ],
  "subject": "acct:bob@honk.hrbrmstr.de"
}

Visit the aliases in a private browser session (so no cookies/etc are used and you see what the world sees) or just curl it from the terminal.

Explore links:

$ curl --header "Accept: application/activity+json" https://honk.hrbrmstr.de/u/bob

drops the following to the terminal (see what happens w/o that custom Accept header, too):

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "followers": "https://honk.hrbrmstr.de/u/bob/followers",
  "following": "https://honk.hrbrmstr.de/u/bob/following",
  "icon": {
    "mediaType": "image/png",
    "type": "Image",
    "url": "https://honk.hrbrmstr.de/a?a=https%3A%2F%2Fhonk.hrbrmstr.de%2Fu%2Fbob&hex=1"
  },
  "id": "https://honk.hrbrmstr.de/u/bob",
  "inbox": "https://honk.hrbrmstr.de/u/bob/inbox",
  "name": "bob",
  "outbox": "https://honk.hrbrmstr.de/u/bob/outbox",
  "preferredUsername": "bob",
  "publicKey": {
    "id": "https://honk.hrbrmstr.de/u/bob#key",
    "owner": "https://honk.hrbrmstr.de/u/bob",
    "publicKeyPem": "CLIPPED B/C TOO BIG FOR SUBSTACK"
  },
  "summary": "BIO CLIPPED B/C TOO BIG FOR SUBSTACK",
  "tag": [
    {
      "href": "https://honk.hrbrmstr.de/o/rstats",
      "name": "#rstats",
      "type": "Hashtag"
    },
    {
      "href": "https://honk.hrbrmstr.de/o/blm",
      "name": "#blm",
      "type": "Hashtag"
    }
  ],
  "type": "Person",
  "url": "https://honk.hrbrmstr.de/u/bob"
}

Try all those endpoints and see what they drop (feel free to hit my server)

FIN

Honk is a great way to explore the various components of the Fediverse and I encourage folks to use it to get more familiar/comfortable with this tech. Drop comments if you run into any issues or have q’s (feel free to honk/toot q’s as well). ☮

After a Twitter convo about weather stations I picked up a WeatherFlow Tempest. Setup was quick, but the sensor package died within 24 hours. I was going to give up on it but I had written an R package (for the REST API & UDP broadcast interfaces) and C++ utility (for just the UDP broadcast interface), and the support staff were both friendly and competent and sent me a replacement super quick.

I’ve blathered about the R package already (on Twitter) so am not going to tag that here, but will link to a few repositories (in various languages) that receive the UDP broadcast messages and at least shove them to stdout.

The C++ one is mostly C but gets the job done (it just posts the messages to stdout). It should run everywhere but I only tested on macOS & Linux, because Windows is a terrible operating system nobody should use.

The Golang one has some structured types to consume about half of the JSON messages (I’ve only seen four in the broadcasts so far, and will add more as I see new ones). It’s only more verbose than the C++ one due to the various record type handling. This should run everywhere, though.

For kicks, I threw together a Swift one that is really just Swift-ified C and is a Frankenstein monster that likely shouldn’t be used. (I’ll be making a SwiftUI macOS/iOS/iPadOS app for the UDP broadcast messages, though, soon).

To round out my obsession I also made a Rust version which I’m just in 💙 with (not because of any skill of my own). It’s the smallest source file and is pretty elegant (100% due to Rust, and, again, not me).

All the code/projects are super small, but the Rust source is so tiny that it won’t be too intrusive to post here:

use std::net::UdpSocket;

fn main() -> std::io::Result<()> {

  let mut buf = [0; 1024]; // 1024 byte buffer is plenty
  let s = UdpSocket::bind("0.0.0.0:50222").expect(r#"{"message":"Could not bind to address/port."}"#);

  loop {

    let (n, _) = s.recv_from(&mut buf).expect(r#"{"message":"No broadcasts received."}"#);

    println!("{}", String::from_utf8(buf[..n].to_vec()).unwrap())

  }

}

FIN

If you’re interested in a low-cost weather station with great DIY programming support, I’d definitely (so far, at least) recommend the Tempest. We’ll see if it survives the forthcoming snowpocalypse.

These are the JSON messages it slings over UDP:

{"serial_number":"HB-00069665","type":"hub_status","firmware_revision":"177","uptime":728643,"rssi":-50,"timestamp":1643246011,"reset_flags":"BOR,PIN,POR","seq":72787,"radio_stats":[25,1,0,3,16637],"mqtt_stats":[10,108]}
{"serial_number":"ST-00055227","type":"rapid_wind","hub_sn":"HB-00069665","ob":[1643246013,0.00,0]}
{"serial_number":"ST-00055227","type":"rapid_wind","hub_sn":"HB-00069665","ob":[1643246015,0.00,0]}
{"serial_number":"ST-00055227","type":"device_status","hub_sn":"HB-00069665","timestamp":1643246016,"uptime":106625,"voltage":2.683,"firmware_revision":165,"rssi":-72,"hub_rssi":-66,"sensor_status":655364,"debug":0}
{"serial_number":"ST-00055227","type":"obs_st","hub_sn":"HB-00069665","obs":[[1643246016,0.00,0.00,0.00,0,3,1024.56,-12.82,47.84,0,0.00,0,0.000000,0,0,0,2.683,1]],"firmware_revision":165}

There are a plethora of amazingly useful Golang libraries, and it has been possible for quite some time to use Go libraries with Swift. The advent of the release of the new Apple Silicon/M1/arm64 architecture for macOS created the need for a new round of “fat”/”universal” binaries and libraries to bridge the gap between legacy Intel Macs and the new breed of Macs.

I didn’t see an “all-in-one-place” snippet for how to build cross-platform + fat/universal Golang static libraries and then use them in Swift to make a fat/universal macOS binary, and it is likely I’m not the only one who wished this existed, so here’s a snippet that takes the static HTML library example from Young Dynasty and shows how to prepare the static library for use in Swift, then how to build a Swift universal binary. This is a command line app and we’re building everything without the Xcode editor to keep it concise and straightforward.

The rest of the explanatory text is in the comments in the code block.

# make a space to play in
mkdir universal-static-test
cd universal-static-test

# libhtmlscraper via: https://youngdynasty.net/posts/writing-mac-apps-in-go/
# make the small HTML escaper library Golang source
cat > main.go << EOF
package main

import (
  "C"
  "html"
)

//export escape_html
func escape_html(input *C.char) *C.char {
  s := html.EscapeString(C.GoString(input))
  return C.CString(s)
}

//export unescape_html
func unescape_html(input *C.char) *C.char {
  s := html.UnescapeString(C.GoString(input))
  return C.CString(s)
}

// We need an entry point; it's ok for this to be empty
func main() {}
EOF

# build the Go library for ARM
CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 go build --buildmode=c-archive -o libhtmlescaper-arm64.a

# build the Go library for AMD
CGO_ENABLED=1 GOOS=darwin GOARCH=amd64 go build --buildmode=c-archive -o libhtmlescaper-amd64.a

# Make a universal static archive
lipo -create libhtmlescaper-arm64.a libhtmlescaper-amd64.a -o libhtmlescaper.a

# we don't need this anymore
rm libhtmlescaper-amd64.h

# this is a better name
mv libhtmlescaper-arm64.h libhtmlescaper.h

# make the objective-c bridging header so we can use the library in Swift
cat > bridge.h <<EOF
#include "libhtmlescaler.h"
EOF

# creaate a lame/super basic test swift file that uses the Go library
cat > main.swift <<EOF
print(String(cString: escape_html(strdup("<b>bold</b>"))))
EOF

# make the swift executatble for amd64
swiftc -target x86_64-apple-macos11.0 -import-objc-header bridge.h main.swift libhtmlescaper.a -o main-amd64

# make the swift executatble for arm64
swiftc -target arm64-apple-macos11.0 -import-objc-header bridge.h main.swift libhtmlescaper.a -o main-arm64 

# Make a universal binary
lipo -create main-amd64 main-arm64 -o main

# Make sure it's universal
file main
## main: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
## main (for architecture x86_64): Mach-O 64-bit executable x86_64
## main (for architecture arm64):  Mach-O 64-bit executable arm64

# try it out
./main.swift
## "<b>bold</b>"