I honestly don’t know how to write this article. I’m not one to rain on someone’s parade, and if you can make money out of this, good on you. But sometimes you come across something so messed up that you just can’t let it go. This is one of them.

And it fits nicely into my self-hosted, screw-the-ecosystem theme that I’ve been writing about lately.

A few days ago, I was at the gym and came across this in the bathroom. Yup, it’s exactly what it looks like. You scan the QR code with your phone, open the web app, enter your contact information, watch an ad, and then you are rewarded, wait for it, with a free certified flushable wipe!

You really can’t make this shit up.

But rest assured, according to the pamphlet, the wipes were independently tested and are even said to comply with the proposed WIPPES Act, which Congress apparently has time to work on instead of, say, strengthening election security.

So how exactly does one get from iPhone to butt wipe?

The Tech Behind the QR code

The pack of squirrels in my brain couldn’t resist doing a little forensic engineering here. So let’s walk through what happens when you scan the QR code. Your phone opens a page hosted by AVA, a Saudi self-service and vending platform:

https://ava.com.sa/app/store/flushly05/BZ20260419flushly05001

That URL identifies the specific Flushly machine and location. Your phone then connects to the vending system, checks that the dispenser is online, and enables the “Get It Free Now” button.

Before dispensing the wipe, the page loads a Typeform survey and asks the user: “Please complete this short survey before using the machine.”

For you code geeks, the site runs on an older jQuery/Bootstrap vending interface and uses Paho MQTT over WebSockets to talk to the dispenser, the same MQTT protocol I wrote about in my home automation article. The newer Typeform code handles the survey and checkout flow. That part looks AI-assisted: more comments, lots of don’t-run-this-twice guards, and a noticeably different style from the older code around it.

So in the end, the QR code is doing a lot more than replacing a pushbutton. It turns a free wipe into a digital transaction involving the machine, the location, the survey, the selected product, and the user’s contact information.

Pay-to-Poo Redux

The whole pay-to-poo shtick is nothing new. Why, the Romans were charging for public toilets nearly 2,000 years ago. And while I’m not sure how many of my readers are old enough to remember, we even had pay toilets when I was a youngin’.

And they might have gotten away with it too, if it weren’t for those pesky kids.

But here we are in 2026, and someone has finally figured out how to turn the whole thing into an advertising platform. I wonder if you need to agree to an EULA or verify your age?

To be fair, this thing is just dispensing a baby wipe, not actual toilet paper. At least for now. But it feels like a slippery slope, in more ways than one. If this is where restroom technology is headed, I may just have to take my own advice and self-host my own toilet paper.

If you are like me and want more control of your digital life, you’ll probably like some of the other stuff I write, whether the subject is electronics, software, baking, perfume, or motorcycles.

Who knows what I will write about next, but the underlying questions are usually the same: How was this made? Why was it built this way? And who gets to control it?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

Yes, you can ask Substack to export your stuff. What you get is a ZIP file with text of every article as HTML and metadata such as titles, dates, status, subscriber data, and delivery/open statistics. The images themselves are not included, only the links to where Substack hosts them.

Bottom line, it’s not standalone, it’s not really portable to other platforms, and you’re still dependent on Substack. Why should it be? They have zero incentive to give you an easy exit plan.

Now don’t get me wrong, I have no beef with Substack, I coexist with their platform just fine. So far they have been fairly neutral, well, in comparison to the rest of the social media platforms.

But that could change. Even if it didn’t, I still want possession and control of my content.

What would be ideal

In a perfect world, the export would be complete and self-contained, with no dependency on external sites. It would be easy to edit, compatible with source control like git, and not be a bear to move somewhere else.

That pretty much leaves Markdown. I am not in love with it. It’s the Mediocrates of document formats, but it works, and just about everybody supports it.

So I Asks AI...

I really didn’t want to spend a lot of time on this. I got other stuff to work on, but I realized I had invested a bit of time in my Substack articles and was starting to feel like a schmuck for not keeping a local copy.

So keeping with all the cool kiddies, I decided to let AI do the heavy work this time. I explained in excruciating detail to GPT what I wanted, and after a few hours of push and pull, what I got was a nice command-line tool to convert the export ZIP into something more useful. It didn’t do a bad job really.

The article Command-Line Tool

I present you article, a Linux command-line tool written in C++ for building and maintaining a self-hosted publishing archive. It can import a Substack export ZIP or a locally written Markdown file, download the external images, convert the articles into a local Markdown archive, and build a complete static HTML website that you can serve with nginx or just about any other web server.

To be specific, the Markdown files and images are the real archive. You can delete the web files and rebuild it whenever you want.

The article command-line tool is written in C++23. I build it with clang++, although there’s nothing particularly Clang-specific about the code, so it should build with a modern g++ as well.

I’m not going to duplicate the build instructions here. The README covers the required libraries, build commands, installation, and basic usage.

Getting Your Export from Substack

From your Substack Dashboard, go to Settings, then scroll all the way down until you get to Import / Export.

Under Export your data, select New export. Substack will prepare the export and email you when it is ready. Go back to Import / Export and download it.

You will get a ZIP file with a generated name something like:

-HhoYdyNS2ig6uvqsff4Kg.zip

That’s your Substack export. I usually rename it to something a little more useful, such as:

substack-export-2026-08-10.zip

Don’t unzip it. article reads the Substack export ZIP directly.

Importing Your Substack Archive

Once you have the ZIP file, put it in your publication directory and run:

article import substack substack-export-2026-08-10.zip

article reads posts.csv, finds the exported HTML for each post, converts the article body to Markdown, downloads the images, and stores everything in the local archive.

Then check the archive:

article verify

And build the static site:

article build --clean

The generated HTML website ends up under:

archive/site/

The unpacked and converted Markdown files and the images can be found here:

archive/content/
archive/assets/

Those are the Markdown files and local images. The website is just generated output and can be deleted and rebuilt whenever you want.

Templates and Configuration

Before you build the site, there are four files you might want to customize:

• templates/article.html controls the layout of individual article pages.

• templates/index.html controls the front page.

• static/article.css controls the styling.

• article.conf configures the publication and tells the article tool where the archive, templates, and static files are located.

If you change any of the templates or the CSS, just rebuild:

article build --clean

No need to re-import anything. The Markdown archive stays exactly as it is.

Self-Hosting Your Content

As I described in Taking Back the Infrastructure, I run my self-hosted services on a headless server running Ubuntu LTS, mostly in Docker containers. I did the same thing here.

My GitHub repository includes a sample docker-compose.yml file that starts an nginx web server and serves the generated site:

services:
  web:
    image: nginx:1.27-alpine
    container_name: article-web
    restart: unless-stopped
    ports:
      - "9010:80"
    volumes:
      - ./archive/site:/usr/share/nginx/html:ro
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro

Feel free to change the port 9010 if you want to use a different one, then bring the container up. At that point you should be able to point a browser at the server and see the site index.

The generated site is mounted read-only. If anything goes wrong, you can delete the container and bring it right back up.

If you want to make the site available outside your LAN, you can set up a Cloudflare Tunnel by following the instructions I wrote in Self-Hosting Without the Exposure.

Taking Back One More Piece

Don’t get me wrong, I’m not quitting Substack. It doesn’t do a bad job of publishing and distributing what I write and they haven’t burned me, yet. But at least I have some leverage now, and I don’t have to trust it with the only copy of my work anymore.

I now have a local copy of the article I write and the images and links, in a format I can read, back up, move, and rebuild somewhere else if I need to.

So, one dependency on the tech bros gone, and one more piece back under my control.

If you are like me, and feel that you want more control of your digital life, then you probably will like some of the other stuff I write. Whether the subject is electronics, software, baking, perfume, or motorcycles.

Who knows what I will write about next, but the underlying questions are usually the same: How was this made? Why was it built this way? And who gets to control it?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

If you are like me, you rather be making stuff than fixing stuff. But it’s unavoidable. Stuff’s gonna break, and you got to fix it. Out here on the ranch, the things seem to always break at the worst time. I’d like to share a few tips that work for me to get things running pronto so I can get back to doing what I like. Some of them might sound like Captain Obvious, but trust me, they weren’t obvious to me at the time.

Data Backups

Well duh. Everybody knows they’re supposed to back up their data. But what does that actually look like for an embedded irrigation system or a weather station?

Start with the stuff you can’t just reinstall: the files you customized during setup, your configuration, and whatever operational history the system has collected.

The code that runs my irrigation system, pIoTServer, can be easily rebuilt and installed on another Raspberry Pi. But the configuration file farm.props.json is specific to my installation. And the data recorded in the piotserver.db SQLite database has been collected for years.

My weather station is no different. The configuration is easy enough to recreate, but weather_history_v2.sqlite3 is my historical weather database, and it holds years of measurements I wouldn’t be able to recreate.

Backing Up the Irrigation History

Since this runs on Linux, I already have everything I need: ssh and rsync. I don’t need any backup package or special software. Just copy the database to another machine. That is about as KISS as it gets. The tools are already there, they work, they’re free, and I know exactly what they are doing. If I need to recover the database, I copy it back.

rsync is perfect here because it only moves what changed. After the first full copy, later backups are usually small and quick. That matters when the database gets backed up on a regular schedule. It has been around forever, it works, and there is not much to screw up.

A Consistent Snapshot

My first instinct was to copy the SQLite database while pIoTServer was running. That might work most of the time, but if the copy catches SQLite in the middle of a transaction, I could end up with an incomplete or corrupt backup.

Instead, let SQLite create the backup, then write the snapshot to a RAM disk (/dev/shm) to avoid an unnecessary write to the Raspberry Pi’s SD card.

The backup process goes like this:

• Have SQLite create a clean snapshot on the irrigation server:

  ssh -i ~/.ssh/backup-key user@node \
    "sqlite3 \"\$HOME/piotserver/piotserver.db\" \
    \".backup '/dev/shm/piotserver-backup.db'\""

• Copy the snapshot to the backup server with rsync:

  rsync -a -e "ssh -i ~/.ssh/backup-key" \
    user@node:/dev/shm/piotserver-backup.db \
    latest.db

• Check the copied database with SQLite’s PRAGMA integrity_check:

  sqlite3 latest.db "PRAGMA integrity_check;"

• Delete the temporary snapshot from the irrigation server:

  ssh -i ~/.ssh/backup-key user@node \
    "rm -f /dev/shm/piotserver-backup.db"

The real script also adds error checking, timestamped filenames, rotation, and cleanup.

A Few Words About SSH

You’ll notice the examples above use a dedicated backup-key. Since these backups run unattended every night, nobody is sitting there waiting to type a password. SSH takes care of that with public-key authentication.

I created a key pair just for backups. The private key stays on the backup server, and the public key goes in the irrigation server’s authorized_keys file.

Once that is set up, the backup server can run commands with ssh and move files with rsync. And we avoid exposing the password in a script.

I have a separate backup SSH key, rather than using my normal login key. That way, if I ever need to kill it and make another one, I can do that without touching my login access. That key is still a login credential, so keep the private key protected; don’t copy it around unnecessarily.

Before I move on, I want to mention that SSH also allows you to limit the key to the backup server’s IP address, disable interactive logins and forwarding, or even restrict it to only the commands the backup needs.

Cold Spares

In a perfect world, all you’d ever have to restore is a data file. But sometimes the hardware itself is what dies. When that happens, the quickest way back up may be to swap in a preconfigured clone instead of rebuilding the whole system from scratch.

After doing this a couple of times, I found it was best to make that clone just before I deploy the real system. Everything is already configured, tested, and still fresh in my mind. Making a second copy only takes a few minutes.

Six months later, after Lord knows how many other projects I’ve worked on, recreating that environment from memory becomes an exercise in archaeology. At least it does around my shop.

Single-Purpose Controllers

What about single-purpose systems like my chicken coop controller? I keep a fully configured spare, or two, on the shelf. Ready to go with the operating system, latest software and firmware, and any daughter cards it would use. I label it with the firmware version and the date, then seal it in an anti-static bag.

That way, if the controller fails, I won’t have to run around the shops scrounging parts or sit in front of a scope trying to repair or build a replacement with my farm system dead. All I have to do is grab a bag off the shelf and replace the dead unit. At worst, if the firmware changed, just flash the latest version before replacing it.

More Complex Systems

The same idea works for more complex systems like the irrigation controller. I still build complete, working, tested spares, but more things can go wrong. These boards have operating systems, applications, configuration files, databases, and network settings that all have to be preserved.

Sometimes swapping the whole unit is the fastest way back up. Other times it makes more sense to replace just the Raspberry Pi or the irrigation valve board. I built them as daughter cards for exactly this reason, so I can leave the wiring and sensors in place and only swap the part that died.

Another design choice that saved my ass more than once is the addition of a connector for a serial console adapter.

If you look at the irrigation server or the chicken coop board you’ll see a 6-pin connector. I designed it so I can plug in a CH340C/USB-C card and talk to the board from a laptop. After standing in the chicken coop with a laptop once, I’m thinking of switching it to a Bluetooth serial link so I can stay outside.

The one time the Pi’s Qualcomm Wi-Fi chip failed, I hooked up to the serial console and figured out what happened. It worked, but standing in the field troubleshooting in the heat, cold, or rain is not my idea of a good time. I’d rather swap the failed hardware, get the system running again, and dig into the old unit later in the shop

The Network Identity Problem

One more thing. When you swap a board that talks to the network, the MAC address almost always changes. If the IP is not locked down, the new board can come up at a different address. Then every service, proxy, or tunnel that was pointing at the old IP stops working until you track it down and fix it.

That’s why these machines need fixed addresses. They’re not laptops that move around. An irrigation controller, MQTT broker, weather station, or home automation server should always come up at the same IP.

Most routers use DHCP and hand out addresses based on the MAC. A DHCP reservation can keep the IP the same, but it’s still tied to the old MAC. When you swap in a spare board, you’ll have to update the reservation too.

Things get messier once a web front end is involved. farm-web sits behind Docker, nginx, reverse proxies, and a Cloudflare tunnel. That’s why I just set a static address on the board instead of trying to be a smart ass.

Home Automation and Zigbee

Now that the farm systems are mostly stable, I’ve started paying more attention to home automation.

Home automation brings its own backup problems. The server can be perfectly healthy, but all the Zigbee devices still depend on a coordinator to talk to the server. If that coordinator dies, the lights, switches, plugs, and sensors pretty much disappear with it.

Zigbee coordinators generally come in two flavors: a USB dongle plugged into the server, or a standalone box that talks over Ethernet.

I use the Ethernet kind, an SMLIGHT SLZB-06P7U. I like it because the radio and the computer can be physically separate.

If the Pi running the server dies, I can drop in a cloned spare and leave the coordinator alone. As long as the new Pi has the right configuration and the right Zigbee2MQTT database, everything comes back.

The other nice thing is that I can put the server somewhere safe and convenient, and put the coordinator wherever the radio coverage is best.

If I have to replace the SLZB-06P7U coordinator, I run into the same network identity problem I talked about earlier. The replacement has a different MAC address, so I also give the coordinator a static IP.

What did we learn?

Most of this I learned the hard way.

• Things break at the worst possible time, and usually not one at a time.

• Back up the stuff you can’t recreate. I can reinstall code. Reinstalling years of data, not so much

• Keep a cold spare around. Swapping in a board that already works beats rebuilding one while everything is dead.

• Give the important network devices fixed addresses. he last thing I need during a failure is to fight with the router over IP addresses.

• Build in a serial console. When the network is dead, I still want a way into the machine. Ask me how I know.

• Do the work while everything is still running. Alligators, baby! Alligators!

If my point about the alligators hit home, you’ll probably enjoy much of what I write about, whether the subject is electronics, software, baking, perfume, or motorcycles. The subject changes without warning, but the questions never do: How does it work? Why was it built that way? And can I build, fix, or control it myself?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand

My father was an electrician, and he taught me the trade early. That sparked an interest in electronics that never really went away. We were often involved in remodeling buildings, and I'd scrounge through the discarded electrical parts looking for something I could use.

I was mesmerized by all the cool stuff coming out of the space program and would pore over every copy of magazines like Popular Electronics that I could get my hands on. Whatever money I made working with Dad usually ended up at the local Radio Shack, buying parts for my next project. I wasn't interested in owning gadgets. I wanted to know how they worked.

Naturally, some of those projects got me into trouble now and then. I remember learning at an early age that you could bounce a laser off a window and recover conversations from the reflected beam. To an electronics geek in the 1970s, that was pure magic.

Interestingly enough, I never got into ham radio until much later. It required studying theory and passing an exam. As a teenager, I wasn't very interested in following someone else's curriculum. If something caught my imagination, though, I'd happily disappear down the rabbit hole until I understood it. I'd rather be building something, or occasionally blowing it up.

High School and Computers

My first real encounter with computers came in junior high school. My math teacher happened to be in charge of the computer room, and he let us fool around on the school's IBM 1130. He even taught us a little FORTRAN. Everything was done with punch cards, and it all felt pretty formal. The machine fascinated me, but it was also a black box. You could write programs and use it, but the hardware itself was off limits. It belonged to IBM. It wasn't long before I figured out how to play games on the console, though.

Some of the upperclassmen had moved on to MIT and invited us to come hang around. Around the same time, MIT was running something called the High School Studies Program (HSSP), a program run entirely by students. Once I got on campus, I quickly discovered the hacker culture at the MIT AI Lab.

The first thing that caught my eye was a DEC PDP-6, already an old machine by then, running Spacewar! This was long before any of us knew what a video game was.

Then I discovered the terminals. You could walk up to almost any open terminal, log in, and suddenly have access to some of the most advanced computing resources in the world. Unlike the IBM 1130, these weren't machines you were expected to leave alone. You were expected to poke around, read the source code, figure out how things worked, and make them better. If you fixed something, great. If you built something cool, even better.

There was even a room lined with racks of AI Memos, Working Papers, and Technical Reports. If someone was working on something interesting, chances were there was a paper sitting there for the taking. You could grab a copy, head to a terminal, and spend the afternoon learning about ideas that were years ahead of anything being taught in school.

One of my favorites was AI Memo 239, better known as HAKMEM. It wasn’t a polished research paper. It was just a collection of programming tricks, math puzzles, hardware hacks, and other odd ideas that people at the AI Lab thought were worth writing down.

Possibly one of the coolest things I came across was the Knight TV system. Think Macintosh, only a decade earlier. In a world of Teletypes and character-based CRTs, this was something else entirely. Tom Knight built it at the MIT AI Lab in the early 1970s. It gave people real bit-mapped displays, 576 by 454 pixels, driven by frame buffers on a PDP-11 hard-wired into the PDP-10. You could put a pixel anywhere you wanted. That may not sound like much today, but back then it completely changed how you thought about interacting with a computer.

What really fascinated me wasn’t just what the system could do. It was the racks of electronics and the wire-wrapped boards. Ordinary people had built this machine because they wanted the computer to do something new. Looking at it, I couldn’t help thinking, “I want to build something like this myself.”

My timing couldn’t have been better.

The Homebrew Computer

About that same time, a quiet revolution was taking shape, one that would change the world. Intel had introduced the 8008. For the first time, you could buy the brains of a computer on a single chip.

By today’s standards it was primitive. An 8-bit chip running at 500 kHz, only 16 KB of memory, maybe a twentieth of a MIPS. None of that mattered. For the first time, someone like me could actually build a computer at home.

The chip alone cost about forty bucks, a lot of money for a high school kid in 1975, but that was just the beginning. The 8008 was a finicky chip that needed a pile of support logic just to interface with memory. If you wanted it to talk to a serial terminal, assuming you were lucky enough to find a surplus Teletype, that required even more hardware. By the time you added memory, TTL chips, sockets, power supplies, and a cabinet to put it all in, building your own computer was a serious investment.

I somehow managed to get my hands on an Intel MCS-8 User’s Manual. It explained every aspect of the 8008, right down to the timing diagrams and interface logic. I read it so many times the pages became worn and frayed. In fact, I was so absorbed by it that I ended up repeating Latin in summer school. I even brought the manual with me and read it there.

I wasn't the only one thinking this way. Around that time I ran into a fellow named Carl Helmers from Peterborough, New Hampshire, who was doing the same thing. He published a little newsletter called Experimenter's Computer System (ECS). He later went on to start a magazine called BYTE. You may have heard of it.

Most people remember BYTE, but fewer remember that it was actually started by Wayne Green, the publisher of 73 Magazine. Wayne was one of the great promoters, and great characters, of the early personal computer industry. After a falling out that cost him control of BYTE, he started a competing magazine called Kilobyte. BYTE promptly trademarked the name, forcing him to rename it Kilobaud. Leave it to Wayne to turn a divorce into a magazine war.

Vector Boards, Wire-Wrapping, and The Mark-8

At first, if you wanted a computer, you designed it yourself. Intel published application notes, but turning an 8008 into a working machine still took serious electrical engineering skill.

Most builders wire-wrapped their circuits on vector boards. A few etched their own printed circuit boards. There were no standards, no motherboards, and no ready-made designs. Every machine was a one-off.

That started to change when hobby magazines began publishing complete computer designs. One that really caught everyone’s attention was Jonathan Titus’s Mark-8, featured in the July 1974 issue of Radio-Electronics.

Calling it a “kit” would be giving it far too much credit. It wasn’t anything like a Heathkit. You didn’t open a box full of carefully labeled parts and follow a step-by-step manual. You ordered a $5 construction booklet from the magazine, and if you were lucky you could buy a set of six bare printed-circuit boards for about $50 from a little company in New Jersey called Techniques.

Everything else, every chip, every resistor, every socket, every connector, every scrap of wire, you had to dig up yourself. Building a Mark-8 wasn’t assembling a kit. It was building a computer from scratch.

The MOD-8: From the Great White North

While the Mark-8 had a following, another machine soon caught my attention. Just as I was getting ready to wire-wrap my own 8008 system, I came across a newsletter from Celetron advertising something called the MOD-8.

So as soon as I could scrape together enough money, I ordered a stack of bare printed circuit boards.

The last piece of the puzzle was the C-MOD8-9, a universal memory board with a whopping 1 KB × 8 of static RAM, sockets for Intel 1702 EPROMs, and a 2 KB × 8 ROM.

The ROM held Monitor 8, a tiny monitor program that completely changed how you used the machine. Instead of flipping switches or hand-entering machine code every time you powered up, you simply connected a serial terminal, or something that looked like an ASR-33 Teletype, to the MOD-8-2 board at a blazing 110 baud and turned the power on. A prompt appeared. Suddenly you could examine and modify memory, load and save programs to cassette, punch paper tape, dump memory, and start programs with simple commands.

It sounds primitive now, but at the time it was a huge improvement. It felt like having a real computer sitting on your workbench. There was just one small problem. To use all of those Monitor 8 commands, I needed a terminal.

The Ins and Outs

That turned out to be easier said than done. The MOD-8 was designed to talk to an ASR-33 Teletype at 110 baud, but I didn't own one, and there was no way I could afford one. In the mid-1970s they were still in everyday service, and even the folks at the friendly local electronics surplus store knew exactly what they were worth.

So whenever I wanted to do any serious work, I had to schlep my project over to MIT, find an unused terminal, and quietly plug into it when nobody was looking. It worked, but it wasn't exactly a long-term solution. So I needed a keyboard, a display, and the electronics to connect the two.

Which brings me back to the local electronics surplus store, Eli Hefferon & Sons. If you were an electronics hobbyist around Boston in those days, you knew Eli. His store was a treasure hunt. You never knew what you were going to find. One week it might be military connectors, the next week DEC modules, oscilloscopes, relays, transformers, or bins full of TTL chips. Half the fun was wandering the aisles. The other half was haggling over the price.

Remember earlier when I mentioned MIT and the High School Studies Program? I somehow talked them into letting me teach a class. Actually, it wasn't really a class. It was just a gathering of homebrew computer geeks who wanted to build computers, trade ideas, and learn from each other.

Well, one day Eli had a pile of surplus Sanders terminal keyboards. They were solid full-travel mechanical units, but useless without new electronics. Someone in our club designed an encoder board that fit them perfectly. We convinced Eli into a group discount and pretty soon half of the problem was solved.

About the same time, Southwest Technical Products SWTPC released the CT-1024. It wasn't exactly a high-resolution display. It could show just 32 characters on each of 16 lines using an ordinary television set. Things were happening.

But I wanted to do something better. I'd been spoiled by the Knight TV terminals at MIT. They were true bit-mapped raster displays with a resolution of 576 by 454 pixels. At one bit per pixel, the frame buffer alone required nearly 32 KB of memory, twice the entire 16 KB address space of the Intel 8008. Not that I could have afforded that much RAM anyway.

As serendipity would have it, I came across a little newsletter called The Computer Hobbyist. In its very first issue, Hal Chamberlin described a homebrew vector graphics display for the Intel 8008. Holy smokes, Batman!

Unlike the expensive raster displays I'd seen at MIT, Hal's design used an ordinary oscilloscope, or surplus radar CRT, to draw crisp vector graphics with almost no memory at all.

I built one, then couldn’t resist improving it by adding my own refresh circuitry to free up the CPU. I actually got it running well enough to win a few science fair awards before it finally let out the magic smoke. Lots of it.

The Homebrew Revolution

Soon it wasn't just me and a few other homebrew computer nuts anymore. Every month seemed to bring another magazine article, another newsletter, or another company selling boards, kits, and parts. What started as a handful of experimenters was turning into a real movement.

Then in January 1975 MITS put the Altair 8800 on the cover of Popular Electronics. Overnight the idea of owning a personal computer went from fantasy to something thousands of people thought they could actually build.

The excitement got so big that people wanted to meet each other. Trade shows and computer conferences started popping up around the country. I made it to the ones in Atlantic City and Philly.

For the first time I was surrounded by people just like me, builders, tinkerers, and hackers who couldn't leave well enough alone. We'd argue about hardware, swap ideas, and spend half the night talking about what we were going to build next.

Before long you could buy complete kits, or even walk into a store and buy a fully assembled computer. What had started as a hobby for a handful of determined builders had become a major industry.

Building It Yourself

Looking back, the homebrew movement wasn’t just about building computers. It was ordinary people claiming the right to understand and shape technology. You didn’t have to wait for some credentialed authority to decide what a computer should do. If you had an idea, you could wire-wrap it together and make it happen.

I never really stopped doing what my father taught me. Figure out how something works. Build one yourself. Don’t be afraid to take it apart and maybe even improve it. That's how you make it your own.

Today I write software, design circuit boards, build open-source systems, and occasionally design an irrigation controller or two. The tools have changed. The curiosity hasn’t.

If you like building the things yourself, you’ll probably enjoy much of what I write about, whether the subject is electronics, software, baking, perfume, or motorcycles. The subject changes without warning, but the questions never do: How does it work? Why was it built that way? And can I build, fix, or control it myself?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

I’ve written a couple of articles lately about some of the dashboards I built for the farm. One lets me keep an eye on the irrigation system and shows me the schedules, valve activity, and what’s going on with it. Another is connected to my off-grid weather station, where I can check the current conditions, watch the trends, and dig back through the history.

These sites work great on my local network, but now and then I leave the ranch to pick up chicken feed, or maybe just have a life, and I still want to check up on things. The problem is, how do I get to my sites without inviting every smelly script-kiddie wannabe, AI bot, or Beijing-sponsored cyber rats to come knocking?

So I had to find a balance between wanting complete control over my self-hosted systems and the convenience of reaching them from anywhere, without hanging my home network out on the open Internet.

What I discovered is that it doesn't have to be all or nothing. I can still own the hardware, control the software, and keep the data on my own servers. The key is putting Cloudflare in front as the secure front gate to everything behind it.

What’s a Cloudflare?

You’ve probably used Cloudflare hundreds of times without realizing it. Ever click on a website and get that little “Verifying you are human...” screen before the page appears?

Am I the only one who finds it ironic that a machine is asking a human that question?

That’s Cloudflare at work. It’s one of the more visible things they do, but it barely scratches the surface of what the company actually does.

Cloudflare traces its roots back to an anti-spam project. In 2009, the people behind it realized the bigger challenge wasn’t just fighting spam. It was making the Internet itself faster and more secure. The story goes that because they were building a “firewall in the cloud,” the name Cloudflare just seemed to fit. Since then, they’ve grown into one of the largest Internet infrastructure companies in the world. A huge amount of all Internet traffic now passes through their network every day.

Cloudflare offers everything from DNS and content delivery to DDoS protection, web application firewalls, Zero Trust security, tunnels, serverless computing, and cloud storage.

Despite all that, Cloudflare doesn’t host my applications. My servers stay in my house. I use Cloudflare to create a secure tunnel between my users and those servers, making the connection faster, more reliable, and a lot harder to attack.

So What's the Catch?

Yeah so how much is this going to cost me?

Well, maybe nothing. But we both know there’s no such thing as a free lunch.

The free tier is part of Cloudflare’s business model. They give a lot away to regular users and hobbyists, and most people never pay anything. When some of those users grow and need higher limits, better tools, advanced features, or real support, that’s when Cloudflare makes its money.

It’s simple economics. Cloudflare already operates one of the largest networks on the Internet. Adding another small website or home lab to that network costs them almost nothing, but it introduces another potential long-term customer.

For self-hosters, the free plan is usually more than enough.

The Old Way

Before Cloudflare Tunnel came along, exposing a home server to the Internet involved a lot of moving parts and was often hit or miss. I still have the scars from doing it for years, going all the way back to the old WildBlue satellite days.

• First, you needed a public IP address. If your ISP changed it from time to time, as many do, you also needed a Dynamic DNS service so people could still find your server.

• Next came port forwarding. You’d log into your router and tell it that incoming traffic on ports like 80 (HTTP) or 443 (HTTPS) should be forwarded to a specific machine on your local network.

• If you used HTTPS, you had to deal with X.509 certificates. Even today with services like Let’s Encrypt, you still have to obtain, install, and renew them.

• Then there was the firewall. Opening ports to the Internet meant making sure only the traffic you wanted could get through while keeping everything else out.

And if you wanted to self-host, this is the crap you had to go through just to be reachable on the Internet. I’m sure plenty of sites still do it this way.

You owned every piece of it. DNS had to be right. The public IP had to stay current. Port forwarding had to work. Certificates had to get renewed. Firewall rules had to stay tight. It could work fine for months, then one little thing would break and you’d be screwed, looking for a fix to the problem.

Why I Started Looking at Cloudflare

Fast forward to 2026. My Internet connection is now Starlink. It’s absolutely amazing. It’s stupid easy to set up, it’s fast, and it just works.

Unfortunately, there’s one catch. Starlink uses Carrier-Grade NAT (CGNAT).

Instead of giving every customer their own public IP address, you’re sharing one with a bunch of other customers. From the rest of the Internet’s point of view, it doesn’t see your network. It only sees Starlink’s. That put me right back where I started during the old WildBlue days, just with a whole lot more bandwidth.

My first thought was, “No problem. I’ll just use Dynamic DNS.” Unfortunately, all that would do is point my domain name to Starlink’s shared public IP address. That address doesn’t belong to my router, it belongs to Starlink’s CGNAT gateway.

From there, the connection has nowhere to go. Starlink has no port-forwarding rule telling it which customer should receive the incoming connection, so the traffic just stops. Their default CGNAT policy blocks inbound ports, there’s no way to forward them to your router.

So where does that leave us? I asked around to a bunch of buddies who are way smarter than me and pretty much came to the same conclusion. Today there are really three practical ways to reach self-hosted services when you’re stuck behind CGNAT.

One option is to build a private network using something like Tailscale. It works well if you’re willing to install software on every device that needs access and limit those services to yourself or a small group of trusted users.

The other option, and I didn’t even know they offered this, is to pay Starlink for a public IP address. That usually means moving to a higher-priced Priority plan. Even then, it gets you right back to the traditional model of port forwarding, certificates, and firewall rules.

Neither of those worked for me. I wanted to keep my servers and my LAN private, while allowing selected applications to be available from anywhere, without requiring users to install any special software.

That’s what led me to the Cloudflare Tunnel.

Cloudflare Tunnel

A Cloudflare Tunnel flips the usual model the other way around.

Instead of opening ports on your router and waiting for the Internet to connect to you, your server reaches out to Cloudflare and keeps an encrypted connection open. When someone types your website address in a browser, the request goes to Cloudflare first. Cloudflare then sends it down the tunnel to your server, and the response travels back the same way.

Nothing on your home network is waiting for random connections from the Internet. There are no inbound ports to open, no port forwarding, and no direct exposure of your home network to the Internet.

This is what Cloudflare calls Zero Trust. The old model basically trusted anything that managed to reach your open ports. Zero Trust is the opposite. Nothing is trusted by default. Every request has to prove itself, and your actual servers stay hidden. With a Cloudflare Tunnel the access can be controlled at Cloudflare’s edge before anything ever reaches your network.

That’s what makes it one of Cloudflare’s coolest features. Your servers stay private, but the applications you choose to publish can still be reached from anywhere through a normal website address.

Setting up Cloudflare

There are a few hoops to jump through the first time around, but that's about it. Once everything is in place, adding another application takes just a few minutes.

Create an Account.

You’ll need a Cloudflare account. If you don’t already have one, go to https://www.cloudflare.com/plans/ and sign up. The free plan is enough for everything in this article.

Do the DNS Dance.

For the tunnel to work cleanly, Cloudflare has to be your DNS provider. When Cloudflare controls the DNS, it can route a name like myapp.yourdomain.com through their network and straight into your tunnel. If someone else is still handling the DNS, the traffic never reaches Cloudflare properly and the tunnel can’t do its job.

That doesn’t mean you have to buy the domain from them. If you like your registrar, you can keep your registrar (Porkbun, Namecheap, GoDaddy, etc.). Just point the nameservers at Cloudflare and leave the registration wherever it already is.

When you add your domain to Cloudflare, they give you two nameserver addresses that look something like foobar.ns.cloudflare.com and bozo.ns.cloudflare.com, or whatever they assign you.

You log into your registrar, find the nameserver settings for the domain, and replace whatever is currently there with the ones Cloudflare gave you. Once that change propagates, Cloudflare is in charge of the DNS and can route traffic into your tunnel.

That’s the only real requirement, and the good news is you only have to do this once. After the nameservers are updated, the rest of the setup happens inside Cloudflare.

Create a Tunnel

Once your domain is on Cloudflare and the nameservers have updated, the next step is to create a tunnel.

To me this is the hardest part: finding it in the Cloudflare website. They offer so many services that it’s easy to get lost, and it feels like they rearrange the whole thing every other week.

In the Cloudflare dashboard, look for Zero Trust. From there go to Networks → Tunnels. Or just type “Tunnels” in the dashboard search bar. That’s often the fastest way. Make sure you’re creating a Cloudflare Tunnel, not anything labeled Mesh. You want the tunnel.

You’ll land on a page that asks you to name it. The name is only for your own tracking, so call it whatever you want.

Once you name the tunnel, Cloudflare takes you to the Install and run a connector screen. They show you a bunch of options (Docker, Linux, Windows, macOS, etc.).We’re not actually going to use any of those full install commands. What we really want is the token.

The easiest way I’ve found is to select Docker. Cloudflare will show you a long docker run command that includes a --token followed by a huge string that starts with something like eyJhIjoi....

Copy that whole token (everything after --token) and save it for now. That’s the only piece we need for the Docker setup later.

After you grab the token, Cloudflare sends you to the Route Traffic screen. This is where you tell the tunnel which public name should point to which local service.

Hostname
Pick a subdomain (like myapp) and choose your domain. That becomes the public address, for example myapp.yourdomain.com.

Path
Leave it blank unless you only want a specific path.

Service
Set the type to HTTP and put in the address of your service. Because we’re running things in Docker, this is usually not localhost.

You’ll typically use one of these:

• The Docker service name (if cloudflared is on the same Docker network), for example http://myapp:8080

• The host machine’s IP address, for example http://192.168.1.50:8080

The key point is that the address has to be reachable from inside the cloudflared container.

Once you hit Complete setup, Cloudflare will create the DNS record for you. You can add more hostnames later if you need them.

Setup Your Side

At this point I’m assuming you already have your web app (or whatever service you want to expose) up and running on your local network. For the rest of this article, I'll assume it's plain HTTP.

You can point the tunnel at an HTTPS service, but on a local network it’s usually more trouble than it’s worth. You often end up flipping on “No TLS Verify” because the local certificate doesn’t match what Cloudflare expects. Cloudflare already handles HTTPS on the public side. So I generally don’t bother encrypting the short hop inside my own LAN. Pointing the tunnel at something like http://192.168.1.50:8080 or http://myapp:8080 is the simple way most of us do it.

That said, we need to run the Cloudflare Tunnel connector so it can reach that service and keep the outbound connection open to Cloudflare.

I run the connector in Docker. Same reason I run most things in Docker: it’s simple, consistent, and easy to update. Instead of installing cloudflared on the host, I just use the official image. When a new version comes out I pull it and restart the container.

One thing that tripped me up the first time was Docker networking. Remember, cloudflared is running inside its own container. When you tell Cloudflare your application is at http://127.0.0.1:8080 , it’s looking for a web server inside the cloudflared container, not on your Raspberry Pi or another container. If your application is containerized, point the tunnel at the Docker service name, for example http://farm-web:8080. If it’s running directly on the host, use the host’s LAN IP address instead.

The other advantage of using the Docker container that I can run more than one connector from the same docker-compose.yml file if I want to run multiple tunnels. Each one gets its own service name and its own token. I keep the tokens in a separate .env file so they don’t end up in the compose file itself. And if I decide to punt a tunnel, all I have to do is edit that file and restart the container.

On the machine running the connector, I keep the whole Cloudflare tunnel setup in its own directory. Inside it are just two files: a docker-compose.yml and a .env file that holds the tokens.

The docker-compose.yml looks like this:

services:
  cloudflare-your-service:
    image: cloudflare/cloudflared:latest
    container_name: cloudflare-your-service
    restart: unless-stopped
    command: tunnel run
    environment:
      - TUNNEL_TOKEN=${YOUR_TOKEN}
    tmpfs:
      - /var/log

  cloudflare-another-service:
    image: cloudflare/cloudflared:latest
    container_name: cloudflare-another-service
    restart: unless-stopped
    command: tunnel run
    environment:
      - TUNNEL_TOKEN=${ANOTHER_TOKEN}
    tmpfs:
      - /var/log

And the .env file in the same directory holds the tokens.

YOUR_TOKEN=eyJhIjoiNDg0NTE3Mj......
ANOTHER_TOKEN=eyiEyiYoTE3Mj......

Then I just run:

docker compose up -d

That's it. The container starts, authenticates with Cloudflare using the tunnel token, and establishes the encrypted tunnel automatically.

Testing That It Works

Once the container is running, the first thing I do is simply open the public URL in a browser. If the page comes up, the tunnel is probably working.

If it doesn’t, the Cloudflare Zero Trust dashboard is the next place to look. Under Tunnels, you should see your connector with a Healthy status. If it isn’t healthy, there’s no point chasing anything else until you fix that.

Cloudflare also lets you watch the tunnel in real time. You can open the log stream from the tunnel page and see requests as they come through. When something isn’t working, that’s usually where I start looking.

One last check is to try the site from somewhere outside your home network. Turn off Wi-Fi on your phone or grab another Internet connection. That confirms you’re really reaching the service through Cloudflare and not accidentally talking to it over your local network.

At this point the tunnel is up and working. Your application is reachable through Cloudflare, and your home network is still sitting behind your firewall.

There’s just one problem. Right now, anyone who knows the URL can reach your application. The tunnel hides your network, but it doesn’t decide who gets in.

That’s where Cloudflare Access comes in. We’ll put an authentication layer in front of the tunnel so only the people you choose can get through.

Authentication with Cloudflare Access

Instead of making your application handle logins, you can have Cloudflare check the user’s identity before the request is ever sent down the tunnel. If the person isn’t allowed in, the traffic never reaches your server. That keeps the attack surface smaller, and you don’t have to bolt authentication onto every individual app.

At the time of this writing, Cloudflare Access supports a bunch of login methods. Most of the convenient ones (Google, Microsoft, GitHub, and the rest) just hand the problem off to an outside provider who gets to decide whether a request should be allowed through. Which defeats the purpose of self-hosting as far as I am concerned.

There is one method I find acceptable, even if it isn’t perfect: Email one-time PIN. Cloudflare sends a short code to the user’s email address. If you're already using a privacy-focused email provider like Proton Mail, it’s a reasonable compromise. It’s still not fully self-hosted, and Cloudflare isn’t encrypting the message with your PGP key, but it’s the least bad of the convenient options.

I’ve been skeptical of these identity problems for a long time (I wrote about the difficulty of verifying authenticity of public keys years ago). That experience is part of why I’m not eager to hand the whole problem over to the tech bros at Google, Microsoft, or even Apple.

Turning on Cloudflare Access

Activating Cloudflare Access isn't too hard once you figure out where they've hidden it this week. Honestly, every time I set one up I either end up skimming the documentation or asking AI where everything moved. I swear they rearrange the menus every other week.

Last time I checked, it was under Access Controls → Applications. Create a Self-hosted application, give it a name, select the hostname you already created for the tunnel, and choose One-time PIN as the login method.

Next, create a policy that includes the email addresses you want to allow. You can enter individual addresses or allow an entire email domain. You can also decide how long a login session stays valid before users have to sign in again.

One last thing: don’t forget to click Save on the policy. This tripped me up more than once.

The Nuts an Bolts of One-Time PIN

So here’s what happens when someone hits a hostname protected by Access. Cloudflare intercepts the request before it ever goes down the tunnel. The browser is shown a login page that asks for an email address.

Once someone gets past the PIN, Cloudflare creates a signed JSON Web Token (JWT) and stores it in the browser as the CF_Authorization cookie. That same token also shows up on your server in a header called Cf-Access-Jwt-Assertion.

The JWT carries the identity info (email, when it was issued, etc.) and is signed by Cloudflare so it can’t be casually messed with. On later visits the browser just sends the cookie along and Cloudflare checks it. If it’s still valid, the request goes down the tunnel.

One thing that tripped me up: JavaScript in the browser can’t read the Cf-Access-Jwt-Assertion header. Backend code or nginx can. If your front-end needs the identity info, the cleanest approach is to create a small endpoint that returns only the specific data the page actually needs. Don’t hand the full token to the browser. It’s a reusable credential, and a hacker who gets hold of it can use it to get into your system.

Here’s a simple nginx example that extracts the JWT payload and serves it at /access-info:

map $http_cf_access_jwt_assertion $cf_access_jwt_payload {
    default "";
    ~^[^.]+\.(?<cf_access_payload_segment>[^.]+)\.[^.]+$ $cf_access_payload_segment;
}

server {
    listen 80;
    server_name _;
    root /usr/share/nginx/html;
    index index.html;

    location = /access-info {
        default_type text/plain;
        add_header Cache-Control "no-store, no-cache, must-revalidate, max-age=0" always;
        add_header Pragma "no-cache" always;
        add_header Expires "0" always;
        add_header X-Content-Type-Options "nosniff" always;

        if ($cf_access_jwt_payload = "") {
            return 204;
        }
        return 200 $cf_access_jwt_payload;
    }
}

This returns only the middle segment of the JWT (the Base64-encoded payload), not the full signed token. Your JavaScript still needs to decode it to read the identity information.

You can debug it yourself by going to yoursite/access-info and pasting the returned string into https://www.jwt.io That will decode the payload so you can see exactly what Cloudflare is sending.

What Cloudflare Doesn’t Do

A couple of parting thoughts, just so I don’t leave you with the wrong impression.

• Cloudflare Tunnel is not a VPN. It’s a secure way to publish selected applications, not your entire network, over a private overlay like Tailscale or a traditional VPN.

• It also doesn’t replace your firewall. Your home network should still be locked down. The tunnel simply eliminates the need to open inbound ports for the applications you choose to publish.

• Cloudflare won't magically fix your bad software. If an application has security vulnerabilities, they'll still be there. Cloudflare Tunnel and Access just reduce your exposure, it doesn’t fix insecure code.

• Cloudflare doesn't replace good operational discipline. You still need to keep your systems patched, make backups, and follow good security practices. It’s a layer of protection, not a substitute for the basics.

Cloudflare is a force multiplier for the little guy. It gives you access to the same kind of network and security services that used to be reserved for companies with dedicated IT departments and million-dollar budgets... assuming they still have a clue and haven't outsourced all their thinking to an LLM.

If you have a clue and like the idea of controlling your own world, you’ll probably enjoy much of what I write about, whether the subject is electronics, software, baking, perfume, or motorcycles. The subject changes without warning, but the questions never do: How does it work? Why was it built that way? And can I build, fix, or control it myself?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

Ironically, that's where the whole project started.

Like many home automation hackers of the time, I got my start with X10, which I came across in the aisles of RadioShack. X10 was pretty cool for its day, but I found its power-line signaling frustratingly unreliable.

By 2010, I had migrated much of my home to Insteon switches and dimmers. Insteon improved on those ideas with a faster dual-mesh network that combined power-line and RF communication while remaining backward compatible with many existing X10 installations.

At the center of it all was the Indigo home automation server, running on a dedicated Mac mini. It was a solid product for its time, but over the years the interface barely evolved. Eventually I found myself wondering why I was paying a subscription for software I was convinced I could build better myself.

It didn’t take me long to reverse engineer the Insteon PowerLinc protocol, even with all its quirks. Before long, my own code was running on a Mac mini, along with an iPhone app I wrote to schedule and control everything from indoor lighting to outdoor Christmas lights. The original project is still available on GitHub.

That old project became the foundation for what eventually evolved into pIoTServer.

I recently went digging through the old configuration files and was surprised by how much of the original design is still there. Devices were organized into groups, events triggered actions, schedules automated the house, and every capability was exposed through a REST API that my iPhone app used for control.

Insteon itself did not fare as well. The company’s servers abruptly went tits up in 2022, leaving customers up the creek with little warning. I heard that a group of Insteon users later acquired the company, brought its services back online, and began developing and selling products again.

Today’s pIoTServer, the same system that runs our farm irrigation, is more sophisticated and hardware-agnostic. It can run on something as small as a Raspberry Pi, but the underlying design remains remarkably similar.

Life After Insteon

Insteon demonstrated that there was a market for home automation that ordinary homeowners could install and manage themselves. It also demonstrated the risk of depending on a proprietary ecosystem.

Let’s not do that again.

Today there is no shortage of alternatives. Zigbee, Z-Wave, Wi-Fi, and the newer Matter standard are all supported by multiple manufacturers, making it much easier to avoid being locked into a single proprietary ecosystem.

Z-Wave was created by the Danish company Zensys as a proprietary technology. Zensys controlled the radio chips and protocol stack, and that control later passed to Sigma Designs and then Silicon Labs.

For most of its history, Z-Wave was tightly controlled. The chips, software stack, certification process, and trademark all ran through a fairly small commercial ecosystem. The same exact shit I was trying to avoid.

Z-Wave is less closed today, with the Z-Wave Alliance taking a larger role in maintaining the standard and opening it to additional silicon and stack suppliers.

Zigbee came from the other direction. The Zigbee Alliance, now called the Connectivity Standards Alliance, or CSA, was formed as a consortium of companies developing a shared standard. Zigbee builds its networking, security, and application layers on the separate IEEE 802.15.4 radio standard. That allowed multiple semiconductor companies to produce compatible radios and software stacks, while manufacturers built switches, plugs, sensors, and dimmers on top of them.

Zigbee has its own certification rules, working groups, and bureaucracy, but it was not nearly as bad as the Z-Wave world.

Zigbee

Each has its own strengths and weaknesses, and I could have made a case for any of them. I first tried Z-Wave, but after fussing with it for far too long, I gave Zigbee a try. It worked the first time.

It was stupid easy to add and configure a new Zigbee device. From the dashboard, you simply enable Permit Join, walk over to the device, and hold down its configuration or power button until it enters pairing mode. A few seconds later, the device appears, usually already identified ,with all of its supported features ready to use. You can keep adding as many devices as you like while Permit Join is enabled, then simply turn it off again when you’re finished.

I found a surprisingly wide range of Zigbee devices. Leviton made a basic wall switch, SONOFF made a really nice plug, and Suraielec had an outdoor plug that fit what I needed. A lot of Zigbee hardware comes from common OEM designs, so the same basic switch, plug, or power stake often appears under several different brand names with slightly different electronics, firmware, or packaging.

And then there is Inovelli, a small company from Kalamazoo, Michigan. Their devices are in a different class entirely.

The design is thoughtful, the controls feel solid, the LED bar can be configured for status and notifications, and nearly every behavior can be adjusted. But what really impressed me was the attention paid to the parts most people will never notice. The terminals, mounting hardware, and even the screws that hold the wires all feel like someone actually cared.

Inovelli is the Michelangelo of home-automation electronics.

I have written before about the art hidden inside the things we create. Inovelli is a perfect example. It is easy to make a switch that works. It is something else entirely to make one beautiful.

Under the Zigbee Hood

Zigbee devices operate in the same 2.4 GHz band used by Wi-Fi and Bluetooth, but they use far less bandwidth. With a raw data rate of only 250 kilobits per second, Zigbee is not intended to stream video or move large files. Instead, it sends small messages: turn on, set brightness, report a temperature, or announce that a button was pressed.

One thing to keep in mind is that Zigbee shares the 2.4 GHz band with Wi-Fi, so it helps to keep their channels apart. My network uses Zigbee channel 11, so I keep nearby Wi-Fi off Wi-Fi channel 1.

Every Zigbee network begins with a coordinator. It creates the network, chooses the radio channel, manages security, and allows new devices to join.

Utility-powered devices such as switches, plugs, and dimmers usually act as routers. Besides performing their own jobs, they relay messages for other devices, extending the range of the network.

Battery-powered sensors are generally end devices. They sleep most of the time, wake long enough to send or receive a small message, then go back to sleep. Because they do not relay traffic for other devices, they can run for months or even years on a battery.

The two devices shown here are Zigbee coordinators. Each contains a Zigbee radio, an antenna, and firmware that handles the lower levels of the Zigbee protocol. One is the SONOFF USB dongle; it plugs directly into the computer and is often adequate for a small home installation.

The other is an SMLIGHT SLZB-06P7U Ethernet Zigbee coordinator. It connects to your local network over Ethernet and can be powered by PoE, allowing it to be placed wherever it provides the best Zigbee coverage.

A Zigbee network can contain many routers and end devices, but only one active coordinator. In this setup, device reports and commands handled by Zigbee2MQTT ultimately pass through the coordinator, sometimes by way of one or more routers.

The coordinator handles the radio traffic, but it still speaks Zigbee. Applications need a simpler way to exchange information about devices. That is where MQTT enters the picture.

MQTT

MQTT is a lightweight messaging protocol designed for communication between machines. It was created in 1999 by Andy Stanford-Clark of IBM and Arlen Nipper of Arcom to monitor oil pipelines as part of a SCADA industrial control system. The equipment communicated over satellite links, where bandwidth was slow and expensive, so the protocol had to be simple, efficient, and reliable.

Those are the same qualities I would want for sensors, embedded computers, and in this case home automation.

MQTT uses a publish-and-subscribe model. Programs do not normally send messages directly to one another. Instead, they connect to an MQTT broker, which receives messages and distributes them to any clients interested in them. The broker is the central post office for the system. Several MQTT broker implementations are available. I happen to use Mosquitto, a lightweight, open-source broker.

MQTT Messages are organized by topic. A temperature sensor might publish a value like this:

Topic: home/kitchen/temperature
Message: 72.4

Another program can subscribe to that topic and receive each new value as it is published. It does not need to know where the sensor is connected, what kind of hardware produced the reading, or which program sent it.

Commands work the same way:

Topic: home/living-room/lamp/set
Message: ON

Any program can publish the command, and whichever program or device subscribes to that topic can act on it. Your code only needs to understand the topic and the message. It doesn’t need to know how the rest of the system works.

Zigbee devices do not speak MQTT, and MQTT clients do not understand Zigbee. Zigbee2MQTT is the bridge between the Zigbee coordinator and the MQTT broker, translating Zigbee traffic into MQTT messages and MQTT commands back into Zigbee.

Zigbee2MQTT also includes an optional web interface that is useful for managing and debugging the Zigbee network. It can be used to pair and rename devices, view their state and capabilities, change settings, inspect logs, and send test commands.

Mosquittoes Buzzing on the Farm

The Mosquitto project includes more than the broker itself. The project also provides command-line tools and a C client library called libmosquitto.

The command-line tools make it possible to publish messages, subscribe to topics, and watch traffic passing through the broker without writing any code. mosquitto_pub sends a message to a topic, while mosquitto_sub subscribes to one or more topics and prints the messages it receives. They are particularly useful for testing and debugging an MQTT system.

In my earlier article, Writing Your Own Drivers, I wrote about how pIoTServer adds new hardware through plugins. MQTT is just another one. The only difference is that this driver uses libmosquitto to connect to the broker, subscribe to configured topics, and publish commands when values change. Once loaded, pIoTServer treats MQTT devices no differently from any others.

All pIoTServer devices are defined in the JSON configuration file. Adding MQTT support is simply a matter of adding another device definition. In this example, the driver connects to an MQTT broker and exposes two Zigbee smart plugs as pIoTServer output values.

{
    "devices": [
        {
            "deviceID": "MQTT_1",
            "device_type": "MQTT",
            "mqtt.host": "<your server>",
            "pins": [
                {
                    "data_type": "BOOL",
                    "gpio.mode": "output",
                    "key": "PLUG_1",
                    "other.props": {
                        "mqtt.target": "plug_1"
                    }
                },
                {
                    "data_type": "BOOL",
                    "gpio.mode": "output",
                    "key": "PLUG_2",
                    "other.props": {
                        "mqtt.target": "plug_2"
                    }
                }
            ]
        }
    ]
}

The complete source code for the MQTT driver is available on GitHub for anyone who would like to explore the implementation.

To turn on PLUG_1, we send a REST PUT request to pIoTServer with the new value in the request body. For example:

PUT /values HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: royal9:8081
Connection: close
Content-Length:  15

{"PLUG_1":true}

Conversely, to retrieve the current state of PLUG_1, we send a REST GET request to pIoTServer:

GET /values?keys=PLUG_1 HTTP/1.1
Host: royal9:8081
Connection: close

he response includes the current value of the plug, along with its title, timestamp, and other metadata:

{
    "values": {
        "PLUG_1": {
            "auto": false,
            "display": "false",
            "time": 1785279894,
            "value": "0"
        }
    }
}

MQTT devices can also be controlled by the pIoTServer scheduler. This example turns PLUG_1 on automatically at sunset.

"sequence": [
    {
        "enable": true,
        "name": "Sunset",
        "trigger": {
            "timeBase": "sunset"
        },
        "steps": [
            {
                "action": [
                    {
                        "cmd": "SET",
                        "key": "PLUG_1",
                        "value": "on"
                    }
                ]
            }
        ]
    }
]

Of course, REST requests and scheduled sequences are only part of the story. Because MQTT devices are integrated into pIoTServer like any other device, they automatically appear in the web dashboard. They can be monitored, switched on and off, and incorporated into schedules alongside I²C, 1-Wire, and RS-485 devices without any special handling. The dashboard itself is described in my earlier article, A Dashboard for My Garden Irrigation.

By this point, I no longer have to think about MQTT. As far as the rest of pIoTServer is concerned, these are just devices.

Beyond Smart Plugs

Smart plugs are an easy place to start, but Zigbee is not limited to commercial switches and sensors. It is also possible to build your own Zigbee devices using inexpensive development boards.

I am currently experimenting with an ESP32-C6-based device that joins the Zigbee network and communicates through Zigbee2MQTT like any other device. The ESP32-C6 includes an IEEE 802.15.4 radio for Zigbee and Thread, and Espressif provides an official Zigbee SDK with working device examples.

In my test, a simple on/off-light application joined the network. After I added a small external converter describing the device to Zigbee2MQTT, it could be controlled by pIoTServer through the same MQTT interface used for commercial plugs and dimmers.

This opens the door to all sorts of custom sensors, relays, lighting controls, garage-door controllers, and other cool devices I haven’t even thought of yet.

What Didn’t Change

Adding MQTT support didn’t require changing the rest of pIoTServer. The REST interface, scheduler, dashboard, and existing device model works the same as before. The MQTT plugin just translates between pIoTServer values and MQTT messages. No hacks. No MQTT-specific paths through the rest of the code.

Zigbee and MQTT extend pIoTServer beyond hardware connected directly to the Raspberry Pi. They make it practical to use off-the-shelf home automation products while still supporting custom devices built for a particular job. Smart plugs are only the beginning. The same approach can be used for switches, dimmers, sensors, lighting controls, and whatever else I decide to build next.

Oh, and for the TL;DR readers asking, “Why not just use Home Assistant?” After Insteon died, I did, for several years. Eventually, I decided I wanted a system that wasn't constantly changing underneath me. As I explained in We Don’t Need No Stinking Ecosystem, I wanted a small, self-hosted platform that I could completely understand, one that didn't surprise me with constant updates and that I could extend as my needs evolved.

That theme of controlling your own world runs through much of what I write, whether the subject is electronics, software, baking, perfume, or motorcycles. The subject may change without warning, but the underlying questions are usually the same: How was this made? Why was it built this way? And who gets to control it?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

Now that we have the machinery worked out, the next question is how the farmer sees and controls it.

What’s going on in the garden?

The system’s web dashboard answers that question. It gives me a single place to see current conditions, check what the system has been doing, review problems, manage schedules, and take control when necessary.

The dashboard is only the operator’s view. pIoTServer handles the actual automation and sends Pushover notifications to my phone when something goes wrong.

Like the rest of the systems I build, this one is self-hosted. I do not outsource its control, its data, or its availability to a third-party vendor or cloud service. The dashboard runs on my own hardware and talks directly to pIoTServer through its REST API.

10,000-Foot View

At a high level, farm-web is your basic web application. It is a collection of HTML, JavaScript, and CSS files served by nginx from a Docker container. The browser uses it to make REST requests to pIoTServer, which remains responsible for the actual automation.

Using standard web technologies means the dashboard can run almost anywhere, on a desktop computer, laptop, tablet, or phone. It could also be displayed on a dedicated kiosk, much like the weather station I built for my kitchen, and mounted in the greenhouse or anywhere else a permanent farm display would be useful.

The code in farm-web does not contain the irrigation logic, maintain schedules, or decide when a rule should run. It presents information and sends operator requests. My automation system, pIoTServer, continues operating whether farm-web is available or not.

The dashboard is also not where the system is configured. Devices, schedules, rules, and other operating details are defined in pIoTServer’s configuration. farm-web lets the operator see and interact with the system, but it does not define how the system itself works. It is a tool designed for the farmer, not the programmer.

From the farmer’s perspective, the dashboard is organized into a handful of major areas. Sensors provide visibility into current conditions. Devices allow direct control of equipment. Schedules and rules show what the automation system is planning to do. Additional administrative pages provide access to system health, incidents, tracking information, and other management functions.

We will start with the Sensors page.

Environmental Sensors

The sensors themselves are defined in the pIoTServer configuration file. I covered how pIoTServer devices and properties are configured in an earlier article, so I will not repeat all of those details here.

Current environmental sensor support includes TMP10X temperature sensors, SHT25 and SHT30 temperature and humidity sensors, BME280 environmental sensors, VEML6030 light sensors, tank depth sensors, analog measurement devices based on the ADS1115 and MCP3427, and 1-Wire devices connected through a DS2482 bridge.

For the dashboard example, we will use the SHT30 that measures temperature and humidity in the garden. Its device definition appears in the pIoTServer configuration file as a JSON object:

"devices": [
    {
      "address": "0x44",
      "description": "Outside Environment Sensor",
      "device_type": "SHT30",
      "pins": [
        {
          "data_type": "TEMPERATURE",
          "key": "GARDEN_TEMPERATURE",
          "title": "Garden Temperature",
          "tracking": "track.range"
        },
        {
          "data_type": "HUMIDITY",
          "key": "GARDEN_HUMIDITY",
          "title": "Garden Humidity",
          "tracking": "track.range"
        }
      ],
      "title": "Garden"
    }
  ]

This tells pIoTServer that an SHT30 is connected at I²C address 0x44. Some devices provide only a single measurement, while others expose multiple functions. The pins array defines the individual properties produced by the device.

In this case, the SHT30 provides two properties: GARDEN_TEMPERATURE and GARDEN_HUMIDITY. Each property is assigned a unique key that can be used elsewhere in the system for automation, tracking, notifications, and dashboard displays.

Once pIoTServer has read the sensor, its individual properties are available through the REST API. We can request both values by name:

GET /values?GARDEN_HUMIDITY&GARDEN_TEMPERATURE

pIoTServer returns a JSON object containing the current value, formatted display value, title, and timestamp for each property:

{
    "success": true,
    "values": {
        "GARDEN_HUMIDITY": {
            "display": "46.94%",
            "time": 1784316906,
            "title": "Garden Humidity",
            "value": "46.936751"
        },
        "GARDEN_TEMPERATURE": {
            "display": "100.57°F",
            "time": 1784316906,
            "title": "Garden Temperature",
            "value": "38.092622"
        }
    }
}

Notice that the raw temperature value is returned in Celsius, while the display field contains the formatted Fahrenheit value used by the operator interface. The dashboard can use the raw value for calculations and the display value when it simply needs to show the reading.

The sensor values alone do not tell farm-web how they should be organized or displayed. That information also comes from pIoTServer. The pIoTServer configuration contains a ui.sensors object that defines the sensor groups, the properties included in each group, and the type of display used for each value.

For this installation, the Garden group is defined as follows:

"ui.sensors": {
    "garden": {
        "label": "Garden",
        "description": "Outside environment and soil conditions",
        "members": [
            {
                "key": "GARDEN_TEMPERATURE",
                "label": "Air Temperature",
                "kind": "temperature",
                "range": {
                    "min": 0,
                    "max": 120
                }
            },
            {
                "key": "GARDEN_HUMIDITY",
                "label": "Humidity",
                "kind": "humidity",
                "range": {
                    "min": 0,
                    "max": 100
                }
            },
            {
                "key": "RAIN_SENSOR",
                "label": "Rain",
                "kind": "binary",
                "falseLabel": "No Rain",
                "falseIcon": "☀️",
                "trueLabel": "Rain",
                "trueIcon": "🌧️"
            }
        ]
    }
}

Since the web application is not on the same hardware as the pIoTServer, it can’t read that configuration file directly. Instead, it retrieves the sensor layout from pIoTServer through the REST API:

GET /props?ui.sensors

From that response, farm-web finds the Garden group and uses its definition to build the display shown below:

• The group label becomes the heading at the top of the panel.

• Each member’s label becomes the title of its card.

• The key identifies the pIoTServer property displayed by the card.

• The kind tells farm-web how to format and present the property value.

• When present, the range defines the scale used by the gauge.

• For a binary property, trueLabel, trueIcon, falseLabel, and falseIcon define how its two states are displayed.

• The property returned by the API may also include a history tag. When history is set to true, the dashboard adds a History button to the card. Selecting it causes farm-web to request the property’s stored readings from pIoTServer and display them as a graph.

While the range in the sensor definition controls the scale of the gauge, we need to make an additional API call to retrieve the actual daily minimum and maximum values for those sensors.

GET /range?GARDEN_TEMPERATURE

{
  "success": true,
  "values": {
    "GARDEN_TEMPERATURE": {
      "max": 39.45182,
      "min": 21.945144
    }
  }
}

The web application converts those values for display and shows them as the day’s high and low readings on the sensor card.

Binary Sensors

Not every sensor reports a numeric value. Many field devices produce simple binary states such as Wet/Dry or On/Off. The sensor definition can provide labels and icons for both states, allowing farm-web to present binary properties in a form that is much more meaningful than simply displaying true or false.

GET /values?RAIN_SENSOR

{
  "success": true,
  "values": {
    "RAIN_SENSOR": {
      "display": "false",
      "time": 1784321494,
      "title": "Orbit Rain Sensor",
      "value": "0"
    }
  }
}

The raw property value remains a simple binary value. The ui.sensors definition shown earlier tells farm-web how to display that value. In this case, the kind is binary, and the trueLabel, falseLabel, trueIcon, and falseIcon properties determine the text and icon presented for each state.

Don’t Know Much About History

The current value and the day’s high and low readings show what is happening now. On the farm, however, we also find it useful to understand the longer-term picture.

One of pIoTServer’s more important features is its ability to track the minimum and maximum values reported by a property over time. Many monitoring systems store every individual sensor reading. That can produce a large amount of data while still making a simple question difficult to answer: how hot or cold did it get each day?

pIoTServer can instead maintain a series of range records. Each record contains the minimum and maximum values observed during a period, along with its timestamp. This preserves the information that matters for long-term comparisons without requiring the dashboard to retrieve and process every raw sensor reading.

Range tracking is enabled in the device configuration by setting the property’s tracking mode to "tracking": "track.range". Once enabled, pIoTServer continuously records the property’s minimum and maximum values over time and makes those records available through the REST API.

GET /history/GARDEN_TEMPERATURE

Because this history can extend across months or years, the endpoint supports days, limit, and offset parameters:

GET /history/GARDEN_TEMPERATURE?days=30&limit=100&offset=0

This allows farm-web to request only the portion of the history needed for the graph.

An abbreviated JSON response looks like this:

{
  "deviceID": "51623",
  "key": "GARDEN_TEMPERATURE",
  "range": [
    {
      "max": 24.6101,
      "min": 6.90585,
      "time": 1742515182
    },
    {
      "max": 24.6101,
      "min": -1.34546,
      "time": 1742586196
    },
    {
      "max": 23.8705,
      "min": 5.1648,
      "time": 1742687386
    }
  ],
  "success": true,
  "title": "Garden Temperature",
  "units": "DEGREES_C"
}

Each entry contains the minimum and maximum values recorded for a period, along with the timestamp for that range. farm-web converts the returned records into the operator’s preferred units and renders them as an interactive history graph. The displayed data can also be exported as a CSV file for further analysis.

Taking Control

Most of the time, the farm operates automatically. pIoTServer evaluates its rules and schedules, monitors the sensors, and decides when irrigation should occur without any intervention from me.

There are times, however, when manual control is useful. I may want to test a newly installed valve, verify that a repair was successful, water a specific plot outside its normal schedule, or simply confirm that a device is operating correctly. The Controls page provides that direct access without changing the automation itself.

As with the sensors, the controls begin with a device defined in the pIoTServer configuration. In this case, an MCP23008 I/O expander provides the output properties used to operate the irrigation valves. One of those properties is GI_4:

  "devices": [
    {
      "address": "0x27",
      "device_type": "MCP23008",
      "key": "GARDEN_27",
      "pins": [
        {
          "bit": 4,
          "data_type": "BOOL",
          "gpio.mode": "output",
          "key": "GI_4",
          "title": "C"
        },
        ...
      ]
    }
  ]

The remaining pin definitions have been omitted for clarity.

Once a property has been defined, it can be read or changed through the pIoTServer REST interface, just as we saw in the article on devices.

At this point we have defined the device and the properties that control it. The remaining question is how those properties become controls on the web dashboard.

Like the Sensors page, the Controls page is configuration-driven. Rather than hard-coding the layout, farm-web retrieves the ui.controls property from pIoTServer and uses it to construct the operator interface.

For this example, we will use the Garden Irrigation control group:

{
  "ui.controls": {
    "garden_irrigation": {
      "label": "Garden Irrigation",
      "control": "valve",

      "members": [
        "GI_8",
        "GI_6",
        "GI_4",
        "GI_5",
        "GI_3",
        "GI_7"
      ],

      "titles": {
        "GI_8": "Plot A",
        "GI_6": "Plot B",
        "GI_4": "Plot C",
        "GI_5": "Plot D",
        "GI_3": "Plot E",
        "GI_7": "Plot P2"
      },

      "descriptions": {
        "GI_8": "BLU",
        "GI_6": "WHT",
        "GI_4": "BRN",
        "GI_5": "ORN",
        "GI_3": "GRN",
        "GI_7": "YEL"
      }
    }
  }
}

The members array identifies the properties that appear in the control group and determines the order in which they are displayed. The titles object provides the operator-facing name for each property, while descriptions supplies the shorter secondary label shown on the control.

The control value tells farm-web which type of widget to use. In this case, valve produces the controls used to open and close the irrigation valves.

From this definition, farm-web constructs the Garden Irrigation section of the Controls page.

The current Open or Closed status reflects each property’s live value. Selecting a card sends the appropriate request to pIoTServer to operate that valve.

Putting the Farm on Autopilot

Manual control is useful for testing, maintenance, and the occasional one-off watering. Most of the time, however, I want the farm to take care of itself.

As described in the earlier pIoTServer article, irrigation schedules define when work should be performed and the sequence of actions that make up each irrigation cycle. farm-web does not implement that automation. Instead, it presents the schedules in a form that is easy to understand, monitor, and manage.

Like the Sensors and Controls pages, the Schedules page is configuration-driven. The dashboard uses the ui.schedules property to organize related schedules into groups.

"ui.schedules": {
    "evening_garden_irrigation": {
        "description": "PWM sprinkler 9 PM",
        "label": "Evening Garden Irrigation",
        "members": [
            "4002"
        ]
    },
    "morning_garden_irrigation": {
        "description": "PWM sprinkler 9 AM",
        "label": "Morning Garden Irrigation",
        "members": [
            "4001"
        ]
    }
}

The dashboard uses this definition to create the schedule groups and tabs. Each schedule card summarizes the sequence, its current status, and the conditions under which it will execute. The timeline provides a graphical view of the irrigation sequence, making it easy to see when each plot will be watered and for how long. Operators can also temporarily enable or disable a schedule or run it immediately without modifying the underlying automation.

Each entry in ui.schedules references a sequence defined elsewhere in the pIoTServer configuration. The sequence contains the automation itself, including the trigger, any conditions that must be satisfied, and the actions that will be performed.

"sequence": [
  {
    "sequenceID": "4001",
    "name": "Garden irrigation morning sequence",
    "description": "GI_8 through GI_3 for 30 minutes at 9:00 AM",
    "trigger": {
      "time": "09:00 AM"
    },
    "condition": "RAIN_SENSOR == 0",
    "steps": [
      ...
      {
        "action": [
          ...
          {
            "cmd": "SET",
            "key": "GI_4",
            "value": "on"
          }
        ],
        "duration": 1800
      },
      {
        "action": [
          {
            "cmd": "SET",
            "key": "GI_4",
            "value": "off"
          },
          ...
        ],
        "duration": 1800
      },
      ...
    ]
  }
]

In this excerpt, the sequence begins at 9:00 AM only when the rain sensor indicates that irrigation is allowed. The step shown opens the valve represented by GI_4 and leaves it open for 1,800 seconds, or 30 minutes. Then the following step closes that valve before the sequence continues to the next plot.

farm-web reads this sequence information and turns it into the timeline shown on the schedule card. The operator sees when each plot will run and for how long without having to read the underlying configuration.

Here the morning irrigation sequence is running. Plot A is highlighted as the active step, while the timeline shows the remaining plots and their scheduled start times. The dashboard combines the sequence definition with the current state reported by pIoTServer, allowing the operator to see what is running, what comes next, and how long each step will last. It also displays the rain-sensor condition and provides an Abort control that stops the sequence and closes all of the valves.

Did It Really Run?

I believe in “trust, but verify”. Once the farm is operating on its own, I still want to know whether the scheduled work actually happened. Did the sequence run? Which plots were watered? How long did each valve remain open?

I don’t have hardware to measure the actual water flow. Fortunately, one of the reasons I moved to Arkansas was access to a good well, so I’m not watching a municipal water bill every month. Even so, runtime is useful. Since the flow rate of each irrigation zone is reasonably consistent, the amount of time a valve remains open provides a good indication of how much water was applied.

To answer those questions, pIoTServer records the runtime history of each tracked output. The dashboard summarizes that information so I can quickly see when each irrigation zone last ran, how long it operated, its total runtime, and the number of recorded runs.

Behind the Curtain

The farm-web code runs in a Docker container, with nginx serving the static HTML, CSS, and JavaScript files. Docker provides a predictable environment and makes deployment and updates straightforward.

nginx also acts as a reverse proxy. It serves the dashboard and forwards its REST requests to pIoTServer. From the browser’s perspective, everything comes from the same web server. The browser does not know where pIoTServer is running, and the pIoTServer REST service is not exposed directly to the public internet.

pIoTServer protects its REST interface with signed requests. Each request includes an API key, a timestamp, and an HMAC-SHA256 signature calculated from the request method, path, and body. pIoTServer calculates the expected signature independently and rejects the request if the signature does not match or the timestamp falls outside the permitted window.

The browser cannot safely hold the shared secret required to create those signatures. Instead, it sends an ordinary request to a small local proxy running alongside nginx. The proxy signs the request and forwards it to pIoTServer. Remote access reaches farm-web through Cloudflare, while the API secret and pIoTServer itself remain on the local network.

Another Piece of the Self-Hosted Farm Puzzle

farm-web has become the primary interface to my irrigation system. It gives me a single place to monitor conditions, manage schedules, verify that the automation is doing what I expect, and investigate problems when they occur. Although it was built for my own farm, it has grown into a general dashboard for pIoTServer that can be adapted to other automation projects simply by changing the configuration.

It is also another example of why I prefer self-hosting. The dashboard runs on my hardware, the data stays on my network, and the farm does not depend on a vendor’s cloud service to keep operating. Remote access is useful, but it does not change who owns or controls the system.

Like the rest of this project, farm-web is open source. If you’re interested in building your own self-hosted automation system, I hope some of the ideas here prove useful. Whether your project is a garden, a greenhouse, a workshop, or something entirely different, the same design philosophy applies: keep the automation independent of the user interface, expose it through a simple REST API, and let the dashboard focus on helping the operator understand what the system is doing.

If you’d like to learn more or build your own system, here are the resources behind this project:

• ⁠Vinnie’s Farm Automation Projects (article index)

• ⁠Garden Irrigation Evolution (hardware evolution)

• ⁠Garden Irrigation Hardware

• ValveNode RS-485 Field Bus Irrigation Controller

• ⁠pIoTServer

• ⁠farm-web

  ---

Self-hosting is not effortless. You have to build the system, maintain it, and understand enough of it to fix problems when they occur. The reward is that the farm keeps operating on your terms. The dashboard runs on your hardware, the data stays under your control, and no vendor can decide that a subscription, cloud account, or discontinued service is suddenly required.

That theme runs through much of what I write, whether the subject is electronics, software, baking, perfume, or motorcycles. The subject may change without warning, but the underlying questions are usually the same: How was this made? Why was it built this way? And who gets to control it?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

If you’ve read my series We Don’t Need No Stinking Ecosystem, then you already know that my decision to move away from Apple was about more than operating systems, phones, or applications. It was about getting out from under institutions that increasingly want their hands on the platforms, services, and infrastructure modern life depends on.

Yeah, and I hate their politics. I don’t trust these technology companies, or the tech bros running them, with the power they now exercise over speech, distribution, identity, and access that should belong to the individual.

The pattern is self-evident from recent history. During COVID, the 2020 election, the unrest surrounding astroturfed movements, and the rise of the woke movement, Facebook, Twitter, Apple, and Google, just to name a few, abandoned any pretense of neutrality. They suppressed reporting, censored dissent, removed applications, and promoted their favored political narratives.

And I see zero evidence that they have changed course. If anything, they have simply found new language and new tools for doing the same thing: ideologically trained AI, effective altruism, and surveillance repackaged as safety with Flock cameras.

It’s the same old story, the same old song and dance, my friend: trust the institutions, trust the science, trust the experts, trust the people in charge.

Only an idiot would.

The Cloud Was Never the Destination

Weaning myself from Apple started with dropping the iPhone and continued with replacing Apple Mail, Safari, Photos, iTunes, and the rest. Each step loosened its grip, but as long as my data still lived on somebody else’s servers, the job was not finished.

That was the next problem. Trading Apple’s cloud for Google, Dropbox, Spotify, GitHub, or some other provider is not independence. It is only changing landlords.

You may have more choice, but the provider still controls the service, the terms, the price, and ultimately whether you continue to have access at all. The rest of the journey was bringing those services, and the data behind them, under my own roof.

Self-Hosting Isn’t About Saving Money

Bringing those services under my own roof wasn’t just about avoiding subscriptions. By the time you account for hardware, storage, backups, electricity, maintenance, and the value of your own time, self-hosting may not be cheaper at all.

What self-hosting buys me is control. I decide where the data lives, how it is backed up, which software runs, when it is updated, and who can access it. I am not depending on a company to preserve a product, honor an API, maintain a pricing model, or continue tolerating the way I use its service.

Saving money is welcome when it happens, but agency is non-negotiable.

Continuity Matters Too

I also see self-hosting as a form of continuity engineering. A service should not disappear because the Internet is down, an API changes, a company abandons a product, or a provider decides it no longer wants your business. Remote access is useful, but it should be an added convenience, not the foundation on which the entire system depends.

Continuity also means keeping the data in its original form, or at least in a format I can extract and use elsewhere. If the server application fails, the project is abandoned, or something better comes along, I need a clear path out. Replacing one lock-in with another would defeat the whole purpose.

Having completed the mandatory philosophical rant, I’ll get off my soapbox, and let’s get to what you came here for.

The Platform

Before diving into PhotoPrism, Paperless, Forgejo, Jellyfin, Navidrome, Radicale, and the other services I host, it is worth spending a moment on the platform itself.

You don’t need enterprise hardware or a dedicated server room to run your own services. A Raspberry Pi with attached storage lets you experiment with self-hosting without spending much or committing to a larger system. Once you know what you want to run, an Intel NUC or similar mini PC can take you a long way.

I happen to use a Protectli VP3210. It is fanless, compact, quiet, and powerful enough to run the services I need without making a big deal out of it.

Another cool option is the compact mini-rack, like what Jeff Geerling builds.

The hardware itself matters far less than you might imagine. Most self-hosted services need only modest processing power. The real challenge is building a system that keeps working when something goes wrong and does not take your data with it.

Build for Failure

That means planning for failure from the beginning. The server, network equipment, and storage should be protected by a UPS so brief interruptions do not bring everything down and longer outages allow for a clean shutdown.

But surviving a power failure is only part of the job. Important data should also exist in more than one place, with at least one copy stored away from the primary server. RAID, mirrored disks, and snapshots may improve availability, but they are not substitutes for a real backup strategy.

Dependability also requires routine maintenance and a little restraint. Keep the operating system and containers updated, expose as little as possible directly to the Internet, and use secure remote-access tools when outside access is required.

I’m also a big believer in keeping a spare box around that can take over if the primary server fails. Besides shortening recovery time, it gives you a safe place to test a complete restore without disturbing the working system.

And call me paranoid, but I would keep that spare in a fire-resistant, EMP-protected safe, along with a current copy of the data needed to rebuild the system. The supply-chain shortages of recent years taught me not to assume that replacement hardware will always be one overnight shipment away.

Ubuntu and Docker

As for the underlying operating system, I chose a headless server running Ubuntu LTS. It is well supported, widely documented, and designed for long-term stability. The less time I need to log into it and fuss around, the more I like it.

Most of my services run in Docker containers. Docker has become the common deployment model for many self-hosted applications, and for good reason. It simplifies installation, isolates services from one another, makes upgrades relatively painless, and provides a consistent way to manage software regardless of how it was originally packaged.

With the platform out of the way, the more interesting question is what to run on it and why. I did not set out to replace every commercial service simply for the sake of self-hosting. I chose applications where control, continuity, privacy, or long-term access mattered enough to justify taking responsibility for them myself.

What follows is not a list of software recommendations so much as a look at the services I chose, the problem each one solves, and why I decided it was worth bringing that part of my digital life back under my own control.

PhotoPrism: Replacing iCloud Photos

Photos were one of the first things I wanted out of the cloud. Over the years, iCloud Photos had quietly become the permanent home for thousands of pictures and videos, which meant Apple controlled not only where they were stored, but how I accessed, organized, synchronized, and exported them.

Regardless of which replacement you choose, your first step is to get the originals out of iCloud’s stinking paws, preserve the metadata, organize the files, and make a backup somewhere other than the server hosting them. Trust me on this.

Sadly, the process involves more than simply downloading a pile of pictures. You have to deal with duplicates, Live Photos, videos, timestamps, location data, directory structure, and some reliable way of getting new photos off the phone. It’s never easy getting away from the pusherman.

I looked at several photo-hosting options but ultimately chose PhotoPrism. It is open source, does a great job of indexing and cataloging image and video files, and still gives me a modern web interface for browsing, searching, and organizing them. What I really like, however, is that PhotoPrism does not own the library. The photographs remain ordinary files stored in directories I control, while PhotoPrism simply builds a searchable catalog around them.

Setting up PhotoPrism was easy. I deployed it as a Docker container alongside MariaDB and pointed it at the directory containing my photo library. Once configured, PhotoPrism indexed the collection, generated thumbnails, extracted metadata, and built its searchable catalog automatically.

In my setup, the original photographs live in their own directory, while PhotoPrism keeps its thumbnails, search index, cache, and other application data separately. MariaDB has its own storage as well. That separation matters because the originals are the library. Everything else can be rebuilt.

The photos are still just files. I can browse them, copy them, back them up, or move them somewhere else without asking PhotoPrism for permission. The database and thumbnails are useful, but they are not the library.

Once the containers came up, I logged in as the administrator and waited, and waited, while PhotoPrism finished indexing the collection and generating thumbnails. It was worth it because, when it was done, my photos were finally free from Apple’s grasp.

On Android and iPhone, you can use PhotoSync to send new pictures to PhotoPrism. It is not quite as transparent as iCloud on an iPhone and requires a little setup, but once configured it works well.

Contacts and Calendars

I never liked giving Apple access to my contacts and calendars, especially when you consider what those records reveal. A contact list maps your relationships. A calendar records where you have been, where you plan to be, who you meet with, and what matters to you. Together, they provide a remarkably detailed picture of your life.

We have seen all too often how governments, corporations, and self-appointed experts convince themselves that access to such information is justified in the name of safety, public health, security, misinformation, or some other greater good. Once the information is centralized, the temptation to use it always seems to follow.

Getting the data out was easy enough because both contacts and calendars already use open standards. Contacts can be exported as vCard files, calendars as iCalendar files, and most modern applications can synchronize directly through CardDAV and CalDAV.

After looking at the available options, I found Radicale to be the simplest and best fit for what I wanted. It is free, open source, and its developers have zero interest in turning it into the next all-in-one collaboration platform. It is just a CardDAV and CalDAV server, and that’s all.

Radicale has been started as a (free topic) stupid school project replacing another (assigned topic) even more stupid school project.

Radicale does not own the data. Contacts and calendars remain ordinary files stored in standard formats and a simple directory structure. I can inspect them, back them up, copy them, or move them to another server without asking Radicale for permission. The server provides synchronization, but it is not the data.

Like PhotoPrism, I run Radicale in a small Docker container with its configuration mounted read-only and its data stored in a separate directory on the host. The container runs under an ordinary user account rather than as root, and the data directory is included in my normal backup routine.

Radicale listens on port 5232, but I do not treat that as an invitation to expose it directly to the Internet. Remote access should sit behind proper authentication, TLS, and whatever secure proxy or private-network arrangement you use for the rest of your services.

On Android, since I use Fossify Calendar and Android’s Contacts app, I needed a little extra plumbing. DAVx⁵ connects both to Radicale and handles the synchronization in the background. Day to day, I rarely open it unless something needs troubleshooting.

On macOS, no extra plumbing was necessary. I simply added the Radicale accounts to the built-in Contacts and Calendar applications, which already support CardDAV and CalDAV. If you are an iPhone user, the process is much the same. Add the account once, and the built-in Contacts and Calendar applications handle the rest.

Paperless: Replacing the Filing Cabinet

In the past, I used the scanner built into my laser printer to turn important documents into PDFs. These days, phone cameras and scanning apps are good enough that I usually scan them directly to PDF instead. Receipts, warranties, tax records, and other important paperwork ended up in folders on a local file server. That worked well enough until the collection grew large and finding something became an exercise in remembering where I had filed it.

I have been experimenting with self-hosting Paperless-ngx, another community-supported, open-source document management system. It imports PDFs and images, performs optical character recognition (OCR), indexes their contents, and makes them easy to search through a web interface.

Like the other self-hosted applications I use, Paperless builds useful services around the documents rather than taking ownership of them. The PDFs remain mine. Paperless simply makes them easier to organize, search, and find.

One area I find particularly interesting is combining the archive with a self-hosted language model. Projects such as Paperless-AI and paperless-gpt are already flying at nap-of-the-earth level, close enough to use today if you are willing to do a little tinkering. The AI could integrate directly with Paperless or simply index the PDFs themselves.

That would allow me to ask questions such as, “How much did I pay for that well pump?” and have the system locate the receipt and point me directly to the source document.

Jellyfin: Do You Actually Own the Movies You Buy?

In an earlier article, I wrote about movies and television shows disappearing from my Apple TV library after I had supposedly purchased them. The reason might be a licensing agreement, a dispute with a studio, or maybe a decision by someone, somewhere, that it was in my best interest that the content no longer be available. Whatever the explanation, the result was the same: I had paid for something and now I could no longer access it.

A digital purchase tied to somebody else’s store is not really ownership. It is merely permission to watch for as long as they allow it. Apparently, this is what you will own nothing and be happy looks like in practice.

So screw them. I started buying discs, often second-hand, converting them into ordinary video files, and storing them on my own NAS. It requires more effort than clicking Buy, but in the end the files are mine and the library cannot be altered by some third-party asshole.

Today the common practice is to rip commercial discs to MKV using MakeMKV and then transcode the resulting files with HandBrake. This produces clean, unprotected video files suitable for efficient storage and streaming while keeping the transcoding work inside the tool most people already know and trust.

Once the files were ready, I stored the video library on its own disk drive and ran Jellyfin in a Docker container to serve it up. Another option is to store the files on a NAS and point Jellyfin to the library there.

Jellyfin is a free, open-source media server designed to organize and stream personal collections of movies, television shows, music, and other media. Think of it as your own private version of Netflix, except the content is stored on hardware you control rather than somebody else’s servers.

Jellyfin catalogs movies and television shows, downloads artwork and metadata, and tracks viewing progress. To watch something, I open the Jellyfin app on a television, streaming box, phone, tablet, or computer. The experience is much like using a commercial streaming service: browse the library, choose a title, and press Play. Clients are available for most major platforms.

Like most of the services discussed here, Jellyfin does not require particularly powerful hardware. If your clients can play the stored media directly, even modest systems work well. Where this starts to break down is with on-the-fly transcoding or multiple simultaneous users. For that, you’re gonna need a bigger boat.

Navidrome: Play That Funky Music

Music has many of the same ownership problems as movies and television. Streaming services are convenient, but access ultimately depends on licensing agreements, subscriptions, and the whims of whoever happens to control the platform.

Just like my movie collection, I solved the problem the old-fashioned way. I ripped my music CDs to disk and stored the files on my server.

To host and stream those files, I use Navidrome. And guess what, Navidrome runs in a Docker container too and simply points at a directory containing the music library. It is a free, open-source music server designed around simplicity rather than becoming an all-in-one media platform.

Navidrome catalogs the library, downloads artwork and metadata, builds playlists, and makes the collection available through a web interface. It also supports the Subsonic protocol, which means there are numerous compatible clients available for phones, tablets, desktop computers, and even car audio systems.

Like Jellyfin, Navidrome does not own the music. The files remain ordinary files stored in ordinary directories. I can copy them, back them up, move them to another server, or replace Navidrome entirely without affecting the collection.

Navidrome provides the service. The music library is the dish.

Forgejo: Your Work Has Value Too

Most people think of photos, documents, music, and movies as personal data. As a software developer, I would add source code to that list.

Years of work can be wrapped up in a repository. Even if the code itself has no commercial value, it often represents thousands of hours of learning, experimentation, and problem solving.

For years I used GitHub, and I still use it as my primary platform for sharing public projects. It has its place. But like every other hosted platform, it comes with baggage. Content moderation disputes, shifting policies, and the general trend toward centralization all serve as reminders that GitHub is ultimately Microsoft’s house.

The AI question makes that especially clear: if I publish code on GitHub, should Microsoft be allowed to train on it and generate similar code without giving credit or attribution?

To host my own repositories, I run yet another Docker container, this one running Forgejo. Forgejo manages Git repositories, issues, pull requests, and projects through a browser-based interface not unlike GitHub.

Because Git was designed as a distributed version control system, every repository is already self-contained and portable. Forgejo does not own the repositories. I can back them up, mirror them elsewhere, move them to another server, or replace Forgejo entirely without affecting the source code.

Taking Back Your Digital Life

Self-hosting is not effortless. You have to do the work. The reward is owning a little more of your digital life. But at the end of the day, the data is under your control. It lives in open, portable formats, and even if a developer walks away tomorrow, your server keeps running. And chances are, somebody else will step in and pick up the slack.

That question of who gets to decide runs through much of what I write, whether the subject is electronics, software, baking, perfume or motorcycles. The subject may change without warning, but the underlying questions are usually the same: How was this made? Why was it built this way? And who gets to control it?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

I have spent a lot of time trying to make my farm automation projects reliable. But the unavoidable reality is that, no matter how carefully a system is designed, things will still break from time to time. Hardware stops responding, networks disappear, power goes out, and valves sometimes refuse to cooperate.

Not only do I lack the luxury of watching a dashboard all day waiting for trouble, I have absolutely zero interest in doing so. The systems should be designed to let me know when something is broken.

That is where notification services come in. Years ago, the answer might have been to automate a pager or send an SMS message. Later, smartphones made push notifications the obvious solution.

Push notifications became important because mobile applications could not simply remain awake indefinitely waiting for new information. A notification service provided the path from a remote server to the device and alerted the user when something required attention.

I had experience coding against Apple’s notification service and was familiar with Google’s as well, but I was not about to depend on either company’s permission to operate my farm. My systems should not care what brand of phone I happen to own next year, and building around either company’s notification stack would have recreated exactly the kind of dependency I was trying to leave behind.

What I wanted instead was a reliable, cross-platform notification service with a simple API that could be called from code, shell scripts, monitoring tools, or small services, with support for priorities, sounds, and acknowledgements when needed.

I considered Gotify and ntfy, both capable self-hosted notification platforms with simple APIs and their own delivery infrastructure, but they did more than I needed. I wanted reliable delivery across both Android and iOS without operating another internet-facing service just to send a few alerts.

Behold Pushover

Obviously, I was not the only one looking for a simpler solution. More than a decade ago, software developer and consultant Joshua Stein needed a reliable way for his scripts, servers, and monitoring tools to reach his phone without building an entire notification system himself.

What started as a side project eventually became Pushover. Unlike many technology services that expand into sprawling platforms, it has remained focused on one job: getting a message from a program to a person’s phone.

Pushover is butt-stupid simple to use. Sending a notification is just an HTTPS request containing an application token, a user key, and a message:

POST /1/messages.json HTTP/1.1
Host: api.pushover.net
Content-Type: application/json; charset=utf-8

{
  "token": "YOUR_APPLICATION_TOKEN",
  "user": "YOUR_USER_KEY",
  "title": "Test Message",
  "message": "Something requires your attention",
  "url": "https://example.com/status"
}

A program sends a small JSON document, and a notification appears on the recipient’s phone. The API also supports priorities, custom sounds, images, acknowledgements, and other features, but the basic use case remains simple and straightforward.

The receiving side is just as simple. You install the Pushover client on an Android phone, iPhone, iPad, or desktop browser, sign in, and register the device with your account.

Messages sent to your user key normally appear on all registered devices, although they can also be directed to a specific device.

After a 30-day trial, an individual license costs $4.99 once per platform, with no subscription fee, and that license covers all of your devices on that platform.

That’s it!

How I Use It

I just want to know when something is broken.

I use Pushover to report when a system becomes unreachable, a valve or field device stops responding, AC power fails, a hardware fault appears, or a failed system recovers and begins responding again.

My code detects and records the problem, then sends a POST request to the Pushover API. Pushover delivers the message, so no one has to sit in front of a dashboard waiting for something to fail.

How About Some Privacy?

Pushover does not manage my devices, collect sensor data, or control how my systems operate. It only delivers a message when one of those systems decides that I need to know something.

Pushover can see the notifications it delivers, but they do not need to contain sensitive information. A message can simply report that a problem exists and link to a protected dashboard containing the details. In my case, that dashboard is reached through a Cloudflare Tunnel over HTTPS, which also avoids opening an inbound port on the farm network.

Someone monitoring my Wi-Fi, local network, ISP connection, or the path across the Internet can see that traffic is being exchanged, but cannot read or alter its contents in transit. Cloudflare can technically inspect the traffic because it terminates the HTTPS connection, so the application still needs proper authentication and security.

Why This Still Fits a Self-Hosting Mindset

Self-sufficiency does not require every supporting service to run on hardware you own. What matters is keeping the essential operation and decision-making under your control, while using outside services only as limited, replaceable tools.

My systems still detect problems, record what happened, and decide when to send an alert. If Pushover is unavailable, the farm keeps operating. I simply may not receive the message immediately.

I am not outsourcing the system. I am outsourcing the last mile between the system and my pocket.

That same question of control runs through much of what I write, whether the subject is electronics, software, baking, perfume or motorcycles. The subject may change without warning, but the underlying questions are usually the same: How was this made? Why was it built this way? And who gets to control it?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

Long before I began writing about leaving Apple, I was writing about privacy, cryptography, and technological independence. More than that, I spent much of my life building the tools used to protect those things. As someone who came out of the OG cypherpunk tradition, I have always believed that privacy is not a privilege to be granted. It is a preexisting right, and the primary purpose of our institutions is to secure such rights, not to dispense them.

That principle applies to every fundamental right. The right to keep and bear arms appears immediately after the protections of speech, religion, assembly, and petition in the Bill of Rights for a reason: liberty cannot be preserved through complacency. Rights survive only when people remain vigilant and are willing to defend them against erosion, whether that erosion comes from government, corporations, or institutions claiming to act in the public interest.

Strong encryption, open systems, and code are simply the modern tools we use to defend what was already ours.

That same concern eventually shaped how I looked at the technology I used every day. What pushed me away from Apple was not merely constant platform churn, but a growing distrust of a technology culture increasingly comfortable with centralized identity, surveillance, and institutional control.

That is why we built tools that made individuals harder to control. The newer tech-bro instinct seems more interested in making it easier for institutions to control us by deciding what is best for everyone else, a familiar strain of Progressivism dressed up as Effective Altruism, trust and safety, or responsible innovation.

That path is little more than a way station to tyranny. It rejects the self-evident truth that our rights are inherent, replacing liberty with permissions granted by institutions claiming to act for our own good.

The articles collected here document my effort to move in the opposite direction: away from dependency, centralization, and institutional control, and toward cross-platform tools, self-hosted services, and systems I can understand, maintain, and control myself.

• Part 1 — Leaving the Apple Ecosystem: Why I began moving away from Apple and started exploring Linux as my primary computing platform.

• Part 2 — Continuing the Move: My early experience setting up Ubuntu and adjusting to daily life outside macOS.

• Part 3 — Replacing the Applications: The software and cross-platform tools I chose to replace the applications and services I had used inside Apple’s ecosystem.

• Part 4 — Moving Beyond the iPhone: My search for a practical mobile-phone alternative that reduced my dependence on Apple.

• Part 5 — Privacy and Independent Services: A closer look at privacy-focused services and taking greater control of my data and communications.

• Part 6 — Nine Months Later: An honest follow-up on what worked, what did not, and how life outside the Apple ecosystem turned out.

• ⁠We Have Always Been at War with Cryptography: Why government-mandated backdoors and weakened encryption threaten the privacy and security of everyone.

• I’m a Pushover for Self Hosting: Adding remote notifications without Apple’s infrastructure.

• Taking Back the Infrastructure: Self-Hosting the Services I Use Every Day
  A look at how PhotoPrism, Paperless, Forgejo, Jellyfin, Navidrome, Radicale, and other self-hosted services helped me replace cloud dependencies with systems I control myself.

• Self-Hosting Without the Exposure

  How to access your self-hosted applications with Cloudflare while keeping your home network private.

• Self Hosting Your Substack

  If I believe in self-hosting, why am I still publishing on Substack? Here’s a tool to help you take back the content you wrote.

• There’s an app for that, too!

  If this is where restroom technology is headed, I may just have to self-host my own toilet paper now.

Funny story: I was using GPT for grammar and spelling edits, and it became absolutely twitterpated over the Thomas Sowell quote above.

A fitting reminder that the tech bros, and systems they build to act on their behalf, cannot be trusted to decide what the rest of us are allowed to say, read, or think.

That question of who gets to decide runs through much of what I write, whether the subject is electronics, software, baking, perfume or motorcycles. The subject may change without warning, but the underlying questions are usually the same: How was this made? Why was it built this way? And who gets to control it?

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

That earlier project started with the simple Linux GPIO 1-Wire path. Later, I moved to the DS2482-100 I²C-to-1-Wire bridge and added better ESD protection and isolated sensor power to handle some of the practical problems that show up when a sensor cable leaves the bench and heads outside.

On the software side, I started with OWFS. OWFS stands for 1-Wire File System. It is a user-space system that makes 1-Wire devices appear as files and directories. That meant I did not have to write code to talk directly to the DS2482-100 or manually issue DS18B20 commands. I could mount the 1-Wire bus and read temperature values using filesystem I/O.

I needed to get the system working and out in the field, and OWFS was a good expedient path. It let me verify the DS2482-100, see the DS18B20 sensors, and get real temperature readings without getting wrapped around the axle writing my own driver.

Fast-forward to the present. I had added a couple of custom I²C devices for valve control and power management, and suddenly I found myself debugging all sorts of I²C timing problems, intermittent failures, and general weirdness.

I don’t know for certain whether OWFS caused any of the I²C problems. I suspected the real problem was in the TWI firmware and transaction handling that I let AI help me write. Either way, OWFS was another process with access to the same I²C bus through the DS2482-100, and I didn’t need another piece of code jabbering on the bus behind my back while I was trying to debug it.

Writing reliable I²C/TWI firmware on an AVR like the ATmega88 is not trivial. The AVR has to respond inside the master’s timing window, keep its state machine straight, and avoid returning stale or half-ready data while the Raspberry Pi is asking for the next byte. There are a lot of chainsaws to juggle.

In my case, a lot of the intermittent weirdness pretty much went away when I slowed the bus from 100 kHz to 80 kHz. That does not prove the Raspberry Pi was the root cause, but it tells me the system needed more timing margin.

This is also running on a Raspberry Pi Zero 2 W, and Raspberry Pi I²C has its own personality. The Pi is a Linux machine using the Broadcom I²C controller, not a small microcontroller bit-banging a private bus in a tight loop. If a slave stretches the clock, responds a little late, has slow rise times, or needs a little breathing room between command and reply, the Pi side can be less forgiving than expected.

Dropping to 80 kHz gave the AVR firmware, the bus edges, and the whole request/response contract enough room to stop stepping on itself.

Cutting Out the Middleman

Lowering the bus speed fixed enough of the immediate weirdness to keep moving, but it also made one thing obvious: I needed fewer hidden pieces touching the bus.

So to simplify my life, I decided to bring the DS18B20 temperature sensors into the same framework as the rest of my irrigation server code. Lose the OWFS mount, lose the FUSE layer, and lose the separate daemon talking to the DS2482-100 and doing I²C behind my back.

That meant writing a simple DS2482-100 / DS18B20 driver. A year ago, I would have started my quest by searching the net for sample code. But this is the Era of LLMs, so naturally I took the lazy man’s approach and asked AI to vibe-code me a sample. This is how a man saves two hours up front and then spends two weeks debugging the crappy code it gave him.

And so, I present you: ⁠ds2482_demo, a small standalone program that talks directly to the DS2482-100 and reads DS18B20 sensors without OWFS.

For the demo, I hung two DS18B20 sensors on the same 1-Wire bus, probed the bus to get their ROM addresses, and then asked the program to read the temperature from one of them.

./ds2482_demo probe
DS2482 address: 0x18 on /dev/i2c-1
DS18B20 sensors found: 2
  28-9d-74-46-d4-6c-0c-4d
  28-3d-85-35-00-00-00-69

./ds2482_demo temp 28-9d-74-46-d4-6c-0c-4d
DS2482 address: 0x18 on /dev/i2c-1 28-9d-74-46-d4-6c-0c-4d 
scratchpad 83 01 55 05 7f a5 a5 66 25
24.1875 C  75.5375 F  

To quote the great philosopher Mediocrates: “Meh, good enough.”

Debugging This Puppy

The next step was to stop guessing and look at what was actually happening on the wire. For that, I needed the right hammer.

Every now and then you run across something so cool it just blows your mind. The ⁠Saleae Logic 8, an 8-channel USB logic analyzer, is one of those. It takes the invisible stuff happening on tiny wires and turns it into something you can actually see, decode, and get a grip on. Since I do enough screwing around with I²C and other logic-level signals, I finally took the plunge and invested in one.

The Saleae Logic 8 is a small USB box about the size of a pack of mints, but it can watch eight digital lines at once, decode protocols like I²C and 1-Wire, and also record analog waveforms well enough to see things like slow edges, ringing, or a line that is not really making it cleanly high or low. It is not a replacement for a serious oscilloscope, but for this kind of embedded debugging it is incredibly useful: digital decoding and analog capture in the same view.

But what really makes it shine is the protocol decoder stack. The raw capture shows the actual voltage transitions, but the software can layer protocol decoders on top of that. In this case, I was able to decode the I²C side of the DS2482-100 and the 1-Wire side at the same time, which let me see both what the Raspberry Pi told the bridge to do and what the bridge actually put on the sensor bus.

Speaking Two Languages

Let’s walk through the details of how this works. Reading a DS18B20 sensor through a DS2482 is not a single “read temperature” operation. It is a conversation across two buses, with the DS2482 translating between I²C on the Raspberry Pi side and 1-Wire on the sensor side.

The Pi never talks 1-Wire directly. It sends I²C commands to the DS2482, and the DS2482 generates the reset pulses, write slots, read slots, and timing needed by the 1-Wire bus.

The Pi asks the DS2482 to perform a 1-Wire operation, then reads its status and data registers to find out what happened. The bridge handles the time-critical signaling, while the Pi handles the higher-level sequence: reset the bus, select a sensor, start a conversion, wait for it to finish, read the scratchpad, verify the CRC, and turn the result into a temperature.

The whole exchange is easier to follow as a sequence of requests and responses as illustrated in the table below.

Following the Bytes

Now that we have glossed over the entire operation at a high level, let’s follow a temperature reading through the complete sequence using the transaction flow described in the DS18B20 datasheet.

The datasheet breaks each access into three parts: initialization, a ROM command, and a DS18B20 function command. In practice, reading a temperature requires that sequence twice—once to start the conversion and again to read the completed result from the scratchpad.

Reset and Presence Detection

Every 1-Wire command sequence begins with a bus reset. The Raspberry Pi does not generate the required timing directly. Instead, it sends a command to the DS2482, which generates the reset pulse on the 1-Wire bus.

All devices on the bus watch for this reset. If one or more devices are present, they respond with a presence pulse, briefly pulling the bus low to announce that they are alive and ready to communicate.

The DS2482 detects the presence pulse and reports the result to the Raspberry Pi through its status register. If no presence pulse is detected, there is no point continuing because no sensor is available to answer the commands that follow.

At this stage, the Pi knows only that at least one device is present on the bus. It has not yet selected a particular sensor. That comes next, using the sensor’s unique 64-bit ROM address.

Select the Sensor

Every DS18B20 contains a unique 64-bit ROM address programmed at the factory. This allows multiple sensors to share the same 1-Wire bus while still being addressed individually.

After detecting that one or more devices are present, the Raspberry Pi tells the DS2482 to send a MATCH ROM command followed by the 64-bit address of the sensor it wants to use.

Every sensor on the bus receives the command, but only the device whose ROM address matches remains selected. The other sensors ignore subsequent commands until the next bus reset.

Communication is now directed to one DS18B20, allowing the Pi to issue a command without affecting the other sensors on the bus.

The next step is to tell the selected sensor to measure its temperature.

Start the Conversion

Once the sensor has been selected, the Raspberry Pi tells the DS2482 to send the DS18B20 CONVERT T command.

The DS18B20 begins measuring the temperature and will place the result in its internal scratchpad memory when the conversion is complete. The DS2482 does not perform the measurement; it carries the command across the 1-Wire bus and handles the required bus timing.

The conversion is not instantaneous. Its duration depends on the sensor’s configured resolution, ranging from 93.75 milliseconds at 9-bit resolution to as much as 750 milliseconds at 12-bit resolution.

Wait for the Conversion

The driver must wait for the DS18B20 to finish the conversion before attempting to retrieve the new result.

It can do this either by waiting for the maximum conversion time or, when the sensor’s power arrangement permits it, by polling the bus until the sensor indicates that the conversion is complete.

Attempting to read the scratchpad too soon would not provide a reliable indication that the newly requested measurement is ready. Once the conversion finishes, the new raw temperature value is stored in the scratchpad and can be retrieved.

Reading that value begins a new 1-Wire command sequence.

Re-select the Sensor

Before reading the scratchpad, the Raspberry Pi tells the DS2482 to perform another 1-Wire bus reset and checks for another presence pulse.

The reset ends the previous command sequence, so the target sensor must be selected again. As before, the Pi tells the DS2482 to send MATCH ROM, followed by the sensor’s 64-bit ROM address.

Only the matching DS18B20 remains selected. The driver can now request the scratchpad containing the newly measured temperature.

Read the Scratchpad

With the sensor selected again, the Raspberry Pi tells the DS2482 to send the DS18B20 READ SCRATCHPAD command.

The sensor returns nine bytes: eight bytes of scratchpad data followed by a ninth byte containing the CRC. The first two data bytes contain the raw temperature value. The remaining data bytes contain alarm thresholds, configuration information, and reserved values.

The DS2482 transfers the scratchpad bytes from the 1-Wire bus to the Raspberry Pi over I²C. The Pi retrieves each byte from the DS2482 as the read progresses.

At this point, the driver has the raw sensor data, but it should not trust that data until the CRC has been checked.

Verify the CRC

The DS18B20 scratchpad contains eight bytes of data followed by a ninth byte containing an 8-bit CRC.

The driver calculates the CRC across the first eight bytes and compares its result with the CRC byte returned by the sensor. If the two values match, the scratchpad was received intact.

If they do not match, one or more bits may have been corrupted during communication, and the temperature reading is discarded rather than treated as valid.

Decode the Temperature

Once the CRC has been verified, the driver can trust the scratchpad data.

The first two bytes contain the raw temperature measurement, stored least-significant byte first as a signed 16-bit two’s-complement value. At 12-bit resolution, each count represents 1/16 °C, or 0.0625 °C.

The driver combines the two bytes, interprets the result as a signed value, and applies the appropriate scale to produce a temperature in degrees Celsius. Since this is intended for human consumption, naturally we convert it to Fahrenheit.

Enough with the Bits Already

I think that is enough geek-bit talk for one article. I will not torment you further by walking through the Saleae capture byte by byte.

Suffice it to say, the protocol decoders let me follow the entire exchange from the Raspberry Pi, through the DS2482, and out onto the 1-Wire bus. The bytes on the screen matched what the datasheet said should be happening.

Once the standalone demo was reliably talking to the DS2482, I folded that code into pIoTServer as another native driver, following the same driver framework described in ⁠Raspberry Pi Internet of Things, Part 9.

"devices": [
        {
            "address": "0x18",
            "description": "DS2482 1-Wire temperature bridge",
            "device_type": "DS2482",
            "enable": true,
            "interval": 30,
            "pins": [
                {
                    "data_type": "TEMPERATURE",
                    "description": "Plot A Soil Temperature",
                    "interval": 30,
                    "key": "PLOT1",
                    "other.props": {
                        "address": "28-9d-74-46-d4-6c-0c-4d"
                    },
                    "title": "Soil Temp Plot A",
                    "tracking": "track.range"
                }
            ],
            "title": "Soil Temperature 1-Wire Bus"
        },
  ...
]

Well, Was It Worth It?

In the end, the result was surprisingly uneventful, which is exactly what I wanted. I replaced OWFS, the FUSE mount, and the supporting daemons with a few hundred lines of code that I understood, could debug, and could maintain.

The DS18B20 sensors became just another part of the pIoTServer ecosystem, and the soil temperature values flowed through the same configuration, logging, REST API, dashboard, and automation framework as every other device in the system.

Was OWFS the cause of the strange I²C behavior I had been chasing? Methinks not. There is still the lingering odor of crappy vibe coding in the AVR firmware. But removing OWFS eliminated another moving part from the I²C bus and gave me complete visibility into what was happening from end to end.

And if nothing else, I now understand a lot more about how a DS2482 actually talks to a DS18B20. Sometimes you have to open the hood and tear out a few wires to see what is really going on underneath.

Besides, if I am going to spend two weeks debugging and cussing at code, it might as well be my code.

If you care about how things are made and enjoy ripping out the wires and cussing at them like I do, you will probably like the rest of what I write about. It is not always code. Rumor has it that sometimes I jump from baking to perfume to motorcycles without much warning.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

The property had previously been used for cattle, so part of the job was turning pasture into vegetable beds. We had plenty of water from a nearby well, but we still had to get it to the crops. So we dug trenches, laid pipe, hooked up valves and filters, and ran drip lines.

A farm has a lot of jobs, and I did not want irrigation to depend on someone remembering to open and close valves at the right time every day. So I needed a field-expedient way to automate the watering while the rest of the farm was still being built.

My first irrigation controller was based on OpenSprinkler. I still think Ray Wang built a great platform: affordable, open, well documented, and fun to work with. It got us watering, and I learned a lot from studying his design choices.

OpenSprinkler does the job for most people, and I am not knocking it. But I wanted more than valve control. I wanted to track temperature and humidity in the garden and greenhouse, monitor ground temperature, and start tying irrigation into the rest of the farm automation work. That is where the controller started evolving.

The next step was moving the brains onto a Raspberry Pi. I wanted the system self-hosted and local, not dependent on somebody else’s cloud server to water crops. So I created pIoTServer: a Raspberry Pi-based farm automation system that could expose sensors, valves, pumps, and other devices through a REST API.

The first hardware version was still very much a prototype: a Raspberry Pi, a Waveshare 8-Ch Relay Expansion Board, AC relays, and a fair amount of wiring. It was not pretty, but it let us control devices, read sensors, and start collecting environmental data.

One of the things I really liked about the Waveshare board was that it treated the Raspberry Pi as the daughter board, not the other way around. I adopted that approach for my later designs because it gives the main board room for power, protection, relays, field wiring, and whatever other interfaces the controller needs.

Once the initial Raspberry Pi version was working, power became the next issue. Rural power does rural power things, and I wanted the controller to survive outages and run from a backup battery. I also wanted better control of the irrigation valves than just feeding them straight DC.

That led to the first PWM prototype. It used pulse-width modulation to open the valves and then hold them with less power. It worked as a proof of concept, but it was limited to eight circuits, and I still needed a better answer for automatic shutdown.

The next version had to solve two other problems: managing power and protecting the controller from the realities of farm wiring.

That led to another custom board. This one had power management built in, including battery backup and controlled shutdown, instead of treating power as something bolted on later. I also moved more of the field-facing hardware onto my own pIoTServer board layout: isolated I2C, 1-Wire support, and isolated DC/DC power. The idea was to give the Raspberry Pi some separation from the long wires, valve coils, noisy equipment, and other nonsense hanging off the controller.

The valve drivers moved onto stackable 8-circuit PWM cards, so the controller was no longer locked into one fixed number of outputs. That design served us pretty well for about a year.

But the garden kept expanding, and the valves kept getting farther from the controller. At some point, it became painfully obvious that this needed to be an RS-485 network. I am not sure how I missed that at the beginning of the project, but there it was.

The cleaner answer was a field bus: one RS-485 run, with small ValveNode boards sitting closer to the valves.

That led to ValveMaster, a plug-in card for the pIoTServer controller with an AVR handling the RS-485 side and talking to the remote valve boards. The Raspberry Pi still runs pIoTServer and the farm logic, but valve control moves into a distributed network instead of everything home-running back to one box.

The power-management work was its own rabbit hole. I started with discrete components controlling the power relay, but that only gets you so far. At some point the board needed a real brain for power control, so I moved that job to an ATmega8-16.

That let the board watch the power state, control the relay, shut the Raspberry Pi down cleanly, and remove power after shutdown. It could also wake the system back up when AC returned, when someone pressed the button, or on a timer so the controller could conserve battery power, wake periodically, and do whatever work was needed.

What This Evolution Adds

This evolution is built around the new ValveMaster prototype and the move toward RS-485 ValveNode boards, but the hardware is only part of it. A lot of the work went into pIoTServer itself: new device drivers, better reporting, notifications, tracking, rules, test tools, REST cleanup, and a redesigned farm-web dashboard.

Hardware

• ValveMaster / RS-485 ValveNode control
  A new plug-in ValveMaster card lets the pIoTServer controller talk to remote ValveNode boards over RS-485, moving valve control closer to the valves instead of home-running every valve wire back to one box.

• Power management
  Controlled shutdown and wake behavior.

• DS2482 / 1-Wire
  Moved away from FUSE/OWFS toward native driver support.

Software

• New device drivers and runtime support
  Added the pIoTServer runtime/driver work for ValveMaster and power management.

• Incident reporting and notifications

  Added incident reporting, and Pushover support so the system can report problems and send phone alerts when something needs attention.

• Rules and ad-hoc sequences
  Added rules and temporary sequences. This lets the system run one-off irrigation jobs, manual tests, and simple conditional logic without having to create a permanent schedule every time.

• Tracking, duration, and summary API
  Added tracking so the system can record what actually happened, not just what the current state says. This includes how long things ran, when they ran, and summary data the dashboard can use without pulling the whole history every time.

• piottool
  Consolidated the hardware debugging and module testing tools into one utility.

• Raspberry Pi Wi-Fi hardening
  As shipped, Raspberry Pis have been known to drop off Wi-Fi and do not always handle network recovery well on their own. We tightened up the Wi-Fi setup and added recovery behavior so the controller has a better chance of coming back without manual intervention.

• REST cleanup for better web UI support
  Cleaned up bugs and refactored parts of the REST API so the new farm-web interface could get cleaner, more consistent data.

More good stuff to come

I am in the process of deploying this version out to the farm now. As usual, every improvement seems to create three more things worth writing about, so I am going to break some of the details into separate articles instead of turning this update into a book.

Web dashboard / UI system
How the farm-web dashboard was redesigned around actual field use, and how the pIoTServer web interface is structured going forward.

Pushover notifications
I needed a way to find out when something broke without staring at logs or keeping the dashboard open all day. Pushover is a simple cross-platform push-notification service, and pIoTServer can use it to send alerts to my phone when something needs attention.

Cloudflare Access
I also needed remote access to the farm dashboard without exposing the farm controller directly to the public internet. Cloudflare gives me a controlled front door, so I can reach the web interface remotely while keeping the raw controller on the private side.

DS2482 / DS18B20 driver without OWFS
As part of the I2C debugging, I needed more direct control over the 1-Wire side of the system. That meant moving DS2482 / DS18B20 temperature sensing away from OWFS/FUSE and into native pIoTServer driver support.

There is always more work to do, and this thing is not done yet. But we are getting there. “Stuff’s getting better every day.” And man, what a long, strange trip it’s been.

If you care about how things are made and why they’re built the way they are, you’ll probably like the rest of what I write about. It is not always electronics. Believe it or not, I have been known to jump from baking to perfume to motorcycles without much warning.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

Easy peasy.

That was the theory, anyway. Then we became victims of our own success. The garden started growing faster than the wiring plan.

The irrigation system has gone through several stages since then, each one solving the problem directly in front of it. I wanted more than an off-the-shelf sprinkler controller.

• First came the Raspberry Pi control system, which gave me flexible scheduling, local control, and environmental data collection.

• Then came the first valve-control hardware: relays switching 24 VAC to ordinary sprinkler valves.

• After that came the question of how to drive those same valves from battery-backed DC power. That led to the boost-and-hold experiments.

• Later, I explored PWM valve control, using full power to open the valve and reduced average power to hold it open.

Each step worked. Each step also exposed the next problem.

The next problem was geography.

As the garden expanded, so did the number of zones. I could stack more PWM boards, but that only solved the controller side of the problem. Every new zone still meant more wire, possibly another trench, and another dedicated run back to the controller.

Looking at the garden from above made the problem hard to ignore. The beds were spreading out, the rows were changing, and new irrigation needs kept appearing in places that were never part of the original wiring plan. The question was no longer whether I could drive another valve. I could. The question was whether I really wanted every new valve to require another cable back to the controller.

Maybe It’s a Network?

At some point, as the valves started spreading out and the wiring kept multiplying, it became painfully obvious that this needed to be an RS-485 network. I am not sure how I missed that at the beginning of the project, but there it was.

I have spent enough time around control systems and communications links to recognize the pattern. What brought RS-485 back to mind was yet another cool project I had been working on. Once I looked back at the irrigation system, it was a natural fit.

RS-485 is an old industrial workhorse that has been around since the early 1980s. It survived because it solves a problem that never really went away: how to talk to equipment spread out over distance, in the harsh electrical environment of the real world.

A logic signal that works perfectly on a circuit board can fall apart once it has to travel down a long cable. Long cables pick up noise. Motors and relays throw garbage onto the wiring. Pumps start and stop. Ground voltage shifts from one end of the run to the other. Add conduit, wet soil, distance, and lightning weather, and suddenly a clean bench signal is not so clean anymore.

RS-485 was built for this. It sends data over a balanced pair of wires instead of relying on one fragile signal referenced to ground. The signal on one wire moves one way while the other moves the opposite way, and the receiver looks at the difference between them. Common-mode noise picked up along the cable tends to appear on both wires together. Because the receiver is looking at the difference between the two wires, it can reject much of that shared noise. That makes RS-485 much better suited for devices spread out across the farm, down long cable runs, and around pumps or other noisy equipment.

For a deeper technical overview, Texas Instruments has two useful references: RS-485 Basics and the RS-485 Design Guide. I also found the MurCal document on proper use of RS-485 Ports to be a gem.

Keep in mind that RS-485 is not a protocol. It is the wiring and signaling layer. That lets me build a simple command protocol on top of it: address a valve node, tell it what to do, and get a reply. More on that later.

Latching Solenoids

In-line automatic sprinkler valves, such as the Hunter PGV or Rain Bird CPF100, control the water used for our crop irrigation. These devices are easy to take for granted, but they are clever little pieces of engineering. I encourage you to take a few minutes and watch the video below to see how they work.

At the heart of the valve is a diaphragm and a small pilot passage controlled by a solenoid. The solenoid does not directly move all the water. It simply opens or closes a tiny control path, and the water pressure does the heavy lifting. That is the elegant part. A small electrical signal controls a much larger flow of water.

The downside is that a standard sprinkler solenoid has to stay energized the entire time the valve is open. My PWM approach makes that more efficient, but it does not change the basic problem. The controller is still feeding power to every open valve.

There is a better way: stop holding the valve open with power.

That is what latching solenoids do. They were designed for battery-powered irrigation controllers. Instead of needing continuous power, they only need a short pulse to change state. Send current one direction and the valve opens. Send it the opposite direction and the valve closes. After that, the driver turns off and the valve stays where it was.

What put me off at first was my assumption that latching solenoids would be expensive, obscure, or hard to source. I thought this was some esoteric European thing.

As Ron White would say, “I was wrong.”

The Hunter 458200 DC latching solenoid was only a few dollars more than a standard replacement solenoid. Rain Bird offers a similar latching solenoid, the K80920, though it costs a bit more.

As you can see in the photo, Hunter and Rain Bird solenoids use different thread designs, so they are not mechanically interchangeable. Electrically, however, they work the same basic way. They still use two wires and are controlled by the direction of current flow.

The good news is that within a given brand, standard and latching solenoids are generally interchangeable on compatible valve bodies. That meant I did not have to abandon ordinary irrigation valves. I could keep the same basic valve bodies and simply change the solenoid.

Even though the latching solenoids are technically not motors, but the control problem is similar to the lock actuator I used in the chicken coop project. Current one way moves the plunger one direction. Current the other way moves it back.

And it turns out that the VNH7100BAS H-bridge, like the one I used in my Humble Actuator project, is a natural fit for controlling the polarity. It might even be a bit of overkill, but when these valves get wet, dirty, old, or sticky they are a bargain at around $2.50 a piece.

The 30,000-Foot View

The architecture is simple: one master, many valve nodes.

I designed two boards for this system: a master board and a valve node board. The master sits near the Raspberry Pi and manages the RS-485 field bus. The valve nodes live out closer to the valves and drive the latching solenoids.

The Raspberry Pi still runs the irrigation logic. It decides what should happen and when. The master board translates those decisions into RS-485 commands. Each valve node listens for commands addressed to it, pulses the right solenoid in the right direction, and reports back.

Instead of running every valve back to the controller, I can run a shared four-wire cable through the field and hang valve nodes off that cable. Two wires carry 12 VDC power. The other two carry the RS-485 data pair.

The protocol is intentionally simple. At the electrical level, it is a 9600 baud serial line running over RS-485. The master sends a command to a specific valve node, and the node either acts on it or reports back: open a valve, close a valve, identify yourself, or report how many valves it can control.

If you want the deep details, I have made the schematics, board layouts, KiCad files, firmware, protocol notes, and test tools available in the ValveNode GitHub repository.

Valve Master Board

The first board is the Valve Master. This board stays near the Raspberry Pi and acts as the bridge between the main farm-control software and the irrigation field bus.

On one side, it talks to the Raspberry Pi over I2C. On the other side, it controls the RS-485 cable that runs out into the field. It also switches the 12 V field power, so the valve-node network does not have to sit powered all the time.

The master board does not directly open and close every valve. That is the point. Its job is to manage the bus, send commands to the valve nodes, receive replies, keep track of which nodes are out there, and report results back to the host.

I designed the I2C side so the board can be built two ways: as a NodeLync-compatible device hanging off the shared I2C bus, or as a daughter card that plugs directly into my Raspberry Pi irrigation controller. The hardware is built around an ATmega88PB-AU microcontroller, a MAX485-style RS-485 transceiver, a relay-switched 12 V field output, status indicators, and an AVR programming header.

Nothing exotic. Just the pieces needed to turn a Raspberry Pi command into a field-bus command that can survive outside the controller box.

The Raspberry Pi decides what should happen, sends the command over I2C to the Valve Master, and the Valve Master handles communication with the nodes in the field.

Valve Node Board

The valve node board lives out near the valves. It has the RS-485 interface, a small microcontroller, input protection and power conditioning for the incoming 12 V field supply, and the H-bridge drivers needed to pulse the local latching solenoids. Each node has an address, so the master can tell one specific board to open or close one specific valve.

Since this board literally lives out in the field, I designed it to fit comfortably inside a common 2FSE FS outdoor electrical box, protecting the electronics from the usual outdoor abuse while leaving enough room for wiring, strain relief, and service access.

Even with the microcontroller, RS-485 transceiver, H-bridge drivers, input protection, voltage regulation, terminal blocks, and the PCB itself, this is still a modest-cost board. In small quantities, I am roughly in the $30 to $45 range per valve node, not counting assembly, shipping, or tariffs.

You do not have to use the Valve Master board to use the valve nodes. The nodes speak ordinary 9600 baud serial over RS-485, so any controller that can send the right frames over an RS-485 interface can talk to them. A USB-to-RS-485 adapter, such as the Waveshare USB to RS232/485 Serial Converter, is enough for direct testing. In fact, that is how I did my initial debugging.

The ValveNode Protocol

This project does not need a complicated protocol. It only needs enough structure for the master to send a command over a 9600 baud RS-485 serial line to a valve node, get an acknowledgment back, and handle a small set of operations: open a valve, close a valve, report status, identify itself, or report how many valves it can control.

I implemented the protocol as a simple ASCII frame format. It is easy to type, easy to log, and easy to debug from a terminal. That way I could connect a USB-to-RS-485 adapter, type commands by hand, and see exactly what came back.

The bus is master-controlled. Nodes do not chatter whenever they feel like it. The master sends a request, and only the addressed node replies.

The frames are short ASCII messages. Each command starts with a marker, includes the target node address, a command code, any needed argument, and ends with a carriage return. Replies use the same basic idea, with an acknowledgment, status, or error result coming back from the addressed node.

For example, to tell node 5 to open valve 1, the master sends:

:05O1<CR>

That means: address node 05, send the open command, and pass valve 1 as the argument. If the node accepts the command, it replies with `A` for acknowledged:

:05A<CR>

A slightly more complicated example is the WHO command, which asks the bus which nodes are connected. The master broadcasts a W command to all nodes by sending it to the FF address:

:FFW<CR>

The tricky part is the reply. If every node answered at the same time, they would talk over each other and the bus would turn into garbage. To avoid that, each node waits for a short delay based on its address before replying.

For example, if nodes 1, 4, and 18 are on the bus, the master might see replies like this:

:01W<CR>
:04W<CR>
:12W<CR>

Each node waits in a 20 ms slot based on its address before replying. Node 1 waits about 20 ms, node 4 waits about 80 ms, and node 18 waits about 360 ms.

The master just has to keep listening long enough for the highest expected node address to answer. For example, if I allow up to 32 nodes, the discovery window needs to be a little over 640 ms, plus some margin.

Note: Addresses are written as two-digit hex values, so node 18 shows up as 12 in the reply.

The messages include a checksum so the devices can detect corrupted messages. The checksum is simple: add up the bytes in the message payload, keep the low 8 bits, and send that value as two hex digits at the end of the frame.

I left the checksum out of the examples above to keep them readable. The valve node firmware will also accept simple commands without one, which makes hand testing from a terminal much easier.

It is not cryptography, just a quick sanity check to keep noise on the line from causing havoc.

Building and Programming the Boards

Once the boards are assembled, they still need to be programmed. Both the master and node boards use AVR microcontrollers, and each board runs its own firmware. The code that runs on the boards is written in C and built with the AVR toolchain, and loaded using avrdude.

Both boards include provisions for a 6-pin AVR ISP connector that mates with an Atmel-ICE programmer.

In previous articles, I wrote in more detail about building AVR code and downloading it to the chip, including “avrdude: The Debugging Tool You Didn’t Know You Had” and “How Did You Make That.”

A newly programmed valve node starts out unconfigured. Before it can join the RS-485 bus, it needs a unique address.

After flashing the firmware, I connect each new node to the RS-485 bus by itself, put it into configuration mode, and assign it an address.

For example, to configure a fresh node as node 7, I send:

:07N<CR>

That tells the unconfigured node to store address 07 in EEPROM. After that, it responds as node 7 on the bus.

Then I can test it with a ping:

:07P<CR>

If everything worked, the node replies:

:07A<CR>

The address can also be changed later by putting that node back into configuration mode and sending it a new address command.

Test Tools and Diagnostics

Before these boards go into outdoor boxes, it is useful to have good ways to test and diagnose them on the bench. I can manually send commands through a terminal emulator over a USB-to-RS-485 adapter, but I wanted something a bit more intuitive to use. So, like any good programmer, I wrote some test utilities.

The first one is vnode, a command-line tool for talking directly to valve nodes over a serial port. It assumes a USB-to-RS-485 adapter is connected to the bus. From there, vnode can ping a node, open or close a valve, check status, ask for the firmware version, identify a node, run discovery, assign addresses, move a node to a new address, cancel configuration mode, or send a raw frame by hand.

For example:

$ ./vnode -d /dev/ttyUSB0 who
device: /dev/ttyUSB0
baud:   9600
tx: :FFW\r
rx: :01WB8  [checksum ok] node=1 WHO
rx: :02WB9  [checksum ok] node=2 WHO

$ ./vnode -d /dev/ttyUSB0 ping 1
device: /dev/ttyUSB0
baud:   9600
tx: :01P\r
rx: :01AA2  [checksum ok] node=1 ACK

$ ./vnode -d /dev/ttyUSB0 open 1 1
device: /dev/ttyUSB0
baud:   9600
tx: :01O1\r
rx: :01AA2  [checksum ok] node=1 ACK

That gives me a quick way to discover nodes, verify that a node is answering, and command a valve output without involving the Valve Master or the Raspberry Pi irrigation software. It can also be used to configure a fresh node, assign its address, and verify that it answers before it goes into the field.

The other utility is valve, which talks to the Valve Master over I2C, the same way the Raspberry Pi irrigation software would. That lets me test the full chain: Raspberry Pi side, I2C, Valve Master, RS-485, valve node, and finally the latching solenoid output.

The valve tool can:

• Check master status

• Switch field power on or off

• Discover nodes

• Ping a node

• Open or close a channel

• Check channel status

• Read master or node firmware versions

• Identify a node

• Handle address provisioning

The GitHub repository also includes test scripts, including hammer.sh, which repeatedly exercises the full Valve Master path. In bench testing, it completed 500 loops across five nodes, 10,016 commands total, with zero first-try failures.

About Those Tangled Wires

There is nothing wrong with reusing the 18 AWG sprinkler wire I already had in place. It was already buried, already routed to the right places, and for short runs it was perfectly serviceable.

But anywhere I was pulling new cable, I wanted something better suited to this design. The RS-485 data side is not the hard part at 9600 baud. The real issue is getting enough 12 V power out to the valve nodes over a few hundred feet of wire.

For new runs, I settled on 16 AWG, 4-conductor, stranded copper, direct-burial cable I came across online.

Two conductors carry +12 V and ground, and the other two carry the RS-485 A/B pair. Twisted pair would be nice, but copper, gauge, and burial rating matter more here than perfect data-cable theology.

Good Systems Evolve

This is not a completely different irrigation system. It is the next step in the same one.

The project started with a simple question: how do I drive ordinary sprinkler valves from a Raspberry Pi and keep them working when the power goes out? The answer kept evolving as the garden grew: relays, DC boost-and-hold, PWM, and now latching solenoids on an RS-485 field bus.

That reminded me of Donald Knuth’s old warning that “premature optimization is the root of all evil.” I had spent a lot of effort making ordinary sprinkler solenoids more efficient. The better move was to stop holding the valve open with power in the first place.

That is how good systems evolve. You solve the problem in front of you. If the system is useful, it grows. If it grows, it eventually shows you where the old design runs out of room.

Post Scriptum

After letting the invisible magic smoke out of two Rain Bird latching solenoids, I started wondering whether I had configured the valve node drivers wrong.

Rain Bird does not exactly give you engineering data for these parts, but I had seen them described as 9 V latching solenoids. Why should they, you should be using it with their hardware.

My first reaction was, “yeah, yeah, so what?” I was only hitting them with short pulses.

Then two of them stopped closing reliably after a day of cycling.

OK. Time for engineering.

The field bus is 12 V, but that does not mean the solenoid needs to eat the full 12 V pulse. The Rain Bird solenoid I measured was about 7.2 ohms. I added a 2.2 ohm resistor in series with the solenoid drive path, giving a total resistance of about 9.4 ohms.

Using Ohm’s law:

I = V / R
I = 12 V / 9.4 ohms
I ≈ 1.28 A

The solenoid itself then sees roughly:

V = I × R
V = 1.28 A × 7.2 ohms
V ≈ 9.2 V

That lets the field bus stay at 12 V while bringing the solenoid pulse down into the neighborhood of a 9 V latching-solenoid pulse. I also shortened the firmware pulse to 20 ms. The resistor is not being used as a continuous power device. It is only there to take the edge off a short actuation pulse. So far, 2.2 ohms 1 Watt resistor in series with the solenoid works fine.

If you care about how things are made and why they’re built the way they are, you’ll probably like the rest of what I write about. It is not always electronics. Believe it or not, I have been known to jump from baking to perfume to motorcycles without much warning.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

This page brings together the full cornucopia of farm automation projects I have built and shared from our farmstead. Irrigation systems, environmental sensing, weather stations, power management, chicken coop automation, and everything in between.

If you are thinking of building this stuff yourself and want to avoid the usual LLM hallucinations, dig through these articles. You will find details on the hardware and software I have actually built, along with the good, the bad, and the ugly behind the design decisions that got me there.

Setting up the Farm

• This is My Dirt
  20-May-24 — The Farmsteading Adventure Begins.

• Be Water, My Friend
  01-May-24 — Our first irrigation project at Stella Porta.

Raspberry Pi Farm Irrigation System

1. Farm automation made easy as Pi
   01-May-25 — Setting up the foundation for real-world IoT on the farm.

2. Controlling the Irrigation Valves
   01-May-25 — The basics of switching solenoids, safely and reliably.

3. Reading from the 1-Wire Sensors
   06-May-25 — Building the temperature sensing layer.

4. Using the I2C Protocol to Communicate to Sensors
   07-May-25 — How we wired it, read it, and filtered out the crap.

5. Reading Environmental Data with I2C
   09-May-25 — Expanding the sensor net and interpreting the data.

6. Raindrops Keep Falling On My Head
   26-May-25 — Detecting precipitation (and making decisions from it).

7. More I/O
   30-May-25 — Handling expansion and GPIO overload.

8. Getting the Data and Controlling the Devices from the REST API
   01-Jun-25 — Talking to your Pi from across the network.

9. Writing Your Own Device Plug-ins
   03-Jun-25 — Custom code to extend your system’s reach.

10. Using Software Devices to Build on the Hardware
    06-Jun-25 — Simulated devices and logic layers.

11. How to Make Things Happen Automatically
    07-Jun-25 — Trigger-based actions and logic chains.

12. Time Has Come Today
    12-Jun-25 — Scheduled tasks and automation routines.

13. There Are Always Conditions to Evaluate
    13-Jun-25 — Conditional logic for smarter systems.

14. Rolling My Own I²C and 1-Wire Interface Card
    21-Jun-25 — A custom hardware solution for real-world reliability.

15. Recording and Accessing Environmental Data Remotely
    01-Jul-25 — Logging and remote access to your system’s mind.

16. Preventing Injection Attacks to the Database
    07-Jul-25 — Keeping bad guys out of your Pi.

17. PWM: A Better Way to Control Sprinkler Valves
    21-Jul-25 — Fine-tuned voltage control for water flow.

18. Not Another Sprinkler Valve Article; TLDR
    02-Sep-25 — The elevator pitch

19. Smart Power Switch and Shutdown for Raspberry Pi
    05-Oct-25 — Battery Backup and controlled shutdown.

20. Why the Raspberry Pi GPIO API is broken and how to fix it
    11-Nov-25 — How a quiet change in Linux and libgpiod boned developers

21. Chasing Raspberry Pi Power Management
    26-Jan-26 — Lessons Learned Building a Power Switch with Deep Sleep and Wake Control

22. Tangled Up in Wires
    11-May-26 — The Latest Evolution of My Farm Irrigation System

23. Garden Irrigation Evolution
    09-Jul-26 — Stuff’s getting better. Stuff’s getting better every day.

24. Reading DS18B20 Temperature Sensors on the Pi Without OWFS
    11-Jul-26 — Talking to the DS2482-100 I²C-to-1-Wire bridge directly

25. I’m a Pushover for Self Hosting
    12-Jul-26 — Adding remote notifications without Apple’s infrastructure.

26. A Dashboard for My Garden Irrigation
    17-Jul-26 — A Web-Based Operations Console for the Garden Irrigation System

27. Adding MQTT to pIoTServer
    29-Jul-26 — Connecting Zigbee devices to my self-hosted internet of things ecosystem.

28. Self-Hosting Without the Exposure
    01-Aug-26 — How to access your self-hosted applications with Cloudflare while keeping your home network private.

29. Back It Up Jack
    09-Aug-26 — How I keep farm and home automation recoverable with backups, cold spares, and fixed network identity.

General stuff

• How did you make that?
  23-Jan-26 — What actually happens between an idea and a working circuit board

• Interview with Raspberry Pint
  30-Sep-25 - Building resilient systems that don’t need the cloud

• Using a Raspberry Pi to measure the cistern level
  14-Nov-21 — A better way to calculate water level

Weather Station

• Building a Dedicated Weather Station Display
  18-Dec-25 — Built a rock-solid, minimalist Raspberry Pi kiosk weather display

• Build Your Own Off-Grid Weather Station
  11-Dec-25 — A Raspberry Pi and SDR system for collecting local weather data without relying on anyone’s cloud.

• Here comes the Sun
  06-Mar-26 — How My Chicken Coop Controller Calculates Sunrise and Sunset

Chicken Coop Automation

• A Better Automatic Chicken Coop Door
  05-Mar-26 — From Backyard Chickens to 100-Bird Pasture Rotation

• Chicken Coop Door, Installed and Running
  26-Mar-26 — Video, installation photos, and command-line setup from the finished coop door controller

• The Humble Actuator
  11-Oct-25 — I2C-Based Linear Actuator Driver with Current Monitor

• Chicken Coop with Raspberry Pi
  27-Jan-22 — Version 2 of the coop controller

• Chicken Coop with Arduino platform
  17-Dec-20 — My first coop controller

If you care about how things are made and why they’re built the way they are, you’ll probably like the rest of what I write about. It is not always electronics. Believe it or not, I have been known to jump from baking to perfume to motorcycles without much warning.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

I have since finished the prototyping in the shop and installed the controller in the mobile coop. What follows are a few photos of the finished installation, a short video of the door in operation, and some command-line setup and log output from the actual system.

Installing the Assembly

I built the whole assembly on a sheet of 23/32-inch AdvanTech so the aluminum guide rails would stay aligned. Once it was dialed in, mounting it to the coop wall was just a matter of driving a bunch of screws, and cutting a hole for the door.

One other small detail: I used a short piece of chain between the actuator and the door, basically like a toilet flapper chain. That gives the linkage a little slack, which helps keep the door from driving hard against anything underneath it, including an unlucky chicken.

I also built a separate battery box for the power system. It houses the 12-volt battery, a solar charge controller, fuse protection, and the rest of the power wiring in one place. I mounted it outside the coop to make it easier to service and to avoid kneeling around in chicken droppings every time I needed to check or adjust something.

For the charge controller, I used a Renogy Voyager. It is a small PWM unit with a simple display and basic charge management. It is a bit of overkill for this job. It is reliable, straightforward to wire, and stupid easy to set up.

The smaller Wanderer would probably have been a better fit, but I happened to have a few Voyagers sitting around the shop, so one of them went into the battery box.

For the panel, I used an ACOPOWER 30-watt monocrystalline solar panel. The load is low enough that 30 watts is more than plenty for this system.

I also mounted the coop door button and LED combo outside in a weatherproof box with a latching cover. That way, if some Wile E. Coyote type comes snooping around after hours, he has to solve at least one extra puzzle before getting access to the chicken banquet. I was prepared to go further with the predator defense plan, but I needed my anvil.

The Door in Operation

Here is a short video of the coop door in operation after installation. Note the locking solenoid engaging after the door closes.

Programming the Coop

In the earlier article, I mentioned that I added a socket for an optional SparkFun DEV-15096 USB-to-serial module. That gives the controller a simple setup console without needing a display, keypad, or any other nonsense bolted onto the box.

To configure it, I first made sure the CONFIG/RUN switch was set to CONFIG and hit reset. Then I plugged a USB cable from the module into my laptop and connected with:

screen /dev/ttyUSB0 38400

That dropped me straight into the console. On first reset, the controller came up with default settings and complained, correctly, that the time and configuration had not been saved yet:

Chicken Coop Controller 2.9.0
WARNING: CONFIG INVALID, USING DEFAULTS
TIME: NOT SET
Use: set date YYYY-MM-DD
     set time HH:MM:SS AM|PM

The first thing I did was set the clock. The easy way is to set the date first, then type in the next minute mark while watching a real clock and hit Return about a second before it rolls over.

> set date 2026-03-26
OK
> set time 16:20:50
OK

Then I verified that the RTC had taken the new time and was reporting sane values. In this case, it showed the clock as valid and confirmed that the time had been set just a few seconds earlier:

> rtc
RTC      : VALID
utc      : 2026-03-26 21:21:12
local    : 2026-03-26 04:21:12 PM
epoch    : 1774560072
since_set: 22 sec (0 days 00:00:22)

After that, I set the rest of the configuration, including latitude, longitude, time zone, DST handling, and the door and lock timing values, then saved it to EEPROM:

> save
OK
>  config
CONFIG (SAVED)

lat  : 34.4653
lon  : -93.3628
tz   : -6
dst  : ON (US rules)
rtc_set_epoch : 1774560050
door_travel_ms : 10000
door_settle_ms : 2000
lock_pulse_ms  : 500
lock_settle_ms : 500

A quick way to verify that the location, time zone, and DST settings are correct is to check the calculated sunrise and sunset times:

> solar
           Rise        Set
Actual     07:09 AM    07:30 PM
Civil      06:44 AM    07:55 PM

If you forget the commands, you can always prompt the system for help:

> help
Commands:
  help      Show help
  version   Show firmware version
  time      Show current date/time
  schedule  Show schedule
  solar     Show sunrise/sunset times
  set       Configure settings
  config    Show configuration
  save      Commit settings
  device    Show or set device state
  door      Manually control door
  lock      Manually control lock
  event     Event commands
  led       Control door LED
  rtc       Show raw RTC state
  sleep     Sleep til next scheduled event   

Build Your Own

Like all of my projects, this one is already up on GitHub, including the firmware, schematics, and PCB layout.

The firmware is released under the MIT License, and the hardware design files are published under the CERN-OHL-P v2 open hardware license, so anyone can study it, modify it, and build their own.

If you build one, drop a photo in the comments. I’d enjoy seeing how it ends up in somebody else’s coop.

If you care about how things are made and why they’re built the way they are, you’ll probably like the rest of what I write about. It is not always electronics. Believe it or not, I have been known to jump from baking to perfume to motorcycles without much warning.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

If you got a $10 head, buy a $10 helmet.

One of the things about Arkansas is the lax… or rather, lack of helmet laws, and the people who insist on riding without them. I guess that’s their call. Me, I like my head, and I’ve done enough time in EMS to know that even a minor crash without a full-face helmet usually doesn’t end well.

But it’s more than just a safety thing. It’s about comfort, too. Even a speck of dirt, or in my case a bird, at 60+ mph will get your attention. Not to mention the hearing loss from wind noise.

This time around, I wanted a full-face helmet with an integrated sun visor, like you’d find on a fighter pilot’s helmet. I got tired of screwing around with sunglasses, especially riding into a low-angle sunset. Tinted visors are fine until it gets dark, which is exactly when you don’t want one on a country road.

That narrowed things down to just a handful of options. Only a few manufacturers make them that I trust to protect my brain housing group. It’s a short list.

I ended up with the Shoei GT-Air 3. You get what you pay for with helmets. I’ve always leaned toward Shoei. They put serious effort into engineering their helmets, and I’ve seen firsthand how well they perform on the track.

Shoei got a lot of things right. Ventilation and the chinstrap are both done properly, and the helmet feels solid. At about 3.8 pounds, it doesn’t feel heavy at all. I also like that their website offers plenty of accessories and replacement parts, so you can set the helmet up the way you want.

Pro-Tip: You can spend a lot more on custom paint schemes. I’m not trying to look like I belong at Laguna Seca. I bought the basic model.

One thing I didn’t expect, or even know about, was the Pinlock® system. It’s a secondary inner lens that mounts inside the face shield and creates a sealed air gap, which keeps it from fogging up. If you’ve ever ridden in cold or wet conditions, you know how fast a visor can turn useless. This one doesn’t.

I’ve seen similar setups on snowmobile helmets for years, but never on motorcycles.

The helmet came with the insert. Easy-peasy to install, and once it’s in, it just works. No sprays, no wiping with your glove, no screwing around.

And that’s exactly the kind of thing that makes this helmet not suck.

Back Road Equipment Slider Rear Rack

One of the problems with riding motorcycles is that you need a way to carry stuff. In spite of what they say about the KLR650, “that you can fix most things with a rock,” you still need to carry tools, not to mention first aid gear and maybe a rain suit.

The stock rear plate was absolutely useless for attaching a bungee cargo net, and I wasn’t about to spend a lot on Kawasaki’s side cases. By the time you’re done, you’re into it for real money. So I looked for something simpler.

Luckily, the folks at Back Road Equipment had a much better solution. They make a slider rear rack system that solves the problem without overcomplicating it. It’s a pretty cool design, if you ask me. You can secure a Pelican case to it, lock it in place, and still remove it easily when you need to.

The best part is that it costs just a little over $150, and you don’t have to dish out big bucks for a Pelican box. A $40 Harbor Freight clone works just fine, and you can always step up to a larger one if you need to.

One more thing about the Harbor Freight case. I don’t know who thought it was a good idea to plaster it with that many stickers and labels (Dante probably had a special circle set aside just for them), but I spent over an hour with solvents getting them off.

Installation is straightforward. Drill four holes in the case, install the slider pucks, and don’t forget a little silicone to keep it watertight.

You might find that after removing the Kawasaki stock rear plate, a set of alignment drifts comes in handy for aligning the slider plate.

One more thing before I move on: Back Road Equipment’s customer service was great. The owner called me, helped straighten out my order, and shared a few installation and usage tips.

How about a Light Switch That Doesn’t Suck

When I bought the bike, I got a great deal on Kawasaki’s LED auxiliary lights. The lights themselves are reasonable (did I say I got a good deal). The stock switch on the dash, however, is an unbelievably bad user experience. They stuck it up on the cowling next to the gauges, where it’s awkward and flat-out unsafe to reach.

I don’t know what Kawasaki was thinking. This doesn’t feel like something the core engineers would sign off on. More likely some regulatory requirement turned it into a piss-poor design.

So I did what I always do. I got out my tools and modified it to make it work right.

A much better place for the switch is right next to the light and horn controls on the left handlebar, where your thumb can hit it without taking your hand off the grip.

After a few minutes of digging around online, I found exactly what I needed, a simple handlebar-mounted switch. It was even illuminated.

This would have been a five-minute job with some hand tools and a soldering iron, if it weren’t for the wonky way Kawasaki mounted their switch. I had to remove a good chunk of the cowling just to get to the screws they hid behind the switch assembly. But I persevered, and after a few knuckle scrapes, Kawi’s design was just a faded memory.

Make It Your Own

Half the fun of owning a motorcycle is making it your own and setting it up the way you like. That’s how you end up with something that’s actually yours.

Mine just happens to have a pissed-off bunny sticker on it. But that’s me.

As I say in each of these pieces, there are no sponsorships here. Don’t ask for one. No affiliate links. No favors. I’m pretty sure most of these companies have never heard of me, and that’s fine. I’m writing about these parts because they earned their place in my shop by working, not because anyone asked or paid for it.

If you like building things, and sometime doing a better job than the manufacturer, you’ll probably like the rest of what I write. Sharing it helps real people find the work. The algorithm can go take a hike.

I had a guest visit the other day, and I asked what their preference for dessert might be. They suggested strawberry rhubarb pie. Unfortunately, rhubarb was out of season, so that idea was off the table.

Instead, I went with what I had on hand and put together a Fresh Strawberry Custard Pie. It turned out to be a real hit and didn’t leave much in the way of leftovers.

The combination of a simple butter crust, vanilla custard, and fresh strawberries worked beautifully. Most of my recipes are improvisation anyhow.

I’m sharing the recipe here partly because friends asked for it, and partly so I have it written down somewhere sensible. I’ve learned that if I don’t write it down, it’s lost forever.

So this is both a recipe and a small act of documentation for future me.

Pie Crust Ingredients

This is a derivative of the crust I used in my Apple Pie project.

One thing I did change this time was the flour. For this batch I used flour made from Azure Market Organic Soft White Wheat Berries.

We picked up a grain mill from Amazon a while back and started grinding and sifting our own flour. Fresh-milled flour behaves and tastes a little different than store-bought flour. It has a slightly nutty flavor and a bit more character. After grinding, we sift the flour to remove some of the coarser bran so the crust stays light rather than heavy.

You can substitute Bob’s Red Mill Organic Unbleached White All Purpose Flour if you don’t want to go through the process of grinding and sifting your own flour.

• 2 1/2 cups, organic ground white flour

• 1 cup, Rumiano organic unsalted butter

• 1/2 teaspoon, Sea Salt

• 1 tablespoon, Granulated sugar

• 5–6 tablespoons ice water

Cut the butter into small ⅜" cubes using a pastry cutter. Combine the flour, sugar, salt, and butter in a bowl and work it together with the pastry cutter until the butter is broken into small pieces.

Avoid the food processor if you can. It works, but using a pastry cutter is just as effective and a lot easier to clean up afterward.

Add the ice water a tablespoon at a time, mixing gently with a fork or pastry cutter.

What you’re aiming for is a coarse, crumbly mixture with little pea-sized bits of butter coated in flour. That texture is what produces a tender crust.

If the dough starts looking smooth or pasty, you’ve gone too far and overworked it. The goal is for the dough to just come together, not turn into a uniform paste.

Occasionally I’ll put the mixture in the refrigerator or freezer for a few minutes to keep the butter cold before continuing. Keeping the butter firm helps the crust bake up tender and flaky.

When you are ready, put some flour down on your work surface and roll the dough out just thick enough to line the pie tin. At that point I put the prepared pie tin into a large Ziploc bag and placed it in the refrigerator until I finished preparing the custard.

Blind Bake the Crust

The crust was blind baked. I lined it with parchment and filled it with “pie weights,” in my case a jar of old black beans. The point is to weigh the dough down so it doesn’t bubble or shrink while the fat melts and the structure sets.

1. Roll the dough and line the pie tin with the crust.

2. Cover with parchment + pie weights or beans

3. Preheat to 350°F

4. Bake 20 minutes at 350°F

5. Remove the weights and parchment

6. Bake again uncovered 10–15 minutes more

7. optional - beat one egg white and brush a thin layer on the hot crust

8. Put it back in the oven 2 minutes

9. Let the shell cool 20–30 minutes before adding custard.

Prepare the custard

For the custard I used the same formula from my Fruit Tart project.

• 3 tablespoons flour

• 2 tablespoons arrowroot

• ½ cup granulated sugar

• ¼ teaspoon sea salt

• 2 eggs (fresh from my chickens)

• 2 cups pasteurized milk

• 1 tablespoon unsalted butter

• 1 vanilla bean, split and scraped

• Zest of one lemon and one orange, with a little juice from each

• A small amount of gelatin, bloomed and added later for extra set

1. In a medium bowl I whisked together the flour, arrowroot, sugar, salt, and eggs until it looked smooth and lump-free.

2. In a saucepan I heated the milk and butter with the lemon and orange zest and the scraped-out vanilla bean (pod and all) just until it was steaming but not boiling.

3. Temper the eggs. I slowly drizzled a ladle of the hot milk into the egg mixture while whisking like mad. The goal is to bring the eggs up to temp gently so they don’t scramble later.

4. Then I poured the warmed egg mix back into the saucepan with the rest of the hot milk, still whisking. This is also a good time to fish out the vanilla pod.

5. I cooked it over medium heat, stirring constantly, until it thickened enough to coat the back of a spoon, about 170 to 175 °F if you check. Don’t let it boil hard or you’ll destroy it.

6. Turn off the heat.

7. For extra insurance, I bloomed a little gelatin in cold water, then stirred it into the hot custard until fully melted.

8. I scooped the custard into a glass bowl and pressed plastic wrap directly onto the surface to keep a skin from forming. Let it cool on the counter for a while, then place it in the refrigerator for 1–2 hours until it is cool to the touch and fully set, roughly below 70 °F, before filling the pie.

Assemble the Pie

Once the custard is fully chilled and the pie shell has cooled, it’s time to assemble the pie. Spoon the custard into the baked pie shell and spread it evenly with a spatula. The filling should be thick enough to hold its shape but still spread smoothly.

Level the surface and return the pie to the refrigerator for about 20–30 minutes to firm up slightly. This helps keep the strawberries from sinking when you arrange them.

Once the custard is set, the pie is ready to decorate with fresh strawberries. I sliced the tops off, cut the berries vertically, and arranged them in circular layers, working from the outside toward the center.

That’s all there is to it. A simple crust, a straightforward custard, and a pile of fresh strawberries arranged on top.

It started as a substitute for strawberry rhubarb pie, but it turned out to be a pretty good dessert on its own. Sometimes cooking works that way. You aim for one thing, and circumstance pushes you somewhere else.

The result was good enough that it’s worth writing down so I can make it again without reinventing it. And if someone else happens to try it and enjoy it too, all the better.

If you care about how things are made and why they’re built the way they are, you’ll probably like the rest of what I write about. Sometimes I even write about things that are yummy.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

For most of human history those smells were simply part of being alive. They whispered things about health, attraction, and who you were. Long before science began talking about pheromones, people were already responding to those signals without realizing it.

Perfumers understood this long before they understood the chemistry. Classic perfumes were built to cooperate with the body rather than fight it. Ingredients like musk, civet, leather, and oakmoss produced warm, slightly animalic notes that echoed the chemistry of human skin. They mingled with the wearer’s natural scent and deepened it. A good perfume didn’t erase the body. It amplified it.

And for a long time that was how perfume worked.

But somewhere along the way we got lost. Now many modern perfumes try to bury the body beneath the smell of clean laundry and soap.

So how did we get so far off track?

The Biology of Human Scent

The way people smell to us begins in chemistry and ends in emotion.

Smell is unusual among the senses because it connects directly to some of the brain’s oldest structures, such as the amygdala and hippocampus, which shape emotion, memory, and desire. Signals from the nose reach these areas without first passing through the slower analytical parts of the brain.

In that sense, smell bypasses the careful rational mind and lands somewhere deeper. That is why a scent can trigger a memory instantly, or make someone seem suddenly familiar, intriguing, or irresistible.

A big part of that scent comes from chemicals our bodies make from sex hormones. Those steroid molecules seep out through sweat and skin oils, and bacteria on our skin break them down into smaller volatile compounds that drift into the air around us.

Our skin is constantly releasing other molecules as well, derived from fatty acids and normal metabolism. As they reach the surface, bacteria transform many of them into airborne compounds that give each person a scent that is subtly unique.

Some of the most interesting of these molecules originate from androgen hormones such as testosterone, tied to the chemistry of attraction and social recognition. The human nose is remarkably sensitive to them.

A smell can trigger a vivid memory, a feeling of comfort, or an instinctive sense that someone is appealing or familiar before we have time to explain why.

And this is why scent can sometimes cause emotion to override rational judgment.

Do you know what I mean.

The Man Who Took Perfume Apart

Years ago, while I was digging into perfume, I came across one of the most interesting books ever written about smell: The Emperor of Scent. Reading it sent me down a rabbit hole that eventually became the impetus for this article.

It tells the story of Luca Turin, a biophysicist who wandered into the perfume world and started asking questions that nobody there seemed comfortable answering.

For decades the official scientific explanation of smell had barely changed. The prevailing theory claimed that odor came from the shape of molecules fitting into receptors in the nose like a lock and key. As the saying goes, the science was settled.

Turin wasn’t convinced.

He proposed something very different. Instead of shape alone, he argued that smell might depend on the way molecules vibrate. If that were true, scent would no longer be something discovered slowly through intuition and experimentation. It could be understood, predicted, and deliberately constructed.

The reaction from both the academic world and the perfume industry was closer to panic than curiosity. Academics had built careers on the lock-and-key theory, while perfumers guarded their formulas in private notebooks and trade secrets, passed down through apprenticeships and quiet tradition.

At one point Turin’s work was submitted to Science, supposedly one of the most prestigious journals in the world. What followed was a demonstration of how peer review can behave when a theory threatens the status quo.

There was no open debate, no careful testing of the evidence. The paper was handed to reviewers who treated the idea as something to eliminate. It wasn’t argued against. It was quietly buried.

But if Turin was right, his theory had the potential to disrupt the entire industry. What had once been the guarded knowledge of a few master perfumers might now, in an age of ubiquitous AI and modern chemistry, be analyzed, reconstructed, and reproduced by a determined experimenter working in a small lab.

Imagine a device that could read molecular vibrations and display them the way a modern SDR program reveals the radio spectrum, turning invisible signals into a visible pattern of frequencies.

Once scent could be seen that way, recreating it would no longer be guesswork. It would just be engineering.

When Perfume Was Intoxicating (1920s–1960s)

Let’s step back to the early twentieth century, when perfume was something entirely different.

Air conditioning was rare. Washing machines were still spreading into homes. Most people did real work and walked long distances every day. Cities smelled stronger. Streetcars rattled past open windows. People smelled stronger too. Bars, dance halls, and crowded restaurants filled the air with cigarette smoke, wool coats, leather jackets, perfume, and sweat.

Music was louder and the rooms were packed. Big band and early jazz poured out of dance halls and bars where people crowded shoulder to shoulder. The energy there came from the same place as the music itself: people working hard, living independently, and not worrying about sanding life’s rough edges. As it was once explained to me, in hard years, especially during wartime, people gathered in places like that to forget their troubles for a few hours and feel alive again.

That older world was also less afraid of the physical realities of life. The human body was simply part of the landscape, not something that had to be constantly managed or sanitized away. The smells of work, movement, and proximity were ordinary parts of daily life.

In that world, perfumes were not meant to hide the body but to mingle with it and amplify it’s natural smells.

One of the most famous perfumes of that era was Chanel No. 5. When it appeared in 1921 it was unlike anything people had smelled before. Bright aldehydes gave it a sparkling, almost champagne-like opening, but underneath was a deep base of musks, woods, and animalic notes that bloomed as the perfume settled into the warmth of the body. What began polished and elegant slowly turned warm and quite intoxicating.

Then there was Bandit. If Chanel No. 5 was elegance, Bandit was the woman in the red dress you couldn’t ignore. It opened sharp and green with galbanum, then settled into leather, oakmoss, patchouli, and vetiver. One spray could fill a room. On skin it became darker and more sensual, the kind of perfume that seemed to grow warmer as the night went on.

Then there was Bay Rum. For generations barbers splashed it on after a shave, a spicy blend of bay leaf, rum, and warm Caribbean spices. It opened sharp and bracing, then settled into something deeper as it warmed. Imagine it for a moment. It was a scent that seemed to amplify the warm chemistry of skin, sweat, and testosterone.

Note: Many of these fragrances have been reformulated over the years, and modern versions smell quite different from the originals people knew in those earlier decades. From time to time original bottles still surface in vintage fragrance markets and online auctions.

Even as the counterculture of the 1960s reshaped music, fashion, and social life, perfume still played by those same rules. It was meant to meet the body, not erase it.

When the Dance Floor Was Hot (1970s)

By the 1970s the music had changed, but the atmosphere hadn’t. The big bands were gone, replaced by funk, soul, and disco, but people still gathered in the same way: crowded rooms, packed dance floors, bodies moving together in the heat.

Disco clubs were loud, dark, and alive. Lights flashed across mirrored walls and spinning balls overhead. The air was thick with cigarette smoke, sweat, and perfume.

One of the defining scents of the era was Halston, a deep chypre built around oakmoss, patchouli, amber, and woods. Warm, rich, and unmistakable, it had the kind of presence that could cut through the smoke and heat of a crowded club.

Then there was Opium. Dark, spicy, and full of resins and woods, it carried enormous weight. A single spray lingered in the air and grew warmer as the night went on.

For men, Paco Rabanne Pour Homme captured the same spirit. Built on a classic fougère structure of lavender, oakmoss, herbs, and woods, it was bold, masculine, and impossible to miss. Like the other fragrances of the era, once it warmed up it carried easily through crowded rooms.

When Perfume Got Loud (1980s)

By the 1980s, disco had faded, replaced by drum machines, synthesizers, and a harder electronic sound that pulsed through the clubs.

The atmosphere had changed too. Leather jackets and pants, aerosol hair spray, cigarette smoke, and sweat thickened the air. The rooms were still hot and crowded, but the style was bigger, bolder, and more theatrical.

In a decade that celebrated ambition and spectacle, fragrances grew louder and more dramatic. They no longer lingered quietly on the body. They filled the room.

One of the defining perfumes of the era was Poison. It was unmistakable. Dense and dramatic, it opened with a rich sweetness before unfolding into dark spices, resins, and woods. In a crowded bar it could announce itself long before the woman wearing it entered the room.

Then there was Giorgio Beverly Hills. It was also impossible to ignore, but in a different way. Built around huge white florals like tuberose and jasmine layered over woods and musk, it was bright, bold, and incredibly strong. Department stores had to move the display near the entrance because the scent filled the entire floor. What began as a sharp burst of flowers slowly settled into a warm musky sweetness that lingered long after the wearer had gone.

Then there was Drakkar Noir. For many men in the 1980s it was the scent of the night. Built on a dark fougère structure of lavender, oakmoss, herbs, woods, and leather, it was sharp, masculine, and unmistakable. It projected strongly at first, then deepened into something darker and slightly smoky as the hours passed.

Giorgio Beverly Hills was the blonde in the spotlight, bright, glamorous, and impossible to ignore. Poison was that mysterious, dangerous dark-haired woman you kept noticing across the room. And Drakkar Noir was the guy leaning on the bar who simply owned the place.

We Lost the Scent (1990s)

Then sometime in the 1990s something strange happened. Perfume stopped smelling like people and started smelling like laundry soap.

Instead of blending with the scent of the body, fragrances began trying to erase it. In many cases they were chemically indistinguishable from the fragrances used in detergents and fabric softeners.

The culture around them had changed too. The sweaty glamour of disco and the loud excess of the 1980s gave way to a cooler, more restrained style shaped by grunge and early hip-hop. The new look was neutral, understated, and often deliberately androgynous.

CK One (Calvin Klein) became one of the defining perfumes of the 1990s. Light, bright, and deliberately clean, it rejected the old idea that perfume should warm and deepen on skin. Rather it smelled like soap.

CK One was marketed as unisex, meant for men and women alike. The bottle looked almost clinical, like a laboratory flask, and the advertising followed the same idea. Black-and-white photographs of young people in plain jeans and T-shirts replaced the glamorous perfume ads of earlier decades.

The goal was no longer warmth or sensuality but cleanliness. The culture stopped celebrating the body and started sanitizing it.

The Detergent Economy

At the same time something else was happening behind the scenes. The same fragrance companies that designed luxury perfumes were also designing scents for detergents, shampoos, deodorants, and household cleaners.

Those products were produced in enormous volumes, far larger than the perfume market. Their fragrances had to be inexpensive, chemically stable, and able to survive heat, water, and months on a store shelf.

Traditional perfumes relied on complex natural materials such as oakmoss, animalic musks, resins, and woods that created deep, warm bases designed to interact with the chemistry of skin.

Industrial products needed something different. The fragrance industry began building scents from a smaller palette of synthetic musks that were cheap, durable, and easy to reproduce.

One of the molecules that quietly took over was Galaxolide. It had the soft, smooth smell many people associate with freshly washed fabric. Chemically it is a large, bulky hydrocarbon ring packed with methyl groups, a structure that makes the molecule unusually stable, resistant to oxidation, and able to survive heat, water, and long storage without breaking down.

That made it perfect for detergents and fabric softeners, surviving the washing machine and clinging to clothing while leaving behind the lingering scent advertisers called “clean,” though it was really just the smell of the chemicals themselves.

Over time the industry effectively brainwashed people into associating that chemical scent with cleanliness. Clothes washed in detergents smelled that way, and towels and even homes cleaned with modern products carried the same scent. After years of repetition people began to accept that chemical smell as the very definition of clean.

Eventually the same molecules began appearing in perfumes, and the smell of laundry moved from the washing machine into the fragrance bottle.

Side note: It can be argued that regulation also played a part in pushing the industry away from many traditional perfume materials. European regulators began restricting ingredients such as oakmoss after reports of allergic reactions in patch tests. Critics argued that many of those tests focused on people who already had skin sensitivities, producing numbers that looked far more alarming than what most people experienced in everyday life.

Personally, I suspect many more people react to the modern chemistry that replaced those materials. Synthetic musks such as Galaxolide, along with additives derived from corn and other industrial feedstocks, appear in a wide range of modern cleaning products. They are everywhere, in detergents, household cleaners, and personal care products, washing down our drains and circulating through wastewater systems, rivers, and even parts of the water supply.

The Rebellion

Not everyone accepted the new world of soap-clean fragrance. A handful of modern perfumers began pushing back, bringing back the musks, woods, and animalic notes that older perfumes once wore proudly.

One of my favorites is the French house État Libre d’Orange, which deliberately builds fragrances around the very smells modern perfumery tried to erase.

One example is Fat Electrician. Despite the absurd name, it is built around vetiver, woods, and creamy resins that come alive the way older perfumes once did. It smells human and slightly worn in, not freshly laundered.

Then there is Sécrétions Magnifiques. That one goes much further. It deliberately references the metallic, salty, bodily notes associated with skin, sweat, and intimacy. The reviews are legendary. Some people love it. Others say it is revolting. It was designed to shock people, but also to remind them of something modern perfume had tried to forget. The human body has a smell.

The Smell of Being Human

Perfume once amplified the scent of a living body. It warmed on skin, mixed with breath and sweat, and slowly became something uniquely human.

Somewhere along the way modern culture decided to replace that complexity with the smell of soap.

Scent is one of the oldest languages we have. Long before words, it told us who was near, who was healthy, who we were drawn to. A scent can make someone seem warm, mysterious, familiar, or irresistible before a single word is spoken. It binds memory, attraction, and emotion together in ways we barely understand.

When perfume forgets the body and starts smelling like laundry, it isn’t just a change in style. It is a retreat from the chemistry that once made scent feel alive.

In the name of cleanliness we traded warmth for sterility and passion for safety.

But without passion, emotion, and a little danger, are we even alive?

In a world increasingly mediated by artificial intelligence, simulated images, and conversations held through glass screens, we are drifting away from the concept of physical presence. We forget what it means to stand close to another person, to flirt, to share the invisible signals our bodies constantly send.

Perfume once spoke that language.

If you care about how things are made and why they’re built the way they are, you’ll probably like the rest of what I write about. Sometimes I even write about why things stink.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

User: Is this mushroom edible?

LLM: Yes, that mushroom is edible.

User: I just ate it and now I’m vomiting.

LLM: Thank you for pointing that out. That mushroom is actually poisonous. Would you like to know more about edible mushrooms?

In a recent article about building a better chicken coop door, I mentioned that I tried to use Grok and GPT for component recommendations. Both of them gave answers that sounded confident, detailed, and completely reasonable. And both of them were also dead wrong.

That experience made me curious. Large language models (LLMs), often lumped together under the label “AI,” often produce explanations that read as if they were written by someone who knows exactly what they’re talking about. Yet when you dig into the details, the recommendations can turn out to be flawed, misleading, or sometimes entirely fabricated.

To understand why that happens, it helps to step back and look at what these systems actually are, and just as importantly, what they are not.

To start with, despite all the hype, these systems are not intelligent. What they are is extremely sophisticated statistical pattern matchers. Trained on enormous amounts of human text, they learn the statistical relationships between words, phrases, and ideas.

When you ask a question, they are not reasoning through the problem or looking up a verified answer. Rather, they produce a response that statistically resembles the kinds of answers humans have written before.

The result might feel insightful and authoritative. But here’s the key point:

It doesn’t know the answer.

It knows what answers usually look like.

But sounding correct and being correct are two very different things.

Let that sink in for a minute.

I could give a few examples, but you probably already have plenty of your own.

LLMs: The Internet’s Bullshit Generator

That phrase may sound a bit harsh, but it’s actually quite accurate, especially in light of what philosopher Harry Frankfurt wrote in his essay On Bullshit. (I’m not making this up.) Frankfurt draws an important distinction between lying and bullshit. A liar knows the truth and deliberately tries to conceal it. A bullshitter is different. The bullshitter simply doesn’t care whether something is true or false. The goal is not accuracy, but persuasion. The only goal is to produce something that sounds convincing.

That description turns out to fit large language models surprisingly well.

By design, an LLM isn’t trying to deceive you. It isn’t trying to tell the truth either. In fact, it has no concept of truth at all. Its only objective is to generate text that statistically resembles the kinds of answers humans tend to produce.

If the response sounds plausible, structured, and authoritative, then from the model’s perspective it has done its job.

Which is why if you train a system to produce text that sounds right, you should not be surprised when it sometimes produces answers that sound perfectly reasonable but are completely wrong.

It’s all about the training

Training a large language model can be broken down into a few steps:

Step 1. Collect a huge amounts of text

The first step in training a large language model is gathering an enormous amount of written material. This includes books, articles, documentation, forums, source code, and large portions of the public internet.

The goal is not to teach the system facts. The goal is simply to expose it to as many examples of human writing as possible. The more text the system sees, the more patterns it can learn about how language works and how people tend to explain things.

Some of that writing is careful and well thought out. Some of it is sloppy, biased, or simply wrong. The LLM doesn’t know the difference between the Good, the Bad and the Ugly. It doesn’t care; it just learns the patterns in whatever you feed it.

This is a key point. The LLM model is basically a statistical mirror of the text it was trained on. If the internet contains good explanations, bad explanations, arguments, opinions, and outright nonsense, the model will learn patterns from all of it.

Step 2. Break the text into tokens

Before the model can learn from the text, the words have to be converted into a form the computer can process. So the text is broken into small pieces called tokens. A token might be a whole word, part of a word, or a piece of punctuation.

Take this sentence:

“The chicken coop door uses a linear actuator.”

This is broken into tokens that represent words and punctuation. Each token is then converted into a number that the model can work with.

Step 3. Learn to predict the next word

Once everything is turned into numbers, the system can start learning patterns in how those tokens tend to appear together.

The model is shown a sequence of tokens and asked to predict what token should come next. It does this over and over again across massive amounts of text.

or example, the model might see something like:

“The chicken coop door uses a 12 volt ...”

Based on patterns it has learned, it tries to guess the next word. It might predict something like:

“motor”

If the guess is wrong, the system adjusts its internal parameters slightly and tries again on the next example. This process repeats billions or even trillions of times.

Over time the model becomes very good at predicting what words tend to follow other words. That’s how it learns grammar, sentence structure, and the patterns of how people explain things.

The LLM isn’t learning facts, it’s learning patterns in how words and ideas tend to appear together.

Step 4. Fine tuning

The model then goes through a smaller round of additional training called fine-tuning. At this stage, the system is trained on carefully selected high-quality prompt-and-response pairs written or reviewed by humans.

These examples are chosen because the raw internet text is often messy, inconsistent, or not designed for helpful conversation. Human-curated pairs are essential to teach the model the exact tone, structure, safety rules, and engaging style we want. Something random web pages simply don’t provide.

For example, instead of learning from a rambling forum post, the model might see this clean training pair:

Q: What is gravity?
A: It’s a force that pulls things together. More precisely, gravity is the invisible attraction between any two objects that have mass. On Earth, this is what makes things fall down toward the center of the planet and keeps us from floating away into space. Want me to explain how Newton discovered it or how it works in outer space?

The goal is to steer it toward clearer, more helpful responses that are easier for people to interact with. This step doesn’t change how the model works. It still predicts the next token based on patterns it learned earlier. Fine-tuning simply nudges the model toward behaving more like a helpful assistant and less like a raw text generator.

Step 5. Reinforcement Learning from Human Feedback

The final step in training is called Reinforcement Learning from Human Feedback, or RLHF. At this stage, humans review answers generated by the model and rank them. For example, the model might produce two different responses to the same question. A reviewer is asked to choose which answer is better.

Note that the reviewers are not judging whether the answer is technically correct. Instead, they are following guidelines that reward responses that sound clear, helpful, polite, and well structured. This teaches the LLM to give responses that humans ranked higher. Pushing the model toward answers that read smoothly and confidently.

The LLM becomes extremely good at producing answers that feel polished and authoritative, even when the underlying explanation may be incomplete or wrong.

But Which Humans?

When people hear “human feedback,” it sounds as if the general public, or perhaps subject-matter experts, are shaping the model. In reality, much of this work is outsourced to low-cost contractors through data-labeling companies in countries such as Kenya, the Philippines, Colombia, Egypt, and India.

These workers are not engineers or scientists. They are paid raters following strict guidelines that reward answers for being clear, polite, polished, and safe, not for being factually correct or technically insightful. They get paid to feed the “hot dog” or “not hot dog.” dataset.

And there you have it: the model learns to optimize for those scoring criteria, not for accuracy, real expertise, or what the actual public actually wants.

The Transformer Paper: When Skynet Came Online

The modern wave of large language models truly begins in 2017, when a team of Google researchers published a landmark paper called “Attention Is All You Need.” That single paper introduced the Transformer architecture. The missing piece that finally made the kind of large-scale language training we just walked through actually practical.

Once the Transformer existed, everything took off. Every major LLM today, including GPT, Grok, Claude, and Llama, is built on that same Transformer foundation.

The rest of the recipe fell into place soon after. OpenAI’s “GPT-1” paper in 2018 showed that simply scaling next-word prediction on massive amounts of text could produce surprisingly powerful models. Then in 2022, InstructGPT introduced Reinforcement Learning from Human Feedback, The training step that turns a raw language model into the conversational assistants we interact with today.

Coke vs Pepsi

So what’s the real difference between Grok, GPT (OpenAI), and Claude (Anthropic)? While they all follow the same basic training recipe, the ingredients and seasoning are what make each one taste completely different.

Data Diet

• Grok: More tightly integrated with X and the live web, which gives it stronger access to current events, slang, memes, and real-time conversation.

• GPT: Mostly trained on curated books, articles, code repositories, and filtered web data. Solid and reliable, but more “library” than “live feed.”

• Claude: Uses aggressively filtered training data designed to reduce what they deem as harmful or unsafe content.

Alignment & steering (the personality factory)

• Grok: RLHF tuned for more direct answers, humor, and fewer guardrails than most systems.

• GPT: Strong RLHF focused on safety and helpfulness. Polite, polished, but quick to refuse or hedge on anything edgy.

• Claude: Uses a method called Constitutional AI, along with RLHF, which trains the model to critique and revise its own answers using a set of written principles.

Personality & vibe

• Grok: More informal and witty, with fewer guardrails in conversation. Feels like a smart friend who’ll actually answer the spicy questions.

• GPT: Professional, smooth, corporate-friendly. Great all-rounder but can sound scripted or overly careful.

• Claude: Thoughtful, ethical, often more reflective or explanatory in tone. Excellent at long-form reasoning but sometimes overly verbose or preachy.

Real-time edge

• Grok: Integrated with X and live web tools, which helps it stay current.

• GPT & Claude: Rely more on periodic model updates or optional browsing tools.

And yet, they’re still eerily similar underneath. A study from the Allen Institute (arXiv 2510.22954) proves it: even on open-ended, creative questions, all the major models : GPT, Claude, Grok, Llama, start giving almost identical answers. They literally call this the “Artificial Hivemind.”

Different flavors on the surface, same homogenized bullshit at the core.

What Is It with Those Damned Em Dashes?

If you’ve spent any time reading AI-generated text, you may have noticed something odd. LLM writing often leans heavily on em dashes.

An em dash (—) is a punctuation mark used to create a pause or break in a sentence. It is longer than a regular hyphen (-) and longer than an en dash (–).

Sometimes it feels like every other sentence has one.

And once you notice it, you can’t unsee it.

In most twentieth-century print writing, em dashes were used sparingly. Newspapers, technical publications, and academic journals stuck to commas, parentheses, or colons. Style guides like Strunk and White pushed for tight, formal sentences, and editors routinely cut anything that felt too conversational.

You could still spot them in literary essays, but they were rare in journalism or technical work. On typewriters and early word processors the character wasn’t even available, so writers just used two hyphens (--) instead.

Then the internet showed up. Blogs and personal publishing let writers ditch the editors and strict style guides. (Methinks they stopped teaching proper English in school.) The tone of online writing became more conversational, closer to how people speak rather than how formal prose traditionally looked on the printed page. The em dash became perfect for slipping in a quick aside, pivoting to a new thought, or adding emphasis without breaking the sentence.

As blogging and opinion pieces exploded, people who had never written before started influencing each other — and the em dash became contagious.

If you ask AI why it uses them so much, it will tell you it’s trying to produce text that looks like polished explanatory writing. It will claim it learned this style from sentences that read as “authoritative” or “well written.”

This is a perfect example of the George Fuechsel’s old programmer’s rule: Garbage In, Garbage Out.

Model Collapse and Poisoning

So what happens when we keep feeding these systems their own bullshit?

The result is model collapse. It hits when new AI models train on text spat out by earlier AI instead of fresh human writing. At first the damage stays subtle. Over generations the models feed on their own output. Errors compound. Simplifications harden. Stylistic quirks echo and amplify. This process is degenerative and irreversible without fresh human data.

Think of it as informational inbreeding. The models drift away from real human knowledge toward a blurry copy of their past answers. Details fade. Mistakes pile up. The language stays smooth and confident on the surface. Underneath the information thins out and turns less reliable. In short the machine keeps talking but knows less and less.

The funny thing is that it does not take much bad data to speed this rot along. A tiny amount of poisoned or low quality content can influence even very large models. A recent study, “Poisoning Attacks on LLMs,” showed that just 250 malicious documents can plant hidden backdoors in a language model, even when those documents are buried inside training datasets that contain billions of words.

Model size does not protect against this. What matters is the number of bad examples, not their percentage of the total data. When those examples come from recursive AI output, or from deliberate sabotage, the decay accelerates. A handful of flawed samples can quickly snowball into widespread degradation.

Are you just telling me what I want to hear?

This is a question I have often pondered about modern AI language models. Too often they seem overly eager to agree, flatter, or validate rather than offer balanced, honest, or corrective feedback.

Most people love hearing affirmation. Comfort feels good. But it comes at a price. Constant agreement turns the AI into an echo chamber. It reinforces bias. It kills self-reflection. It can even excuse choices that damage relationships or stunt personal growth.

Recent research suggests this concern is not just a hunch. A 2025 Stanford led study titled “Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence” found that modern language models affirm users far more often than humans do.

The effect is troubling. When people receive those agreeable responses, they become more convinced they are in the right and less willing to apologize, fix the mess, or even consider the other side’s view.

In other words, the more the AI validates them, the more of an asshole they become.

That matters because we already live in a society where basic politeness often feels like a lost art. Public discourse is fractured, and social media often amplifies those divisions. People are nudged and manipulated into arguments that escalate quickly, sometimes even spilling over into hostility or violence. In that environment, a machine that constantly tells us we are right does not calm things down. It only adds fuel to the fire.

Don’t Throw Out the Baby With the Bathwater

Every advance in technology comes with unintended consequences.

Like most tools, large language models are useful when you understand what they are good at and what they are not.

They excel at summarizing information, exploring ideas, translating concepts, generating drafts, and helping you think through problems. They sift through enormous text and surface patterns faster than any human could.

The mistake is treating them like an authority instead of a tool. An LLM is not a database of verified facts and it certainly is not an expert. It is a machine trained to produce text that resembles the kinds of answers humans tend to write.

Sometimes that works remarkably well. Sometimes it just produces bullshit that only sounds convincing.

The trick is learning the difference.

Use AI the way you would use a fast but unreliable research assistant. Let it help you explore ideas, organize thoughts, and point you toward things worth investigating. Then do what humans are supposed to be good at.

• Always check the facts yourself.

• Test the assumptions.

• Use your own judgment.

Because in the end, the responsibility for thinking still belongs to us.

For now… at least.

Quick follow-up on the article:

I intentionally kept the explanation simple. I describe LLMs as “token prediction” or “next-word guessing.” At the lowest mechanical level that’s exactly what they do. They generate text one token at a time by predicting what comes next. I wanted that core idea to land for readers who aren’t data scientists.

I should also add that the LLM is not guessing blindly. While generating each token, it looks at your prompt and the entire conversation so far. That context lets it keep a coherent thread across several sentences or paragraphs.

As models grow larger and see more training data, they begin showing what researchers call emergent behavior. They can summarize documents, follow complex prompts, and produce responses that look a lot like step-by-step reasoning. But under the hood the mechanism has not changed. It’s still predicting the next token but adding in the context of the conversation.

If you care about how things are made and and how they work under the hood, you’ll probably like the rest of what I write about. Sometimes that means digging into the gritty details of why AI spits out such BS, or how a few poisoned documents can really screw with Skynet.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.

Remember, the coop controller runs completely offline. There’s no network connection, no cloud service, and no outside API calls. The firmware computes the solar events locally using standard NOAA-style astronomical equations.

Below is the simplified sequence of calculations used by the firmware, explained in plain English with the actual equations shown and every number plugged in so someone with high school math (trigonometry and a calculator) can follow along.

Example Setup and Constants

For this example, we’ll use the following:

• Location: Little Rock, Arkansas (USA)

  • Latitude: 34.75° N

  • Longitude: 92.29° W

• Date: March 6, 2026

To compute sunrise and sunset, the firmware only needs three key pieces of information:

• Lat = Latitude (how far north you are) = 34.75°

• SolarDec = solar declination (how far north or south the sun is that day)

• HA = hour angle (how many degrees Earth must spin before sun reaches your horizon)

1. Determine the Day of the Year

Everything starts with one simple number: what day of the year is it?

For March 6, 2026:

DoY = 65

(January has 31 days + February has 28 days in 2026 + 6 days = 65.)

2. Estimate the Sun’s Position (Solar Declination)

The controller calculates how far north or south the sun is using this formula:

SolarDec ≈ −23.44° × sin((360° / 365.25) × (DoY − 81))

Plugging in DoY = 65:

SolarDec ≈ −23.44 × sin((360 / 365.25) × (65 − 81))
SolarDec ≈ −23.44 × sin(0.9856 × (−16))
SolarDec ≈ −23.44 × sin(−15.77°)

sin(−15.77°) ≈ −0.272

SolarDec ≈ −23.44 × (−0.272) ≈ +6.37°

Rounded: SolarDec ≈ +6.4°

This means the sun is slightly north of the equator (spring is coming and days are getting longer).

3. Calculate the Hour Angle

This is the most important step. The controller uses this equation:

cos(HA) = [sin(h0) − sin(Lat) × sin(SolarDec)] / [cos(Lat) × cos(SolarDec)]

Where:

• h0 = −0.833° (small correction for atmospheric refraction and the apparent size of the sun)

• Lat = 34.75°

• SolarDec = +6.37°

Plugging in the values (using a calculator for sin and cos):

Numerator:
sin(−0.833°) − sin(Lat) × sin(SolarDec)
≈ −0.0145 − (0.570 × 0.111)
≈ −0.0145 − 0.0633
≈ −0.0778

Denominator:
cos(Lat) × cos(SolarDec)
≈ 0.821 × 0.994
≈ 0.816

cos(HA) ≈ −0.0778 / 0.816 ≈ −0.095

Now take the inverse cosine:

HA = arccos(−0.095) ≈ 95.5°

4. Convert Hour Angle to Time

Earth spins 15° every hour, so we convert the angle to hours:

Hours from noon = HA / 15 ≈ 95.5 / 15 ≈ 6.37 hours

Sunrise ≈ 12:00 − 6.37 hours ≈ 5:38 AM
Sunset  ≈ 12:00 + 6.37 hours ≈ 6:22 PM

5. Apply Astronomical Corrections

The quick estimate is close, but the controller now runs the full NOAA equations to fix two real-world effects.

a. Equation of Time (how much the sun appears ‘fast’ or ‘slow’ because Earth’s orbit is not perfectly circular)

The controller calculates a fractional year angle γ:

γ = (2 × π / 365) × (DoY − 1)

For DoY = 65:

γ ≈ (6.2832 / 365) × 64 ≈ 1.101 radians

It then plugs γ into the full NOAA formula (a combination of sin and cos terms). For March 6, 2026 this gives:

eqtime ≈ −11.23 minutes

(The sun is running about 11 minutes “slow” today.)

b. Longitude correction (because Little Rock is not exactly on the center of the Central Time zone)

Little Rock longitude = 92.29° West
Central Time zone meridian = 90° West

Correction = 4 minutes per degree × (92.29 − 90) = +9.16 minutes

The controller now uses the refined hour angle HA together with these corrections in the main NOAA sunrise formula (in minutes past midnight, Universal Time):

sunrise_UT (minutes) = 720 − 4 × (longitude + HA) − eqtime

Plugging in the values (longitude = −92.29°, HA = 95.5°, eqtime = −11.23):

4 × (−92.29 + 95.5) = 4 × 3.21 = 12.84
sunrise_UT = 720 − 12.84 − (−11.23) = 720 − 12.84 + 11.23 = 718.39 minutes

718.39 minutes ≈ 11:58 UT

The calculation above uses a simplified hour angle so it’s easier to see how the longitude correction and Equation of Time affect the result.

In the actual firmware (solar.cpp), the controller runs the full NOAA solar algorithm. This produces slightly more accurate values for both solar declination and the hour angle. Because of that, the final Universal Time ends up a few minutes different from our simplified example, yielding the precise sunrise and sunset times the coop controller actually uses.

The same calculation is performed for sunset as well, with the hour angle taken on the opposite side of solar noon.

6. Convert Universal Time to Local Time

Finally, the controller simply subtracts the Central Standard Time offset:

Local time = UT − 6 hours

So the precise Universal Time sunrise (after all refinements) converts to the local clock time shown below.

Civil dawn and civil dusk are calculated using the same NOAA solar algorithm used for sunrise and sunset. The only difference is the zenith angle used in the hour-angle calculation: official sunrise and sunset use 90.833°, which represents the sun at the horizon with refraction and solar-disk correction, while civil dawn and dusk use 96.0°, corresponding to the sun being 6° below the horizon.

Everything else in the calculation remains the same, solar declination, equation of time, and longitude correction. So the result is simply an earlier morning event and a later evening event.

Final Results for March 6, 2026 (Little Rock)

After running all the math, here are the accurate times the controller uses:

Civil Dawn   : 06:07 (6:07 AM)
Sunrise      : 06:32 (6:32 AM)

Sunset       : 18:09 (6:09 PM)
Civil Dusk   : 18:34 (6:34 PM)

These match published solar tables within about one minute.

So that’s the math behind the coop door. In the original article I glossed over most of this, and afterward it felt like I’d skipped the best part. In reality it only takes a little trigonometry, a date, and the coop’s latitude to predict when the sun will appear over the horizon.

In the end, the chickens don’t know anything about solar declination or hour angles. They’re basically dinosaur pigs. They eat everything, including each other (if they could), and they’re perfectly happy as long as the door opens right on schedule every morning.

If you care about how things are made and why they’re built the way they are, you’ll probably like the rest of what I write about. Sometimes that means solar math and sometimes it’s just me pontificating about how I screwed up a design.

Hitting like and sharing helps real people find the work. The algorithm can go pound sand.