Quick Demo of ticketscli

Say we’re starting a new project called “FaceSpace” - the first thing we do is install the ticketscli APM package targeting our particular AI coding agent, in this case OpenCode:

mkdir facespace
cd facespace
git init .

apm install https://github.com/polycloudinc/ticketscli.git --dev --target opencode

This installs the following agent skills:

• tickets-init
• tickets-create
• tickets-list
• tickets-review
• tickets-execution-plan
• tickets-transition
• tickets-rank

Next we’ll initialize the tickets system for this project. This is a one-time step that sets the ticket code prefix:

npx @polycloudinc/ticketscli@latest init --code-prefix FCSP

This means our tickets will get codes FCSP001, FCPC002, etc. This command created the storage directory and s couple of files:

ls .tickets/

settings.yaml  statistics.yaml

Next we fire up the AI coding agent and create our first ticket with a prompt such as:

Create a ticket for scaffolding the backend of FaceSpace using express js

The coding agent identifies that it should use the tickets-create agent skills, and uses that skill to help it correctly create the new ticket. The agent then uses it’s model to determine the name for the ticket based on the user prompt, and uses the CLI to create the ticket:

Thought: Using ticket creation skill · 1.4s
  → Skill "tickets-create"
  + Thought: Inspecting command requirements · 953ms

I’ll create the ticket with the requested Express.js backend scaffolding scope, then verify and complete its generated template fields.

Thought · 258ms
  $ npx @polycloudinc/ticketscli@latest create -n "Scaffold FaceSpace Backend With Express.js"
  Created: /workspaces/facespace/.tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md

Thought: Reviewing ticket details · 1.9s
  → Read .tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md
  ✱ Glob "**/Ticket.md" in . (1 match)

Thought: Editing project requirements · 1.1s

The ticket was created as FCSP001; its front matter already matches the template. I’m replacing the generated placeholders with the project purpose, outcome requirements, and the explicit Express.js decision.

Next, following the skill instructions, the agent fills in the Introduction, Requirements and Technical Solution sections based on the initial prompt:

Thought: Revising patch context · 996ms

  ← Patched /workspaces/facespace/.tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md
    11   ticket_completed:
    12   ---
    13   # Introduction
    14
    15 - TODO: A one to two sentence explanation of the purpose of the ticket without getting into deep requirements and without getting into the solution.
    15 + Establish the initial backend foundation for FaceSpace so server-side features can be developed consistently. The scaffold should provide a runnable Express.js application with a clear structure for future API work.
    16
    17   # Requirements
    18
    19 - TODO: Detailed requirements captured in a way that focuses on outcomes while keeping the solution negotiable.  Note the Requirements section may be organized into subsectiona if appropriate.
    19 + - The FaceSpace backend can be installed and started through documented project commands.
    20 + - The application exposes a basic health or readiness endpoint that confirms the server is running.
    21 + - The backend has a maintainable structure for adding routes, middleware, configuration, and services.
    22 + - The scaffold includes appropriate baseline error handling and JSON request/response support.
    23 + - The setup provides a foundation that can be extended with FaceSpace domain APIs without restructuring the project.
    24
    25   # Technical Solution
    26
    23 - TODO: Capture enough detail on the technical solution for there to be no ambiguities when it comes time to implement the solution.  This may mean capturing key technical decisions, architecture outline, project structure layout, interface descriptions or other elements that define the shape of the implementation.  The Technical Solution section should never contain large blocks of code.
    27 + Use Express.js as the HTTP server framework for the FaceSpace backend. Create a minimal application entry point and separate the server startup from the Express app configuration where practical.  Organize the scaffold so routes, middleware, configuration, and service logic have clear locations, and add a simple health endpoint plus baseline JSON parsing and error handling.

Before calling the job done, the agent uses the validate CLI subcommand to check for any structural issues with the ticket:

Thought: Verifying fields and validation · 1.3s
  → Read .tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md
  $ npx @polycloudinc/ticketscli@latest validate

    Error: --all or --ticket is required
    Usage: tickets validate [--all | --ticket <code>] [options]
    Options:
      -a, --all                 Validate all tickets
      -t, --ticket <code>       Validate a single ticket by code
      -d, --tickets-dir <path>  Path to tickets directory (default: _tickets)
      -h, --help                Show this help message

Thought · 218ms
  $ npx @polycloudinc/ticketscli@latest validate --ticket FCSP001
    Validating: .tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md
    Created ticket FCSP001: .tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md

    Validation passed.

We can see the new ticket in our backlog with the list CLI subcommand:

npx @polycloudinc/ticketscli@latest list

Code     Subject                                                  Rank Status
-------- ------------------------------------------------------- ----- ------------
FCSP001  Scaffold FaceSpace Backend With Express.js                  1 backlog
-------- ------------------------------------------------------- ----- ------------
1 matching from 1 total tickets

From here we typically go into a review and prompt cycle to refine the requirements and factor our decisions into the ticket. Once those sections are to our liking we render the ticket down to an execution plan:

Define execution plan

The agent follows the tickets-execution-plan skill instructions to produce a linear execution plan that is broken into logical phases with verification steps for each phase:

Thought · 231ms
  → Skill "tickets-execution-plan"
  + Thought: Defining an editing plan · 996ms

I’ll replace the placeholder on FCSP001 with phased, checkbox-based implementation tasks and deterministic verification steps for each phase.

  → Read .tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md
  + Thought: Planning patch phases · 2.0s

← Patched /workspaces/facespace/.tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md
  27   Use Express.js as the HTTP server framework for the FaceSpace backend. Create a minimal application entry point and separate the server startup from the Express app configuration where practical.  Organize the scaffold so routes, middleware, configuration, and service logic have clear locations, and add a simple health endpoint plus baseline JSON parsing and error handling.
  28
  29   # Execution Plan
  30
  31 - TODO
  31 + ## Project Foundation
  32 +
  33 + ### Tasks
  34 + - [ ] Define the backend package metadata, dependency installation, and development/start commands.
  35 + - [ ] Create the Express application entry point and separate server startup from app configuration.
  36 + - [ ] Add environment configuration with a safe default port and documented local setup.
  37 +
  38 + ### Verification Steps
  39 + - [ ] Install dependencies and start the backend using the documented command; confirm the process listens on the configured port.
  40 +
  41 + ## API Baseline
  42 +
  43 + ### Tasks
  44 + - [ ] Add JSON request parsing and the initial route registration structure.
  45 + - [ ] Implement a health or readiness endpoint with a stable JSON response.
  46 + - [ ] Add a not-found response and centralized error-handling middleware.
  47 +
  48 + ### Verification Steps
  49 + - [ ] Run a scripted HTTP smoke test that checks the health endpoint, JSON response, unknown-route handling, and malformed-request behavior.
  50 +
  51 + ## Maintainability And Documentation
  52 +
  53 + ### Tasks
  54 + - [ ] Organize routes, middleware, configuration, and service directories for future FaceSpace APIs.
  55 + - [ ] Add or update the backend README with setup, run, endpoint, and configuration instructions.
  56 + - [ ] Add automated tests for the app bootstrap and health endpoint using the project test framework.
  57 +
  58 + ### Verification Steps
  59 + - [ ] Run the backend test suite and confirm all tests pass from a clean dependency installation.

Thought · 389ms

  $ npx @polycloudinc/ticketscli@latest validate --ticket FCSP001
  Validating: .tickets/FCSP001 - Scaffold FaceSpace Backend With Express.js.md

Defined the execution plan for FCSP001 with three phases:
  - Project Foundation
  - API Baseline
  - Maintainability And Documentation

Each phase includes implementation tasks and deterministic verification steps. Ticket validation passed.

From here we review the execution plan and make any necessary adjustments, then ask the agent to execute that plan to deliver the work. We might ask it to execute task-by-task, phase-by-phase or end-to-end depending on the level of oversight we want to give the agent. Regardless of the scope of execution the agent will use the tickets-execute skill for guidance and guardrails.

See the full docs for ticketscli ⁵on GitHub.

Atlassian

I first used Atlassian JIRA and Confluence in 2005 and worked at several shops that used those products over the ensuing years. I also operated my own instances for about a decade, however I moved away from those tools to Git-tracked Markdown by around 2022.

Move to Markdown

I started practicing and promoting the use of Markdown for software specs and design docs in the same repo as code sometime around 2014/2015 with the main driver being versioning design docs alongside the code it pertains to. I even got non-technical users in my team to follow this practice, made easier using the pretty nice WYSIWYG Markdown tool Typora which at the time was in beta and was free to use. Through this practice I was able to move all the content from the Confluence servers I managed out to Git-tracked Markdown by around 2018.

Obsidian

Around 2020 while working at AWS I was introduced to Obsidian and that opened up a whole new set of possibilities with Markdown. Through a combination of Obsidian’s built-in templates system, and the Dataview and Tasks plugins I came up with a ticketing workflow where each ticket is a Markdown file with YAML front matter for the structured metadata about the ticket such as it’s status. By around 2023 I had migrated all the tickets from my JIRA servers to Git-tracked Markdown as well. I still use that Obsidian workflow to this day for my personal projects and life management as well as for professional purposes.

Spec-Driven Development

With the advent of coding agents in early 2024, I started using those Markdown tickets as my way of doing SDD - and once we had Opus 4.5, that started working remarkably well. Not because my system is doing anything special, just because the models are so good now, and you don’t need megabytes of specs to steer them effectively.

For a while I was quite happy with nothing more than a template Ticket.md and some simple instructions in AGENTS.md. I could tell the agent something like “Create a ticket for adding database connection pooling to the FooBar microservice” and it would create the fresh ticket in the designated .tickets/ directory and from there I could iterate with the agent to flesh out the requirements, stipulate technical design decisions and eventually render a flat, linear execution plan which it would then execute for me.

Rough Edges

It worked great but over time a few small issues came up:

1. Sometimes the agent would mess up the ticket creation or other tasks, picking the wrong sequence number, using the wrong filename convention or some other little thing that I would have to steer back.
2. Sometimes I had to be overly verbose as well as repetitive in my prompts to get the agent to follow the intended workflow.
3. Once the number of tickets in .tickets/ gets beyond 20 or so it becomes difficult to understand and plan through the backlog, and to understand what’s done already.

ticketscli

To solve these issues I decided to formalize my tickets system with:

• A CLI that handles the mechanical aspects such as creating tickets and transitioning status.
• A set of agent skills that provides much crisper workflow guidance for the coding agent.

Thus was born ticketscli.

Today it’s very functional for building and curating a backlog and delivering work SDD style, and I have a backlog of improvements coming that should help it scale up beyond a solo developer scenario and handle projects that run for multiple years. I have personally been using this workflow for about two years and using the ticketscli implementation of the workflow since I first built it in roughly March 2026.

See the full docs on GitHub.

You can get the Grafana dashboard code for this dashboard from here: dashboard.json

The details of the dashboard are very specific to my environment in terms of network interfaces and hosts so it’s not a drop-in solution, but with an understanding of how it works it’s trivial to customize it to your needs.

To recap - this dashboard shows network statistics pulled from my pfSense router by Prometheus via SNMP. Actually it also shows stats from key interfaces on my UniFi 24-port managed switch similarly obtained via SNMP.

There’s a bit going on here but there’s only three distinct types of content:

• Network traffic - showing the traffic in and out from an interface - this accounts for most of the panels
• pfSense State Table - the number of tracked states and the percentage of state table memory used
• Firewall Actions - showing the number of bytes passed or blocked across all interfaces

Let’s look at each in turn and how to customize the dashboard:

Network Traffic

In my dashboard there are several network traffic panels, which are all the same but just for different interfaces:

• bkk80router WAN
• bkk80router GREEN
• bkk80router PURPLE
• etc

As a side-note - many many years ago my first homelab firewall was based on Smoothwall where interfaces are usually described as GREEN, RED, etc. I still use that terminology out of habit although I’m considering changing it to improve the readability of my firewall rules. GREEN is LAN, PURPLE is trusted WiFi, BLUE is IoT WiFi,etc.

Each network panel shows octets (8-bit bytes) in and out with in colored blue and out colored purple and both plotted from a common zero axis but with independent vertical scales - this is so that when there’s a lot of traffic in one direction, you can still see the peaks on the other. It does mean you need to pay attention to the Y-axis scales which may not be to everyone’s liking but it’s how I want to see the data:

If you want both directions to use the same scale you can move the OUT metric to the left axis.

Note that I’m still using the now-legacy Graph visualization and not the newer Time Series visualization because although Time Series supports dual (actually multiple) axes, I haven’t been able to get get it to center zero yet but I will be updating my dashboard once I figure that out.

Each network panel has two queries, one for IN and one for OUT like this:

These queries are very similar:

# IN
irate(ifInOctets{job="snmp_pfsense",instance="10.80.2.2",ifIndex="2"}[5m])

# OUT
irate(ifOutOctets{job="snmp_pfsense",instance="10.80.2.2",ifIndex="2"}[5m])*-1

For IN we’re doing an irate() of ifInOctets and for OUT of ifOutOctets. We filter on the job and instance labels to get to the right device (in my case either my pfSense box or my UniFi switch) and then ifIndex (interface index) for the specific interface we want to monitor.

Note I am using irate() not rate(). irate() works off the last 2 metrics regardless of window size and rate() works off the entire window. If you prefer you could do a rate() of a small window - say 30s - e.g.:

# IN
rate(ifInOctets{job="snmp_pfsense",instance="10.80.2.2",ifIndex="2"}[30s])

The question is how do we know what to use for ifIndex? You can find out the index of each interface by querying for ifDescr in your Prometheus console:

Here we can see that, for example:

• re0 - my WAN interface is index 1
• re1 - my LAN interface (aka GREEN) is index 2
• re1.90 - VLAN tag 90 on my LAN interface which is my IoT WiFi network is index 11
• tun_wg0 - my WireGuard site-to-site VPN is index 7
• etc

Now that you know the index, it’s easy to adapt the queries above to show the data for the interface you’re interested in.

pfSense State Table

In the early days of my adoption of pfSense I had a performance issue that seemed to be correlated to a state table with more than 2% utilization - for that reason I wanted to have this in my dashboard.

I have two stat panels - one showing the number of states and the other showing the utilization of the state table:

The count of states is obtained from one of the pfSense-specific SNMP metrics:

pfStateTableCount{job="snmp_pfsense",instance="10.80.2.2"}

While the utilization is the percentage of the total size:

pfStateTableCount{job="snmp_pfsense",instance="10.80.2.2"}/360000

360000 is hardcoded as it’s not available from a SNMP metric. I got this number from the pfSense console:

This does vary based on RAM so check your state table size and update as appropriate.

Firewall Actions

Firewall actions are shown on a two axis graph similar to the network activity graphs, with PASS traffic in blue and BLOCK traffic in purple.

The queries are:

# PASS
sum(irate(pfInterfacesIf4BytesInPass[5m]))

# BLOCK
sum(irate(pfInterfacesIf4BytesInBlock[5m]))*-1

Conclusion

This network dashboard gives me a high-level idea of what traffic volume looks like on my home network. If I see something interesting in this high-level view and want to know the details then the next step would be to take a look at the pfSense state table to see recently passed traffic, or run a packet capture in pfSense or remotely with Wireshark over SSH into the router.

Let me know if this helps you or if you notice bugs or have ideas to improve it. In the future I may do another update as I continue to enhance it for my needs.

I decided to design my stands around a mag-safe-style USB cable, and ended up grabbing three of these “elough”-brand cables:

As usual the design process started with taking a bunch of measurements of the cable and phone:

These measurements then were defined as parameters in Fusion360:

The modelling process was a bit finicky because of the 10-degree angle I needed to put on the stand bed, and the various cutouts I had to make for the cable track. The final design looked something like this:

Finally here’s the print in white PLA with 20% infill:

The tolerances on the cable channel and the socket for the magnetic plug were a bit tight on the first print, but after enlarging with a blade and a power drill I got it to friction-fit snugly into place:

Finally with the phone mounted in it:

The design came out great. The only issue is the base is very light given the small amount of plastic, so when you grab the phone the whole stand comes with it. Actually I had the same problem with my official Apple phone stand for my iPhone and just solved it with Blutac (of course) so I’ll probably do the same here.

The sources are available under the Apache License 2.0 from the GitHub project and a pre-built Docker image supporting armv7 and amd64 architectures is available from Docker Hub.

So far I’m using this in my homelab to measure transfer speeds between my two sites (BKK80 and HEA92) and from public servers in London, New York and Singapore to my two sites:

The install was uneventful and went according to the documentation so there’s nothing much to add there.

Their docs provide a sample Python script for reading the energy values via the I2C bus and this was also easy to get going. After getting the sample running I decided to put together a Prometheus exporter based on the script_exporter by Rico Berger. This is fully working now:

curl http://localhost:9469/probe?script=piups

# HELP script_success Script exit status (0 = error, 1 = success).
# TYPE script_success gauge
script_success{script="piups"} 1
# HELP script_duration_seconds Script execution time, in seconds.
# TYPE script_duration_seconds gauge
script_duration_seconds{script="piups"} 0.280330
# HELP piups_bus_voltage_volts The voltage supplied to the connected device(s)
# TYPE piups_bus_voltage_volts gauge
piups_bus_voltage_volts 5.208
# HELP piups_bus_current_milliamps The current drawn by the connected device(s)
# TYPE piups_bus_current_milliamps gauge
piups_bus_current_milliamps 136.805
# HELP piups_power_milliwatts The power consumed by the connected devices in milli-watts
# TYPE piups_power_milliwatts gauge
piups_power_milliwatts 737.561
# HELP piups_shunt_voltage_millivolts TBD
# TYPE piups_shunt_voltage_millivolts gauge
piups_shunt_voltage_millivolts 7.080

In my weather station dashboard I’ve added a side-bar showing the energy consumption of the station as queried every 15 seconds from the UPS board:

I have packaged my solution as piups_exporter on GitHub and the README in that project elaborates on how to get it setup, which is very easy.

It was only after I got my UPS board that I realized there’s a newer model than the one that I ordered - the EP-0136 “UPS Plus” which provides a lot more metrics - so naturally I’ve ordered one of those and once I have it I will come back and update piups_exporter to also ship those metrics.

Hardware

Overview

So here’s what the weather station looks like currently:

The new things are:

• PoE+ adapter and 3D-printed mounting bracket
• Replacement SSD enclosure and 3D-printed mounting bracket
• LED indicators for UPS board power, Pi power status and SSD activity
• SI1145 light sensor

The AS3935 sensor plugged in at the right isn’t connected yet - that’s just so I don’t misplace it.

PoE+ Adapter

I have a few these PoE+ adapters in my parts cabinet so I just threw one on to see what would happen with power being supplied by my UniFi 24 port switch. It worked, but after about 5 days the system lost power.

After fully recharging the batteries via the barrel jack I hooked up my Ruideng USB in-line multimeter to determine that when the UPS board is already fully charged it draws about 1 milliamp from the PoE+ adapter:

Even though the Pi is running and drawing around half an amp:

In other words the UPS is driving the board off batteries even when connected to power. Eventually it hits a threshold and starts charging again, but apparently not effectively enough because eventually the batteries run flat and the system dies. In short - this is not a UPS, it’s a powerbank.

I have ordered a 52pi UPS board. I believe it’s a better unit - but it’s really hard to tell from the data sheet so has to be tested. This board has a built-in RTC (a nice-to-have for my application) and a power meter so you can read draw and battery level via I2C which will be cool if I can get that to work. Ideally I will read those values and export them to Prometheus.

Replacement SSD Enclosure

On Tuesday morning last week I woke up to a dead weather station. From the Prometheus-exported host telemetry I could see that it went offline around 2:20am local time.

Through a process of elimination I determined that it was the interface board in the SSD enclosure that had failed and that the SSD itself was fine so I ordered in a new Orico brand enclosure. This one was a different size to the original so I had to print a new bracket for it:

LED Indicators

Eventually this thing is going to be mounted in a cabinet on a pole in a field, so it would be helpful if the power and HDD lights were easy to see. Fortunately Raspberry Pi supports breaking out LED’s so it was a 5 minute job to add external power and HDD activity LED’s. I also connected a LED to one of the JST outputs on the UPS board to indicate when the board is supplying power:

Endurance Test

Even though the current UPS board is no good, I ran an endurance test to see how long these two 18650 cells could run the host for and the result was just under 16 hours.

Once I get the replacement UPS board I will re-run the endurance test, but this is already a very good indicative result - being able to survive a power outage of more than half-a-day will make this a very robust system.

Sensors

I added one new sensor in this iteration. Originally I was going to add the I2C lightning sensor which was already in my parts bin but during the week last week the GY/SI1145 light sensor came in so I hooked that up instead.

I broke out the I2C bus to a breadboard and reconnected the BME280 plus added the SI1145 - blue is data and orange is clock:

Verified connectivity and found the SI1145 at 0x60 while the BME280 is still at 0x76:

pi@hea92weather01:~ $ i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: 60 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- 76 --

Then added the code to read the SI1145 and include it in the Influx line protocol message published to MQTT:

import SI1145.SI1145 as SI1145

...

#
# GY1145 / SI1145
#

si1145 = SI1145.SI1145()

...

# SI1145
lightVisible = si1145.readVisible()
lightUV = si1145.readUV()
lightIR = si1145.readIR()

# Publish
influx = "weather,station=" + weather_station_id + " " + \
	"humidity=" + str(humidity) + \
	",pressure=" + str(pressure) + \
	",temperature=" + str(temperature) + \
	",visible=" + str(lightVisible) + \
	",uv=" + str(lightUV) + \
	",ir=" + str(lightIR)

And added some panels to my weather station dashboard in my main Grafana instances:

Data Plane

Overview

Most of my time on this iteration was spent on the data plane. The improvements here were:

• Handle SIGTERM
• MQTT Auto-Reconnect
• Containerized the sensor-reader Python script
• Added a custom bridge network to connect the three Docker Compose stacks
• Added an on-prem MQTT broker and InfluxDB storage
• Configured the edge MQTT broker to bridge to the on-prem MQTT broker
• Connect Grafana to use the on-prem InfluxDB instead of edge InfluxDB
• Connected my Home Assistant instance to MQTT and added visualizations
• De-hardcode sensor-reader

The data plane now looks like this:

Handle SIGTERM

In order to be a good container citizen I added support for SIGTERM and SIGINT Linux signals - this is the skeleton for signal handling:

class SensorReader:

	def __init__(self):
		self.stopped = False
        
	def run():
		while not self.stopped:
            # Process sensors
            sleep(1)

	def stop(self, signal, frame):
		print("Stopping SensorReader")
		self.stopped = True
        
def main():
	sensorReader = SensorReader()
	signal.signal(signal.SIGINT, sensorReader.stop)
	signal.signal(signal.SIGTERM, sensorReader.stop)
	sensorReader.run()

if __name__ == "__main__":
	main()

This was adapted from a post but I’ve misplaced the reference.

MQTT Auto-Reconnect

My original MQTT implementation was bare bones. I’ve added code based on some helpful articles from Steves Internet Guide to track connection state and leverage Paho MQTT’s built-in auto-reconnect functionality:

		#
		# Configure MQTT CLient
		#

		def mqtt_on_connect(client, userdata, flags, rc):
			if rc == 0:
				print("Connected to MQTT broker")
				client.connected_flag = True
			else:
				print("Failed to connect")

		def mqtt_on_disconnect(client, userdata, rc):
			print("Disconnecting reason " + str(rc))
			client.connected_flag = False
			client.disconnect_flag = True

		client = mqtt.Client(weather_station_id)
		client.on_connect = mqtt_on_connect
		client.on_disconnect = mqtt_on_disconnect
		client.connected_flag = False
		client.username_pw_set(broker_username, broker_password)
		client.connect(broker_host_name, broker_port)
		client.loop_start()

		while not self.stopped and not client.connected_flag:
			print("Waiting for connection")
			sleep(1)

         while not self.stopped:

			if (client.connected_flag):
                
				# Read sensors and publish to MQTT

				sleep(15)

			else:
				print("Connection lost.  Waiting for reconnection")
				sleep(5)

		client.loop_stop()
		client.disconnect()

This now means if the local MQTT broker goes down, sensor-reader will retry until it can connect again and the sensor readings will flow once again.

Containerize sensor-reader

The daemon that reads sensors and publishes the values to MQTT is called sensor-reader and it’s implemented as a simple Python3 script. To Dockerize this I just had to create a requirements.txt to enumerate the packages that I had previously installed manually:

RPi.bme280
paho-mqtt
SI1145

Then created a simple Dockerfile based on the official Python image:

FROM python:3.9.5-buster

RUN mkdir /app
WORKDIR /app

COPY main.py ./
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

CMD [ "python3", "./main.py" ]

I initially wanted to use the alpine-based image but there is a problem running it on 32-bit RPi OS without privileged mode, so I fell back to the Debian Buster based image instead.

On first run I got this error:

pi@hea92weather01:~/als_sys_host_031_hea92weather01/sensor-reader $ ./run.sh
Traceback (most recent call last):
  File "/app/./main.py", line 92, in <module>
    main()
...
FileNotFoundError: [Errno 2] No such file or directory: '/dev/i2c-1'

Which was solved by mapping that device through when running the container:

sudo docker run \
    -it --rm \
    --device /dev/i2c-1 \
    sensor-reader:latest

After successfully testing the image I created a docker-compose config for easier startup:

version: "3"
services:

  #
  # SensorReader
  #

  sensorreader:
    image: "aleisium/sensor-reader:latest"
    restart: "always"
    devices:
      - "/dev/i2c-1:/dev/i2c-1"

Custom Bridge Network

I intentionally have three separate docker-compose stacks - the reason for this is that I want to be able to easily start and stop the MQTT broker, the local InfluxDB storage and the sensor-reader independently. This could also be achieved by having a single stack with convenience scripts to selectively stop and remove containers, and I may yet do that, but for now it’s nicer having separate stacks.

The problem with separate stacks is if you make them communicate via the physical eth0 interface, connectivity is lost when the network cable is pulled.

To solve this I created a custom bridge network in Docker and connected the three stacks to it.

The bridge is created with the script network/up.sh:

#!/bin/bash

sudo docker network create --driver=bridge --subnet=10.12.1.0/16 weather0

Then each of the three stacks are connected to that network - for example, sensor-reader:

version: "3"
services:

  #
  # SensorReader
  #

  sensorreader:
    image: "aleisium/sensor-reader:latest"
    networks:
      - "weather0"
    restart: "always"
    devices:
      - "/dev/i2c-1:/dev/i2c-1"

networks:
  weather0:
    external: true
    name: "weather0"

On-Prem MQTT Broker

I established a new “central” MQTT broker on a VM under ProxMox. I also now have an InfluxDB 1.8 instance with a Telegraf agent that reads from that central MQTT and stores measurements on-prem.

The central MQTT, Influx and Telegraf configs are separate from my weather station GitHub but I’ll be adding a similar config soon to make it easier for others to re-use.

Edge MQTT Bridge to On-Prem MQTT

I’m using the venerable Mosquitto MQTT broker which supports bridging - meaning an instance of the broker can be instructed to act as a client to another broker to read and / or write messages.

I added the following bridge config to broker/config/mosquitto.conf for the MQTT broker on the weather station:

connection central
address 10.80.2.60:1883
remote_username hea92weather01
remote_password password
topic # out

This causes the weather station to forward all messages it receives to the broker at 10.80.2.60 using the specified credentials. In this syntax # is not a comment marker but a wildcard indicating “all” messages.

References:

• mosquitto_conf Documentation

Point Grafana to On-Prem InfluxDB

As I use Grafana Provisioning for infrastructure-as-code of my telemetry stack, I just had to commit the address change to the data source definition in Git:

apiVersion: 1

# list of datasources that should be deleted from the database
deleteDatasources:
  - name: "hea92weather01"
    orgId: 1

# list of datasources to insert/update depending
# what's available in the database
datasources:
  - name: "hea92weather01"
    type: "influxdb"
    access: "proxy"
    database: "weather"
    user: "grafana"
    password: "password"
    url: "http://10.80.2.60:8086"

Then re-launch the telemetry stack.

Home Assistant Integration

Defined a connection to the central MQTT broker in Home Assistant’s configuration.yaml:

mqtt:
  broker: "10.80.2.60"
  port: 1883
  client_id: "bkk80ha"
  username: "bkk80ha"
  password: "password"

Added sensors to subscribe to values of interest from MQTT:

  - platform: "mqtt"
    name: "HEA92 Temperature"
    state_topic: "weather/hea92weather01/temperature"
    device_class: "temperature"
    unit_of_measurement: "°C"
    value_template: "0"
  - platform: "mqtt"
    name: "HEA92 Humidity"
    state_topic: "weather/hea92weather01/humidity"
    device_class: "humidity"
    unit_of_measurement: "%"
    value_template: "0"
  - platform: "mqtt"
    name: "HEA92 Pressure"
    state_topic: "weather/hea92weather01/pressure"
    device_class: "pressure"
    unit_of_measurement: "mbar"
    value_template: "0"
  - platform: "mqtt"
    name: "HEA92 Visible Light"
    state_topic: "weather/hea92weather01/visible"
    device_class: "illuminance"
  - platform: "mqtt"
    name: "HEA92 UV"
    state_topic: "weather/hea92weather01/uv"
    device_class: "illuminance"
  - platform: "mqtt"
    name: "HEA92 IR"
    state_topic: "weather/hea92weather01/ir"
    device_class: "illuminance"

Finally added some cards to a dashboard in Home Assistant:

Further elaboration is available in the README.md.

References:

• Home Assistant MQTT Broker
• Home Assistant MQTT Sensor

De-hardcode sensor-reader

Finally as a move towards making my weather station stack easily re-usable by others, I started parameterizing some of the hardcoded values in sensor-reader and moving them to environment variables so that they can be passed through the docker-compose file:

  sensorreader:
    environment:
      - "WEATHER_STATION_ID=hea92weather01"
      - "BROKER_USERNAME=hea92weather01"
      - "BROKER_PASSWORD=password"
      - "SAMPLE_INTERVAL=15"
    image: "aleisium/sensor-reader:latest"
    networks:
      - "weather0"
    restart: "always"
    devices:
      - "/dev/i2c-1:/dev/i2c-1"

Next Steps

The next improvements are:

• Processing of light readings into meaningful values
• Improve documentation and scripts to make it easier for others to re-use my configuration
• More sensors
• Test the new UPS board when it arrives
• Enclosure for the electronics
• Trial field deployment (on our condo balcony)

A few weeks back we renovated our balcony in Bangkok and as part of that exercise I completely re-installed the drip irrigation system that waters our many plants.

In the original install I had the flexible pipe just running along the tiles behind the pots. The problem with this that the pipe becomes a dirt trap, collecting soil particles and leaves behind it. For the re-install I wanted to get the pipe off the ground and thought maybe I could fabricate some clips to hold it under the lowest level of the balcony railing which would also put it somewhat out of sight as an added benefit.

I took some quick dimensions of the pipe and the railing:

Then sketched the shape in Fusion360 - this took all of 5 minutes:

It took 3 or 4 iterations of printing and testing to get the shape dialed in. The adjustments were reducing the inner dimension to make the clip hold onto the railing tightly, and progressively thickening the shape and increasing infill to make it stronger.

Here’s the test of the one that became the final version:

With the model finalized I printed 14 of them out and did the final installation:

The result was good. The clips are strong and haven’t deformed after being under load for a couple of weeks now, and the irrigation pipe is now mostly out of sight and more importantly is off the tiles making it easier for us to keep the balcony clean.

It remains to be seen if these PLA prints last long-term being exposed to the rain and sun - will be interesting to check back in on them in 6 months.