Folk Computer

This is an old revision of the document!

Written by Jacob Haip on April 17, 2024

I am interested in integrating more physical hardware and sensors into Folk Computer to help expand the scope to the full room and to take advantage of the affordances of hardware that the projected AR system alone can't do. Things like sensing the noise level of the environment, motors that spin a fan and create wind, and the physical feel of a knob that clicks when turning off.

WiFi-connected microcontrollers are the primary tool I have been using to integrate sensors and actuators with Folk. Boards I have used include the Adafruit Feather ESP32, RPi Pico W, and the Particle Photon. They are either programmed using Arduino or MicroPython. I have also integrated microcontrollers using BLE or a wired serial connection but I tend to like WiFi for long-lived stable devices embedded in my space.

Folk has the ability to create arbitrary HTTP endpoints and a Websocket connection that proxies through Folk Statements, but neither are currently bidirectional and both are heavy protocols for microcontrollers to send or get data from Folk.

Lately I have stated using MQTT as the protocol for microcontrollers to talk to the Folk System. It is a lighter weight Pub/Sub messaging protocol that is primarily done on top of a TCP connection. Microcontrollers talk to a MQTT Broker running alongside Folk and then a Folk program talks to the broker as well.

I am using the Mosquitto MQTT Broker which runs on port 1883.

sudo apt install mosquitto mosquitto-clients -y
sudo systemctl status mosquitto
sudo systemctl start mosquitto
sudo systemctl enable mosquitto
mosquitto_sub -h address -t topic
mosquitto_pub -h address -t topic -m “message”

The bridge from the MQTT broker to Folk is the mqtt.folk program which adds support for wishes like Wish MQTT publish $message on topic $feed at timestamp $timestamp and When MQTT claims message /message/ topic /topic/ timestamp /t/ { ... }.

mqtt.folk

# This program depends on the https://chiselapp.com/user/schelte/repository/mqtt/home MQT library being installed at the /home/folk/mqtt directory
Start process mqtt {
    Wish $::thisProcess receives statements like \
        [list /someone/ wishes MQTT publish /...anything/]
    Wish $::thisProcess receives statements like \
        [list $::thisNode has step count /...anything/]
    Wish $::thisProcess shares statements like \
        [list MQTT claims message /message/ topic /topic/ timestamp /timestamp/]
 
    ::tcl::tm::path add /home/folk/mqtt
    package require mqtt
 
    set client [mqtt new]
    set ::mqttclient $client
 
    proc cb {topic content status} {
        set message [encoding convertfrom utf-8 $content]
        puts "MQTT CALLBACK: $topic: $message"
        set now [clock milliseconds]
        Assert MQTT claims message $message topic $topic timestamp $now

        # Retract all events that are more than 5 seconds old.
        set events [Statements::findMatches [list MQTT claims message /message/ topic /topic/ timestamp /timestamp/]]
        foreach event $events {
            dict with event {
                if {$now - $timestamp > 5000} {
                    Retract MQTT claims message $message topic $topic timestamp $timestamp
                }
            }
        }
    }
    $client connect test-client localhost 1883
    $client subscribe "#" cb
 
    When /someone/ wishes MQTT publish /m/ on topic /t/ at timestamp /ts/ {
        # puts "publishing $m on $t at $ts"
        $::mqttclient publish "$t" "$m"
    }
 
    while true {
        # Step for folk, update for TCL and MQTT
        Step
        update
    }
}

Example code using the Adafruit MagTag E-Ink Microcontroller

Adafruit MagTag MicroPython Code that displays texts messages received on the MQTT /magtag/message topic, and publishes a heartbeat message on the MQTT /feeds/test topic.

magtag-micropython.py

import os
import time
import ssl
import socketpool
import wifi
import adafruit_minimqtt.adafruit_minimqtt as MQTT
from adafruit_magtag.magtag import MagTag
 
magtag = MagTag()
magtag.add_text(
    text_position=(
        50,
        (magtag.graphics.display.height // 2) - 1,
    ),
    text_scale=3,
)
magtag.set_text("Hello World")
 
print(f"Connecting to {os.getenv('CIRCUITPY_WIFI_SSID')}")
print(f"Connecting to {os.getenv('CIRCUITPY_WIFI_PASSWORD')}")
wifi.radio.connect(os.getenv("CIRCUITPY_WIFI_SSID"), os.getenv("CIRCUITPY_WIFI_PASSWORD"))
print(f"Connected to {os.getenv('CIRCUITPY_WIFI_SSID')}!")
 
sending_feed = "/feeds/test"
message_feed = "/magtag/message"
 
def connected(client, userdata, flags, rc):
    client.subscribe(message_feed)
 
def disconnected(client, userdata, rc):
    print("Disconnected")
 
def message(client, topic, message):
    print(f"New message on topic {topic}: {message}")
    magtag.set_text(message)
 
pool = socketpool.SocketPool(wifi.radio)
ssl_context = ssl.create_default_context()
mqtt_client = MQTT.MQTT(
    broker="folk-haip.local",
    port=1883,
    socket_pool=pool,
    ssl_context=ssl_context,
)
 
# Setup the callback methods above
mqtt_client.on_connect = connected
mqtt_client.on_disconnect = disconnected
mqtt_client.on_message = message
 
# Connect the client to the MQTT broker.
print("Connecting...")
mqtt_client.connect()
 
test_val = 0
while True:
    # Poll the message queue
    mqtt_client.loop(timeout=1)
 
    # Send a new message
    print(f"Sending value: {test_val}...")
    mqtt_client.publish(sending_feed, test_val)
    print("Sent!")
    test_val += 1

A folk program to make a crude clock on the MagTag display:

crude-clock.folk

When /node/ has step count /c/ {
  if {[expr {int($c) % 5000}] == 0} {
    set ts [clock milliseconds]
    Wish MQTT publish $ts on topic jhaip/feeds/onoff at timestamp $ts
  }
}

A folk program that shows the latest published message from the MagTag device:

show-message.folk

When MQTT claims message /message/ topic "/feeds/test" timestamp /t/ {
   Wish $this is labelled "MagTag value: $message"
}