Differences

This shows you the differences between two versions of the page.

--- notes:mqtt [2024/04/18 19:44] – discord
+++ notes:mqtt [2024/04/20 20:46] (current) – discord
@@ Line 1: / Line 1: @@
-Written by Jacob Haip on April 17, 2024
+====== MQTT and Microcontroller Folk Integrations ======
+Jacob Haip, April 2024
-I am interested in integrating more physical hardware and sensors into Folk Computer to help expand the scope to the full room and to take advantage of the affordances of hardware that the projected AR system alone can't do. Things like sensing the noise level of the environment, motors that spin a fan and create wind, and the physical feel of a knob that clicks when turning off.
+I am interested in integrating more physical hardware and sensors into Folk Computer to expand computing to the size of the room and to take advantage of the affordances of hardware that the projected AR system alone can't do. Things like sensing the noise level of the environment, motors that spin a fan and create wind, and the physical feel of a knob that clicks when turning off.
 WiFi-connected microcontrollers are the primary tool I have been using to integrate sensors and actuators with Folk. Boards I have used include the [[https://www.adafruit.com/product/5400|Adafruit Feather ESP32]], [[https://www.adafruit.com/product/5526|RPi Pico W]], and the [[https://store.particle.io/products/photon-2|Particle Photon]]. They are either programmed using Arduino or MicroPython. I have also integrated microcontrollers using BLE or a wired serial connection but I tend to like WiFi for long-lived stable devices embedded in my space.
@@ Line 72: / Line 73: @@
 </code>
-=== Getting a button input into Folk ===
+===== Getting a button input into Folk =====
-TODO: GIF of button press.
+{{micro-button-press.gif}}
 <code c rpi-pico-w-micropython-button-press.py>
@@ Line 174: / Line 175: @@
 </code>
-=== Example code using the Adafruit MagTag E-Ink Microcontroller ===
+===== Output from Folk to Microcontroller =====
-[[https://www.adafruit.com/product/4800|Adafruit MagTag]] MicroPython Code that displays texts messages received on the MQTT /magtag/message topic, and publishes a heartbeat message on the MQTT /feeds/test topic.
+I have this cool [[https://www.adafruit.com/product/4745|Adafruit MatrixPortal M4]] Pixel Screen that can be programmed with Arduino. I set it up to listen on the /home/matrixportal MQTT topic where a message is a new screen to display.
-<code python magtag-micropython.py>
-import os
-import time
-import ssl
-import socketpool
-import wifi
-import adafruit_minimqtt.adafruit_minimqtt as MQTT
-from adafruit_magtag.magtag import MagTag
-magtag = MagTag()
+<code c matrix-portal.c>
-magtag.add_text(
+#include <SPI.h>
-    text_position=(
+#include <WiFiNINA.h>
-,
+#include <Adafruit_Protomatter.h>
-        (magtag.graphics.display.height // 2) - 1,
+#include <PubSubClient.h>
-    ),
-    text_scale=3,
-)
-magtag.set_text("Hello World")
-print(f"Connecting to {os.getenv('CIRCUITPY_WIFI_SSID')}")
+char ssid[] = "XXXX";   // your network SSID (name)
-print(f"Connecting to {os.getenv('CIRCUITPY_WIFI_PASSWORD')}")
+char pass[] = "XXXX"; // your network password (use for WPA, or use as key for WEP)
-wifi.radio.connect(os.getenv("CIRCUITPY_WIFI_SSID"), os.getenv("CIRCUITPY_WIFI_PASSWORD"))
-print(f"Connected to {os.getenv('CIRCUITPY_WIFI_SSID')}!")
-sending_feed = "/feeds/test"
+const char* mqttServer = "192.168.1.34";
-message_feed = "/magtag/message"
+const int mqttPort = 1883;
+const char* mqttTopic = "/home/matrixportal";
-def connected(client, userdata, flags, rc):
+WiFiClient espClient;
-    client.subscribe(message_feed)
+PubSubClient client(espClient);
-def disconnected(client, userdata, rc):
+uint8_t rgbPins[] = {7, 8, 9, 10, 11, 12};
-    print("Disconnected")
+uint8_t addrPins[] = {17, 18, 19, 20};
+uint8_t clockPin = 14;
+uint8_t latchPin = 15;
+uint8_t oePin = 16;
-def message(client, topic, message):
+Adafruit_Protomatter matrix(
-    print(f"New message on topic {topic}: {message}")
+, 4, 1, rgbPins, 4, addrPins, clockPin, latchPin, oePin, false);
-    magtag.set_text(message)
-pool = socketpool.SocketPool(wifi.radio)
+uint64_t pixelPalette[] = {
-ssl_context = ssl.create_default_context()
+    matrix.color565(0, 0, 0),
-mqtt_client = MQTT.MQTT(
+    matrix.color565(157, 157, 157),
-    broker="folk-haip.local",
+    matrix.color565(255, 255, 255),
-    port=1883,
+    matrix.color565(190, 38, 51),
-    socket_pool=pool,
+    matrix.color565(224, 111, 139),
-    ssl_context=ssl_context,
+    matrix.color565(73, 60, 43),
-)
+    matrix.color565(164, 100, 34),
+    matrix.color565(235, 137, 49),
+    matrix.color565(247, 226, 107),
+    matrix.color565(47, 72, 78),
+    matrix.color565(68, 137, 26),
+    matrix.color565(163, 206, 39),
+    matrix.color565(27, 38, 50),
+    matrix.color565(0, 87, 132),
+    matrix.color565(49, 162, 242),
+    matrix.color565(178, 220, 239),
+    matrix.color565(255, 0, 255)};
-# Setup the callback methods above
-mqtt_client.on_connect = connected
-mqtt_client.on_disconnect = disconnected
-mqtt_client.on_message = message
-# Connect the client to the MQTT broker.
+//--------- WIFI -------------------------------------------
-print("Connecting...")
-mqtt_client.connect()
-test_val = 0
+void wifi_connect() {
-while True:
+  Serial.print("Starting connecting WiFi.");
-    # Poll the message queue
+  delay(10);
-    mqtt_client.loop(timeout=1)
+  WiFi.begin(ssid, pass);
+  while (WiFi.status() != WL_CONNECTED) {
+    delay(500);
+    Serial.print(".");
+  }
+  Serial.println("WiFi connected");
+  Serial.println("IP address: ");
+  Serial.println(WiFi.localIP());
+}
-    # Send a new message
+//------------------ MQTT ----------------------------------
-    print(f"Sending value: {test_val}...")
+void mqtt_setup() {
-    mqtt_client.publish(sending_feed, test_val)
+  client.setServer(mqttServer, mqttPort);
-    print("Sent!")
+  client.setBufferSize(64*32 + 64);
-    test_val += 1
+  client.setCallback(callback);
+  Serial.println("Connecting to MQTT…");
+  while (!client.connected()) {
+      String clientId = "ESP32Client-";
+      clientId += String(random(0xffff), HEX);
+      if (client.connect(clientId.c_str())) {
+          Serial.println("connected");
+      } else {
+          Serial.print("failed with state  ");
+          Serial.println(client.state());
+          delay(2000);
+      }
+  }
+  client.subscribe(mqttTopic);
+}
+void callback(char* topic, byte* payload, unsigned int length) {
+    Serial.print("Message arrived in topic: ");
+    Serial.println(topic);
+    String byteRead = "";
+    Serial.print("Message: ");
+    for (int i = 0; i < length; i++) {
+        byteRead += (char)payload[i];
+    }
+    Serial.println(byteRead);
+    matrix.fillScreen(matrix.color565(0, 0, 0));
+    for (int i = 0; i < byteRead.length(); i++)
+    {
+        uint64_t pixelColor = matrix.color565(0, 0, 0);
+        String pixelChar = String(byteRead[i]);
+        int colorIndex = (int)strtol(&pixelChar[0], NULL, 16);
+        if (colorIndex >= 0 && colorIndex < 16)
+        {
+            pixelColor = pixelPalette[colorIndex];
+        }
+        matrix.drawPixel(i % 64, i / 64, pixelColor);
+    }
+    matrix.show(); // Copy data to matrix buffers
+}
+void setup()
+{
+    Serial.begin(9600);
+    // Initialize matrix...
+    ProtomatterStatus pmstatus = matrix.begin();
+    Serial.print("Protomatter begin() status: ");
+    Serial.println((int)pmstatus);
+    if (pmstatus != PROTOMATTER_OK)
+    {
+        for (;;)
+            ;
+    }
+    matrix.println("wifi..."); // Default text color is white
+    matrix.show();                   // Copy data to matrix buffers
+    wifi_connect();
+    matrix.println("mqtt..."); // Default text color is white
+    matrix.show();                   // Copy data to matrix buffers
+    mqtt_setup();
+}
+void loop()
+{
+    client.loop();
+}
 </code>
-A folk program to make a crude clock on the MagTag display:
+And with this you can make a crude clock that fills up the screen horizontally as time passes.
-<code tcl crude-clock.folk>
+{{crude-clock.gif}}
+<code tcl crude-matrixportal-clock.folk>
 When /node/ has step count /c/ {
-  if {[expr {int($c) % 5000}] == 0} {
+  if {[expr {int($c) % 1000}] == 0} {
-    set ts [clock milliseconds]
+    set currentTimeMillis [clock milliseconds]
-    Wish MQTT publish $ts on topic jhaip/feeds/onoff at timestamp $ts
+    set millisSinceMinuteStart [expr {$currentTimeMillis % 60000}]
+    set fillValue [expr {$millisSinceMinuteStart * 64 / 60000}]
+    set matrixportaloutput ""
+    for {set y 0} {$y < 32} {incr y} {
+      for {set x 0} {$x < 64} {incr x} {
+        if {$x < $fillValue} {
+          set matrixportaloutput "9$matrixportaloutput"
+        } else {
+          set matrixportaloutput "0$matrixportaloutput"
+        }
+      }
+    }
+    set c [clock milliseconds]
+    Wish MQTT publish "$matrixportaloutput" on topic /home/matrixportal at timestamp $currentTimeMillis
   }
 }
 </code>
-A folk program that shows the latest published message from the MagTag device:
+Additionally, you can make a web app that also speaks to the MQTT Broker using [[https://github.com/mqttjs/MQTT.js|MQTT.js]]. In this case, the Folk system actually isn't used at all because all the logic is within the Javascript of the web page to edit the pixels and send MQTT messages.
-<code tcl show-message.folk>
+{{matrixportal-live-edit.gif}}
-When MQTT claims message /message/ topic "/feeds/test" timestamp /t/ {
-   Wish $this is labelled "MagTag value: $message"
+===== Using MQTT on web pages for bidirectional communication with Folk =====
+Making simple web pages using the [[https://github.com/mqttjs/MQTT.js|MQTT.js]] library also bring bidirectional communication to Folk. In the current state of the Folk code, you had to long poll for updates but using MQTT like this is nice because you don't have to poll. I like these basic web apps as a way to bring phones and laptops into the Folk system as new inputs and outputs.
+First we make a web app that plays a sound when it receives an MQTT messages:
+<code html play-sound.html>
+<html>
+  <head><title>Websocket client</title></head>
+  <body>
+    <h1>Plays a sound when /web/playsound receives a MQTT message</h1>
+    <audio id="myAudio" controls preload="preload">
+        <source src="https://www.w3schools.com/html/horse.ogg" type="audio/ogg">
+        <source src="https://www.w3schools.com/html/horse.mp3" type="audio/mpeg">
+        Your browser does not support the audio element.
+    </audio>
+    <script src="https://unpkg.com/mqtt/dist/mqtt.min.js"></script>
+    <script>
+const clientId = 'mqttjs_' + Math.random().toString(16).substr(2, 8)
+const host = 'ws://192.168.1.34.local:8080'
+const options = {
+  keepalive: 60,
+  clientId: clientId,
+  protocolId: 'MQTT',
+  protocolVersion: 4,
+  clean: true,
+  reconnectPeriod: 1000,
+  connectTimeout: 30 * 1000,
+  will: {
+    topic: 'WillMsg',
+    payload: 'Connection Closed abnormally..!',
+    qos: 0,
+    retain: false
+  },
+}
+console.log('Connecting mqtt client')
+const client = mqtt.connect(host, options)
+client.on('error', (err) => {
+  console.log('Connection error: ', err)
+  client.end()
+})
+client.on('reconnect', () => {
+  console.log('Reconnecting...')
+})
+client.on('connect', () => {
+  console.log('Client connected:' + clientId)
+  client.subscribe('/web/playsound', { qos: 0 })
+});
+client.on('message', (topic, message, packet) => {
+  console.log('Received Message: ' + message.toString() + '\nOn topic: ' + topic)
+  document.getElementById("myAudio").currentTime = 0;
+  document.getElementById("myAudio").play();
+});
+</script>
+  </body>
+</html>
+</code>
+And then in folk you can make every phone and laptop with that web page open play a sound when a button is pressed:
+{{button-horse.mp4}}
+<code tcl trigger-sound.folk>
+When MQTT claims message /message/ topic "/button1" timestamp /t/ {
+   Wish $this is labelled "BUTTON 1 PRESSED"
+   set ts [clock milliseconds]
+   Wish MQTT publish "make some noise" on topic "/web/playsound" at timestamp $ts
 }
 </code>