Differences

This shows you the differences between two versions of the page.

--- newsletters:2025-01 [2025/02/03 02:53] – [General Folk system improvements] osnr
+++ newsletters:2025-01 [2025/02/03 03:46] (current) – [Portable Folk gadget] osnr
@@ Line 76: / Line 76: @@
 ==== New parallel evaluator ====
-Omar: Progress on this has been limited this month.
+Omar: Progress on this has been limited this month. A lot of looking at these specific spans in Tracy where there was a blink-out of a tag outline:
-A lot of looking at these specific spans in Tracy where there was a blink-out of a tag outline:
+{{:newsletters:pasted:20250131-195852.png?400px}} {{:newsletters:pasted:20250131-195919.png?400px}}
-{{:newsletters:pasted:20250131-195852.png?400px}} {{:newsletters:pasted:20250131-195919.png}}
+You can see the drawcount plot mentioned at the bottom -- when that dips from 4 (the edges of the page outline) to 0, you know you have an unwanted blink (assuming the page was visible on camera the whole time). Tried adding more messages so we can see the actual statements that are evaluating (and getting stuck on).
-You can see the drawcount plot at the bottom -- when that dips from 4 to 0, you know you have an unwanted blink. Tried adding more messages so we can see the actual statement.
+You can see some of the red spans in the above images where threads are getting preempted while running Folk When code, which delays all downstream statements, and they might be delayed for 10ms+ while the thread isn't on the CPU, which means you'll miss the frame deadline and the page blinks out! Here's another one (the SAY I'm hovering over and the HOLD above it have both been preempted and delayed in that way):
-Red span
+{{:newsletters:pasted:20250203-031534.png?500px}}
-I cleaned up the detection of past versions (it makes an explicit folk.pid file) and have been rethinking our thread spawning/killing policies. I think a lot of the blinking is just because we have too many threads for the number of CPUs and they're preempting each other mid-task, which means that the evaluation doesn't converge in time for the frame.
+After getting back in late January, I ran into issues running the real-time scheduler branch from last month (it's hard to find & kill those threads by process name? do they dominate their CPUs too much?). So I cleaned up the detection of past versions (Folk now makes an explicit folk.pid file) and have been rethinking our thread spawning/killing policies.
+Again, I think a lot of the blinking is just because we have too many threads for the number of CPUs and they're preempting each other mid-task, which means that the evaluation doesn't converge in time for the frame. So need to manage the number of threads in a more reasonable way. Maybe based on I/O-bound-ness (is the thread in Linux Running or Blocked state?) instead of just pure execution time which was always a hack...
 ==== Portable Folk gadget ====
-All the components are in. This revision uses the Orange Pi 5.
+Omar: I've been working on a new portable Folk gadget that uses off-the-shelf parts, so anyone will be able to build/buy one, along with some other improvements: built-in battery, trigger button you can press with your finger, built-in speaker.
+All the components are in place and I'm starting to fix up the software to make Folk work on the platform.
 {{:newsletters:pasted:20250131-194410.jpeg?250px}} {{:newsletters:pasted:20250131-194400.jpeg?250px}}
+This revision uses the Orange Pi 5 instead of the Raspberry Pi 5, and a stereo USB camera instead of Pi wide-angle camera, which has caused various random issues.
 === Network setup ===
@@ Line 100: / Line 108: @@
   * Still struggling to make it do ad-hoc or access point mode by default
     * (ideally it would work in concurrent mode, so could both get on known existing networks and make its own for wild environments:)
-      * {{:newsletters:pasted:20250131-200028.png?300px}}
+      * {{:newsletters:pasted:20250131-200028.png?400px}}
 === The stereo camera ===
-The stereo camera works! Its field of view is huge, though:
+The stereo camera works! Its field of view is huge, though (the subregion we actually care about in each stereo image is mostly the red-circled region where we can project):
-{{newsletters:imagestereo.png?400px}}
+{{newsletters:foooo.jpg?400px}}
-Added crop support to just take the center part of the left image for now so we don't waste time processing the rest and can start trying to calibrate right away.
+[[https://github.com/FolkComputer/folk/pull/189|Added crop support]] to just take the center part of the left image for now so we don't waste time processing the rest and can start trying to calibrate right away.
-=== Current state & next steps ===
+Haven't been able to calibrate yet, though; I think I'm having issues with exposure so I can't read enough tags to calibrate.
-really heavy
+=== Current state & next steps ===
-button works
-battery works
+Overall, the new gadget revision is in OK shape, although not done yet:
-sound not working yet
+  * It has a working trigger button! That'll open up a lot of fun point-and-drag and lasso interactions. Here I'm having the 'marching ants' rectangle turn from white to green when I press the button down:
+    * {{newsletters:img_7370.mp4}}
+    * This is the program that implements that (I just put it in setup.folk, since we don't have physical programs here yet):
+      * <code>
+exec gpio mode 7 up
-need to figure out how to calibrate.
+When display /disp/ has width /w/ height /h/ {
+    When the clock time is /t/ {
+        set pressed [expr {![exec gpio read 7]}]
+        set color [expr {$pressed ? "green" : "white"}]
+        Wish to draw a dashed stroke with points [list [list 0 0] [list $w 0] [list $w $h] [list 0 $h] [list 0 0]] \
+          color $color width 10 dashlength 40 dashoffset [expr {fmod($t, 10)*-120}]
+    }
+}
+</code>
+      * it basically just sets a pullup resistor on GPIO 7 (where I plugged the button in) & shells out to [[https://github.com/orangepi-xunlong/wiringOP|wiringOP]] every frame to read that pin; to avoid the multi-ms overhead of shelling out, would be good to make a Tcl-C library to call into wiringOP and do this (I think the C header and library is already on the device anyway!)
+  * It has a working battery, so you don't need to plug it in & carry a long extension cord! I haven't tested battery life but probably at least an hour? And I can even get live reports over serial about battery percentage, just like a phone or laptop shows battery percentage:
+    * {{:newsletters:pasted:20250203-034338.png?600px}}
+    * (will be fun to get this info into Folk and display it ambiently in the corner or on the marching ants border or whatever)
+  * It's really heavy and kind of big, notably heavier than the older gadget (because it has 3 18650 batteries and a heavier mini projector with a full chassis). Not sure what to do about this. Maybe go back to the concept of the detachable battery pack (a standard USB-C battery pack would be ideal, but I don't think I could quite get that to work without more custom power stuff).
+  * It has a speaker! I haven't gotten this to work yet (I haven't gotten the Orange Pi 5 3.5mm audio jack to work at all. Linux audio struggles). Also curious how loud it'll be without an amplifier.
+  * The biggest issue is that I haven't been able to calibrate it yet, but I just need to spend more time on that...
-need to do some physical revision
+Finally, need to finish physical design (I want to put on the back panel so people feel more comfortable handling it, less wires poking out, fix the speaker in place, maybe use a different camera, make the grip feel less precarious since it only has 1 bolt) and publish on GitHub.
 ==== RFID localization ====
@@ Line 144: / Line 169: @@
   * Omar: Finish the new gadget revision (back panel, audio support, calibrate camera) and document it on GitHub for people to try.
   * Omar: Set up the out-of-band RFID radio and maybe start on localization.
-  * Omar: New evaluator
+  * Omar: Figure out thread management on the new evaluator so work items don't get delayed for 10ms at a time (hopefully will fix blinking)
-  * Andrés: Video
+  * Andrés: Work on video
-  * Andrés: Dot detector
+  * Andrés: Dot detector port to work under 3D calibration
   * We'll be speaking at the [[https://interactiondesign.sva.edu/|School of Visual Arts' Interaction Design]] department’s 2025 winter/spring lecture series. The talk will be recorded so we'll link it here as soon as it's online.
   * We'll also be showing Folk as an interactive prototype in a workshop in the Design in Action course at Columbia GSAPP. The students' [[https://gsapp-cdp.github.io/archive/|past work]] is very Folk-adjacent so we expect to get a lot of interesting feedback and are excited to show an enthusiastic technical audience Folk up close.