Differences

This shows you the differences between two versions of the page.

--- newsletters:2026-02 [2026/03/10 02:30] – [How the redirection works] osnr
+++ newsletters:2026-02 [2026/03/11 19:14] (current) – osnr
@@ Line 2: / Line 2: @@
 {{htmlmetatags>metatag-og:title=(Folk Computer February 2026 newsletter)
-metatag-media-og:image=(newsletters:IMAGE_FILENAME_HERE.jpg)
+metatag-media-og:image=(newsletters:pasted:20260311-191224.jpeg)
-metatag-og:description=(Brief description of key updates this month)
+metatag-og:description=(Python FFI, image segmentation, text recognition; sound wheel; stdout and stderr redirection; Folk February; Hamish talk; post-calibration refinement)
 }}
@@ Line 210: / Line 210: @@
 and with this in place, a program can just When to get the detector function and call it on a camera frame / camera slice Image and get back a list of text boxes. (The FFI will automatically convert any Python list/dict/etc into a Tcl object by using JSON serialization, and vice versa, in arg or return types.)
+{{newsletters:img_2224.jpeg?250px}}
 (I do wonder if this binding even needs to be built into Folk or if it should just be part of the user program on the table, it's so short... and it would've been nice, at the last open house, if people could just see this caller implementation on the table, rather than having to take our word for it.)
@@ Line 230: / Line 232: @@
 === Text detection + handwriting recognition ===
+[[https://github.com/FolkComputer/folk/blob/c4d80ff2474fad6de32c327b1a3c6b8ed7b3fde6/builtin-programs/recognition/trocr.folk|Here's the binding to the TrOCR text recognizer.]]
+TrOCR binding side-by-side with the CRAFT text detector binding:
+{{.:pasted:20260310-024607.png?600px}}
 This doesn't work great yet -- I wonder if we need higher resolution or what -- but here it's finding the slice where "text" is handwritten and then OCRing it:
@@ Line 235: / Line 243: @@
 {{newsletters:img_1874.mp4?300px}}
+=== Image segmentation using SAM2 ===
+I don't have a good photo or video of this yet (needs to be set up in the new space and have more interesting demo built around it, maybe whole table instead of camera slice), but [[https://github.com/FolkComputer/folk/blob/3efc3b2687050eb8b3560134e009a28654be6b84/builtin-programs/recognition/sam2.folk|here's the binding]] for [[https://github.com/facebookresearch/sam2|SAM2]].
+Would be useful for pointing to an arbitrary object and tracking or projection mapping it.
 ==== ''Expect'' experiment ====
@@ Line 265: / Line 278: @@
 {{.:pasted:20260307-012646.png?500px}}
-Have been picking back up what used to be ''table-refine.folk'' -- a post-calibration refinement step, basically a new kind of calibration that is interactive. It still requires that you do the traditional calibration step first, because it needs a guess that's good enough (lets it see enough tags) that it can refine it.
+Have been [[https://github.com/FolkComputer/folk/tree/osnr/interactively-refine-calibration|picking back up]] what used to be ''table-refine.folk'' and is now [[https://github.com/FolkComputer/folk/blob/200f1c8e9197a09f7d063fe13420f95f1b50b760/builtin-programs/calibrate/interactively-refine.folk|''interactively-refine.folk'']] -- a post-calibration refinement step, basically a new kind of calibration that is interactive or 'online'. You set the calibration board in a pose and the system projects and adjusts its full end-to-end calibration for a while until it can nail that pose:
-TODO: originally, problem was I wasn't giving enough residuals, so the system was underdetermined?
+{{newsletters:img_2220.mp4?400px}}
+This uses the Levenberg-Marquardt optimization that we also use in the traditional calibration, but in the middle of the evaluation function, we project with the current candidate calibration and fully loop that through the camera and tag detector, so we get a 'real' end-to-end error.
+It still requires that you do the traditional calibration step first, because it needs a guess that's good enough (lets it already see enough tags) that it can refine it.
+=== Various issues ===
+Originally, problem was I typoed and wasn't giving enough residuals (need more residuals than parameters), so the system was underdetermined and would fail:
 {{.:pasted:20260307-012756.png?400px}}
-TODO: for a while, problem was LM step size wasn't large enough, so the changes to estimate gradient were below the noise floor, so it wouldn't improve. you can see that it improves a lot now:
+For a while, problem was LM step size wasn't large enough (it wasn't tuning the parameters by a big enough offset, like 0.00001 instead of 0.001), so the changes to estimate gradient were below the noise floor of tag detection, so it couldn't estimate the gradient and know which direction to improve things.
+You can see that the refinement improves a lot now (orignorm -> bestnorm):
 {{.:pasted:20260307-012844.png?500px}}
@@ Line 292: / Line 315: @@
 === Open house ===
-  * We hosted our last monthly open house at our Hex House studio on Thursday, February 26th, a few of Andrés' students from SVA stopped by and we had fun demoing the animation program as well as segmentation that Omar's been working on and a sound-effect wheel that Andrés has been making:
+  * We hosted our last monthly open house at our Hex House studio on Thursday, February 26th -- a few of Andrés' students from SVA stopped by and we had fun demoing the animation program, as well as segmentation that Omar's been working on, and a sound-effect wheel that Andrés has been making:
-    * {{.:pasted:20260305-190452.jpeg?400px}} {{.:pasted:20260305-190510.jpeg?400px}}
+    * {{.:pasted:20260305-190510.jpeg?0x200px}} {{.:pasted:20260305-190541.jpeg?0x200px}}
-    * {{.:pasted:20260305-190541.jpeg?400px}}
+    * {{.:pasted:20260305-190531.jpeg?0x300px}} {{newsletters:img_2222.jpeg?0x300px}} {{newsletters:img_2226.mp4?200px}}
-    * {{.:pasted:20260305-190531.jpeg?300px}}
+    * {{newsletters:img_2228.jpeg?0x300px}} {{newsletters:img_2232.jpeg?0x300px}}{{newsletters:img_2233.jpeg?0x300px}} {{newsletters:img_2237.jpeg?0x300px}}
+== Atomically bug ==
+Our friend [[https://x.com/perlinwarp|Peter Walkington]] found a bug with Atomically that shows up after a little while on this program:
+{{newsletters:img_2242.jpeg?400px}}
+It should alternate between :O and :), but it ends up in this superposition where both :O and :) are visible at the same time:
+{{newsletters:img_2241.jpeg?0x400px}}
+It looks like we're leaking versions, like we're 20,000 versions behind on item 5 here (1557's clock time):
+{{newsletters:screenshot-2026-02-27-at-4.48.51-pm.png?500px}}
+We saw similar issues with Atomically at the party end of last year -- it's nice to have a concrete example and a sense of what the issue is. Need to debug.
 ===== What we'll be up to in March =====