Differences

This shows you the differences between two versions of the page.

--- newsletters:2026-06 [2026/07/16 19:41] – osnr
+++ newsletters:2026-06 [2026/07/18 13:30] (current) – [SVA system fixes] osnr
@@ Line 1: / Line 1: @@
 ====== June 2026 newsletter ======
 {{htmlmetatags>metatag-og:title=(Folk Computer June 2026 newsletter)
-metatag-media-og:image=(newsletters:20260705-214508.jpg)
+metatag-media-og:image=(newsletters:img_5748.jpg)
-metatag-og:description=(Contour object tracking; ShaderToy demos; fields demo; performance analysis; color spinner)
+metatag-og:description=(Contour object tracking; new drawing API; ShaderToy demos; fields in code printout; performance analysis; color spinner)
 }}
 Our next Folk open house will be in the evening on **[[https://luma.com/vk6nnzl5|Wednesday, July 29]]**, at our studio in Williamsburg, Brooklyn.
+Also: [[https://danielpipk.in/|Daniel Pipkin]] and [[https://github.com/smj-edison|Mason Jones]] will be exhibiting Folk at [[
+https://craftlakecity.com/diy-festival/|Craft Lake City's DIY Festival]] August 7-9 in Salt Lake City.
 ===== What we've been up to =====
 ==== General system improvements ====
@@ Line 22: / Line 26: @@
 === New drawing API ===
-Andrés: TODO
+Andrés: I finally merged the [[https://github.com/FolkComputer/folk/pull/261 |Revive drawing capabilities (new and improved!)]] PR! So now you can do:
-Units on everything.
-Error reporting requires you to look at the shapes.folk log right now:
+  * Wish $this draws a circle
+  * Wish $this draws a circle with color gold radius 20mm
+  * Wish $this draws a circle with color magenta radius 20cm position [list x 20% 50%]
+    * and you get this:
+      * {{.:pasted:20260718-114822.mp4}}
-{{newsletters:screenshot-2026-06-17-at-10.51.21-am.png?400px}}
+We enforce units (mm, cm, m, %) on everything in the new drawing system which is nice as it makes it easy to tell, using your real-world brain, where the shapes you’re drawing are going to land.
+  * Error reporting requires you to look at the shapes.folk log on the web dashboard right now:
+    * {{newsletters:screenshot-2026-06-17-at-10.51.21-am.png?400px}}
-Causality TODO
+  * By the end of July we should have [[https://github.com/FolkComputer/folk/pull/272 |a fix that bubbles up errors to the top level reporting mechanism]] on the table, which should make this annoying reliance on the web dashboard for shapes.folk errors disappear.
 === ShaderToy compatibility ===
@@ Line 40: / Line 48: @@
 {{.:pasted:20260705-212644.jpeg?0x300px}} {{newsletters:img_5975.jpeg?0x300px}}
-We've had an [[https://github.com/FolkComputer/folk/issues/106|issue about this]] for a while, since the internal shader/pipeline API is mostly just meant for use by the drawing primitives, not for creative coding, but this new feature was immediately motivated by talking to [[https://ktrosenberg.github.io/|Karl]] [[https://discord.com/channels/956758212152025098/956765077883744266/1512876563669057726|on Discord]] about richer drawing APIs -- [[https://danielpipk.in/Daniel Pipkin]] did an [[https://github.com/FolkComputer/folk/pull/270|initial attempt]], and then Omar built on the approach and figured we might as well go all the way to ShaderToy compatibility if we're going to have a user-facing API for toy shaders.
+We've had an [[https://github.com/FolkComputer/folk/issues/106|issue about this]] for a while, since the internal shader/pipeline API is mostly just meant for use by the drawing primitives, not for creative coding, but this new feature was immediately motivated by talking to [[https://ktrosenberg.github.io/|Karl]] [[https://discord.com/channels/956758212152025098/956765077883744266/1512876563669057726|on Discord]] about richer drawing APIs -- [[https://danielpipk.in/|Daniel Pipkin]] did an [[https://github.com/FolkComputer/folk/pull/270|initial attempt]], and then Omar built on the approach and figured we might as well go all the way to ShaderToy compatibility if we're going to have a user-facing API for toy shaders.
 ==== Fields + local feedback calibration / nudging experiment ====
@@ Line 120: / Line 128: @@
 First, you can see how contours all change color from frame to frame -- we can't identify that a contour is 'the same' object from frame to frame without some additional logic.
-You could imagine tricks for this: we could compute an average color or color distribution for each contour, and we could use that to distinguish a blue cup contour from a red stapler contour (identity by color) (that also might be a nice visualization anyway). We could look at the contour shape (identity by shape) (although this runs into rotation and the distortion problem described below). We could find the closest contour from previous frame and identify it with new contour in the new frame (so identity by location/trajectory). But the point is, it's not something you get for free from the contour detector (unless you assume you're looking for exactly 1 object in the frame or something).
+You could imagine tricks for this: we could compute an average color or color distribution for each contour, and we could use that to distinguish a blue cup contour from a red stapler contour (identity by color) (that also might be a nice visualization anyway). We could look at the contour shape (identity by shape) (although this runs into rotation and the distortion problem described below). We could find the closest contour from previous frame and identify it with new contour in the new frame (so identity by location/trajectory).
-== Contour failures ==
+But the point is, it's not something you get for free from the contour detector (unless you assume you're looking for exactly 1 object in the frame or something).
+== Distinct contours not corresponding with distinct objects ==
 Contours are traced from a binarized image, so all dark areas that are adjacent get merged into the same contour. You might have one object that accidentally splits into multiple contours (if it has light sections). You might have multiple objects (and shadows) that get blobbed into one contour!
@@ Line 134: / Line 144: @@
 It's hard for me to imagine a scheme that can deal with this with pure contours -- you'd need to get the threshold exactly right to cleave the cup away from its shadow, and they look pretty close in luminance to me.
-== Distortion ==
+== Contour distortion from camera angle ==
-This is a subtle issue, but it might be the hardest issue.
+This issue is subtle, but it might be the hardest issue to actually fix.
+For the DrawTalking demo and a lot of other purposes, we kind of want a contour that is the 'foot' of the object, the part touching the table or viewport or program underneath. But the contour you get is from the POV of the camera, which is generally overhead and off on the side, so the contour gets this weird position-dependent section of a bunch of the upper part of the object:
 {{.:pasted:20260714-173541.png?200px}}
@@ Line 146: / Line 158: @@
 {{newsletters:img_5752_cropped.mp4}} {{newsletters:screenshot-2026-06-10-at-5.25.34-pm-1815-jpeg-image-320-218-pixels-.png?0x200px}}
-Karl's idea is that during setup/intake, you would draw a manual contour to impose, given the cup center:
+Karl's idea is that during setup/intake, you would draw a manual foot contour to impose, given the cup center:
 {{newsletters:img_5969.jpeg?200px}}
+But how do you get the cup center to do that? You still need to go from distorted contour to center somehow.
+The identity and separation problems from earlier I think are generally addressable with SAM2 (which we'll talk more about next month) or other simple or prebaked techniques. This distortion problem requires some object-specific knowledge: even if you know that it is a cup, you have to know how to undistort it, and you need probably a different policy for each object, or a 3D model to fit pose for each object.
 === Dual camera experiment ===
@@ Line 160: / Line 176: @@
 {{.:pasted:20260714-173451.jpeg?200px}} {{.:pasted:20260714-173522.jpeg?200px}}
-I think this is still viable, but the physical setup is annoying to switch to/from, and it pushes the USB bus hard, and there's this giant shadow. So shelving for now.
+I think this is still viable, but the physical setup is annoying to switch to/from, and it pushes the USB bus hard (the Brio 4K wrecks whatever hub it's connected to), and there's this giant shadow. And the Brio 4K can't do 60fps at 4K! So shelving for now. I want to try some simpler schemes.
 === SAM2 experiments on laptop ===
@@ Line 196: / Line 212: @@
 {{newsletters:screenshot-2026-06-18-at-4.26.26-pm.png?500px}}
+Disadvantages of magic-trace: it needs an explicit trigger + it only runs on literal Intel CPUs (no AMD, no ARM / Raspberry Pi / Orange Pi / Apple Silicon)
 === Mason's cross-thread branch from last year ===
@@ Line 250: / Line 267: @@
 == On Mason's cross-thread ==
-And here are a couple of traces on Mason's cross-thread-object branch:
+Here's a good trace on Mason's branch after I removed tail-call optimization:
-{{newsletters:screenshot-2026-06-18-at-4.13.27-pm-hex-ct6.fxt.gz-12-mb-magic-trace.png?0x200px}} {{newsletters:screenshot-2026-06-18-at-4.48.39-pm-mt3.fxt.gz-141-mb-magic-trace.png?0x200px}}
-Here's a really useful trace on Mason's branch after I removed tail-call optimization:
 {{newsletters:screenshot-2026-06-18-at-5.03.32-pm-hex-ct7-nosib.fxt.gz-12-mb-magic-trace.png}}
 You can see a few things here. Basically, a lot of the shimmering (especially the color map one) does drop out, but the rest of the trace is slower enough that the whole thing is still slower overall.
+The two peaks (each has a JimInvokeCommand at top) representing the two Wishes are still present.
+On the cross-thread branch, each of these Wish calls takes 40ish microseconds:
+{{newsletters:screenshot-2026-06-18-at-4.44.41-pm-hex-ct7-nosib.fxt.gz-12-mb-magic-trace.png?500px}}
+On main, it's pretty similar, but each Wish takes 24ish microseconds:
+{{newsletters:screenshot-2026-06-18-at-4.48.39-pm-mt3.fxt.gz-141-mb-magic-trace.png?500px}}
 > it's weird because structurally main and cross-thread have similar traces, but 1. cross-thread is completely missing the expensive string->dict because it presumably is sharing the color map and other stuff 2. everything in cross-thread is just like 30% slower
@@ Line 265: / Line 288: @@
 Our theory is that the multithreaded reference counting is 2x more expensive which slows down all interpreter operations.
+There are some immediate lessons here for both branches.
+  * You can see that the [[https://github.com/FolkComputer/folk/blob/33e89446f6838288905a127862ccd2c130891be2/prelude.tcl#L401|term-processing for loop in Say]] is actually pretty expensive (the section under ''Jim_ForCoreCommand''). We could probably optimize that loop (use foreach, a switch statement, stuff like that) or even move it to C. Would help a lot in workloads that make a lot of statements.
+  * We could probably speed up the db operations a bit.
+  * The env and source line/file management is also surprisingly slow -- need to deal with this somehow.
 === Zicl update ===
@@ Line 286: / Line 315: @@
 === SVA system fixes ===
-TODO Omar, Andrés, and Brian
+Omar, Andrés, and Brian went in and fixed up the system at SVA in Manhattan.
 {{newsletters:img_5858.jpeg?250px}}
@@ Line 329: / Line 358: @@
   * Daniel will be exhibiting Folk at [[
 https://craftlakecity.com/diy-festival/|Craft Lake City's DIY Festival]] August 7-9 in Salt Lake City. Reach out to him on Discord if you have any questions!
-  * Omar: further push on performance to make the in-system editor really usable
+  * Omar: further push on performance using trace to make the in-system editor really usable
   * Omar: new gadget design; our current one is falling apart, so it might finally be time
-  * Omar: SAM2 object-tracking system, distortion correction
+  * Omar: SAM2 object-tracking system: distortion correction, make some interactions around it
+  * Omar: Pick RFID project back up
+  * Andrés: Fixing [[https://github.com/FolkComputer/folk/pull/272|silent shapes.folk errors]]
 ===== Links we've enjoyed =====
 ==== Andrés ====
-  * TODO
+  * [[https://danielsmith.com/color-map/| Interactive Color
+Map (CIE-Lab)]]
+  * [[https://www.hpl.hp.com/techreports/93/HPL-93-61.pdf|Mark Weiser - Some Computer Science Issues in Ubiquitous Computing (1993)]]
+  * [[https://www.filfre.net/|The Digital Antiquarian - Deep Histories of Analytical & Text-Based Games]]
+  * [[https://cie.co.at/|CIE Color Spaces & Uniformity Guidelines for Vision/UI Ergonomics]]
+  * [[https://www.allaboutcircuits.com/technical-articles/introduction-to-hall-effect-sensors/|Understanding Hall Effect Switches & Magnetic Prototyping Mechanics]]
 ==== Omar ====
   * https://en.wikipedia.org/wiki/Exposure_fusion