newsletters:2026-06
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| newsletters:2026-06 [2026/07/16 20:02] – [A specific trace of quad masking] osnr | newsletters:2026-06 [2026/07/18 13:30] (current) – [SVA system fixes] osnr | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== June 2026 newsletter ====== | ====== June 2026 newsletter ====== | ||
| + | |||
| {{htmlmetatags> | {{htmlmetatags> | ||
| - | metatag-media-og: | + | metatag-media-og: |
| - | metatag-og: | + | metatag-og: |
| }} | }} | ||
| Our next Folk open house will be in the evening on **[[https:// | Our next Folk open house will be in the evening on **[[https:// | ||
| + | |||
| + | Also: [[https:// | ||
| + | https:// | ||
| ===== What we've been up to ===== | ===== What we've been up to ===== | ||
| ==== General system improvements ==== | ==== General system improvements ==== | ||
| Line 22: | Line 26: | ||
| === New drawing API === | === New drawing API === | ||
| - | Andrés: | + | Andrés: |
| - | + | ||
| - | Units on everything. | + | |
| - | Error reporting requires | + | * Wish $this draws a circle |
| + | * Wish $this draws a circle with color gold radius 20mm | ||
| + | * Wish $this draws a circle with color magenta radius 20cm position [list x 20% 50%] | ||
| + | * and you get this: | ||
| + | * {{.:pasted: | ||
| - | {{newsletters: | + | We enforce units (mm, cm, m, %) on everything in the new drawing system which is nice as it makes it easy to tell, using your real-world brain, where the shapes you’re drawing are going to land. |
| + | * Error reporting requires you to look at the shapes.folk log on the web dashboard right now: | ||
| + | * {{newsletters: | ||
| - | Causality TODO | + | * By the end of July we should have [[https:// |
| === ShaderToy compatibility === | === ShaderToy compatibility === | ||
| Line 40: | Line 48: | ||
| {{.: | {{.: | ||
| - | We've had an [[https:// | + | We've had an [[https:// |
| ==== Fields + local feedback calibration / nudging experiment ==== | ==== Fields + local feedback calibration / nudging experiment ==== | ||
| Line 120: | Line 128: | ||
| First, you can see how contours all change color from frame to frame -- we can't identify that a contour is 'the same' object from frame to frame without some additional logic. | First, you can see how contours all change color from frame to frame -- we can't identify that a contour is 'the same' object from frame to frame without some additional logic. | ||
| - | You could imagine tricks for this: we could compute an average color or color distribution for each contour, and we could use that to distinguish a blue cup contour from a red stapler contour (identity by color) (that also might be a nice visualization anyway). We could look at the contour shape (identity by shape) (although this runs into rotation and the distortion problem described below). We could find the closest contour from previous frame and identify it with new contour in the new frame (so identity by location/ | + | You could imagine tricks for this: we could compute an average color or color distribution for each contour, and we could use that to distinguish a blue cup contour from a red stapler contour (identity by color) (that also might be a nice visualization anyway). We could look at the contour shape (identity by shape) (although this runs into rotation and the distortion problem described below). We could find the closest contour from previous frame and identify it with new contour in the new frame (so identity by location/ |
| - | == Contour failures | + | But the point is, it's not something you get for free from the contour detector (unless you assume you're looking for exactly 1 object in the frame or something). |
| + | |||
| + | == Distinct contours not corresponding with distinct objects | ||
| Contours are traced from a binarized image, so all dark areas that are adjacent get merged into the same contour. You might have one object that accidentally splits into multiple contours (if it has light sections). You might have multiple objects (and shadows) that get blobbed into one contour! | Contours are traced from a binarized image, so all dark areas that are adjacent get merged into the same contour. You might have one object that accidentally splits into multiple contours (if it has light sections). You might have multiple objects (and shadows) that get blobbed into one contour! | ||
| Line 134: | Line 144: | ||
| It's hard for me to imagine a scheme that can deal with this with pure contours -- you'd need to get the threshold exactly right to cleave the cup away from its shadow, and they look pretty close in luminance to me. | It's hard for me to imagine a scheme that can deal with this with pure contours -- you'd need to get the threshold exactly right to cleave the cup away from its shadow, and they look pretty close in luminance to me. | ||
| - | == Distortion | + | == Contour distortion from camera angle == |
| + | |||
| + | This issue is subtle, but it might be the hardest issue to actually fix. | ||
| - | This is a subtle issue, but it might be the hardest issue. | + | For the DrawTalking demo and a lot of other purposes, we kind of want a contour that is the ' |
| {{.: | {{.: | ||
| Line 146: | Line 158: | ||
| {{newsletters: | {{newsletters: | ||
| - | Karl's idea is that during setup/ | + | Karl's idea is that during setup/ |
| {{newsletters: | {{newsletters: | ||
| + | |||
| + | But how do you get the cup center to do that? You still need to go from distorted contour to center somehow. | ||
| + | |||
| + | The identity and separation problems from earlier I think are generally addressable with SAM2 (which we'll talk more about next month) or other simple or prebaked techniques. This distortion problem requires some object-specific knowledge: even if you know that it is a cup, you have to know how to undistort it, and you need probably a different policy for each object, or a 3D model to fit pose for each object. | ||
| === Dual camera experiment === | === Dual camera experiment === | ||
| Line 160: | Line 176: | ||
| {{.: | {{.: | ||
| - | I think this is still viable, but the physical setup is annoying to switch to/from, and it pushes the USB bus hard, and there' | + | I think this is still viable, but the physical setup is annoying to switch to/from, and it pushes the USB bus hard (the Brio 4K wrecks whatever hub it's connected to), and there' |
| === SAM2 experiments on laptop === | === SAM2 experiments on laptop === | ||
| Line 196: | Line 212: | ||
| {{newsletters: | {{newsletters: | ||
| + | Disadvantages of magic-trace: | ||
| === Mason' | === Mason' | ||
| Line 256: | Line 273: | ||
| You can see a few things here. Basically, a lot of the shimmering (especially the color map one) does drop out, but the rest of the trace is slower enough that the whole thing is still slower overall. | You can see a few things here. Basically, a lot of the shimmering (especially the color map one) does drop out, but the rest of the trace is slower enough that the whole thing is still slower overall. | ||
| - | The two peaks (each has a JimInvokeCommand at top) representing the two Wishes are still present. You can see that the [[https:// | + | The two peaks (each has a JimInvokeCommand at top) representing the two Wishes are still present. |
| On the cross-thread branch, each of these Wish calls takes 40ish microseconds: | On the cross-thread branch, each of these Wish calls takes 40ish microseconds: | ||
| Line 271: | Line 288: | ||
| Our theory is that the multithreaded reference counting is 2x more expensive which slows down all interpreter operations. | Our theory is that the multithreaded reference counting is 2x more expensive which slows down all interpreter operations. | ||
| + | |||
| + | There are some immediate lessons here for both branches. | ||
| + | |||
| + | * You can see that the [[https:// | ||
| + | * We could probably speed up the db operations a bit. | ||
| + | * The env and source line/file management is also surprisingly slow -- need to deal with this somehow. | ||
| === Zicl update === | === Zicl update === | ||
| Line 292: | Line 315: | ||
| === SVA system fixes === | === SVA system fixes === | ||
| - | TODO Omar, Andrés, and Brian | + | Omar, Andrés, and Brian went in and fixed up the system at SVA in Manhattan. |
| {{newsletters: | {{newsletters: | ||
| Line 335: | Line 358: | ||
| * Daniel will be exhibiting Folk at [[ | * Daniel will be exhibiting Folk at [[ | ||
| https:// | https:// | ||
| - | * Omar: further push on performance to make the in-system editor really usable | + | * Omar: further push on performance |
| * Omar: new gadget design; our current one is falling apart, so it might finally be time | * Omar: new gadget design; our current one is falling apart, so it might finally be time | ||
| - | * Omar: SAM2 object-tracking system, distortion correction | + | * Omar: SAM2 object-tracking system: distortion correction, make some interactions around it |
| + | * Omar: Pick RFID project back up | ||
| + | * Andrés: Fixing [[https:// | ||
| ===== Links we've enjoyed ===== | ===== Links we've enjoyed ===== | ||
| ==== Andrés ==== | ==== Andrés ==== | ||
| - | * TODO | + | * [[https:// |
| + | Map (CIE-Lab)]] | ||
| + | * [[https:// | ||
| + | * [[https:// | ||
| + | * [[https:// | ||
| + | * [[https:// | ||
| ==== Omar ==== | ==== Omar ==== | ||
| * https:// | * https:// | ||
newsletters/2026-06.1784232158.txt.gz · Last modified: by osnr