Differences

This shows you the differences between two versions of the page.

--- newsletters:2024-06 [2024/06/30 06:58] – [Real-world/3D calibration] osnr
+++ newsletters:2024-06 [2024/07/01 08:37] (current) – osnr
@@ Line 1: / Line 1: @@
-====== June 2024 newsletter (WIP) ======
+====== June 2024 newsletter ======
 ===== What we've been up to =====
@@ Line 60: / Line 60: @@
     * Block out skew parameter of cam/proj intrinsics, can assume that it is always 0 (OpenCV does this so we copy them and assume it's OK)
     * Instead of averaging extrinsics (3D cam->proj rotation and translation) derived from each pose, which doesn't make sense (you can't average rotations), we accumulate a list of point pairs from camera->projector space and then use Kabsch algorithm to get a rotation and translation estimate, then do the nonlinear refinement mentioned above
-    * [[https://social.omar.website/@omar/statuses/01J0F5H0GW4HMG3G6B5RR0TQ9T|"Virtual visual servoing"]] pose estimation. Almost completely fixes weird flicker issues! This was surprisingly huge, and it's a really simple technique
+    * [[https://social.omar.website/@omar/statuses/01J0F5H0GW4HMG3G6B5RR0TQ9T|"Virtual visual servoing"]] pose estimation. Almost completely fixes flicker issues with 3D poses! This was surprisingly huge, and it's a really simple technique
+      * Keeps a tiny bit of state (pose from prev frame) and finds the most likely pose for current frame using tag detection + prev state
+      * Can get stuck in weird poses but fixes itself if you cover & uncover tag, this is tolerable for now I think
+      * I really like that this is both pose estimation and 'smoothing' at the same time, and it has some intrinsic model that this is a thing moving around in space that we're trying to estimate -- wonder if same technique could work for RFID
+        * Wonder if we could also teach it about the square tag instead of doing servoing for each point on its own, would that make it more robust
     * Spent a while on performance micro-optimization, went from like 300ms latency to 200ms. Didn't really help on folk0 (and folk-live at my apartment is not that much faster even though its calibration is way better), so ruling out the performance theory of why folk0 doesn't work as well
     * Spent a while on making a separate correct-extrinsics step that you do at the end of calibration, where you leave the board flat on your table and it reprojects tags and measures the offset and tries to correct the extrinsics:
@@ Line 81: / Line 85: @@
 I've also made it almost fully backward-compatible with all existing code -- it now derives 2D projector-space region claims from the 3D calibration, so all code that uses regions still works. I deleted all the old 2D calibration and homography and region code (calibrate.tcl), so 3D calibration backs everything on the branch.
-I'd like to get it to at least be on par with the existing 2D calibration in most cases before we merge it, and we're not there yet...
+I'd like to get it to at least be on par with the existing 2D calibration in most cases before we merge it, and we're not there yet... (working theory is still that the individual intrinsics of camera and projector are derived pretty well, but the extrinsics seem off)
 ----
@@ Line 180: / Line 184: @@
 ===== What we'll be up to in July =====
-  * **Our next [[https://partiful.com/e/U0A3JFfs0w9mgzynBcrt|Folk open house is in the afternoon of Tuesday, July 30th]] at our studio in East Williamsburg, Brooklyn.**
+  * **Our next [[https://partiful.com/e/U0A3JFfs0w9mgzynBcrt|Folk open house is on the night of Saturday, July 27]] at our studio in East Williamsburg, Brooklyn.**
   * Getting 3D calibration to work well on all systems, then merging it.......
   * Figuring out retraction and transaction behavior for parallel evaluator; testing it more
   * Gadget iteration (need to fix swiveling, fix power issues, improve software)
   * Video support
-  * Documentation (language reference, new kits and printable worksheets for setting up and learning the Folk language)
+  * Documentation (language reference, new kits and printable worksheets for setting up and learning the Folk language), open up GitHub repo
@@ Line 208: / Line 212: @@
   * [[https://www.e-ink-info.com/tags/transparency|printable mini e-ink displays with their own batteries]]
   * [[https://www.instagram.com/reel/C8a3g27ppHM/|The first few art objects in this video]] remind me of large, polished, but simplistic versions of Kevin Kwok's e-ink crank
+  * [[https://www.instagram.com/reel/C7Vu5sDOzHj/?igsh=MTgwOWo2bjVrNzNjcA==|Fascinating multi-colored stamp]] — watching this over and over, it’s inspiring a bunch of ideas around hand stamped fiducial tools :)