Differences

This shows you the differences between two versions of the page.

--- newsletters:2026-07 [2026/08/17 18:55] – Remove (words), fix timecodes, order of DL laser link cwervo
+++ newsletters:2026-07 [2026/08/18 17:21] (current) – [Correction experiments] osnr
@@ Line 121: / Line 121: @@
 === New viewport ===
-Dynamic viewport issue
+Last month, we worked on a dynamic viewport where you move the two corners (gold) to resize a larger-than-page-sized viewport (green):
-Acrylic sheet, put tag on
+{{newsletters:img_6439.jpeg?200px}}
+This dynamic viewport has issues with actually getting it on the right plane so you can accurately projection-map it. (Do you take the plane from one of the corner programs? They'll have subtly different planes, like a couple millimeters variation in pose between the two of them. Do you average the corner programs? Do you try to find a plane underneath both? Do you preregister the table plane at calibration time and downproject onto it?)
+{{newsletters:screenshot-2026-07-10-at-12.56.17-pm.png?300px}}
+This month, we've been prototyping with a big acrylic viewport with tag in corner and manually enlarged geometry (on left).
+{{.:pasted:20260818-170321.jpeg?300px}}
+This works pretty well and is about the right size that Karl wants. We should be able to go back to the old dynamic viewport later, but punting on it for now.
 === SAM2 + contour tracing ===
@@ Line 220: / Line 230: @@
 This is a pretty high-level API, although you still need to think about camera images/frames/slices and pixel locations (mostly because a camera frame also reflects a specific time that your prompt is from). But it's nice that it hides most of the track loop from you and lets you think in terms of objects semantically.
+(We made APIs on top that you can invoke that give you contour or foot contour or center, which have more geometric information than the mask. You could imagine ultimately getting a rough quad for the object, so you can projection map it or do points-at / other queries, or maybe we generalize those so they work with contours.)
 === Physical interface for multiple object registration ===
@@ Line 245: / Line 257: @@
 {{.:pasted:20260813-162034.png?400px}}
+All these pages have a 1-1 correspondence between registration page and object. You need to print multiple registration pages to track multiple objects in parallel. We might change this soon. (It is nice because you can just make the object ID from the registration page ID and point to the registration page to clear/hold state.)
 === Web UIs ===
 We have a few interesting web debugging UIs for SAM2 tracking.
-SAM2 per-object mask views:
+SAM2 per-object mask views, so you can see what SAM2 has locked onto if anything:
 {{newsletters:screenshot-2026-07-10-at-3.34.23-pm.png?300px}}
-Camera slice /camera-slice/VIEWPORT view -- this is what the camera sees, rewarped to make the viewport rectangular:
+Camera slice /camera-slice/ID view -- this is what the camera sees, rewarped to make the object ID you're slicing rectangular and axis-aligned (two slices side-by-side here: the viewport and a blank page inside the viewport):
-{{newsletters:screenshot-2026-07-10-at-3.56.20-pm.png?300px}}
 {{newsletters:screenshot-2026-07-24-at-2.50.55-pm.png?600px}}
@@ Line 284: / Line 294: @@
 == Camera-facing arc experiment ==
-Omar had the idea that we just look at where the object is relative to the camera and exclude the side that's likely to be distorted.
+Omar had the idea that we just look at where the object is relative to the camera and exclude the side that's likely to be distorted. Here, the camera is above and to the right of the viewport, so we cast rays from the camera to the convex hull of the contour of the object and take that green arc between the two tangent points:
 {{newsletters:screenshot-2026-07-24-at-5.11.38-am.png?400px}}
 {{newsletters:img_6973.mp4?350px}}
+That should get us the 'side' of the contour that is touching the viewport plane (should be unaffected by moving the cup further away or to different sides of the camera, unlike just finding centroid of contour), and then we fit a circle of the right diameter to it? More on this next month.
 === Issues ===