Differences

This shows you the differences between two versions of the page.

--- newsletters:2026-07 [2026/08/18 17:03] – [New viewport] osnr
+++ newsletters:2026-07 [2026/08/18 17:21] (current) – [Correction experiments] osnr
@@ Line 230: / Line 230: @@
 This is a pretty high-level API, although you still need to think about camera images/frames/slices and pixel locations (mostly because a camera frame also reflects a specific time that your prompt is from). But it's nice that it hides most of the track loop from you and lets you think in terms of objects semantically.
+(We made APIs on top that you can invoke that give you contour or foot contour or center, which have more geometric information than the mask. You could imagine ultimately getting a rough quad for the object, so you can projection map it or do points-at / other queries, or maybe we generalize those so they work with contours.)
 === Physical interface for multiple object registration ===
@@ Line 255: / Line 257: @@
 {{.:pasted:20260813-162034.png?400px}}
+All these pages have a 1-1 correspondence between registration page and object. You need to print multiple registration pages to track multiple objects in parallel. We might change this soon. (It is nice because you can just make the object ID from the registration page ID and point to the registration page to clear/hold state.)
 === Web UIs ===
 We have a few interesting web debugging UIs for SAM2 tracking.
-SAM2 per-object mask views:
+SAM2 per-object mask views, so you can see what SAM2 has locked onto if anything:
 {{newsletters:screenshot-2026-07-10-at-3.34.23-pm.png?300px}}
-Camera slice /camera-slice/VIEWPORT view -- this is what the camera sees, rewarped to make the viewport rectangular:
+Camera slice /camera-slice/ID view -- this is what the camera sees, rewarped to make the object ID you're slicing rectangular and axis-aligned (two slices side-by-side here: the viewport and a blank page inside the viewport):
-{{newsletters:screenshot-2026-07-10-at-3.56.20-pm.png?300px}}
 {{newsletters:screenshot-2026-07-24-at-2.50.55-pm.png?600px}}
@@ Line 294: / Line 294: @@
 == Camera-facing arc experiment ==
-Omar had the idea that we just look at where the object is relative to the camera and exclude the side that's likely to be distorted.
+Omar had the idea that we just look at where the object is relative to the camera and exclude the side that's likely to be distorted. Here, the camera is above and to the right of the viewport, so we cast rays from the camera to the convex hull of the contour of the object and take that green arc between the two tangent points:
 {{newsletters:screenshot-2026-07-24-at-5.11.38-am.png?400px}}
 {{newsletters:img_6973.mp4?350px}}
+That should get us the 'side' of the contour that is touching the viewport plane (should be unaffected by moving the cup further away or to different sides of the camera, unlike just finding centroid of contour), and then we fit a circle of the right diameter to it? More on this next month.
 === Issues ===