Differences

This shows you the differences between two versions of the page.

--- newsletters:2024-09 [2024/10/01 03:29] – [New parallel evaluator] osnr
+++ newsletters:2024-09 [2024/10/01 03:47] (current) – [New parallel evaluator] osnr
@@ Line 5: / Line 5: @@
   * Andrés will be giving the opening keynote at [[https://berlin.causalislands.com/#program|Causal Islands Berlin]] on this Friday, October 4.
     * The talk will use a Folk gadget to control slides, demonstrate Folk, and explain the technology and vision behind Folk. We'll link the recording of the talk in the newsletter next month.
-    * It looks like [[https://ti.to/causal-islands/berlin-2024|livestream tickets are also available]] if you're interested
+    * It looks like [[https://ti.to/causal-islands/berlin-2024|livestream tickets are also available]] for anyone interested
+  * Our next [[https://partiful.com/e/62C2MjzYBVIpLP7qG330|Folk open house is on the night of Monday, October 21]], at our studio in East Williamsburg, Brooklyn.
 ==== Applications and demos ====
@@ Line 59: / Line 60: @@
     * + tell libcamera to configure (reduce) camera exposure time so that projected tags for calibration don't get washed out
-These fixed the worst issues -- it now actually calibrates! We can project onto programs!
+These fixed the worst issues -- it now actually calibrates! We can project onto programs! (it's still fairly annoying to calibrate, takes 20-30 minutes, but it's doable, and you ideally only have to do it once. I feel like we should ultimately be able to calibrate it really well because the projector is a laser scanning projector and has no lens distortion, also.)
 First calibrated use of Omar's gadget (gadget-blue) and of Andrés's gadget (gadget-red), respectively:
@@ Line 96: / Line 97: @@
 === Sysmon and thread pool ===
-Omar added workqueue display on threads Web page, which made it clear that many of the random issues are just that work-stealing was breaking down and work items would get stuck on some thread that was running a permanent task.
+Started working on sysmon thread which wakes up every few milliseconds and manages the size of the thread pool (too many threads looking for work? kill some; too few threads compared to number of CPUs? spawn some).
+There are subtleties here where you want to avoid churning the thread pool and constantly killing and re-spawning stuff. Haven't figured out yet. It needs heuristics about what threads are occupied and how long they'll be occupied (microseconds? milliseconds? seconds?) to determine this.
 === Sustain / time-to-live field on statements or Holds ===
@@ Line 102: / Line 105: @@
 Idea: sustain/TTL field where statements can get held for some number of milliseconds after their parent is retracted.
-sysmon is extended (maybe will rename to custodian) to also handle reaping sustained statements when they hit their deadline
+sysmon (which previously was just managing the size of the thread pool) is extended (maybe will rename to custodian) to also handle reaping sustained statements when they hit their deadline. it feels nice to do it here rather than complicating the priority queue further, since sysmon is already waking up every couple milliseconds anyway (and it doesn't need to run any more than that, fixed ticks are fine).
-We had a discussion about this proposal in Discord:
+We had a good discussion about this proposal in Discord:
-> Omar: when you make a When (or a statement) in parallel Folk, we might have an option for "sustain" or "time-to-live" where you can be like, I want this statement to stick around for 1ms (or 10ms or 100ms) even if its parent is retracted, in case it comes back in on the next frame
+{{:newsletters:pasted:20241001-034059.png?550px}}
->
-> just thinking about anti-blink measures. it wouldn't surprise me if we need more little hints like this to make the system work
+<details>
->
+<summary>More discussion of sustain/TTL</summary>
->
->> Is there a particular context where this came up?
+{{:newsletters:pasted:20241001-034119.png?550px}}
-> it's pretty general -- you'll see page outlines blink out in the new evaluator, because they get retracted because the old camera frame is retracted & that is a race against the new camera frame getting processed (and if the retract happens before the page gets re-added due to the new frame, it blinks out)
->
+{{:newsletters:pasted:20241001-034138.png?550px}}
-> the specific one that just came up is that we were exec-ing some keymap command a lot because the keyboard page kept blinking in and out (downstream of the above)
+</details>
->
-> we also do this de facto in single-threaded Folk right now by ordering retractions after new assertions in the priority queue
->
->> Why does the old frame get retracted?
-> retracting old statements is part of the core loop of the system -- you can kind of think of the camera subsystem as a while loop that just goes like this
-> <code>set prevFrame {}
-while true {
-   set frame [camera readFrame] ;# blocking, takes 16ms
-   Assert the camera frame is $frame
-   Retract the camera frame is $prevFrame
-   set prevFrame $frame
-}</code>
-> and then when the Assert happens, the new set of active pages is calculated, and the new outlines of all pages are calculated, and then the shader display list is calculated, etc, and that's all in place
->
-> and then the Retract happens and rips out the old set of active pages (but any pages that are still on the table are now supported by the new Assert so they don't blink out)
->
-> (and then only once the single-threaded evaluator has converged and all tasks are complete, after all of this, the final display list is rendered on the GPU, so you don't see the transient retracted old statements)
->> And in the parallel case Retract is potentially happening before the set of active pages are reactively set as a result of the Assert?
-> Yes, exactly -- it's a race since those operations are parallel-scheduled in the new evaluator
->> So the TTL could live in the camera frame claim or maybe the tag detector to make detected programs stickier. I think I got it. There is something that feels a little wrong about putting a time on it instead of declaring some sort of dependency, but I get it.
-> Yeah
-> I'm hoping that we would only need to use them internally in a couple specific places
-> and the end user programming experience would broadly be the same
->
-> hmm yeah i think the problem is that you don't really want to introduce a hard dependency because then programs can arbitrarily block the control loop if they're slow (which is sort of the situation we have in current single-threaded Folk)
->
-> it feels like at some level, you do actually want to say, hey, if you can't resolve within a few milliseconds, we're just gonna go ahead and continue with the next frame
->
-> anyway this (blinking) remains an open problem so i'm gonna try it when i get the chance and see how it feels
->
->> This is a really good point. We would want a timeout on a lock anyway.
 === Other new evaluator stuff ===
-fixed some keyboard issues
+Added workqueue display on /threads Web page, which made it clear that many of the random issues with folk2 are just that work-stealing was breaking down and work items would get permanently stuck on some thread that was stuck running a permanent task (instead of the work getting stolen and executed elsewhere).
+Fixed some keyboard issues where the keyboard process was erroring or exec-ing stuff all the time (because of lack of persistence / because of blinking), and where the grabber was broken because it wasn't inter-thread-safe.
+Scheming about this memory management idea again, since it could simplify a lot of the implementation:
+{{:newsletters:pasted:20241001-033956.png?550px}}
+<details>
+<summary>More discussion of memory management</summary>
+{{:newsletters:pasted:20241001-034016.png?550px}}
+</details>
 ==== Friends and outreach ====
@@ Line 163: / Line 146: @@
   * **Our next [[https://partiful.com/e/62C2MjzYBVIpLP7qG330|Folk open house is on the night of Monday, October 21]], at our studio in East Williamsburg, Brooklyn.**
-  * Andrés will give the talk at Causal Islands Berlin
+  * Andrés will give their talk about Folk at Causal Islands Berlin
     * Maybe some tasks afterward around merging the slides system they built and following up with people
   * Andrés continuing docs and desksaver projects
+  * Andrés and Omar will work with our new partner [[https://www.kosmik.app|Kosmik]] on knowledge management and phone/laptop integrations and demos with Folk
   * Daniel continuing to work on hand tracking now that the Kinect data is in Folk
   * Omar wants to respin the gadget to make the front panel more solid, maybe add a second camera, & is meeting with Ultimems (the mini projector supplier)