Differences

This shows you the differences between two versions of the page.

--- newsletters:2025-12 [2026/01/17 10:12] – [Statement reuse bug fix (fixes editor freeze)] osnr
+++ newsletters:2025-12 [2026/01/17 23:48] (current) – [folk2 launch party] osnr
@@ Line 39: / Line 39: @@
   * [[https://github.com/FolkComputer/folk/commit/4185bf4936894986aac65d5ff66d619e089b174d|Don't reproject during calibration if it's gonna go outside the projector area]] (and, likely, get stuck that way, since you can't then detect it and iterate to a new pose)
   * Scheduler now scales number of Folk worker threads up to roughly the CPU count, to make better use of bigger Folk PC we have (the scheduler probably still needs a lot of work to be robust against a lot of different workloads)
-  * Daniel [[https://github.com/FolkComputer/folk/commit/83479b1315926c6ab4aeda2e83ae080a5e1bfcb5|fixed printer forwarding]] and program forwarding so we could print from downstairs systems through folk-hex and use those programs on all systems in the room
+  * Daniel [[https://github.com/FolkComputer/folk/commit/83479b1315926c6ab4aeda2e83ae080a5e1bfcb5|fixed printer forwarding]] and program forwarding so we could print from downstairs systems through the upstairs folk-hex system (and use those programs on all systems in the room after printing)
+    * {{.:pasted:20260117-151940.jpeg?300px}}
   * Improved "Printed!" and "Saved!" notices
 === Vulkan memory allocation ===
-As we tested the system before the party, we noticed huge amounts of blinking when you had a bunch of camera slices / animation out. And the blinking got worse and worse over time. I profiled the system and saw that we would be spending tens of milliseconds on GPU memory allocation of new textures for the camera slices / animations.
+As we tested the system before the party, we noticed huge amounts of blinking when you had a bunch of camera slices / animation / sprites out.
-(which is a form of workload that Folk really struggles with; we can cope with a long-running task, and we can do lots of short tasks, but lots of medium-length tasks is really hard for us. The blinking is a symptom of underlying slowdown / bottleneck in the evaluator, where we're missing deadlines to respond to things because we're spending too much compute time elsewhere)
+{{newsletters:img_7263.mp4?250px}}
-Well, memory allocation shouldn't take that long. The problem was that I had hand-rolled the memory allocation, which you're really actually not supposed to do. Vulkan doesn't defragment or anything, so your allocation performance just gets worse and worse over time if you're making lots of little allocation (it's fragmented, has to walk to find new areas, whatever).
+And the blinking got worse and worse over time. I profiled the system and saw that we would be spending milliseconds up to tens of milliseconds on each GPU memory allocation of a new texture for the camera slices / animations / sprites. See how long the gpu/textures.folk:669 blocks take here:
+{{newsletters:image-alloc.png?500px}}
+{{.:pasted:20260117-231138.png?500px}}
+(this is a form of workload that Folk really struggles with; we can cope with a long-running task, and we can do lots of short tasks, but lots of medium-length tasks is really hard for us. The blinking is a symptom of underlying slowdown / bottleneck in the evaluator, where we're missing deadlines to respond to things because we're spending too much compute time elsewhere)
+Well, memory allocation shouldn't take that long. The problem was that I had hand-rolled the memory allocation, which you're really not supposed to do. Vulkan doesn't defragment or anything, so your allocation performance just gets worse and worse over time if you're making lots of little allocations (it's fragmented, has to walk to find new areas, whatever).
+[[https://github.com/FolkComputer/folk/compare/f508b523...c559c8ce|I switched to use the Vulkan Memory Allocator library]] that [[https://old.reddit.com/r/vulkan/comments/u076gp/is_it_a_good_idea_to_use_vma_for_memory/|everyone]] [[https://stackoverflow.com/questions/74814762/how-to-deal-with-memory-fragmentation-in-vulkan#comment132036100_74814762|recommends]], and this immediately fixed everything, and now performance of camera slices and animations is great and doesn't degrade at all.
+The best part was that I [[https://github.com/FolkComputer/folk/commit/c559c8ced5cd6bab747883e2efb1dd220b9f8432|made this work at like 1PM on the day of the merge party]] -- a really amazing moment before the party (when we had been worried that the system just wouldn't work and now it was all magically working).
+{{.:pasted:20260117-102045.png?500px}}
+{{newsletters:small_pxl_20251212_181546070_20251219233938.mp4?500px}}
-I switched to use the Vulkan Memory Allocator library that everyone recommends, and this immediately fixed everything, and now performance of camera slices and animations is great and doesn't degrade at all.
 ==== /setup page ====
@@ Line 62: / Line 78: @@
 === JPEG frames ===
-TODO: Decompress and do AprilTag detect, making the normal Folk pipeline work again.
+We started on using JPEG frames in the system for /setup / live preview last month. This month, we added decompress and AprilTag detect steps on top, making the normal Folk pipeline work again.
 [[https://github.com/Rooprob|Rob Fielding]] suggested moving toward claiming decompressed color frames in the system as well. Currently, the system only exposes the decompressed grayscale channel (which is all you need for AprilTag detection). By using the raw JPEG data from webcams, we can potentially allow color use cases to coexist with grayscale detection without redundant decompression/compression cycles.
@@ Line 123: / Line 139: @@
 Whenever I got the editor freeze bug, I'd see that that Hold! block was executing, and most of the statements from the block were in the database, but one was missing & would never show up as a result to any queries (so we'd be lacking a ''max cursor x'' or ''view dimensions'' or ''cursor position'' statement, which meant the editor Whens wouldn't fire, breaking the editor).
+I walked the database trie by hand and confirmed that the ''has max cursor x'' statement wasn't there at all (there's no ''max'' branch under ''has''):
+{{.:pasted:20260117-231251.png?580px}}
 This made me feel that there was some really bad race in the evaluator (this behavior violates what I would think of as core Folk invariants), so I did more digging.
@@ Line 158: / Line 178: @@
 ==== Editor refactor ====
-Mason did a big [[https://github.com/FolkComputer/folk/pull/240|editor refactor]] that removes traces of the old "code view" terminology, cleans up the state, and makes the line numbers render more cleanly. Scroll also works better (retains more context)
+Mason did a big [[https://github.com/FolkComputer/folk/pull/240|editor refactor]] that removes traces of the old "code view" terminology, cleans up the state, and makes the line numbers render more cleanly (no more snapping around when you go from 1-digit to 2-digit line numbers + dynamic left margin so the line numbers don't take up a tiny or huge amount of space on extreme zoom). Scroll also works better (retains more context).
-This mostly came out of us trying to fix a bug where the scroll viewport was weirdly small on editor reboot (due to some race in the old way we were doing state).
+This mostly came out of a bug where the scroll viewport was weirdly small on editor reboot (due to some race in the old way we were doing state). The refactor fixes this bug.
+{{newsletters:img_7668.mp4?400px}}
+Some other funny line number situations we saw (and fixed) while doing the refactor:
+{{.:pasted:20260117-232906.jpeg?300px}} {{newsletters:img_7924.jpeg?300px}}
+Now the editor feels pretty solid and comfortable across zooms, restarts, etc.
 ==== Folk for DnD ====
@@ Line 173: / Line 200: @@
 {{.:pasted:20260105-142658.png?450px}}
+The branch works okay in small tests, but slows down the system a lot when I run the whole thing.
 === Richer captures ===
@@ Line 191: / Line 219: @@
 I want to clean it up and do some more stuff (maybe texture the quads with the camera slices of each quad). It'd be cool for this to be a livestream UI with these 'computational objects' in the stream for all viewers to interact with.
+==== New gadget1 ====
+Omar: I printed a new (pink!) gadget1 to bring to China. This one uses top/bottom threading for the front panel so we don't need space-eating wells to screw the front panel into.
+{{.:pasted:20260117-232319.jpeg?250px}}
+It also now has a bottom hole for the trigger button, so we can do trigger button stuff on the gadget1 as well. And it supports dual Pi cameras to mount and to stick through front panel.
+{{.:pasted:20260117-231800.png?300px}}
+New (left) and old (right):
+{{newsletters:img_8057.jpeg?250px}}
+I've just been doing each one by hand, but it feels like we now have a number of implicit variants:
+  * Battery (gadget2) vs. no-battery (gadget1)
+  * Orange Pi (gadget2) vs. Raspberry Pi (gadget1)
+    * retail AnyBeam (gadget2) vs. Ultimems devkit (gadget1)
+      * (the retail AnyBeam requires USB-C video in, so needs the Orange Pi)
+      * (it is somewhat more stable to mount the devkit because it has multiple threaded holes underneath, though)
+  * ELP stereo camera (gadget2) vs. dual Raspberry Pi cameras (gadget1)
+It would be nice to be able to mix and match these attributes.
+I haven't yet updated the repo with this info or with the new CAD model, but I also figured out the pins to plug the trigger button into on the gadget1's Raspberry Pi (as opposed to the gadget2's Orange Pi).
+{{.:pasted:20260117-231948.jpeg?300px}}
+See how it changes output when I press the trigger button:
+{{.:pasted:20260117-231935.png?500px}}
 ==== Experimental tcl interpreter update ====
@@ Line 205: / Line 267: @@
 ==== Outreach ====
-Our friend [[https://a9.io|Max Krieger]] visited:
+Our friends [[https://a9.io|Max Krieger]] and [[https://www.tophtucker.com|Toph Tucker]] visited:
-{{newsletters:img_6965.jpeg?300px}}
+{{newsletters:img_6965.jpeg?300px}} {{newsletters:img_7229.jpeg?300px}} {{newsletters:img_7232.jpeg?300px}}
 === folk2 launch party ===
-We had a [[https://partiful.com/e/Y5L66oPibgDt4PcRQptx|launch party for folk2]] on Friday, Dec 12. It was very fun. [[https://github.com/brianshl007|Brian Lee]] helped out, and Daniel came in from Utah to help out as well. Was a good way to bring people in who hadn't seen Folk in a while, announce nonprofit and folk2 updates, and force ourselves to make progress on public-facing niceties in the Folk system. (We probably made a couple months' worth of progress on little nits in the week before the party.)
+We had a [[https://partiful.com/e/Y5L66oPibgDt4PcRQptx|launch party for folk2]] on Friday, Dec 12. It was very fun. [[https://github.com/brianshl007|Brian Lee]] helped out, and Daniel came in from Utah to help out as well.
+{{newsletters:img_7318.jpeg?0x300px}} {{newsletters:img_7322.jpeg?0x300px}}
+{{newsletters:img_7261.mp4}}
+Was a good way to bring people in who hadn't seen Folk in a while, announce nonprofit and folk2 updates, and force ourselves to make progress on public-facing niceties in the Folk system. (We probably made a couple months' worth of progress on little nits in the week before the party.)
 Omar made some programs to do the merge live on stage:
@@ Line 226: / Line 294: @@
 {{newsletters:board.jpeg?400px}}
-Omar and Andrés gave a short talk about their work on folk2 and the hopes they have for the future of the project. Afterward Daniel talked about the non-profit and funding this work:
+Omar and Andrés gave a short talk about their work on folk2 and the hopes they have for the future of the project. We had a /camera livestream of a table (via the Folk Web sevrer) projected on the screen, to show the actual merge and to show what people were doing with the system:
+{{newsletters:img_7327.jpeg?300px}}
+Afterward Daniel talked about the non-profit and funding this work:
-{{newsletters:daniel_speech.jpeg?400px}}
+{{newsletters:daniel_speech.jpeg?0x300}} {{newsletters:img_7366.jpeg?0x300}}
 === Folk gadgets at SVA ===