Differences

This shows you the differences between two versions of the page.

--- newsletters:2025-12 [2025/12/29 00:32] – Edit link bullets cwervo
+++ newsletters:2025-12 [2026/01/17 23:48] (current) – [folk2 launch party] osnr
@@ Line 1: / Line 1: @@
-====== December 2025 newsletter (WIP) ======
+====== December 2025 newsletter ======
+Our next [[https://luma.com/49d1t0ng|Folk open house is in the evening on Thursday, January 29th, in East Williamsburg, Brooklyn.]]
 ===== What we've been up to =====
@@ Line 5: / Line 7: @@
 ==== Nonprofit 501(c)3 ====
-We are now officially a 501c3 nonprofit. This means we can accept tax-deductible donations and are eligible to apply for grants.
+[[https://danielpipk.in|Daniel Pipkin]]: We are now officially a 501c3 nonprofit. This means we can accept tax-deductible donations and are eligible to apply for grants.
 Folk is a long-term project. Many people want to work on it and even more people want to support its growth and development. When thinking of the possible organizational structures that could sustain this development, we decided that a nonprofit was the best fit. It allows us to continue providing something valuable without needing to rope off parts of that value behind a paywall. We want as many people as possible to be able to participate in this exciting new computing paradigm.
@@ Line 18: / Line 20: @@
 ==== folk2 merge ====
-We've merged the new version of folk into the main branch.
+Omar: We've [[https://github.com/FolkComputer/folk/pull/217|merged the new version of folk into the main branch.]]
-It's a nice feeling that the changes we make are finally going to be visible on the main repo and get used by people.
+{{newsletters:e42d1d59-4378-4d94-879c-161ad455a4bc-26887-0000149b61e49e34.png?500px}}
-folk2 isn't quite caught up (a lot of features are still not implemented, and documentation isn't great), but we weren't really supporting folk1 anyway (we wouldn't want anyone to submit a patch for it, for instance), and folk2 is so much faster that I think we want it to be the default experience for people.
+It's a nice feeling that the work we've been doing is finally going to be visible on the main repo and get used by people out of the box.
+Along with the nonprofit news, I think we want to use this as a trigger to enter a new, more public phase where we do more applications and demos (and work more on fundraising).
+folk2 isn't quite caught up to folk1 feature-wise (a lot of features are still not implemented, and documentation isn't great), but we weren't really supporting folk1 anyway (we wouldn't want anyone to submit a patch for it, for instance).
+And folk2 is so much faster (and more expressive for complicated integrations) that I think we want it to be the default experience for people. Still lots to do to communicate the new stuff you can do in it.
 === Minor improvements ===
+from the lead-up to merge & party:
   * [[https://github.com/FolkComputer/folk/commit/5ce48da6139005ed7a3835bf4d35f597eef5a841|Renamed virtual-programs/ to builtin-programs/]]
   * [[https://github.com/FolkComputer/folk/commit/4185bf4936894986aac65d5ff66d619e089b174d|Don't reproject during calibration if it's gonna go outside the projector area]] (and, likely, get stuck that way, since you can't then detect it and iterate to a new pose)
-  *
+  * Scheduler now scales number of Folk worker threads up to roughly the CPU count, to make better use of bigger Folk PC we have (the scheduler probably still needs a lot of work to be robust against a lot of different workloads)
+  * Daniel [[https://github.com/FolkComputer/folk/commit/83479b1315926c6ab4aeda2e83ae080a5e1bfcb5|fixed printer forwarding]] and program forwarding so we could print from downstairs systems through the upstairs folk-hex system (and use those programs on all systems in the room after printing)
+    * {{.:pasted:20260117-151940.jpeg?300px}}
+  * Improved "Printed!" and "Saved!" notices
+=== Vulkan memory allocation ===
+As we tested the system before the party, we noticed huge amounts of blinking when you had a bunch of camera slices / animation / sprites out.
+{{newsletters:img_7263.mp4?250px}}
+And the blinking got worse and worse over time. I profiled the system and saw that we would be spending milliseconds up to tens of milliseconds on each GPU memory allocation of a new texture for the camera slices / animations / sprites. See how long the gpu/textures.folk:669 blocks take here:
+{{newsletters:image-alloc.png?500px}}
+{{.:pasted:20260117-231138.png?500px}}
+(this is a form of workload that Folk really struggles with; we can cope with a long-running task, and we can do lots of short tasks, but lots of medium-length tasks is really hard for us. The blinking is a symptom of underlying slowdown / bottleneck in the evaluator, where we're missing deadlines to respond to things because we're spending too much compute time elsewhere)
+Well, memory allocation shouldn't take that long. The problem was that I had hand-rolled the memory allocation, which you're really not supposed to do. Vulkan doesn't defragment or anything, so your allocation performance just gets worse and worse over time if you're making lots of little allocations (it's fragmented, has to walk to find new areas, whatever).
+[[https://github.com/FolkComputer/folk/compare/f508b523...c559c8ce|I switched to use the Vulkan Memory Allocator library]] that [[https://old.reddit.com/r/vulkan/comments/u076gp/is_it_a_good_idea_to_use_vma_for_memory/|everyone]] [[https://stackoverflow.com/questions/74814762/how-to-deal-with-memory-fragmentation-in-vulkan#comment132036100_74814762|recommends]], and this immediately fixed everything, and now performance of camera slices and animations is great and doesn't degrade at all.
+The best part was that I [[https://github.com/FolkComputer/folk/commit/c559c8ced5cd6bab747883e2efb1dd220b9f8432|made this work at like 1PM on the day of the merge party]] -- a really amazing moment before the party (when we had been worried that the system just wouldn't work and now it was all magically working).
+{{.:pasted:20260117-102045.png?500px}}
+{{newsletters:small_pxl_20251212_181546070_20251219233938.mp4?500px}}
 ==== /setup page ====
-FIXME: final /setup layout
+Omar: The /setup page which now lets you set up camera and projector and initiate calibration looks pretty good now. (it replaces the old setup.folk file)
-FIXME: live display/camera selection cool
+You can see the /setup interface on the left here, and some of its source code on the right (which uses the new HtmlWhen construct):
-FIXME: GPU splitup
+{{newsletters:img_7052.jpeg?600px}}
+Live display and camera selection doesn't work perfectly yet (switching displays crashes), but it feels really cool and shows the dynamism of the system and the display/camera modules. The live preview of camera when you choose a resolution is really nice.
 === JPEG frames ===
-Decompress and do AprilTag detect, making the normal Folk pipeline work again.
+We started on using JPEG frames in the system for /setup / live preview last month. This month, we added decompress and AprilTag detect steps on top, making the normal Folk pipeline work again.
+[[https://github.com/Rooprob|Rob Fielding]] suggested moving toward claiming decompressed color frames in the system as well. Currently, the system only exposes the decompressed grayscale channel (which is all you need for AprilTag detection). By using the raw JPEG data from webcams, we can potentially allow color use cases to coexist with grayscale detection without redundant decompression/compression cycles.
 === HtmlWhen improvements ===
+I [[https://github.com/FolkComputer/folk/commit/68a9fe80eca043b05a956ced5350fb5bfa432ff4|implemented]] the HtmlWhen construct [[newsletters/2025-11#htmlwhen|that I discussed last month]], made web/programs use it, and [[https://github.com/FolkComputer/folk/commit/0db6d064e56da3e9151093df00861927678e4780|made web/setup use it.]]
+Made some improvements after initial implementation:
+  * [[https://github.com/FolkComputer/folk/commit/55e7be471c7d7ede16665d0956b267c1ed6028e2|Initialize on page load]], so the initial HTML you get from the endpoint already has the data, not just on change
+  * [[https://github.com/FolkComputer/folk/commit/1f25e5540e30c671dd4bdd53b9334421124febe5|Scope the HTML element per-HtmlWhen]]  so you can have more than one HtmlWhen on the same page
+== Idiomorph ==
-==== Statement reuse bug fix ====
+Last month, we mentioned wanting to use a morphing library to preserve HTML elements (including their internal state: focus, selection, details-open, input value) when the HtmlWhen updates. I [[https://github.com/FolkComputer/folk/commit/7bc42355b92c7bb6f6d6ed613603a3ee6ae273d4|added support for idiomorph]], a library from the [[https://htmx.org|HTMX]] people, which pretty much just works for this!
-Fixes editor nonresponsive
+This is as long as you put HTML ids on all your elements so they can be reconciled across the update. [[https://github.com/bigskysoftware/idiomorph/issues/7|You might also need to add custom JS handlers to filter out attributes that you don't want to be destroyed by the update.]]
-Fixes Added pileup
+Here's the templated Folk-HTML that [[https://github.com/FolkComputer/folk/blob/7bc42355b92c7bb6f6d6ed613603a3ee6ae273d4/builtin-programs/web/programs.folk|implements the /programs endpoint now]], including the ''-beforeAttributeUpdated'' filter to preserve the <details> tag open state across updates:
+<code>
+<html>
+<head>
+    <title>Running programs</title>
+    <link rel="stylesheet" href="/style.css">
+    <style>
+        body {
+            font-family: math;
+        }
+        summary {
+            font-family: monospace;
+            font-size: 2em;
+        }
+    </style>
+    <script src="/lib/folk.js"></script>
+    <script src="/vendor/idiomorph.js"></script>
+    <script>
+      const folk = new FolkWS();
+    </script>
+</head>
+<body>
+[HtmlWhen the collected results for [list /programName/ has program code /programCode/] \
+          are /programs/ {
+    emitHtmlForPrograms $programs
+} -beforeAttributeUpdated {(attributeName, node, mutationType) => {
+    if (attributeName === "open") { return false; }
+}}]
+</body>
+</html>
+</code>
+==== Statement reuse bug fix (fixes editor freeze) ====
+Omar: We noticed before and during the folk2 launch party that the editor would often freeze after a minute or so of usage. You'd be typing stuff and suddenly your keystrokes wouldn't do anything, and you'd have to un-cover and re-cover the editor or even reboot the system to get the editor to work again.
+I dug into the problem for a few days. I started by finding that for some reason the [[https://github.com/FolkComputer/folk/commit/a4851a93c62ee640ffb6f88ab52622a7108a952d|last statement in this block]] sometimes was mysteriously gone. Tried reordering to see if that would cause a different statement to be missing:
+{{.:pasted:20260117-084633.png}}
+Whenever I got the editor freeze bug, I'd see that that Hold! block was executing, and most of the statements from the block were in the database, but one was missing & would never show up as a result to any queries (so we'd be lacking a ''max cursor x'' or ''view dimensions'' or ''cursor position'' statement, which meant the editor Whens wouldn't fire, breaking the editor).
+I walked the database trie by hand and confirmed that the ''has max cursor x'' statement wasn't there at all (there's no ''max'' branch under ''has''):
+{{.:pasted:20260117-231251.png?580px}}
+This made me feel that there was some really bad race in the evaluator (this behavior violates what I would think of as core Folk invariants), so I did more digging.
+I found that the missing statement (s4409:77 here) //was// actually present in the database and was alive, but was not queryable (didn't exist in the trie):
+{{.:pasted:20260117-090019.png?600px}}
+{{.:pasted:20260117-090044.png?700px}}
+(whereas the other two child statements of the block, s4408:77 and s4419:77, were properly indexed and queryable)
+I added some print statements and found this violation, where we're adding a new statement ''max cursor x 19'' **while we haven't yet removed an old ''max cursor x 19''!**
+{{.:pasted:20260117-090229.png?550px}}
+After this point, the statement is in the weird state we saw earlier (alive but not in trie):
+{{.:pasted:20260117-091547.png?550px}}
+(Extremely alarming and a clear invariant violation to have two different living statements with an identical clause.)
+It turned out the problem was in statement reuse (which makes sense, since these were statements about editor state that are often the same across updates). I had misunderstood what ''continue'' in a do-while loop does.
+When statement reuse succeeds, we ''return NULL'', since we don't need to change the database trie. When statement reuse fails (because the reusee was removed before we could acquire it or increment its refcount), we generally ''continue'' to retry adding the statement from scratch.
+But I assumed that ''continue'' repeats the loop from the top of loop body. I didn't realize that the ''continue'' actually still tests the `while` expression at the bottom of the loop first, which swaps ''newClauseToStatementRef'' into ''db->clauseToStatementRef'', which succeeds and terminates the loop. And the ''newClauseToStatementRef'' in this reuse case is the same as the old ''clauseToStatementRef'', so we don't actually change the trie. So we create a new statement data structure and think we added it to the trie, but it's actually not queryable. There's our problem.
+[[https://github.com/FolkComputer/folk/commit/b22ffd29ab64ac06e9d5fd7f730043ce99cf8d20#diff-57df756227997036d791c93ca1c8b3da658de85d114d3189010f3807a1569cfdL1302|The fix was to rerun the loop as long as the new trie is same as old trie, so we actually do a next iteration from scratch on continue.]]
+{{.:pasted:20260117-100905.png?550px}}
+This one-line fix fixed the editor freeze bug, and it also seemingly fixed [[newsletters/2025-11#added-tag-pileup-not-fixed-yet|the Added tag pileup that has been bothering me for months]], and it also probably fixes other mysterious bugs. I feel a lot happier with the evaluator now.
+==== Editor refactor ====
+Mason did a big [[https://github.com/FolkComputer/folk/pull/240|editor refactor]] that removes traces of the old "code view" terminology, cleans up the state, and makes the line numbers render more cleanly (no more snapping around when you go from 1-digit to 2-digit line numbers + dynamic left margin so the line numbers don't take up a tiny or huge amount of space on extreme zoom). Scroll also works better (retains more context).
+This mostly came out of a bug where the scroll viewport was weirdly small on editor reboot (due to some race in the old way we were doing state). The refactor fixes this bug.
+{{newsletters:img_7668.mp4?400px}}
+Some other funny line number situations we saw (and fixed) while doing the refactor:
+{{.:pasted:20260117-232906.jpeg?300px}} {{newsletters:img_7924.jpeg?300px}}
+Now the editor feels pretty solid and comfortable across zooms, restarts, etc.
+==== Folk for DnD ====
+Paul and Naveen discussed building a "DnDKit" for Folk. The idea is to have a DM laptop interface that projects a symbolic map onto the table, allowing the DM to selectively hide or expose rooms (fog of war) while players move physical miniatures around.
+==== with /...options/ options ====
+Omar: I've been [[https://github.com/FolkComputer/folk/tree/osnr/when-with-sugar|experimenting with]] trying to break out ''When with /...options/'' into ''When with x /x/ y /y/'' while still supporting any option order and letting unused options pass through. This is nice because it makes the When declaration better reflect the actual options needed by the body.
+{{.:pasted:20260105-142658.png?450px}}
+The branch works okay in small tests, but slows down the system a lot when I run the whole thing.
+=== Richer captures ===
+Omar and Mason brainstormed some more ways to make Folk's ''When'' captures more expressive, instead of having ad hoc code in the body of the When to destructure arguments and check argument validity.
+  * Optional and default values: We're exploring syntax like ''/layer?/'' for optional captures or ''/layer or 0/'' for default values.
+  * Type constraints and filters: There is interest in "contracts" for captures, such as ''/x where [string is double $x]/'' or even range checks.
+  * The "with" keyword: We are looking at making ''with'' a bare keyword in statements to better denote special flags or contexts without complicating the dictionary matching.
+The main challenge here is tokenization—balancing the ergonomics of the forward-slash ''/capture/'' syntax with the need to parse more complex expressions inside them.
+==== /quads experiment ====
+Omar: I've felt for a while that we're not doing enough with the full 3D geometry that we have of all quads (program areas) and of the camera and projector. I wanted to do a little 3D visualization of the 3D space that you can pan and zoom. Slapped this together just using CDNed three.js. It shows Folk's perception of the 3D space in front of the camera:
+{{newsletters:screen-recording-2025-12-18-at-5.29.46 pm.mp4?400px}}
+I want to clean it up and do some more stuff (maybe texture the quads with the camera slices of each quad). It'd be cool for this to be a livestream UI with these 'computational objects' in the stream for all viewers to interact with.
+==== New gadget1 ====
+Omar: I printed a new (pink!) gadget1 to bring to China. This one uses top/bottom threading for the front panel so we don't need space-eating wells to screw the front panel into.
+{{.:pasted:20260117-232319.jpeg?250px}}
+It also now has a bottom hole for the trigger button, so we can do trigger button stuff on the gadget1 as well. And it supports dual Pi cameras to mount and to stick through front panel.
+{{.:pasted:20260117-231800.png?300px}}
+New (left) and old (right):
+{{newsletters:img_8057.jpeg?250px}}
+I've just been doing each one by hand, but it feels like we now have a number of implicit variants:
+  * Battery (gadget2) vs. no-battery (gadget1)
+  * Orange Pi (gadget2) vs. Raspberry Pi (gadget1)
+    * retail AnyBeam (gadget2) vs. Ultimems devkit (gadget1)
+      * (the retail AnyBeam requires USB-C video in, so needs the Orange Pi)
+      * (it is somewhat more stable to mount the devkit because it has multiple threaded holes underneath, though)
+  * ELP stereo camera (gadget2) vs. dual Raspberry Pi cameras (gadget1)
+It would be nice to be able to mix and match these attributes.
+I haven't yet updated the repo with this info or with the new CAD model, but I also figured out the pins to plug the trigger button into on the gadget1's Raspberry Pi (as opposed to the gadget2's Orange Pi).
+{{.:pasted:20260117-231948.jpeg?300px}}
+See how it changes output when I press the trigger button:
+{{.:pasted:20260117-231935.png?500px}}
+==== Experimental tcl interpreter update ====
+Mason: slowly but surely, the custom interpreter is coming together! The core object APIs are now in place, and the interpreter works, so now it's mainly working on the standard library.
+Expression parsing ended up being a lot easier to figure out than I expected—precedence climbing is surprisingly intuitive! There were some infinite recursion and infinite loop bugs that took some time to hunt down, but it's working pretty well now.
+I ran into a really nasty reference counting bug the other day, because I've been swapping objects in place whenever they need to be replaced. This created some really scary situations where I accidentally freed an object that was still in use by the caller, so I've opted against swapping in place except for the outermost API. I'm mostly done refactoring it out, but there's still more stuff to clean up.
+Overall, I'm feeling really good about the design. Having a custom allocator lets me do some really cool stuff, like splitting an existing allocation into individual parts in-place, which lets me do some efficient list shimmering (I store objects directly in the list, so if one of the objects became shared when freeing the list, I need some way to keep the objects in place. I can split the list's allocation into individual objects, and free all the non-shared objects).
+I still haven't been able to do much performance testing though, since the pieces kind of all interlock, so I have a nagging feeling that may be my interpreter will be slower than Jim. But, I've built in a lot of room for modifying when structures are copied/referenced, so I have room to tune. So stay tuned for when I can get enough standard functions in place, so I can start benchmarking Folk!
 ==== Outreach ====
+Our friends [[https://a9.io|Max Krieger]] and [[https://www.tophtucker.com|Toph Tucker]] visited:
+{{newsletters:img_6965.jpeg?300px}} {{newsletters:img_7229.jpeg?300px}} {{newsletters:img_7232.jpeg?300px}}
 === folk2 launch party ===
+We had a [[https://partiful.com/e/Y5L66oPibgDt4PcRQptx|launch party for folk2]] on Friday, Dec 12. It was very fun. [[https://github.com/brianshl007|Brian Lee]] helped out, and Daniel came in from Utah to help out as well.
+{{newsletters:img_7318.jpeg?0x300px}} {{newsletters:img_7322.jpeg?0x300px}}
+{{newsletters:img_7261.mp4}}
+Was a good way to bring people in who hadn't seen Folk in a while, announce nonprofit and folk2 updates, and force ourselves to make progress on public-facing niceties in the Folk system. (We probably made a couple months' worth of progress on little nits in the week before the party.)
 Omar made some programs to do the merge live on stage:
@@ Line 64: / Line 290: @@
 {{newsletters:editor_with_cat.jpeg?0x400px}}
-[[https://github.com/brianshl007|Brian Lee]] made this poster board with various demo programs people could try out on one of the systems downstairs and was a convenient surface to point gadgets at to demonstrate that they can run all the same programs a bigger system can:
+Brian made this poster board with various demo programs people could try out on one of the systems downstairs and was a convenient surface to point gadgets at to demonstrate that they can run all the same programs a bigger system can:
 {{newsletters:board.jpeg?400px}}
-Omar and Andrés gave a short talk about their work on folk2 and the hopes they have for the future of the project. Afterward Daniel talked about the non-profit and funding this work:
+Omar and Andrés gave a short talk about their work on folk2 and the hopes they have for the future of the project. We had a /camera livestream of a table (via the Folk Web sevrer) projected on the screen, to show the actual merge and to show what people were doing with the system:
+{{newsletters:img_7327.jpeg?300px}}
+Afterward Daniel talked about the non-profit and funding this work:
+{{newsletters:daniel_speech.jpeg?0x300}} {{newsletters:img_7366.jpeg?0x300}}
+=== Folk gadgets at SVA ===
+Andrés presented to a class they were teaching on UX/UI Fundamentals at the SVA Interaction Design masters program about the future of interfaces. It culminated in a live demo of the Folk gadget, which students had a lot of interesting ideas for building prototypes with:
+{{newsletters:img_2720-medium.jpeg?400px}} {{newsletters:img_2719-medium.jpeg?400px}}
-{{newsletters:daniel_speech.jpeg?400px}}
+{{newsletters:img_2716-medium.jpeg?0x400px}} {{newsletters:img_2714-medium.jpeg?0x400px}}
 ===== What we'll be up to in January =====
-  * TODO: Next open house
+  * Our next [[https://luma.com/49d1t0ng|open house is on Thursday, January 29th]]
-  * Omar will be in China and mostly working on the gadget
+  * Omar will mostly be in China and mostly working on the gadget (calibration, setup process, new hardware iterations/capabilities) and maybe RFID
+  * Omar wants to do some more model/perception experiments as well
   * Andrés will be working on getting dot detection demos working again and exploring dot interaction demos
@@ Line 82: / Line 321: @@
 ==== Omar ====
+  * https://www.youtube.com/watch?v=ifKRmkGJgOE
+  * https://tech.marksblogg.com/apple-iphone-15-pro-depth-map-heic.html
+  * https://xania.org/202506/how-compiler-explorer-works
+  * https://www.cups.org/blog/2018-06-06-demystifying-cups-development.html
+  * https://sel4.systems/About/seL4-whitepaper.pdf
+  * https://barrelfish.org/publications/nevill-master-capabilities.pdf
 ==== Andrés ====
@@ Line 88: / Line 333: @@
   * [[https://social.nouveau.community/@andnull/115714417216432637|Fungus: A 2D Programming Environment]]
   * [[https://www.instagram.com/p/DRkSuBrjNkl/|Bert Hodgson's "weird pebbles" (projection mapping on stones)]]
+  * [[https://www.youtube.com/watch?v=IroPQ150F6c|A Practical Guide to Applying Data Oriented Design]]