Our next Folk open house is in the evening on Thursday, January 29th, in East Williamsburg, Brooklyn.

Daniel Pipkin: We are now officially a 501c3 nonprofit. This means we can accept tax-deductible donations and are eligible to apply for grants.

Folk is a long-term project. Many people want to work on it and even more people want to support its growth and development. When thinking of the possible organizational structures that could sustain this development, we decided that a nonprofit was the best fit. It allows us to continue providing something valuable without needing to rope off parts of that value behind a paywall. We want as many people as possible to be able to participate in this exciting new computing paradigm.

Funding is a big challenge for a nonprofit. It will take some time to grow. Because we have no plan to charge for Folk, we can't sell portions of ownership to venture capital or other investors. This is good long-term. We can avoid the pitfalls that come from developing a technology in service of shareholder value. But it means we can't rapidly grow into a 10-50 person team the way a traditional startup would.

The first major milestone will be hiring Omar full-time and Andrés part-time. They will continue development of Folk and the gadget. We also want to cover studio rent and hardware costs. In the future, we could expand this to include things like growing the core team, covering conference attendance, more gadget development, or anything else that advances our vision of tangible computing for everyone.

If you would like to support us, please join the 30+ people already donating on GitHub Sponsors. A special thank you to all of you who are already sponsoring the project.

Omar: We've merged the new version of folk into the main branch.

It's a nice feeling that the work we've been doing is finally going to be visible on the main repo and get used by people out of the box.

Along with the nonprofit news, I think we want to use this as a trigger to enter a new, more public phase where we do more applications and demos (and work more on fundraising).

folk2 isn't quite caught up to folk1 feature-wise (a lot of features are still not implemented, and documentation isn't great), but we weren't really supporting folk1 anyway (we wouldn't want anyone to submit a patch for it, for instance).

And folk2 is so much faster (and more expressive for complicated integrations) that I think we want it to be the default experience for people. Still lots to do to communicate the new stuff you can do in it.

from the lead-up to merge & party:

• Renamed virtual-programs/ to builtin-programs/
• Don't reproject during calibration if it's gonna go outside the projector area (and, likely, get stuck that way, since you can't then detect it and iterate to a new pose)
• Scheduler now scales number of Folk worker threads up to roughly the CPU count, to make better use of bigger Folk PC we have (the scheduler probably still needs a lot of work to be robust against a lot of different workloads)
• Daniel fixed printer forwarding and program forwarding so we could print from downstairs systems through the upstairs folk-hex system (and use those programs on all systems in the room after printing)
  • 
• Improved “Printed!” and “Saved!” notices

As we tested the system before the party, we noticed huge amounts of blinking when you had a bunch of camera slices / animation / sprites out.

And the blinking got worse and worse over time. I profiled the system and saw that we would be spending milliseconds up to tens of milliseconds on each GPU memory allocation of a new texture for the camera slices / animations / sprites. See how long the gpu/textures.folk:669 blocks take here:

(this is a form of workload that Folk really struggles with; we can cope with a long-running task, and we can do lots of short tasks, but lots of medium-length tasks is really hard for us. The blinking is a symptom of underlying slowdown / bottleneck in the evaluator, where we're missing deadlines to respond to things because we're spending too much compute time elsewhere)

Well, memory allocation shouldn't take that long. The problem was that I had hand-rolled the memory allocation, which you're really not supposed to do. Vulkan doesn't defragment or anything, so your allocation performance just gets worse and worse over time if you're making lots of little allocations (it's fragmented, has to walk to find new areas, whatever).

I switched to use the Vulkan Memory Allocator library that everyone recommends, and this immediately fixed everything, and now performance of camera slices and animations is great and doesn't degrade at all.

The best part was that I made this work at like 1PM on the day of the merge party – a really amazing moment before the party (when we had been worried that the system just wouldn't work and now it was all magically working).

Omar: The /setup page which now lets you set up camera and projector and initiate calibration looks pretty good now. (it replaces the old setup.folk file)

You can see the /setup interface on the left here, and some of its source code on the right (which uses the new HtmlWhen construct):

Live display and camera selection doesn't work perfectly yet (switching displays crashes), but it feels really cool and shows the dynamism of the system and the display/camera modules. The live preview of camera when you choose a resolution is really nice.

We started on using JPEG frames in the system for /setup / live preview last month. This month, we added decompress and AprilTag detect steps on top, making the normal Folk pipeline work again.

Rob Fielding suggested moving toward claiming decompressed color frames in the system as well. Currently, the system only exposes the decompressed grayscale channel (which is all you need for AprilTag detection). By using the raw JPEG data from webcams, we can potentially allow color use cases to coexist with grayscale detection without redundant decompression/compression cycles.

I implemented the HtmlWhen construct that I discussed last month, made web/programs use it, and made web/setup use it.

Made some improvements after initial implementation:

• Initialize on page load, so the initial HTML you get from the endpoint already has the data, not just on change
• Scope the HTML element per-HtmlWhen so you can have more than one HtmlWhen on the same page

Last month, we mentioned wanting to use a morphing library to preserve HTML elements (including their internal state: focus, selection, details-open, input value) when the HtmlWhen updates. I added support for idiomorph, a library from the HTMX people, which pretty much just works for this!

This is as long as you put HTML ids on all your elements so they can be reconciled across the update. You might also need to add custom JS handlers to filter out attributes that you don't want to be destroyed by the update.

Here's the templated Folk-HTML that implements the /programs endpoint now, including the -beforeAttributeUpdated filter to preserve the <details> tag open state across updates:

<html>
<head>
    <title>Running programs</title>
    <link rel="stylesheet" href="/style.css">
    <style>
        body {
            font-family: math;
        }
        summary {
            font-family: monospace;
            font-size: 2em;
        }
    </style>
    <script src="/lib/folk.js"></script>
    <script src="/vendor/idiomorph.js"></script>
    <script>
      const folk = new FolkWS();
    </script>
</head>
<body>
[HtmlWhen the collected results for [list /programName/ has program code /programCode/] \
          are /programs/ {
    emitHtmlForPrograms $programs

} -beforeAttributeUpdated {(attributeName, node, mutationType) => {
    if (attributeName === "open") { return false; }
}}]
</body>
</html>

Omar: We noticed before and during the folk2 launch party that the editor would often freeze after a minute or so of usage. You'd be typing stuff and suddenly your keystrokes wouldn't do anything, and you'd have to un-cover and re-cover the editor or even reboot the system to get the editor to work again.

I dug into the problem for a few days. I started by finding that for some reason the last statement in this block sometimes was mysteriously gone. Tried reordering to see if that would cause a different statement to be missing:

Whenever I got the editor freeze bug, I'd see that that Hold! block was executing, and most of the statements from the block were in the database, but one was missing & would never show up as a result to any queries (so we'd be lacking a max cursor x or view dimensions or cursor position statement, which meant the editor Whens wouldn't fire, breaking the editor).

I walked the database trie by hand and confirmed that the has max cursor x statement wasn't there at all (there's no max branch under has):

This made me feel that there was some really bad race in the evaluator (this behavior violates what I would think of as core Folk invariants), so I did more digging.

I found that the missing statement (s4409:77 here) was actually present in the database and was alive, but was not queryable (didn't exist in the trie):

(whereas the other two child statements of the block, s4408:77 and s4419:77, were properly indexed and queryable)

I added some print statements and found this violation, where we're adding a new statement max cursor x 19 while we haven't yet removed an old max cursor x 19!

After this point, the statement is in the weird state we saw earlier (alive but not in trie):

(Extremely alarming and a clear invariant violation to have two different living statements with an identical clause.)

It turned out the problem was in statement reuse (which makes sense, since these were statements about editor state that are often the same across updates). I had misunderstood what continue in a do-while loop does.

When statement reuse succeeds, we return NULL, since we don't need to change the database trie. When statement reuse fails (because the reusee was removed before we could acquire it or increment its refcount), we generally continue to retry adding the statement from scratch.

But I assumed that continue repeats the loop from the top of loop body. I didn't realize that the continue actually still tests the `while` expression at the bottom of the loop first, which swaps newClauseToStatementRef into db->clauseToStatementRef, which succeeds and terminates the loop. And the newClauseToStatementRef in this reuse case is the same as the old clauseToStatementRef, so we don't actually change the trie. So we create a new statement data structure and think we added it to the trie, but it's actually not queryable. There's our problem.

The fix was to rerun the loop as long as the new trie is same as old trie, so we actually do a next iteration from scratch on continue.

This one-line fix fixed the editor freeze bug, and it also seemingly fixed the Added tag pileup that has been bothering me for months, and it also probably fixes other mysterious bugs. I feel a lot happier with the evaluator now.

Mason did a big editor refactor that removes traces of the old “code view” terminology, cleans up the state, and makes the line numbers render more cleanly. Scroll also works better (retains more context)

This mostly came out of us trying to fix a bug where the scroll viewport was weirdly small on editor reboot (due to some race in the old way we were doing state).

Paul and Naveen discussed building a “DnDKit” for Folk. The idea is to have a DM laptop interface that projects a symbolic map onto the table, allowing the DM to selectively hide or expose rooms (fog of war) while players move physical miniatures around.

Omar: I've been experimenting with trying to break out When with /...options/ into When with x /x/ y /y/ while still supporting any option order and letting unused options pass through. This is nice because it makes the When declaration better reflect the actual options needed by the body.

The branch works okay in small tests, but slows down the system a lot when I run the whole thing.

Omar and Mason brainstormed some more ways to make Folk's When captures more expressive, instead of having ad hoc code in the body of the When to destructure arguments and check argument validity.

• Optional and default values: We're exploring syntax like /layer?/ for optional captures or /layer or 0/ for default values.
• Type constraints and filters: There is interest in “contracts” for captures, such as /x where [string is double $x]/ or even range checks.
• The “with” keyword: We are looking at making with a bare keyword in statements to better denote special flags or contexts without complicating the dictionary matching.

The main challenge here is tokenization—balancing the ergonomics of the forward-slash /capture/ syntax with the need to parse more complex expressions inside them.

Omar: I've felt for a while that we're not doing enough with the full 3D geometry that we have of all quads (program areas) and of the camera and projector. I wanted to do a little 3D visualization of the 3D space that you can pan and zoom. Slapped this together just using CDNed three.js. It shows Folk's perception of the 3D space in front of the camera:

I want to clean it up and do some more stuff (maybe texture the quads with the camera slices of each quad). It'd be cool for this to be a livestream UI with these 'computational objects' in the stream for all viewers to interact with.

Mason: slowly but surely, the custom interpreter is coming together! The core object APIs are now in place, and the interpreter works, so now it's mainly working on the standard library.

Expression parsing ended up being a lot easier to figure out than I expected—precedence climbing is surprisingly intuitive! There were some infinite recursion and infinite loop bugs that took some time to hunt down, but it's working pretty well now.

I ran into a really nasty reference counting bug the other day, because I've been swapping objects in place whenever they need to be replaced. This created some really scary situations where I accidentally freed an object that was still in use by the caller, so I've opted against swapping in place except for the outermost API. I'm mostly done refactoring it out, but there's still more stuff to clean up.

Overall, I'm feeling really good about the design. Having a custom allocator lets me do some really cool stuff, like splitting an existing allocation into individual parts in-place, which lets me do some efficient list shimmering (I store objects directly in the list, so if one of the objects became shared when freeing the list, I need some way to keep the objects in place. I can split the list's allocation into individual objects, and free all the non-shared objects).

I still haven't been able to do much performance testing though, since the pieces kind of all interlock, so I have a nagging feeling that may be my interpreter will be slower than Jim. But, I've built in a lot of room for modifying when structures are copied/referenced, so I have room to tune. So stay tuned for when I can get enough standard functions in place, so I can start benchmarking Folk!

Our friends Max Krieger and Toph Tucker visited:

We had a launch party for folk2 on Friday, Dec 12. It was very fun. Brian Lee helped out, and Daniel came in from Utah to help out as well. Was a good way to bring people in who hadn't seen Folk in a while, announce nonprofit and folk2 updates, and force ourselves to make progress on public-facing niceties in the Folk system. (We probably made a couple months' worth of progress on little nits in the week before the party.)

Omar made some programs to do the merge live on stage:

We had multiple systems — the main system (folk-hex), two portable systems set up on the first floor of Hex House, and three gadgets (one of which was Daniel's home-made gadget) being passed around. People had a lot of fun trying out folk2 by making buttons, playing with the editors, and using points-at programs to have programs effect each other:

Brian made this poster board with various demo programs people could try out on one of the systems downstairs and was a convenient surface to point gadgets at to demonstrate that they can run all the same programs a bigger system can:

Omar and Andrés gave a short talk about their work on folk2 and the hopes they have for the future of the project. Afterward Daniel talked about the non-profit and funding this work:

Andrés presented to a class they were teaching on UX/UI Fundamentals at the SVA Interaction Design masters program about the future of interfaces. It culminated in a live demo of the Folk gadget, which students had a lot of interesting ideas for building prototypes with:

• Our next open house is on Thursday, January 29th
• Omar will mostly be in China and mostly working on the gadget (calibration, setup process, new hardware iterations/capabilities) and maybe RFID
• Omar wants to do some more model/perception experiments as well
• Andrés will be working on getting dot detection demos working again and exploring dot interaction demos

• https://www.youtube.com/watch?v=ifKRmkGJgOE
• https://tech.marksblogg.com/apple-iphone-15-pro-depth-map-heic.html
• https://xania.org/202506/how-compiler-explorer-works
• https://www.cups.org/blog/2018-06-06-demystifying-cups-development.html
• https://sel4.systems/About/seL4-whitepaper.pdf
• https://barrelfish.org/publications/nevill-master-capabilities.pdf

• Solid State Volumetric Display using glass etching
• Unihertz Tank 3 Pro (Projector Phone)
• Fungus: A 2D Programming Environment
• Bert Hodgson's "weird pebbles" (projection mapping on stones)
• A Practical Guide to Applying Data Oriented Design