Differences

This shows you the differences between two versions of the page.

--- newsletters:2025-04 [2025/04/30 18:27] – [New Folk evaluator] osnr
+++ newsletters:2025-04 [2025/05/01 01:57] (current) – osnr
@@ Line 1: / Line 1: @@
-====== April 2025 newsletter (WIP) ======
+====== April 2025 newsletter ======
 We've had a funding shortfall recently -- we'd really appreciate it if you [[https://github.com/sponsors/FolkComputer|sponsor Folk on GitHub Sponsors]] if you can, so that we can keep doing this free and open work :-)
-We’ll be set up at Rhizome World on this Sunday, May 4 (at 161 Water Street in the Financial District). Feel free to say hello; we’ll also be running [[https://rhizome.itm.studio/m/a-folk-computer-workshop|a Folk Computer workshop there at noon.]]
+We’ll be setting up at [[https://rhizome.org/editorial/2025/apr/03/rhizome-world/|Rhizome World]] this Sunday, May 4 (at 161 Water Street in the Financial District). Feel free to say hello; we’ll also be running [[https://rhizome.itm.studio/m/a-folk-computer-workshop|a Folk Computer workshop there at noon.]]
 If you want to stop by the studio, our next Folk open house is **[[https://lu.ma/ydi03zax|in the afternoon on Saturday, May 31]], in East Williamsburg, Brooklyn.**
@@ Line 11: / Line 11: @@
 ==== Demos and applications ====
-  *
+  * (Andrés) I gave a small talk and workshop to [[https://www.linkedin.com/in/christinahuang96/|Christina Huang]]'s Learning Analytics class at NYU. I was so busy demoing that I didn't take any photos myself, but there was a photographer there from NYU's [[https://steinhardt.nyu.edu/programs/educational-communication-and-technology|Educational Communication and Technology]] Wireframe Newsletter — when I get those back I'll share them here.
 ==== General system improvements ====
-  * (Andrés) We merged (TODO (Andrés): merge all) [[https://www.linkedin.com/in/robertfielding/|Rob Fielding]]'s PR's to:
+  * (Andrés) We merged [[https://www.linkedin.com/in/robertfielding/|Rob Fielding]]'s PR's to:
     * Re-activate [[https://haiperspace.com/|Jacob Haip]]'s blob-detection code, introducing a spruced up demo program:
       * {{youtube>qiPEaeuK2yo?}}
       * This also means the laser capability is back! (Side note: I love the dog getting to play with the laser here ... it makes me wonder what a Folk system specifically for entertaining / training pets could look like!)
         * {{youtube>wgaSJ6B9uwc?}}
-    * Add `Wish $this draws text "example"` to the system, making text work like our general shape syntax:
+    * Add ''Wish $this draws text "example"'' to the system, making text work like our general shape syntax:
       * {{youtube>NFN6DbwPsF8?}}
     * Tools for creating subregions:
@@ Line 75: / Line 75: @@
 Omar: I would say work this month has been driven by three different pushes (not really separable that cleanly, but oh well):
-. Continuing the performance/anti-blink push from last month: this drove page quads
+. Continuing the performance/anti-blink push from last month: this drove page quads / render-to-texture, trying to get rid of blinking both semantically and by improving performance
-. Gadget: memory leak
+. Run folk2 on my gadget: it quickly leaks memory and runs out of RAM (the gadget has much less RAM than desktops do), which made me try to hunt down memory leaks
-. Fix calibrate on folk2 (given the changes from 1)
+. Fix calibrate and other functionality on folk2 so we can actually merge it: render-to-texture is forcing us to make more changes to calibrate and editor at least to get them to work again
 I feel pretty good about folk2 right now. Beyond fixing the memory leak, a lot of the remaining work is functionality work, honestly, where we need to get calibration and the editor running again under the new regime (as well as camera slices, animation, other smaller modules). I've been putting that off because I really want to fix the memory leaks...
@@ Line 87: / Line 87: @@
 I've spent a lot of this month working on the [[newsletters/2025-03#page-quads|page quads / render-to-texture]] stuff that I started last month, which has produced huge performance improvements (almost more than everything else in folk2 combined). Really excited to get this merged now.
-I added support for drawing text and images and AprilTags on the textures, which was hard to debug, so I added a /images endpoint that shows all textures in memory (including placeholder image 0, font atlases, page textures, projected calibration board texture, other images):
+Weird design choice for now: I got rid of the implicit parameter that passes screen resolution into every shader/pipeline -- you still need it a lot of the time, but you have to do it explicitly now. This is because pipelines can now be used both to draw to the screen and to draw to a writable image. It's also because we're really tight on push constant space for the glyph shader in particular (it's max 128 bytes and we now need to pass a 3x3 matrix into most shaders), and manually specifying and ordering all the push constants lets you save a bunch of padding space. Maybe this is fine, but we should also think about [[https://github.com/FolkComputer/folk/issues/106|the higher-level shader notation idea]] again as a layer on top?
+== /images debug page ==
+I added support for drawing text and images and AprilTags on the textures, which was hard to debug, so I added a [[https://github.com/FolkComputer/folk/blob/4f62dd2daa9a5abebd751c6c5b29f9403915cbdf/virtual-programs/web/images.folk|/images endpoint that shows all textures in memory]] (including placeholder image 0, font atlases, page textures, projected calibration board texture, other images):
 {{newsletters:images-endpoint.png?0x400px}} {{newsletters:calibration-image-2.png?0x400px}}
-== Destructors ==
+The ability to get 'the stuff projected onto this page' as an image feels really cool and useful in itself, like Folk is now a compositor like Wayland or macOS Quartz and it has internal semantically useful knowledge of what all the layers are. I think we could have some cool applications for this gpu->cpu image transfer capability, like compositing the rendered page graphics on top of a livestream of the camera, or or printing what's projected on a page.
+(most writable image textures are square 1024x1024 by default, and then we just resample them when we project them onto the display, which is why they're weird and square in the debugger view here)
+== Destructors and ForEach! ==
+To cleanly do GPU texture destruction when a page is flipped over, I added support for //per-statement destructors//, where you can pass a ''-destructor'' parameter to ''Claim'' or ''Wish'' to run some code once it's guaranteed that no one is looking at that statement anymore.
-To cleanly do texture destruction when a page is flipped over, I added support for per-statement destructors:
+I also use this to clean up camera images now, which feels really nice:
+{{:newsletters:pasted:20250430-190101.png?400px}}
-I also use this to clean up camera images now, which feels really nice (instead of just timing out camera images after some fixed period of time):
+(instead of just timing out camera images after some fixed period of time, or doing it on match unmatch which is unsafe since it doesn't guarantee no concurrent accessors to the statement)
+This involved some changes to statement acquisition behavior and some new constructs:
+  * When a When matches a statement, the statement is now acquired for the duration of the When block's execution, so the statement's destructor (if applicable) will not run until the When has terminated. This feels like the correct behavior.
+  * But Query!, which returns a static set of result statements, **does not acquire any of those statements.** **So the destructor of any statement in the result set may run at any time once Query! has returned.** So it'd be unsafe to access a camera image that you got from Query! without explicitly acquiring the owning statement first to make sure it sticks around (and that it's still around).
+  * So ForEach! has been added for both ergonomic and safety reasons: ForEach! is like Query!, but you pass it a code block as last arg; it iterates and binds each result statement and runs the code block. In other words, it's like a mix of Query! (imperative semantics, doesn't yield) and When! (acquires each matched statement, binds variables for you, safe to use terms in statement inside your code block)
+    * <code>
+set omarAdjs [list]
+ForEach! Omar is /adj/ {
+  lappend omarAdjs $adj
+}
+puts "Omar is all of these: $omarAdjs"
+</code>
+    * (Andrés and Mason helped to name this one -- I suspect we'll be using it a lot)
+  * Manual StatementAcquire! and StatementRelease! have also been added if you want to use those with Query! (some of your queried statements may be invalid, though, and you'd have to ignore/toss those)
 == Bilinear interpolation ==
@@ Line 113: / Line 136: @@
 === Memory leak ===
+Because I've been trying to run folk2 on the (RAM-constrained) gadget, I've been really wanting to fix the memory leaks in it. It's been tough. My hypothesis (from seeing Jim allocations get out of control and from manually inspecting the heap) is that it's mostly little Jim Tcl heap objects (Jim_Objs, mostly). But hard to trace causation beyond that point so far.
 <details>
@@ Line 118: / Line 143: @@
   - Address Sanitizer (new this month)
+    - (This has been really useful for finding more serious bugs, too -- I found some use-after-frees and stuff like that)
     - {{newsletters:6386e479-848f-4d65-a809-eef01649e267-1040-00001f35ba2c028e.png?400px}}
   - pprof (libtcmalloc) heap profiler
@@ Line 132: / Line 158: @@
 </details>
+It's getting kind of ridiculous how many different execution modes we have for folk2:
+{{:newsletters:pasted:20250430-192041.png?350px}}
+(it is a lot easier to do all this in folk2 than in folk1, though, because folk2 is all one big process and most of its kernel is statically compiled)
 === Disabling cache ===
@@ Line 158: / Line 190: @@
 For now, I guess we'll try to merge folk2 without even the cache, just string reparsing every block, and then later see if some of these other approaches can be revived and can help further.
-==== Outreach and other systems ====
-  * TODO: Discord updates
 === Open house ===
@@ Line 168: / Line 196: @@
   * {{newsletters:20250427-ned-keyboard.jpg?0x300px}}{{newsletters:20250427-open-house-crowd.jpg?0x300px}}{{newsletters:20250427-yuwei-animation.jpg?0x300px}}
   * {{newsletters:img_9855.jpeg?0x300px}} {{newsletters:yuwei-cat.gif?0x300px}}
 === aci-d club visit ===
@@ Line 184: / Line 211: @@
 === Other visitors and interactions ===
-  * Andrés went to  Christina Huang's Learning Analytics class at NYU to demonstrate Folk, it was a fun and productive class:
+  * (Andrés) I went to  Christina Huang's Learning Analytics class at NYU to demonstrate Folk, it was a fun and productive class. I'll add pictures in this newsletter once they're published.
-    * TODO (Andrés): pictures
 ===== What we'll be up to in May =====
@@ Line 214: / Line 240: @@
 ==== Andrés ====
-  * Vox graphics
+  * I've been reading about "time-based media conservation" and stumbled on [[https://www.tbmworkshops.com/workshops/artwithaplug2020|this fun workshop called "Art With A Plug"]]
+  * I also really like NYU's Institute of Fine Arts' video [[https://vimeo.com/677750584|How to Become a Time-Based Media Conservator]]
+  * Fascinating paper on [[https://www.youtube.com/watch?v=5fiRjtGrQjQ|Curved origami with tunable stiffness
+]] — basically making small compliant mechanism machines using scoring and folding
+  * [[https://medium.com/swlh/a-bit-about-interface-builder-ceffaf484580|A Bit About Interface Builder]]
+  * [[https://www.behance.net/gabmd|Gabrielle Merite]] is making a lot of fascinating [[https://x.com/rsnous/status/1916912489301791027|post-Tufte data visualizations]]