Differences

This shows you the differences between two versions of the page.

--- newsletters:2025-03 [2025/04/01 01:42] – [General system improvements] osnr
+++ newsletters:2025-03 [2025/04/01 20:40] (current) – discord
@@ Line 8: / Line 8: @@
   * (Andrés) Animation binder
-    * I took the animation programs that Cristóbal made and put them in a binder, added a binder clip for stacks of paper, and added a wheel for people to adjust the number of frames
+    * I took the animation programs that [[https://cristobal.arquipelago.org|Cristóbal Sciutto]] made and put them in a binder, added a binder clip for stacks of paper, and added a wheel for people to adjust the number of frames
       * {{newsletters:img_1070.gif}}
       * {{newsletters:folk-anim-tinnei.gif}}
   * (Andrés) Frog folder
-    * I've been trying to explore making more didactic Folk programs, the first of these is the **origami frog folder**, which uses a mix of shapes and text to guide you through making an origami frog. If you have ideas for other things Folk might be useful for teaching feel free to email me at **[[mailto:cwervo@folk.computer|cwervo@folk.computer]]**
+    * I've been trying to explore making more didactic Folk programs; the first of these is the **origami frog folder**, which uses a mix of shapes and text to guide you through making an origami frog. If you have ideas for other things Folk might be useful for teaching, feel free to email me at **[[mailto:cwervo@folk.computer|cwervo@folk.computer]]**
-      * {{newsletters:folk_origami_frog_march_14_2025-1-.gif}}
+      * {{newsletters:folk_origami_frog_march_14_2025-1-.gif}} {{newsletters:frog-binder-cover.jpeg?200px}}
-      * {{newsletters:frog-binder-cover.jpeg?300px}}
 ==== General system improvements ====
@@ Line 27: / Line 26: @@
       * Omar: I've been really excited about video -- I think making a physical video book or diorama is a really simple and really compelling application of Folk (that you can't do without a system like Folk), and it's not even that computational or programming-y
   * (Andrés) [[https://github.com/FolkComputer/folk/pull/201|Shapes refactor]] — I fixed ''shapes.folk'' so that all shapes are drawn relative to ''$this'':
-    * {{newsletters:folk-shapes-tuneup.gif}} {{newsletters:folk-shapes-tuneup-text-large.jpeg?0x400}}
+    * {{newsletters:folk-shapes-tuneup.gif?0x200}} {{newsletters:folk-shapes-tuneup-text-large.jpeg?0x200}}
   * (Andrés) Dot detector — not much progress this month, will get to it early April
-  * (Mason) [[https://github.com/FolkComputer/folk/pull/198|Basic program grouping]] — a program that runs a list of programs, even if they're not visible (it helps with saving table space as programs accumulate, I like stacking the programs below the group!)
+  * ([[https://github.com/smj-edison|Mason Jones]]) [[https://github.com/FolkComputer/folk/pull/198|Basic program grouping]] — a program that runs a list of programs, even if they're not visible (it helps with saving table space as programs accumulate, I like stacking the programs below the group!)
 ==== Folk gadget & QR scanner ====
-<details>
+Omar: as a side project for [[https://qrshow.nyc|QR Show]] ([[https://qrshow.nyc/retrospective.html|see retrospective]]), I wrote a [[https://github.com/osnr/folk-qr|QR scanner program in Folk]] for the handheld gadget.
-<summary>Omar: as a side project for [[https://qrshow.nyc|QR Show]] ([[https://qrshow.nyc/retrospective.html|see retrospective]]), I wrote a [[https://github.com/osnr/folk-qr|QR scanner program in Folk]] for the handheld gadget.  (Click to see more)</summary>
 > [[https://bsky.app/profile/omar.website/post/3lkm3yv4ups2c|QR Show was my excuse]] to (*extremely* last-minute) hack together a prototype that I've been thinking about for years…
@@ Line 44: / Line 42: @@
 > It kind of half-seriously answers the critique of, like, restaurant menu QR codes, where scanning a QR code locks you into your phone & isn't a process that people around you can participate in. Here, you scan the QR code in a 'social' way, and 'in situ', and other people can see/participate too
-This is an interaction I'd had in mind for a few years, since I first got the Nebra AnyBeam in like 2021, so it's great to get to play it out now (and it was [[https://bsky.app/profile/omar.website/post/3lkm3yv4ups2c|not hard to write]] -- came up with the idea and implemented it in like 100 lines of code in a couple of hours before showing up).
+This is an interaction I'd had in mind for a few years, since I first got the Nebra AnyBeam in like 2021, so it's great to get to play it out now.
+And it was [[https://bsky.app/profile/omar.website/post/3lkm3yv4ups2c|not hard to write]] -- came up with the idea and implemented it in like 100 lines of code in a couple of hours before showing up. (This is also a consequence of all the work we've put into making stable gadget hardware over the last few months; the gadget is a pretty good hardware platform for this in its current state as is!)
 Here, I'm scanning a URL that Owen put up for his project (https://owentrueblood.com/experiments/autonomous_response/), and that web page gets rendered (using ''exec wkhtmltoimage'') and projected by Folk on the right:
@@ Line 73: / Line 73: @@
   * slow processes (synchronous-ish QR detect, asynchronous-ish rendering Web page)
   * uses the network, have to get the gadget on local Wi-Fi or whatever
-</details>
 === Other gadget stuff ===
-<details>
+== Wi-Fi issues ==
-<summary>Wi-Fi failing and upcoming gadget plans (Click to see more)</summary>
-**Wi-Fi failing**
+I mentioned this last month -- I've had such trouble getting the gadget (with its Orange Pi 5 and USB Wi-Fi dongle) connected in wild environments (where it doesn't know the Wi-Fi upfront). The self-access point mode on the dongle just doesn't seem to work.
-Following up on [[newsletters/2025-02#calibrating-the-new-gadget|last month]], I did successfully calibrate this (new, ELP dual-camera) gadget.
+== Successful calibration ==
+Following up on [[newsletters/2025-02#calibrating-the-new-gadget|last month]], I did successfully calibrate this (new, ELP dual-camera) gadget.
 (it hasn't been a priority, but I figured I might as well since I've been bringing it out so much for QR, and everything was in place already to try it)
-wasn't too bad! the calibration is a bit off when trying to use it on top of folk0 (TODO: photo?), but I hope some of the physical improvements below will help.
+wasn't too bad! the calibration is a bit off -- a couple of centimeters -- when trying to use it on top of folk0, but I hope some of the physical improvements below will help.
-**Upcoming**
+=== Upcoming ===
 There's still a lot to do on the gadget. Bringing it out so often for QR demo has helped clarify what the outstanding issues are, at least:
@@ Line 98: / Line 98: @@
     * display Wi-Fi status like how we display battery status? (can we do both of these in a nicer/more ambient way?)
   * Back panel is getting looser over time and one bolt never really stuck; probably should use heat inserts to screw it in. it comes on and off a lot, too...
-    * have a good way to expose a USB port or two? they're just generally handy
+    * have a good way to expose a USB port or two on the back panel too? they're just generally handy
   * Projector swivels around still (which could be very bad, breaks calibration).
     * Maybe use elastics and slits in the top?
@@ Line 105: / Line 105: @@
   * Button is fraying over time as it gets bent outward. Needs a cover that you press on top, like in the original grip design (I made a basic casing for our button but not a cover)
   * microSD card sometimes totally fails either before boot or while the system is running, need to power cycle
-    * use M.2 SSD instead?
+    * use M.2 SSD instead? I ordered a couple
-</details>
 ==== folk2 ====
-<details>
+Omar: Still a lot of work left on the new interpreter -- I've been biting off more and more as I continue to encounter blinking problems and dropped frames -- but I'm also increasingly excited about the potential frontier of performance (making Folk as responsive as possible), and I think the new system will be semantically better and cleaner, too (and more flexible for people to do new things). A lot of the changes are things we've wanted to do for years.
-<summary>
-Omar: Still a lot of work left on the new interpreter -- I've been biting off more and more as I continue to encounter blinking problems and dropped frames -- but I'm also increasingly excited about the potential frontier of performance (making Folk as responsive as possible), and I think the new system will be semantically better and cleaner, too (and more flexible for people to do new things). A lot of the changes are things we've wanted to do for years. (Click to see more)</summary>
 A few different branches are in play right now:
-**C collect**
+=== C collect ===
 I noticed in the Tracy profiles that we were spending a lot of time doing recollections, which is implemented in this both-elegant-and-hacky pure-Folk way in folk2:
@@ Line 150: / Line 148: @@
 (Even in folk1, Collect is implemented natively in C, so it's not surprising that this would be a step backward in performance.)
-I made a new folk2 branch where Collect is implemented in C.
+So I made a new folk2 branch where Collect is implemented in C, which should cut almost all of that database traffic.
 It feels a lot more responsive compared to main folk2. The major bug (and why we haven't merged it into main folk2) is that it seems to introduce long periods where pages just blink out and vanish, like this:
@@ Line 158: / Line 156: @@
 Still need to debug this.
-**Page quads**
+=== Page quads ===
-This is the most important branch and probably the biggest potential performance improvement / hold-up to merging folk2 in general. I've mostly been working on this for the last couple of weeks.
+This is the most important branch and probably the biggest potential performance improvement / hold-up to merging folk2. I've mostly been working on this for the last couple of weeks.
 The goals here are to 1. render page graphics to textures in memory and composite those instead of drawing directly to screen buffer and 2. always represent draw coordinates in 'meters with respect to the page', all the way until the draw op hits the GPU. That means that almost all draw statements will be stable across frames, since the outline of a page doesn't change in that coordinate frame. That means we can hopefully get rid of like 80% of database traffic every frame, because we don't have to recompute every draw statement when a page wiggles slightly.
@@ Line 168: / Line 166: @@
 (//and// it's a correctness improvement -- it means we should be able to draw text non-distorted in the real world, not relying on the projector pixel grid as we do now.)
-Broke up the Vulkan gpu.folk more cleanly into gpu.folk, gpu/images.folk, and a new gpu/writable-images.folk. (Had to improve C import/extend technology so we could access the gpu C code from the other modules.)
+Broke up the Vulkan gpu.folk more cleanly into gpu.folk, gpu/images.folk, and a new gpu/writable-images.folk. (Had to improve C import/extend technology so we could access the gpu C code from the other modules -- you can even use global variables from another module now, at least by a ''_ptr()'' getter I added. Had to figure out the right Vulkan render pass and image format stuff so we can still have single pipelines that we reuse across both image-target and screen-target draw operations.)
 Made more of the operations run concurrently (and in arbitrary threads) in normal Whens instead of one big GPU loop, which should make the system more robust (and mean that stuff like new-shader compilation doesn't freeze the whole system) (and let others extend it further in Folk modules and get access to state that's just stored in statements instead of loop-local variables), but does mean we need more locking and fences and semaphores and stuff to synchronize the GPU actions.
@@ Line 174: / Line 172: @@
 It's cool to think about how much draw logic can be implemented using idiomatic Folk constructs, and it's a good test of how expressive we can make Folk.
-**Laptop universe and safety concerns**
+Lots of weird images like this to test drawing in an image and then drawing that image onto the screen:
-I've been testing a lot on my laptop, which has produced some side issues:
+{{:newsletters:pasted:20250401-021250.png?350px}}
-  * TODO: Brought back hot swap!
-  * Now working on something like ''exit()'' (well, I'm calling it ''conclude!''), so I can write tests that terminate cleanly, so I can test the whole writable-image flow more easily.
-These two have a similar energy: they both are part of a whole constellation of safety concerns I have about Folk programs. Basically, a Folk program needs to be cancelable, it needs to be hot-swappable (that is, you retract it, clean up all its stuff, then assert a new version of it that needs to do a lot of the same things). That's a pretty high bar! [[https://social.omar.website/@omar/statuses/01JQM63EE1NKSZM5R7W8SCW4G1|Unix can do it with processes]], but we want to share more state than Unix processes do.
+(here, was also encountering a bug with line drawing that turned out to be that I forgot about triangle facing / vertex order.)
-**Inter-thread shared objects**
+=== Laptop universe and safety concerns ===
+I've been testing a lot on my laptop, which has produced some side quests:
+  * Brought back [[https://github.com/FolkComputer/folk/blob/4c96d35cf1e6fa35aaac5bf37cebb3126ef78331/virtual-programs/fswatch.folk|hot swap]] so you can live-edit even pretty deep virtual programs on laptop (needed this to debug GPU texture stuff more easily, turn off draw operations, mess around with coordinate transforms until it was correct)
+    * (isn't it cool that this can just be a blocking infinite loop now, because folk2 supports that? no need to think about the event loop or use ''readable'' or anything)
+  * New version of ''exit()'' -- I'm calling it ''__conclude'' -- so we can write tests that terminate cleanly, so I can test the whole writable-image flow more easily.
+  * New finalizer concept so we can release GPU resources in statements safely?
+    * {{newsletters:imag11e.png?300px}}
+These all have a similar energy: they both are part of a whole constellation of safety concerns I have about Folk programs. Basically, a Folk program needs to be cancelable, it needs to be hot-swappable (that is, you retract it, clean up all its stuff, then assert a new version of it that presumably wants to do a lot of the same things). That's a pretty high bar! [[https://social.omar.website/@omar/statuses/01JQM63EE1NKSZM5R7W8SCW4G1|Unix can do it with processes]], but we want to share more state than Unix processes do. How do you release all the resources cleanly, do it in the right order, etc?
+Another class of issue is queueing issues, where a thread (usually the sysmon thread) freezes and the rest of the system gets overrun by clock events and/or garbage and crashes:
+{{:newsletters:pasted:20250401-021357.png?450px}}
+=== Inter-thread shared objects ===
 (I still don't know if we'll merge this, but) I've been noodling some more on the concept of inter-thread shared objects (see [[newsletters/2025-02#shared-immutable-immortal-objects|last month]]), so we don't have to copy and reparse (or cache-lookup) term objects every time we use a statement in a different thread.
-** Mason's dive into not shimmering shared objects **
+I've been trying an approach from last month (which does feel actually-implementable) where we only /temporarily/ shimmer objects if they're inter-thread-shared, which only requires some pretty mechanical changes to all the shimmering functions in the Jim Tcl interpreter.
-Mason: I've been tinkering with preventing shared objects from shimmering, so that they can be safely shared across threads. It's been fairly straightforward, though there's some edge-cases I'm still trying to figure out (programs and variable names still need to shimmer so they can do fast execution and lookup, how to deal with program transfer?). I also haven't tested the code yet, so maybe it'll all blow up. ¯\_(ツ)_/¯
-</details>
+{{newsletters:c66f2b55-c5e0-4d2d-8560-12ec42f861bc-22511-0000170b2cd6a9d0.png?400px}}
+But there are so many trade-offs (now shimmering is just temporary for shared objects, now we have to track lots more stuff, etc) that I don't know if it's worth it compared to the current folk2 approach of just caching everything. Hmmmm.
+== Mason's dive into not shimmering shared objects ==
+Mason: I've been tinkering with preventing shared objects from shimmering, so that they can be safely shared across threads. It's been fairly straightforward, though there's some edge-cases I'm still trying to figure out (programs and variable names still need to shimmer so they can do fast execution and lookup, how to deal with program transfer?). I also haven't tested the code yet, so maybe it'll all blow up. ¯\_(ツ)_/¯
 ==== Outreach and other systems ====
-  * [[https://github.com/smj-edison|Mason Jones]] updated the [[https://folk.computer/supercollider|SuperCollider instructions]] to make booting up more consistent:
+  * Mason updated the [[https://folk.computer/supercollider|SuperCollider instructions]] to make booting up more consistent:
     * {{newsletters:screenshot-2025-03-31-at-5.17.34 pm.png?400px}}
-  * Mason and [[[[https://x.com/_rpf|Rob Fielding]] discussed wanting scrolling in the editor, how to edit printed programs, and physical debugging tools.
+  * Mason and [[https://x.com/_rpf|Rob Fielding]] discussed wanting scrolling in the editor, how to edit printed programs, and physical debugging tools.
-  * Paul: "pics from our meetup last week, had someone at the meetup talk about a "chemistry computer" which would allow users to manipulate cards that represented atoms and the computer would simulate their interactions, so we messed around with a (very) rough prototype"
+  * Paul from [[https://www.synergymill.com|SynergyMill]]: "pics from our meetup last week, had someone at the meetup talk about a 'chemistry computer' which would allow users to manipulate cards that represented atoms and the computer would simulate their interactions, so we messed around with a (very) rough prototype"
     * {{newsletters:march-5-meetup.jpg?0x250px}} {{newsletters:folk-chemistry.gif}}
   * Mason tested out the portability of ''folk-peridot'':
@@ Line 202: / Line 220: @@
   * Rob got an AAXA 4K1 projector and used it to fashion his own Folk lamp:
     * {{newsletters:img_9691.jpg?0x250}}{{newsletters:img_9685.jpg?0x250}}{{newsletters:iggmg.jpg?0x250}}
-  * Rob, Naveen Michaud-Agrawal, Paul from [[https://www.synergymill.com|SynergyMill]] (Paul is now in Philadelphia), and Andrés talked about how to talk about Folk (if you have another way of describing Folk to people, let us know over email or in our Discord):
+  * Rob, Naveen, Paul, and Andrés talked about how to talk about Folk (if you have another way of describing Folk to people, let us know over email or in our Discord):
     * {{newsletters:screenshot-2025-03-31-at-5.42.51 pm_discord_convo_1.png?500px}}
     * {{newsletters:screenshot-2025-03-31-at-5.43.03 pm.png?500px}}
@@ Line 226: / Line 244: @@
   * **Our next Folk open house is [[https://partiful.com/e/khO9CQ55Phn6FEsdK8dd|in the evening on Sunday, April 27]], at our studio in East Williamsburg, Brooklyn.**
-  * Omar: do another design pass, put up gadget design on GitHub
+  * Omar: do another gadget design pass, put up gadget design on GitHub
   * Omar: finish render-to-texture and make folk2 pull request
   * Maybe start working on dual-camera and self-calibration/refinement tech?
-  * Andrés:
+    * Omar: I'm bothered by the baseline-mediocre state of calibration and think we need some new approaches to keep the system calibrated over time -- maybe self-calibration by periodically projecting a tag somewhere on the table; maybe a lightweight form of calibration that just adjusts extrinsics and is easy; maybe an iterative calibration that continuously adjusts (as opposed to the 10-pose batch status quo) so you can really tune the fringes of the projector-camera volume
+  * Andrés: finish video playback
+  * Andrés: get dot detector demo working in 3D coordiantes
+  * Andrés: Kosmik integration demo
+  * Andrés: guest lecture at Christina Huang's Learning Analytics class
 ===== Links we've enjoyed =====
@@ Line 243: / Line 265: @@
   * https://poetical.day
   * https://boofcv.org/index.php?title=Performance:QrCode
+  * https://www.gofundme.com/f/rescue-and-restore-the-aesthedes-1
 ==== Andrés ====
   * [[https://wattenberger.com/thoughts/boo-chatbots|"Why Chatbots Are Not the Future"]] — A really thoughtful piece about how limiting chatbot interfaces are
   * [[https://wattenberger.com/thoughts/our-interfaces-have-lost-their-senses|"Our interfaces have lost their senses"]] — A sequel-of-sorts to the above piece that feel like a very Folk-endorsed idea: computers used to be big and tangible, and as they've gotten smaller, and more convenient, we've also lost all the texture and ability that comes with large, room-scale physical interfaces.