Folk Computer

User Tools

Site Tools

Trace: • 2025-03 • 2025-04 • 2024-08 • 2025-08 • 2024-03 • 2024-05 • 2024-11 • 2025-06 • orbitalkit • 2024-10
newsletters:2025-03

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
newsletters:2025-03 [2025/04/01 01:54] – [Folk gadget & QR scanner] osnrnewsletters:2025-03 [2025/04/01 20:40] (current) – discord
Line 28: Line 28:
     * {{newsletters:folk-shapes-tuneup.gif?0x200}} {{newsletters:folk-shapes-tuneup-text-large.jpeg?0x200}}     * {{newsletters:folk-shapes-tuneup.gif?0x200}} {{newsletters:folk-shapes-tuneup-text-large.jpeg?0x200}}
   * (Andrés) Dot detector — not much progress this month, will get to it early April   * (Andrés) Dot detector — not much progress this month, will get to it early April
-  * (Mason) [[https://github.com/FolkComputer/folk/pull/198|Basic program grouping]] — a program that runs a list of programs, even if they're not visible (it helps with saving table space as programs accumulate, I like stacking the programs below the group!)+  * ([[https://github.com/smj-edison|Mason Jones]]) [[https://github.com/FolkComputer/folk/pull/198|Basic program grouping]] — a program that runs a list of programs, even if they're not visible (it helps with saving table space as programs accumulate, I like stacking the programs below the group!)
  
 ==== Folk gadget & QR scanner ==== ==== Folk gadget & QR scanner ====
Line 42: Line 42:
 > It kind of half-seriously answers the critique of, like, restaurant menu QR codes, where scanning a QR code locks you into your phone & isn't a process that people around you can participate in. Here, you scan the QR code in a 'social' way, and 'in situ', and other people can see/participate too > It kind of half-seriously answers the critique of, like, restaurant menu QR codes, where scanning a QR code locks you into your phone & isn't a process that people around you can participate in. Here, you scan the QR code in a 'social' way, and 'in situ', and other people can see/participate too
  
-This is an interaction I'd had in mind for a few years, since I first got the Nebra AnyBeam in like 2021, so it's great to get to play it out now (and it was [[https://bsky.app/profile/omar.website/post/3lkm3yv4ups2c|not hard to write]] -- came up with the idea and implemented it in like 100 lines of code in a couple of hours before showing up).+This is an interaction I'd had in mind for a few years, since I first got the Nebra AnyBeam in like 2021, so it's great to get to play it out now. 
 + 
 +And it was [[https://bsky.app/profile/omar.website/post/3lkm3yv4ups2c|not hard to write]] -- came up with the idea and implemented it in like 100 lines of code in a couple of hours before showing up. (This is also a consequence of all the work we've put into making stable gadget hardware over the last few months; the gadget is a pretty good hardware platform for this in its current state as is!)
  
 Here, I'm scanning a URL that Owen put up for his project (https://owentrueblood.com/experiments/autonomous_response/), and that web page gets rendered (using ''exec wkhtmltoimage'') and projected by Folk on the right: Here, I'm scanning a URL that Owen put up for his project (https://owentrueblood.com/experiments/autonomous_response/), and that web page gets rendered (using ''exec wkhtmltoimage'') and projected by Folk on the right:
Line 76: Line 78:
 == Wi-Fi issues == == Wi-Fi issues ==
  
-TODO: Wi-Fi failing+I mentioned this last month -- I've had such trouble getting the gadget (with its Orange Pi 5 and USB Wi-Fi dongle) connected in wild environments (where it doesn't know the Wi-Fi upfront). The self-access point mode on the dongle just doesn't seem to work.
  
 == Successful calibration == == Successful calibration ==
  
-Following up on [[newsletters/2025-02#calibrating-the-new-gadget|last month]], I did successfully calibrate this (new, ELP dual-camera) gadget.+Following up on [[newsletters/2025-02#calibrating-the-new-gadget|last month]], I did successfully calibrate this (new, ELP dual-camera) gadget. 
  
 (it hasn't been a priority, but I figured I might as well since I've been bringing it out so much for QR, and everything was in place already to try it) (it hasn't been a priority, but I figured I might as well since I've been bringing it out so much for QR, and everything was in place already to try it)
  
-wasn't too bad! the calibration is a bit off when trying to use it on top of folk0 (TODO: photo?), but I hope some of the physical improvements below will help.+wasn't too bad! the calibration is a bit off -- a couple of centimeters -- when trying to use it on top of folk0, but I hope some of the physical improvements below will help.
  
 === Upcoming === === Upcoming ===
Line 96: Line 98:
     * display Wi-Fi status like how we display battery status? (can we do both of these in a nicer/more ambient way?)     * display Wi-Fi status like how we display battery status? (can we do both of these in a nicer/more ambient way?)
   * Back panel is getting looser over time and one bolt never really stuck; probably should use heat inserts to screw it in. it comes on and off a lot, too...   * Back panel is getting looser over time and one bolt never really stuck; probably should use heat inserts to screw it in. it comes on and off a lot, too...
-    * have a good way to expose a USB port or two? they're just generally handy+    * have a good way to expose a USB port or two on the back panel too? they're just generally handy
   * Projector swivels around still (which could be very bad, breaks calibration).   * Projector swivels around still (which could be very bad, breaks calibration).
     * Maybe use elastics and slits in the top?     * Maybe use elastics and slits in the top?
Line 103: Line 105:
   * Button is fraying over time as it gets bent outward. Needs a cover that you press on top, like in the original grip design (I made a basic casing for our button but not a cover)   * Button is fraying over time as it gets bent outward. Needs a cover that you press on top, like in the original grip design (I made a basic casing for our button but not a cover)
   * microSD card sometimes totally fails either before boot or while the system is running, need to power cycle   * microSD card sometimes totally fails either before boot or while the system is running, need to power cycle
-    * use M.2 SSD instead?+    * use M.2 SSD instead? I ordered a couple
  
  
Line 146: Line 148:
 (Even in folk1, Collect is implemented natively in C, so it's not surprising that this would be a step backward in performance.) (Even in folk1, Collect is implemented natively in C, so it's not surprising that this would be a step backward in performance.)
  
-I made a new folk2 branch where Collect is implemented in C.+So I made a new folk2 branch where Collect is implemented in C, which should cut almost all of that database traffic.
  
 It feels a lot more responsive compared to main folk2. The major bug (and why we haven't merged it into main folk2) is that it seems to introduce long periods where pages just blink out and vanish, like this: It feels a lot more responsive compared to main folk2. The major bug (and why we haven't merged it into main folk2) is that it seems to introduce long periods where pages just blink out and vanish, like this:
Line 156: Line 158:
 === Page quads === === Page quads ===
  
-This is the most important branch and probably the biggest potential performance improvement / hold-up to merging folk2 in general. I've mostly been working on this for the last couple of weeks. +This is the most important branch and probably the biggest potential performance improvement / hold-up to merging folk2. I've mostly been working on this for the last couple of weeks. 
  
 The goals here are to 1. render page graphics to textures in memory and composite those instead of drawing directly to screen buffer and 2. always represent draw coordinates in 'meters with respect to the page', all the way until the draw op hits the GPU. That means that almost all draw statements will be stable across frames, since the outline of a page doesn't change in that coordinate frame. That means we can hopefully get rid of like 80% of database traffic every frame, because we don't have to recompute every draw statement when a page wiggles slightly. The goals here are to 1. render page graphics to textures in memory and composite those instead of drawing directly to screen buffer and 2. always represent draw coordinates in 'meters with respect to the page', all the way until the draw op hits the GPU. That means that almost all draw statements will be stable across frames, since the outline of a page doesn't change in that coordinate frame. That means we can hopefully get rid of like 80% of database traffic every frame, because we don't have to recompute every draw statement when a page wiggles slightly.
Line 164: Line 166:
 (//and// it's a correctness improvement -- it means we should be able to draw text non-distorted in the real world, not relying on the projector pixel grid as we do now.) (//and// it's a correctness improvement -- it means we should be able to draw text non-distorted in the real world, not relying on the projector pixel grid as we do now.)
  
-Broke up the Vulkan gpu.folk more cleanly into gpu.folk, gpu/images.folk, and a new gpu/writable-images.folk. (Had to improve C import/extend technology so we could access the gpu C code from the other modules.)+Broke up the Vulkan gpu.folk more cleanly into gpu.folk, gpu/images.folk, and a new gpu/writable-images.folk. (Had to improve C import/extend technology so we could access the gpu C code from the other modules -- you can even use global variables from another module now, at least by a ''_ptr()'' getter I added. Had to figure out the right Vulkan render pass and image format stuff so we can still have single pipelines that we reuse across both image-target and screen-target draw operations.)
  
 Made more of the operations run concurrently (and in arbitrary threads) in normal Whens instead of one big GPU loop, which should make the system more robust (and mean that stuff like new-shader compilation doesn't freeze the whole system) (and let others extend it further in Folk modules and get access to state that's just stored in statements instead of loop-local variables), but does mean we need more locking and fences and semaphores and stuff to synchronize the GPU actions. Made more of the operations run concurrently (and in arbitrary threads) in normal Whens instead of one big GPU loop, which should make the system more robust (and mean that stuff like new-shader compilation doesn't freeze the whole system) (and let others extend it further in Folk modules and get access to state that's just stored in statements instead of loop-local variables), but does mean we need more locking and fences and semaphores and stuff to synchronize the GPU actions.
  
 It's cool to think about how much draw logic can be implemented using idiomatic Folk constructs, and it's a good test of how expressive we can make Folk. It's cool to think about how much draw logic can be implemented using idiomatic Folk constructs, and it's a good test of how expressive we can make Folk.
 +
 +Lots of weird images like this to test drawing in an image and then drawing that image onto the screen:
 +
 +{{:newsletters:pasted:20250401-021250.png?350px}}
 +
 +(here, was also encountering a bug with line drawing that turned out to be that I forgot about triangle facing / vertex order.)
  
 === Laptop universe and safety concerns === === Laptop universe and safety concerns ===
  
-I've been testing a lot on my laptop, which has produced some side issues: +I've been testing a lot on my laptop, which has produced some side quests: 
-  * TODO: Brought back hot swap! + 
-  * Now working on something like ''exit()'' (well, I'm calling it ''conclude!''), so I can write tests that terminate cleanly, so I can test the whole writable-image flow more easily.+  * Brought back [[https://github.com/FolkComputer/folk/blob/4c96d35cf1e6fa35aaac5bf37cebb3126ef78331/virtual-programs/fswatch.folk|hot swap]] so you can live-edit even pretty deep virtual programs on laptop (needed this to debug GPU texture stuff more easily, turn off draw operations, mess around with coordinate transforms until it was correct) 
 +    * (isn't it cool that this can just be a blocking infinite loop now, because folk2 supports that? no need to think about the event loop or use ''readable'' or anything) 
 +  * New version of ''exit()'' -- I'm calling it ''__conclude'' -- so we can write tests that terminate cleanly, so I can test the whole writable-image flow more easily. 
 +  * New finalizer concept so we can release GPU resources in statements safely? 
 +    * {{newsletters:imag11e.png?300px}} 
 + 
 +These all have a similar energy: they both are part of a whole constellation of safety concerns I have about Folk programs. Basically, a Folk program needs to be cancelable, it needs to be hot-swappable (that is, you retract it, clean up all its stuff, then assert a new version of it that presumably wants to do a lot of the same things). That's a pretty high bar! [[https://social.omar.website/@omar/statuses/01JQM63EE1NKSZM5R7W8SCW4G1|Unix can do it with processes]], but we want to share more state than Unix processes do. How do you release all the resources cleanly, do it in the right order, etc? 
 + 
 +Another class of issue is queueing issues, where a thread (usually the sysmon thread) freezes and the rest of the system gets overrun by clock events and/or garbage and crashes: 
 + 
 +{{:newsletters:pasted:20250401-021357.png?450px}}
  
-These two have a similar energy: they both are part of a whole constellation of safety concerns I have about Folk programs. Basically, a Folk program needs to be cancelable, it needs to be hot-swappable (that is, you retract it, clean up all its stuff, then assert a new version of it that needs to do a lot of the same things). That's a pretty high bar! [[https://social.omar.website/@omar/statuses/01JQM63EE1NKSZM5R7W8SCW4G1|Unix can do it with processes]], but we want to share more state than Unix processes do. 
  
 === Inter-thread shared objects === === Inter-thread shared objects ===
  
 (I still don't know if we'll merge this, but) I've been noodling some more on the concept of inter-thread shared objects (see [[newsletters/2025-02#shared-immutable-immortal-objects|last month]]), so we don't have to copy and reparse (or cache-lookup) term objects every time we use a statement in a different thread. (I still don't know if we'll merge this, but) I've been noodling some more on the concept of inter-thread shared objects (see [[newsletters/2025-02#shared-immutable-immortal-objects|last month]]), so we don't have to copy and reparse (or cache-lookup) term objects every time we use a statement in a different thread.
 +
 +I've been trying an approach from last month (which does feel actually-implementable) where we only /temporarily/ shimmer objects if they're inter-thread-shared, which only requires some pretty mechanical changes to all the shimmering functions in the Jim Tcl interpreter.
 +
 +{{newsletters:c66f2b55-c5e0-4d2d-8560-12ec42f861bc-22511-0000170b2cd6a9d0.png?400px}}
 +
 +But there are so many trade-offs (now shimmering is just temporary for shared objects, now we have to track lots more stuff, etc) that I don't know if it's worth it compared to the current folk2 approach of just caching everything. Hmmmm.
  
 == Mason's dive into not shimmering shared objects == == Mason's dive into not shimmering shared objects ==
Line 188: Line 211:
 ==== Outreach and other systems ==== ==== Outreach and other systems ====
  
-  * [[https://github.com/smj-edison|Mason Jones]] updated the [[https://folk.computer/supercollider|SuperCollider instructions]] to make booting up more consistent:+  * Mason updated the [[https://folk.computer/supercollider|SuperCollider instructions]] to make booting up more consistent:
     * {{newsletters:screenshot-2025-03-31-at-5.17.34 pm.png?400px}}     * {{newsletters:screenshot-2025-03-31-at-5.17.34 pm.png?400px}}
   * Mason and [[https://x.com/_rpf|Rob Fielding]] discussed wanting scrolling in the editor, how to edit printed programs, and physical debugging tools.   * Mason and [[https://x.com/_rpf|Rob Fielding]] discussed wanting scrolling in the editor, how to edit printed programs, and physical debugging tools.
Line 197: Line 220:
   * Rob got an AAXA 4K1 projector and used it to fashion his own Folk lamp:   * Rob got an AAXA 4K1 projector and used it to fashion his own Folk lamp:
     * {{newsletters:img_9691.jpg?0x250}}{{newsletters:img_9685.jpg?0x250}}{{newsletters:iggmg.jpg?0x250}}     * {{newsletters:img_9691.jpg?0x250}}{{newsletters:img_9685.jpg?0x250}}{{newsletters:iggmg.jpg?0x250}}
-  * Rob, Naveen Michaud-Agrawal, Paul, and Andrés talked about how to talk about Folk (if you have another way of describing Folk to people, let us know over email or in our Discord):+  * Rob, Naveen, Paul, and Andrés talked about how to talk about Folk (if you have another way of describing Folk to people, let us know over email or in our Discord):
     * {{newsletters:screenshot-2025-03-31-at-5.42.51 pm_discord_convo_1.png?500px}}     * {{newsletters:screenshot-2025-03-31-at-5.42.51 pm_discord_convo_1.png?500px}}
     * {{newsletters:screenshot-2025-03-31-at-5.43.03 pm.png?500px}}     * {{newsletters:screenshot-2025-03-31-at-5.43.03 pm.png?500px}}
Line 224: Line 247:
   * Omar: finish render-to-texture and make folk2 pull request   * Omar: finish render-to-texture and make folk2 pull request
   * Maybe start working on dual-camera and self-calibration/refinement tech?   * Maybe start working on dual-camera and self-calibration/refinement tech?
-  * Andrés: continue work on video playback, fix remaining bugs+    * Omar: I'm bothered by the baseline-mediocre state of calibration and think we need some new approaches to keep the system calibrated over time -- maybe self-calibration by periodically projecting a tag somewhere on the table; maybe a lightweight form of calibration that just adjusts extrinsics and is easy; maybe an iterative calibration that continuously adjusts (as opposed to the 10-pose batch status quo) so you can really tune the fringes of the projector-camera volume 
 +  * Andrés: finish video playback 
 +  * Andrés: get dot detector demo working in 3D coordiantes 
 +  * Andrés: Kosmik integration demo 
 +  * Andrés: guest lecture at Christina Huang's Learning Analytics class
  
 ===== Links we've enjoyed ===== ===== Links we've enjoyed =====
Line 238: Line 265:
   * https://poetical.day   * https://poetical.day
   * https://boofcv.org/index.php?title=Performance:QrCode   * https://boofcv.org/index.php?title=Performance:QrCode
 +  * https://www.gofundme.com/f/rescue-and-restore-the-aesthedes-1
  
 ==== Andrés ==== ==== Andrés ====
   * [[https://wattenberger.com/thoughts/boo-chatbots|"Why Chatbots Are Not the Future"]] — A really thoughtful piece about how limiting chatbot interfaces are   * [[https://wattenberger.com/thoughts/boo-chatbots|"Why Chatbots Are Not the Future"]] — A really thoughtful piece about how limiting chatbot interfaces are
   * [[https://wattenberger.com/thoughts/our-interfaces-have-lost-their-senses|"Our interfaces have lost their senses"]] — A sequel-of-sorts to the above piece that feel like a very Folk-endorsed idea: computers used to be big and tangible, and as they've gotten smaller, and more convenient, we've also lost all the texture and ability that comes with large, room-scale physical interfaces.   * [[https://wattenberger.com/thoughts/our-interfaces-have-lost-their-senses|"Our interfaces have lost their senses"]] — A sequel-of-sorts to the above piece that feel like a very Folk-endorsed idea: computers used to be big and tangible, and as they've gotten smaller, and more convenient, we've also lost all the texture and ability that comes with large, room-scale physical interfaces.
newsletters/2025-03.1743472473.txt.gz · Last modified: 2025/04/01 01:54 by osnr
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki