newsletters:2026-09
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| newsletters:2026-09 [2026/10/15 02:34] – Convert to table format cwervo | newsletters:2026-09 [2026/10/24 00:10] (current) – discordbot | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== | + | ====== |
| - | Folk Computer | + | {{htmlmetatags> |
| + | metatag-media-og:image=(newsletters:img_8752.jpeg) | ||
| + | metatag-og: | ||
| - | ==== Base Installation | $30,000 ==== | ||
| - | ^ Category ^ Included Items ^ | + | Our next Folk open house will be in the evening on **[[https:// |
| - | | Hardware | Projector | | + | |
| - | | ::: | Camera | | + | |
| - | | ::: | Computer pre-flashed with folk | | + | |
| - | | ::: | Table-top laser printer | | + | |
| - | | ::: | Mounting hardware* | | + | |
| - | | Services | Installation service | + | |
| - | ==== Bespoke Large Installation | $40,000 ==== | ||
| - | ^ Category ^ Included Items ^ | + | ===== What we've been up to ===== |
| - | | Hardware | 4k projector | | + | |
| - | | ::: | 4k webcam @ 30 fps | | + | |
| - | | ::: | Beelink Computer w/ Folk | | + | |
| - | | Services | Consultation for custom mounting hardware (mounting hardware is an additional cost) | | + | |
| - | | ::: | Consultation in-person with an orientation workshop for you/your team (covering setup, programming in Folk, and system integration) | | + | |
| - | ==== Industrial Installation | Contact for pricing | + | ==== General system improvements |
| - | ^ Overview ^ Contact ^ | + | * editor: [[https:// |
| - | | For installing | + | * Fixes this issue where an editor pointing at two programs at once results in two unusable/ |
| + | * {{: | ||
| + | * (especially annoying when on a viewport, since the editor would always point at both the viewport and any program inside the viewport) | ||
| + | * [[https:// | ||
| + | * (for example, if you just put '' | ||
| + | * prelude: [[https:// | ||
| + | * Should finally fix the random space changer failures which would fill up the log! (was happening when space changer initialized to null changer and then filled later, but original changer function was not actually replaced) | ||
| + | * / | ||
| + | * [[https:// | ||
| + | * Python FFI: [[https:// | ||
| + | |||
| + | === Atomically === | ||
| + | |||
| + | [[https:// | ||
| + | |||
| + | I saw a lot of these issues with RC visitors who were making a bouncing ball (clock time atomically) whose color was based on color wheel (quad atomically). Blinking: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Fixes: | ||
| + | |||
| + | * [[https:// | ||
| + | * Should fix " | ||
| + | * [[https:// | ||
| + | * Support auto-atomically in first pattern in & join | ||
| + | * Collect retains previous version when atomic | ||
| + | * One-way hasConverged (latch convergence) to prevent ' | ||
| + | * Timeout by parent removal instead of by time of last convergence: | ||
| + | * You can also use the -keep option on the When to set a custom timeout, for things where you expect re-convergence to take a long time or just never want expiry (it's a ' | ||
| + | * [[https:// | ||
| + | |||
| + | Also comes with a [[https:// | ||
| + | |||
| + | {{: | ||
| + | |||
| + | === Graphics speedup === | ||
| + | |||
| + | Omar: As long as I can remember, the editor has been the slowest thing in the system, where you point an editor at a program and the whole system slows down noticeably. It's especially bad on programs that have a whole page of code (that' | ||
| + | |||
| + | I benchmarked it this month and found that a 2,000-glyph editor slows the system down to 8fps (and this is on the global GPU thread, so it slows everything else down too). | ||
| + | |||
| + | [[https:// | ||
| + | |||
| + | Next, I want to do more editing and terminal I/O in the system. I think we can also push draw call batching further and maybe then bring back the validation layer. | ||
| + | |||
| + | ==== RFID localization ==== | ||
| + | |||
| + | Current state of RFID localization is that it can talk to the tag at up to around 1.1 meters (but drops off afterward) and can also localize it reasonably well to within a couple cm. We went up from like 30% successful tag detection rate at 1.1m to around 90%: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | At home -- it talks to the tag fine on table but not once I move it further outward to couch: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | In September: | ||
| + | |||
| + | * **Fixed channel estimation bug**: we were mistakenly treating all bits as flipping between reflective state and non-reflective halfway through, but only ' | ||
| + | * Subtract baseline carrier wave to improve detection range | ||
| + | * Faster RFID protocol implementation (integer arithmetic) so we are better at making the ACK deadline | ||
| + | * Retain the channel at each hop so we can still localize even if we didn't get a valid hop last round | ||
| + | * Persist stuff to disk, run headless for testing loop | ||
| + | |||
| + | Hop alignment between RFID reader (in-band) and out-of-band localization helper radio looks pretty good: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Uncalibrated localization (we pick that first peak to probably be the line-of-sight path and tell us the estimated distance to the tag from that): | ||
| + | |||
| + | {{: | ||
| + | |||
| + | It's still a sort of coarse localization -- we need to use more fine phase data and also start triangulating from more than one OOB radio, I think. Would also be nice to increase speed. Will also do a Folk projector demo like what we used to have to make the detection accuracy more visceral. | ||
| + | |||
| + | ==== Object tracking / DrawTalking ==== | ||
| + | |||
| + | === DINOv2 (embedding model) === | ||
| + | |||
| + | Omar: This month, I've been trying object tracking using [[https:// | ||
| + | |||
| + | It lets you do // | ||
| + | |||
| + | Embeddings are fast -- you can compute an embedding of a pose or viewport in like 10ms -- and are fast on almost any hardware, even my low-end home Intel N95 GPU (which can run DINOv2 in 100 or 200ms but takes 2 seconds to run SAM2). | ||
| + | |||
| + | I started with pose registration that takes the embedding of the pose, and a ' | ||
| + | |||
| + | {{: | ||
| + | |||
| + | (Inelegant: pose registration step still requires SAM2, because SAM2 is how we cut the object out of the page/ | ||
| + | |||
| + | Then I started doing localization, | ||
| + | |||
| + | Here, the top heatmap shows bright army-toy-like regions, and the bottom heatmap shows bright cup-like regions (notice how they follow when I move the objects): | ||
| + | |||
| + | {{: | ||
| + | |||
| + | More heatmaps: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | == DINOv2 with normalized cross-correlation == | ||
| + | |||
| + | You can find the hottest point in the heatmap to get a coarse location of the cup or army toy or whatever, but to get a fine-grained location, you need a refinement step that uses something else. We could use SAM2 again here, but that seems slow and may suffer from distortion again. Better idea (inspired by my old ScreenMatcher project): use [[https:// | ||
| + | |||
| + | Here's DINOv2 with NCC, which is remarkably fast and accurate for a radially symmetric object like this cup: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | You can get pretty impressive fine centers: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Only problem is that this doesn' | ||
| + | |||
| + | == Rotation tracking == | ||
| + | |||
| + | Solution: 1. record multiple physical poses with the object rotated differently, | ||
| + | |||
| + | [[https:// | ||
| + | |||
| + | It tracks the orientation of this army toy fairly well (except it sometimes flips by 180 degrees, which makes sense for this toy): | ||
| + | |||
| + | {{: | ||
| + | |||
| + | == Next steps == | ||
| + | |||
| + | The center (fine-grained localization) is still a bit unstable and sometimes makes mistakes or jumps a few centimeters, | ||
| + | |||
| + | I did a lot of speedup work: do FFT convolution, | ||
| + | |||
| + | === DrawTalking integration === | ||
| + | |||
| + | We've started talking about the graphics pipeline that would be needed to draw many-point curves for sketches from DrawTalking. Some discussion about how to translate from DT procedural mindset to Folk reactive/ | ||
| + | |||
| + | {{: | ||
| + | |||
| + | The graphics optimizations for the editor should help a lot here. Instead of reusing the existing line popeline, we probably want a custom pipeline with push constants for point A, point B, and transform, and then only the transform usually varies from frame to frame. | ||
| + | |||
| + | I think this integration will be a good test of Folk performance for drawing complex objects and passing complex data through the DB (which is definitely something we want to support). Like a DT object with thousands or tens of thousands or hundreds of thousands of points. | ||
| + | |||
| + | ==== folk-cnc ==== | ||
| + | |||
| + | Omar: As part of the RTBW project, Brian, John Williams, and I have been mounting and bringing up the CNC system. | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Mounted steel strut to ceiling, hung 2 cameras and a projector (cantilevered on wood block). | ||
| + | |||
| + | {{: | ||
| + | |||
| + | It took a while to figure out how to mount the projector, because it projects in a triangle upward from its bottom plate, and our strut wasn't really in the right spot to mount it widescreen. We tried complicated rigs like this to move the projector vertically away from the strut: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | but felt uncomfortable mounting them. Eventually just went with turning it 90 degrees and doing the cantilever for now. As you can see, it's still not perfectly aligned with the left edge of the cut area on the bed, so we'll probably tweak it soon: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Weirdly, both of our Intel N95-based mini PCs seemed to have fried SSDs when I tried them. I ran into this issue on my home system after about a year, too. Replaced the SSD and put a 3D-printed spacer in to make the new SSD the right length: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Verified that the CNC machine itself works to cut some wood for the mount: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | === folk-convivial === | ||
| + | |||
| + | We also remounted folk-convivial from the ceiling, getting rid of the C-stand that was always at risk of tipping over (and took up floor space, and made the table impossible to move without toppling the stand): | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Good alignment of projector and camera now: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | === Telephoto camera === | ||
| + | |||
| + | For the RTBW project, I've been mounting [[https:// | ||
| + | |||
| + | I can configure the lens to the right field of view, but can't mechanically get it to the right place to focus it while keeping it all attached. Need to buy a ring adapter to push the C-mount lens further out, I think? Haven' | ||
| + | |||
| + | {{: | ||
| + | |||
| + | ==== [Expect] and rewindability ==== | ||
| + | Mason: Omar and I (among others) have been continuing to discuss what an [Expect] command would look like. The idea is you could use [Expect] like: | ||
| + | < | ||
| + | Expect Omar likes color / | ||
| + | Expect Mason likes color / | ||
| + | if {$omarColor eq $masonColor} { | ||
| + | Wish $this is labelled "Mason and Omar like the same color" | ||
| + | } else { | ||
| + | Wish $this is labelled "Mason and Omar like different colors" | ||
| + | } | ||
| + | </ | ||
| + | |||
| + | This doesn' | ||
| + | < | ||
| + | proc pointProject {fromPoint toSpace} { | ||
| + | Expect tag [lindex $fromPoint 1] has quad / | ||
| + | Expect tag $toSpace has quad /toQuad/ | ||
| + | # Conversion code here | ||
| + | return $converted | ||
| + | } | ||
| + | set projected [pointProject [point 0.1 0.1] 123] | ||
| + | </ | ||
| + | |||
| + | Now " | ||
| + | < | ||
| + | Expect the filename is / | ||
| + | set fd [open $filename] | ||
| + | set contents [read $fd] | ||
| + | # What if Expect gets a new value right here, and rolls back? [close] will | ||
| + | # never be called, so we'll leak the fd. | ||
| + | close $fd | ||
| + | </ | ||
| + | |||
| + | So, you might say, just close every resource that's still open, right? Well, that opens up a different can of worms (though this is more contrived): | ||
| + | < | ||
| + | Expect the filename is / | ||
| + | set fd [open $filename] | ||
| + | Hold! the fd is $fd | ||
| + | # What if Expect gets a new value here? It will roll back, closing fd in | ||
| + | # the process, even though fd stays visible. | ||
| + | </ | ||
| + | |||
| + | I think in general I'd prefer the second option, because it means that things are closed instead of leaked. | ||
| + | |||
| + | Another (more cursed) idea I had that stems from rewinding is propagating errors by rewinding to the problematic statement, and having the error raised there. For example: | ||
| + | < | ||
| + | Wish to draw a circle onto 123 with cnter [list 0.2 0.2] ;# Has cnter typo | ||
| + | Wish $this is labelled " | ||
| + | # Code has now run both wishes | ||
| + | </ | ||
| + | Later... | ||
| + | < | ||
| + | When /someone/ wishes to draw a circle onto /p/ with / | ||
| + | # Error is raised here, since no center was provided | ||
| + | } | ||
| + | </ | ||
| + | Now, the error would propagate up to the original wisher, and the code in that block would //rewind// back to the problematic statement. That means the " | ||
| + | |||
| + | < | ||
| + | < | ||
| + | This is pretty closely related to multi-resumable effect handlers, so I've been looking at what they do to implement the second resumption. Essentially what is needed is to be able to snapshot the interpreter and stack at a particular point in time, and to be able to restore that state. That means a non-recursive engine //groan//. Currently both Jimtcl and Zicl are what are called recursive interpreters, | ||
| + | |||
| + | A non-recursive engine on the other hand doesn' | ||
| + | |||
| + | But, with an NRE, we could suspend entire programs to multiplex interpreters, | ||
| + | |||
| + | Oh, one other detail, specific to rewinding: there needs to be some way to undo every side effect caused by the scope(s) during rewind. I'm thinking if we have an interpreter log of every side effect that happens, you could run that log in reverse when rewinding, up to the point of the Expect. This would avoid any side effects from being missed. The cool thing about this log is when running in reverse it might even go back up stack frames, so your Expect could be five function calls deep, and the log would just run all the way through the stack back to the Expect. | ||
| + | </ | ||
| + | |||
| + | |||
| + | ==== Open Sound Control API ==== | ||
| + | |||
| + | Paul: I am in the process of building out a Folk music toolkit, and one of the core components needed for this system is the ability to send OSC messages from my Folk machine to arbitrary OSC servers to control synthesizers. I managed to implement simple OSC send functionality which allows Folk users to send OSC messages using the native Folk '' | ||
| + | |||
| + | Next I will add support for OSC bundles, which will allow us to write proper tidalcycles-esque pattern languages native in Folk! | ||
| + | |||
| + | {{newsletters: | ||
| + | |||
| + | {{newsletters: | ||
| + | |||
| + | ==== Outreach ==== | ||
| + | |||
| + | === Interface Studies podcast === | ||
| + | |||
| + | Omar and Daniel showcased Folk on the first episode of " | ||
| + | |||
| + | {{youtube> | ||
| + | |||
| + | Andrés: I've watched [[https:// | ||
| + | |||
| + | {{newsletters: | ||
| + | |||
| + | === Recurse Center and SVA visitors === | ||
| + | |||
| + | Tracey Le has been bringing in smaller groups from the current Recurse Center batches who want to do some programming in our systems at 87R (and maybe bring knowledge and practices back to the RC system). | ||
| + | |||
| + | {{: | ||
| + | |||
| + | == Friday, September 4 == | ||
| + | |||
| + | {{: | ||
| + | |||
| + | == Friday, September 18 == | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Tracey has been showing the raw contour + color-wheel stuff to a lot of visitors: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | == Friday, September 25 == | ||
| + | |||
| + | {{: | ||
| + | |||
| + | We messed with the object-tracking label program to label with real-world centimeter positions: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | and to make big labels: | ||
| + | |||
| + | {{: | ||
| + | |||
| + | |||
| + | |||
| + | === Open house === | ||
| + | |||
| + | Had a pretty big open house (we've been having big open houses for the last few months). Tracey brought a bunch of people from Recurse Center; Alan Laidlaw stopped in for the first time in a while; Paul came up from Philadelphia. | ||
| + | |||
| + | {{: | ||
| + | |||
| + | Nice to have: New fast editor + lack of multi-editor bug when pointed at multiple things. | ||
| + | |||
| + | Ran into bug with editor still crashing/ | ||
| + | |||
| + | {{: | ||
| + | |||
| + | === Utah Meetup === | ||
| + | We had our first Utah meetup in Lehi, UT. We chatted about the reactive database and introduced the system to our host Michael. | ||
| + | |||
| + | We plan to do this monthly. If you are interested, reach out on discord or email <daniel@folk.computer> | ||
| + | |||
| + | {{: | ||
| + | |||
| + | |||
| + | ===== What we'll be up to in October ===== | ||
| + | |||
| + | * Our next [[https:// | ||
| + | * Omar: Ship RFID tracker; do projection demo of distance | ||
| + | * Omar: Build automatic folk-cnc driver and preview application | ||
| + | * Omar: Calibrate RTBW CNC system with 2 cameras | ||
| + | * Omar: More work to make editor usable on table | ||
| + | * Omar: Filter DINOv2+NCC+rotation tracker to be more stable; integrate with DrawTalking strokes | ||
| + | * Andrés: Lamp! I'm working on a gadget-inside-a-lamp project using a few [[https:// | ||
| + | |||
| + | ===== Links we've enjoyed ===== | ||
| + | |||
| + | ==== Andrés ==== | ||
| + | * [[https:// | ||
| + | * [[https:// | ||
| + | ==== Omar ==== | ||
| + | * http:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | * https:// | ||
| + | |||
| + | |||
| + | === paul_18583 | 2026-09-26 01:09 UTC === | ||
| + | who needs tidalcycles | ||
| + | |||
| + | {{newsletters: | ||
| - | ===== Installation Consulting Service ===== | ||
| - | ^ Package ^ Typical Services ^ Schedule / Details ^ | ||
| - | | Base Installation | 2-day virtual set up & orientation consultation (in-person available if in NYC) | **Day 1:** Discuss, measurements of the space, discuss physical space needs, and software/ | ||
| - | | Bespoke Large Installation | Custom mounting & in-person workshop | Consultation for custom mounting hardware (mounting hardware is an additional cost) \\ Consultation in-person with an orientation workshop for you/your team (covering setup, programming in Folk, and system integration) | | ||
newsletters/2026-09.1792031699.txt.gz · Last modified: by cwervo
