Folk Computer

User Tools

Site Tools

Trace: • 2024-02
newsletters:2024-02

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
newsletters:2024-02 [2024/02/29 15:40] – [Friends and outreach] osnrnewsletters:2024-02 [2024/02/29 22:22] (current) – osnr
Line 13: Line 13:
     * Integration with [[https://cuttle.xyz/@forresto/folk-computer-cuttle-xyz-FL0EJkXOEVCd|Cuttle]]: {{newsletters:image.webp?200}}     * Integration with [[https://cuttle.xyz/@forresto/folk-computer-cuttle-xyz-FL0EJkXOEVCd|Cuttle]]: {{newsletters:image.webp?200}}
  
-  * Andrés magazine article highlight+  * Andrés made a demo of a Folk-highlighted quote using an excerpt from [[ https://spikeartmagazine.com/articles/essay-emily-dickinson-apple-computer-house|this Spike magazine article about Folk]]. The program displays a letter-page-sized border and highlights the text of the quote on the printed-out article. To the right is a program that pulls out the highlighted text and projects it. This is a simple proof of concept, but you could imagine feeding these quotes into a spatial note-taking system on the table, or associating different kinds of data (definitions, related articles, comments on the same passage) to the highlights. 
 +      * {{newsletters:highlightquotedemo.jpg?500px}}
  
 (Omar: Both Forrest and Vedran's demos [and the receipt printer work from last year] indicate the potential importance of custom regions and accurate calibration, instead of the hard-coded 'half-size page' that we currently have. Ideally, it would be easy to make the program region follow the outline of the hex tile or the train or whatever else.) (Omar: Both Forrest and Vedran's demos [and the receipt printer work from last year] indicate the potential importance of custom regions and accurate calibration, instead of the hard-coded 'half-size page' that we currently have. Ideally, it would be easy to make the program region follow the outline of the hex tile or the train or whatever else.)
Line 22: Line 23:
  
 ==== Template matching ==== ==== Template matching ====
 +
 +Andrés: I've spent most of this month working on template matching (finding the location of an image in a larger image) to enable tracking custom game pieces in Folk. I wanted a basic demo where you can:
 +
 +  - capture a small, square image from the table, this is known as a template.
 +  - run a program that continuously searches for the template and highlight its location
 +
 +To that end, I wrapped a few programs into this "Image Slice Matcher" folder made from cardstock:
 +
 +{{newsletters:templatefoldercover.jpeg?300px}}
 +
 +{{newsletters:templatefolderinside.jpeg?400px}}
 +
 +When you open the folder, the left side includes two programs: on the full-size page is a program titled "slice" that will show if you've captured a template image, plus a program that pulls out to the side allowing you to trigger the template capture. Like so:
 +
 +{{newsletters:templatecapture.gif}}
 +
 +In a previous version, you had to manually take a photo of the table in order to recognize a template. Now the program reads the global camera image from Folk's main process. This allows the program that looks for and highlights matches to operate much faster and work continuously. Here it is searching for the cover of a small zine as I move it around the desk:
 +
 +{{newsletters:templatematcherdemo.gif}}
 +
 +Next month, I'll be extending this to track game pieces in a booklet of games I've been working on.
  
 ==== Parallel evaluator ==== ==== Parallel evaluator ====
Line 38: Line 60:
  
 We have reached this point again; I'm regularly frustrated with how slow things feel on folk0 when more than 1-2 things are on the table, even though low-load performance is still pretty good. (I think a lot of people's home systems don't suffer from this as much because they don't have random stuff out, they're working on one thing at a time, and they're mostly using it on their own, not showing it off.) We have reached this point again; I'm regularly frustrated with how slow things feel on folk0 when more than 1-2 things are on the table, even though low-load performance is still pretty good. (I think a lot of people's home systems don't suffer from this as much because they don't have random stuff out, they're working on one thing at a time, and they're mostly using it on their own, not showing it off.)
 +
 +But I'm hoping that this rewrite will be a permanent improvement where each program doesn't have to make latency worse at the margin, once we have scheduling and preemptive multitasking -- we'll be much more like a real operating system where the marginal program doesn't slow down the rest of the programs.
  
 ---- ----
Line 60: Line 84:
 ---- ----
  
-Example of how you'll be able to write programs/Whens that run concurrently, with arbitrary blocking / loops:+Example of how you'll be able to write programs/Whens that run concurrently, with arbitrary blocking & loops:
  
 {{newsletters:514a9267-67aa-471f-9bad-9cdd24b99173-1088-0001c53f776cebc5.png?500}} {{newsletters:514a9267-67aa-471f-9bad-9cdd24b99173-1088-0001c53f776cebc5.png?500}}
  
-This is useful for talking to webcams: you can have a virtual program that is a simple infinite loop, where the body blocks on the webcam read from Linux, then ''Hold!''s a statement with the resulting camera frame. Or for making HTTP requests: you can write a program that literally does ''exec curl'' and then uses the result without halting the rest of Folk.+This is useful, for example, for talking to webcams: you can have a virtual program that is a simple infinite loop, where the body blocks on the webcam read from Linux, then ''Hold!''s a statement with the resulting camera frame. Or for making HTTP requests: you can write a program that literally does ''exec curl'' and then uses the result without halting the rest of Folk. I really like that naive straight-line imperative code will just work and do the reasonable and expected thing. 
 + 
 +Also notice how the other thread gets arbitrary samples of the counter value (0, 97, 107, etc), it doesn't actually evaluate for every intermediate state. This is the behavior that you want for statements! If processing takes a while, you don't want to fall behind by processing intermediate page locations or whatever as the user moves a page around, you want to catch up to the latest stuff as soon as possible.  
 + 
 +(If you _do_ want to process intermediate states, you want something with different semantics from normal statements, like event statements or something. This is an open question.) 
 + 
 +---- 
 + 
 +{{:newsletters:pasted:20240229-160222.png?500}} 
 + 
 +(under the new evaluator, I'm hoping you can just beam stuff out to the GPU at 60fps, using ''Query!'' to sample from whatever happens to be in the database at that moment; you don't need to wait for a system convergence that includes all programs)
  
 ---- ----
  
-Ported the web server, which (elegantly) can now just be a normal virtual program, since its blocking on clients/sockets is perfectly OK in the new evaluator.+Ported the web server, which (elegantly) can now just be a normal virtual program, since it blocking on clients/sockets is perfectly OK in the new evaluator.
  
 {{newsletters:7231068f-f116-4e12-a2e9-b600f8233bb7-1088-0001c6f584d11daf.png?400}} {{newsletters:7231068f-f116-4e12-a2e9-b600f8233bb7-1088-0001c6f584d11daf.png?400}}
  
-Ported most virtual programs (camera, AprilTag, GPU/display). Also elegant: finally trashing the pi/ directory and inlining everything (talking to GPU, cameras, other libraries) into the virtual programs. (We want to do this partly to make everything live-editable inside the system; it should help with readability/understandability too.)+Ported most virtual programs (camera, AprilTag, GPU/display). Also elegant: finally trashing the pi/ directory and inlining everything (talking to GPU, cameras, other libraries) into the virtual programs. (We want to do this partly to make everything live-editable inside the system; it should help with readability/understandability, too, since you'll know exactly what files to read to understand those subsystems, and there'll be fewer intimidating random folders/files in the repo.)
  
-Even mostly works on my laptop (because we're not reliant on ''fork'' which completely breaks GPU access on macOS):+Even mostly works on my laptop (because we're no longer reliant on ''fork'' which completely breaks GPU access on macOS):
  
 {{newsletters:ee71abc5-035d-4ea3-87a1-7c672d51c6c2-1088-0001c6db080fde93.png?400}} {{newsletters:ee71abc5-035d-4ea3-87a1-7c672d51c6c2-1088-0001c6db080fde93.png?400}}
Line 93: Line 127:
  
 {{:newsletters:pasted:20240229-153535.png?500}} {{:newsletters:pasted:20240229-153535.png?500}}
 +
 +----
 +
 +Next steps here are to figure out the data structures/locking/atomics so that tests reliably pass, then do a run on a table with all the major components of the system (camera, AprilTags, GPU) and see how it performs compared to the old evaluator (that end-to-end test is really the only way to find out, I think)
 +
 ==== RFID ==== ==== RFID ====
  
Line 110: Line 149:
     * {{:newsletters:pasted:20240229-135506.png?400}}     * {{:newsletters:pasted:20240229-135506.png?400}}
   * Now trying to speed up the IB decoder even more to give us more headroom. Simpler pulse detector that uses a window instead of needing to scan 2*buf, removing heap allocations on the decode path, stuff like that   * Now trying to speed up the IB decoder even more to give us more headroom. Simpler pulse detector that uses a window instead of needing to scan 2*buf, removing heap allocations on the decode path, stuff like that
-    * (Sometimes I wonder if I'd be better served writing Verilog instead...)+    * (sometimes I wonder if I'd be better served writing Verilog instead) 
 + 
 +There must be some bit of hidden state in the IB radio that causes it to go off the rails and consistently fail after that 1-2 rounds... fix that & fix error recovery, then set up OOB(s), then set up localization.
  
 ==== Friends and outreach ==== ==== Friends and outreach ====
Line 128: Line 169:
     * We even tried hanging the webcam by its cable, just for fun:     * We even tried hanging the webcam by its cable, just for fun:
       * {{newsletters:img_4408.mp4?200}}       * {{newsletters:img_4408.mp4?200}}
 +
 ==== Other work ==== ==== Other work ====
  
   * Got folk0's printer on USB instead of Wi-Fi after it completely failed during open house   * Got folk0's printer on USB instead of Wi-Fi after it completely failed during open house
     * (It's been an irritant for a long time, it failing to get on the network or getting on the wrong one or ending up in weird dead-end states)     * (It's been an irritant for a long time, it failing to get on the network or getting on the wrong one or ending up in weird dead-end states)
-  * Andrés fixed editor bug TODO+  * Andrés fixed a bug in the table editor that was preventing printing more than one program from an editor. Now you can print to your heart's desire right from the table.
  
 ===== What we'll be up to in March ===== ===== What we'll be up to in March =====
Line 138: Line 180:
   * **[[https://partiful.com/e/d9l6Mj6IwL0Y2KCMeoNl|Our next Folk open house will be on the evening of Thursday, March 21]], at our studio in East Williamsburg, Brooklyn.**   * **[[https://partiful.com/e/d9l6Mj6IwL0Y2KCMeoNl|Our next Folk open house will be on the evening of Thursday, March 21]], at our studio in East Williamsburg, Brooklyn.**
  
-  * New collaboration (to be announced) on deeper phone/laptop integration and knowledge management+  * New collaboration (still TBA) on deeper phone/laptop integration and knowledge management
     * (Omar: I did some prototyping last weekend on localizing a phone using its camera + a tag on a laptop screen. I feel like you could do clever stuff with phone sensors and dead reckoning and smoothing to make this a nice experience even if tag tracking cuts out. Ideally you could just use a valid QR code, not even a specialized fiducial. Also thinking about Joy-Cons and Wii Remotes and how they do mouse-like pointing.)     * (Omar: I did some prototyping last weekend on localizing a phone using its camera + a tag on a laptop screen. I feel like you could do clever stuff with phone sensors and dead reckoning and smoothing to make this a nice experience even if tag tracking cuts out. Ideally you could just use a valid QR code, not even a specialized fiducial. Also thinking about Joy-Cons and Wii Remotes and how they do mouse-like pointing.)
 +  
 +  * Fine-tuning the template matching and using it to create small board-game interactions
  
   * Polishing Folk/CNC interface for use by other people at Hex House and in general   * Polishing Folk/CNC interface for use by other people at Hex House and in general
-    * USB stick install / Linux distro (can we make it a turnkey install if you have a CNC machine and want a Folk UI for it?) +    * USB stick install / Linux distro packaging, both Owen's demo and Folk more broadly (this will be a big deal for getting people up and running, I think) 
-    * System reliability (can we feel sure that people can walk up to it and use it?)+    * Better Web UI
     * Finishing up the 3D/high-accuracy calibration work from late last year to actually do the machine preview in Folk     * Finishing up the 3D/high-accuracy calibration work from late last year to actually do the machine preview in Folk
  
Line 150: Line 194:
   * RFID: Continuing to try to debug why we lose contact after 1-2 rounds of IB communication, work on the imgui GUI, hopefully start on OOB   * RFID: Continuing to try to debug why we lose contact after 1-2 rounds of IB communication, work on the imgui GUI, hopefully start on OOB
  
-  * Parallel evaluator: continuing to work on locking and scheduling, hopefully do a table test after that to see how it performs+  * Parallel evaluator: continuing to work on locking and scheduling, goal is to do a table test after that to see how it performs 
 ===== Links we've enjoyed ===== ===== Links we've enjoyed =====
  
Line 157: Line 202:
   * https://electronics.stackexchange.com/questions/110506/dead-reckoning-with-accelerometer-gyro-possible   * https://electronics.stackexchange.com/questions/110506/dead-reckoning-with-accelerometer-gyro-possible
   * http://factor-language.blogspot.com/2008/04/performance-improvements.html   * http://factor-language.blogspot.com/2008/04/performance-improvements.html
 +  * https://lwn.net/Articles/565734/
 +  * https://itty.bitty.site/
 +  * https://downpour.games
  
 ==== Andrés ==== ==== Andrés ====
 +
 +  * [[https://github.com/prathyvsh/Illustrated-Primers|Illustrated Primers
 + — Books that makes heavy use of visuals for scientific communication]]
 +  * [[https://alarmingdevelopment.org/?p=1708|"you should write an Onward! paper"]]
newsletters/2024-02.1709221227.txt.gz · Last modified: by osnr
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki