Differences

This shows you the differences between two versions of the page.

--- newsletters:2023-10 [2023/10/31 23:55] – [CNC and calibration] osnr
+++ newsletters:2023-10 [2023/11/01 23:01] (current) – admin
@@ Line 5: / Line 5: @@
 ==== Applications and demos ====
+  * Andrés: [[guides/button|New guide on how to make a paper button:]] [[guides/button|{{guides:3_6_yayyyyy.gif?300}}]]
+    * Really want to do more of these very specific technical/physical guides -- that's kind of what's nice about this newsletter, it can be very concrete and not explicitly philosophical but still communicate the underlying philosophical points
+  * [[https://cristobal.space|Cristóbal Sciutto]] has been working on a series of iterations of the concept of [[notes:tableshots|'tableshots']] ([[https://twitter.com/tobyshooters/status/1710395304669843548|tweet]])
+    * {{:notes:cristobal_tableshot.gif?0x200|}} {{newsletters:img_8594.jpg?0x200}}
+    * {{newsletters:img_8617.mp4}}
+      * (Omar: You can start to see the real motivation for this work on tableshots: building a hybrid digital-physical 'reading desk' where you can take 'screenshots' of passages of real books. It's a project I've been excited about for years, so I'm glad it's starting to happen.)
+      * {{:newsletters:pasted:20231101-040656.png?400}}
   * [[http://twitter.com/naveenma|Naveen Michaud-Agrawal]] made [[https://discord.com/channels/956758212152025098/956758650700046366/1160573888451854366|dials]] on cardboard tokens:
@@ Line 11: / Line 20: @@
   * Naveen also made an observability demo where you can point at a performance metric and see a live plot of that metric:
-    * {{newsletters:pxl_20231015_1434173842.mp4?250}}
+    * {{newsletters:pxl_20231015_1434173842.mp4?400}}
     * (also check out how his system uses [[https://gist.github.com/nmichaud/2a43c15a7c1810170c88e4ff37625239|dual tags]] and a different tag family -- it's always a nice example of the flexibility of Folk)
   * Naveen also made warping/portals:
-    * {{newsletters:pxl_20231010_0252000052.mp4?300}}
+    * {{:newsletters:pasted:20231101-224529.mp4?400}}
   * Andrés: [[https://x.com/cwervo/status/1716172470858760462|Pumpkins.]] Happy Halloween!
     * {{newsletters:img_2392.jpg?200}}
-  * Andrés: Embroidered AprilTag
+  * Andrés made an [[https://discord.com/channels/956758212152025098/956758650700046366/1159570264670085250|embroidered AprilTag]]:
+    * {{newsletters:apriltag-embroidery.gif?0x250}} {{newsletters:img_1765_large.jpeg?0x250}}
-  * [[https://cristobal.space|Cristóbal Sciutto]] has been working on a series of iterations of the concept of [[notes:tableshots|'tableshots']] ([[https://twitter.com/tobyshooters/status/1710395304669843548|tweet]])
-    * {{:notes:cristobal_tableshot.gif?0x200|}} {{newsletters:img_8594.jpg?0x200}}
-    * {{newsletters:img_8617.mp4}}
-      * (Omar: You can start to see the real motivation for this work on tableshots: building a hybrid digital-physical 'reading desk' where you can take 'screenshots' of passages of real books. It's a project I've been excited about for years, so I'm glad it's starting to happen.)
-  * Andrés: [[https://folk.computer/guides/button|New guide on how to make a paper button:]] [[https://folk.computer/guides/button|{{guides:3_6_yayyyyy.gif?300}}]]
-    * Really want to do more of these very specific technical/physical guides -- that's kind of what's nice about this newsletter, it can be very concrete and not explicitly philosophical but still communicate the underlying philosophical points
 ==== Outreach and new systems ====
@@ Line 61: / Line 62: @@
       * There's an interesting set of choices here. We want to use the parts we have (AprilTag emitter and detector, projector, camera, Web server, printer) and not bloat the system with new computer vision pipelines...
     * Talked about this in previous months -- this is exciting because full intrinsic projector/camera calibration will finally break us out of the 2D planes / page paradigm, and it'll let us have mobile systems (including phones that understand the world map), and high accuracy will break us out of coarse moving-pages-around -- will allow checking boxes, handwriting in text fields, etc
-    * Here is the 'calibration rig' right now... (the calibration virtual program sets up a Web endpoint where you get a calibration board PDF to print; then you'll measure its real-world metric and trigger end-to-end calibration in Folk) {{:newsletters:pasted:20231031-214225.png?500}} {{:newsletters:pasted:20231031-085430.png?300}}
+    * Here is the 'calibration rig' right now... (the calibration virtual program sets up a Web endpoint where you get a calibration board PDF to print; then you'll measure its real-world metric and trigger end-to-end calibration in Folk, which will project more AprilTags into the gaps on the board) {{:newsletters:pasted:20231031-214225.png?500}} {{:newsletters:pasted:20231031-085430.png?300}}
+    * It's inspiring to look at papers on projector-camera calibration, because the //baseline// from like 2009 -- not even the state of the art -- is error of like 0.7 pixels, while Folk still has error of like an entire inch.
+      * [[http://www.ok.sc.e.titech.ac.jp/res/PCS/publications/procams2009.pdf|{{:newsletters:pasted:20231101-011921.png?300}}]]
 === Owen's projection-mapped CNC 'print preview' prototype ===
@@ Line 88: / Line 91: @@
   * Cristóbal made a number of [[https://github.com/FolkComputer/folk/pull/108|small quality-of-life improvements]]: image scaling now works (easy to do now that we're on the GPU, just change the shader vertices); you can now do ''When $this has neighbor /n/'' without needing to explicitly wish for neighbors; he also papered over [[https://github.com/FolkComputer/folk/issues/105|a nasty collect bug]] for outlines
-=== TODO: Andrés's tabletop editor ===
+=== Andrés's tabletop editor ===
+  * Andrés has been working on the editor. It supports:
+      * Writing Folk code into a text area; as you type code is evaluated in realtime
+      * ''Ctrl + r'' — reset editor text to ''Wish $this is outlined green'', a basic piece of code to outline the active program.
+      * ''Ctrl + p'' — print out code to
+      * ''Ctrl + a/e'' — Move cursor to the beginning/end of the line
+      * ''Ctrl + u/k'' — Delete text from cursor to beginning of line
+      * Bugs to be worked out:
+        * Standardize on connecting to the keyboard via: ''Claim $this is keyboard $MACaddress''
+        * Drawing a green cursor-line object — debugging pic of this below
+        * Document setting a keyboard via Bluetooth
+  * {{newsletters:andres_editor_multi_line.jpeg?400}} {{newsletters:andres_editor_cursor_debugging.jpeg?300}}
 === Omar's proposal for reactive Variables ===
@@ Line 102: / Line 117: @@
-==== Graphics ====
+==== Display ====
   * Merged [[https://github.com/FolkComputer/folk/pull/93|the Vulkan branch]] and the [[https://github.com/FolkComputer/folk/pull/103|follow-up draw refactor branch]], which together basically completely reimplement Folk's display system
     * All Folk drawing is now GPU-accelerated! Drawing generally runs at 60fps now
-    * All Folk drawing now happens through wishes with keyword/optional arguments -- feels really clean and extensible. All drawing primitives now live in [[https://github.com/FolkComputer/folk/tree/main/virtual-programs/display|cleanly separated files in virtual-programs/display/]] (which also serve as examples of how to use the GPU)
+    * All Folk drawing now happens through wishes with keyword/optional arguments -- feels really clean and extensible. **All Display:: calls have been deprecated and will visibly warn on use.** All drawing primitives now live in [[https://github.com/FolkComputer/folk/tree/main/virtual-programs/display|cleanly separated files in virtual-programs/display/]] (which also serve as examples of how to use the GPU)
       * [[https://github.com/FolkComputer/folk/blob/e21bb5eac49dfa9ad1ee5646a76fa6638253bf11/virtual-programs/display/circle.folk|Example: circle.folk implements circles]]: {{:newsletters:pasted:20231031-005316.png?400}}
         * Notice how the rest pattern ''/...options/'' is used to take the tail of the statement as the ''options'' object, and then parameters can just be pulled out of that object
         * A program can draw using ''Wish to draw a circle with radius 3 center [list 200 300] thickness 2 color green'', for instance
     * Any (virtual or paper) program can now compile and run a custom shader; the built-in draw 'primitives' aren't privileged in any way :-)
-      * TODO: Document shader example
+      * On the to-do list: We [[https://github.com/FolkComputer/folk/issues/106|want to make a cleaner Shadertoy-like interface]] for writing pixel shaders, where can just shade a region or object and don't have to give clip vertices and stuff
-      * TODO: We [[https://github.com/FolkComputer/folk/issues/106|want to make a cleaner Shadertoy-like interface]] for writing pixel shaders, where can just shade a region or object and don't have to give clip vertices and stuff
         * {{:newsletters:pasted:20231031-005723.png?500}}
-    * Instructions (to install headless Vulkan on Linux / to install Folk) still need work but are reasonably solid except for Jacob's Nvidia system
+    * Instructions (to install headless Vulkan on Linux / to install Folk) still need work but seem reasonably OK, except for Nvidia GPUs (which have a different Vulkan implementation)
-  * [[https://arcades.agency|Arcade Wise]] added support for custom fonts and wrote a guide, so you can [[https://folk.computer/guides/createafont|import pretty much arbitrary TrueType and OpenType fonts]] by generating an SDF atlas, and they automatically become available inside Folk
+  * [[https://arcades.agency|Arcade Wise]] added [[https://github.com/FolkComputer/folk/pull/97|support for custom fonts]] and wrote a guide, so you can [[guides/createafont|import pretty much arbitrary TrueType and OpenType fonts]] by generating an SDF atlas, and they automatically become available inside Folk
-    * TODO: Andrés made a font booklet that can list fonts, show examples of fonts, etc.
-  * Naveen: Contributed a drawing primitive for [[https://discord.com/channels/956758212152025098/956758650700046366/1160023903251476591|arcs]]
+  * Naveen [[https://github.com/FolkComputer/folk/pull/98|contributed]] a drawing primitive for [[https://discord.com/channels/956758212152025098/956758650700046366/1160023903251476591|arcs]]
     * {{newsletters:pxl_20231007_0119124392.mp4}}
+  * Andrés made a few programs for fonts:
+    * {{newsletters:andres_font_big_if_true.jpeg?0x300}}
+      * A tool for viewing installed fonts on the Folk system.
+    * {{newsletters:andres_font_sizes_demo.gif?0x350}}
+      * A tool to preview fonts at different sizes
+    * {{newsletters:andres_folder_font_book.jpeg?0x300}}
 === The new shader language ===
-Folk now implements a new 'GPU FFI'((There's an obvious analogy to our 'C FFI': both systems have you embed arbitrary C/GLSL code in a Folk program and make it easy to call from Folk with native Tcl-object parameters.)) which allows you to compile 'pipelines', which consist of a vertex shader and a fragment shader, which tell the GPU how to draw something:
+Folk now implements a new 'GPU FFI'((There's an obvious analogy to our 'C FFI': both systems have you embed arbitrary foreign (C or GLSL) code in a Folk program and make it easy to call from Folk with native Tcl-object parameters.)) which allows you to compile 'pipelines', which consist of a vertex shader and a fragment shader, which tell the GPU how to draw something.
-TODO
+There aren't great helpers for this yet, like shading a region or anything like that, so you need to work in projector coordinates by hand for now. Let me know what you think would be useful to have.
-=== TODO: Side quests ===
+Here is a very basic shader that you can just paste as a virtual program (or print). It will color a triangle underneath itself green-blue.
-Versioned heap
+<code>
+Wish the GPU compiles pipeline "$this's triangle" {
+    {vec2 p0 vec2 p1 vec2 p2} {
+        vec2 vertices[4] = vec2[4](p0, p1, p2, p0);
+        return vertices[gl_VertexIndex];
+    } {
+        return vec4(0, 0.1, 0.1, 1);
+    }
+}
-New subprocess model so you can send code to subprocesses later (so display can be managed from multiple programs)
+Wish $this is outlined blue
+When $this has region /r/ {
+    set r' [region move $r down 100%]
+    Wish the GPU draws pipeline "$this's triangle" with arguments [list {*}[lrange [region vertices ${r'}] 0 2]]
+}
+</code>
+Notice how you compile the pipeline once (when the program is put down), and then it gets executed repeatedly (whenever the program's region changes).
+The pipeline object that you pass after ''Wish the GPU compiles pipeline NAME'' has the form ''{ALL-ARGS VERT-SHADER [FRAG-ARGS] FRAG-SHADER}''. See [[guides/gpu|the GPU guide]] for more information about these pipeline parameters.
+----
+There's a nice layering and consistency and comprehensibility to how graphics works now, all the way down. See the ladder:
+  - Object-centric: ''Wish $this draws a circle with radius 3 ...''
+  - Natural-language wish: ''Wish to draw a circle with radius 3 center {100 200} ...''
+  - Vulkan pipeline wish: ''Wish the GPU draws pipeline "circle" with arguments {3 {100 200} ...}''
+  - Tcl imperative: ''Gpu::draw $circlePipeline 3 {100 200} ...''
+  - C FFI: ''Gpu::drawImpl $circlePipeline [encodeArgs $circlePipelineArgs]''
+  - Vulkan/GPU driver: ''vkCmdDraw(...)''
+=== Side quests on the way to Vulkan display ===
+----
+Had to rewrite calibrate to use shaders instead of plotting pixels. Made its print output slightly friendlier
+----
+The Folk interprocess heap has been extended to allow freeing of heap allocations (it's now based on [[http://gee.cs.oswego.edu/dl/html/malloc.html|dlmalloc]] instead of just being a bump allocator) and to tag each heap slot with a random 64-bit 'version', as a hacky way to evict stale images that happen to reuse the same heap address (so we know to recopy them to the GPU)
+This implementation is pretty inefficient and unsafe (it walks a capped-256 array of all allocations and has a single machine-wide lock) and we may want to replace it with an interval tree or something at some point & introduce finer synchronization
+----
+C FFI improvements: struct getter functions, some errors actually abort and throw to Tcl (using longjmp)
+----
+''dict_getdef'' has been added
+----
+Subprocess code deployment has changed. You boot a subprocess now with ''Start process NAME'', like:
+<code>
+Start process cool {
+  Wish $::thisProcess shares statements like [list /someone/ wishes /z/ is labelled /x/]
+  Wish $this is labelled "Hello!"
+}
+</code>
+You can send more code to an already-existing subprocess with ''On process NAME'':
+<code>
+On process cool {
+  Wish $this is labelled "Wow"
+}
+</code>
+This is used to deploy code from individual programs to the GPU process, so all GPU process lgoic doesn't have to be in one monolithic file anymore.
+There are some catches / ugly bits: process names are global & you now have to explicitly opt into all sharing (either share or receive); there's no sharing by default anymore.
+----
 === A note on the graphics rewrite ===
@@ Line 167: / Line 260: @@
 ===== What we'll be up to in November =====
-  * TODO: We're having our next open house (at our studio in East Williamsburg, Brooklyn) on the evening of Wednesday, November 29. Come by!
+  * We're having [[https://partiful.com/e/3RtkNpS4KTFUj2tZNXbX|our next open house on the evening of Wednesday, November 29]], at our studio in East Williamsburg, Brooklyn. Come by!
-  * Editor work
+  * Tabletop editor
   * Still need to complete RFID refactor
-  * Continuing work on new calibration (maybe followed by super-accurate camera slicing and OCR)
+  * Continuing work on new calibration (followed by refactoring a lot of the system to use real-world 3D coordinates, maybe followed by super-accurate camera slicing and OCR)
+  * Want to do some follow-up work on Vulkan: user-facing shader language, resource disposal, increase the image limit maybe
 ===== Links we've enjoyed =====
@@ Line 189: / Line 283: @@
 ==== Andrés ====
+  * [[https://www.youtube.com/watch?v=cBYudbaqHAk&t=6704s%20|Cellpond - Spatial programming without escape (1:51:44 timecode)]] I've been thinking about this talk for hours this week. If you watch it let me know what you think: cwervo@folk.computer
+  * [[https://drfreckles42.itch.io/youre-the-os|You're the OS]] — //Do you sometimes get frustrated at how slow your computer's operating system can be? Now is your chance to prove that you could do a better job!//
+  * [[https://www.baeldung.com/linux/bluetooth-via-terminal|Connect to a Bluetooth Device via the Terminal]]
+  * [[https://www.youtube.com/watch?v=NEaUSP4YerM| Clearly Explained]]
+==== Arcade ====
+  * [[https://zine-machine.glitch.me/|Zine Machine: CSS that makes your website a printable zine!]]
+  * [[https://creativecommons.org/2004/04/05/woodyguthriefreeculture/|Woodie Guthrie on Free Culture]] one of my favorite folk musicians :>
+  * [[https://www.robinsloan.com/notes/home-cooked-app/|An app can be a home-cooked meal]]