User Tools

Site Tools


newsletters:2025-02

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
newsletters:2025-02 [2025/02/28 21:20] – [New Folk evaluator] osnrnewsletters:2025-02 [2025/03/31 12:18] (current) – [What we'll be up to in February] osnr
Line 42: Line 42:
  
 === Snapshots / state-saving / program-saving === === Snapshots / state-saving / program-saving ===
 +
 +>> Is there a way to Hold something on disk so a page has permanent information?
 +>> Or is snapshotting the goal?
 +>
 +> we don't have this right now but i think it would be great to have
 +>
 +> 1. subsume awkward hacks where we use files in ad-hoc ways for persistence (.folk, .meta.folk, calibration)
 +>
 +> 2. crash resilience (your editor state can get preserved across crashes, etc)
 +>
 +> i think of snapshotting as a way to preserve/version programs and persistent Hold as a way to preserve data i guess, i don't know if that distinction makes sense but that's how it is in my head
 +
 +(snapshotting being "where you have virtual thumbnails on the printed page to represent all the programs, so they're still pointable & have regions etc… 
 +the thumbnail regions could all be captured from table session when the snapshot is taken (& shrunk down to fit on page), or i guess they could just be a grid if you have a statement that is just a straight list of program numbers like you propose?")
 +
 +>> Could you have a separate process that journals the holds and writes to disk on crash or quit?
  
 === Intercept discussion === === Intercept discussion ===
Line 48: Line 64:
  
 {{:newsletters:pasted:20250228-193350.png?500px}} {{:newsletters:pasted:20250228-193350.png?500px}}
 +
 +(Omar: "side note: it would be cool if (maybe if we had statement blocking) you could implement smoothing and freezing without needing to modify the virtual program, like you could just impose your own mixin 'filters' to set smoothing policy for individual pages")
  
 Mason proposal to have Intercept block: Mason proposal to have Intercept block:
Line 138: Line 156:
  
   * Fixed [[https://github.com/FolkComputer/folk/commit/ba72061b844206bb07f5f29472b303f38fdf8fdc|horrible sign error bug]] that made ''keep ms'' not actually work -- this fixed most of the outline blinking!  (because we keep the tag detection for ~16ms, until the next frame's detection comes in, and it actually keeps now)   * Fixed [[https://github.com/FolkComputer/folk/commit/ba72061b844206bb07f5f29472b303f38fdf8fdc|horrible sign error bug]] that made ''keep ms'' not actually work -- this fixed most of the outline blinking!  (because we keep the tag detection for ~16ms, until the next frame's detection comes in, and it actually keeps now)
-  * [[https://github.com/FolkComputer/folk/commit/d9f0f154639c92f883a1bfaa003daf17e83270e2|New scheduling]] that uses [[https://github.com/FolkComputer/folk/commit/2f91d34d87a5a7b506dded892e90c36c50a52bdb|I/O-boundness]] (by checking /proc for each thread) +  * [[https://github.com/FolkComputer/folk/commit/d9f0f154639c92f883a1bfaa003daf17e83270e2|New scheduling]] that uses [[https://github.com/FolkComputer/folk/commit/2f91d34d87a5a7b506dded892e90c36c50a52bdb|I/O-boundness]] (by checking /proc for each thread) to figure out if new threads need to be awakened to stay responsive. Maintain a pool (10-20) of threads that are mostly asleep so we don't have to literally spawn new threads (slow) for this. 
-  * Inlined a lot of operations (Claim, Wish, When) so you don't have to trampoline to the workqueue (slow) to do them. Can do this because we don't lock the db anymore for those so is ok to just do them right away. The workqueue is pretty much just for queueing Tcl evals now+    * Seems to work well in practice! Utilization of CPUs in Tracy looks pretty good 
 +  * Inlined a lot of operations (Claim, Wish, When) so we don't have to trampoline to the workqueue (slow) to do them. Can do this because we don't lock the db anymore for those so is ok to just do them right away. The workqueue is pretty much just for queueing Tcl evals now
   * Per-thread object caching    * Per-thread object caching 
     * Lots of little improvements to object reuse (don't set source info because that stomps the whole object representation)     * Lots of little improvements to object reuse (don't set source info because that stomps the whole object representation)
Line 147: Line 166:
   * Overall fixed keyboard so you can actually edit stuff; reap keyboard events off-thread   * Overall fixed keyboard so you can actually edit stuff; reap keyboard events off-thread
   * Reap AprilTags when flipped over (fixes issue Rob has below)   * Reap AprilTags when flipped over (fixes issue Rob has below)
-  * Fixed stdout printingthis has been a minor annoyance for ages+  * Fixed stdout printing -- this has been a minor annoyance in folk2 for ages
   * Fixed repeated reloading of C modules which was a big performance drag   * Fixed repeated reloading of C modules which was a big performance drag
   * ./folk PROGRAM.folk now runs the PROGRAM in a When context, instead of a bare context, so you can do Wish/Claim/When normally inside your program. Great for writing quick tests. It also just feels nice to have Folk able to act as a 'normal' self-contained programming language runtime that can run scripts if needed   * ./folk PROGRAM.folk now runs the PROGRAM in a When context, instead of a bare context, so you can do Wish/Claim/When normally inside your program. Great for writing quick tests. It also just feels nice to have Folk able to act as a 'normal' self-contained programming language runtime that can run scripts if needed
-  * Failed project: the 'environment store'+  * Failed project (for now): the 'environment store' which was meant to reduce the amount of captured data from lexical scope around functions/When bodies that we're shipping around, dedupe scopes that are actually the same by giving them pointer IDs and keeping environments in a global multithreaded store 
 +    * Burned a lot of time just getting the reference counting of these envs to be correct (given destructors have envs, functions may outlive their original When, etc) 
 +    * Broke too much stuff: it breaks a lot of When deduplication because those pointers are always unique, so two Whens with identical captured variables no longer merge 
 +    * Also would create awkward JS-like bugs if you call Hold! in a loop (the environment ID & code is always the same, so the Hold! doesn't run more than once)
  
 Most important, I've reconciled folk2 so it now lives in a real branch on the Folk repo (and not a separate repo that can't actually merged), can be tested by other people and ultimately PRed and merged. Deleted a lot of extra files from folk1 while doing that reconciliation, so the repo looks pretty clean. Most important, I've reconciled folk2 so it now lives in a real branch on the Folk repo (and not a separate repo that can't actually merged), can be tested by other people and ultimately PRed and merged. Deleted a lot of extra files from folk1 while doing that reconciliation, so the repo looks pretty clean.
  
 Anyway, immediate next steps are to fix editor blinking while you type, then make a draft pull request. Anyway, immediate next steps are to fix editor blinking while you type, then make a draft pull request.
 +
 +=== Editor blinking ===
 +
 +
 +Notice the blinking of the editor -- this is extreme (other slow stuff is happening on the table), but it happens occasionally even while just normally typing/backspacing:
 +
 +{{newsletters:img_7894.mp4?300px}}
 +
 +Tracy view when I type a bunch of stuff (see the blips on the keyboard thread for when I'm pressing keys) and the editor blinks out (see the dips on the plot at the bottom):
 +
 +{{newsletters:263550d2-1c28-4792-9e18-60b3d280c982-6265-0000034beb6ac656.png?450px}}
 +
 +I did optimize the editor a bit by [[https://github.com/FolkComputer/folk/commit/a88d925f541efde46169e55ad9828620232d5f5f|decoupling cursor state from rendering]], so it only blinks when you actually edit the code, not when you're moving the cursor. (It's cool how just by separating your logic into different independent Whens, your program feels like it gets more and more parallel and reliable...)
  
 === Rob's test === === Rob's test ===
Line 166: Line 201:
 === Shared immutable (immortal) objects === === Shared immutable (immortal) objects ===
  
-Omar: Mason and I spent a while throwing around ideas in Discord about how to share objects between threads (separate Jim interpreters) in folk2 without copying/reparsing.+Omar: Mason and I spent a while throwing around ideas in Discord about how to share objects between threads (separate Jim interpreters) in folk2 without copying/reparsing (which is what both folk1 and folk2 do now, although it's a more acute problem in folk2 because stuff comes fresh out of the DB for every When execution)
  
 ('immortal' is [[https://peps.python.org/pep-0683/|a term from Python]] for what I think is a similar concept) ('immortal' is [[https://peps.python.org/pep-0683/|a term from Python]] for what I think is a similar concept)
  
-I think we have an OK plan for this, actually, and I [[https://github.com/FolkComputer/folk/tree/osnr/folk2-transients|started hacking on it]], but haven't really fully implemented it yet. Something to keep in pocket if we need it, maybe.+I think we have an OK plan for this, actually, and I [[https://github.com/FolkComputer/folk/tree/osnr/folk2-transients|started hacking on it]], but haven't really fully implemented it yet. Something to keep in pocket if we need it, maybe. (for now, the aggressive per-thread object cache mitigates the copying and reparsing a lot)
 ==== Friends and outreach ==== ==== Friends and outreach ====
  
Line 233: Line 268:
       * Folk plugin for Procession that tells the LLM how to input/output stuff       * Folk plugin for Procession that tells the LLM how to input/output stuff
  
-===== What we'll be up to in February =====+===== What we'll be up to in March =====
  
   * **Our next Folk open house is [[https://partiful.com/e/8zlZKFFAvyeFbngr9PUe|in the evening on Wednesday, March 26]], at our studio in East Williamsburg, Brooklyn.**   * **Our next Folk open house is [[https://partiful.com/e/8zlZKFFAvyeFbngr9PUe|in the evening on Wednesday, March 26]], at our studio in East Williamsburg, Brooklyn.**
Line 259: Line 294:
  
 ==== Andrés ==== ==== Andrés ====
-  * +  * [[https://www.instagram.com/reel/DFdi2U9tgEA/|Liquid paper sculptures]] 
 +  * [[https://endtimes.dev/why-you-should-dither-images/|Why Your Website Should Use Dithered Images]] 
 +  * [[https://www.greendesign.io/dithering-images|Dither images]] 
 +  * [[https://x.com/kevintwohy/status/1867964091769430262?s=12&t=AUhNsAKs8G7beopf-_iD3Q|Projected wayfinding in Newark Airport]]
newsletters/2025-02.1740777631.txt.gz · Last modified: 2025/02/28 21:20 by osnr

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki