| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| newsletters:2024-04 [2024/05/01 21:32] – osnr | newsletters:2024-04 [2024/05/02 02:07] (current) – Add NYU specifics admin |
|---|
| Omar: as part of getting the CNC project (next section) polished and distributed for a wider audience, I've been making a [[https://github.com/FolkComputer/folk-live-build|self-contained Folk live USB for amd64 PCs. It's mostly ready! You can download it now!]] | Omar: as part of getting the CNC project (next section) polished and distributed for a wider audience, I've been making a [[https://github.com/FolkComputer/folk-live-build|self-contained Folk live USB for amd64 PCs. It's mostly ready! You can download it now!]] |
| |
| It just uses display 0 and webcam 0, so the live USB stick (plugged in on the right) can run straight on a laptop: | It just uses whatever is display 0 and whatever is webcam 0, so the live USB stick (plugged in on the right) runs straight on a cheap laptop: |
| |
| {{:newsletters:pasted:20240501-080545.png?300px}} | {{:newsletters:pasted:20240501-080545.png?300px}} |
| (You can think of this as a replacement for most of the traditional [[https://git.folk.computer/folk/about/|normal manual Folk setup process]] where you install Linux, install a bunch of dependencies, clone Folk, etc.) | (You can think of this as a replacement for most of the traditional [[https://git.folk.computer/folk/about/|normal manual Folk setup process]] where you install Linux, install a bunch of dependencies, clone Folk, etc.) |
| |
| It's in pretty good shape now -- you can flash the image to a USB stick, configure/update the Folk system from a computer (there's a writable FAT32 partition that contains all the Folk stuff that isn't the core Linux OS, so it's easy to update both Folk repo and any custom programs you want to preinstall), plug it into a PC, and the PC should boot right to a working network-connected Folk as long as you set up Wi-Fi and have /dev/video0 and display 0. | It's in pretty good shape now: you download the .img file, flash it to a USB stick, and configure/update the Folk system on the USB stick from your normal computer. There's a writable FAT32 partition that contains all the Folk stuff that isn't the core Linux OS, so it's easy to update both the Folk repo and any custom programs you want to preinstall: |
| | |
| | {{:newsletters:pasted:20240502-011852.png?600px}} |
| | |
| | Then you plug the USB stick into the PC that you want to run Folk, and the PC should boot right to a working network-connected Folk as long as you configured Wi-Fi and have /dev/video0 and display 0. |
| |
| Some problems I encountered while building the live USB: | Some problems I encountered while building the live USB: |
| * although the writable partition has ended up being pretty cool and useful and inspiring | * although the writable partition has ended up being pretty cool and useful and inspiring |
| * way easier to update the live USB / live image and iterate on it since the OS doesn't need to be reflashed | * way easier to update the live USB / live image and iterate on it since the OS doesn't need to be reflashed |
| * makes Folk feel much more stable and appliance-like; I can just boot an extra Chromebook into Folk to test stuff and know exactly what I'm depending on, not worry about messing up the PC, know that if I reboot it'll come up without needing to be touched | * makes Folk feel much more stable and appliance-like; I can just boot an extra Chromebook into Folk to test something and know exactly what I'm depending on, not worry about messing up the PC, know that if I reboot it'll come up without needing to be touched |
| * feels kind of like microPython or something in how you can program it like a USB drive | * feels kind of like microPython or something in how you can program it like a USB drive |
| * Memory leak freezes/crashes Folk on Chromebook for some reason | * Memory leak freezes/crashes Folk on Chromebook for some reason |
| Remaining to do on live USB (although it's already pretty useful): | Remaining to do on live USB (although it's already pretty useful): |
| |
| * Support configuring which display & camera to use at minimum, ideally support multiple | * Support configuring which display & camera to use at minimum, ideally support multiple (this is important for getting laptops to be useful Folk systems, since they already have a built-in webcam and display that you wouldn't want to lock onto) |
| * Build out the setup process (calibration, printer setup, maybe interactive Wi-Fi setup) (although there will probably always be an [[https://social.omar.website/@omar/statuses/01HW6GAPSTB294WG193GG5WDCM|escape hatch to Linux terminal]], we can't cover all problems with custom UI); get this stuff persisted on the writable partition across reboots | * Build out the setup process (calibration, printer setup, maybe interactive Wi-Fi setup) (although there will probably always be an [[https://social.omar.website/@omar/statuses/01HW6GAPSTB294WG193GG5WDCM|escape hatch to Linux terminal]], we can't cover all problems with custom UI); get this setup persisted on the writable partition across reboots |
| * Build/test the process of permanent installation to hard drive -- we've just been using it in live mode so far | * Build/test the process of permanent installation to hard drive -- we've just been using it in live mode so far |
| * Debug memory leak | * Debug memory leak |
| ==== folk-cnc ==== | ==== folk-cnc ==== |
| |
| Omar: I've been porting Owen's [[https://github.com/jmpinit/projected-toolpath-preview|projected CNC toolpath preview tool]] from last fall to Folk, so you should be able to flash Folk to a live USB, clone folk-cnc onto the live USB, and boot a PC (that's connected to a webcam and projector) right into the CNC preview system. | Omar: I've been porting [[https://owentrueblood.com|Owen's]] [[https://github.com/jmpinit/projected-toolpath-preview|projected CNC toolpath preview tool]] from last fall to Folk, so you should be able to flash Folk to a live USB, clone folk-cnc onto the live USB, and boot a PC (that's connected to a webcam and projector) right into Folk with the CNC preview application pre-loaded. |
| |
| Then (at least for now) you use a Web UI (much like the one in Owen's system) to calibrate the preview to your projector & camera & CNC machine and the material that you want to cut, and you upload a gcode file to preview. Here's the Web UI: | Then, at least for now, you use a Web frontend much like the one in Owen's system to set up the preview: you calibrate the preview to your projector & camera & CNC machine and the material that you want to cut, and you upload a gcode file to preview. Here's the Web UI: |
| |
| {{:newsletters:pasted:20240501-190329.png?500px}} | {{:newsletters:pasted:20240501-190329.png?500px}} |
| ([[https://poormansprofiler.org|"poor man's profiler"]] energy) | ([[https://poormansprofiler.org|"poor man's profiler"]] energy) |
| |
| You can see from the ''Run when'' descriptions above that thread 1 (3394) is running the camera loop, thread 2 (3401) is running the Web loop, thread 4 (3403) is running the GPU loop. | You can see from the ''Run when'' descriptions above that thread 1 (3394) is running the camera loop, thread 2 (3401) is running the Web loop, thread 4 (3403) is running the GPU loop. (note that these are not //static// assignments, though! I never assigned those processes to threads by hand, I just wrote the straight-line Folk programs that each blocked on different stuff, and the folk2 scheduler assigned them to threads in the thread pool at runtime.) |
| |
| You can see by squinting at the user stack traces below that thread 1 (3394) is running the camera loop, thread 10 (3476) is running an AprilTag detector worker thread, and thread 11 (3497) is running some Vulkan-related thread: | You can see by squinting at the user stack traces below that thread 1 (3394) is running the camera loop, thread 10 (3476) is running an AprilTag detector worker thread, and thread 11 (3497) is running some Vulkan-related thread: |
| I wanted to count inter-frame delays on various subprocesses: how long does it take between one camera frame and the next? between one AprilTag detection and the next? one GPU draw and the next? How many of each of these are we doing a second? is it what I'd expect (30Hz or 60Hz)? | I wanted to count inter-frame delays on various subprocesses: how long does it take between one camera frame and the next? between one AprilTag detection and the next? one GPU draw and the next? How many of each of these are we doing a second? is it what I'd expect (30Hz or 60Hz)? |
| |
| perf should provide a whole suite to count and plot and correlate stuff once we give it simple tracepoint hooks from our program, and it saves me needing to make custom thread-safe data structures and functions in Folk to maintain event counts and so on. | perf should provide a whole suite to count and plot and correlate things once we give it simple tracepoint hooks from our program, and it saves me needing to make custom thread-safe data structures and functions in Folk to maintain event counts and so on. |
| |
| Here's how you provide an event to perf in your Folk code: | Here's how you provide an event to perf in your Folk code: |
| (although it took a bit [[https://social.omar.website/@omar/statuses/01HWBB79447C991Q130YC7FERP|to even get perf to work]] on what should be a bog-standard amd64 Debian system... it [[https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2019247|wouldn't detect libtraceevent]] to record user events, and I had to recompile it myself) | (although it took a bit [[https://social.omar.website/@omar/statuses/01HWBB79447C991Q130YC7FERP|to even get perf to work]] on what should be a bog-standard amd64 Debian system... it [[https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2019247|wouldn't detect libtraceevent]] to record user events, and I had to recompile it myself) |
| |
| Counting perf events immediately paid off, because I figured out that (part of) the reason that folk2 is slow is probably that it does AprilTag detection in an infinite ''while 1'' loop, 100+ times a second, rather than just when the camera frame changes, 30/60x per second: | Counting perf events immediately paid off, because I figured out from the above that (part of) the reason that folk2 is slow is probably that it does AprilTag detection in an infinite ''while 1'' loop, 100+ times a second, rather than just when the camera frame changes, 30/60x per second. |
| |
| and the reason I made the detection process an infinite loop was because the AprilTag detector, like all C objects, was only callable from the thread it was created on. So I decided it was time to do a C refactor I'd been meaning to do for a while, to make the detector callable from any thread, so we can call it from a normal When body that responds to the camera frame (which can get scheduled onto any thread). | and the reason I made the detection process an infinite loop was because the AprilTag detector, like all C objects, was only callable from the thread it was created on. So I decided it was time to do a C refactor I'd been meaning to do for a while, to make the detector callable from any thread, so we can call it from a normal When body that responds to the camera frame (which can get scheduled onto any thread). |
| It feels uglier now that 1. the C library object needs to get stored in the Tcl object so that Tcl methods can call into the C library and 2. we need to hook the ''unknown'' method if we want the Tcl object to also let its users call C methods and 3. the Tcl object doesn't transparently get shared across threads, since it's a Jim lambda reference. | It feels uglier now that 1. the C library object needs to get stored in the Tcl object so that Tcl methods can call into the C library and 2. we need to hook the ''unknown'' method if we want the Tcl object to also let its users call C methods and 3. the Tcl object doesn't transparently get shared across threads, since it's a Jim lambda reference. |
| |
| At the very least, it would be good to make a construct on top of ''class'' that factors out some of the common stuff like the C library member, the ''compile'' call, and the ''unknown'' handler. | At the very least, it would be good to make a construct on top of ''class'' that factors out some of the common logic like the C library member, the ''compile'' call, and the ''unknown'' handler. |
| ==== Friends and outreach ==== | ==== Friends and outreach ==== |
| |
| * **Our next [[https://partiful.com/e/yPGAy7VccfYivP557hCn|Folk open house is in the afternoon of Sunday, May 26]] at our studio in East Williamsburg, Brooklyn.** | * **Our next [[https://partiful.com/e/yPGAy7VccfYivP557hCn|Folk open house is in the afternoon of Sunday, May 26]] at our studio in East Williamsburg, Brooklyn.** |
| * More side hacking on Folk gadget: calibration, hardware design | * More side hacking on Folk gadget: calibration, hardware design |
| | * Maybe revisit 3D calibration for this |
| * Finish up the CNC port, test it, and do a writeup/post (including end-to-end install instructions using live USB) | * Finish up the CNC port, test it, and do a writeup/post (including end-to-end install instructions using live USB) |
| * Continue to improve new parallel evaluator, especially performance, and port more functionality (text, images, Collect) | * Continue to improve new parallel evaluator, especially performance, and port more functionality (text, images, Collect) |
| * FIXME Andrés ITP show? | * Maybe revisit RFID |
| | * Andrés will be showing Folk, specifically OpenCV game experiments, in an NYU ITP Project Fellows art show at [[https://tisch.nyu.edu/clivedavisgallery|The Clive Davis Gallery]] from May 23rd to May 30th. |
| |
| ===== Links we've enjoyed ===== | ===== Links we've enjoyed ===== |
