This is an old revision of the document!
Table of Contents
January 2025 newsletter
What we've been up to
General Folk system improvements
- Andrés and Mason Jones improved calibration instructions: clarified wording to be more explicit about lifting the board above the table and added an example video of calibration; a lot of people have been really confused by calibration and have reported that it doesn't work (when actually they aren't calibrating it right; usually they're only training on the table plane so it's degenerate), so we've been meaning to clear it up
- Omar added crop support to camera driver so you can focus on a subregion of the camera image (see gadget section below)
Updating the dot detector
Andrés: I made a wiki page for Naveen Michaud-Agrawal's circle/dot detector that integrates OpenCV's circle detection with Folk. Unfortunately when I tried this on the most recent version of Folk I noticed it needed to be updated to translate coordinates to our 3D coordinate system. Omar helped me figure out the linear algebra to do this translation and I have a branch that's starting to work. I should have an updated example the first week of February and am looking forward to developing demos that use dots as interface elements (connect-the-dots & count dots to generate numbers come to mind).
Handwriting recognition proposal
Omar: I've finally starting to look at putting in blanks, recognition regions, etc inline w/ code on printed programs. I think it would be cool to provide an API like this:
where you get "X" as a string as the return value of Blank2. This would be great instead of (or in addition to) a more explicit API where you have to like Wish $this prints rectangle with width 4cm height 2cm x 0cm y 10cm and When query for the camera slice of the field and run OCR on that etc etc. You'd be able to use handwriting input out of the box, as easily as you read from the terminal in basic programming on a laptop.
You could imagine more easy primitives like this built into Folk, like multiple-choice bubbles that the user fills in, or a slider, or whatever.
I also like how this starts to really use the fact that the code is printed on the program. Now the printed code is not just for documentation; you can start to modify the program by physically modifying the code (which is also nice since it's a different interaction – using pen or pencil – as opposed to the usual coarse moving pieces of paper around).
Smoothing
New parallel evaluator
Omar: Progress on this has been limited this month.
A lot of looking at these specific spans in Tracy where there was a blink-out of a tag outline:
You can see the drawcount plot at the bottom – when that dips from 4 to 0, you know you have an unwanted blink. Tried adding more messages so we can see the actual statement.
Red span
I cleaned up the detection of past versions (it makes an explicit folk.pid file) and have been rethinking our thread spawning/killing policies. I think a lot of the blinking is just because we have too many threads for the number of CPUs and they're preempting each other mid-task, which means that the evaluation doesn't converge in time for the frame.
Portable Folk gadget
All the components are in. This revision uses the Orange Pi 5.
Network setup
Annoyingly, the Orange Pi 5 doesn't have built-in Wi-Fi, so I'm using an Edimax EW-7611ULB USB dongle (RTL8723BU chipset) I had lying around. (Another problem is that we have really limited USB ports on the OPi 5 – it only has two, we're already using one for the camera, so with the Wi-Fi dongle, there are no free ones left for keyboard or anything.)
- Set up Netplan, struggled to get it on my network, realized that it only supports 2.4GHz Wi-Fi.
- Still struggling to make it do ad-hoc or access point mode by default
- (ideally it would work in concurrent mode, so could both get on known existing networks and make its own for wild environments:)
The stereo camera
The stereo camera works! Its field of view is huge, though:
Added crop support to just take the center part of the left image for now so we don't waste time processing the rest and can start trying to calibrate right away.
Current state & next steps
really heavy
button works
battery works
sound not working yet
need to figure out how to calibrate.
need to do some physical revision
RFID localization
Omar: Good progress here. My networking code (writefull and readfull) was broken, which was causing failures after a few seconds, and it seems reliable now.
Hit rate looks pretty good when live-streaming data from the IB radio – see how almost all hops are green:
(Rounds are made of hops; each green hop is a valid roundtrip to the RFID tag where the result passed checksum, so they are guaranteed good data for localization + they tell you the tag ID. Right now, a round takes about 250ms and a hop takes about 15ms, so we're not quite at real-time interaction rate yet, but hopefully we can get there.)
Next step is to set up the OOB radio – can just have it read at 915MHz to start (and that'll help make sure the right stuff is happening on IB & that we can sync).
Friends and outreach
- Small open house at the end of January:
What we'll be up to in February
- Our next Folk open house is in the evening on Wednesday, February ??, at our studio in East Williamsburg, Brooklyn.
- We'll be speaking at the School of Visual Arts' Interaction Design department’s 2025 winter/spring lecture series. The talk will be recorded so we'll link it here as soon as it's online.
- We'll also be showing Folk as an interactive prototype in a workshop in the Design in Action course at Columbia GSAPP. The students' past work is very Folk-adjacent so we expect to get a lot of interesting feedback and are excited to show an enthusiastic technical audience Folk up-close.