Table of Contents

January 2026 newsletter

Our next Folk open house will be on the evening on Thursday, February 26th, in East Williamsburg, Brooklyn. RSVP here:

RSVP for the Folk open house on Feb 26

Last month, we officially formed a nonprofit for Folk. We’re trying to get more serious about fundraising — we'd love for you to sponsor us so we can keep doing this work:

Sponsor Folk on GitHub Sponsors

What we've been up to

OCR experiments

Omar: I've been wanting to do non-AprilTag inputs for a while, especially handwriting input (see proposal from last year, for example).

I started looking at OCR/handwriting recognition more seriously a couple of months ago, and I concluded that TrOCR was probably the right building block (AI model) for this. It runs pretty fast (0.1s to 1s) on most systems on GPU and CPU, I was able to run it locally, it's trained for handwriting and not print. I did some prototyping in Python for that.

Now that we have folk2, it's a lot easier to run synchronous tasks like handwriting recognition without blocking the rest of Folk, so I want to finally get stuff like this deeply integrated (and have the integration be pretty simple and idiomatic) and prototype how to interact with it.

So at the end of January, I finally started hacking on a TrOCR integration that runs inside Folk, so we can actually write on the table and get live text output.

img_1463.jpeg

Here's a draft trocr.folk that exports a callable function (more verbose / AI-y than it probably should be, but it's fully self-contained, which is great).

Easy invocation on a static on-disk image (to use live, put it in an infinite loop and query for camera slice): 

When the image library is / imageLib/ &\
     the TrOCR function is /TrOCR/ {
  fn TrOCR
  set im [$imageLib loadJpeg "a01-122-02.jpg"]
  puts [TrOCR $im]
}

It runs really fast on folk-hex when CUDA is on (maybe 10x faster, multiple frames per second). That liveness is fun to play with.

Excited to show more demos around this next month.

A lot of this is built on uvx, which I just learned about this month. uvx gives us the ability to run Python with arbitrary dependencies in 'immediate mode', which aligns a lot better with Folk (wanting self-contained declarative Folk programs, wanting to be able to change them to do anything at any time, etc) than a traditional project/env-oriented Python workflow would. You just run uvx with all dependencies as arguments and then can eval any Python you want, and it automatically caches the dependencies so it doesn't have to reinstall from scratch every time.

Some issues to resolve:

Text detection with CRAFT

I started experimenting with the CRAFT text detector, which seems to be what people generally pair with TrOCR.

I tuned it to use the GPU and to be more aggressive in linking regions, so we capture the whole “wow” in one box here:

Here's the draft code that does both CRAFT and TrOCR, which I haven't tested on a table (I'm now working on a more general Python FFI so that the OCR pipeline isn't one big monolith, and so we can try other models easily).

Stuff from China

Omar: I was in China for a few weeks (for some workshops and conferences in Shanghai and Shenzhen). I prioritized getting stuff I couldn't find in the US (often these are not even on Aliexpress, only on Taobao or in person). Got a few interesting Folk-related gadgets:

In general, my hope was that we could find competitively-priced devices in these categories (that maybe are only available in China), but was only kind of successful. Let us know if you know of anything:

Briefly showed off the new pink gadget1 I made last month:

619449636_18568814476029757_6219918909947160428_n.jpg

Also started looking into PCB design for a side project. As a result, I'm thinking very seriously about making a smaller gadget that fits in a barcode scanner-like chassis (something like the Deli one above). Would need a custom PCB and heat dissipation strategy, but we could just use the bare Ultimems projector board (no heat sink), and we could use just one 18650 battery. That would go a long way in reducing weight and volume.

Outreach

Open house

What we'll be up to in February

Omar

Andrés