| Both sides previous revisionPrevious revision | |
| newsletters:2025-11 [2025/12/08 19:20] – [AtomicallyVersion leak (not fixed yet)] osnr | newsletters:2025-11 [2025/12/08 20:37] (current) – [JPEG frames and Term struct] osnr |
|---|
| === JPEG frames and Term struct === | === JPEG frames and Term struct === |
| |
| I was thinking about how to speed up the /camera-frame view when we're streaming it in setup and calibrate, and I realized that we're decompressing every webcam frame (for AprilTag detect) (since webcams send you frames as a JPEG stream), then recompressing every frame as a JPEG to send to the web client. Why not put the original JPEG from the webcam in a statement, before we decompress, and just send that to the web client? | I was thinking about how to speed up the /camera-frame view when we're streaming it in setup and calibrate, and I realized that we're decompressing every webcam frame (for AprilTag detect) (since webcams send you frames as a JPEG stream), then recompressing every frame as a JPEG, writing to disk, then reading back from disk to send to the web client. Why not put the original JPEG from the webcam in a statement, before we decompress, and just send that buffer to the web client? |
| |
| That sent me on a journey to try to embed the raw JPEG data in terms/statement clauses/the trie, which led me to [[https://github.com/FolkComputer/folk/compare/90d0a582...3cb882e0|replace the null-terminated C strings with a new length-prefixed Term struct]]. Now you can (in theory) have null characters in terms, so you can put binary data in terms directly. That's a change that Mason wanted to try a few months ago, since it seems both safer and potentially faster. | That sent me on a journey to try to embed the raw JPEG data in terms/statement clauses/the trie, which led me to [[https://github.com/FolkComputer/folk/compare/90d0a582...3cb882e0|replace the null-terminated C strings with a new length-prefixed Term struct]]. Now you can (in theory) have null characters in terms, so you can put binary data in terms directly. That's a change that Mason wanted to try a few months ago, since it seems both safer and potentially faster. |
| |
| I didn't end up using this for JPEG transmission -- instead making an Image-like struct that points at the JPEG data on the heap -- because I was paranoid about term copying in and out of interpreters, and the indirect struct lets us only copy when we actually want to read the JPEG data (on output to web client and on decompress). | I didn't end up using this for JPEG transmission -- instead making an Image-like struct that points at the JPEG data on the heap -- because I was paranoid about term copying in and out of interpreters, and the indirect struct lets us only copy when we actually want to read the JPEG data (on output to web client and on decompress). |
| | |
| | Here's what /camera-frame looks like after this change (no more recompress, no more write to disk): |
| | |
| | {{.:pasted:20251208-203638.png?600px}} |
| |
| Anyway, more on using the JPEG data next month -- it gives us color preview and faster livestream, which is great, and I'm hoping to use the color in more places soon. I [[https://github.com/FolkComputer/folk/commit/4f901e20cc56f198622f8825deaab8e6b9ebc5b0|switched to the TurboJPEG API]], which I think might let us [[https://graphicdesign.stackexchange.com/a/56130|crop the JPEG without decompressing it first]], which would be great for cropped cameras and stereo camera. | Anyway, more on using the JPEG data next month -- it gives us color preview and faster livestream, which is great, and I'm hoping to use the color in more places soon. I [[https://github.com/FolkComputer/folk/commit/4f901e20cc56f198622f8825deaab8e6b9ebc5b0|switched to the TurboJPEG API]], which I think might let us [[https://graphicdesign.stackexchange.com/a/56130|crop the JPEG without decompressing it first]], which would be great for cropped cameras and stereo camera. |
