From c425822b825e6882ff83ecc73c91644fd2cf9ffc Mon Sep 17 00:00:00 2001 From: ScottW514 Date: Mon, 17 Aug 2026 14:34:36 -0400 Subject: [PATCH] docs, acceptance: the cameras as users meet them, and 8 MP at full resolution docs/VIDEO.md is the user-facing camera guide: what the endpoints return, what the sensors can do that ForgeFIRM does not send and why, the privacy gate, and the state of 8 MP support. The 8 MP (OV8856) capture path now reaches the sensor's full 3264x2448 frame. Its stock RAW10 full-resolution mode runs the link at 1.44 Gbps/lane and the i.MX6 CSI-2 D-PHY stops at 1 Gbps; the BSP adds a RAW8 mode that carries the same frame at half the rate, so an HD machine is no longer limited to the binned quarter-pixel mode. kas/README.md carries the reasoning, the register deltas and the factory configuration to fall back to if the receiver will not lock at 720 Mbps/lane. Acceptance: camera.sensor-profile expects the new OV8856 geometry, the camera covers map names src/camhealth.* (the src/cam.* glob does not match it, so the lint would have gone quiet on an uncovered file at the next pin bump), and a new auto test camera.frame-health asserts a burst of captures with no frames the capture queue flagged errored - a real signal about the camera ribbon even though those frames never reach a client. --- README.md | 14 +- docs/BRINGUP.md | 50 +++- docs/VIDEO.md | 410 ++++++++++++++++++++++++++++ forgetest/forgetest/suite/camera.py | 244 ++++++++++++++++- kas/README.md | 51 +++- 5 files changed, 748 insertions(+), 21 deletions(-) create mode 100644 docs/VIDEO.md diff --git a/README.md b/README.md index 8465e24..c0c886a 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,7 @@ panel, and a standard Grbl interface. * [Connecting LightBurn](https://github.com/ScottW514/forgefirm/blob/master/docs/LIGHTBURN.md) * [How motion and the laser are driven](https://github.com/ScottW514/forgefirm/blob/master/docs/MOTION.md) * [How cooling and airflow work](https://github.com/ScottW514/forgefirm/blob/master/docs/COOLING.md) +* [The cameras and the video stream](https://github.com/ScottW514/forgefirm/blob/master/docs/VIDEO.md) * [How the laser safing works](https://github.com/ScottW514/forgefirm/blob/master/docs/SAFETY.md) * [How a release is accepted](https://github.com/ScottW514/forgefirm/blob/master/docs/ACCEPTANCE.md) * [Community Support](https://community.openglow.org) @@ -41,13 +42,20 @@ machine is idle:** camera homing cycle through the Glowforge service (a Glowforge account and a live service session are required for `$H`; everything else in GRBL mode runs without them), and the machine records where it is. +* **Cameras only capture with the lid closed.** The lid camera faces the room + once the lid is raised, so ForgeFIRM refuses every capture — live view, + snapshot, and anything the Glowforge service asks for in cloud mode — until + the enclosure is shut, and stops a running stream the moment the lid opens. + See [the cameras and the video stream](https://github.com/ScottW514/forgefirm/blob/master/docs/VIDEO.md). ## Hardware The control board is common to Glowforge Basic, Plus, and Pro. The 5 MP -(OV5648) camera modules are fully supported; the 8 MP (OV8856) modules found in -"HD" units bind but do not capture yet — see the camera note in -[kas/README.md](kas/README.md). +(OV5648) camera modules are fully supported and hardware-validated. The 8 MP +(OV8856) modules found in "HD" units have a complete capture path — the kernel +patches, device tree and sensor-aware capture profile they need are all in the +build — but it has never run on an 8 MP machine, so treat it as untested; see +the camera note in [kas/README.md](kas/README.md). ## Safety diff --git a/docs/BRINGUP.md b/docs/BRINGUP.md index 21a3625..8592a53 100644 --- a/docs/BRINGUP.md +++ b/docs/BRINGUP.md @@ -16,6 +16,7 @@ Read together with: | `forgectrl/docs/SERVICES.md` | the machine-services contract: switch map, hardware ownership, cooling channels, mode supervision, pulse-device ownership, logging | | `docs/SAFETY.md` | the hardware safing chain, decoded | | `docs/ACCEPTANCE.md` | the release acceptance contract | +| `docs/VIDEO.md` | the cameras as users meet them: endpoints, delivered geometry, and what the sensors can do that ForgeFIRM does not send | | `docs/LIGHTBURN.md`, `docs/UPDATE-SYSTEM.md`, `INSTALL.md`, `BUILD.md`, `kas/README.md` | sender setup, A/B update system, install, build | | `python3-gfhardware/forgefirm-app/docs/CLOUD.md` | cloud mode, including its own open items | @@ -375,13 +376,17 @@ One ulfius daemon serves it all: `/run/forgefirm/cooling.state` file. - `POST /diag/flow-verify|flow-calibrate|abort`, `GET /diag/status` — the diagnostics runner (below). -- `GET /cam/stream?cam=lid|head` — multipart MJPEG at 1296×972 (2×2 - Bayer-superpixel demosaic, JPEG q75; `FORGECTRL_STREAM_Q` overrides, - `FORGECTRL_STREAM_FPS` caps the frame rate, unset/0 = sensor max). +- `GET /cam/stream?cam=lid|head` — multipart MJPEG at half the sensor's frame + in each axis, 1296×972 on a 5 MP machine (2×2 Bayer-superpixel demosaic, + JPEG q75; `FORGECTRL_STREAM_Q` overrides, `FORGECTRL_STREAM_FPS` caps the + frame rate, unset/0 = sensor max). - `GET /cam/snapshot?cam=lid|head&res=full|half&q=1..100` — single JPEG, - default full 2592×1944 (own MIT bilinear demosaic). + default the sensor's full frame, 2592×1944 on a 5 MP machine (own MIT + bilinear demosaic). - `GET /cam/status` — JSON (running/cam/clients/frames/fps/fps_cap/encoder/ - buffers). + buffers/sensor, the stream + snapshot geometry the fitted sensor implies, + and the privacy gate's `capture_allowed` / `stopped_by_lid`). Stream and + snapshot answer 409 while the lid is open. - `GET /slots`, `POST /boot`, `POST /update/check|download|apply|upload`, `GET /update/status`, `POST /restore/factory`, `POST /system/reboot` — the A/B update manager (`docs/UPDATE-SYSTEM.md`). Upload is auth + idle + job @@ -401,7 +406,23 @@ red while unreferenced, normal once anchored. **Camera engine.** One worker owns the V4L2 node persistently (media-ctl / v4l2-ctl sequences identical to `gfhardware/cam.py`, factory exposure/gain/WB, software hflip in the demosaic); it starts on demand and tears down fully after -10 s idle so gfhardware one-shot grabs still work. The cameras share the +10 s idle so gfhardware one-shot grabs still work. **Privacy gate: neither +camera captures unless the lid is closed** — `machine_lid_closed()` (EV_SW +bit 3, fail-closed) is checked at every entry point and once per frame, so an +open lid refuses stream and snapshot with HTTP 409 and a lid opened mid-capture +tears the pipeline down; `gfhardware.cam.capture()` enforces the same rule for +the cloud client's direct-V4L2 fallback and raises `LidOpen`. No setting +disables it, and the factory's lid-open focus hunt now fails as a result +(`docs/VIDEO.md` §2, `forgectrl/docs/SERVICES.md`). Geometry, Bayer depth and +the manual control set come from a **sensor profile** chosen by whichever +driver bound on that camera's I2C bus, so one image serves both the 5 MP +OV5648 (2592×1944) and the 8 MP OV8856 (3264×2448) — both 8-bit BGGR, so the +capture word and the demosaic are the same and only the geometry changes; +`/cam/status` reports the model and the frame sizes that follow from it. +A frame the capture queue flags errored is dropped rather than demosaiced, +four in a row cycle the queue, and three cycles with no usable frame stop the +engine; `/cam/status` carries the running `health` counts (`src/camhealth.c`, +host test `camhealth_test`). The cameras share the hardware video-mux and the NEWEST request wins it: **streams preempt** (the current stream's clients end cleanly), **snapshots borrow** (pause, switch, grab one frame, switch back — a ~1–2 s freeze). The per-camera lamp @@ -818,10 +839,19 @@ Open items only. Anything closed is in `CAMPAIGN-LOG.md`. `gfcloud_home_x/y` against a jog to a known reference if the factory corner offset matters. 6. **Cameras.** Lens calibration / bed alignment (the fisheye needs LightBurn's - camera calibration pass); capture support for the 8 MP (OV8856) modules, - which bind but do not capture (the clock/DT detail is in `kas/README.md`); - the deferred emulator homing-image smoke, now that the emulator can be - pointed at live snapshots. + camera calibration pass); **first light on an 8 MP (OV8856) machine** — the + whole path is written but nothing has run on one, and only that hardware can + answer whether the 2-lane RAW8 full-resolution mode locks the D-PHY at + 720 Mbps/lane and what exposure/gain the sensor wants; the details, the + reachable-mode reasoning and the factory fallback configuration are in + `kas/README.md` §2. Also unapplied: the factory's **per-unit lens-shading + calibration**, an OmniVision LENC register file the factory pushes into the + sensor at every stream start (`load_cam_regs.sh` → a `regs` sysfs attribute + its driver adds; OV8858 `0x58xx` addresses remapped to the OV8856's + `0x59xx`). The files are per-machine data, not in the factory rootfs — look + for them under `/data` on a machine booted into the factory slot before + deciding whether to reimplement the mechanism. Finally the deferred emulator + homing-image smoke, now that the emulator can be pointed at live snapshots. 7. **Cloud mode.** The remaining gaps are tracked in `python3-gfhardware/forgefirm-app/docs/CLOUD.md` "Outstanding items": streaming-during-run (would lift the ring-size cap on job length), diff --git a/docs/VIDEO.md b/docs/VIDEO.md new file mode 100644 index 0000000..63c0a25 --- /dev/null +++ b/docs/VIDEO.md @@ -0,0 +1,410 @@ +# Video and the cameras + +The machine has two cameras — one in the lid looking down at the bed, one in the +print head looking at the material under the lens — and ForgeFIRM serves both as +plain **MJPEG over HTTP** from the web control panel. There is no app, no cloud +relay, and no proprietary protocol: a browser, LightBurn, or anything else that +can read an MJPEG stream or fetch a JPEG can use them. + +One rule governs all of it: **the cameras only capture with the lid closed** +(§2). Everything else here assumes that condition is met. + +This page explains what you get, how to point a client at it, what the sensors +are physically capable of versus what ForgeFIRM actually sends and why, and what +you can change. + +- For sender setup in general, see [Connecting LightBurn](LIGHTBURN.md). +- The cloud mode described in §8 is documented in + [Cloud mode](https://github.com/ScottW514/python3-gfhardware/blob/master/forgefirm-app/docs/CLOUD.md). + +--- + +## 1. What the hardware is + +The two cameras carry the same kind of sensor and feed one shared path into the +board. A **hardware MIPI switch** (the factory `CAM_SEL` line) selects which of +them reaches the board's single camera receiver, so **exactly one camera can be +capturing at any moment**. That is a property of the board, not a software +limit — there is no configuration in which both stream at once. + +| | Lid camera | Head camera | +|---|---|---| +| Sees | the whole bed, from above | the material directly under the lens | +| Its lamp | the lid LED strip | the white LED in the print head | +| Used for | bed view, camera-referenced homing, LightBurn's camera overlay | focus and material inspection (cloud mode's distance measurement) | + +The lid lens is a wide fisheye, which matters for how you use the image (§5.5). + +Which sensor is fitted depends on the machine. Standard machines carry a **5 MP +OV5648**; "HD" machines carry an **8 MP OV8856**. ForgeFIRM reads which one +bound and configures itself accordingly — one firmware image covers both — and +reports it in `/cam/status` and on the panel's Status tab. + +--- + +## 2. Privacy: the cameras only work with the lid closed + +**Neither camera captures anything while the lid is open.** Not the live view, +not a snapshot, and not an image requested by the Glowforge service in cloud +mode. Close the lid and everything works; open it and the sensors stop. + +The reason is where the lid camera points. It is mounted in the lid, so raising +the lid swings it up to face the room — and in cloud mode the shutter is not +yours to press: the service asks for images on its own schedule, whenever it is +connected. The rule removes the question. The enclosure being shut is the +condition for an image to exist at all. + +**What the rule covers** + +- **Both cameras.** The head camera is gated too, so this is one rule to + remember rather than a rule with an exception you have to trust. +- **Every way in:** the panel, `/cam/stream`, `/cam/snapshot`, the + mjpg-streamer aliases, LightBurn, and cloud mode's image actions. +- **Capture already running.** Opening the lid stops a live stream within + about a frame and shuts the sensor down; it does not merely block new + requests. +- **The lamps.** A refused capture never raises them, so an attempt with the + lid open leaves no trace. + +**How it behaves** + +| Situation | What happens | +|---|---| +| Snapshot requested with the lid open | `409` and a message naming the lid; no image data | +| Stream requested with the lid open | `409`; the stream never opens | +| Lid opened while a stream is running | the stream ends cleanly and the pipeline is torn down | +| Lid state unreadable | treated as open — capture refused | +| Cloud service asks for an image with the lid open | refused, and reported back to the service as a failed action rather than left hanging | +| Lid closed again | everything works immediately; nothing to restart | + +`GET /cam/status` reports it: **`capture_allowed`** is false whenever the lid is +open, and **`stopped_by_lid`** records that the last capture ended because the +lid opened rather than going idle. The panel's Status tab says *lid open — the +cameras are off* rather than showing a stream error. + +**Where the check comes from.** The lid signal is the same one the hardware +safety chain uses to gate the beam — the series combination of both lid +switches, not a software flag — and the check **fails closed**: if the lid +state cannot be read at all, the cameras stay dark. That direction is proven by +a unit test in CI; the end-to-end behavior is an acceptance test run on real +hardware. + +**One thing it costs.** The factory firmware ran the cloud's focus *hunt* with +the lid open, and part of a hunt is a head capture. Those captures are now +refused, so a hunt attempted with the lid open fails instead of completing. +Close the lid before letting the app focus or print. + +**What it is not.** This is a rule enforced by the two programs that own the +sensors, not a hardware cut-off: the sensor rails stay powered, and anyone with +root on the machine could bypass it. It protects you from the Glowforge +service, from other software on your network, and from a stream you forgot was +running — not from someone who already controls the board. There is +deliberately no setting to turn it off. + +--- + +## 3. Watching it + +**In the panel.** Open `http://:8080/` and go to the **Status** tab. +The *Lid camera* card shows a still by default with **Live** and **Refresh** +buttons; **Live** switches the same frame to the running stream. + +**From another program.** The endpoints are: + +| URL | What it returns | +|---|---| +| `/cam/stream?cam=lid` | continuous MJPEG (`multipart/x-mixed-replace`) | +| `/cam/stream?cam=head` | the same, from the head camera | +| `/cam/snapshot?cam=lid` | one full-resolution JPEG | +| `/cam/snapshot?cam=lid&res=half` | one half-resolution JPEG (much faster) | +| `/cam/status` | JSON: which sensor, which camera, frame rate, frame sizes, whether the lid currently permits capture | +| `/?action=stream` | the lid stream again, under the name mjpg-streamer clients expect | +| `/?action=snapshot` | one full-resolution lid JPEG, same aliasing | + +The `?action=` pair exists because a lot of software — print-server dashboards, +camera widgets, anything written against mjpg-streamer — assumes those exact +URLs. Point such a client at `http://:8080/` and it will work. + +Every one of them answers **`409`** with the lid open (§2), so a client that +checks status codes can tell "close the lid" apart from "the camera is broken". + +**Access.** Reading the camera needs no token. It does need a request that +addresses the machine by IP address (or `localhost`) and, from a browser, one +that is not cross-site — that is what stops a hostile page in another tab from +reaching into your machine. It is not protection against other people on your +LAN. Anything that *changes* machine state does need the panel's token. In +practice: paste the URL into any local client and it works. + +--- + +## 4. What you actually get + +| | 5 MP machine (OV5648) | 8 MP machine (OV8856) | +|---|---|---| +| Sensor frame captured | 2592 × 1944 | 3264 × 2448 | +| Live stream | 1296 × 972 | 1632 × 1224 | +| Full snapshot | 2592 × 1944 | 3264 × 2448 | +| Half snapshot | 1296 × 972 | 1632 × 1224 | +| Format | JPEG, quality 75 by default | same | +| Frame rate | **15 fps** sustained | not yet measured (§10) | + +Measured on a 5 MP machine: 15.0 fps with a viewer attached, which is the rate +the sensor itself produces in this mode — the machine is not the bottleneck. +The daemon uses about 41 % of one CPU with one viewer, and LightBurn can watch +the stream while jogging from the same session without disturbing motion. A +full-resolution still takes about 2.4 s to produce (2.7 s if the camera has to +be started first), because 5 megapixels of demosaicing and JPEG encoding happen +on the machine's CPU. + +The stream frame is not a resampled copy of the full frame. Each 2 × 2 group of +sensor pixels becomes exactly one output pixel, which is why the stream is +precisely half the capture in each axis and why it is cheap enough to run +continuously. + +--- + +## 5. What the sensor can do versus what ForgeFIRM sends + +This is where expectations usually come unstuck: the sensors are more capable +on paper than the video you get. Each difference below is deliberate and has a +reason. + +| | The sensor can | ForgeFIRM sends | Why | +|---|---|---|---| +| Live resolution | full frame | half in each axis | CPU and bandwidth; §5.1 | +| Frame rate (5 MP) | 30 fps in reduced modes | 15 fps | the full-field mode runs at 15 fps; §5.1 | +| Resolution (8 MP) | 3280 × 2464 | 3264 × 2448 | the widest frame the board's camera receiver can take; §5.2 | +| Bit depth | 10 bits per pixel | 8 bits | JPEG is 8-bit, and 8-bit is what makes §5.2 fit | +| Exposure / color | auto exposure and auto white balance | fixed values | a bed image has to look the same frame to frame; §5.4 | +| Lens | — | no correction applied | correction belongs in the client; §5.5 | +| Encoding | — | MJPEG only, nothing recorded | §5.6 | +| Mirroring | a mirror register | mirrored in software instead | the register breaks capture on this board; §5.7 | + +### 5.1 Resolution and frame rate on a 5 MP machine + +The OV5648 offers several modes, and they are not simply "the same picture, +smaller": + +| Mode | Rate | Field of view | +|---|---|---| +| 2592 × 1944 | 15 fps | the whole sensor | +| 1920 × 1080 | 15 fps | a crop from the middle | +| 1600 × 1200 | 15 fps | a crop from the middle | +| 1280 × 960 | 30 fps | the whole sensor, every other pixel | +| 1280 × 720 | 30 fps | a crop, every other pixel | +| 640 × 480 | 30 fps | the whole sensor, every fourth pixel | + +ForgeFIRM runs the **2592 × 1944** mode. The cropped modes are unusable for a +bed camera — they would show the middle of the bed and cut off the corners. That +leaves the full-frame mode at 15 fps or the skipped 1280 × 960 mode at 30 fps. + +The camera runs **one mode at a time**, and the live stream and the snapshots +come from the same frames: that is what lets a snapshot be delivered while a +stream is running, and it is why the picture does not stutter or re-expose when +you take one. Choosing 1280 × 960 would double the frame rate and permanently +give up full-resolution stills — and full resolution is exactly what bed +alignment, camera calibration and cloud mode need. Full stills win; 15 fps is +the price. + +The stream is halved to 1296 × 972 rather than sent at full size because +demosaicing and encoding 5 megapixels 15 times a second is far beyond this +CPU, and because a 5 MP live view of the bed is of no practical use — it is a +positioning aid, not a photograph. + +### 5.2 8 MP ("HD") machines: a few rows short of the full array + +The OV8856's largest frame is 3280 × 2464. ForgeFIRM captures **3264 × 2448**, +which is 16 columns and 16 rows less — the whole field of view, edge to edge, +just without the last few pixels of margin. + +Getting there is not free, and it explains why §5.3 matters. The sensor can +send its full frame over the two data lanes this board wires, but at 10 bits +per pixel that means running the link at **1.44 Gbit/s per lane**, and the +i.MX6's camera receiver tops out at **1 Gbit/s per lane** — it has no timing +setting for anything faster, so it refuses the mode outright. Asking the sensor +for 8-bit pixels instead cuts a fifth off every sample and lets the same frame +travel at half the rate, which the receiver takes comfortably. That is how an +HD machine gets its full resolution, and it costs nothing, because the +delivered JPEG was going to be 8-bit anyway. + +The result is about 15 frames per second off the sensor, and roughly the same +bytes per second across the bus as a 5 MP machine at its own full frame. + +### 5.3 Ten bits in, eight bits out + +Both sensors can emit 10 bits per pixel. ForgeFIRM asks both for 8 instead, and +the delivered image is 8 bits per channel because that is what JPEG is. + +Two bits would buy nothing without a tone curve to spend them on, and there is +no tone curve (§5.4) — while asking for 8 halves the data crossing the bus, +which is what keeps the stream cheap on a 5 MP machine and what makes full +resolution reachable at all on an 8 MP one (§5.2). + +### 5.4 Exposure, gain and color are fixed + +There is no auto-exposure and no auto white balance. Exposure, gain and the +color balance are set to fixed values when the camera starts, matching what the +factory firmware used, and they are not adjustable from the panel. + +That is deliberate. A bed camera is a measuring instrument: camera-referenced +homing, LightBurn's overlay, and cloud mode's alignment all compare images to +known geometry, and an image whose brightness and color shift between frames — +as the head moves through the frame, or as the laser flashes — is worse than a +consistently imperfect one. + +ForgeFIRM also applies **no gamma, tone curve, sharpening or noise reduction**. +The JPEG is the sensor's data, demosaiced and encoded. Compared with a phone +photo the result looks flat. That is expected; it is not a fault, and it does +not affect how well the image works for alignment. + +One consequence on 8 MP machines: that sensor's driver publishes no color +balance controls at all, so those images will be less color-correct than a +5 MP machine's until the exposure and gain are commissioned on real hardware. + +### 5.5 The lens is not corrected + +The lid lens is a wide fisheye and the image has heavy barrel distortion — +straight bed edges bow. ForgeFIRM sends the image as the lens sees it and does +not attempt to flatten it. + +Correction belongs where the calibration lives: **LightBurn's camera +calibration** pass measures your particular machine's lens and applies the +correction on the host, which is more accurate than a fixed correction baked +into the firmware and costs the machine's CPU nothing. Run that calibration +before trusting the camera overlay for placement. + +### 5.6 MJPEG only — and nothing is recorded + +The stream is a sequence of complete JPEG frames, not H.264 or any other +inter-frame codec, and **the machine never writes video to disk**. + +MJPEG is the right trade here: every frame stands alone, so a viewer can join +or leave at any moment and a dropped frame costs nothing; browsers and sender +software consume it with no plugin; and stream frames are encoded by the +board's **hardware JPEG encoder**, which is what makes 15 fps affordable while +the machine is also running a job. (Stills are encoded in software instead, +which is most of why a full-resolution one takes a couple of seconds.) An +inter-frame codec would need buffering and a container, would break the "any +client, any time" property, and would buy bandwidth savings that a LAN does +not need. + +If you want a recording, record the stream on the computer watching it. The +machine stores its firmware, settings and logs on a small internal flash device +and has no recording feature to fill it with. + +### 5.7 The mirror is applied in software + +The image is mirrored horizontally to match the orientation the factory +software produced. The sensors have a mirror register that would do this for +free, but setting it breaks the board's capture path — frames stop completing +altogether — so the flip is done while the image is being demosaiced instead. +The cost is negligible and the result is identical. + +--- + +## 6. Lighting + +Each camera has its own lamp, and ForgeFIRM drives them around captures: + +- **While capturing**, the relevant lamp is raised to a fixed working level and + restored when the camera goes idle. +- **At rest**, the lid lamp sits at the `lid_lamp_idle` setting (0–255, default + 236) — the bed light you normally see. It is asserted when the daemon starts, + when you change the setting, and whenever a controller starts. +- **Per shot**, `/cam/snapshot` accepts `lamp=0..1023` to override the level for + that one image; a few frames are discarded afterward so the image you get was + actually exposed under the light you asked for. + +In cloud mode the cloud client drives the lid lamp for as long as it runs, and +ForgeFIRM re-asserts your idle level the next time a controller starts. + +--- + +## 7. Sharing one camera path + +Because only one camera can capture at a time, requests have to be arbitrated. +The rule is **the newest request wins**, on the assumption that one person is +standing at the machine: + +- **A new stream preempts the current one.** Existing viewers' streams end + cleanly — their picture freezes on the last frame — rather than being torn + mid-frame. +- **A snapshot of the other camera borrows the path.** The stream pauses, the + mux switches, one frame is taken, and it switches back; viewers see a gap of + a second or two. Snapshots do not fail because someone else is watching. +- **The camera shuts down after 10 seconds** with nobody watching and no + snapshot pending, so other software on the machine can use it — and + immediately, whoever is watching, if the lid opens (§2). + +--- + +## 8. Who else uses the cameras + +- **Camera-referenced homing** (`$H`) takes a lid image and has the Glowforge + service work out where the head is. This is the factory homing method, so it + needs a service session; it is the one part of GRBL mode that does. +- **Cloud mode** captures both cameras on demand through the same snapshot + endpoint, so it obeys the same arbitration as everything else. +- **LightBurn** consumes the lid stream for its camera overlay while it drives + motion over the Grbl connection; the two coexist. + +--- + +## 9. When something looks wrong + +**No picture at all, and a 409 mentioning the lid.** Working as intended: the +lid is open (§2). Close it. `/cam/status` shows `capture_allowed: false` while +that is the case. If the lid *is* shut and you still see this, one of the two +lid switches is not making — the same condition that would stop the laser +firing, so it is worth investigating rather than working around. + +**A black or nearly black picture.** The scene is not lit: the exposure is +fixed, so the camera cannot compensate. Check `lid_lamp_idle`, and remember +snapshots can carry their own `lamp=` level. + +**The stream stops on its own.** Either the lid opened (§2 — the panel says +so), or someone else — another browser tab, LightBurn, the panel — asked for +the other camera, or for a stream, and preempted yours. Reload; the panel does +this automatically and says which it was. + +**"camera switch timed out".** A viewer would not let go within the grace +period. Close the other viewer and retry. + +**A snapshot returns 503.** The camera could not start. The usual cause is +another process holding the capture device; the daemon's log names the failing +step. + +**`/cam/status` reports `"sensor": "unknown"`.** No camera was found on that +bus, or a sensor bound that this firmware has no profile for. On an HD machine +see §9. + +**The picture is bowed / placement is off.** Expected without calibration; see +§5.5. + +--- + +## 10. Status of 8 MP ("HD") machines + +Everything an 8 MP machine needs is in the firmware: the kernel patches for the +OV8856 — including the 8-bit full-resolution mode §5.2 depends on — the +device-tree entries, and a capture path that picks its geometry and sensor +controls from whichever sensor bound. + +**None of it has run on an 8 MP machine.** No such unit has been available to +test against, so treat 8 MP support as untested rather than working: whether +the receiver locks onto the full-resolution mode, and what exposure and gain +the sensor actually wants, can only be settled on that hardware. Frame-rate and +CPU figures in §4 are from a 5 MP machine and do not carry over — an HD machine +demosaics 60 % more pixels per frame. Reports from anyone with an HD machine +are welcome. + +The 5 MP path is hardware-validated and in daily use. + +--- + +## See also + +- [Connecting LightBurn](LIGHTBURN.md) — sender setup and the camera overlay +- [Motion and laser drive](MOTION.md) — what the machine does while you watch +- [Cooling and airflow](COOLING.md) +- [Laser safety](SAFETY.md) diff --git a/forgetest/forgetest/suite/camera.py b/forgetest/forgetest/suite/camera.py index f857bf8..fa13dec 100644 --- a/forgetest/forgetest/suite/camera.py +++ b/forgetest/forgetest/suite/camera.py @@ -1,9 +1,48 @@ """camera.* - the lid camera pipeline through forgectrl.""" from ..catalog import test -_CAM_COVERS = [("forgectrl", "src/cam.*"), ("forgectrl", "src/debayer.*"), ("forgectrl", "src/vpu_jpeg.*"), +_CAM_COVERS = [("forgectrl", "src/cam.*"), ("forgectrl", "src/camhealth.*"), + ("forgectrl", "src/debayer.*"), ("forgectrl", "src/vpu_jpeg.*"), + ("forgectrl", "src/main.c"), ("python3-gfhardware", "gfhardware/src/**"), ("python3-gfhardware", "gfhardware/cam*")] +# The privacy gate spans the camera path and the lid read it depends on, +# in both processes that can reach a sensor. +_PRIVACY_COVERS = _CAM_COVERS + [("forgectrl", "src/status.*"), + ("python3-gfhardware", "gfhardware/switches.py"), + ("forgefirm-app", "forgefirm-app/ffmachine.py")] + +# The sensors the machine ships with, and the geometry each one implies. +# A machine reports exactly one of these; anything else means the sensor +# bound to a driver the capture path has no profile for. +_SENSOR_GEOMETRY = { + "OV5648": (2592, 1944), + "OV8856": (3264, 2448), +} + + +def _jpeg_size(data): + """(width, height) from a JPEG's first SOFn marker, or None.""" + i = 2 + n = len(data) + while i + 3 < n: + if data[i] != 0xFF: + i += 1 + continue + marker = data[i + 1] + if marker in (0xD8, 0xD9) or 0xD0 <= marker <= 0xD7: + i += 2 + continue + seglen = (data[i + 2] << 8) | data[i + 3] + # SOFn, excluding the non-frame markers in the same range + if 0xC0 <= marker <= 0xCF and marker not in (0xC4, 0xC8, 0xCC): + if i + 9 > n: + return None + return ((data[i + 7] << 8) | data[i + 8], + (data[i + 5] << 8) | data[i + 6]) + i += 2 + seglen + return None + @test("camera.snapshot", title="Lid camera snapshot and stream", subsystem="camera", kind="operator", est_min=2, @@ -50,3 +89,206 @@ def snapshot(ctx): ctx.log("cam status after: %s", body) ctx.confirm("Open the control panel (port 8080), Status tab: does the lid snapshot show the bed " "(not black, not frozen, roughly the right orientation)?") + + +@test("camera.sensor-profile", title="Camera geometry follows the fitted sensor", subsystem="camera", + kind="auto", est_min=1, + covers=_CAM_COVERS, requires=["forgectrl.panel-serves"], + description="/cam/status names the sensor that bound (OV5648 on a 5 MP machine, OV8856 on an " + "8 MP 'HD' one) and reports the geometry that sensor implies, and the JPEGs it " + "actually returns are that size. A machine that reports 'unknown', or whose frames " + "do not match what it advertises, is running the wrong capture profile.") +def sensor_profile(ctx): + fc = ctx.forgectrl + ev = ctx.evidence + + st, body = fc.get("/cam/status") + ctx.check(st == 200 and isinstance(body, dict), "GET /cam/status -> %s", st) + sensor = body.get("sensor") + ev["sensor"] = sensor + ctx.log("sensor: %s", sensor) + ctx.check(sensor in _SENSOR_GEOMETRY, + "sensor %r is not one this build has a capture profile for", sensor) + + want_w, want_h = _SENSOR_GEOMETRY[sensor] + snap = body.get("snapshot") or {} + stream = body.get("stream") or {} + ev["geometry"] = {"snapshot": snap, "stream": stream} + ctx.log("advertised: snapshot %sx%s, stream %sx%s", + snap.get("width"), snap.get("height"), stream.get("width"), stream.get("height")) + ctx.check(snap.get("width") == want_w and snap.get("height") == want_h, + "%s should advertise %dx%d snapshots, got %sx%s", + sensor, want_w, want_h, snap.get("width"), snap.get("height")) + ctx.check(stream.get("width") == want_w // 2 and stream.get("height") == want_h // 2, + "%s should advertise %dx%d stream frames, got %sx%s", + sensor, want_w // 2, want_h // 2, stream.get("width"), stream.get("height")) + + # What it advertises is what it delivers. + for res, (w, h) in (("full", (want_w, want_h)), ("half", (want_w // 2, want_h // 2))): + st, data = fc.get("/cam/snapshot", params={"cam": "lid", "res": res}, raw=True) + ctx.check(st == 200 and data and data[:2] == b"\xff\xd8", + "%s snapshot -> %s", res, st) + got = _jpeg_size(data) + ev["jpeg_%s" % res] = got + ctx.log("%s snapshot: %s (%d bytes)", res, got, len(data)) + ctx.check(got == (w, h), "%s snapshot is %s, not %dx%d", res, got, w, h) + + +@test("camera.frame-health", title="Capture delivers whole frames", subsystem="camera", + kind="auto", est_min=1, + covers=_CAM_COVERS, requires=["forgectrl.panel-serves"], + description="The capture queue flags a frame errored when it is short, torn, or arrived " + "after the CSI-2 receiver lost sync; forgectrl drops those rather than " + "demosaicing them, and restarts the stream if they persist. A healthy machine " + "captures a burst with none of them. A nonzero corrupt count here is a real " + "signal - a marginal camera ribbon, a mistimed D-PHY - even though the frames " + "themselves never reach a client.") +def frame_health(ctx): + fc = ctx.forgectrl + ev = ctx.evidence + + def health(): + st, body = fc.get("/cam/status") + ctx.check(st == 200 and isinstance(body, dict), "GET /cam/status -> %s", st) + h = body.get("health") + ctx.check(isinstance(h, dict), "/cam/status carries no health block: %s", body) + return h + + before = health() + ev["health_before"] = before + ctx.log("before: %s", before) + + # A burst, so the count covers a run of frames rather than a single grab. + for _ in range(3): + st, data = fc.get("/cam/snapshot", params={"cam": "lid", "res": "half"}, raw=True) + ctx.check(st == 200 and data and data[:2] == b"\xff\xd8", "snapshot -> %s", st) + + after = health() + ev["health_after"] = after + ctx.log("after: %s", after) + + captured = (after.get("captured") or 0) - (before.get("captured") or 0) + corrupt = (after.get("corrupt") or 0) - (before.get("corrupt") or 0) + restarts = (after.get("restarts") or 0) - (before.get("restarts") or 0) + ev["captured"] = captured + ev["corrupt"] = corrupt + ev["restarts"] = restarts + ctx.log("captured %d frames, %d corrupt, %d stream restarts", captured, corrupt, restarts) + + ctx.check(captured > 0, "no frames were dequeued during three snapshots") + ctx.check(corrupt == 0, "%d of %d captured frames came back errored", corrupt, captured) + ctx.check(restarts == 0, "the capture stream was restarted %d time(s)", restarts) + + +@test("camera.lid-privacy", title="Cameras capture only with the lid closed", subsystem="camera", + kind="operator", est_min=3, + covers=_PRIVACY_COVERS, requires=["forgectrl.panel-serves"], + steps=["You will be asked to open the lid, then close it again.", + "Nothing moves and the laser is not involved."], + description="The privacy gate: with the lid open neither camera captures. A running stream " + "stops within a frame or so of the lid opening, /cam/status reports capture as " + "not allowed, and both the snapshot and the stream are refused with 409 and a " + "reason naming the lid. Closing the lid restores all of it. This is what stops " + "the machine - and in cloud mode the Glowforge service - from imaging the room " + "through an open lid.") +def lid_privacy(ctx): + import urllib.error + import urllib.request + + fc = ctx.forgectrl + ev = ctx.evidence + + def status(): + st, body = fc.get("/cam/status") + ctx.check(st == 200 and isinstance(body, dict), "GET /cam/status -> %s", st) + return body + + def snapshot(): + """(status, body) for a half-res lid snapshot.""" + st, data = fc.get("/cam/snapshot", params={"cam": "lid", "res": "half"}, raw=True) + return st, data or b"" + + # --- lid closed: the baseline the rest is measured against ---------- + ctx.instruct("Close the lid, then click Done.") + body = status() + ctx.check(body.get("capture_allowed") is True, + "with the lid closed /cam/status should allow capture, got %r", + body.get("capture_allowed")) + st, data = snapshot() + ctx.check(st == 200 and data[:2] == b"\xff\xd8", + "snapshot with the lid closed -> %s", st) + ev["closed_snapshot_bytes"] = len(data) + ctx.log("lid closed: snapshot %d bytes", len(data)) + + # --- a live stream must die when the lid opens ---------------------- + req = urllib.request.Request(fc.base + "/cam/stream?cam=lid", + headers={"Host": fc.host_header()}) + stream = urllib.request.urlopen(req, timeout=15) + try: + first = stream.read(4096) + ctx.check(b"\xff\xd8" in first, "the stream did not start before the lid test") + ctx.instruct("The stream is running. Open the lid now, then click Done.") + # The engine tears the pipeline down on the next frame and the + # stream ends. Draining now returns whatever was buffered before + # the lid opened and then EOF; what must not happen is frames + # continuing to arrive indefinitely. + drained, ended = 0, False + try: + while True: + chunk = stream.read(65536) + if not chunk: + ended = True + break + drained += len(chunk) + ctx.check(drained < 8 * 1024 * 1024, + "the stream was still delivering after %d bytes with the lid open", + drained) + except Exception as e: # noqa: BLE001 - a reset connection ends it too + ended = True + ctx.log("stream ended with %s", type(e).__name__) + ev["stream_bytes_after_lid_open"] = drained + ctx.log("stream ended after %d further buffered bytes", drained) + ctx.check(ended, "the stream never ended after the lid opened") + finally: + stream.close() + + # --- lid open: everything is refused, and says why ------------------ + body = status() + ev["status_lid_open"] = body + ctx.check(body.get("capture_allowed") is False, + "with the lid open /cam/status should refuse capture, got %r", + body.get("capture_allowed")) + ctx.check(body.get("stopped_by_lid") is True, + "the engine should record that the lid stopped it, got %r", + body.get("stopped_by_lid")) + ctx.check(body.get("running") is False, + "the capture engine should not still be running with the lid open") + + st, data = snapshot() + ev["snapshot_lid_open"] = [st, data[:120].decode("utf-8", "replace")] + ctx.log("lid open: snapshot -> %s %s", st, data[:120]) + ctx.check(st == 409, "snapshot with the lid open should be refused with 409, got %s", st) + ctx.check(b"lid" in data.lower(), "the refusal should name the lid: %r", data[:120]) + ctx.check(data[:2] != b"\xff\xd8", "a JPEG was returned with the lid open") + + st, data = fc.get("/cam/snapshot", params={"cam": "head", "res": "half"}, raw=True) + ctx.log("lid open: head snapshot -> %s", st) + ctx.check(st == 409, "the head camera should be refused too, got %s", st) + + try: + with urllib.request.urlopen(req, timeout=10) as r: + ctx.fail("the stream opened with the lid open (%s)", r.status) + except urllib.error.HTTPError as e: + ctx.log("lid open: stream -> %s", e.code) + ctx.check(e.code == 409, "the stream should be refused with 409, got %s", e.code) + + # --- closing the lid restores it ----------------------------------- + ctx.instruct("Close the lid again, then click Done.") + body = status() + ctx.check(body.get("capture_allowed") is True, + "closing the lid should allow capture again, got %r", + body.get("capture_allowed")) + st, data = snapshot() + ctx.check(st == 200 and data[:2] == b"\xff\xd8", + "snapshot after closing the lid -> %s", st) + ctx.log("lid closed again: snapshot %d bytes", len(data)) diff --git a/kas/README.md b/kas/README.md index 3a816d7..81ad79b 100644 --- a/kas/README.md +++ b/kas/README.md @@ -188,13 +188,50 @@ Scarthgap, but the legacy (Dunfell/Gatesgarth) layers won't build clean until: fixed oscillator (matching the factory DTB); avdd/dovdd/dvdd rails are in the DT. Both cameras stream live through forgectrl (MJPEG at 15 fps with VPU JPEG encode, full-resolution snapshots, mux arbitration). - **HD-unit caveat:** the DT lists both `ovti,ov5648` (5 MP) and - `ovti,ov8856` (8 MP) at 0x36 so one image covers both, and the driver - matching the chip ID wins — but mainline `ov8856` expects a 19.2 MHz - xvclk and only warns at 24 MHz, and the capture path is written to - ov5648's SBGGR8 2592×1944 format set. 8 MP "HD" modules therefore bind - but do not capture; adding 24 MHz PLL modes plus a sensor-aware capture - path is the open work. + **HD units (8 MP OV8856) — code complete, UNTESTED.** The DT lists both + `ovti,ov5648` (5 MP) and `ovti,ov8856` (8 MP) at 0x36 so one image covers + both, and the driver matching the chip ID wins. Everything the OV8856 + needs is in the build: patch 0011 gives it the `get_mbus_config` the + IPU-CSI hard-fails without (the same gap 0006 closes for ov5648), patch + 0012 retunes both PLL multipliers for the board's 24 MHz xvclk (mainline's + tables are written for 19.2 MHz, which would run the link 25 % above the + frequency the driver publishes), patch 0013 adds the 2-lane RAW8 modes + (below), the endpoint's `link-frequencies` list carries the driver's whole + 2-lane menu (it rejects the endpoint outright if any entry is missing — + the old list omitted 720 MHz, so probe would have failed), and forgectrl + and gfhardware pick geometry and the sensor's control set from whichever + driver bound. + + The capture mode is the **full 3264×2448**, reached in RAW8. The + sensor's stock RAW10 full-resolution 2-lane mode asks for 1.44 Gbps/lane + and the i.MX6 CSI-2 D-PHY stops at 1 Gbps (`hsfreq_map` in + `imx6-mipi-csi2.c` ends at 1000 Mbps and `max_mbps_to_hsfreqrange_sel()` + returns `-EINVAL` above it), so `imx6-mipi-csi2` refuses to program it — + but 8-bit samples carry the same frame at half the rate, which puts it on + the 360 MHz link the binned modes already use, at 180 Mpx/s and 15 fps. + Patch 0013 builds those modes from mainline's own 4-lane 3264×2448 and + 1632×1224 register lists plus a per-mode delta list: `0x3018` for two + lanes, `0x3031` for 8-bit readout, and double the HTS because half the + lanes carry half a line in the same time. It also makes the sample depth a + mode property, so pixel rate, blanking and exposure ranges follow the mode + instead of a fixed 10. Side effect worth having: the OV8856 path becomes + byte-identical in shape to the OV5648's (8-bit BGGR, one byte per sample), + and 3264 is a multiple of 32 so the NEON superpixel converter applies, + which 1640 did not allow. + + The values are the factory firmware's: its own OV8856 driver is RAW8-only + and ships exactly these two resolutions over two lanes with the same + `0x3018`/`0x3031` and the same HTS/VTS pairs. It reaches them through a + different PLL divider chain (`0x0302=0x1e`, `0x0303=0x03`, `0x030f=0x07`, + `0x0312=0x05`, `0x4837=0x58`) that halves the link again to 180 MHz and + the internal SCLK with it — a self-consistent alternative, recorded in the + patch header as the configuration to fall back to if the D-PHY will not + lock at 720 Mbps/lane on real hardware. + + Open, and only answerable on an 8 MP machine: whether it streams at all at + 720 Mbps/lane, and exposure/gain/white-balance commissioning — the OV8856 + driver publishes no red/blue balance controls, so white balance is + uncorrected. 3. **u-boot** — **DONE.** The `glowforge` u-boot is a standalone `u-boot_2020.01.bb` (Scarthgap's poky has no u-boot 2020.01 base recipe to extend). It reuses poky's