vh/forgefirm
1
0
Fork 0
mirror of https://github.com/openglow-org/forgefirm.git synced 2026-09-28 01:01:12 -07:00
Code Issues Packages Projects Releases Wiki Activity

docs, acceptance: the cameras as users meet them, and 8 MP at full resolution

Browse Source 
docs/VIDEO.md is the user-facing camera guide: what the endpoints return, what
the sensors can do that ForgeFIRM does not send and why, the privacy gate, and
the state of 8 MP support.

The 8 MP (OV8856) capture path now reaches the sensor's full 3264x2448 frame.
Its stock RAW10 full-resolution mode runs the link at 1.44 Gbps/lane and the
i.MX6 CSI-2 D-PHY stops at 1 Gbps; the BSP adds a RAW8 mode that carries the
same frame at half the rate, so an HD machine is no longer limited to the
binned quarter-pixel mode. kas/README.md carries the reasoning, the register
deltas and the factory configuration to fall back to if the receiver will not
lock at 720 Mbps/lane.

Acceptance: camera.sensor-profile expects the new OV8856 geometry, the camera
covers map names src/camhealth.* (the src/cam.* glob does not match it, so the
lint would have gone quiet on an uncovered file at the next pin bump), and a
new auto test camera.frame-health asserts a burst of captures with no frames
the capture queue flagged errored - a real signal about the camera ribbon even
though those frames never reach a client.
This commit is contained in:
ScottW514
2026-08-17 14:34:36 -04:00
parent d3bab940b3
commit c425822b82
5 changed files with 748 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/VIDEO.md
+410
View File
@@ -0,0 +1,410 @@
# Video and the cameras
The machine has two cameras — one in the lid looking down at the bed, one in the
print head looking at the material under the lens — and ForgeFIRM serves both as
plain **MJPEG over HTTP** from the web control panel. There is no app, no cloud
relay, and no proprietary protocol: a browser, LightBurn, or anything else that
can read an MJPEG stream or fetch a JPEG can use them.
One rule governs all of it: **the cameras only capture with the lid closed**
(§2). Everything else here assumes that condition is met.
This page explains what you get, how to point a client at it, what the sensors
are physically capable of versus what ForgeFIRM actually sends and why, and what
you can change.
- For sender setup in general, see [Connecting LightBurn](LIGHTBURN.md).
- The cloud mode described in §8 is documented in
[Cloud mode](https://github.com/ScottW514/python3-gfhardware/blob/master/forgefirm-app/docs/CLOUD.md).
---
## 1. What the hardware is
The two cameras carry the same kind of sensor and feed one shared path into the
board. A **hardware MIPI switch** (the factory `CAM_SEL` line) selects which of
them reaches the board's single camera receiver, so **exactly one camera can be
capturing at any moment**. That is a property of the board, not a software
limit — there is no configuration in which both stream at once.
| | Lid camera | Head camera |
|---|---|---|
| Sees | the whole bed, from above | the material directly under the lens |
| Its lamp | the lid LED strip | the white LED in the print head |
| Used for | bed view, camera-referenced homing, LightBurn's camera overlay | focus and material inspection (cloud mode's distance measurement) |
The lid lens is a wide fisheye, which matters for how you use the image (§5.5).
Which sensor is fitted depends on the machine. Standard machines carry a **5 MP
OV5648**; "HD" machines carry an **8 MP OV8856**. ForgeFIRM reads which one
bound and configures itself accordingly — one firmware image covers both — and
reports it in `/cam/status` and on the panel's Status tab.
---
## 2. Privacy: the cameras only work with the lid closed
**Neither camera captures anything while the lid is open.** Not the live view,
not a snapshot, and not an image requested by the Glowforge service in cloud
mode. Close the lid and everything works; open it and the sensors stop.
The reason is where the lid camera points. It is mounted in the lid, so raising
the lid swings it up to face the room — and in cloud mode the shutter is not
yours to press: the service asks for images on its own schedule, whenever it is
connected. The rule removes the question. The enclosure being shut is the
condition for an image to exist at all.
**What the rule covers**
- **Both cameras.** The head camera is gated too, so this is one rule to
remember rather than a rule with an exception you have to trust.
- **Every way in:** the panel, `/cam/stream`, `/cam/snapshot`, the
mjpg-streamer aliases, LightBurn, and cloud mode's image actions.
- **Capture already running.** Opening the lid stops a live stream within
about a frame and shuts the sensor down; it does not merely block new
requests.
- **The lamps.** A refused capture never raises them, so an attempt with the
lid open leaves no trace.
**How it behaves**
| Situation | What happens |
|---|---|
| Snapshot requested with the lid open | `409` and a message naming the lid; no image data |
| Stream requested with the lid open | `409`; the stream never opens |
| Lid opened while a stream is running | the stream ends cleanly and the pipeline is torn down |
| Lid state unreadable | treated as open — capture refused |
| Cloud service asks for an image with the lid open | refused, and reported back to the service as a failed action rather than left hanging |
| Lid closed again | everything works immediately; nothing to restart |
`GET /cam/status` reports it: **`capture_allowed`** is false whenever the lid is
open, and **`stopped_by_lid`** records that the last capture ended because the
lid opened rather than going idle. The panel's Status tab says *lid open — the
cameras are off* rather than showing a stream error.
**Where the check comes from.** The lid signal is the same one the hardware
safety chain uses to gate the beam — the series combination of both lid
switches, not a software flag — and the check **fails closed**: if the lid
state cannot be read at all, the cameras stay dark. That direction is proven by
a unit test in CI; the end-to-end behavior is an acceptance test run on real
hardware.
**One thing it costs.** The factory firmware ran the cloud's focus *hunt* with
the lid open, and part of a hunt is a head capture. Those captures are now
refused, so a hunt attempted with the lid open fails instead of completing.
Close the lid before letting the app focus or print.
**What it is not.** This is a rule enforced by the two programs that own the
sensors, not a hardware cut-off: the sensor rails stay powered, and anyone with
root on the machine could bypass it. It protects you from the Glowforge
service, from other software on your network, and from a stream you forgot was
running — not from someone who already controls the board. There is
deliberately no setting to turn it off.
---
## 3. Watching it
**In the panel.** Open `http://<machine>:8080/` and go to the **Status** tab.
The *Lid camera* card shows a still by default with **Live** and **Refresh**
buttons; **Live** switches the same frame to the running stream.
**From another program.** The endpoints are:
| URL | What it returns |
|---|---|
| `/cam/stream?cam=lid` | continuous MJPEG (`multipart/x-mixed-replace`) |
| `/cam/stream?cam=head` | the same, from the head camera |
| `/cam/snapshot?cam=lid` | one full-resolution JPEG |
| `/cam/snapshot?cam=lid&res=half` | one half-resolution JPEG (much faster) |
| `/cam/status` | JSON: which sensor, which camera, frame rate, frame sizes, whether the lid currently permits capture |
| `/?action=stream` | the lid stream again, under the name mjpg-streamer clients expect |
| `/?action=snapshot` | one full-resolution lid JPEG, same aliasing |
The `?action=` pair exists because a lot of software — print-server dashboards,
camera widgets, anything written against mjpg-streamer — assumes those exact
URLs. Point such a client at `http://<machine>:8080/` and it will work.
Every one of them answers **`409`** with the lid open (§2), so a client that
checks status codes can tell "close the lid" apart from "the camera is broken".
**Access.** Reading the camera needs no token. It does need a request that
addresses the machine by IP address (or `localhost`) and, from a browser, one
that is not cross-site — that is what stops a hostile page in another tab from
reaching into your machine. It is not protection against other people on your
LAN. Anything that *changes* machine state does need the panel's token. In
practice: paste the URL into any local client and it works.
---
## 4. What you actually get
| | 5 MP machine (OV5648) | 8 MP machine (OV8856) |
|---|---|---|
| Sensor frame captured | 2592 × 1944 | 3264 × 2448 |
| Live stream | 1296 × 972 | 1632 × 1224 |
| Full snapshot | 2592 × 1944 | 3264 × 2448 |
| Half snapshot | 1296 × 972 | 1632 × 1224 |
| Format | JPEG, quality 75 by default | same |
| Frame rate | **15 fps** sustained | not yet measured (§10) |
Measured on a 5 MP machine: 15.0 fps with a viewer attached, which is the rate
the sensor itself produces in this mode — the machine is not the bottleneck.
The daemon uses about 41 % of one CPU with one viewer, and LightBurn can watch
the stream while jogging from the same session without disturbing motion. A
full-resolution still takes about 2.4 s to produce (2.7 s if the camera has to
be started first), because 5 megapixels of demosaicing and JPEG encoding happen
on the machine's CPU.
The stream frame is not a resampled copy of the full frame. Each 2 × 2 group of
sensor pixels becomes exactly one output pixel, which is why the stream is
precisely half the capture in each axis and why it is cheap enough to run
continuously.
---
## 5. What the sensor can do versus what ForgeFIRM sends
This is where expectations usually come unstuck: the sensors are more capable
on paper than the video you get. Each difference below is deliberate and has a
reason.
| | The sensor can | ForgeFIRM sends | Why |
|---|---|---|---|
| Live resolution | full frame | half in each axis | CPU and bandwidth; §5.1 |
| Frame rate (5 MP) | 30 fps in reduced modes | 15 fps | the full-field mode runs at 15 fps; §5.1 |
| Resolution (8 MP) | 3280 × 2464 | 3264 × 2448 | the widest frame the board's camera receiver can take; §5.2 |
| Bit depth | 10 bits per pixel | 8 bits | JPEG is 8-bit, and 8-bit is what makes §5.2 fit |
| Exposure / color | auto exposure and auto white balance | fixed values | a bed image has to look the same frame to frame; §5.4 |
| Lens | — | no correction applied | correction belongs in the client; §5.5 |
| Encoding | — | MJPEG only, nothing recorded | §5.6 |
| Mirroring | a mirror register | mirrored in software instead | the register breaks capture on this board; §5.7 |
### 5.1 Resolution and frame rate on a 5 MP machine
The OV5648 offers several modes, and they are not simply "the same picture,
smaller":
| Mode | Rate | Field of view |
|---|---|---|
| 2592 × 1944 | 15 fps | the whole sensor |
| 1920 × 1080 | 15 fps | a crop from the middle |
| 1600 × 1200 | 15 fps | a crop from the middle |
| 1280 × 960 | 30 fps | the whole sensor, every other pixel |
| 1280 × 720 | 30 fps | a crop, every other pixel |
| 640 × 480 | 30 fps | the whole sensor, every fourth pixel |
ForgeFIRM runs the **2592 × 1944** mode. The cropped modes are unusable for a
bed camera — they would show the middle of the bed and cut off the corners. That
leaves the full-frame mode at 15 fps or the skipped 1280 × 960 mode at 30 fps.
The camera runs **one mode at a time**, and the live stream and the snapshots
come from the same frames: that is what lets a snapshot be delivered while a
stream is running, and it is why the picture does not stutter or re-expose when
you take one. Choosing 1280 × 960 would double the frame rate and permanently
give up full-resolution stills — and full resolution is exactly what bed
alignment, camera calibration and cloud mode need. Full stills win; 15 fps is
the price.
The stream is halved to 1296 × 972 rather than sent at full size because
demosaicing and encoding 5 megapixels 15 times a second is far beyond this
CPU, and because a 5 MP live view of the bed is of no practical use — it is a
positioning aid, not a photograph.
### 5.2 8 MP ("HD") machines: a few rows short of the full array
The OV8856's largest frame is 3280 × 2464. ForgeFIRM captures **3264 × 2448**,
which is 16 columns and 16 rows less — the whole field of view, edge to edge,
just without the last few pixels of margin.
Getting there is not free, and it explains why §5.3 matters. The sensor can
send its full frame over the two data lanes this board wires, but at 10 bits
per pixel that means running the link at **1.44 Gbit/s per lane**, and the
i.MX6's camera receiver tops out at **1 Gbit/s per lane** — it has no timing
setting for anything faster, so it refuses the mode outright. Asking the sensor
for 8-bit pixels instead cuts a fifth off every sample and lets the same frame
travel at half the rate, which the receiver takes comfortably. That is how an
HD machine gets its full resolution, and it costs nothing, because the
delivered JPEG was going to be 8-bit anyway.
The result is about 15 frames per second off the sensor, and roughly the same
bytes per second across the bus as a 5 MP machine at its own full frame.
### 5.3 Ten bits in, eight bits out
Both sensors can emit 10 bits per pixel. ForgeFIRM asks both for 8 instead, and
the delivered image is 8 bits per channel because that is what JPEG is.
Two bits would buy nothing without a tone curve to spend them on, and there is
no tone curve (§5.4) — while asking for 8 halves the data crossing the bus,
which is what keeps the stream cheap on a 5 MP machine and what makes full
resolution reachable at all on an 8 MP one (§5.2).
### 5.4 Exposure, gain and color are fixed
There is no auto-exposure and no auto white balance. Exposure, gain and the
color balance are set to fixed values when the camera starts, matching what the
factory firmware used, and they are not adjustable from the panel.
That is deliberate. A bed camera is a measuring instrument: camera-referenced
homing, LightBurn's overlay, and cloud mode's alignment all compare images to
known geometry, and an image whose brightness and color shift between frames —
as the head moves through the frame, or as the laser flashes — is worse than a
consistently imperfect one.
ForgeFIRM also applies **no gamma, tone curve, sharpening or noise reduction**.
The JPEG is the sensor's data, demosaiced and encoded. Compared with a phone
photo the result looks flat. That is expected; it is not a fault, and it does
not affect how well the image works for alignment.
One consequence on 8 MP machines: that sensor's driver publishes no color
balance controls at all, so those images will be less color-correct than a
5 MP machine's until the exposure and gain are commissioned on real hardware.
### 5.5 The lens is not corrected
The lid lens is a wide fisheye and the image has heavy barrel distortion —
straight bed edges bow. ForgeFIRM sends the image as the lens sees it and does
not attempt to flatten it.
Correction belongs where the calibration lives: **LightBurn's camera
calibration** pass measures your particular machine's lens and applies the
correction on the host, which is more accurate than a fixed correction baked
into the firmware and costs the machine's CPU nothing. Run that calibration
before trusting the camera overlay for placement.
### 5.6 MJPEG only — and nothing is recorded
The stream is a sequence of complete JPEG frames, not H.264 or any other
inter-frame codec, and **the machine never writes video to disk**.
MJPEG is the right trade here: every frame stands alone, so a viewer can join
or leave at any moment and a dropped frame costs nothing; browsers and sender
software consume it with no plugin; and stream frames are encoded by the
board's **hardware JPEG encoder**, which is what makes 15 fps affordable while
the machine is also running a job. (Stills are encoded in software instead,
which is most of why a full-resolution one takes a couple of seconds.) An
inter-frame codec would need buffering and a container, would break the "any
client, any time" property, and would buy bandwidth savings that a LAN does
not need.
If you want a recording, record the stream on the computer watching it. The
machine stores its firmware, settings and logs on a small internal flash device
and has no recording feature to fill it with.
### 5.7 The mirror is applied in software
The image is mirrored horizontally to match the orientation the factory
software produced. The sensors have a mirror register that would do this for
free, but setting it breaks the board's capture path — frames stop completing
altogether — so the flip is done while the image is being demosaiced instead.
The cost is negligible and the result is identical.
---
## 6. Lighting
Each camera has its own lamp, and ForgeFIRM drives them around captures:
- **While capturing**, the relevant lamp is raised to a fixed working level and
restored when the camera goes idle.
- **At rest**, the lid lamp sits at the `lid_lamp_idle` setting (0–255, default
236) — the bed light you normally see. It is asserted when the daemon starts,
when you change the setting, and whenever a controller starts.
- **Per shot**, `/cam/snapshot` accepts `lamp=0..1023` to override the level for
that one image; a few frames are discarded afterward so the image you get was
actually exposed under the light you asked for.
In cloud mode the cloud client drives the lid lamp for as long as it runs, and
ForgeFIRM re-asserts your idle level the next time a controller starts.
---
## 7. Sharing one camera path
Because only one camera can capture at a time, requests have to be arbitrated.
The rule is **the newest request wins**, on the assumption that one person is
standing at the machine:
- **A new stream preempts the current one.** Existing viewers' streams end
cleanly — their picture freezes on the last frame — rather than being torn
mid-frame.
- **A snapshot of the other camera borrows the path.** The stream pauses, the
mux switches, one frame is taken, and it switches back; viewers see a gap of
a second or two. Snapshots do not fail because someone else is watching.
- **The camera shuts down after 10 seconds** with nobody watching and no
snapshot pending, so other software on the machine can use it — and
immediately, whoever is watching, if the lid opens (§2).
---
## 8. Who else uses the cameras
- **Camera-referenced homing** (`$H`) takes a lid image and has the Glowforge
service work out where the head is. This is the factory homing method, so it
needs a service session; it is the one part of GRBL mode that does.
- **Cloud mode** captures both cameras on demand through the same snapshot
endpoint, so it obeys the same arbitration as everything else.
- **LightBurn** consumes the lid stream for its camera overlay while it drives
motion over the Grbl connection; the two coexist.
---
## 9. When something looks wrong
**No picture at all, and a 409 mentioning the lid.** Working as intended: the
lid is open (§2). Close it. `/cam/status` shows `capture_allowed: false` while
that is the case. If the lid *is* shut and you still see this, one of the two
lid switches is not making — the same condition that would stop the laser
firing, so it is worth investigating rather than working around.
**A black or nearly black picture.** The scene is not lit: the exposure is
fixed, so the camera cannot compensate. Check `lid_lamp_idle`, and remember
snapshots can carry their own `lamp=` level.
**The stream stops on its own.** Either the lid opened (§2 — the panel says
so), or someone else — another browser tab, LightBurn, the panel — asked for
the other camera, or for a stream, and preempted yours. Reload; the panel does
this automatically and says which it was.
**"camera switch timed out".** A viewer would not let go within the grace
period. Close the other viewer and retry.
**A snapshot returns 503.** The camera could not start. The usual cause is
another process holding the capture device; the daemon's log names the failing
step.
**`/cam/status` reports `"sensor": "unknown"`.** No camera was found on that
bus, or a sensor bound that this firmware has no profile for. On an HD machine
see §9.
**The picture is bowed / placement is off.** Expected without calibration; see
§5.5.
---
## 10. Status of 8 MP ("HD") machines
Everything an 8 MP machine needs is in the firmware: the kernel patches for the
OV8856 — including the 8-bit full-resolution mode §5.2 depends on — the
device-tree entries, and a capture path that picks its geometry and sensor
controls from whichever sensor bound.
**None of it has run on an 8 MP machine.** No such unit has been available to
test against, so treat 8 MP support as untested rather than working: whether
the receiver locks onto the full-resolution mode, and what exposure and gain
the sensor actually wants, can only be settled on that hardware. Frame-rate and
CPU figures in §4 are from a 5 MP machine and do not carry over — an HD machine
demosaics 60 % more pixels per frame. Reports from anyone with an HD machine
are welcome.
The 5 MP path is hardware-validated and in daily use.
---
## See also
- [Connecting LightBurn](LIGHTBURN.md) — sender setup and the camera overlay
- [Motion and laser drive](MOTION.md) — what the machine does while you watch
- [Cooling and airflow](COOLING.md)
- [Laser safety](SAFETY.md)