test(visual): a shadow-gap measurement, and the spawn height cleared
Tools/shadow_gap.py measures the vertical pixel gap between an object's lowest pixels and the top of its shadow, as a fraction of the object's height, and calls it contact or floating. It classifies a shadow by per-channel scaling - a shadow darkens every channel by the same proportion, an object does not - which separates the two where an absolute delta cannot. Four unit tests cover contact, floating, an empty box and the border median. On the real spawn view it returns 'unknown': the crop contains two surfaces (pale wall behind, dark road below) and one global background cannot classify both. Recorded as the next step, not presented as working. The spawn height is measured and cleared: tracing down from 30 m, the topmost surface under the player is Z 366 and the capsule's feet are at 369 - 3 cm, the capsule skin. If the character still reads as floating, the gap is between the rendered pavement and its collision surface, not the spawn Z, and that is what to measure next.
This commit is contained in:
1 parent
dde451fd69
commit
f1d6283384
5 files changed
+230
-1
No files matched your search
@@ -99,6 +99,7 @@ Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side b
|
||||
| `vehicle_from_template.py` | Converts the owner's `hot hatch.blend` into UE-ready assets: normalises the scale to a documented 4.05 m, splits systems (body/trim/glass/lights/wheels), adds a `UCX_` hull, decimates two LODs, exports FBX per system and writes a report. | `--out`, `--report` | `blender -b "Art/blender/vehicle/hot hatch.blend" --python Tools/vehicle_from_template.py -- --out /tmp/gigline/hatch` |
|
||||
| `blender_screenshots.py` | Renders review screenshots out of a .blend: one per mesh object (`--mode objects`) and four turntable angles (`--mode scene`), framed from the scene's own bounds, into `Screenshots/blender/`. | `--label`, `--mode`, `--out`, `--report` | `blender -b "Art/blender/vehicle/hot hatch.blend" --python Tools/blender_screenshots.py -- --label vehicle --mode both` |
|
||||
| `build_comparison_md.py` | Regenerates the `Blender vs in-game` section of `Screenshots/README.md` between `<!-- blender-comparison:begin/end -->` markers: in-game captures beside their Blender renders in one table, plus a signoff table with one row per in-game screen. Pairs by name with an alias map for camera-named shots. Run it after every capture or Blender render. | (no arguments) | `python3 Tools/build_comparison_md.py` |
|
||||
| `shadow_gap.py` | Measures the vertical pixel gap between an object's lowest pixels and the top of its shadow, as a fraction of the object's height, and calls it `contact` or `floating`. Objective replacement for asking a vision model whether something floats. | `--image`, `--object-box`, `--gap-fraction`, `--out` | `python3 Tools/shadow_gap.py --image Screenshots/demo/review-demo-spawn-view.png --object-box 0.44,0.48,0.60,0.99` |
|
||||
|
||||
## Tests
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Measure how far an object sits above its own shadow.
|
||||
|
||||
python3 Tools/shadow_gap.py --image Screenshots/demo/review-demo-spawn-view.png \
|
||||
--object-box 0.52,0.55,0.60,0.75 --out /tmp/gigline/shadow-gap.json
|
||||
|
||||
Why a measurement instead of another opinion: a still cannot distinguish "the object floats"
|
||||
from "the object stands on a raised kerb and its shadow falls past it onto the road", and a
|
||||
vision model asked to arbitrate gives a different answer each run. The vertical pixel gap
|
||||
between the lowest object pixel and the top of its shadow is objective.
|
||||
|
||||
Method, deliberately simple and stated so it can be argued with:
|
||||
|
||||
1. Crop the region of interest (the object box, fractions of the frame).
|
||||
2. The object is the set of pixels that differ from the local background - the median colour of
|
||||
the crop's border.
|
||||
3. The shadow is the darkest connected region *below* the object.
|
||||
4. Report the gap in pixels between the object's lowest row and the shadow's top row, and the
|
||||
object's height in pixels, so the gap can be read as a fraction of the object.
|
||||
|
||||
A large gap relative to the object's height is the floating signal; a small gap is contact.
|
||||
The thresholds are arguments, not constants, because the right values depend on the camera.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
from PIL import Image
|
||||
|
||||
DEFAULT_GAP_FRACTION = 0.12 # gap > 12 % of the object's pixel height reads as floating
|
||||
SHADOW_DELTA = 38 # how much darker than the background a pixel must be to be shadow
|
||||
|
||||
|
||||
def border_median(image: Image.Image):
|
||||
"""Median colour of the crop's border, used as the local background."""
|
||||
pixels = image.load()
|
||||
width, height = image.size
|
||||
samples = []
|
||||
step = max(1, width // 60)
|
||||
for x in range(0, width, step):
|
||||
samples.append(pixels[x, 1])
|
||||
samples.append(pixels[x, height - 2])
|
||||
for y in range(0, height, max(1, height // 60)):
|
||||
samples.append(pixels[1, y])
|
||||
samples.append(pixels[width - 2, y])
|
||||
if not samples:
|
||||
return (128, 128, 128)
|
||||
channels = list(zip(*samples))
|
||||
return tuple(sorted(channel)[len(channel) // 2] for channel in channels)
|
||||
|
||||
|
||||
def classify_pixels(image: Image.Image, background, object_delta=18):
|
||||
"""Return two masks: the object (differs from background) and the shadow (darker)."""
|
||||
pixels = image.load()
|
||||
width, height = image.size
|
||||
object_rows = [0] * height
|
||||
shadow_rows = [0] * height
|
||||
for y in range(height):
|
||||
for x in range(width):
|
||||
r, g, b = pixels[x, y]
|
||||
delta = max(abs(r - background[0]), abs(g - background[1]), abs(b - background[2]))
|
||||
luminance = 0.2126 * r + 0.7152 * g + 0.0722 * b
|
||||
background_luminance = (0.2126 * background[0] + 0.7152 * background[1]
|
||||
+ 0.0722 * background[2])
|
||||
# A shadow is a darkened version of the surface; an object is much darker or a
|
||||
# different colour. Classifying by absolute delta alone put every shadow in the
|
||||
# object mask, which is why the first runs reported "no shadow found".
|
||||
ratio = luminance / max(1.0, background_luminance)
|
||||
# A shadow darkens every channel by the same proportion; an object does not.
|
||||
# Comparing the per-channel scale factors is what separates them - an absolute
|
||||
# delta cannot, because a shadow's delta is as large as a dark object's.
|
||||
scales = [rgb / max(1.0, base) for rgb, base in zip((r, g, b), background)]
|
||||
spread = max(scales) - min(scales)
|
||||
if 0.45 <= ratio < 0.90 and spread < 0.15:
|
||||
shadow_rows[y] += 1
|
||||
elif ratio < 0.45 or delta > object_delta:
|
||||
object_rows[y] += 1
|
||||
return object_rows, shadow_rows
|
||||
|
||||
|
||||
def measure(image_path: str, box, gap_fraction=DEFAULT_GAP_FRACTION) -> dict:
|
||||
image = Image.open(image_path).convert("RGB")
|
||||
width, height = image.size
|
||||
x0, y0, x1, y1 = (int(box[0] * width), int(box[1] * height),
|
||||
int(box[2] * width), int(box[3] * height))
|
||||
crop = image.crop((x0, y0, x1, y1))
|
||||
background = border_median(crop)
|
||||
object_rows, shadow_rows = classify_pixels(crop, background)
|
||||
|
||||
object_rows_nonzero = [y for y, count in enumerate(object_rows) if count > 4]
|
||||
if not object_rows_nonzero:
|
||||
return {"error": "no object found in the box", "background": background}
|
||||
object_top, object_bottom = object_rows_nonzero[0], object_rows_nonzero[-1]
|
||||
object_height = max(1, object_bottom - object_top)
|
||||
|
||||
# Shadow rows below the object's lowest row.
|
||||
shadow_below = [y for y, count in enumerate(shadow_rows)
|
||||
if y > object_bottom and count > 4]
|
||||
shadow_top = shadow_below[0] if shadow_below else None
|
||||
gap = None if shadow_top is None else shadow_top - object_bottom
|
||||
|
||||
verdict = "unknown"
|
||||
if gap is not None:
|
||||
verdict = "floating" if gap > object_height * gap_fraction else "contact"
|
||||
return {
|
||||
"image": os.path.abspath(image_path),
|
||||
"box_pixels": [x0, y0, x1, y1],
|
||||
"background": list(background),
|
||||
"object_top_row": object_top,
|
||||
"object_bottom_row": object_bottom,
|
||||
"object_height_px": object_height,
|
||||
"shadow_top_row": shadow_top,
|
||||
"gap_px": gap,
|
||||
"gap_fraction_of_object": None if gap is None else round(gap / object_height, 3),
|
||||
"threshold_fraction": gap_fraction,
|
||||
"verdict": verdict,
|
||||
}
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--image", required=True)
|
||||
ap.add_argument("--object-box", required=True,
|
||||
help="x0,y0,x1,y1 as fractions of the frame around the object and its shadow")
|
||||
ap.add_argument("--gap-fraction", type=float, default=DEFAULT_GAP_FRACTION,
|
||||
help="gap/height above this reads as floating (default 0.12)")
|
||||
ap.add_argument("--out", default="/tmp/gigline/shadow-gap.json")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
box = [float(v) for v in args.object_box.split(",")]
|
||||
if len(box) != 4:
|
||||
print("error: --object-box needs four comma-separated fractions", file=sys.stderr)
|
||||
return 2
|
||||
if not os.path.exists(args.image):
|
||||
print(f"error: no such image: {args.image}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
result = measure(args.image, box, args.gap_fraction)
|
||||
os.makedirs(os.path.dirname(os.path.abspath(args.out)) or ".", exist_ok=True)
|
||||
with open(args.out, "w") as handle:
|
||||
json.dump(result, handle, indent=2)
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0 if result.get("verdict") != "unknown" else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,56 @@
|
||||
"""Tests for the pure parts of Tools/shadow_gap.py."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
import shadow_gap as sg # noqa: E402
|
||||
|
||||
|
||||
def synthetic(object_bottom, shadow_top, size=(80, 120), background=(200, 200, 196)):
|
||||
"""A grey frame with a dark block (object) and a dark strip (shadow) below it."""
|
||||
image = Image.new("RGB", size, background)
|
||||
pixels = image.load()
|
||||
for y in range(20, object_bottom):
|
||||
for x in range(30, 50):
|
||||
pixels[x, y] = (18, 18, 20) # object: far darker than the surface
|
||||
for y in range(shadow_top, min(size[1] - 2, shadow_top + 12)):
|
||||
for x in range(30, 50):
|
||||
# shadow: a darkened version of the surface, as a real shadow is
|
||||
pixels[x, y] = (140, 140, 138)
|
||||
return image
|
||||
|
||||
|
||||
class MeasureTests(unittest.TestCase):
|
||||
def test_contact_reads_as_contact(self):
|
||||
image = synthetic(object_bottom=60, shadow_top=61)
|
||||
image.save("/tmp/gigline-shadow-contact.png")
|
||||
result = sg.measure("/tmp/gigline-shadow-contact.png", (0.2, 0.05, 0.8, 0.95))
|
||||
self.assertEqual(result["verdict"], "contact")
|
||||
self.assertLessEqual(result["gap_px"], 2)
|
||||
|
||||
def test_a_gap_reads_as_floating(self):
|
||||
image = synthetic(object_bottom=60, shadow_top=100)
|
||||
image.save("/tmp/gigline-shadow-float.png")
|
||||
result = sg.measure("/tmp/gigline-shadow-float.png", (0.2, 0.05, 0.8, 0.95))
|
||||
self.assertEqual(result["verdict"], "floating")
|
||||
self.assertGreater(result["gap_px"], result["object_height_px"] * 0.12)
|
||||
|
||||
def test_empty_box_reports_an_error(self):
|
||||
image = Image.new("RGB", (40, 40), (200, 200, 200))
|
||||
image.save("/tmp/gigline-shadow-empty.png")
|
||||
result = sg.measure("/tmp/gigline-shadow-empty.png", (0.1, 0.1, 0.9, 0.9))
|
||||
self.assertIn("error", result)
|
||||
|
||||
|
||||
class BorderTests(unittest.TestCase):
|
||||
def test_border_median_is_the_frame_colour(self):
|
||||
image = Image.new("RGB", (40, 40), (180, 190, 200))
|
||||
self.assertEqual(sg.border_median(image), (180, 190, 200))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user