test(visual): a shadow-gap measurement, and the spawn height cleared
Tools/shadow_gap.py measures the vertical pixel gap between an object's lowest pixels and the top of its shadow, as a fraction of the object's height, and calls it contact or floating. It classifies a shadow by per-channel scaling - a shadow darkens every channel by the same proportion, an object does not - which separates the two where an absolute delta cannot. Four unit tests cover contact, floating, an empty box and the border median. On the real spawn view it returns 'unknown': the crop contains two surfaces (pale wall behind, dark road below) and one global background cannot classify both. Recorded as the next step, not presented as working. The spawn height is measured and cleared: tracing down from 30 m, the topmost surface under the player is Z 366 and the capsule's feet are at 369 - 3 cm, the capsule skin. If the character still reads as floating, the gap is between the rendered pavement and its collision surface, not the spawn Z, and that is what to measure next.
This commit is contained in:
zwitschi committed
1 parent
dde451fd69
commit
f1d6283384
5 files changed
+230
-1
No files matched your search
@@ -382,6 +382,23 @@ void AGigLineCharacter::LogFootProbes()
|
||||
static const ECollisionChannel Channels[] = {ECC_Visibility, ECC_WorldStatic, ECC_Pawn};
|
||||
const float CapsuleHalf = GetCapsuleComponent()
|
||||
? GetCapsuleComponent()->GetScaledCapsuleHalfHeight() : 0.f;
|
||||
// Trace from well ABOVE the head down: the first hit is the topmost surface, i.e. the
|
||||
// pavement the player can see. Tracing up from the feet finds whatever is below them,
|
||||
// which is how "ground 366" got reported while the pavement on screen was higher.
|
||||
if (UWorld* WorldTop = GetWorld())
|
||||
{
|
||||
FHitResult Top;
|
||||
FCollisionQueryParams QTop(TEXT("GigLineTop"), false, this);
|
||||
const FVector From = GetActorLocation() + FVector(0.f, 0.f, 3000.f);
|
||||
const bool bTop = WorldTop->LineTraceSingleByChannel(Top, From,
|
||||
From - FVector(0.f, 0.f, 6000.f), ECC_WorldStatic, QTop);
|
||||
UE_LOG(LogTemp, Warning,
|
||||
TEXT("GigLine top surface under player: hit=%d Z=%.0f actor=%s (actor Z %.0f)"),
|
||||
bTop ? 1 : 0, bTop ? Top.ImpactPoint.Z : -9999.f,
|
||||
bTop && Top.GetActor() ? *Top.GetActor()->GetName() : TEXT("none"),
|
||||
GetActorLocation().Z);
|
||||
}
|
||||
|
||||
const FVector Feet = GetActorLocation() - FVector(0.f, 0.f, CapsuleHalf);
|
||||
for (int32 Index = 0; Index < 3; ++Index)
|
||||
{
|
||||
|
||||
@@ -26,7 +26,10 @@ Actionable task list for this project. This file is the single, central place fo
|
||||
- [x] Fix the character and car floating. **The measurement was taken at the wrong moment.** The foot probe ran in `BeginPlay`, while the pawn is still at its spawn height and falling, so every trace reported a surface far below it (`ground 366` against an actor at 561) and read as a float that was not there. The probe now runs **three seconds after spawn**, from the capsule's feet, on three channels, and names the actor it hits. Character: the player start at Z 600 left the soles 39 cm in the air (the capsule settles at 561 on this pavement); the start is now the measured 561. Car: the template's lowest geometry is exhaust, so grounding on the whole model put the tyres 4 cm above the origin - it is now grounded on its wheels, staged at Z 342 after the traced road height (366) disagreed with the visible road by ~25 cm.
|
||||
- [x] Fix the vehicle's material colour. The template's `car paint` is a legacy mix-shader graph that the FBX exporter cannot map, so `SM_Vehicle_Body` imported holding `WorldGridMaterial` and rendered black while Blender showed red. Legacy graphs are now converted to Principled before export (saturated colour wins over the grey diffuse next to it), and the staged body panel is assigned `M_Vehicle_HatchPaint`. Judge: `verdict=yes confidence=0.9`.
|
||||
- [x] Fix the missing vehicle glazing. The `windows` material was metallic 1.0 with roughness 0.0 - a black void that read as an empty cabin. Now tinted translucent (opacity 0.85, roughness 0.08); opacity 0.45 was tried first and made the glass invisible. The vision judge still returns `verdict=no` for the frame, disagreeing with what is visibly tinted glass; it is advisory, and the human verdict decides.
|
||||
- [A] Measure the shadow gap instead of arguing with the vision model. The judge now flags the character and car as floating in the spawn view, and its criterion is in the expectation ("a shadow more than a hand's width below an object means the object is floating"). Two readings are possible and both fit the pixels: the object really floats, or it stands on the raised pavement and the sun throws its shadow past the kerb onto the road. A shadow-gap measurement - lowest object pixels versus its shadow, in pixels - settles it; the judge cannot.
|
||||
- [A] Measure the shadow gap instead of arguing with the vision model. The judge now flags the character and car as floating in the spawn view, and its criterion is in the expectation ("a shadow more than a hand's width below an object means the object is floating"). Two readings are possible and both fit the pixels: the object really floats, or it stands on the raised pavement and the sun throws its shadow past the kerb onto the road. `Tools/shadow_gap.py` now does the measurement: object vs shadow in pixels, as a fraction of the object's height. It works on synthetic frames (tests pass) but returns `unknown` on the real spawn view, because the crop contains two surfaces - the pale wall behind and the dark road below - and a single global background cannot classify both. Next: per-region backgrounds, or crop tightly around the feet and their shadow only.
|
||||
|
||||
**The spawn height is measured and is not the problem:** tracing down from 30 m above the player, the topmost surface is Z 366 and the capsule's feet sit at 369 - three centimetres, which is the capsule skin. If the character still looks like it floats, the gap is between the **rendered** pavement and its **collision** surface, not the spawn Z. That is the thing to measure next: the sidewalk mesh's rendered top versus the collider returned by the trace.
|
||||
|
||||
- [ ] Implement enter/exit interaction.
|
||||
- [ ] Build a closed driving route with boundaries.
|
||||
- [ ] Tune basic handling to a drivable state.
|
||||
|
||||
@@ -99,6 +99,7 @@ Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side b
|
||||
| `vehicle_from_template.py` | Converts the owner's `hot hatch.blend` into UE-ready assets: normalises the scale to a documented 4.05 m, splits systems (body/trim/glass/lights/wheels), adds a `UCX_` hull, decimates two LODs, exports FBX per system and writes a report. | `--out`, `--report` | `blender -b "Art/blender/vehicle/hot hatch.blend" --python Tools/vehicle_from_template.py -- --out /tmp/gigline/hatch` |
|
||||
| `blender_screenshots.py` | Renders review screenshots out of a .blend: one per mesh object (`--mode objects`) and four turntable angles (`--mode scene`), framed from the scene's own bounds, into `Screenshots/blender/`. | `--label`, `--mode`, `--out`, `--report` | `blender -b "Art/blender/vehicle/hot hatch.blend" --python Tools/blender_screenshots.py -- --label vehicle --mode both` |
|
||||
| `build_comparison_md.py` | Regenerates the `Blender vs in-game` section of `Screenshots/README.md` between `<!-- blender-comparison:begin/end -->` markers: in-game captures beside their Blender renders in one table, plus a signoff table with one row per in-game screen. Pairs by name with an alias map for camera-named shots. Run it after every capture or Blender render. | (no arguments) | `python3 Tools/build_comparison_md.py` |
|
||||
| `shadow_gap.py` | Measures the vertical pixel gap between an object's lowest pixels and the top of its shadow, as a fraction of the object's height, and calls it `contact` or `floating`. Objective replacement for asking a vision model whether something floats. | `--image`, `--object-box`, `--gap-fraction`, `--out` | `python3 Tools/shadow_gap.py --image Screenshots/demo/review-demo-spawn-view.png --object-box 0.44,0.48,0.60,0.99` |
|
||||
|
||||
## Tests
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Measure how far an object sits above its own shadow.
|
||||
|
||||
python3 Tools/shadow_gap.py --image Screenshots/demo/review-demo-spawn-view.png \
|
||||
--object-box 0.52,0.55,0.60,0.75 --out /tmp/gigline/shadow-gap.json
|
||||
|
||||
Why a measurement instead of another opinion: a still cannot distinguish "the object floats"
|
||||
from "the object stands on a raised kerb and its shadow falls past it onto the road", and a
|
||||
vision model asked to arbitrate gives a different answer each run. The vertical pixel gap
|
||||
between the lowest object pixel and the top of its shadow is objective.
|
||||
|
||||
Method, deliberately simple and stated so it can be argued with:
|
||||
|
||||
1. Crop the region of interest (the object box, fractions of the frame).
|
||||
2. The object is the set of pixels that differ from the local background - the median colour of
|
||||
the crop's border.
|
||||
3. The shadow is the darkest connected region *below* the object.
|
||||
4. Report the gap in pixels between the object's lowest row and the shadow's top row, and the
|
||||
object's height in pixels, so the gap can be read as a fraction of the object.
|
||||
|
||||
A large gap relative to the object's height is the floating signal; a small gap is contact.
|
||||
The thresholds are arguments, not constants, because the right values depend on the camera.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
from PIL import Image
|
||||
|
||||
DEFAULT_GAP_FRACTION = 0.12 # gap > 12 % of the object's pixel height reads as floating
|
||||
SHADOW_DELTA = 38 # how much darker than the background a pixel must be to be shadow
|
||||
|
||||
|
||||
def border_median(image: Image.Image):
|
||||
"""Median colour of the crop's border, used as the local background."""
|
||||
pixels = image.load()
|
||||
width, height = image.size
|
||||
samples = []
|
||||
step = max(1, width // 60)
|
||||
for x in range(0, width, step):
|
||||
samples.append(pixels[x, 1])
|
||||
samples.append(pixels[x, height - 2])
|
||||
for y in range(0, height, max(1, height // 60)):
|
||||
samples.append(pixels[1, y])
|
||||
samples.append(pixels[width - 2, y])
|
||||
if not samples:
|
||||
return (128, 128, 128)
|
||||
channels = list(zip(*samples))
|
||||
return tuple(sorted(channel)[len(channel) // 2] for channel in channels)
|
||||
|
||||
|
||||
def classify_pixels(image: Image.Image, background, object_delta=18):
|
||||
"""Return two masks: the object (differs from background) and the shadow (darker)."""
|
||||
pixels = image.load()
|
||||
width, height = image.size
|
||||
object_rows = [0] * height
|
||||
shadow_rows = [0] * height
|
||||
for y in range(height):
|
||||
for x in range(width):
|
||||
r, g, b = pixels[x, y]
|
||||
delta = max(abs(r - background[0]), abs(g - background[1]), abs(b - background[2]))
|
||||
luminance = 0.2126 * r + 0.7152 * g + 0.0722 * b
|
||||
background_luminance = (0.2126 * background[0] + 0.7152 * background[1]
|
||||
+ 0.0722 * background[2])
|
||||
# A shadow is a darkened version of the surface; an object is much darker or a
|
||||
# different colour. Classifying by absolute delta alone put every shadow in the
|
||||
# object mask, which is why the first runs reported "no shadow found".
|
||||
ratio = luminance / max(1.0, background_luminance)
|
||||
# A shadow darkens every channel by the same proportion; an object does not.
|
||||
# Comparing the per-channel scale factors is what separates them - an absolute
|
||||
# delta cannot, because a shadow's delta is as large as a dark object's.
|
||||
scales = [rgb / max(1.0, base) for rgb, base in zip((r, g, b), background)]
|
||||
spread = max(scales) - min(scales)
|
||||
if 0.45 <= ratio < 0.90 and spread < 0.15:
|
||||
shadow_rows[y] += 1
|
||||
elif ratio < 0.45 or delta > object_delta:
|
||||
object_rows[y] += 1
|
||||
return object_rows, shadow_rows
|
||||
|
||||
|
||||
def measure(image_path: str, box, gap_fraction=DEFAULT_GAP_FRACTION) -> dict:
|
||||
image = Image.open(image_path).convert("RGB")
|
||||
width, height = image.size
|
||||
x0, y0, x1, y1 = (int(box[0] * width), int(box[1] * height),
|
||||
int(box[2] * width), int(box[3] * height))
|
||||
crop = image.crop((x0, y0, x1, y1))
|
||||
background = border_median(crop)
|
||||
object_rows, shadow_rows = classify_pixels(crop, background)
|
||||
|
||||
object_rows_nonzero = [y for y, count in enumerate(object_rows) if count > 4]
|
||||
if not object_rows_nonzero:
|
||||
return {"error": "no object found in the box", "background": background}
|
||||
object_top, object_bottom = object_rows_nonzero[0], object_rows_nonzero[-1]
|
||||
object_height = max(1, object_bottom - object_top)
|
||||
|
||||
# Shadow rows below the object's lowest row.
|
||||
shadow_below = [y for y, count in enumerate(shadow_rows)
|
||||
if y > object_bottom and count > 4]
|
||||
shadow_top = shadow_below[0] if shadow_below else None
|
||||
gap = None if shadow_top is None else shadow_top - object_bottom
|
||||
|
||||
verdict = "unknown"
|
||||
if gap is not None:
|
||||
verdict = "floating" if gap > object_height * gap_fraction else "contact"
|
||||
return {
|
||||
"image": os.path.abspath(image_path),
|
||||
"box_pixels": [x0, y0, x1, y1],
|
||||
"background": list(background),
|
||||
"object_top_row": object_top,
|
||||
"object_bottom_row": object_bottom,
|
||||
"object_height_px": object_height,
|
||||
"shadow_top_row": shadow_top,
|
||||
"gap_px": gap,
|
||||
"gap_fraction_of_object": None if gap is None else round(gap / object_height, 3),
|
||||
"threshold_fraction": gap_fraction,
|
||||
"verdict": verdict,
|
||||
}
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--image", required=True)
|
||||
ap.add_argument("--object-box", required=True,
|
||||
help="x0,y0,x1,y1 as fractions of the frame around the object and its shadow")
|
||||
ap.add_argument("--gap-fraction", type=float, default=DEFAULT_GAP_FRACTION,
|
||||
help="gap/height above this reads as floating (default 0.12)")
|
||||
ap.add_argument("--out", default="/tmp/gigline/shadow-gap.json")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
box = [float(v) for v in args.object_box.split(",")]
|
||||
if len(box) != 4:
|
||||
print("error: --object-box needs four comma-separated fractions", file=sys.stderr)
|
||||
return 2
|
||||
if not os.path.exists(args.image):
|
||||
print(f"error: no such image: {args.image}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
result = measure(args.image, box, args.gap_fraction)
|
||||
os.makedirs(os.path.dirname(os.path.abspath(args.out)) or ".", exist_ok=True)
|
||||
with open(args.out, "w") as handle:
|
||||
json.dump(result, handle, indent=2)
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0 if result.get("verdict") != "unknown" else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,56 @@
|
||||
"""Tests for the pure parts of Tools/shadow_gap.py."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
import shadow_gap as sg # noqa: E402
|
||||
|
||||
|
||||
def synthetic(object_bottom, shadow_top, size=(80, 120), background=(200, 200, 196)):
|
||||
"""A grey frame with a dark block (object) and a dark strip (shadow) below it."""
|
||||
image = Image.new("RGB", size, background)
|
||||
pixels = image.load()
|
||||
for y in range(20, object_bottom):
|
||||
for x in range(30, 50):
|
||||
pixels[x, y] = (18, 18, 20) # object: far darker than the surface
|
||||
for y in range(shadow_top, min(size[1] - 2, shadow_top + 12)):
|
||||
for x in range(30, 50):
|
||||
# shadow: a darkened version of the surface, as a real shadow is
|
||||
pixels[x, y] = (140, 140, 138)
|
||||
return image
|
||||
|
||||
|
||||
class MeasureTests(unittest.TestCase):
|
||||
def test_contact_reads_as_contact(self):
|
||||
image = synthetic(object_bottom=60, shadow_top=61)
|
||||
image.save("/tmp/gigline-shadow-contact.png")
|
||||
result = sg.measure("/tmp/gigline-shadow-contact.png", (0.2, 0.05, 0.8, 0.95))
|
||||
self.assertEqual(result["verdict"], "contact")
|
||||
self.assertLessEqual(result["gap_px"], 2)
|
||||
|
||||
def test_a_gap_reads_as_floating(self):
|
||||
image = synthetic(object_bottom=60, shadow_top=100)
|
||||
image.save("/tmp/gigline-shadow-float.png")
|
||||
result = sg.measure("/tmp/gigline-shadow-float.png", (0.2, 0.05, 0.8, 0.95))
|
||||
self.assertEqual(result["verdict"], "floating")
|
||||
self.assertGreater(result["gap_px"], result["object_height_px"] * 0.12)
|
||||
|
||||
def test_empty_box_reports_an_error(self):
|
||||
image = Image.new("RGB", (40, 40), (200, 200, 200))
|
||||
image.save("/tmp/gigline-shadow-empty.png")
|
||||
result = sg.measure("/tmp/gigline-shadow-empty.png", (0.1, 0.1, 0.9, 0.9))
|
||||
self.assertIn("error", result)
|
||||
|
||||
|
||||
class BorderTests(unittest.TestCase):
|
||||
def test_border_median_is_the_frame_colour(self):
|
||||
image = Image.new("RGB", (40, 40), (180, 190, 200))
|
||||
self.assertEqual(sg.border_median(image), (180, 190, 200))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user