zwitschi/security-game
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Make the vision judge discover cost-effective OpenRouter models
Full build / Import + headless tests (push) Successful in 40s
Details
Full build / Full desktop build + optional release (push) Successful in 4m6s
Details

Browse Source 
Add openrouter_judge.py to rank live image+JSON models by advertised price,
with ordered fallbacks, auth-error handling and auto selection. Wire both judges
to --model auto, record the model used, and add fake-catalogue/mocked-call
tests. Re-run the judges with the valid key: 13/13 screen designs match, 0 diff;
the Forward+ A/B report records the selected model and ~$0.006 spend. Headless
231/231, Python 94/94.
This commit is contained in:
zwitschi committed 2026-11-01 19:55:43 +01:00
1 parent 28f2a48811
commit 659b0e35b2
15 files changed
+535 -65
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/screens/realism/2026-11-01-forward-plus/report.json
+156 -6
View File
@@ -1,6 +1,14 @@
{
"run": "docs/screens/realism/2026-11-01-forward-plus-eval",
"judge_model": null,
"run": "docs/screens/realism/2026-11-01-forward-plus",
"judge_model": "z-ai/glm-5.3-flash",
"judge_fallbacks": [
"z-ai/glm-5.3-flash",
"qwen/qwen3.7-flash",
"google/gemma-3-4b-it",
"google/gemma-3-12b-it",
"inclusionai/ling-3.0-flash-vl",
"prism-ml/ternary-bonsai-2-9b"
],
"sets": {
"baseline": {
"entrance_late": {
@@ -19,7 +27,52 @@
"saturation_mean": 159.44,
"colour_spread": 26.61
},
"metric_gap": 1.568
"metric_gap": 1.568,
"judge": {
"scores": {
"characters": 4.0,
"overall": 5.0,
"lighting": 5.0,
"texture_detail": 6.0,
"materials": 6.0,
"backdrop": 5.0
},
"score_range": {
"characters": [
4.0,
4.0
],
"overall": [
5.0,
5.0
],
"lighting": [
5.0,
5.0
],
"texture_detail": [
6.0,
6.0
],
"materials": [
6.0,
6.0
],
"backdrop": [
5.0,
5.0
]
},
"runs": 1,
"model": "z-ai/glm-5.3-flash",
"fixes": [
"Increase contrast and deepen shadows across the entire scene, especially in the foreground and under the building overhang, to achieve a more cinematic mood.",
"Boost the reflectivity and wetness of the pavement to create dramatic, high-contrast reflections of the streetlights and sign.",
"Increase the density and volumetric scattering of the atmospheric haze/fog to enhance the sense of depth and distance, matching the target's mood.",
"Improve the character's silhouette and integration by adding strong rim lighting or darkening the model to make it feel more like a dramatic figure rather than a visible asset.",
"Add more micro-detail, grime, and color variation to the brick and concrete materials to make them feel less uniform and more weathered."
]
}
},
"street_late": {
"target": "docs/screens/concepts/boulevard_skyline_dusk.png",
@@ -37,7 +90,14 @@
"saturation_mean": 188.16,
"colour_spread": 35.38
},
"metric_gap": 0.247
"metric_gap": 0.247,
"judge": {
"scores": {},
"score_range": {},
"runs": 1,
"model": "z-ai/glm-5.3-flash",
"fixes": []
}
}
},
"candidate": {
@@ -57,7 +117,52 @@
"saturation_mean": 159.44,
"colour_spread": 26.61
},
"metric_gap": 1.187
"metric_gap": 1.187,
"judge": {
"scores": {
"characters": 5.0,
"overall": 5.0,
"lighting": 5.0,
"texture_detail": 6.0,
"materials": 6.0,
"backdrop": 4.0
},
"score_range": {
"characters": [
5.0,
5.0
],
"overall": [
5.0,
5.0
],
"lighting": [
5.0,
5.0
],
"texture_detail": [
6.0,
6.0
],
"materials": [
6.0,
6.0
],
"backdrop": [
4.0,
4.0
]
},
"runs": 1,
"model": "z-ai/glm-5.3-flash",
"fixes": [
"Implement volumetric fog and heavy atmospheric haze to increase depth and match the target's moody atmosphere.",
"Increase ground material specularity and darken the asphalt to enhance the wet, reflective look.",
"Adjust global illumination and ambient light to increase contrast, allowing for deeper shadows and more dramatic highlights (e.g., on the sign).",
"Apply rim lighting to the character to integrate them into the scene's dramatic lighting scheme.",
"Enhance the color grading towards a high-contrast, desaturated, warm-yellow/orange cinematic palette."
]
}
},
"street_late": {
"target": "docs/screens/concepts/boulevard_skyline_dusk.png",
@@ -75,7 +180,52 @@
"saturation_mean": 188.16,
"colour_spread": 35.38
},
"metric_gap": 0.258
"metric_gap": 0.258,
"judge": {
"scores": {
"characters": 3.0,
"overall": 4.0,
"lighting": 4.0,
"texture_detail": 3.0,
"materials": 3.0,
"backdrop": 5.0
},
"score_range": {
"characters": [
3.0,
3.0
],
"overall": [
4.0,
4.0
],
"lighting": [
4.0,
4.0
],
"texture_detail": [
3.0,
3.0
],
"materials": [
3.0,
3.0
],
"backdrop": [
5.0,
5.0
]
},
"runs": 1,
"model": "z-ai/glm-5.3-flash",
"fixes": [
"Implement advanced volumetric lighting and bloom effects on street lamps to achieve the dramatic contrast seen in the target.",
"Upgrade material shaders (PBR) for primary surfaces (glass, concrete) to increase definition and reduce the current muddy, low-resolution appearance.",
"Enhance atmospheric scattering and haze in the distance to create greater depth and match the cinematic quality of the target's skyline.",
"Increase the resolution and detail of foreground assets (building facades, road textures) to move away from the current blocky look.",
"Refine the sky's color gradient and lighting to better capture the specific warm, dramatic twilight palette of the target."
]
}
}
}
}