Refactor load testing scripts to support Bash and PowerShell
CI / test (push) Successful in 2m50s
CI / docker (push) Successful in 5m12s

- Updated README.md to reflect changes in load testing commands and added usage examples for Bash scripts.
- Converted PowerShell load testing scripts to Bash equivalents for better cross-platform compatibility.
- Added new Bash scripts: run-loadtests.sh, summarize-artillery.sh, and summarize-k6.sh to mirror existing PowerShell functionality.
- Updated admin-api.js and ws-chat.yml for consistency and improved performance metrics reporting.
- Enhanced error handling and output formatting in summary scripts for better readability.
This commit is contained in:
zwitschi committed 2026-09-02 19:47:34 +02:00
1 parent 9f60087a4c
commit 33b6296c5e
82 files changed
+19682 -21133

No files matched your search

+20 -49
View File
@@ -2,66 +2,42 @@
These files back the [scaling/performance playbook](../../docs/operations/scaling-performance-playbook.md).
## WebSocket chat
## WebSocket chat (artillery)
Run with an explicit WebSocket URL, for example:
Run with an explicit WebSocket URL, capturing output to an artifact file:
```bash
WS_URL=wss://linearvoid.tv/ws?username=loadtest&room=main npx artillery run scripts/loadtest/ws-chat.yml
```
Capture output to an artifact file for the worksheet:
```bash
WS_URL=wss://linearvoid.tv/ws?username=loadtest&room=main npx artillery run scripts/loadtest/ws-chat.yml --output artifacts/ws-chat-report.json
```
PowerShell equivalent:
```powershell
$env:WS_URL = 'wss://linearvoid.tv/ws?username=loadtest&room=main'
New-Item -ItemType Directory -Force artifacts | Out-Null
export WS_URL='wss://linearvoid.tv/ws?username=loadtest&room=main'
mkdir -p artifacts
npx artillery run scripts/loadtest/ws-chat.yml --output artifacts/ws-chat-report.json
```
Summarize key counters, rates, and available percentile stats from a saved Artillery JSON report:
```powershell
pwsh -File scripts/loadtest/summarize-artillery.ps1 -ReportFile artifacts/ws-chat-report.json
```bash
scripts/loadtest/summarize-artillery.sh artifacts/ws-chat-report.json
```
## Admin API
## Admin API (k6)
Run with a cookie or session header that can access admin endpoints:
Run with a cookie or session header that can access admin endpoints, capturing machine-readable metrics and a text summary:
```bash
ADMIN_BASE_URL=https://linearvoid.tv ADMIN_SESSION_COOKIE="session=..." k6 run scripts/loadtest/admin-api.js
```
Capture machine-readable metrics and a text summary:
```bash
ADMIN_BASE_URL=https://linearvoid.tv ADMIN_SESSION_COOKIE="session=..." k6 run --summary-export artifacts/admin-api-summary.json scripts/loadtest/admin-api.js > artifacts/admin-api-output.txt
```
PowerShell equivalent:
```powershell
$env:ADMIN_BASE_URL = 'https://linearvoid.tv'
$env:ADMIN_SESSION_COOKIE = 'session=...'
New-Item -ItemType Directory -Force artifacts | Out-Null
k6 run --summary-export artifacts/admin-api-summary.json scripts/loadtest/admin-api.js | Tee-Object -FilePath artifacts/admin-api-output.txt
export ADMIN_BASE_URL='https://linearvoid.tv'
export ADMIN_SESSION_COOKIE='session=...'
mkdir -p artifacts
k6 run --summary-export artifacts/admin-api-summary.json scripts/loadtest/admin-api.js | tee artifacts/admin-api-output.txt
```
The script emits global and per-endpoint thresholds (overview, analytics, chat-log, users) so p95 and error-rate regressions are visible per route.
Summarize endpoint p95 and failure rates from a saved k6 JSON summary:
```powershell
pwsh -File scripts/loadtest/summarize-k6.ps1 -SummaryFile artifacts/admin-api-summary.json
```bash
scripts/loadtest/summarize-k6.sh artifacts/admin-api-summary.json
```
## PowerShell helper
## Runner helper
Use the helper to run one scenario or both and automatically write outputs into a dated artifact folder:
@@ -69,16 +45,11 @@ Use the helper to run one scenario or both and automatically write outputs into
- WebSocket runs also emit `ws-chat-summary.txt` with connection/error counters and available summary percentiles.
- Admin runs also emit `admin-api-endpoint-summary.txt` with per-endpoint p95/error values.
```powershell
pwsh -File scripts/loadtest/run-loadtests.ps1 -Scenario ws -WsUrl 'wss://linearvoid.tv/ws?username=loadtest&room=main'
```
```powershell
pwsh -File scripts/loadtest/run-loadtests.ps1 -Scenario admin -AdminBaseUrl 'https://linearvoid.tv' -AdminSessionCookie 'session=...'
```
```powershell
pwsh -File scripts/loadtest/run-loadtests.ps1 -Scenario all -WsUrl 'wss://linearvoid.tv/ws?username=loadtest&room=main' -AdminBaseUrl 'https://linearvoid.tv' -AdminSessionCookie 'session=...'
```bash
scripts/loadtest/run-loadtests.sh -s ws -w 'wss://linearvoid.tv/ws?username=loadtest&room=main'
scripts/loadtest/run-loadtests.sh -s admin -a 'https://linearvoid.tv' -c 'session=...'
scripts/loadtest/run-loadtests.sh -s all -w 'wss://linearvoid.tv/ws?username=loadtest&room=main' \
-a 'https://linearvoid.tv' -c 'session=...'
```
## Notes
+58 -58
View File
@@ -1,58 +1,58 @@
import http from "k6/http";
import { check, sleep } from "k6";
const baseUrl = __ENV.ADMIN_BASE_URL || "https://linearvoid.tv";
const adminCookie = __ENV.ADMIN_SESSION_COOKIE || "";
const requestParams = adminCookie ? { headers: { Cookie: adminCookie } } : {};
const endpoints = [
{ name: "overview", path: "/api/admin/overview" },
{ name: "analytics", path: "/api/admin/analytics" },
{ name: "chat_log", path: "/api/admin/chat-log" },
{ name: "users", path: "/api/admin/users" },
];
export const options = {
stages: [
{ duration: "30s", target: 10 },
{ duration: "60s", target: 50 },
{ duration: "60s", target: 100 },
{ duration: "30s", target: 0 },
],
thresholds: {
http_req_duration: ["p(95)<1000"],
http_req_failed: ["rate<0.01"],
"http_req_duration{endpoint:overview}": ["p(95)<1000"],
"http_req_duration{endpoint:analytics}": ["p(95)<1000"],
"http_req_duration{endpoint:chat_log}": ["p(95)<1000"],
"http_req_duration{endpoint:users}": ["p(95)<1000"],
"http_req_failed{endpoint:overview}": ["rate<0.01"],
"http_req_failed{endpoint:analytics}": ["rate<0.01"],
"http_req_failed{endpoint:chat_log}": ["rate<0.01"],
"http_req_failed{endpoint:users}": ["rate<0.01"],
},
};
export default function () {
const requests = endpoints.map((endpoint) => [
"GET",
`${baseUrl}${endpoint.path}`,
null,
{
...requestParams,
tags: {
endpoint: endpoint.name,
},
},
]);
const responses = http.batch(requests);
responses.forEach((response, index) => {
const endpointPath = endpoints[index].path;
check(response, {
[`status is 200 for ${endpointPath}`]: (r) => r.status === 200,
});
});
sleep(1);
}
import http from "k6/http";
import { check, sleep } from "k6";
const baseUrl = __ENV.ADMIN_BASE_URL || "https://linearvoid.tv";
const adminCookie = __ENV.ADMIN_SESSION_COOKIE || "";
const requestParams = adminCookie ? { headers: { Cookie: adminCookie } } : {};
const endpoints = [
{ name: "overview", path: "/api/admin/overview" },
{ name: "analytics", path: "/api/admin/analytics" },
{ name: "chat_log", path: "/api/admin/chat-log" },
{ name: "users", path: "/api/admin/users" },
];
export const options = {
stages: [
{ duration: "30s", target: 10 },
{ duration: "60s", target: 50 },
{ duration: "60s", target: 100 },
{ duration: "30s", target: 0 },
],
thresholds: {
http_req_duration: ["p(95)<1000"],
http_req_failed: ["rate<0.01"],
"http_req_duration{endpoint:overview}": ["p(95)<1000"],
"http_req_duration{endpoint:analytics}": ["p(95)<1000"],
"http_req_duration{endpoint:chat_log}": ["p(95)<1000"],
"http_req_duration{endpoint:users}": ["p(95)<1000"],
"http_req_failed{endpoint:overview}": ["rate<0.01"],
"http_req_failed{endpoint:analytics}": ["rate<0.01"],
"http_req_failed{endpoint:chat_log}": ["rate<0.01"],
"http_req_failed{endpoint:users}": ["rate<0.01"],
},
};
export default function () {
const requests = endpoints.map((endpoint) => [
"GET",
`${baseUrl}${endpoint.path}`,
null,
{
...requestParams,
tags: {
endpoint: endpoint.name,
},
},
]);
const responses = http.batch(requests);
responses.forEach((response, index) => {
const endpointPath = endpoints[index].path;
check(response, {
[`status is 200 for ${endpointPath}`]: (r) => r.status === 200,
});
});
sleep(1);
}
+98 -98
View File
@@ -1,99 +1,99 @@
#!/usr/bin/env pwsh
param(
[ValidateSet('ws', 'admin', 'all')]
[string]$Scenario = 'all',
[string]$ArtifactRoot = 'artifacts',
[string]$WsUrl = $env:WS_URL,
[string]$AdminBaseUrl = $env:ADMIN_BASE_URL,
[string]$AdminSessionCookie = $env:ADMIN_SESSION_COOKIE
)
$ErrorActionPreference = 'Stop'
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$repoRoot = Resolve-Path (Join-Path $scriptDir '..\..')
$wsScenarioPath = Join-Path $scriptDir 'ws-chat.yml'
$wsSummaryScriptPath = Join-Path $scriptDir 'summarize-artillery.ps1'
$adminScriptPath = Join-Path $scriptDir 'admin-api.js'
$adminSummaryScriptPath = Join-Path $scriptDir 'summarize-k6.ps1'
if ([System.IO.Path]::IsPathRooted($ArtifactRoot)) {
$artifactBase = $ArtifactRoot
} else {
$artifactBase = Join-Path $repoRoot $ArtifactRoot
}
if ($Scenario -in @('ws', 'all')) {
if (-not (Get-Command npx -ErrorAction SilentlyContinue)) {
throw 'npx is required for ws scenario but was not found in PATH.'
}
}
if ($Scenario -in @('admin', 'all')) {
if (-not (Get-Command k6 -ErrorAction SilentlyContinue)) {
throw 'k6 is required for admin scenario but was not found in PATH.'
}
}
$timestamp = Get-Date -Format 'yyyy-MM-dd_HHmmss'
$artifactDir = Join-Path $artifactBase $timestamp
New-Item -ItemType Directory -Force -Path $artifactDir | Out-Null
function Invoke-WebSocketLoadTest {
param(
[string]$TargetWsUrl,
[string]$OutputPath
)
if ([string]::IsNullOrWhiteSpace($TargetWsUrl)) {
throw 'WS_URL is required for ws scenario. Pass -WsUrl or set $env:WS_URL.'
}
$env:WS_URL = $TargetWsUrl
& npx artillery run $wsScenarioPath --output $OutputPath
}
function Invoke-AdminApiLoadTest {
param(
[string]$BaseUrl,
[string]$SessionCookie,
[string]$SummaryPath,
[string]$OutputPath
)
if ([string]::IsNullOrWhiteSpace($BaseUrl)) {
throw 'ADMIN_BASE_URL is required for admin scenario. Pass -AdminBaseUrl or set $env:ADMIN_BASE_URL.'
}
if ([string]::IsNullOrWhiteSpace($SessionCookie)) {
throw 'ADMIN_SESSION_COOKIE is required for admin scenario. Pass -AdminSessionCookie or set $env:ADMIN_SESSION_COOKIE.'
}
$env:ADMIN_BASE_URL = $BaseUrl
$env:ADMIN_SESSION_COOKIE = $SessionCookie
& k6 run --summary-export $SummaryPath $adminScriptPath | Tee-Object -FilePath $OutputPath
}
if ($Scenario -in @('ws', 'all')) {
if (-not (Test-Path $wsSummaryScriptPath)) {
throw "Missing summary helper: $wsSummaryScriptPath"
}
$wsOutput = Join-Path $artifactDir 'ws-chat-report.json'
$wsSummary = Join-Path $artifactDir 'ws-chat-summary.txt'
Invoke-WebSocketLoadTest -TargetWsUrl $WsUrl -OutputPath $wsOutput
& pwsh -File $wsSummaryScriptPath -ReportFile $wsOutput | Tee-Object -FilePath $wsSummary
}
if ($Scenario -in @('admin', 'all')) {
if (-not (Test-Path $adminSummaryScriptPath)) {
throw "Missing summary helper: $adminSummaryScriptPath"
}
$adminSummary = Join-Path $artifactDir 'admin-api-summary.json'
$adminOutput = Join-Path $artifactDir 'admin-api-output.txt'
$adminEndpointSummary = Join-Path $artifactDir 'admin-api-endpoint-summary.txt'
Invoke-AdminApiLoadTest -BaseUrl $AdminBaseUrl -SessionCookie $AdminSessionCookie -SummaryPath $adminSummary -OutputPath $adminOutput
& pwsh -File $adminSummaryScriptPath -SummaryFile $adminSummary | Tee-Object -FilePath $adminEndpointSummary
}
#!/usr/bin/env pwsh
param(
[ValidateSet('ws', 'admin', 'all')]
[string]$Scenario = 'all',
[string]$ArtifactRoot = 'artifacts',
[string]$WsUrl = $env:WS_URL,
[string]$AdminBaseUrl = $env:ADMIN_BASE_URL,
[string]$AdminSessionCookie = $env:ADMIN_SESSION_COOKIE
)
$ErrorActionPreference = 'Stop'
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$repoRoot = Resolve-Path (Join-Path $scriptDir '..\..')
$wsScenarioPath = Join-Path $scriptDir 'ws-chat.yml'
$wsSummaryScriptPath = Join-Path $scriptDir 'summarize-artillery.ps1'
$adminScriptPath = Join-Path $scriptDir 'admin-api.js'
$adminSummaryScriptPath = Join-Path $scriptDir 'summarize-k6.ps1'
if ([System.IO.Path]::IsPathRooted($ArtifactRoot)) {
$artifactBase = $ArtifactRoot
} else {
$artifactBase = Join-Path $repoRoot $ArtifactRoot
}
if ($Scenario -in @('ws', 'all')) {
if (-not (Get-Command npx -ErrorAction SilentlyContinue)) {
throw 'npx is required for ws scenario but was not found in PATH.'
}
}
if ($Scenario -in @('admin', 'all')) {
if (-not (Get-Command k6 -ErrorAction SilentlyContinue)) {
throw 'k6 is required for admin scenario but was not found in PATH.'
}
}
$timestamp = Get-Date -Format 'yyyy-MM-dd_HHmmss'
$artifactDir = Join-Path $artifactBase $timestamp
New-Item -ItemType Directory -Force -Path $artifactDir | Out-Null
function Invoke-WebSocketLoadTest {
param(
[string]$TargetWsUrl,
[string]$OutputPath
)
if ([string]::IsNullOrWhiteSpace($TargetWsUrl)) {
throw 'WS_URL is required for ws scenario. Pass -WsUrl or set $env:WS_URL.'
}
$env:WS_URL = $TargetWsUrl
& npx artillery run $wsScenarioPath --output $OutputPath
}
function Invoke-AdminApiLoadTest {
param(
[string]$BaseUrl,
[string]$SessionCookie,
[string]$SummaryPath,
[string]$OutputPath
)
if ([string]::IsNullOrWhiteSpace($BaseUrl)) {
throw 'ADMIN_BASE_URL is required for admin scenario. Pass -AdminBaseUrl or set $env:ADMIN_BASE_URL.'
}
if ([string]::IsNullOrWhiteSpace($SessionCookie)) {
throw 'ADMIN_SESSION_COOKIE is required for admin scenario. Pass -AdminSessionCookie or set $env:ADMIN_SESSION_COOKIE.'
}
$env:ADMIN_BASE_URL = $BaseUrl
$env:ADMIN_SESSION_COOKIE = $SessionCookie
& k6 run --summary-export $SummaryPath $adminScriptPath | Tee-Object -FilePath $OutputPath
}
if ($Scenario -in @('ws', 'all')) {
if (-not (Test-Path $wsSummaryScriptPath)) {
throw "Missing summary helper: $wsSummaryScriptPath"
}
$wsOutput = Join-Path $artifactDir 'ws-chat-report.json'
$wsSummary = Join-Path $artifactDir 'ws-chat-summary.txt'
Invoke-WebSocketLoadTest -TargetWsUrl $WsUrl -OutputPath $wsOutput
& pwsh -File $wsSummaryScriptPath -ReportFile $wsOutput | Tee-Object -FilePath $wsSummary
}
if ($Scenario -in @('admin', 'all')) {
if (-not (Test-Path $adminSummaryScriptPath)) {
throw "Missing summary helper: $adminSummaryScriptPath"
}
$adminSummary = Join-Path $artifactDir 'admin-api-summary.json'
$adminOutput = Join-Path $artifactDir 'admin-api-output.txt'
$adminEndpointSummary = Join-Path $artifactDir 'admin-api-endpoint-summary.txt'
Invoke-AdminApiLoadTest -BaseUrl $AdminBaseUrl -SessionCookie $AdminSessionCookie -SummaryPath $adminSummary -OutputPath $adminOutput
& pwsh -File $adminSummaryScriptPath -SummaryFile $adminSummary | Tee-Object -FilePath $adminEndpointSummary
}
Write-Host "Artifacts written to $artifactDir"
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env bash
# run-loadtests.sh — Run an artillery (ws) and/or k6 (admin) load-test scenario
# and write outputs into a dated artifacts folder.
# Mirrors scripts/loadtest/run-loadtests.ps1.
#
# Usage:
# ./scripts/loadtest/run-loadtests.sh -s ws -w 'wss://...'
# ./scripts/loadtest/run-loadtests.sh -s admin -a 'https://...' -c 'session=...'
# ./scripts/loadtest/run-loadtests.sh -s all -w 'wss://...' -a 'https://...' -c 'session=...'
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_root="$(cd "$script_dir/../.." && pwd)"
SCENARIO="all"
ARTIFACT_ROOT="artifacts"
WS_URL="${WS_URL:-}"
ADMIN_BASE_URL="${ADMIN_BASE_URL:-}"
ADMIN_SESSION_COOKIE="${ADMIN_SESSION_COOKIE:-}"
usage() {
echo "Usage: $0 [-s ws|admin|all] [-w ws_url] [-a admin_base_url] [-c admin_session_cookie] [-r artifact_root]" >&2
exit 2
}
while [[ $# -gt 0 ]]; do
case "$1" in
-s) SCENARIO="$2"; shift 2;;
-w) WS_URL="$2"; shift 2;;
-a) ADMIN_BASE_URL="$2"; shift 2;;
-c) ADMIN_SESSION_COOKIE="$2"; shift 2;;
-r) ARTIFACT_ROOT="$2"; shift 2;;
-h|--help) usage;;
*) echo "Unknown option: $1" >&2; usage;;
esac
done
case "$SCENARIO" in ws|admin|all) ;; *) echo "Scenario must be ws|admin|all" >&2; exit 2;; esac
if [[ "$SCENARIO" == ws || "$SCENARIO" == all ]] && ! command -v npx >/dev/null 2>&1; then
echo "npx is required for ws scenario but was not found in PATH." >&2; exit 1
fi
if [[ "$SCENARIO" == admin || "$SCENARIO" == all ]] && ! command -v k6 >/dev/null 2>&1; then
echo "k6 is required for admin scenario but was not found in PATH." >&2; exit 1
fi
if [[ "$ARTIFACT_ROOT" = /* ]]; then artifact_base="$ARTIFACT_ROOT"; else artifact_base="$repo_root/$ARTIFACT_ROOT"; fi
artifact_dir="$artifact_base/$(date +%Y-%m-%d_%H%M%S)"
mkdir -p "$artifact_dir"
if [[ "$SCENARIO" == ws || "$SCENARIO" == all ]]; then
[[ -n "$WS_URL" ]] || { echo "WS_URL is required for ws scenario. Pass -w or set \$WS_URL." >&2; exit 1; }
ws_out="$artifact_dir/ws-chat-report.json"
ws_sum="$artifact_dir/ws-chat-summary.txt"
echo "== Running artillery ws scenario =="
WS_URL="$WS_URL" npx artillery run "$script_dir/ws-chat.yml" --output "$ws_out"
"$script_dir/summarize-artillery.sh" "$ws_out" | tee "$ws_sum"
fi
if [[ "$SCENARIO" == admin || "$SCENARIO" == all ]]; then
[[ -n "$ADMIN_BASE_URL" ]] || { echo "ADMIN_BASE_URL is required for admin scenario. Pass -a or set \$ADMIN_BASE_URL." >&2; exit 1; }
[[ -n "$ADMIN_SESSION_COOKIE" ]] || { echo "ADMIN_SESSION_COOKIE is required for admin scenario. Pass -c or set \$ADMIN_SESSION_COOKIE." >&2; exit 1; }
admin_sum="$artifact_dir/admin-api-summary.json"
admin_out="$artifact_dir/admin-api-output.txt"
admin_ep_sum="$artifact_dir/admin-api-endpoint-summary.txt"
echo "== Running k6 admin scenario =="
ADMIN_BASE_URL="$ADMIN_BASE_URL" ADMIN_SESSION_COOKIE="$ADMIN_SESSION_COOKIE" \
k6 run --summary-export "$admin_sum" "$script_dir/admin-api.js" | tee "$admin_out"
"$script_dir/summarize-k6.sh" "$admin_sum" | tee "$admin_ep_sum"
fi
echo "Artifacts written to $artifact_dir"
+115 -115
View File
@@ -1,115 +1,115 @@
#!/usr/bin/env pwsh
param(
[Parameter(Mandatory = $true)]
[string]$ReportFile
)
$ErrorActionPreference = 'Stop'
if (-not (Test-Path $ReportFile)) {
throw "Report file not found: $ReportFile"
}
$report = Get-Content -Raw -Path $ReportFile | ConvertFrom-Json -Depth 100
$aggregate = $report.aggregate
if (-not $aggregate) {
throw 'Artillery report is missing aggregate data.'
}
function Get-PropertyValue {
param(
[Parameter(Mandatory = $true)]
[object]$Object,
[Parameter(Mandatory = $true)]
[string]$Name
)
if (-not $Object) {
return $null
}
$property = $Object.PSObject.Properties[$Name]
if ($property) {
return $property.Value
}
return $null
}
function Format-StatLine {
param(
[string]$Prefix,
[object]$Stats
)
if (-not $Stats) {
return $null
}
$parts = @()
foreach ($name in @('count', 'min', 'max', 'mean', 'median', 'p95', 'p99')) {
$value = Get-PropertyValue -Object $Stats -Name $name
if ($null -ne $value) {
$parts += "$name=$value"
}
}
if ($parts.Count -eq 0) {
return $null
}
return "$Prefix: $($parts -join ', ')"
}
$counters = Get-PropertyValue -Object $aggregate -Name 'counters'
$rates = Get-PropertyValue -Object $aggregate -Name 'rates'
$summaries = Get-PropertyValue -Object $aggregate -Name 'summaries'
$created = [double](Get-PropertyValue -Object $counters -Name 'vusers.created' ?? 0)
$failed = [double](Get-PropertyValue -Object $counters -Name 'vusers.failed' ?? 0)
$completed = [double](Get-PropertyValue -Object $counters -Name 'vusers.completed' ?? 0)
$errorRate = if ($created -gt 0) { [math]::Round(($failed / $created) * 100, 2) } else { 0 }
Write-Output '=== Artillery WebSocket Summary ==='
Write-Output "source: $ReportFile"
$timestamp = Get-PropertyValue -Object $report -Name 'timestamp'
if ($timestamp) {
Write-Output "timestamp: $timestamp"
}
if ($null -ne $created) {
Write-Output "vusers.created: $created"
}
if ($null -ne $completed) {
Write-Output "vusers.completed: $completed"
}
if ($null -ne $failed) {
Write-Output "vusers.failed: $failed"
}
Write-Output "vusers.failure_rate_percent: $errorRate"
if ($counters) {
foreach ($metric in @('ws.messages_sent', 'ws.messages_received', 'ws.ping', 'plugins.metrics-by-endpoint.response_time')) {
$value = Get-PropertyValue -Object $counters -Name $metric
if ($null -ne $value) {
Write-Output "$metric: $value"
}
}
}
if ($rates) {
foreach ($property in $rates.PSObject.Properties) {
Write-Output "rate.$($property.Name): $($property.Value)"
}
}
if ($summaries) {
foreach ($property in $summaries.PSObject.Properties) {
$line = Format-StatLine -Prefix "summary.$($property.Name)" -Stats $property.Value
if ($line) {
Write-Output $line
}
}
}
#!/usr/bin/env pwsh
param(
[Parameter(Mandatory = $true)]
[string]$ReportFile
)
$ErrorActionPreference = 'Stop'
if (-not (Test-Path $ReportFile)) {
throw "Report file not found: $ReportFile"
}
$report = Get-Content -Raw -Path $ReportFile | ConvertFrom-Json -Depth 100
$aggregate = $report.aggregate
if (-not $aggregate) {
throw 'Artillery report is missing aggregate data.'
}
function Get-PropertyValue {
param(
[Parameter(Mandatory = $true)]
[object]$Object,
[Parameter(Mandatory = $true)]
[string]$Name
)
if (-not $Object) {
return $null
}
$property = $Object.PSObject.Properties[$Name]
if ($property) {
return $property.Value
}
return $null
}
function Format-StatLine {
param(
[string]$Prefix,
[object]$Stats
)
if (-not $Stats) {
return $null
}
$parts = @()
foreach ($name in @('count', 'min', 'max', 'mean', 'median', 'p95', 'p99')) {
$value = Get-PropertyValue -Object $Stats -Name $name
if ($null -ne $value) {
$parts += "$name=$value"
}
}
if ($parts.Count -eq 0) {
return $null
}
return "$Prefix: $($parts -join ', ')"
}
$counters = Get-PropertyValue -Object $aggregate -Name 'counters'
$rates = Get-PropertyValue -Object $aggregate -Name 'rates'
$summaries = Get-PropertyValue -Object $aggregate -Name 'summaries'
$created = [double](Get-PropertyValue -Object $counters -Name 'vusers.created' ?? 0)
$failed = [double](Get-PropertyValue -Object $counters -Name 'vusers.failed' ?? 0)
$completed = [double](Get-PropertyValue -Object $counters -Name 'vusers.completed' ?? 0)
$errorRate = if ($created -gt 0) { [math]::Round(($failed / $created) * 100, 2) } else { 0 }
Write-Output '=== Artillery WebSocket Summary ==='
Write-Output "source: $ReportFile"
$timestamp = Get-PropertyValue -Object $report -Name 'timestamp'
if ($timestamp) {
Write-Output "timestamp: $timestamp"
}
if ($null -ne $created) {
Write-Output "vusers.created: $created"
}
if ($null -ne $completed) {
Write-Output "vusers.completed: $completed"
}
if ($null -ne $failed) {
Write-Output "vusers.failed: $failed"
}
Write-Output "vusers.failure_rate_percent: $errorRate"
if ($counters) {
foreach ($metric in @('ws.messages_sent', 'ws.messages_received', 'ws.ping', 'plugins.metrics-by-endpoint.response_time')) {
$value = Get-PropertyValue -Object $counters -Name $metric
if ($null -ne $value) {
Write-Output "$metric: $value"
}
}
}
if ($rates) {
foreach ($property in $rates.PSObject.Properties) {
Write-Output "rate.$($property.Name): $($property.Value)"
}
}
if ($summaries) {
foreach ($property in $summaries.PSObject.Properties) {
$line = Format-StatLine -Prefix "summary.$($property.Name)" -Stats $property.Value
if ($line) {
Write-Output $line
}
}
}
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env bash
# summarize-artillery.sh — Extract counters, rates and percentile stats from an
# Artillery JSON report. Mirrors scripts/loadtest/summarize-artillery.ps1.
# Usage: ./scripts/loadtest/summarize-artillery.sh <artillery-report.json>
set -euo pipefail
if [[ $# -lt 1 ]]; then
echo "Usage: $0 <artillery-report.json>" >&2
exit 2
fi
report_file="$1"
[[ -f "$report_file" ]] || { echo "Report file not found: $report_file" >&2; exit 1; }
REPORT_FILE="$report_file" node <<'EOF'
const fs = require('fs');
const report = JSON.parse(fs.readFileSync(process.env.REPORT_FILE, 'utf8'));
const aggregate = report.aggregate;
if (!aggregate) throw new Error('Artillery report is missing aggregate data.');
const get = (obj, name) => (obj && obj[name] !== undefined) ? obj[name] : undefined;
const counters = aggregate.counters || {};
const rates = aggregate.rates || {};
const summaries = aggregate.summaries || {};
const created = Number(get(counters, 'vusers.created') || 0);
const failed = Number(get(counters, 'vusers.failed') || 0);
const completed = Number(get(counters, 'vusers.completed') || 0);
const errorRate = created > 0 ? +((failed / created) * 100).toFixed(2) : 0;
console.log('=== Artillery WebSocket Summary ===');
console.log(`source: ${process.env.REPORT_FILE}`);
if (report.timestamp) console.log(`timestamp: ${report.timestamp}`);
console.log(`vusers.created: ${created}`);
console.log(`vusers.completed: ${completed}`);
console.log(`vusers.failed: ${failed}`);
console.log(`vusers.failure_rate_percent: ${errorRate}`);
for (const metric of ['ws.messages_sent', 'ws.messages_received', 'ws.ping', 'plugins.metrics-by-endpoint.response_time']) {
const v = get(counters, metric);
if (v !== undefined) console.log(`${metric}: ${v}`);
}
for (const [k, v] of Object.entries(rates || {})) console.log(`rate.${k}: ${v}`);
const fmt = (prefix, stats) => {
if (!stats) return;
const parts = [];
for (const name of ['count', 'min', 'max', 'mean', 'median', 'p95', 'p99']) {
const v = get(stats, name);
if (v !== undefined) parts.push(`${name}=${v}`);
}
if (parts.length) console.log(`${prefix}: ${parts.join(', ')}`);
};
for (const [k, v] of Object.entries(summaries)) fmt(`summary.${k}`, v);
EOF
+68 -68
View File
@@ -1,68 +1,68 @@
#!/usr/bin/env pwsh
param(
[Parameter(Mandatory = $true)]
[string]$SummaryFile
)
$ErrorActionPreference = 'Stop'
if (-not (Test-Path $SummaryFile)) {
throw "Summary file not found: $SummaryFile"
}
$summary = Get-Content -Raw -Path $SummaryFile | ConvertFrom-Json
$metrics = $summary.metrics
if (-not $metrics) {
throw 'Invalid k6 summary file: missing metrics object.'
}
$endpoints = @('overview', 'analytics', 'chat_log', 'users')
function Get-MetricValue {
param(
[object]$Metric,
[string]$Key
)
if (-not $Metric) {
return $null
}
if (-not $Metric.values) {
return $null
}
$value = $Metric.values.$Key
if ($null -ne $value) {
return [double]$value
}
return $null
}
$globalDuration = Get-MetricValue -Metric $metrics.http_req_duration -Key 'p(95)'
$globalFailRate = Get-MetricValue -Metric $metrics.http_req_failed -Key 'rate'
Write-Output "k6 Summary: $SummaryFile"
if ($null -ne $globalDuration) {
Write-Output ("Global p95 duration (ms): {0:N2}" -f $globalDuration)
}
if ($null -ne $globalFailRate) {
Write-Output ("Global failure rate: {0:P2}" -f $globalFailRate)
}
Write-Output ''
$rows = foreach ($endpoint in $endpoints) {
$durationMetricName = "http_req_duration{endpoint:$endpoint}"
$failMetricName = "http_req_failed{endpoint:$endpoint}"
$durationP95 = Get-MetricValue -Metric $metrics.$durationMetricName -Key 'p(95)'
$failRate = Get-MetricValue -Metric $metrics.$failMetricName -Key 'rate'
[PSCustomObject]@{
Endpoint = $endpoint
P95Ms = if ($null -ne $durationP95) { [math]::Round($durationP95, 2) } else { $null }
FailurePct = if ($null -ne $failRate) { [math]::Round($failRate * 100, 4) } else { $null }
}
}
$rows | Format-Table -AutoSize | Out-String | Write-Output
#!/usr/bin/env pwsh
param(
[Parameter(Mandatory = $true)]
[string]$SummaryFile
)
$ErrorActionPreference = 'Stop'
if (-not (Test-Path $SummaryFile)) {
throw "Summary file not found: $SummaryFile"
}
$summary = Get-Content -Raw -Path $SummaryFile | ConvertFrom-Json
$metrics = $summary.metrics
if (-not $metrics) {
throw 'Invalid k6 summary file: missing metrics object.'
}
$endpoints = @('overview', 'analytics', 'chat_log', 'users')
function Get-MetricValue {
param(
[object]$Metric,
[string]$Key
)
if (-not $Metric) {
return $null
}
if (-not $Metric.values) {
return $null
}
$value = $Metric.values.$Key
if ($null -ne $value) {
return [double]$value
}
return $null
}
$globalDuration = Get-MetricValue -Metric $metrics.http_req_duration -Key 'p(95)'
$globalFailRate = Get-MetricValue -Metric $metrics.http_req_failed -Key 'rate'
Write-Output "k6 Summary: $SummaryFile"
if ($null -ne $globalDuration) {
Write-Output ("Global p95 duration (ms): {0:N2}" -f $globalDuration)
}
if ($null -ne $globalFailRate) {
Write-Output ("Global failure rate: {0:P2}" -f $globalFailRate)
}
Write-Output ''
$rows = foreach ($endpoint in $endpoints) {
$durationMetricName = "http_req_duration{endpoint:$endpoint}"
$failMetricName = "http_req_failed{endpoint:$endpoint}"
$durationP95 = Get-MetricValue -Metric $metrics.$durationMetricName -Key 'p(95)'
$failRate = Get-MetricValue -Metric $metrics.$failMetricName -Key 'rate'
[PSCustomObject]@{
Endpoint = $endpoint
P95Ms = if ($null -ne $durationP95) { [math]::Round($durationP95, 2) } else { $null }
FailurePct = if ($null -ne $failRate) { [math]::Round($failRate * 100, 4) } else { $null }
}
}
$rows | Format-Table -AutoSize | Out-String | Write-Output
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env bash
# summarize-k6.sh — Extract per-endpoint p95 and failure rates from a k6 JSON
# summary export. Mirrors scripts/loadtest/summarize-k6.ps1.
# Usage: ./scripts/loadtest/summarize-k6.sh <k6-summary.json>
set -euo pipefail
if [[ $# -lt 1 ]]; then
echo "Usage: $0 <k6-summary.json>" >&2
exit 2
fi
summary_file="$1"
[[ -f "$summary_file" ]] || { echo "Summary file not found: $summary_file" >&2; exit 1; }
SUMMARY_FILE="$summary_file" node <<'EOF'
const fs = require('fs');
const summary = JSON.parse(fs.readFileSync(process.env.SUMMARY_FILE, 'utf8'));
const metrics = summary.metrics;
if (!metrics) throw new Error('Invalid k6 summary file: missing metrics object.');
const val = (m, key) => (m && m.values && m.values[key] !== undefined) ? Number(m.values[key]) : undefined;
const globalDuration = val(metrics.http_req_duration, 'p(95)');
const globalFailRate = val(metrics.http_req_failed, 'rate');
console.log(`k6 Summary: ${process.env.SUMMARY_FILE}`);
if (globalDuration !== undefined) console.log(`Global p95 duration (ms): ${globalDuration.toFixed(2)}`);
if (globalFailRate !== undefined) console.log(`Global failure rate: ${(globalFailRate * 100).toFixed(2)}%`);
console.log('');
const endpoints = ['overview', 'analytics', 'chat_log', 'users'];
const rows = endpoints.map((ep) => {
const dP95 = val(metrics[`http_req_duration{endpoint:${ep}}`], 'p(95)');
const fRate = val(metrics[`http_req_failed{endpoint:${ep}}`], 'rate');
return {
Endpoint: ep,
P95Ms: dP95 !== undefined ? dP95.toFixed(2) : 'n/a',
FailurePct: fRate !== undefined ? (fRate * 100).toFixed(4) : 'n/a',
};
});
const pad = (s, n) => String(s).padEnd(n);
console.log(pad('ENDPOINT', 12) + pad('P95_MS', 10) + 'FAILURE_PCT');
for (const r of rows) console.log(pad(r.Endpoint, 12) + pad(r.P95Ms, 10) + r.FailurePct);
EOF
+28 -28
View File
@@ -1,28 +1,28 @@
config:
target: "{{ $processEnvironment.WS_URL }}"
phases:
- duration: 60
arrivalRate: 10
name: warmup-10
- duration: 120
arrivalRate: 100
name: sustain-100
- duration: 180
arrivalRate: 500
name: sustain-500
- duration: 240
arrivalRate: 1000
name: sustain-1000
engines:
ws: {}
scenarios:
- engine: ws
name: chat-presence-and-send
flow:
- think: 1
- send:
text: '{"type":"ping","text":"loadtest"}'
- think: 1
- send:
text: '{"type":"ping","text":"loadtest-2"}'
- think: 1
config:
target: "{{ $processEnvironment.WS_URL }}"
phases:
- duration: 60
arrivalRate: 10
name: warmup-10
- duration: 120
arrivalRate: 100
name: sustain-100
- duration: 180
arrivalRate: 500
name: sustain-500
- duration: 240
arrivalRate: 1000
name: sustain-1000
engines:
ws: {}
scenarios:
- engine: ws
name: chat-presence-and-send
flow:
- think: 1
- send:
text: '{"type":"ping","text":"loadtest"}'
- think: 1
- send:
text: '{"type":"ping","text":"loadtest-2"}'
- think: 1