ZOA TestKit

DeterministicTestFixture

class

The base class to inherit whenever a test touches randomness, time or shared state. Seeds the random source and gives you a clock you advance by hand.

• protected Random DeterministicRandom { get; }
• protected double TestTime { get; }
• protected float DeltaTime { get; }
• protected int CurrentSeed { get; }
• protected virtual int FixedSeed => 0
• protected virtual void OnSetUp()
• protected virtual void OnTearDown()
• protected void AdvanceTime(float delta)
• protected void SimulateFrames(int frameCount, float fixedDelta = 1f / 60f)
• protected void ResetTime()
• protected float RandomFloat()
• protected int RandomInt(int min, int max)
• protected T RandomPick<T>(IList<T> items)

IGoldenDataTestRunner

interface

Registers expected payloads and compares actual output against them, returning a hashed result with a diff.

• void RegisterGoldenData(GoldenTestId testId, string expectedJson)
• GoldenTestResult RunTest(GoldenTestId testId, string actualJson)
• IList<GoldenTestResult> RunAll()
• IList<GoldenTestId> GetRegisteredTests()

GoldenTestResult

class

The outcome of one comparison: pass or fail, both hashes, the diff that explains a failure, and how long it took.

• GoldenTestId TestId { get; set; }
• bool Passed { get; set; }
• string ExpectedHash { get; set; }
• string ActualHash { get; set; }
• string Diff { get; set; }
• long ExecutionTimeMs { get; set; }

GoldenTestId

struct

Readonly id with implicit conversions to and from string, so a literal is valid at any call site.

• string Value { get; }

TestHashUtility

class

SHA256 over UTF-8 bytes returned as hex, plus the JSON comparison that produces the diff a failing golden test reports.

• static string ComputeHash(string data)
• static bool CompareJson(string expected, string actual, out string diff)

IScenarioRunner

interface

Owns registration and ordering. Execution is delegated to an IScenarioExecutor so the same scenario can run against different harnesses.

• void RegisterScenario(ScenarioDefinition scenario)
• IList<ScenarioStep> RunScenario(string scenarioId, IScenarioExecutor executor)
• IList<ScenarioDefinition> GetRegisteredScenarios()

IScenarioExecutor

interface

What a step actually does. Return false to fail the step and expose the reason through GetStepOutput.

• bool ExecuteStep(ScenarioStep step)
• string GetStepOutput()

ScenarioDefinition

class

A named, ordered sequence of steps built fluently. Tags make a subset selectable.

• ScenarioDefinition(string id, string name, string description)
• ScenarioDefinition AddStep(string description, string action, string expectedOutcome)
• ScenarioDefinition AddTag(string tag)
• IReadOnlyList<ScenarioStep> Steps { get; }
• IReadOnlyList<string> Tags { get; }

ScenarioStep

class

One step and its result. Passed and ErrorMessage are filled in by the run, so a report names the step that failed and what it expected.

• int StepIndex { get; set; }
• string Description { get; set; }
• string Action { get; set; }
• string ExpectedOutcome { get; set; }
• bool Passed { get; set; }
• string ErrorMessage { get; set; }

ScenarioRunner

class

The shipped runner. Constructed with stopOnFirstFailure, defaulting to true, so a scenario stops at the first broken step rather than cascading.

• ScenarioRunner(bool stopOnFirstFailure = true)

FlakyTestDetector

class

Scores each test on how often it flips between pass and fail across a rolling window. Alternation is flakiness; consistent failure is a bug, and the score separates them.

• FlakyTestDetector(int windowSize = 10, double flakinessThreshold = 0.3)
• void RecordResult(string testFullName, bool passed, double durationMs = 0)
• IReadOnlyList<FlakyTestReport> GetFlakyTests()
• FlakyTestReport GetReport(string testFullName)
• void Reset()

FlakyTestReport

class

One test's record: flakiness score, pass rate, how many runs are in the window, and average duration.

• string TestFullName { get; }
• double FlakinessScore { get; }
• double PassRate { get; }
• int RunCount { get; }
• double AverageDurationMs { get; }

TestCoveragePolicy

class

Per-package minimum test counts, tiered by how much depends on the package. The release gate reads this, so falling below the threshold fails the gate.

• static IReadOnlyDictionary<string, int> PackageThresholds { get; }
• const int DefaultMinimumTests = 2
• static int GetThreshold(string packageName)

SmokeTestDefinition

class

What a wizard should look like: its type name, its expected step count, and the fields it must expose. Catches a wizard that still opens but has lost a step.

• SmokeTestDefinition(string wizardTypeName, int expectedStepCount)
• SmokeTestDefinition AddRequiredField(string fieldName)
• SmokeTestDefinition AddTag(string tag)
• IReadOnlyList<string> RequiredFields { get; }
• IReadOnlyList<string> Tags { get; }

TestMatrixEntry

class

One cell of the render-pipeline validation matrix: which pipeline, which sample, which test, whether it passed, and the screenshot captured for it.

• string Pipeline { get; set; }
• string SampleName { get; set; }
• string TestName { get; set; }
• bool Passed { get; set; }
• string ErrorMessage { get; set; }
• string ScreenshotPath { get; set; }