Validate negative test error types in test262 harness by lahma · Pull Request #2361 · sebastienros/jint (original) (raw)
and others added 5 commits
Previously the test262 harness only checked testCase.Negative (a boolean) when validating negative tests, without verifying that the thrown error type matches what the test expects. This could cause false passes when Jint throws the wrong error type (e.g. TypeError instead of SyntaxError).
Add error type validation that maps Jint exceptions to Test262 ExpectedErrorType and compares against NegativeTestCase.Type. Covers ScriptPreparationException, JavaScriptException (via error name property), and internal SyntaxError/TypeError/RangeError exceptions.
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
The initial validation returned null for non-object JS throws (like strings from $DONOTEVALUATE), causing false passes for negative tests where the parser failed to reject invalid syntax. Map non-object throws and unknown error names to Test262Error so they correctly mismatch against expected SyntaxError/TypeError/etc.
Add 24 test files to exclusion list for known parser gaps (return at global level, await-using/using in single-statement positions, HTML close comment on first line, invalid JSON import attributes).
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- Disable AllowReturnOutsideFunction in test262 harness to match ECMAScript spec (return at global scope is a SyntaxError). The engine default remains true for backward compatibility.
- Throw SyntaxError instead of generic JavaScriptException for invalid JSON module imports in ModuleFactory.BuildJsonModule.
- Remove 12 test exclusions now passing (11 return + 1 JSON import). 14 exclusions remain for Acornima parser limitations (using/await-using in single-statement positions, HTML close comment on first line).
Result: 95,978 passed (+23), 1,376 skipped (-23), 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- using/await-using exclusions: link to Acornima fix PR sebastienros#32
- HTML comment exclusions: root cause is missing EvalError in Test262Harness NuGet ExpectedErrorType enum, not Acornima
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
Update Test262Harness NuGet package and console tool to 1.0.4 which adds EvalError and URIError to ExpectedErrorType enum. Add EvalError and URIError mapping in GetErrorTypeFromJsError.
Remove 3 HTML close comment test exclusions that were caused by the missing EvalError enum variant — Acornima handles --> on first line correctly.
Result: 95,981 passed (+3), 1,373 skipped (-3), 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
This was referenced
Mar 27, 2026
This was referenced
Jun 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})