-
Notifications
You must be signed in to change notification settings - Fork 17
[FIX] web support #84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FIX] web support #84
Conversation
Remove platform-specific test expectations in preparation for cross-platform fixes. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The ieee754 package's toFloat16Bytes() uses ByteData.setInt16() which behaves incorrectly on JS where bitwise operators are limited to 32-bit signed integers. This implementation constructs bytes directly. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
On JS, double.infinity and double.nan might incorrectly match the int type check, leading to BigInt.from(infinity) which throws. This fix checks for NaN/Infinity before the int type check. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
BigInt.isValidInt is unreliable on JS for values > 2^53-1. This fix adds explicit bitLength checks to ensure safe integer handling across all platforms. - arg.dart: operator ~() now returns ArgBigInt when result exceeds JS safe integer range - int.dart: CborInt factory uses explicit safe integer check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ByteData.getUint64()/setUint64() don't exist on JS/WASM. This fix: - Adds platform detection constants for JS and WASM - Implements 64-bit integer read/write using two 32-bit operations - Uses multiplication instead of bit shifts to avoid JS limitations - Properly handles signed integers via two's complement conversion Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove Uint64List type expectation since web platforms return List<int> instead. The value check still ensures correctness. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The ieee754 package's fromFloat16Bytes() uses ByteData.getInt16() which interprets bytes as a signed integer on JS. For values with the high bit set (negative numbers, large exponents), this produces incorrect results. For example, -4.0 (bytes 0xC400) was decoded as -33554428 instead of -4.0. This fix: - Adds fromFloat16Bytes() to custom float16.dart utility - Uses only unsigned byte operations that work consistently across platforms - Updates all decoder call sites to use the custom implementation - Documents the rationale in the utility file Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove ieee754 package dependency entirely and implement custom float encoding/decoding utilities that work correctly across all platforms (VM, dart2js, dart2wasm). Key changes: - Rename float16.dart to float_utils.dart and add float32/float64 helpers - Add isFloat16Lossless, isFloat32Lossless, isFloat64Lossless functions - Add toFloat32Bytes, toFloat64Bytes, fromFloat32Bytes, fromFloat64Bytes - Update double.dart, stage3.dart, pretty_print.dart to use float_utils - Fix negative zero encoding in CborSmallInt (check value == 0 first) - Use `identical(0, 0.0)` for reliable JS platform detection - Update tests with platform-specific expectations for integer-valued doubles that cannot be distinguished from integers on JS This fixes remaining JS test failures by properly detecting and handling: - Large doubles exceeding safe integer range (now encoded as floats) - Negative zero in integer encoding (now correctly encodes as 0) - JSON encoding of integer-valued doubles on JS All 423 VM tests, 411 JS tests, and 411 WASM tests now pass. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
| /// // Use: | ||
| /// cborEncode(CborFloat(1.0)); // Always encoded as float | ||
| /// ``` | ||
| class CborSimpleCodec extends Codec<Object?, List<int>> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I personally wonder if this should be deprecated (to force people to use cborEncode and actually specify CborInt/CborFloat etc, which prevents many web issues.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the documentation you have added here is sufficient to warn web users. I don't want to threaten to remove this from native/wasm users who aren't affected by it.
| // are indistinguishable from integers and get encoded as CBOR integers. | ||
| // Note: This does NOT apply to WASM (dart2wasm), which handles this correctly. | ||
| // Using `identical(0, 0.0)` which returns true only on JS where int/double are same. | ||
| final bool kIsJs = identical(0, 0.0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: bool.fromEnvironment('dart.library.js_interop') was not working reliably
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting, this comes from flutter here.
Maybe the Dart runtime is different with this then.
Add comprehensive test coverage for the custom float encoding/decoding utilities that replaced the ieee754 package. Tests cover: - Float16 (half-precision): encoding, decoding, round-trips - Special values: NaN, +/-Infinity, +/-0.0 - Normal and subnormal values - Overflow to infinity, underflow to zero - RFC 8949 Appendix A test vectors - Float32 (single-precision): encoding, decoding, round-trips - Float64 (double-precision): encoding, decoding, round-trips - isFloat16Lossless, isFloat32Lossless, isFloat64Lossless checks - Cross-platform consistency tests (JS-specific bug fixes) - Edge cases: byte array lengths, big-endian order 105 new tests, all passing on VM, JS (dart2js), and WASM (dart2wasm). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
66260d8 to
b41c2b6
Compare
Add ieee754 package as dev dependency and use it to validate that the custom float_utils implementation produces identical results to ieee754 on non-JS platforms. Validation is performed inline within each existing test using helper functions that compare results against ieee754 (skipped on JS where ieee754 has known issues). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
I added one last commit that uses the basically tests the Tbh we can prob remove that commit and dependency altogether? I wanted this mostly as a cross-reference to catch any potential issues with local implementation. Note: Mention in some comment(s) too, but the |
|
Thanks for this, I'll merge it into its own branch for now and have a look at it. |
* [FIX] web support (#84) * test: unify web and VM test expectations Remove platform-specific test expectations in preparation for cross-platform fixes. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: implement cross-platform half-precision float encoding The ieee754 package's toFloat16Bytes() uses ByteData.setInt16() which behaves incorrectly on JS where bitwise operators are limited to 32-bit signed integers. This implementation constructs bytes directly. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: handle Infinity encoding by checking special doubles first On JS, double.infinity and double.nan might incorrectly match the int type check, leading to BigInt.from(infinity) which throws. This fix checks for NaN/Infinity before the int type check. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: use explicit safe integer bounds for BigInt handling on JS BigInt.isValidInt is unreliable on JS for values > 2^53-1. This fix adds explicit bitLength checks to ensure safe integer handling across all platforms. - arg.dart: operator ~() now returns ArgBigInt when result exceeds JS safe integer range - int.dart: CborInt factory uses explicit safe integer check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add web platform support for Uint64/Int64 typed arrays ByteData.getUint64()/setUint64() don't exist on JS/WASM. This fix: - Adds platform detection constants for JS and WASM - Implements 64-bit integer read/write using two 32-bit operations - Uses multiplication instead of bit shifts to avoid JS limitations - Properly handles signed integers via two's complement conversion Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: update Uint64 test to work cross-platform Remove Uint64List type expectation since web platforms return List<int> instead. The value check still ensures correctness. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: implement cross-platform half-precision float decoding The ieee754 package's fromFloat16Bytes() uses ByteData.getInt16() which interprets bytes as a signed integer on JS. For values with the high bit set (negative numbers, large exponents), this produces incorrect results. For example, -4.0 (bytes 0xC400) was decoded as -33554428 instead of -4.0. This fix: - Adds fromFloat16Bytes() to custom float16.dart utility - Uses only unsigned byte operations that work consistently across platforms - Updates all decoder call sites to use the custom implementation - Documents the rationale in the utility file Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: replace ieee754 dependency with custom float_utils Remove ieee754 package dependency entirely and implement custom float encoding/decoding utilities that work correctly across all platforms (VM, dart2js, dart2wasm). Key changes: - Rename float16.dart to float_utils.dart and add float32/float64 helpers - Add isFloat16Lossless, isFloat32Lossless, isFloat64Lossless functions - Add toFloat32Bytes, toFloat64Bytes, fromFloat32Bytes, fromFloat64Bytes - Update double.dart, stage3.dart, pretty_print.dart to use float_utils - Fix negative zero encoding in CborSmallInt (check value == 0 first) - Use `identical(0, 0.0)` for reliable JS platform detection - Update tests with platform-specific expectations for integer-valued doubles that cannot be distinguished from integers on JS This fixes remaining JS test failures by properly detecting and handling: - Large doubles exceeding safe integer range (now encoded as floats) - Negative zero in integer encoding (now correctly encodes as 0) - JSON encoding of integer-valued doubles on JS All 423 VM tests, 411 JS tests, and 411 WASM tests now pass. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Removed ieee dep * [chore] auto-formatted code * test: add exhaustive tests for float_utils.dart Add comprehensive test coverage for the custom float encoding/decoding utilities that replaced the ieee754 package. Tests cover: - Float16 (half-precision): encoding, decoding, round-trips - Special values: NaN, +/-Infinity, +/-0.0 - Normal and subnormal values - Overflow to infinity, underflow to zero - RFC 8949 Appendix A test vectors - Float32 (single-precision): encoding, decoding, round-trips - Float64 (double-precision): encoding, decoding, round-trips - isFloat16Lossless, isFloat32Lossless, isFloat64Lossless checks - Cross-platform consistency tests (JS-specific bug fixes) - Edge cases: byte array lengths, big-endian order 105 new tests, all passing on VM, JS (dart2js), and WASM (dart2wasm). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add ieee754 validation tests for float_utils Add ieee754 package as dev dependency and use it to validate that the custom float_utils implementation produces identical results to ieee754 on non-JS platforms. Validation is performed inline within each existing test using helper functions that compare results against ieee754 (skipped on JS where ieee754 has known issues). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * DCM fixes --------- Co-authored-by: Alexandru Dochioiu <38853913+AlexDochioiu@users.noreply.github.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Noticed that there are quite a lot of web issues, for multiple reasons. A few of them are JS limitations which cannot possibly be resolved (cause we cannot distinguish int from float). But many others are resolvable.
P.s. I removed the
ieee754dependency because a lot of issues were caused by itNote - I used/guided Claude heavily for this work