Constant sequential filling operation needs output shape parameter

[fdwr]

See constant(start, end, step, type) at https://www.w3.org/TR/webnn/#dom-mlgraphbuilder-constant-start-end-step-type

Problem

Computing the output shape indirectly from a floating-point calculation is brittle because the caller, depending on its equation and rounding, can yield a different size from WebNN which will cause graph execution to fail due to mismatched tensors. Such incompatible off-by-one differences can occur via differing calculation precisions (e.g. the caller computes the output shape in float64 whereas Chromium computes it in float32) and via differing handling of the inclusive/exclusive end value (see a non-hypothetical example below). So it's prudent for the library to pass WebNN the shape explicitly, and we've increasingly been moving toward more explicitness anyway (e.g. removing the special null values in Reshape). The output shape was in the fillSequence operator, but got dropped, and it reoccurred to me while reviewing Bin's Chromium implementation.

Proposal

- constant(start, end, step, type) method steps are:
+ constant(start, step, outputShape, type) method steps are:
...
- Let size be max(0, ceil((end - start)/step)).
- Set descriptor.[dimensions](https://www.w3.org/TR/webnn/#dom-mloperanddescriptor-dimensions) to [size].
+ Set descriptor.[dimensions](https://www.w3.org/TR/webnn/#dom-mloperanddescriptor-dimensions) to [outputShape].
+ Let outputElementCount be the product of all elements in descriptor.[dimensions](https://www.w3.org/TR/webnn/#dom-mloperanddescriptor-dimensions).
...
- [For each](https://infra.spec.whatwg.org/#map-iterate) index in [the range](https://infra.spec.whatwg.org/#the-range) 0 to size, exclusive:
+ [For each](https://infra.spec.whatwg.org/#map-iterate) index in [the range](https://infra.spec.whatwg.org/#the-range) 0 to outputElementCount, exclusive:

* Note the end is now redundant, since it was only used for size calculation, not needed for the core start + (index * step) formula.

Problematic Library differences

We see differences between TF and PT with the exact same inputs due to how they handle the end result (included vs excluded), which emphasizes why WebNN should be clearly informed by the caller of the expected shape:

import tensorflow
result = tensorflow.range(5, 5, 2)
print("value:", result)

# output = [], shape= (0,)

import torch
result = torch.range(5, 5, 2)
print("value:", result)

# output = [5], shape= (1,)

Usage Examples

... = constant(0, 1, [3,3], "float32"); // Fill with an increasing sequence.
// output = [[0,1,2],[3,4,5],[6,7,8]

... = constant(9, -1, [3,3], "float32"); // Fill with a decreasing sequence.
// output = [[9,8,7],[6,5,4],[3,2,1]

... = constant(1, 0, [3,3], "float16"); // Quickly fill large tensor with a constant value, like numpy.ones(outputShape).
// output = [[1,1,1],[1,1,1],[1,1,1]    // Note this is currently awkward otherwise for float16, because you have to
                                        // pass 0x3C00 (1.0) to fill a Uint16Array.

@wchao1115 @huningxin

[huningxin]

constant(start, step, outputShape, type)

Could the parameters outputShape and type be encapsulated by MLOperandDescriptor?

[miaobin]

constant(start, step, outputShape, type)

So the final proposal is like this: constant(start, step, operand_descriptor), right (I guess the order of parameters is like this)? @huningxin @fdwr WDYT?

[fdwr]

constant(start, step, outputShape, type)

Could the parameters outputShape and type be encapsulated by MLOperandDescriptor?

I'm fine with it either way. If you as a caller already have shape and data type variables, then requiring a temporary tensor desc is a little more bothersome than just passing them directly as parameters, but if you already have a tensor desc, then passing that directly is slightly more convenient. 🤷‍♂️ Since the internal implementation complexity doesn't really change either way, maybe asking someone from the WebNN API caller point of view (like Wanming from ORT) would be a nice tie breaker?

Whatever we do, we should use the same pattern in the future with similar operators. So if we introduce, for example, a diagonal matrix operator that fills a new tensor with 1's along the diagonal and 0's everywhere else, then it should take a tensordesc or two separate parameters. I don't see that we have much existing precedent though. Any ops that take a shape currently (like reshape or expand) take the shape as a separate parameter, and ops that take a data type take them as a separate parameter (like cast), but we don't have any others that take both shape and data type.

[Honry]

Could the parameters outputShape and type be encapsulated by MLOperandDescriptor?

No much impact to WebNN EP, I am fine with it either way as well.

[zolkis]

In Web APIs they usually prefer passing options in dictionaries.
Semantically important parameters make sense to be passed as args, if there is not many of them.

[fdwr]

In Web APIs they usually prefer passing options in dictionaries. Semantically important parameters make sense to be passed as args, if there is not many of them.

Well these are essential parameters, not options, but it would be more consistent with the existing constant(descriptor, bufferView) overload to accept a tensor description MLOperandDescriptor. So, seeing that precedent, I more clearly favor making it take MLOperandDescriptor too. Then we would have these two (or three) overloads:

1. constant(descriptor, bufferView);
2. constant(descriptor, start, step = 0);
3. constant(value, type) // I recall Ningxin proposed removing this one elsewhere due to implementation complexity.

Usage:

const x = builder.constant({dataType: 'uint32', dimensions: [4]}, new Float32Array([1,3,5,7]));
const y = builder.constant({dataType: 'uint32', dimensions: [4]}, 1.0, 2.0); // 1,3,5,7

I guess the order of parameters is like this)

If it causes no ambiguity in the IDL, then I'd like both overload parameters to go in the same order, rather than inconsistently reversed between them. Also if you make step optional (defaulting to 0), then that satisfies the reasonably likely use case of wanting to fill with a solid value (const y = builder.constant({dataType: 'uint32', dimensions: [4]}, 1.0)). Conveniently, I think that also obviates the constant(value, type) overload?

[wacky6]

In Web APIs they usually prefer passing options in dictionaries. Semantically important parameters make sense to be passed as args, if there is not many of them.

I think marking fields are required in IDL dictionary is fine (it's not uncommon). So the proposal in #492 (comment) looks reasonable to me.

---

The following is also fine IMO.

dictionary FillSequenceDescriptor {
  required double start;
  double step = 0;
};

constant(MLOperandDescriptor desc, FillSequenceDescriptor fillSequence);

The type we use for start and step might be tricky. I think we'll at least need a double to cover int32 and float32 (fp32 can't accurately represent large int32 start value).

How to support for int64 remains an open question :)

[fdwr]

I think we'll at least need a double to cover int32 and float32

One of the three current constant overloads already takes a double...

MLOperand constant(double value, optional MLOperandDataType type = "float32");

...and so there's precedent for passing at least that much precision. I agree that float32 is not sufficient to cover the accuracy needed (it cannot even represent odd int32 values >= 16777216 accurately).

How to support for int64 remains an open question

One operator (split) already accepts parameters of differing types:

partial interface MLGraphBuilder {
    sequence<MLOperand> split(
        MLOperand input,
        (unsigned long or sequence<unsigned long>) splits, <----------------
        optional MLSplitOptions options = {}
    );
};

So (barring any IDL issues I'm not seeing?) we could accept (double or BigInt) to accurately cover the full floating point and integer range:

MLOperand constant(MLOperandDescriptor descriptor, (double or BigInt) start, (double or BigInt) step = 0.0);

Curiously I see a BigUint64Array, but I don't see a matching BigUint type? 🙃 I suppose it doesn't matter because a BigInt can hold unsigned values too, with the precise type uint64 being clarified by the operand desc's dataType (and I see that BigInt.asUintN(64, (2n ** 64n) - 1n) runs fine, printing 18446744073709551615n as expected).

[miaobin]

So (barring any IDL issues I'm not seeing?) we could accept (double or BigInt) to accurately cover the full floating point and integer range:

MLOperand constant(MLOperandDescriptor descriptor, (double or BigInt) start, (double or BigInt) step = 0.0);

I thought of the same method as you. And I am trying to add bigint to verify the feasibility of the implementation.

[miaobin]

Updates to the constant sequential filling operation:

1. CL5091540 (extends the build method of constant) and CL5243778 (defines mojom and DirectML node) are developed according to IDL:MLOperand constant(MLOperandDescriptor desc, (double or bigint) start, (double or bigint) step);

2. We met the problem of using the keyword or in IDL to create a union type containing double and bigint. In the generated file "v8_union_bigint_double.cc", it was only judged that the incoming data was of JS number type and there is no judgment that it is BigInt, which causes an error to be reported when BigInt data is passed in when calling this API using JS code. (For example, when we call constant op, we pass in start=123n). @fdwr Maybe we should file an issue to chromium for this problem?

On one hand, we can continue to review these two CLs to find potential problems and verify feasibility. On the other hand, can we first create an updated specification PR which matches what is being implemented and get folks with more WebIDL and JavaScript API design knowledge to take a look at it to make sure it will work the way we intend it to as Reilly said? WDYT @huningxin @fdwr @wacky6