GL pixel shader generator creates `dot(vec3, float)`

[JayFoxRox]

Revision cb8a97a

---

Generated code is like this: r1.rgb = vec3(dot(r0.rgb, r0.a));

I'm not sure what exactly this does. I believe the dotproduct is only valid for rgb channels. We should probably hw-test this.

A workaround would be to cast all inputs to vec3; however, this probably wouldn't be correct:

nv2a_psh.c

QString *ab;
if (output.ab_op == PS_COMBINEROUTPUT_AB_DOT_PRODUCT) {
	ab = qstring_from_fmt("dot(vec3(%s), vec3(%s))",
						  qstring_get_str(a), qstring_get_str(b));
} else {
	ab = qstring_from_fmt("(%s * %s)",
						  qstring_get_str(a), qstring_get_str(b));
}

QString *cd;
if (output.cd_op == PS_COMBINEROUTPUT_CD_DOT_PRODUCT) {
	cd = qstring_from_fmt("dot(vec3(%s), vec3(%s))",
						  qstring_get_str(c), qstring_get_str(d));
} else {
	cd = qstring_from_fmt("(%s * %s)",
						  qstring_get_str(c), qstring_get_str(d));
}

Affected games:

• Mace Griffin Bounty Hunter
• Tony Hawk's American Wasteland
• Halo 2

[JayFoxRox]

An update on this: This violates the OpenGL spec (NV_register_combiners), so we need hardware testing.

If the parameter is ALPHA, specifying a non-FALSE value for either of the parameters <abDotProduct> or <cdDotProduct>, generates an INVALID_VALUE error.