Adding support for `Metadata` class

[adespawn]

This is an umbrella issue for all endpoints of the Metadata class

List of possible enpoints:

• checkSchemaAgreement
• clearPrepared
• getAggregate
• getAggregates
• getFunction
• getFunctions
• getMaterializedView
• getReplicas
• getTable
• getTokenRanges
• getTokenRangesForHost
• getTrace
• getUdt
• isDbaas
• newToken
• newTokenRange
• refreshKeyspace
• refreshKeyspaces

[adespawn]

Let's start step by step. If at some point @wprzytula you decide there is too much going on in this single issue, we can split the discussion into multiple ones.

refreshKeyspace(s)

It does the following: Gets the keyspace metadata information and updates the internal state of the driver. (or across all keyspaces. For simplicyty let's talk only about this version. Everything is the same in the "plural" version of this function).

As all other client enpoints it can be used either with async or with callback.

<smallRant>
Hover, compared to client methods, the return type is not documented in any way... First part that is not documented is what happens with the return value.

refreshKeyspace(name, callback) {
        return promiseUtils.optionalCallback(
            this._refreshKeyspace(name),
            callback,
        );
    }

With _refreshKeyspace being async, this means that if a callback is provided, then once _refreshKeyspace finishes, it's called with the return value. Otherwise, this returs _refreshKeyspace promise, which means refreshKeyspace behaves as an async function, without the magic async keyword or any JSdoc type information.

The other problem here is that the return type (keyspace metadata) is... Well, first of all, it's not visible at the caller level due to the abstraction present, and secondly, it's improperly defined.

The type of the metadata is as follows:

{name, durableWrites, strategy, strategyOptions, tokenToReplica, udts, tables, functions, aggregates},

with no TS type definition available (did I mentioned that TS types do not mention return value of refreshKeyspace, making it impossible to use this metadata in TS code?).

name, durableWrites, strategy, strategyOptions, tokenToReplica, virtual (not mentioned in type definition) are properly filled during construction.

graphEngine (as per this discussion it's likely to be deprecated) is (again not mentioned in type definition and) set to undefined in constructor, and later set to some value.

udts, tables, functions, aggregates from what I was able to search are never set (very high confidence on aggregates, medium confidence on others). In the constructor they are set to {}
</smallRant>

Considering the above, what may be the best approach here?

1. Improve type information for this part of the API
2. Consider to deprecate current name (but still support for API compatibility reasons). The name suggests something else than will happen with our implementation. We would probably want to have the same approach as discussed in Re-implement schema Metadata support scylladb-zpp-2025-csharp-rs-driver/csharp-driver#89, where the Rust driver is responsible for fetching new metadata. We could replace it with a better named endpoint.
3. Add support for virtual in the same way @wprzytula in C# driver
4. Ensure udts, tables, functions, aggregates are not set in a sneaky part of the code, and probably remove them.

[adespawn]

getTable

While it suffers the same callaback-hell type problem as refreshKeyspace, there is a proper definition of expected fields:

class TableMetadata {
    this.name;
    this.replicateOnWrite = true;
    this.memtableFlushPeriod = 0;
    this.indexInterval = null;
    this.isCompact = false;
    this.indexes = null;
    this.cdc = null;
    this.virtual = false;
}

This class definition on the other hand does not seem to contain all of the fields set by the DSx driver:

nodejs-rs-driver/lib/metadata/schema-parser.js

Lines 452 to 614 in a812258

	async _parseTableOrView(
	tableInfo,
	tableRow,
	columnRows,
	indexRows,
	virtual,
	) {
	// All the tableInfo parsing in V1 is sync, it uses a async function because the super class defines one
	// to support other versions.
	let c, name, types;
	const encoder = this.cc.getEncoder();
	const columnsKeyed = {};
	let partitionKeys = [];
	let clusteringKeys = [];
	tableInfo.bloomFilterFalsePositiveChance =
	tableRow["bloom_filter_fp_chance"];
	tableInfo.caching = tableRow["caching"];
	tableInfo.comment = tableRow["comment"];
	tableInfo.compactionClass = tableRow["compaction_strategy_class"];
	tableInfo.compactionOptions = JSON.parse(
	tableRow["compaction_strategy_options"],
	);
	tableInfo.compression = JSON.parse(tableRow["compression_parameters"]);
	tableInfo.gcGraceSeconds = tableRow["gc_grace_seconds"];
	tableInfo.localReadRepairChance = tableRow["local_read_repair_chance"];
	tableInfo.readRepairChance = tableRow["read_repair_chance"];
	tableInfo.populateCacheOnFlush =
	tableRow["populate_io_cache_on_flush"] ||
	tableInfo.populateCacheOnFlush;
	tableInfo.memtableFlushPeriod =
	tableRow["memtable_flush_period_in_ms"] ||
	tableInfo.memtableFlushPeriod;
	tableInfo.defaultTtl =
	tableRow["default_time_to_live"] || tableInfo.defaultTtl;
	tableInfo.speculativeRetry =
	tableRow["speculative_retry"] || tableInfo.speculativeRetry;
	tableInfo.indexInterval =
	tableRow["index_interval"] || tableInfo.indexInterval;
	if (typeof tableRow["min_index_interval"] !== "undefined") {
	// Cassandra 2.1+
	tableInfo.minIndexInterval =
	tableRow["min_index_interval"] || tableInfo.minIndexInterval;
	tableInfo.maxIndexInterval =
	tableRow["max_index_interval"] || tableInfo.maxIndexInterval;
	} else {
	// set to null
	tableInfo.minIndexInterval = null;
	tableInfo.maxIndexInterval = null;
	}
	if (typeof tableRow["replicate_on_write"] !== "undefined") {
	// leave the default otherwise
	tableInfo.replicateOnWrite = tableRow["replicate_on_write"];
	}
	tableInfo.columns = [];
	for (let i = 0; i < columnRows.length; i++) {
	const row = columnRows[i];
	const type = encoder.parseFqTypeName(row["validator"]);
	c = {
	name: row["column_name"],
	type: type,
	isStatic: false,
	};
	tableInfo.columns.push(c);
	columnsKeyed[c.name] = c;
	switch (row["type"]) {
	case "partition_key":
	partitionKeys.push({
	c: c,
	index: row["component_index"] || 0,
	});
	break;
	case "clustering_key":
	clusteringKeys.push({
	c: c,
	index: row["component_index"] || 0,
	order: c.type.options.reversed ? "DESC" : "ASC",
	});
	break;
	case "static":
	// C* 2.0.6+ supports static columns
	c.isStatic = true;
	break;
	}
	}
	if (partitionKeys.length > 0) {
	tableInfo.partitionKeys = partitionKeys
	.sort(utils.propCompare("index"))
	.map((item) => item.c);
	clusteringKeys.sort(utils.propCompare("index"));
	tableInfo.clusteringKeys = clusteringKeys.map((item) => item.c);
	tableInfo.clusteringOrder = clusteringKeys.map(
	(item) => item.order,
	);
	}
	// In C* 1.2, keys are not stored on the schema_columns table
	const keysStoredInTableRow = tableInfo.partitionKeys.length === 0;
	if (keysStoredInTableRow && tableRow["key_aliases"]) {
	// In C* 1.2, keys are not stored on the schema_columns table
	partitionKeys = JSON.parse(tableRow["key_aliases"]);
	types = encoder.parseKeyTypes(tableRow["key_validator"]).types;
	for (let i = 0; i < partitionKeys.length; i++) {
	name = partitionKeys[i];
	c = columnsKeyed[name];
	if (!c) {
	c = {
	name: name,
	type: types[i],
	};
	tableInfo.columns.push(c);
	}
	tableInfo.partitionKeys.push(c);
	}
	}
	const comparator = encoder.parseKeyTypes(tableRow["comparator"]);
	if (keysStoredInTableRow && tableRow["column_aliases"]) {
	clusteringKeys = JSON.parse(tableRow["column_aliases"]);
	for (let i = 0; i < clusteringKeys.length; i++) {
	name = clusteringKeys[i];
	c = columnsKeyed[name];
	if (!c) {
	c = {
	name: name,
	type: comparator.types[i],
	};
	tableInfo.columns.push(c);
	}
	tableInfo.clusteringKeys.push(c);
	tableInfo.clusteringOrder.push(
	c.type.options.reversed ? "DESC" : "ASC",
	);
	}
	}
	tableInfo.isCompact = !!tableRow["is_dense"];
	if (!tableInfo.isCompact) {
	// is_dense column does not exist in previous versions of Cassandra
	// also, compact pk, ck and val appear as is_dense false
	// clusteringKeys != comparator types - 1
	// or not composite (comparator)
	tableInfo.isCompact =
	// clustering keys are not marked as composite
	!comparator.isComposite ||
	// only 1 column not part of the partition or clustering keys
	(!comparator.hasCollections &&
	tableInfo.clusteringKeys.length !==
	comparator.types.length - 1);
	}
	name = tableRow["value_alias"];
	if (tableInfo.isCompact && name && !columnsKeyed[name]) {
	// additional column in C* 1.2 as value_alias
	c = {
	name: name,
	type: encoder.parseFqTypeName(tableRow["default_validator"]),
	};
	tableInfo.columns.push(c);
	columnsKeyed[name] = c;
	}
	tableInfo.columnsByName = columnsKeyed;
	tableInfo.indexes = Index.fromColumnRows(
	columnRows,
	tableInfo.columnsByName,
	);
	}

Either way, the metadata in the JS driver is much more vast than what is provided by the Rust driver.
Additionally, the DSx driver catches the table metadata, unless it's in <drum roll sound> user-provided cache (third argument to the getTable). Btw, it only checks the cache and doesn't insert to it.

Considering the above, what may be the best approach here?

I'm not really sure...
The existing code fetches the metadata with names different from those present on my test Scylla instance:
local_read_repair_chance, while dclocal_read_repair_chance is present as a column.
It also attempts to read metadata not present in my metadata, like populate_io_cache_on_flush.

This is quite a big mess, so I'm starting to consider whether it wouldn't be better to redesign the metadata from scratch.

[adespawn]

getAggregate(s)

Appears to retrieve data from either system_schema.aggregates or system.schema_aggregates tables.
Suffers from callback hell type problem. Very vague type definition of the returned value.
Does not appear to be present in the rust driver.

getFunction(s)

Appears to retrieve data from either system_schema.functions or system_schema.functions tables.
Suffers from callback hell type problem. Very vague type definition of the returned value. Uses user defined cache.
Does not appear to be present in the rust driver.