Review the serialization of tables in triplet format

[ceberam]

The TripletTableSerializer aims at serializing tables to a text format that could be understood by LLMs for downstream applications like RAG.
The implementation connects a column, a row, and the content of the cell in the given column and row to create sentences of the style A, B = C.
The implementation needs to be reviewed, since it takes assumptions on the role of the first column and the first row to build the triplet representation of the table cells.

For instance, the inner table in test/data/doc/rich_table.gt.html:

will be serialized as follows (refer to the ground truth file 0c_out_chunks.json:

cell 0,0, 1 = cell 0,1. cell 1,0, 1 = <em><p>text in italic</p></em>. <ul>\n<li>list item 1</li>\n<li>list item 2</li>\n</ul>, 1 = cell 2,1. cell 3,0, 1 = inner cell 0,0, 1 = inner cell 0,1. inner cell 0,0, 2 = inner cell 0,2. inner cell 1,0, 1 = inner cell 1,1. inner cell 1,0, 2 = inner cell 1,2. <p>Some text in a generic group.</p>\n<p>More text in the group.</p>, 1 = cell 4,1

[wanadzhar913]

Hi there, I'd like to take this on. Let me draw up a plan.

[wanadzhar913]

Hi @ceberam! Just wanted to clarify further regarding:

1. Expected outputs and;
2. Confirm if the file to change is docling-core/transforms/chunker/hierarchical_chunker.py, specifically these lines:

docling-core/docling_core/transforms/chunker/hierarchical_chunker.py

Lines 74 to 101 in 2457411

	# Handle single-column tables
	if table_df.shape[1] == 1:
	# For single-column tables, use first row as column name
	# and remaining rows as values
	col_name = str(table_df.iloc[0, 0]).strip()
	values = [str(val).strip() for val in table_df.iloc[1:, 0].to_list()]
	table_text_parts = [f"{col_name} = {val}" for val in values]
	table_text = ". ".join(table_text_parts)
	parts.append(create_ser_result(text=table_text, span_source=item))
	else:
	# For multi-column tables
	# copy header as first row and shift all rows by one
	table_df.loc[-1] = table_df.columns # type: ignore[call-overload]
	table_df.index = table_df.index + 1
	table_df = table_df.sort_index()
	rows = [str(item).strip() for item in table_df.iloc[:, 0].to_list()]
	cols = [str(item).strip() for item in table_df.iloc[0, :].to_list()]
	nrows = table_df.shape[0]
	ncols = table_df.shape[1]
	table_text_parts = [
	f"{rows[i]}, {cols[j]} = {str(table_df.iloc[i, j]).strip()}"
	for i in range(1, nrows)
	for j in range(1, ncols)
	]
	table_text = ". ".join(table_text_parts)
	parts.append(create_ser_result(text=table_text, span_source=item))

My changes will essentially:

1. Make the serializer check first whether the table actually has row/column headers before deciding how to format the output.
2. We then get output depending on:
   • Table has only column headers (cells explicitly marked as headers in the data) → use the original triplet format: row_value, col_header = cell_value.
   • Table has only row headers → use the row header values for rows but label columns positionally: row_header, Column 1 = cell_value.
   • Has both -> use the original triplet format: row_value, col_header = cell_value.
   • Table has no headers at all → use straightforward positional labels: Row 0, Column 0 = cell 0,0. Row 0, Column 1 = cell 0,1. ...

Here's some sample code to visualize. Say we have the below dataframe extracted from Docling:

df = pd.DataFrame({
    'Country': ['Italy', 'Canada', 'Switzerland'],
    'Continent': ['Europe', 'North America', 'Europe'],
    'Language': ['Italian', 'English, French', 'Swiss']
})

df.head(3)

index	Country	Continent	Language
0	Italy	Europe	Italian
1	Canada	North America	English, French
2	Switzerland	Europe	Swiss

For some function, here are the example outputs:

def build_triplets(
    table_df: pd.DataFrame,
    *,
    has_col_headers: bool,
    has_row_headers: bool,
) -> list[str]:
    ...

print(build_triplets(table_df=df, has_col_headers=True, has_row_headers=False))
>> ['Italy, Continent = Europe', 'Italy, Language = Italian', 'Canada, Continent = North America', 'Canada, Language = English, French', 'Switzerland, Continent = Europe', 'Switzerland, Language = Swiss']

print(build_triplets(table_df=df, has_col_headers=False, has_row_headers=True))
>> ['Italy, Column 1 = Europe', 'Italy, Column 2 = Italian', 'Canada, Column 1 = North America', 'Canada, Column 2 = English, French', 'Switzerland, Column 1 = Europe', 'Switzerland, Column 2 = Swiss']

print(build_triplets(table_df=df, has_col_headers=True, has_row_headers=True))
>> ['Italy, Continent = Europe', 'Italy, Language = Italian', 'Canada, Continent = North America', 'Canada, Language = English, French', 'Switzerland, Continent = Europe', 'Switzerland, Language = Swiss']

print(build_triplets(table_df=df, has_col_headers=False, has_row_headers=False))
>> ['Row 0, Column 0 = Italy', 'Row 0, Column 1 = Europe', 'Row 0, Column 2 = Italian', 'Row 1, Column 0 = Canada', 'Row 1, Column 1 = North America', 'Row 1, Column 2 = English, French', 'Row 2, Column 0 = Switzerland', 'Row 2, Column 1 = Europe', 'Row 2, Column 2 = Swiss']

Do let me know what the right output should be, especially for when there's no headers for both (has_col_headers=False, has_row_headers=False) and if we only have row headers (has_row_headers=True)?

[passionworkeer]

问题在于当前实现假设第一列是行标题、第一行是列标题。建议：1) 检查是否所有表格都遵循这个约定；2) 对于没有表头或表头在别处的表格，需要一个回退逻辑；3) 可以考虑使用 HTML 的 标签来识别真正的表头单元格，而不是依赖位置；4) 添加更多测试用例覆盖不同表格结构（如嵌套表格、合并单元格等）。