Add support for decimal types

[brendanyounger]

Can now read parquet files with decimal types.

[za-arthur]

Thank you @brendanyounger for the new feature. Could you also add tests and fix README?

[za-arthur]

Shouldn't be DECIMAL128 and DECIMAL256 used here similar to bytes_to_postgres_type?

[Magmatrix]

@brendanyounger Hi, I tested this in PG 15 but couldn't get it to work fully.

I have tried creating the foreign table with price column of type NUMERIC, and later with type FLOAT. Regardless of the type used, price always show up as 0. Example:

parquet=# select price,price_m,price_e from "Order" limit 10;
 price | price_m | price_e
-------+---------+---------
     0 |    6000 |      -3
     0 |    2320 |      -3
     0 |   10000 |      -3
     0 |    1530 |      -3
     0 |   66200 |      -3
     0 |    3000 |      -3
     0 |    4010 |      -3
     0 |     232 |      -3
     0 |    7420 |      -3
     0 |  291940 |      -3
(10 rows)

However, if I cast it from NUMERIC to FLOAT (or vice versa), all price columns EXCEPT the first row show up correctly:

parquet=# select price::NUMERIC,price_m,price_e from "Order" limit 10;
 price  | price_m | price_e
--------+---------+---------
      0 |    6000 |      -3
   2.32 |    2320 |      -3
     10 |   10000 |      -3
   1.53 |    1530 |      -3
   66.2 |   66200 |      -3
      3 |    3000 |      -3
   4.01 |    4010 |      -3
  0.232 |     232 |      -3
   7.42 |    7420 |      -3
 291.94 |  291940 |      -3
(10 rows)

The cast obviously triggers something (after the first row), but I have no idea where to start looking for this bug. Any ideas?

[Magmatrix]

NB: If I count the number of orders with price==0 this happens:

parquet=# select count(*) from "Order" where price = 0;
ERROR:  parquet_fdw: failed to extract row groups from Parquet file: row group filter match failed: cache lookup failed for function 0 ('/srv/parquet-test/Order/2025/02/10/10/Order.parquet')

But with a cast, it finds 105 cases where the price wrongly shows up as 0:

parquet=# select count(*) from "Order" where price::numeric = 0;
 count
-------
   105

This table is constructed from exactly 105 files, so it's the first price in each file that gets misinterpreted when using a cast (or every row if not using a cast). I guess this might help in finding the bug.

[Magmatrix]

And a bit more testing shows that if you add another cast, then the results becomes random (different results every time):

parquet=# select count(*) from "Order" where price::numeric::float = 0;
 count
--------
 477094
(1 row)

parquet=# select count(*) from "Order" where price::numeric::float = 0;
 count
--------
 459599
(1 row)

parquet=# select count(*) from "Order" where price::numeric::float <> 0;
 count
--------
 405207
(1 row)

parquet=# select count(*) from "Order" where price::numeric::float <> 0;
 count
--------
 427107
(1 row)

[MisterRaindrop]

Converting the type to a string and then processing it with numeric_in is relatively inefficient, right? At least in our tests, it is significantly slower than Spark’s parsing. Are there better solutions to handle this type of conversion?