Column pruning not handled properly

[Novemser]

SQL:

select id_dt from full_data_type_table where id_dt not in (select id_dt from full_data_type_table  where tp_decimal <> 1E10)

Plan:

== Physical Plan ==
*Project [id_dt#138L]
+- BroadcastNestedLoopJoin BuildRight, LeftAnti, ((id_dt#138L = id_dt#138L#258L) || isnull((id_dt#138L = id_dt#138L#258L)))
   :- TiDB CoprocessorRDD{[table: full_data_type_table] , Ranges: Start:[-9223372036854775808], End: [9223372036854775807], Columns: [id_dt], [tp_varchar], [tp_datetime], [tp_blob], [tp_binary], [tp_date], [tp_timestamp], [tp_year], [tp_bigint], [tp_decimal], [tp_double], [tp_float], [tp_int], [tp_mediumint], [tp_real], [tp_smallint], [tp_tinyint], [tp_char], [tp_nvarchar], [tp_longtext], [tp_mediumtext], [tp_text], [tp_tinytext], [tp_bit], [tp_time], [tp_enum], [tp_set]}
   +- BroadcastExchange IdentityBroadcastMode
      +- *Project [id_dt#138L AS id_dt#138L#258L]
         +- TiDB CoprocessorRDD{[table: full_data_type_table] , Ranges: Start:[-9223372036854775808], End: [9223372036854775807], Columns: [id_dt], [tp_decimal], Aggregates: Not(IsNull([tp_decimal])), <>([tp_decimal], 10000000000)}

Seem we didn't properly prune the plan and pushed down too many things.

[Novemser]

@birdstorm After fixing this issue, regression testing is needed.

[Novemser]

Schema

mysql> desc full_data_type_table;
+---------------+-----------------------+------+------+-------------------+-------+
| Field         | Type                  | Null | Key  | Default           | Extra |
+---------------+-----------------------+------+------+-------------------+-------+
| id_dt         | int(11)               | NO   | PRI  | NULL              |       |
| tp_varchar    | varchar(45)           | YES  |      | NULL              |       |
| tp_datetime   | datetime              | YES  |      | CURRENT_TIMESTAMP |       |
| tp_blob       | blob                  | YES  |      | NULL              |       |
| tp_binary     | binary(2)             | YES  |      | NULL              |       |
| tp_date       | date                  | YES  |      | NULL              |       |
| tp_timestamp  | timestamp             | NO   |      | CURRENT_TIMESTAMP |       |
| tp_year       | year UNSIGNED         | YES  |      | NULL              |       |
| tp_bigint     | bigint(20)            | YES  |      | NULL              |       |
| tp_decimal    | decimal               | YES  |      | NULL              |       |
| tp_double     | double                | YES  |      | NULL              |       |
| tp_float      | float                 | YES  |      | NULL              |       |
| tp_int        | int(11)               | YES  | MUL  | NULL              |       |
| tp_mediumint  | mediumint(9)          | YES  |      | NULL              |       |
| tp_real       | double                | YES  |      | NULL              |       |
| tp_smallint   | smallint(6)           | YES  |      | NULL              |       |
| tp_tinyint    | tinyint(4)            | YES  |      | NULL              |       |
| tp_char       | char(10)              | YES  |      | NULL              |       |
| tp_nvarchar   | varchar(40)           | YES  |      | NULL              |       |
| tp_longtext   | longtext              | YES  |      | NULL              |       |
| tp_mediumtext | mediumtext            | YES  |      | NULL              |       |
| tp_text       | text                  | YES  |      | NULL              |       |
| tp_tinytext   | tinytext              | YES  |      | NULL              |       |
| tp_bit        | bit(1)                | YES  |      | NULL              |       |
| tp_time       | time                  | YES  |      | NULL              |       |
| tp_enum       | enum('1','2','3','4') | YES  |      | NULL              |       |
| tp_set        | set('a','b','c','d')  | YES  |      | NULL              |       |
+---------------+-----------------------+------+------+-------------------+-------+
27 rows in set (0.01 sec)