For the query like below:
select msg from data2 where t."$date" < now() limit 1;
The initial filter operation generated by Presto is LESS_THAN(CAST(DEREFERENCE(t, 0)), 7204114125742080), where the CAST(DEREFERENCE(t, 0)) is cast the first (0-th) nested field of t to the same type of the second operands, which is of TIMESTAMP WITH TIMEZONE data type. It is a packed 64-bit value used for TIMESTAMP WITH TIME ZONE, where the upper bits are epoch milliseconds (shifted left by 12 bits) and the lower 12 bits are the time-zone key.
In ClpFilterToKqlConverter.handleLogicalBinary(), the ClpExpression doesn't accept these two kinds of operands so they are just set to Optional.empty() and there is no pushdown generated.
ClpExpression leftExpression = left.accept(this, null);
ClpExpression rightExpression = right.accept(this, null);
Also, the result of this query is empty which is incorrect.
presto:default> select msg from data2 where t."$date" < now() limit 1;
msg
-----
(0 rows)
Query 20250925_174625_00005_n2nnx, FINISHED, 1 node
Splits: 2 total, 2 done (100.00%)
[Latency: client-side: 0:01, server-side: 0:01] [683K rows, 0B] [1.24M rows/s, 0B/s]
I validated that the TIMESTAMP type can be converted to TIMESTAMP WITH TIMEZONE type, so this is not caused by we compare a TIMESTAMP column with a TIMESTAMP WITH TIMEZONE value, and it is our own bug (see here, the from_unixtime returns a TIMESTAMP type data):
presto:default> select (from_unixtime(1700000000) AT TIME ZONE 'America/Toronto');
_col0
-----------------------------------------
2023-11-14 17:13:20.000 America/Toronto
(1 row)
For the query like below:
The initial filter operation generated by Presto is
LESS_THAN(CAST(DEREFERENCE(t, 0)), 7204114125742080), where theCAST(DEREFERENCE(t, 0))is cast the first (0-th) nested field oftto the same type of the second operands, which is ofTIMESTAMP WITH TIMEZONEdata type. It is a packed 64-bit value used forTIMESTAMP WITH TIME ZONE, where the upper bits are epoch milliseconds (shifted left by 12 bits) and the lower 12 bits are the time-zone key.In
ClpFilterToKqlConverter.handleLogicalBinary(), theClpExpressiondoesn't accept these two kinds of operands so they are just set toOptional.empty()and there is no pushdown generated.Also, the result of this query is empty which is incorrect.
I validated that the
TIMESTAMPtype can be converted toTIMESTAMP WITH TIMEZONEtype, so this is not caused by we compare aTIMESTAMPcolumn with aTIMESTAMP WITH TIMEZONEvalue, and it is our own bug (see here, thefrom_unixtimereturns aTIMESTAMPtype data):