Skip to content

Always chunk Avro on uncompressed bytes#4165

Merged
ilidemi merged 1 commit intomainfrom
always-chunk-uncompressed
Apr 14, 2026
Merged

Always chunk Avro on uncompressed bytes#4165
ilidemi merged 1 commit intomainfrom
always-chunk-uncompressed

Conversation

@ilidemi
Copy link
Copy Markdown
Contributor

@ilidemi ilidemi commented Apr 11, 2026 •
edited
Loading

It's been running for some time without any issues

Closes DBI-581

@ilidemi ilidemi requested a review from jgao54 April 11, 2026 05:38
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 11, 2026

❌ 57 Tests Failed:

Tests completed Failed Passed Skipped
2198 57 2141 183
View the top 3 failed test(s) by shortest run time 
github.com/PeerDB-io/peerdb/flow/e2e::TestGenericBQ
Stack Traces | 0s run time 
=== RUN   TestGenericBQ
=== PAUSE TestGenericBQ
=== CONT  TestGenericBQ
--- FAIL: TestGenericBQ (0.00s)
2026/04/11 05:43:24 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:43:24 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg
Stack Traces | 0.01s run time 
=== RUN   TestApiPg
=== PAUSE TestApiPg
=== CONT  TestApiPg
--- FAIL: TestApiPg (0.01s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH
Stack Traces | 0.01s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH (0.01s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuitePG_CH
Stack Traces | 0.01s run time 
=== RUN   TestPeerFlowE2ETestSuitePG_CH
=== PAUSE TestPeerFlowE2ETestSuitePG_CH
=== CONT  TestPeerFlowE2ETestSuitePG_CH
--- FAIL: TestPeerFlowE2ETestSuitePG_CH (0.01s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_InitialLoadOnly_No_Primary_Key
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_InitialLoadOnly_No_Primary_Key
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_InitialLoadOnly_No_Primary_Key
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_InitialLoadOnly_No_Primary_Key
    clickhouse_test.go:2155: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_InitialLoadOnly_No_Primary_Key (31.07s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Chunking_Initial_Load_Parts_Per_Partition
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Chunking_Initial_Load_Parts_Per_Partition
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Chunking_Initial_Load_Parts_Per_Partition
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Chunking_Initial_Load_Parts_Per_Partition
2026/04/11 05:52:54 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:54 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:405: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Chunking_Initial_Load_Parts_Per_Partition (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Composite_PKey
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Composite_PKey
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Composite_PKey
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Composite_PKey
2026/04/11 05:52:54 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:54 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:3566: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Composite_PKey (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Extra_CH_Columns
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Extra_CH_Columns
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Extra_CH_Columns
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Extra_CH_Columns
2026/04/11 05:52:54 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:54 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:2577: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Extra_CH_Columns (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Bit
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Bit
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Bit
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Bit
2026/04/11 05:52:52 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:157: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Bit (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Enum
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Enum
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Enum
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Enum
2026/04/11 05:52:51 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:51 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:264: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Enum (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Geometric_Types
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Geometric_Types
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Geometric_Types
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Geometric_Types
2026/04/11 05:52:51 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:51 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:406: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Geometric_Types (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_NumToVarcharCoercion
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_NumToVarcharCoercion
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_NumToVarcharCoercion
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_NumToVarcharCoercion
2026/04/11 05:52:41 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:41 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:962: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_NumToVarcharCoercion (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Time
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Time
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Time
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Time
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:109: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
2026/04/11 05:52:54 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Time (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Vector
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Vector
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Vector
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Vector
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:311: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Vector (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullEngine
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullEngine
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullEngine
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullEngine
2026/04/11 05:52:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:2611: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullEngine (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableColumnSetting
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableColumnSetting
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableColumnSetting
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableColumnSetting
2026/04/11 05:52:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:285: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableColumnSetting (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableMirrorSetting
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableMirrorSetting
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableMirrorSetting
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableMirrorSetting
2026/04/11 05:52:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:238: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_NullableMirrorSetting (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Nullable_Schema_Change
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Nullable_Schema_Change
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Nullable_Schema_Change
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Nullable_Schema_Change
2026/04/11 05:52:20 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:20 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:1947: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Nullable_Schema_Change (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionBy
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionBy
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionBy
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionBy
2026/04/11 05:52:20 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:20 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:3024: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionBy (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionByExpr
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionByExpr
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionByExpr
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionByExpr
2026/04/11 05:52:20 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:20 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:3076: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_PartitionByExpr (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Empty
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Empty
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Empty
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Empty
2026/04/11 05:52:17 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:17 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:2924: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Empty (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_SchemaAsColumn
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_SchemaAsColumn
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_SchemaAsColumn
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_SchemaAsColumn
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:2529: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_SchemaAsColumn (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_SkipSnapshotExport
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_SkipSnapshotExport
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_SkipSnapshotExport
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_SkipSnapshotExport
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:2497: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_SkipSnapshotExport (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Disabled
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Disabled
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Disabled
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Disabled
2026/04/11 05:51:50 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:50 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:322: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Disabled (31.08s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Addition_Removal
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Addition_Removal
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Addition_Removal
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Addition_Removal
2026/04/11 05:51:49 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:49 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:116: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Addition_Removal (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Column_Exclusion
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Column_Exclusion
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Column_Exclusion
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Column_Exclusion
    clickhouse_test.go:1900: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Column_Exclusion (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Blobs
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Blobs
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Blobs
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Blobs
2026/04/11 05:52:52 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:212: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Blobs (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_DateCoercion
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_DateCoercion
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_DateCoercion
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_DateCoercion
2026/04/11 05:52:52 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:1030: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_DateCoercion (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Numbers
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Numbers
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Numbers
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Numbers
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:352: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Numbers (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Specific_Geometric_Types
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Specific_Geometric_Types
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Specific_Geometric_Types
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Specific_Geometric_Types
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:23 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:519: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Specific_Geometric_Types (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Time64
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Time64
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Time64
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Time64
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:1364: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Time64 (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_UnsignedMySQL
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_UnsignedMySQL
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_UnsignedMySQL
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_UnsignedMySQL
2026/04/11 05:51:50 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:50 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:57: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_UnsignedMySQL (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Enabled
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Enabled
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Enabled
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Enabled
2026/04/11 05:51:50 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:50 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:363: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Update_PKey_Env_Enabled (31.09s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_JSON_CH
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_JSON_CH
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_JSON_CH
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_JSON_CH
2026/04/11 05:52:53 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:53 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:1702: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_JSON_CH (31.10s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_GhOst_Schema_Changes
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_GhOst_Schema_Changes
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_GhOst_Schema_Changes
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_GhOst_Schema_Changes
2026/04/11 05:52:49 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:49 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_mysql_test.go:776: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_GhOst_Schema_Changes (31.10s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Schema_Changes
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Schema_Changes
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Schema_Changes
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Schema_Changes
    clickhouse_mysql_test.go:657: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_MySQL_Schema_Changes (31.10s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCancelAddCancel
Stack Traces | 31.1s run time 
=== RUN   TestApiPg/TestCancelAddCancel
=== PAUSE TestApiPg/TestCancelAddCancel
=== CONT  TestApiPg/TestCancelAddCancel
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    cancel_table_addition_test.go:787: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    api_test.go:46: begin tearing down postgres schema api_t0c0vglh
--- FAIL: TestApiPg/TestCancelAddCancel (31.14s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCancelErrorOnPostgresZeroOIDs
Stack Traces | 31.1s run time 
=== RUN   TestApiPg/TestCancelErrorOnPostgresZeroOIDs
=== PAUSE TestApiPg/TestCancelErrorOnPostgresZeroOIDs
=== CONT  TestApiPg/TestCancelErrorOnPostgresZeroOIDs
2026/04/11 05:51:49 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:49 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    cancel_table_addition_test.go:1024: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    api_test.go:46: begin tearing down postgres schema api_txjue8qf
--- FAIL: TestApiPg/TestCancelErrorOnPostgresZeroOIDs (31.14s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCreateCDCFlowAttachIdempotentAfterContinueAsNew
Stack Traces | 31.1s run time 
=== RUN   TestApiPg/TestCreateCDCFlowAttachIdempotentAfterContinueAsNew
=== PAUSE TestApiPg/TestCreateCDCFlowAttachIdempotentAfterContinueAsNew
=== CONT  TestApiPg/TestCreateCDCFlowAttachIdempotentAfterContinueAsNew
2026/04/11 05:51:18 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:18 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2735: UNEXPECTED STATUS TIMEOUT STATUS_SNAPSHOT
    api_test.go:46: begin tearing down postgres schema api_m6gwuee6
--- FAIL: TestApiPg/TestCreateCDCFlowAttachIdempotentAfterContinueAsNew (31.14s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCancelTableAdditionDuringSetupFlow
Stack Traces | 31.1s run time 
=== RUN   TestApiPg/TestCancelTableAdditionDuringSetupFlow
=== PAUSE TestApiPg/TestCancelTableAdditionDuringSetupFlow
=== CONT  TestApiPg/TestCancelTableAdditionDuringSetupFlow
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    cancel_table_addition_test.go:1163: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    api_test.go:46: begin tearing down postgres schema api_r0vdi0pm
--- FAIL: TestApiPg/TestCancelTableAdditionDuringSetupFlow (31.15s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCancelTableAdditionRemoveAddRemove
Stack Traces | 31.1s run time 
=== RUN   TestApiPg/TestCancelTableAdditionRemoveAddRemove
=== PAUSE TestApiPg/TestCancelTableAdditionRemoveAddRemove
=== CONT  TestApiPg/TestCancelTableAdditionRemoveAddRemove
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    cancel_table_addition_test.go:636: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    api_test.go:46: begin tearing down postgres schema api_wfiejeys
--- FAIL: TestApiPg/TestCancelTableAdditionRemoveAddRemove (31.15s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCreateCDCFlowAttachConcurrentRequests
Stack Traces | 31.1s run time 
=== RUN   TestApiPg/TestCreateCDCFlowAttachConcurrentRequests
=== PAUSE TestApiPg/TestCreateCDCFlowAttachConcurrentRequests
=== CONT  TestApiPg/TestCreateCDCFlowAttachConcurrentRequests
2026/04/11 05:51:19 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:19 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2391: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    api_test.go:46: begin tearing down postgres schema api_aqjsecdj
--- FAIL: TestApiPg/TestCreateCDCFlowAttachConcurrentRequests (31.15s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCreateCDCFlowAttachSequentialRequests
Stack Traces | 31.1s run time 
=== RUN   TestApiPg/TestCreateCDCFlowAttachSequentialRequests
=== PAUSE TestApiPg/TestCreateCDCFlowAttachSequentialRequests
=== CONT  TestApiPg/TestCreateCDCFlowAttachSequentialRequests
2026/04/11 05:51:18 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:18 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2551: UNEXPECTED STATUS TIMEOUT STATUS_SNAPSHOT
    api_test.go:46: begin tearing down postgres schema api_hoiyto6o
--- FAIL: TestApiPg/TestCreateCDCFlowAttachSequentialRequests (31.15s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Integer
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Integer
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Integer
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Integer
2026/04/11 05:52:10 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:52:10 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:2817: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Integer (31.15s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Null
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Null
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Null
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Null
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:2981: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Null (31.15s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Timestamp
Stack Traces | 31.1s run time 
=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Timestamp
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Timestamp
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Timestamp
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:2877: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH/Test_Partition_Key_Timestamp (31.15s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCreateCDCFlowAttachExternalFlowEntry
Stack Traces | 31.2s run time 
=== RUN   TestApiPg/TestCreateCDCFlowAttachExternalFlowEntry
=== PAUSE TestApiPg/TestCreateCDCFlowAttachExternalFlowEntry
=== CONT  TestApiPg/TestCreateCDCFlowAttachExternalFlowEntry
2026/04/11 05:51:18 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:18 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2624: UNEXPECTED STATUS TIMEOUT STATUS_SNAPSHOT
    api_test.go:46: begin tearing down postgres schema api_4n1xbae5
--- FAIL: TestApiPg/TestCreateCDCFlowAttachExternalFlowEntry (31.18s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCancelTableAddition_NoRemovalAssumed
Stack Traces | 31.2s run time 
=== RUN   TestApiPg/TestCancelTableAddition_NoRemovalAssumed
=== PAUSE TestApiPg/TestCancelTableAddition_NoRemovalAssumed
=== CONT  TestApiPg/TestCancelTableAddition_NoRemovalAssumed
2026/04/11 05:51:21 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    cancel_table_addition_test.go:361: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    api_test.go:46: begin tearing down postgres schema api_ejuafiuw
--- FAIL: TestApiPg/TestCancelTableAddition_NoRemovalAssumed (31.19s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCancelTableAddition_WithRemoval
Stack Traces | 31.2s run time 
=== RUN   TestApiPg/TestCancelTableAddition_WithRemoval
=== PAUSE TestApiPg/TestCancelTableAddition_WithRemoval
=== CONT  TestApiPg/TestCancelTableAddition_WithRemoval
    cancel_table_addition_test.go:361: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    api_test.go:46: begin tearing down postgres schema api_bz4bcwml
2026/04/11 05:51:52 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
--- FAIL: TestApiPg/TestCancelTableAddition_WithRemoval (31.20s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCancelTableAddition_NoRemovalAssumedWithRemoval
Stack Traces | 31.2s run time 
=== RUN   TestApiPg/TestCancelTableAddition_NoRemovalAssumedWithRemoval
=== PAUSE TestApiPg/TestCancelTableAddition_NoRemovalAssumedWithRemoval
=== CONT  TestApiPg/TestCancelTableAddition_NoRemovalAssumedWithRemoval
2026/04/11 05:51:19 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:19 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    cancel_table_addition_test.go:361: UNEXPECTED STATUS TIMEOUT STATUS_SNAPSHOT
    api_test.go:46: begin tearing down postgres schema api_caq10odo
--- FAIL: TestApiPg/TestCancelTableAddition_NoRemovalAssumedWithRemoval (31.22s)

github.com/PeerDB-io/peerdb/flow/e2e::TestGenericBQ/Test_Inheritance_Table_With_Dynamic_Setting
Stack Traces | 33.5s run time 
=== RUN   TestGenericBQ/Test_Inheritance_Table_With_Dynamic_Setting
=== PAUSE TestGenericBQ/Test_Inheritance_Table_With_Dynamic_Setting
=== CONT  TestGenericBQ/Test_Inheritance_Table_With_Dynamic_Setting
2026/04/11 05:42:03 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column FROM e2e_test_bq_h5p1j5jl_20260411054133.\"test_lost_column_bug\" ORDER BY id"
2026/04/11 05:42:03 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column FROM e2e_test_bq_h5p1j5jl_20260411054133.\"test_lost_column_bug\" ORDER BY id"
2026/04/11 05:42:03 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_8563865259168345730 CURSOR FOR SELECT id,good_column FROM e2e_test_bq_h5p1j5jl_20260411054133.\"test_lost_column_bug\" ORDER BY id" args=[]
2026/04/11 05:42:03 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column FROM e2e_test_bq_h5p1j5jl_20260411054133.\"test_lost_column_bug\" ORDER BY id" channelLen=0
2026/04/11 05:42:03 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8563865259168345730
2026/04/11 05:42:03 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8563865259168345730 records=1 bytes=9 channelLen=0
2026/04/11 05:42:03 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column FROM e2e_test_bq_h5p1j5jl_20260411054133.\"test_lost_column_bug\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/11 05:42:03 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8563865259168345730
2026/04/11 05:42:03 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8563865259168345730 records=0 bytes=0 channelLen=0
2026/04/11 05:42:03 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column FROM e2e_test_bq_h5p1j5jl_20260411054133.\"test_lost_column_bug\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:42:03 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:42:03 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column FROM e2e_test_bq_h5p1j5jl_20260411054133.\"test_lost_column_bug\" ORDER BY id" rows=1 bytes=9 channelLen=0
    generic_test.go:1103: UNEXPECTED STATUS TIMEOUT STATUS_SNAPSHOT
    bigquery.go:86: begin tearing down postgres schema bq_subg69qi_20260411054203
--- FAIL: TestGenericBQ/Test_Inheritance_Table_With_Dynamic_Setting (33.54s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestCreateCDCFlowAttachCanceledWorkflow
Stack Traces | 60.2s run time 
=== RUN   TestApiPg/TestCreateCDCFlowAttachCanceledWorkflow
=== PAUSE TestApiPg/TestCreateCDCFlowAttachCanceledWorkflow
=== CONT  TestApiPg/TestCreateCDCFlowAttachCanceledWorkflow
2026/04/11 05:51:20 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:20 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2666: WaitFor finish 2026-04-11 05:51:20.584172584 +0000 UTC m=+590.609673578
    api_test.go:2666: UNEXPECTED TIMEOUT finish 2026-04-11 05:52:20.698528012 +0000 UTC m=+650.724029005
    api_test.go:46: begin tearing down postgres schema api_ubtcinko
--- FAIL: TestApiPg/TestCreateCDCFlowAttachCanceledWorkflow (60.22s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestQRep
Stack Traces | 63.2s run time 
=== RUN   TestApiPg/TestQRep
=== PAUSE TestApiPg/TestQRep
=== CONT  TestApiPg/TestQRep
2026/04/11 05:51:06 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:06 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:06 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:06 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2126: WaitFor qrep initial load 2026-04-11 05:51:06.834675633 +0000 UTC m=+576.860176625
2026/04/11 05:51:06 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id"
2026/04/11 05:51:06 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id"
2026/04/11 05:51:06 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_1987681146408319552 CURSOR FOR SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" args=[]
2026/04/11 05:51:06 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" channelLen=0
2026/04/11 05:51:06 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1987681146408319552
2026/04/11 05:51:06 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1987681146408319552 records=1 bytes=9 channelLen=0
2026/04/11 05:51:06 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/11 05:51:06 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1987681146408319552
2026/04/11 05:51:06 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1987681146408319552 records=0 bytes=0 channelLen=0
2026/04/11 05:51:06 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:06 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:06 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=1 bytes=9 channelLen=0
    api_test.go:2126: code: 60, message: Unknown table expression identifier 'qrepapi_api_qdfe3vza' in scope SELECT id, val FROM qrepapi_api_qdfe3vza FINAL WHERE _peerdb_is_deleted = 0 ORDER BY 1 ASC SETTINGS use_query_cache = false
2026/04/11 05:51:06 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id"
2026/04/11 05:51:06 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id"
2026/04/11 05:51:06 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_1580434540469517171 CURSOR FOR SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" args=[]
2026/04/11 05:51:06 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" channelLen=0
2026/04/11 05:51:06 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1580434540469517171
2026/04/11 05:51:06 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1580434540469517171 records=2 bytes=19 channelLen=1
2026/04/11 05:51:06 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" rows=2 bytes=19 channelLen=1
2026/04/11 05:51:06 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1580434540469517171
2026/04/11 05:51:06 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1580434540469517171 records=0 bytes=0 channelLen=0
2026/04/11 05:51:06 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:06 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:06 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" rows=2 bytes=19 channelLen=0
    api_test.go:2131: WaitFor insert post qrep initial load 2026-04-11 05:51:07.853920017 +0000 UTC m=+577.879421010
2026/04/11 05:51:07 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id"
2026/04/11 05:51:07 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id"
2026/04/11 05:51:07 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_2042465824664475662 CURSOR FOR SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" args=[]
2026/04/11 05:51:07 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" channelLen=0
2026/04/11 05:51:07 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2042465824664475662
2026/04/11 05:51:07 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2042465824664475662 records=2 bytes=19 channelLen=1
2026/04/11 05:51:07 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=2 bytes=19 channelLen=1
2026/04/11 05:51:07 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2042465824664475662
2026/04/11 05:51:07 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2042465824664475662 records=0 bytes=0 channelLen=0
2026/04/11 05:51:07 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:07 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:07 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=2 bytes=19 channelLen=0
    api_test.go:2131: q.NumRecords: 2
    api_test.go:2131: other.NumRecords: 1
2026/04/11 05:51:07 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id"
2026/04/11 05:51:07 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id"
2026/04/11 05:51:07 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_5671649729855059666 CURSOR FOR SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" args=[]
2026/04/11 05:51:07 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" channelLen=0
2026/04/11 05:51:07 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5671649729855059666
2026/04/11 05:51:07 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5671649729855059666 records=2 bytes=19 channelLen=1
2026/04/11 05:51:07 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" rows=2 bytes=19 channelLen=1
2026/04/11 05:51:07 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5671649729855059666
2026/04/11 05:51:07 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5671649729855059666 records=0 bytes=0 channelLen=0
2026/04/11 05:51:07 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:07 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:07 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_8376yjxw.\"table1\" ORDER BY id" rows=2 bytes=19 channelLen=0
    api_test.go:2131: q.NumRecords: 2
    api_test.go:2131: other.NumRecords: 1
2026/04/11 05:51:08 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:08 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2150: WaitFor finish 2026-04-11 05:51:09.877923376 +0000 UTC m=+579.903424369
    api_test.go:2150: UNEXPECTED TIMEOUT finish 2026-04-11 05:52:09.990089089 +0000 UTC m=+640.015590072
    api_test.go:46: begin tearing down postgres schema api_qdfe3vza
--- FAIL: TestApiPg/TestQRep (63.24s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestResyncCompleted
Stack Traces | 71.2s run time 
=== RUN   TestApiPg/TestResyncCompleted
=== PAUSE TestApiPg/TestResyncCompleted
=== CONT  TestApiPg/TestResyncCompleted
    api_test.go:1302: WaitFor finish 2026-04-11 05:51:11.86922386 +0000 UTC m=+581.894724853
    api_test.go:1302: Finished Status Completed
2026/04/11 05:51:11 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id"
2026/04/11 05:51:11 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id"
2026/04/11 05:51:11 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_8859011116172099669 CURSOR FOR SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" args=[]
2026/04/11 05:51:11 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" channelLen=0
2026/04/11 05:51:11 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8859011116172099669
2026/04/11 05:51:11 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8859011116172099669 records=1 bytes=9 channelLen=0
2026/04/11 05:51:11 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/11 05:51:11 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8859011116172099669
2026/04/11 05:51:11 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8859011116172099669 records=0 bytes=0 channelLen=0
2026/04/11 05:51:11 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:11 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:11 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/11 05:51:12 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id"
2026/04/11 05:51:12 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id"
2026/04/11 05:51:12 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_2439582860489305022 CURSOR FOR SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" args=[]
2026/04/11 05:51:12 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" channelLen=0
2026/04/11 05:51:12 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2439582860489305022
2026/04/11 05:51:12 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2439582860489305022 records=3 bytes=26 channelLen=2
2026/04/11 05:51:12 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" rows=3 bytes=26 channelLen=2
2026/04/11 05:51:12 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2439582860489305022
2026/04/11 05:51:12 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2439582860489305022 records=0 bytes=0 channelLen=0
2026/04/11 05:51:12 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:12 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:12 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" rows=3 bytes=26 channelLen=0
    api_test.go:1337: WaitFor resync 2026-04-11 05:51:12.342022563 +0000 UTC m=+582.367523556
2026/04/11 05:51:12 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id"
2026/04/11 05:51:12 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id"
2026/04/11 05:51:12 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_18114842374987617105 CURSOR FOR SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" args=[]
2026/04/11 05:51:12 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" channelLen=0
2026/04/11 05:51:12 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_18114842374987617105
2026/04/11 05:51:12 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_18114842374987617105 records=2 bytes=19 channelLen=1
2026/04/11 05:51:12 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=2 bytes=19 channelLen=1
2026/04/11 05:51:12 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_18114842374987617105
2026/04/11 05:51:12 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_18114842374987617105 records=0 bytes=0 channelLen=0
2026/04/11 05:51:12 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:12 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:12 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=2 bytes=19 channelLen=0
    api_test.go:1337: q.NumRecords: 2
    api_test.go:1337: other.NumRecords: 1
    api_test.go:1337: q.NumRecords: 2
    api_test.go:1337: other.NumRecords: 1
2026/04/11 05:51:13 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_ma5xoes3.\"status_test\" ORDER BY id"
2026/04/11 05:51:13 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_ma5xoes3.\"status_test\" ORDER BY id"
2026/04/11 05:51:13 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_17505448926758431931 CURSOR FOR SELECT id,val FROM e2e_test_api_ma5xoes3.\"status_test\" ORDER BY id" args=[]
2026/04/11 05:51:13 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_ma5xoes3.\"status_test\" ORDER BY id" channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_17505448926758431931
2026/04/11 05:51:13 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_17505448926758431931 records=1 bytes=9 channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_ma5xoes3.\"status_test\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_17505448926758431931
2026/04/11 05:51:13 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_17505448926758431931 records=0 bytes=0 channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_ma5xoes3.\"status_test\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:13 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_ma5xoes3.\"status_test\" ORDER BY id" rows=1 bytes=9 channelLen=0
    api_test.go:1337: q.NumRecords: 2
    api_test.go:1337: other.NumRecords: 1
    api_test.go:1337: q.NumRecords: 2
    api_test.go:1337: other.NumRecords: 1
    api_test.go:1337: q.NumRecords: 2
    api_test.go:1337: other.NumRecords: 1
    api_test.go:1340: WaitFor finish 2026-04-11 05:51:17.382899654 +0000 UTC m=+587.408400637
    api_test.go:1340: Finished Status Completed
    api_test.go:1348: WaitFor resync 2 2026-04-11 05:51:17.828622716 +0000 UTC m=+587.854123709
2026/04/11 05:51:17 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id"
2026/04/11 05:51:17 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id"
2026/04/11 05:51:17 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_6323833982969360125 CURSOR FOR SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" args=[]
2026/04/11 05:51:17 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" channelLen=0
2026/04/11 05:51:17 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_6323833982969360125
2026/04/11 05:51:17 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_6323833982969360125 records=2 bytes=19 channelLen=1
2026/04/11 05:51:17 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=2 bytes=19 channelLen=1
2026/04/11 05:51:17 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_6323833982969360125
2026/04/11 05:51:17 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_6323833982969360125 records=0 bytes=0 channelLen=0
2026/04/11 05:51:17 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:17 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:17 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_tk6rk1zx.\"valid\" ORDER BY id" rows=2 bytes=19 channelLen=0
    api_test.go:1351: WaitFor finish 2026-04-11 05:51:17.835266854 +0000 UTC m=+587.860767848
    api_test.go:1351: UNEXPECTED TIMEOUT finish 2026-04-11 05:52:17.951219359 +0000 UTC m=+647.976720342
    api_test.go:46: begin tearing down postgres schema api_tk6rk1zx
--- FAIL: TestApiPg/TestResyncCompleted (71.20s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestDoubleClickCancelTableAddition
Stack Traces | 184s run time 
=== RUN   TestApiPg/TestDoubleClickCancelTableAddition
=== PAUSE TestApiPg/TestDoubleClickCancelTableAddition
=== CONT  TestApiPg/TestDoubleClickCancelTableAddition
2026/04/11 05:51:15 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:15 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:15 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id"
2026/04/11 05:51:15 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id"
2026/04/11 05:51:15 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_1781652461868312995 CURSOR FOR SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" args=[]
2026/04/11 05:51:15 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" channelLen=0
2026/04/11 05:51:15 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1781652461868312995
2026/04/11 05:51:15 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1781652461868312995 records=3 bytes=26 channelLen=2
2026/04/11 05:51:15 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" rows=3 bytes=26 channelLen=2
2026/04/11 05:51:15 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1781652461868312995
2026/04/11 05:51:15 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_1781652461868312995 records=0 bytes=0 channelLen=0
2026/04/11 05:51:15 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:15 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:15 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_0frggoso.\"test_nullengine\" ORDER BY id" rows=3 bytes=26 channelLen=0
    cancel_table_addition_test.go:1367: WaitFor wait for initial load to finish 2026-04-11 05:51:19.371523821 +0000 UTC m=+589.397024814
2026/04/11 05:51:19 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_phdlev4c.\"t1\" ORDER BY id"
2026/04/11 05:51:19 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_phdlev4c.\"t1\" ORDER BY id"
2026/04/11 05:51:19 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_3881843741033228288 CURSOR FOR SELECT id,val FROM e2e_test_api_phdlev4c.\"t1\" ORDER BY id" args=[]
2026/04/11 05:51:19 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_phdlev4c.\"t1\" ORDER BY id" channelLen=0
2026/04/11 05:51:19 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_3881843741033228288
2026/04/11 05:51:19 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_3881843741033228288 records=1 bytes=9 channelLen=0
2026/04/11 05:51:19 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_phdlev4c.\"t1\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/11 05:51:19 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_3881843741033228288
2026/04/11 05:51:19 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_3881843741033228288 records=0 bytes=0 channelLen=0
2026/04/11 05:51:19 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_phdlev4c.\"t1\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:19 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:19 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_phdlev4c.\"t1\" ORDER BY id" rows=1 bytes=9 channelLen=0
    cancel_table_addition_test.go:1378: WaitFor wait for pause 2026-04-11 05:51:19.394179952 +0000 UTC m=+589.419680935
    cancel_table_addition_test.go:1378: UNEXPECTED TIMEOUT wait for pause 2026-04-11 05:54:19.579831452 +0000 UTC m=+769.605332435
    cancel_table_addition_test.go:1310: begin tearing down postgres schema double_cancel_api_phdlev4c
    api_test.go:46: begin tearing down postgres schema api_phdlev4c
--- FAIL: TestApiPg/TestDoubleClickCancelTableAddition (184.39s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestEditTablesBeforeResync
Stack Traces | 186s run time 
=== RUN   TestApiPg/TestEditTablesBeforeResync
=== PAUSE TestApiPg/TestEditTablesBeforeResync
=== CONT  TestApiPg/TestEditTablesBeforeResync
2026/04/11 05:51:09 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:09 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:51:09 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id"
2026/04/11 05:51:09 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id"
2026/04/11 05:51:09 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_960402774844791573 CURSOR FOR SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" args=[]
2026/04/11 05:51:09 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" channelLen=0
2026/04/11 05:51:09 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_960402774844791573
2026/04/11 05:51:09 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_960402774844791573 records=2 bytes=19 channelLen=1
2026/04/11 05:51:09 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=2 bytes=19 channelLen=1
2026/04/11 05:51:09 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_960402774844791573
2026/04/11 05:51:09 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_960402774844791573 records=0 bytes=0 channelLen=0
2026/04/11 05:51:09 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:09 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:09 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_qdfe3vza.\"qrepapi_api_qdfe3vza\" ORDER BY id" rows=2 bytes=19 channelLen=0
    api_test.go:1636: WaitFor wait for initial load to finish 2026-04-11 05:51:13.93448518 +0000 UTC m=+583.959986163
2026/04/11 05:51:13 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7axpiror.\"original\" ORDER BY id"
2026/04/11 05:51:13 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7axpiror.\"original\" ORDER BY id"
2026/04/11 05:51:13 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_5464087110120811667 CURSOR FOR SELECT id,val FROM e2e_test_api_7axpiror.\"original\" ORDER BY id" args=[]
2026/04/11 05:51:13 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7axpiror.\"original\" ORDER BY id" channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5464087110120811667
2026/04/11 05:51:13 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5464087110120811667 records=1 bytes=9 channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7axpiror.\"original\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5464087110120811667
2026/04/11 05:51:13 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5464087110120811667 records=0 bytes=0 channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7axpiror.\"original\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:51:13 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:51:13 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7axpiror.\"original\" ORDER BY id" rows=1 bytes=9 channelLen=0
    api_test.go:1647: WaitFor wait for pause for add table 2026-04-11 05:51:13.952810102 +0000 UTC m=+583.978311085
    api_test.go:1667: WaitFor wait for table addition to finish 2026-04-11 05:51:15.96082447 +0000 UTC m=+585.986325463
    api_test.go:1667: UNEXPECTED TIMEOUT wait for table addition to finish 2026-04-11 05:54:16.127322842 +0000 UTC m=+766.152823825
    api_test.go:46: begin tearing down postgres schema api_7axpiror
--- FAIL: TestApiPg/TestEditTablesBeforeResync (186.32s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuitePG_CH/Test_Normalize_Metadata_With_Retry
Stack Traces | 337s run time 
=== RUN   TestPeerFlowE2ETestSuitePG_CH/Test_Normalize_Metadata_With_Retry
=== PAUSE TestPeerFlowE2ETestSuitePG_CH/Test_Normalize_Metadata_With_Retry
=== CONT  TestPeerFlowE2ETestSuitePG_CH/Test_Normalize_Metadata_With_Retry
2026/04/11 05:50:08 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:50:08 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/11 05:50:09 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,\"Ac2\",\"Ac3\",c4 FROM e2e_test_pgch_iflidwkl.\"test_nullable_sc_ch_replident_full\" ORDER BY id"
2026/04/11 05:50:09 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,\"Ac2\",\"Ac3\",c4 FROM e2e_test_pgch_iflidwkl.\"test_nullable_sc_ch_replident_full\" ORDER BY id"
2026/04/11 05:50:09 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_4914245048878442029 CURSOR FOR SELECT id,c1,\"Ac2\",\"Ac3\",c4 FROM e2e_test_pgch_iflidwkl.\"test_nullable_sc_ch_replident_full\" ORDER BY id" args=[]
2026/04/11 05:50:09 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,\"Ac2\",\"Ac3\",c4 FROM e2e_test_pgch_iflidwkl.\"test_nullable_sc_ch_replident_full\" ORDER BY id" channelLen=0
2026/04/11 05:50:09 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4914245048878442029
2026/04/11 05:50:09 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4914245048878442029 records=1 bytes=16 channelLen=0
2026/04/11 05:50:09 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,\"Ac2\",\"Ac3\",c4 FROM e2e_test_pgch_iflidwkl.\"test_nullable_sc_ch_replident_full\" ORDER BY id" rows=1 bytes=16 channelLen=0
2026/04/11 05:50:09 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4914245048878442029
2026/04/11 05:50:09 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4914245048878442029 records=0 bytes=0 channelLen=0
2026/04/11 05:50:09 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,\"Ac2\",\"Ac3\",c4 FROM e2e_test_pgch_iflidwkl.\"test_nullable_sc_ch_replident_full\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:09 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:50:09 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,\"Ac2\",\"Ac3\",c4 FROM e2e_test_pgch_iflidwkl.\"test_nullable_sc_ch_replident_full\" ORDER BY id" rows=1 bytes=16 channelLen=0
    clickhouse_test.go:2209: WaitFor waiting on initial 2026-04-11 05:50:14.656216254 +0000 UTC m=+524.681717247
2026/04/11 05:50:14 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_gu054cmi.\"test_normalize_metadata_with_retry_1\" ORDER BY id"
2026/04/11 05:50:14 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_gu054cmi.\"test_normalize_metadata_with_retry_1\" ORDER BY id"
2026/04/11 05:50:14 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_4346904883441606543 CURSOR FOR SELECT id,\"key\" FROM e2e_test_pgch_gu054cmi.\"test_normalize_metadata_with_retry_1\" ORDER BY id" args=[]
2026/04/11 05:50:14 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_gu054cmi.\"test_normalize_metadata_with_retry_1\" ORDER BY id" channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4346904883441606543
2026/04/11 05:50:14 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4346904883441606543 records=4 bytes=29 channelLen=3
2026/04/11 05:50:14 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_gu054cmi.\"test_normalize_metadata_with_retry_1\" ORDER BY id" rows=4 bytes=29 channelLen=3
2026/04/11 05:50:14 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4346904883441606543
2026/04/11 05:50:14 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4346904883441606543 records=0 bytes=0 channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_gu054cmi.\"test_normalize_metadata_with_retry_1\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:50:14 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgch_gu054cmi.\"test_normalize_metadata_with_retry_1\" ORDER BY id" rows=4 bytes=29 channelLen=0
    clickhouse_test.go:2223: WaitFor waiting for first sync to complete 2026-04-11 05:50:14.672421285 +0000 UTC m=+524.697922268
2026/04/11 05:50:14 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'"
2026/04/11 05:50:14 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'"
2026/04/11 05:50:14 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_9194749516629789367 CURSOR FOR SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" args=[]
2026/04/11 05:50:14 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_9194749516629789367
2026/04/11 05:50:14 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_9194749516629789367 records=0 bytes=0 channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:50:14 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:14 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgch_mh9gfcee.\"test_infinite_time\" ORDER BY id"
2026/04/11 05:50:14 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgch_mh9gfcee.\"test_infinite_time\" ORDER BY id"
2026/04/11 05:50:14 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_2741957424632065416 CURSOR FOR SELECT id FROM e2e_test_pgch_mh9gfcee.\"test_infinite_time\" ORDER BY id" args=[]
2026/04/11 05:50:14 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgch_mh9gfcee.\"test_infinite_time\" ORDER BY id" channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2741957424632065416
2026/04/11 05:50:14 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2741957424632065416 records=2 bytes=8 channelLen=1
2026/04/11 05:50:14 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgch_mh9gfcee.\"test_infinite_time\" ORDER BY id" rows=2 bytes=8 channelLen=1
2026/04/11 05:50:14 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2741957424632065416
2026/04/11 05:50:14 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2741957424632065416 records=0 bytes=0 channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgch_mh9gfcee.\"test_infinite_time\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:14 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:50:14 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgch_mh9gfcee.\"test_infinite_time\" ORDER BY id" rows=2 bytes=8 channelLen=0
    clickhouse_test.go:2238: WaitFor waiting for raw table push to complete 2026-04-11 05:50:30.718301086 +0000 UTC m=+540.743802078
    clickhouse_test.go:2251: WaitFor waiting for normalize error 2026-04-11 05:50:30.721711448 +0000 UTC m=+540.747212441
2026/04/11 05:50:30 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('test_normalize_metadata_with_retry_pgch_gu054cmi' in flow_name) > 0\n\t\tAND error_message ILIKE '%error while inserting into target clickhouse table%'"
2026/04/11 05:50:30 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('test_normalize_metadata_with_retry_pgch_gu054cmi' in flow_name) > 0\n\t\tAND error_message ILIKE '%error while inserting into target clickhouse table%'"
2026/04/11 05:50:30 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_4840137256472042525 CURSOR FOR \n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('test_normalize_metadata_with_retry_pgch_gu054cmi' in flow_name) > 0\n\t\tAND error_message ILIKE '%error while inserting into target clickhouse table%'" args=[]
2026/04/11 05:50:30 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('test_normalize_metadata_with_retry_pgch_gu054cmi' in flow_name) > 0\n\t\tAND error_message ILIKE '%error while inserting into target clickhouse table%'" channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4840137256472042525
2026/04/11 05:50:30 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4840137256472042525 records=1 bytes=8 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('test_normalize_metadata_with_retry_pgch_gu054cmi' in flow_name) > 0\n\t\tAND error_message ILIKE '%error while inserting into target clickhouse table%'" rows=1 bytes=8 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4840137256472042525
2026/04/11 05:50:30 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4840137256472042525 records=0 bytes=0 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('test_normalize_metadata_with_retry_pgch_gu054cmi' in flow_name) > 0\n\t\tAND error_message ILIKE '%error while inserting into target clickhouse table%'" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:50:30 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('test_normalize_metadata_with_retry_pgch_gu054cmi' in flow_name) > 0\n\t\tAND error_message ILIKE '%error while inserting into target clickhouse table%'" rows=1 bytes=8 channelLen=0
    clickhouse_test.go:2264: WaitFor waiting for second sync to complete 2026-04-11 05:50:30.732514858 +0000 UTC m=+540.758015851
2026/04/11 05:50:30 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'"
2026/04/11 05:50:30 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'"
2026/04/11 05:50:30 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_14784667678060310534 CURSOR FOR SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" args=[]
2026/04/11 05:50:30 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_14784667678060310534
2026/04/11 05:50:30 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_14784667678060310534 records=1 bytes=8 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" rows=1 bytes=8 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_14784667678060310534
2026/04/11 05:50:30 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_14784667678060310534 records=0 bytes=0 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:50:30 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgch_gu054cmi'" rows=1 bytes=8 channelLen=0
2026/04/11 05:50:30 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,t FROM e2e_test_pgch_77lbvh88.\"test_exclude_ch\" ORDER BY id"
2026/04/11 05:50:30 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,t FROM e2e_test_pgch_77lbvh88.\"test_exclude_ch\" ORDER BY id"
2026/04/11 05:50:30 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_12810985032408526073 CURSOR FOR SELECT id,c1,t FROM e2e_test_pgch_77lbvh88.\"test_exclude_ch\" ORDER BY id" args=[]
2026/04/11 05:50:30 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,t FROM e2e_test_pgch_77lbvh88.\"test_exclude_ch\" ORDER BY id" channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_12810985032408526073
2026/04/11 05:50:30 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_12810985032408526073 records=4 bytes=80 channelLen=3
2026/04/11 05:50:30 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,t FROM e2e_test_pgch_77lbvh88.\"test_exclude_ch\" ORDER BY id" rows=4 bytes=80 channelLen=3
2026/04/11 05:50:30 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_12810985032408526073
2026/04/11 05:50:30 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_12810985032408526073 records=0 bytes=0 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,t FROM e2e_test_pgch_77lbvh88.\"test_exclude_ch\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:30 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:50:30 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,t FROM e2e_test_pgch_77lbvh88.\"test_exclude_ch\" ORDER BY id" rows=4 bytes=80 channelLen=0
    clickhouse_test.go:2278: WaitFor waiting for second raw table push to complete 2026-04-11 05:50:45.781586252 +0000 UTC m=+555.807087245
2026/04/11 05:50:45 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,json,jsonb FROM e2e_test_pgch_dpt4h7al.\"test_json\" ORDER BY id"
2026/04/11 05:50:45 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,json,jsonb FROM e2e_test_pgch_dpt4h7al.\"test_json\" ORDER BY id"
2026/04/11 05:50:45 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_7714055421362300598 CURSOR FOR SELECT id,json,jsonb FROM e2e_test_pgch_dpt4h7al.\"test_json\" ORDER BY id" args=[]
2026/04/11 05:50:45 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,json,jsonb FROM e2e_test_pgch_dpt4h7al.\"test_json\" ORDER BY id" channelLen=0
2026/04/11 05:50:45 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7714055421362300598
2026/04/11 05:50:45 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7714055421362300598 records=14 bytes=1516 channelLen=13
2026/04/11 05:50:45 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,json,jsonb FROM e2e_test_pgch_dpt4h7al.\"test_json\" ORDER BY id" rows=14 bytes=1516 channelLen=13
2026/04/11 05:50:45 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7714055421362300598
2026/04/11 05:50:45 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7714055421362300598 records=0 bytes=0 channelLen=0
2026/04/11 05:50:45 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,json,jsonb FROM e2e_test_pgch_dpt4h7al.\"test_json\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/11 05:50:45 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/11 05:50:45 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,json,jsonb FROM e2e_test_pgch_dpt4h7al.\"test_json\" ORDER BY id" rows=14 bytes=1516 channelLen=0
    clickhouse_test.go:2278: UNEXPECTED TIMEOUT waiting for second raw table push to complete 2026-04-11 05:55:45.963546113 +0000 UTC m=+855.989047105
    clickhouse.go:122: begin tearing down postgres schema pgch_gu054cmi
--- FAIL: TestPeerFlowE2ETestSuitePG_CH/Test_Normalize_Metadata_With_Retry (337.43s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 11, 2026

🔄 Flaky Test Detected

Analysis: All failures show UNEXPECTED STATUS TIMEOUT STATUS_SETUP / UNEXPECTED TIMEOUT across multiple unrelated test suites (MySQL_CH, ApiPg, PG_CH) that all hit their timeouts at the same wall-clock time (~05:54 UTC), indicating CI runner resource exhaustion rather than a code regression.
Confidence: 0.95

✅ Automatically retrying the workflow

View workflow run

@pfcoperez pfcoperez mentioned this pull request Apr 13, 2026
Open
jgao54
jgao54 approved these changes Apr 14, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@jgao54 jgao54 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@ilidemi ilidemi merged commit f7ba7a9 into main Apr 14, 2026
17 of 18 checks passed
@ilidemi ilidemi deleted the always-chunk-uncompressed branch April 14, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jgao54 jgao54 jgao54 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ilidemi @jgao54