table scoped lfc

[jizezhang]

Which issue does this PR close?

• Closes #.

Rationale for this change

This PR explores one way to make ListFilesCache table scoped. A session level cache is still used, but the cache key is made a "table-scoped" path, for which a new struct

pub struct TableScopedPath(pub Option<TableReference>, pub Path);

is defined. TableReference comes from CreateExternalTable passed to ListingTableFactory::create.

Additionally, when a table is dropped, all entries related to a table is dropped by modifying SessionContext::find_and_deregister method.

Testing (change on adding list_files_cache() for cli is included for easier testing).

• Testing cache reuse on a single table.

> \object_store_profiling summary
ObjectStore Profile mode set to Summary
> create external table test
stored as parquet
location 's3://overturemaps-us-west-2/release/2025-12-17.0/theme=base/';
0 row(s) fetched. 
Elapsed 14.878 seconds.

Object Store Profiling
Instrumented Object Store: instrument_mode: Summary, inner: AmazonS3(overturemaps-us-west-2)
Summaries:
+-----------+----------+-----------+-----------+-------------+-------------+-------+
| Operation | Metric   | min       | max       | avg         | sum         | count |
+-----------+----------+-----------+-----------+-------------+-------------+-------+
| Get       | duration | 0.030597s | 0.209235s | 0.082396s   | 36.254189s  | 440   |
| Get       | size     | 204782 B  | 857230 B  | 497304.88 B | 218814144 B | 440   |
| List      | duration | 0.192037s | 0.192037s | 0.192037s   | 0.192037s   | 1     |
| List      | size     |           |           |             |             | 1     |
+-----------+----------+-----------+-----------+-------------+-------------+-------+
> select table, path, unnest(metadata_list) from list_files_cache() limit 1;
+-------+---------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| table | path                            | UNNEST(list_files_cache().metadata_list)                                                                                                                                                                                                                  |
+-------+---------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| test  | release/2025-12-17.0/theme=base | {file_path: release/2025-12-17.0/theme=base/type=bathymetry/part-00000-dd0f2f50-b436-4710-996f-f1b06181a3a1-c000.zstd.parquet, file_modified: 2025-12-17T22:32:50, file_size_bytes: 40280159, e_tag: "15090401f8f936c3f83bb498cb99a41d-3", version: NULL} |
+-------+---------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched. 
Elapsed 0.058 seconds.

Object Store Profiling
> select count(*) from test where type = 'infrastructure';
+-----------+
| count(*)  |
+-----------+
| 142969564 |
+-----------+
1 row(s) fetched. 
Elapsed 0.028 seconds.

Object Store Profiling

• Test separate cache entries are created for two tables with same path

> create external table test2
stored as parquet
location 's3://overturemaps-us-west-2/release/2025-12-17.0/theme=base/';
0 row(s) fetched. 
Elapsed 14.798 seconds.

Object Store Profiling
Instrumented Object Store: instrument_mode: Summary, inner: AmazonS3(overturemaps-us-west-2)
Summaries:
+-----------+----------+-----------+-----------+-------------+-------------+-------+
| Operation | Metric   | min       | max       | avg         | sum         | count |
+-----------+----------+-----------+-----------+-------------+-------------+-------+
| Get       | duration | 0.030238s | 0.350465s | 0.073670s   | 32.414654s  | 440   |
| Get       | size     | 204782 B  | 857230 B  | 497304.88 B | 218814144 B | 440   |
| List      | duration | 0.133334s | 0.133334s | 0.133334s   | 0.133334s   | 1     |
| List      | size     |           |           |             |             | 1     |
+-----------+----------+-----------+-----------+-------------+-------------+-------+
> select count(*) from test2 where type = 'bathymetry';
+----------+
| count(*) |
+----------+
| 59963    |
+----------+
1 row(s) fetched. 
Elapsed 0.009 seconds.

Object Store Profiling
> select table, path from list_files_cache();
+-------+---------------------------------+
| table | path                            |
+-------+---------------------------------+
| test  | release/2025-12-17.0/theme=base |
| test2 | release/2025-12-17.0/theme=base |
+-------+---------------------------------+
2 row(s) fetched. 
Elapsed 0.004 seconds.

• Test cache associated with a table is dropped when table is dropped, and the other table with same path is unaffected.

> drop table test;
0 row(s) fetched. 
Elapsed 0.015 seconds.

Object Store Profiling
> select table, path from list_files_cache();
+-------+---------------------------------+
| table | path                            |
+-------+---------------------------------+
| test2 | release/2025-12-17.0/theme=base |
+-------+---------------------------------+
1 row(s) fetched. 
Elapsed 0.005 seconds.

Object Store Profiling
> select count(*) from list_files_cache() where table = 'test';
+----------+
| count(*) |
+----------+
| 0        |
+----------+
1 row(s) fetched. 
Elapsed 0.014 seconds.
> select count(*) from test2 where type = 'infrastructure';
+-----------+
| count(*)  |
+-----------+
| 142969564 |
+-----------+
1 row(s) fetched. 
Elapsed 0.013 seconds.

Object Store Profiling

• Test that dropping a view does not remove cache

> create view test2_view as (select * from test2 where type = 'infrastructure');
0 row(s) fetched. 
Elapsed 0.103 seconds.

Object Store Profiling
> select count(*) from test2_view;
+-----------+
| count(*)  |
+-----------+
| 142969564 |
+-----------+
1 row(s) fetched. 
Elapsed 0.094 seconds.

Object Store Profiling
> drop view test2_view;
0 row(s) fetched. 
Elapsed 0.002 seconds.

Object Store Profiling
> select table, path from list_files_cache();
+-------+---------------------------------+
| table | path                            |
+-------+---------------------------------+
| test2 | release/2025-12-17.0/theme=base |
+-------+---------------------------------+
1 row(s) fetched. 
Elapsed 0.007 seconds.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?