docs: Fix upgrade guide API examples for FileScanConfigBuilder and ParquetSource

[ShashidharM0118]

Which issue does this PR close?

Closes #19393

Rationale for this change

The upgrade guide contained incorrect API examples that didn't match the current implementation

What changes are included in this PR?

• Updated ParquetExecBuilder migration example in the upgrade guide to use correct API:
  • Changed ParquetSource::new(parquet_options) to ParquetSource::new(table_schema) with proper TableSchema creation
  • Changed FileScanConfig::new(url, file_schema, source) to FileScanConfigBuilder::new(url, source) (removed schema parameter)
  • Removed non-existent with_table_partition_cols() method call
  • Updated with_projection() to with_projection_indices() with proper error handling
  • Fixed final execution plan creation to use DataSourceExec::from_data_source()

Are these changes tested?

Are there any user-facing changes?

No

[Copilot]

Pull request overview

This PR fixes incorrect API examples in the upgrade guide documentation for migrating from ParquetExecBuilder to the new FileScanConfigBuilder and DataSourceExec pattern. The examples were updated to reflect the current API implementation, correcting method signatures, parameter types, and builder patterns that had evolved since the original documentation was written.

Key Changes

• Updated ParquetSource instantiation to use TableSchema parameter instead of parquet options
• Replaced FileScanConfig::new() with FileScanConfigBuilder pattern and removed the schema parameter
• Changed with_projection() to with_projection_indices() with proper error handling
• Updated final execution plan creation to use DataSourceExec::from_data_source()

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[alamb]

This is an upgrade guide for an older version (which had a different API) -- Why would the upgrade match the current version?

[ShashidharM0118]

This is an upgrade guide for an older version (which had a different API) -- Why would the upgrade match the current version?

Apologies — I misunderstood the purpose of the upgrade guide itself. 😅.

[mag1c1an1]

This is an upgrade guide for an older version (which had a different API) -- Why would the upgrade match the current version?

When I upgraded from version 50 to 51, this guide left me confused, or perhaps there was something I overlooked.

[alamb]

Hi @mag1c1an1

When I upgraded from version 50 to 51, this guide left me confused, or perhaps there was something I overlooked.

I am not sure where your screen shots come from. This one shows the arguments from main (not the 51 branch)

The actual released 51 docs are here:

https://docs.rs/datafusion/51.0.0/datafusion/datasource/physical_plan/struct.FileScanConfigBuilder.html#method.new

I wonder if you are trying to upgrade to the main branch (what will become 52.0.0) -- it is still labeled (confusingly) as 51.0.0

[mag1c1an1]

Hi @alamb . I'm very sorry that I only saw your comment now. I upgraded to version 51 instead of the main branch. What puzzles me is the following description in the upgrading guide:

FileSource constructors now require TableSchema: All built-in file sources now take the schema in their constructor:

- let source = ParquetSource::default();
+ let source = ParquetSource::new(table_schema);

But in api doc , the new function of ParquetSource take TableParquetOptions.

FileScanConfigBuilder no longer takes schema as a parameter: The schema is now passed via the FileSource:

- FileScanConfigBuilder::new(url, schema, source)
+ FileScanConfigBuilder::new(url, source)

But in api doc , the new function of FileScanConfigBuilder still needs a file schema.

In my understanding, the meaning of the above code block is that the API of 51 has changed now. Please make the necessary modifications, such as removing the code starting with '- ' and replacing it with the code starting with '+ '. Am I mistaken in my interpretation? All the code blocks come from the "DataFusion 51" section of the upgrading guide. I hope my poor expression skills can correctly convey my intention.

[Jefffrey]

@mag1c1an1 you're correct that the upgrade guide is incorrect, it seems the PR was originally slated for v51 but was pushed to v52 but I guess we forget to amend the upgrade guide

• Force FileSource to be constructed with a Schema #18386

@ShashidharM0118 Please doublecheck where we're supposed to be fixing the upgrade guide as per what @mag1c1an1 stated

cc @adriangb

[adriangb]

Apologies if I put docs in the wrong version and caused confusion. This was a big change and I'm sure I made mistakes. It seems like there's already a lot of discussion underway, and I myself am confused after reading this about what is in what version 😄, I'm happy to help clarify anything if needed otherwise I will allow the work to continue.

[ShashidharM0118]

Thanks @mag1c1an1 and @Jefffrey for the clarification! I've updated the docs

[Jefffrey]

This is still not correct @ShashidharM0118; can you please double check my comment above? We should not be modifying the existing text in-place under v51 as it is not supposed to be in v51, it is meant to be in v52. Moreover these edits don't make sense; the existing text is left to state:

FileSource constructors now require TableSchema:

But the example has been changed so this is not the case here so now it is inconsistent.

[mag1c1an1]

Hi @ShashidharM0118, thanks for your contribution to #19393!

Regarding the update, could you please remove the sections I mentioned? Since these API changes haven't been introduced in v51 yet, this part remains the same for users upgrading from v50 or earlier. Keeping it might cause confusion, so it's best to revert those specific examples to match the current v51 implementation.