Add PageIndexBuilder and PageIndexProvider for Parquet page indexes

[etseidl]

Which issue does this PR close?

• Part of Support load page index metadata with specified column id instead of all columns #7582
• Closes parquet: PageIndex cannot be constructed outside the crate #10824.

Rationale for this change

This grew out of a discussion in #10784 and relates to apache/datafusion#24288 (comment). This PR provides a new builder for creating page index structures, and also adds a new PageIndexProvider trait to allow more performant implementations.

What changes are included in this PR?

Adds PageIndexBuilder, PageIndexProvider, implements PageIndexProvider for PageIndex, and adds RowGroupPageIndex as a helper to support fetching page indexes for a specific row group (replaces the X_index_for_rowgroup() functions on PageIndex).

Are these changes tested?

Yes, should be covered by existing tests

Are there any user-facing changes?

Yes, this changes the public API for accessing page index information

Created with the aid of Claude Code, but I own the changes.

[etseidl]

Thanks @alamb! I think I've addressed all of your comments, PTAL when you can. I still need to add issues for the serialization gap and memory accounting. I think the former is a pretty easy fix, but the latter will take some thought (we've gone down this rabbit hole before (e.g. #9138)

Edit: I have the serialization ready to go once this merges. Non-breaking so it can wait for 60.1.0.

[bharath-techie]

Thanks for the changes @etseidl

[alamb]

Thanks @etseidl -- I went through this one again

I filed some follow on tickets

• #11028
• #11029
• #11030

And then I pushed some commits to:

1. Update the TODO comments with links to issues
2. Fix some other random small cleanups found while reviewing

I think we are ready now. Maybe you can take one last quick peek and then merge it in!

[alamb]

• filed Support reporting heap size of custom PageIndexProvider #11028

[etseidl]

Thanks @alamb, your changes look good. I'll merge once CI finishes.

[adriangb]

Amazing to see this land! @etseidl do you plan to use this to work on alternative storage / representation for page indexes (particularly for wide tables with sparse reads)?

[alamb]

We did it!

[etseidl]

Amazing to see this land! @etseidl do you plan to use this to work on alternative storage / representation for page indexes (particularly for wide tables with sparse reads)?

Thanks @adriangb. TBH I haven't really thought that far ahead. I was hoping to get more requirements from the Datafusion side before doing too much further engineering. I'm certainly interested in having other providers in the parquet crate.