test(dataset): characterize metadata amplification on wide tables

[hiltonhe]

Summary

Characterization tests that pin the observable metadata costs of the
full-snapshot manifest on a 256-column dataset, using the built-in
object-store IO tracker (same pattern as the lazy-column-metadata tests):

• adding one column rewrites at least the previous manifest size of bytes;
• a cold open reads at least the manifest size, even for a narrow query;
• after another writer commits, the next checkout_latest re-reads the
  full manifest even though the reader's projection did not change.

Motivation

These document current behavior for wide tables — context and measurements
in the RFC discussion #9096. They are expected to be relaxed by any future
change that reduces metadata amplification; that change should update
these assertions in the same PR.

How tested

• cargo test -p lance --lib dataset::tests::dataset_wide_table (3 tests pass)
• cargo clippy -p lance --all-targets -- -D warnings

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

The read-side tests are isolated, fast, and directly characterize the full-manifest cost. The write-side characterization has a blind spot: total add_columns bytes include data and transaction files, so its threshold may stay green after the full-manifest rewrite is removed. Measuring the manifest object itself would make that assertion a durable signal.

[lance-gatekeeper]

io.written_bytes is operation-wide, not manifest-only, so this comparison does not establish that a full manifest was written. On this head, an instrumented run measured a 16,231-byte base manifest, a 16,859-byte new manifest, and 53,846 total bytes; subtracting the current manifest still leaves about 37 KB, already above base_manifest. The assertion can therefore remain green if a future implementation stops rewriting the full manifest—the exact improvement this test says should relax it. A manifest-path/size assertion (or the manifest writer’s reported size) would isolate the intended cost.