Skip to content

fix(molrecbench_wild): give the converter its required track column - #1668

Merged
ssiq merged 1 commit into
open-compass:mainfrom
Anai-Guo:fix/molrecbench-wild-track-column
Sep 23, 2026
Merged

ssiq merged 1 commit into
open-compass:mainfrom
Anai-Guo:fix/molrecbench-wild-track-column

Conversation

@Anai-Guo

@Anai-Guo Anai-Guo commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

The bug

MolRecBenchWildDataset.evaluate() cannot run at all. It builds a two-column
frame and then calls the converter with a track= keyword:

selected = pd.DataFrame({
    'index': sample_ids,
    'prediction': [prediction_map[index] for index in selected_indices],
})

from .utils.molrecbench_wild import convert_dataframe, score_records

converted, _ = convert_dataframe(selected, track=self.track)

But convert_dataframe takes no track argument — it reads the track from a
required column:

def convert_dataframe(dataframe, *, sheet="Sheet1", id_suffix=""):
    required_columns = {"index", "track", "prediction"}

So every run of any of the three MolRecBench-Wild tracks raises:

TypeError: convert_dataframe() got an unexpected keyword argument 'track'

and simply deleting the keyword is not enough, because the frame that
evaluate() builds has no track column either:

WorkbookConversionError: sheet 'Sheet1' is missing columns: track

(score_records(ground_truth, converted, self.track, ...) a few lines below is
correct — track really is a positional parameter there, which is likely
where the mix-up came from.)

The fix

Supply the track as the column the converter asks for, and drop the keyword.

Verification

Ran the real vlmeval/dataset/utils/molrecbench_wild package at main
(5a91183) against the exact frame evaluate() constructs.

Before — both failure modes:

--- convert_dataframe(selected, track=self.track) ---
TypeError: convert_dataframe() got an unexpected keyword argument 'track'

--- keyword removed but column still absent ---
WorkbookConversionError: sheet 'Sheet1' is missing columns: track

After — records convert, no diagnostics:

records= 2 issues= 0
  id= mol_0001.png smiles= 'CCO'
  id= mol_0002.png smiles= 'c1ccccc1'

(second prediction was fenced ```json output, to check the normal
extraction path still runs rather than falling back to empty records).

pre-commit's flake8 (7.1.2, --max-line-length=120 --ignore=W503) and isort
6.0.1 are clean on the changed file, and yapf produces the same diff before and
after the change — the added lines are untouched by it.

🤖 Generated with Claude Code

MolRecBenchWildDataset.evaluate() calls
convert_dataframe(selected, track=self.track), but convert_dataframe's
signature is (dataframe, *, sheet, id_suffix) - it reads the track from a
required "track" column instead of a keyword argument. Every evaluation
run therefore dies with

    TypeError: convert_dataframe() got an unexpected keyword argument 'track'

and dropping the keyword alone is not enough, because the frame that
evaluate() builds only has "index" and "prediction":

    WorkbookConversionError: sheet 'Sheet1' is missing columns: track

Add the track column to the frame and drop the keyword.
@ssiq
ssiq added this pull request to the merge queue Sep 23, 2026
Merged via the queue into open-compass:main with commit ed12fe1 Sep 23, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants