Skip to content

add generate_quantities timing support - #1168

Merged
jgabry merged 5 commits into
masterfrom
gq-timing
Jul 18, 2026
Merged

add generate_quantities timing support#1168
jgabry merged 5 commits into
masterfrom
gq-timing

Conversation

@avehtari

@avehtari avehtari commented Mar 29, 2026

Copy link
Copy Markdown
Member

Closes #1070 after the next CmdStan release.

CmdStan develop version adds now timing information to standalone generated quantities csv. This PR adds CmdStanR support so that the timing reporting with $generate_quantities() matches $sample().

For CmdStan 2.38 and earlier the timing reported stays the same, that is, showing 0.0 seconds.

Example output with CmdStan 2.39+

Running standalone generated quantities after 4 MCMC chains, all chains in parallel ...

Chain 3 finished in 1.3 seconds.
Chain 4 finished in 1.4 seconds.
Chain 2 finished in 1.7 seconds.
Chain 1 finished in 1.9 seconds.

All 4 chains finished successfully.
Mean chain execution time: 1.6 seconds.
Total execution time: 2.0 seconds.

Copyright and Licensing

Please list the copyright holder for the work you are submitting
(this will be you or your assignee, such as a university or company):
Aki Vehtari

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses:

@avehtari
avehtari requested a review from jgabry March 29, 2026 16:57
@jgabry

jgabry commented Mar 30, 2026

Copy link
Copy Markdown
Member

Thanks Aki. Looks like some tests are failing.

@avehtari

Copy link
Copy Markdown
Member Author

Some tests may fail if not using the develop version of CmdStan. I only thought that the actual code would work with old CmdStan, but did not think that it would be easier if the tests also work with old versions. Will fix tomorrow

@avehtari

Copy link
Copy Markdown
Member Author

Can we have conditional tests based on the CmdStan version? I tried to look at examples in tests, but didn't find a clear example

Comment thread tests/testthat/test-csv.R Outdated
Comment thread tests/testthat/test-fit-gq.R Outdated
@jgabry

jgabry commented Mar 30, 2026

Copy link
Copy Markdown
Member

Can we have conditional tests based on the CmdStan version? I tried to look at examples in tests, but didn't find a clear example

We used to have a lot of tests like this for older versions of CmdStan but they were removed recently because for v1.0 we will require at least CmdStan v2.35.0. You can do things like this:

if (cmdstan_version() >= "2.39.0") {
  expect_true(...)
} 

But if the development version of CmdStan is not installed on any of the GitHub actions runs then the new functionality won't actually be tested. We could either setup one of the CI runs to use the development version so we can test it now or we could wait to merge this until the next CmdStan release.

@codecov-commenter

codecov-commenter commented Mar 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.75000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.46%. Comparing base (24acf13) to head (9f9f877).

Files with missing lines Patch % Lines
R/csv.R 95.83% 1 Missing ⚠️
R/run.R 87.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1168      +/-   ##
==========================================
+ Coverage   90.57%   91.46%   +0.88%     
==========================================
  Files          15       15              
  Lines        6131     6149      +18     
==========================================
+ Hits         5553     5624      +71     
+ Misses        578      525      -53     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jgabry

jgabry commented Jul 17, 2026

Copy link
Copy Markdown
Member

Now that CmdStan 2.39 was released we can go ahead with this. I found a couple of small issues, but I'll make some commits soon and then we can merge.

@jgabry jgabry left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I made a few changes and we can merge when all the checks pass

@jgabry jgabry added this to the v1.0.0 - release milestone Jul 17, 2026
@jgabry
jgabry merged commit e27a88f into master Jul 18, 2026
15 checks passed
@jgabry
jgabry deleted the gq-timing branch July 18, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

generate quantities method always reports completion of chains in 0s

3 participants