fix: avoid exponential MSVC compile time in covariant try_get

[dornbirndevelops]

Problem:

• MSVC compile time and memory grow exponentially with the number of reference dependencies passed to sm's constructor. From about 28 dependencies on, it fails with C1060 "compiler is out of heap space".
• Cause: the covariant try_get overloads #8/#9 from Google's NiceMock<MyMock> dependencies fail to compile if dependency is passed as reference template parameter #467 deduce D from a pool_type<D&> / pool_type<const D&> base of the pool. When many bases match that pattern, MSVC's deduction is exponential in their number.
• The same deduction fails on every compiler as soon as more than one base matches: #8 only found a derived dependency when the pool held exactly one reference dependency, #9 only when it held exactly one const reference dependency. Otherwise the lookup fell back to missing_ctor_parameter, which is a compile error for Base&, but silently a dangling default-constructed object for const Base&.

Solution:

• Replace the deducing overloads #8/#9 with a single try_get whose slot is computed by covariant_dep. Nothing is deduced from the pool's bases anymore.
• The candidates are the pool's reference dependencies D& / const D& whose D* converts to const T*, tested by overload resolution. A public, unambiguous base is a candidate. A private, protected or ambiguous base is a candidate that is a compile error to read. An incomplete D has no known bases, so it is no candidate; nothing tests whether it is complete. The list is built once per pool.
• The covariant lookup only applies to a class T for which none of the exact overloads #2–#7 matches (has_exact_dep). A T held by value (#1) does not block it. Lookups of non-class types (void*, int, ...) are left to #1–#7 and the fallback, as before.
• One candidate is read; with several, the single const one (keeping #9's result for e.g. D1& + const D2&); otherwise it is a static_assert instead of the silent fallback. Instead of that error, and instead of reading a private or ambiguous base, a T held by value is read by #1, except for a const T& parameter, where that copy would dangle.
• pool(init, ...) reads each slot through try_get_slot. A Base& slot only considers D& deps: with D1& + const D2&, Base& reads D1, while const Base& and Base read D2. A const Base& that shares the Base& slot, because the sm also takes Base&, reads D1.

Behaviour changes:

1. A derived dependency used as Base&, const Base& or Base works alongside other reference dependencies. This changes the runtime behaviour of code that already compiled. A const Base& previously bound a dangling default-constructed temporary, a Base by value got a default object, and with BOOST_SML_CREATE_DEFAULT_CONSTRUCTIBLE_DEPS a Base& got an owned default object. Now they get the passed object. The same holds for the lookup of the state machine class and of sub-state-machine classes. A Base passed by value next to a derived reference dependency and other reference dependencies: master read the Base by value; now the derived dependency is read, as master did when it was the only reference dependency.
2. An exact dependency #2–#7 (Base&, const Base&, Base*, const Base*) wins over a derived one. Before, partial ordering let #9 win: with Base& and const Derived&, a const Base& got the derived object and a Base& failed to compile. A Base held by value does not win over a derived dependency.
3. Several dependencies derived from the requested type are a compile error ("State Machine has several constructor parameters derived from a requested type!"). For const Base& and Base this applies without a single const one. For Base& it applies to several mutable ones; const ones do not count, so e.g. D1& + D2& + const D3& is an error for Base&. With BOOST_SML_CREATE_DEFAULT_CONSTRUCTIBLE_DEPS, master gave a sliced copy of D3 there. It is not an error when Base is also passed by value (except for const Base&): that one is read, as on master. Before, the lookup fell back to a default, dangling or null object.
4. A dependency with the requested type as a private, protected or ambiguous base is a compile error when the lookup selects it, also next to other reference dependencies. The exception is when Base is also passed by value (except for const Base&): then that one is read, as master did next to other reference dependencies. Before, it was only diagnosed when it was the only reference dependency.
5. Reference dependencies to forward-declared types compile. Before, GCC and MSVC rejected an incomplete type that was the only (const) reference dependency. An incomplete type is never a covariant candidate, so the sm has to see the definition where the type is to be found as a base. A lookup of another type made while it is incomplete, e.g. of the state machine class, does not hide it from later lookups. Nothing tests completeness, so gcc 16 doesn't warn (-Wsfinae-incomplete).
6. A Base* parameter with a single derived reference dependency next to other reference dependencies is a compile error. Before, it silently got nullptr.
7. MSVC in /permissive mode (the default for C++14/17) deduced D from the first matching base, so it silently picked the first of several derived dependencies. Now that is the error from 3, or with Base also passed by value, that one is read. (MSVC /permissive still accepts a conversion to an ambiguous base that is also a direct base, as master did.)
8. With D1& + const D2&, a Base& parameter binds D1. Master gave a compile error, or with BOOST_SML_CREATE_DEFAULT_CONSTRUCTIBLE_DEPS a sliced copy of D2.
9. A volatile derived dependency is no candidate for a non-volatile Base.

Tests:

• test/ft/issue_715_many_ctor_deps.cpp: the reproducer from MSVC: exponential compile time and C1060 "compiler is out of heap space" #715 (28 dependencies), plus a derived dependency among 27 others (macro off). With the master header, the first one runs out of heap on MSVC and the second one fails to compile on every compiler.
• test/ft/dependencies.cpp, new cases that fail on master: an exact base& preferred over a const derived&, a const derived dependency among other const reference dependencies, a const derived dependency next to a mutable one, void* parameters next to a derived dependency, a covariant lookup unaffected by a user's declarations found by ADL, and a forward-declared type as the only reference dependency.
• test/ft/errors/several_derived_deps.cpp, test/ft/errors/private_base_dep.cpp: must not compile (WILL_FAIL).
• Review round (5dfe302):
  • test/ft/dependencies.cpp:
    • mutable_derived_dep_not_hidden_by_const_derived_dep: fails to compile on 191c554 and on master.
    • derived_dep_preferred_over_base_held_by_value: dangles on 191c554.
    • derived_dep_preferred_over_base_held_by_value_among_other_ref_deps: master read the by-value base.
    • mutable_derived_dep_bound_not_copied_next_to_const_derived_dep: with the macro, master and 191c554 used a sliced copy.
    • The ADL test now also declares a try_get_slot in the dep's namespace.
    • Guards for the by-value rule, which pass on master (gcc/clang) and on 191c554: base_held_by_value_read_next_to_several_derived_deps, base_held_by_value_read_next_to_private_derived_dep and volatile_derived_dep_is_no_candidate.
  • test/ft/issue_715_incomplete_deps.cpp (macro off): a dep that was incomplete where a first sm was built is found as a base later (dangles on 191c554 with clang). A static sm whose reference dep is defined later (-Wsfinae-incomplete error on 191c554 with gcc 16 -Werror).
  • test/ft/errors/private_base_dep_next_to_base_by_value.cpp: must not compile; it dangles on 191c554.

Benchmark (the #715 reproducer, compile only, C++20, debug flags:

• MSVC /Od /Ob0 /RTC1 /Zi /MDd /bigobj, gcc/clang -O0 -g; peak memory is MSVC job commit or max RSS; best of 3 below 5 s; i7-13700K, 64 GB RAM):

MSVC 19.44.35229 x64 (VS 2022):

deps	before	after
16	0.26 s / 78 MB	0.22 s / 70 MB
24	10.4 s / 8.6 GB	0.26 s / 94 MB
28	C1060 / ~39 GB	0.29 s / 108 MB
64	–	0.72 s / 289 MB
128	–	3.0 s / 826 MB

gcc 13.3 and clang 18.1 never had the blow-up and are unchanged (128 deps: gcc 0.85 → 0.86 s, clang 0.79 → 0.80 s).

Lookups without an exact dependency (e.g. the state machine class itself, or parameters that are not passed) scan the pool's reference dependencies once each, so they cost O(N). With 128 reference dependencies and 9 such lookups the file compiles in 0.48 s on MSVC. 128 reference dependencies with 128 such lookups take 5.8 s / 1.7 GB, where master runs out of heap.

The review round (5dfe302) leaves the #715 reproducer unchanged on MSVC, gcc 16 and clang 23. Measured side by side with 191c554, 128 reference dependencies with 128 such lookups use 1.4 instead of 1.6 GB peak memory on MSVC, at the same time.

Verified:

• MSVC 2022, C++20: all 50 tests pass.
• gcc 14, gcc 16 and clang 23 at C++14/17/20, plain and with -fsanitize=address,undefined: all tests pass.

Issue: #715 #723

Reviewers: @kris-jusiak @PavelGuzenfeld

[PavelGuzenfeld]

@dornbirndevelops, could you please add a benchmark for the fix to the test suite?
Also, please update the documentation with a usage example.

[dornbirndevelops]

Hi @PavelGuzenfeld,

Could you please provide benchmark for the fix as part of the test suite?

I've integrated the minimum reproducible example from the original issue in commit 5ac2fec into the test suite.
Could you please be more specific what you mean benchmark as part of the test suite?

[PavelGuzenfeld]

Hi @PavelGuzenfeld,

Could you please provide benchmark for the fix as part of the test suite?

I've integrated the minimum reproducible example from the original issue in commit 5ac2fec into the test suite. Could you please be more specific what you mean benchmark as part of the test suite?

I misunderstood. I was curious about the compile time improvement.
Does the fix solve both the heap allocation limit and the compile time, or just the compile time?
The API isn't affected, so disregard the second request.

[PavelGuzenfeld]

See review comments per file.

[PavelGuzenfeld]

1. Test passes against master so reverting the fix would leave CI green.
2. The test doesn't cover the covariant overloads.

[dornbirndevelops]

You're right on both counts. That test only had exact dependencies: with the master header it passed on gcc and clang (only MSVC ran out of heap), and it never reached #8/#9.

issue_715_many_ctor_deps.cpp replaces it and now also has derived_dep_among_many_ctor_dependencies: an implementation of an abstract interface among 27 other reference dependencies, taken as const interface&. With the master header it fails to compile on every compiler, and the reproducer next to it still runs out of heap on MSVC. That file doesn't define BOOST_SML_CREATE_DEFAULT_CONSTRUCTIBLE_DEPS, so the covariant path is covered with the macro off too.

dependencies.cpp adds cases that fail on master as well (exact base& vs const derived&, const derived among other const reference deps, const vs mutable derived, void* next to a derived dep, ADL, a single forward-declared dep), and test/ft/errors checks that ambiguous and private bases don't compile.

[PavelGuzenfeld]

The PR deletes the missing_ctor_parameter<T> sentinel fallback. Now it's a hard fail. Please restore it.

[dornbirndevelops]

Good catch, that revision did remove it: it renamed try_get(...) to try_get_impl(...) behind a wrapper that only accepted pool pointers. sml's own call sites all pass a pool, so a missing dependency still ended up at missing_ctor_parameter<T>, but try_get<T> on anything else became a hard error.

Since then the try_get(...) declaration and definition are back unchanged, and #1–#7 are untouched. The covariant lookup is one additional overload that drops out when there is no candidate, so a missing dependency still resolves to the sentinel. The one exception is intended: several dependencies derived from the requested type are now a static_assert instead of silently falling back to a default object.

[dornbirndevelops]

Hi @PavelGuzenfeld,

I was curious about the compile time improvement.
Does the fix solve both the heap allocation limit and the compile time, or just the compile time?

The solution attempts to solve both the heap allocation limit as well as the compile time.
I revisited my previous attempt and also added some benchmarks as requested.

[PavelGuzenfeld]

In context of class template 'has_exact_dep'

  struct e1 {};
  struct base { int val = 1; };
  struct der : base { der() { val = 715; } };

  struct c {
    auto operator()() noexcept {
      using namespace sml;
      return make_transition_table(*"idle"_s + event<e1> / [](const base& b) { std::printf("val=%d\n", b.val); } = X);
    }
  };

  int main() {
    der d;
    base b;
    b.val = 5;
    sml::sm<c> sm{std::move(b), d};
    sm.process_event(e1{});
  }

gcc 14 -fsanitize=address gives stack-use-after-return instead of val=715 as on master (the by-value b counts as exact, so the const base& slot binds to a temporary). Could you add this as a test?

[PavelGuzenfeld]

In context of class template 'unique_covariant_dep'

  struct e1 {};
  struct iface { virtual ~iface() = default; virtual int id() const = 0; };
  struct a : iface { int id() const override { return 1; } };
  struct b : iface { int id() const override { return 2; } };

  struct c {
    auto operator()() noexcept {
      using namespace sml;
      return make_transition_table(*"idle"_s + event<e1> / [](iface& f) { std::printf("%d\n", f.id()); } = X);
    }
  };

  int main() {
    a x;
    const b y{};
    sml::sm<c> sm{x, y};
    sm.process_event(e1{});
  }

Fails with no matching function for call to 'pool_type<iface&>::pool_type(const iface&)' instead of printing 1 (only x binds to iface&, but the single const y is picked).

[PavelGuzenfeld]

In context of class template 'covariant_slot'

  struct e1 {};
  struct base { int val = 1; };
  struct der;

  struct c1 {
    auto operator()() noexcept {
      using namespace sml;
      return make_transition_table(*"idle"_s + event<e1> / [](der&, int&) {} = X);
    }
  };
  void f1(der& d, int& i) { sml::sm<c1> sm{d, i}; }

  struct der : base { der() { val = 715; } };

  struct c2 {
    auto operator()() noexcept {
      using namespace sml;
      return make_transition_table(*"idle"_s + event<e1> / [](const base& b, int&) { std::printf("val=%d\n", b.val); } = X);
    }
  };

  int main() {
    der d;
    int i = 0;
    f1(d, i);
    sml::sm<c2> sm{d, i};
    sm.process_event(e1{});
  }

clang 18 -fsanitize=address gives stack-use-after-return instead of val=715 (der was incomplete when f1 first instantiated the slot. gcc 14 prints val=715 as well).

[dornbirndevelops]

In context of class template 'has_exact_dep'

Thanks, confirmed. The base passed by value counted as an exact dep, so try_get #1 won and returned a copy, which the const base& slot bound as a temporary. On master #8 won over #1 by partial ordering.

A T held by value is no longer an exact dep, so the derived dep wins again and this prints val=715. Where the derived lookup is ambiguous, or the only derived dep has T as a private or ambiguous base, the T held by value is still read, as on master. The exception is a const T& parameter: that copy would dangle, so there it stays a compile error.

Added as derived_dep_preferred_over_base_held_by_value, plus variants next to it, in dependencies.cpp (5dfe302).

[dornbirndevelops]

In context of class template 'unique_covariant_dep'

Confirmed. It didn't compile on master either, because #9 picked y there as well. A T& parameter now only considers mutable candidates, so with a& + const b& it binds x and prints 1. A const iface& or iface parameter still reads y, as #9 did.

One consequence: if the same sm takes both iface& and const iface&, sml keeps a single iface& slot for both (collapse_const_refs), so both read x.

Added as mutable_derived_dep_not_hidden_by_const_derived_dep (5dfe302).

[dornbirndevelops]

In context of class template 'covariant_slot'

Confirmed, thanks. covariant_slot<der&> cached whether der was complete. clang instantiates f1's constexpr constructor on the spot, so the later sm<c2> reused "incomplete".

The lookup no longer tests completeness at all. A dep is a candidate when D* converts to const T*, decided per (requested type, dep), so the earlier lookup of c1 no longer affects the later lookup of base. That also gets rid of gcc 16's new -Wsfinae-incomplete warning, which fired e.g. for a static sm whose reference dep is defined later.

Both are in the new test/ft/issue_715_incomplete_deps.cpp (5dfe302). What can't work in general is looking up a base of der itself while der is incomplete, or in a TU where it is incomplete. That's documented next to the lookup.

[PavelGuzenfeld]

In context of alias template 'slot_lookup_t'

  struct e1 {};
  struct base { int val = 1; };
  struct der : base { der() { val = 715; } };

  struct c {
    auto operator()() noexcept {
      using namespace sml;
      return make_transition_table(*"idle"_s + event<e1> / [](der&, int&, base* p) { std::printf("%s\n", p ? "set" : "null"); } = X);
    }
  };

  int main() {
    der d;
    int i = 0;
    sml::sm<c> sm{d, i};
    sm.process_event(e1{});
  }

Fails with no matching function for call to 'pool_type<base*>::pool_type(base&)' instead of printing null as on master (slot_lookup_t strips the pointer, so the base* slot takes the covariant der&).

[PavelGuzenfeld]

In context of function template 'try_get_slot'

  struct e1 {};
  struct base { int val = 0; };
  struct a : base { a() { val = 1; } };
  struct b : base { b() { val = 2; } };

  struct c {
    auto operator()() noexcept {
      using namespace sml;
      return make_transition_table(*"idle"_s + event<e1> / [](base& r, base v) { std::printf("%d %d\n", r.val, v.val); } = X);
    }
  };

  int main() {
    a x;
    const b y{};
    sml::sm<c> sm{x, y};
    sm.process_event(e1{});
  }

Prints 1 2 instead of a compile error as on master (the base& slot reads x, the base slot reads y).

[PavelGuzenfeld]

Dead code in context of class template 'slot_kind'

  struct e1 {};
  struct base { int val = 5; };
  struct der : base { der() { val = 715; } };

  struct c {
    auto operator()() noexcept {
      using namespace sml;
      return make_transition_table(*"idle"_s + event<e1> / [](base*& p, base* const& q, der&, int&) { std::printf("%d %d\n", p->val, q->val); } = X);
    }
  };

  int main() {
    base b;
    base* bp = &b;
    der d;
    int i = 0;
    sml::sm<c> sm{bp, d, i};
    sm.process_event(e1{});
  }

Prints 5 5 with slot_kind<T *&> and slot_kind<T *const &> removed, same as with them (slot_lookup_t<base *&> is base *, not a class, so covariant_dep is off whatever the slot kind). Please drop them.

[PavelGuzenfeld]

Many thank for your contribution!
After these 3 I think it would be LGTM.
In addition, I've posted several followup issues found only on master (found along the way during code review) - help would be much appreciated on a separate pr.