Skip to content

Add FuzzTest integration, recreating the existing xmlio test - #2941

Open
seang-ilm wants to merge 1 commit into
AcademySoftwareFoundation:mainfrom
seang-ilm:fuzztest-integration
Open

seang-ilm wants to merge 1 commit into
AcademySoftwareFoundation:mainfrom
seang-ilm:fuzztest-integration

Conversation

@seang-ilm

Copy link
Copy Markdown
Contributor

Integrate Google TestFuzz into MaterialX, and as a proof of concept re-create the existing Xmlio Fuzz Test using resources/Materials/Examples as a starter corpus

./MaterialXFuzz   --replay_corpus=MaterialXXmlIoFuzz.ParseXmlString 
...
Note: Google Test filter = MaterialXXmlIoFuzz.ParseXmlString
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from MaterialXXmlIoFuzz
[ RUN      ] MaterialXXmlIoFuzz.ParseXmlString
FUZZTEST_PRNG_SEED=MW5FQRTxoOIxOHCzmQCr17OSwQlpATxEgm9FetA_TN8
[*] Corpus size:     1 | Edges covered:   1438 | Fuzzing time:       9.201566ms | Total runs:  1.00e+00 | Runs/secs:   108 | Max stack usage:    50688
[*] Corpus size:     2 | Edges covered:   1450 | Fuzzing time:      31.130168ms | Total runs:  2.00e+00 | Runs/secs:    64 | Max stack usage:    50688
...
[*] Corpus size:    47 | Edges covered:   2790 | Fuzzing time:   4m44.48424777s | Total runs:  9.00e+01 | Runs/secs:     0 | Max stack usage:    50688
[*] Corpus size:    48 | Edges covered:   2791 | Fuzzing time:  4m44.531136569s | Total runs:  9.20e+01 | Runs/secs:     0 | Max stack usage:    50688
[       OK ] MaterialXXmlIoFuzz.ParseXmlString (287101 ms)
[----------] 1 test from MaterialXXmlIoFuzz (287101 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (287101 ms total)
[  PASSED  ] 1 test.

Note we've got some inputs in the resources/Materials/Examples that are very big which is why we see slow "Runs/secs" in the fuzzer. And you'll see some warnings about some test inputs in that folder that are too big for limit we've set in the Fuzzer of 1M inputs (/MaterialXFuzz/XmlIo_fuzz.cpp:22: INVALID_ARGUMENT: Invalid value in aggregate >> Invalid size: 1671504. Max size: 1048576:)

Sharing for initial feedback.

@linux-foundation-easycla

linux-foundation-easycla Bot commented May 19, 2026 •

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: seang-ilm / name: Sean Gilligan (1d2f65f)

@seang-ilm
seang-ilm force-pushed the fuzztest-integration branch from b11e231 to 864af7e Compare October 2, 2026 08:25
@seang-ilm

seang-ilm commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor Author

Pushing a fix to only seed the fuzzer with .mtlx documents. Including other large files in the examples/ dir like the images causes some very slow running. While fuzzing from valid XML I'm seeing steady 1,300 run/s second for a core while ASan in enabled.

[*] Corpus size:  1752 | Edges covered:   3916 | Fuzzing time:  9m56.837074352s | Total runs:  7.81e+05 | Runs/secs:  1308 | Max stack usage:     8368

[.] Fuzzing was terminated.

=================================================================
=== Fuzzing stats

Elapsed time: 10m0.001227882s
Total runs: 788050
Edges covered: 3916
Total edges: 96670
Corpus size: 1752
Max stack used: 8368

This approach still has some drawbacks. This is currently using random mutation on the initial document set, so you burn a lot of cycles trying to generate valid XML. In future this would benefit from a more detailed FuzzTest - Domain that is more likely to generate valid XML more often. Currently the fuzzer has been mainly useful for finding slow input paths than actual security issues.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants