Skip to content

test(node): Port Anthropic integration tests to span streaming #25082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nicohrubec merged 12 commits into from
Oct 7, 2026
Merged

test(node): Port Anthropic integration tests to span streaming #25082

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
c5f9ed4
test(node): Port Anthropic integration tests to span streaming
nicohrubec Oct 6, 2026
255ccf7
test(node): Remove redundant anthropic assertions
nicohrubec Oct 6, 2026
03bbcb8
test(node): Collect anthropic spans across envelopes
nicohrubec Oct 6, 2026
550ecdf
test(node): Match the intended Anthropic root spans
nicohrubec Oct 6, 2026
aaf387f
test(node): Use the SENTRY_ORIGIN convention in anthropic tests
nicohrubec Oct 6, 2026
b4f3b2a
test(node): Wait for Anthropic beta helpers to finish before counting…
nicohrubec Oct 6, 2026
f637a47
test(node): Use convention constants for anthropic span attributes
nicohrubec Oct 6, 2026
3692a56
Merge branch 'develop' into feat/anthropic-tests-span-streaming
nicohrubec Oct 6, 2026
dfa3f30
test(node): Report Anthropic assertions outside envelope matching
nicohrubec Oct 7, 2026
92d57fb
Merge develop before Anthropic runner cleanup
nicohrubec Oct 7, 2026
20d8f15
test(node): Remove unnecessary Anthropic unordered matching
nicohrubec Oct 7, 2026
139269a
test(node): Simplify ordered Anthropic span assertions
nicohrubec Oct 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions dev-packages/node-integration-tests/suites/tracing/anthropic/instrument-span-streaming.mjs
View file
Open in desktop

This file was deleted.

8 changes: 0 additions & 8 deletions dev-packages/node-integration-tests/suites/tracing/anthropic/instrument-with-options.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ import * as Sentry from '@sentry/node';
import { loggingTransport } from '@sentry-internal/node-integration-tests';

Sentry.init({
traceLifecycle: 'static',
dsn: 'https://public@dsn.ingest.sentry.io/1337',
release: '1.0',
tracesSampleRate: 1.0,
Expand All @@ -14,11 +13,4 @@ Sentry.init({
recordOutputs: true,
}),
],
beforeSendTransaction: event => {
// Filter out mock express server transactions
if (event.transaction.includes('/anthropic/v1/')) {
return null;
}
return event;
},
});
8 changes: 0 additions & 8 deletions dev-packages/node-integration-tests/suites/tracing/anthropic/instrument-with-pii.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,9 @@ import * as Sentry from '@sentry/node';
import { loggingTransport } from '@sentry-internal/node-integration-tests';

Sentry.init({
traceLifecycle: 'static',
dsn: 'https://public@dsn.ingest.sentry.io/1337',
release: '1.0',
tracesSampleRate: 1.0,
dataCollection: { genAI: { inputs: true, outputs: true } },
transport: loggingTransport,
beforeSendTransaction: event => {
// Filter out mock express server transactions
if (event.transaction.includes('/anthropic/v1/')) {
return null;
}
return event;
},
});
8 changes: 0 additions & 8 deletions dev-packages/node-integration-tests/suites/tracing/anthropic/instrument.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,9 @@ import * as Sentry from '@sentry/node';
import { loggingTransport } from '@sentry-internal/node-integration-tests';

Sentry.init({
traceLifecycle: 'static',
dsn: 'https://public@dsn.ingest.sentry.io/1337',
release: '1.0',
tracesSampleRate: 1.0,
dataCollection: { genAI: { inputs: false, outputs: false } },
transport: loggingTransport,
beforeSendTransaction: event => {
// Filter out mock express server transactions
if (event.transaction.includes('/anthropic/v1/')) {
return null;
}
return event;
},
});
406 changes: 191 additions & 215 deletions dev-packages/node-integration-tests/suites/tracing/anthropic/test.ts
View file
Open in desktop

Large diffs are not rendered by default.

10 changes: 0 additions & 10 deletions ...s/node-integration-tests/suites/tracing/anthropic/v0.129/instrument-default-lifecycle.mjs
View file
Open in desktop

This file was deleted.

8 changes: 0 additions & 8 deletions dev-packages/node-integration-tests/suites/tracing/anthropic/v0.129/instrument.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,9 @@ import * as Sentry from '@sentry/node';
import { loggingTransport } from '@sentry-internal/node-integration-tests';

Sentry.init({
traceLifecycle: 'static',
dsn: 'https://public@dsn.ingest.sentry.io/1337',
release: '1.0',
tracesSampleRate: 1.0,
dataCollection: { genAI: { inputs: false, outputs: false } },
transport: loggingTransport,
beforeSendTransaction: event => {
// Filter out mock express server transactions

@Lms24 Lms24 Oct 6, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to be removed without ignoreSpans replacement?

@nicohrubec nicohrubec Oct 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes should be good. this was needed to ignore transactions that we don't assert on. this PR changes the pattern a bit to accumulate arriving spans across envelopes and only fails if the assertions don't hold across all incoming spans. drawback of this pattern is that the runner in the failure case won't be giving us nice assertion failures anymore (instead it times out because it never knows if the correct spans might still come). if this gets annoying we might have to fix this in the runner, haven't found a cleaner solution yet.

if (event.transaction.includes('/anthropic/v1/')) {
return null;
}
return event;
},
});
11 changes: 6 additions & 5 deletions dev-packages/node-integration-tests/suites/tracing/anthropic/v0.129/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { GEN_AI_RESPONSE_ID, GEN_AI_RESPONSE_STREAMING } from '@sentry/conventions/attributes';
import { GEN_AI_RESPONSE_ID, GEN_AI_RESPONSE_STREAMING, SENTRY_OP } from '@sentry/conventions/attributes';
import { afterAll, describe, expect } from 'vitest';
import { cleanupChildProcesses, createEsmAndCjsTests } from '../../../../utils/runner';

Expand All @@ -18,14 +18,14 @@ describe('Anthropic integration (0.129)', () => {
(createRunner, test) => {
test('emits one span for messages.stream(), not a second one for its internal create', async () => {
await createRunner()
.expect({ transaction: { transaction: 'main' } })
.expect({
span: container => {
expect(container.items.find(span => span.is_segment && span.name === 'main')).toBeDefined();
const streamingSpans = container.items.filter(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'msg_stream_1',
);
expect(streamingSpans).toHaveLength(1);
expect(streamingSpans[0]!.attributes['sentry.op']).toEqual({ type: 'string', value: 'gen_ai.chat' });
expect(streamingSpans[0]!.attributes[SENTRY_OP]).toEqual({ type: 'string', value: 'gen_ai.chat' });
expect(streamingSpans[0]!.attributes[GEN_AI_RESPONSE_STREAMING]).toEqual({
type: 'boolean',
value: true,
Expand All @@ -48,16 +48,17 @@ describe('Anthropic integration (0.129)', () => {
createEsmAndCjsTests(
__dirname,
'scenario-beta-stream-helpers.mjs',
'instrument-default-lifecycle.mjs',
'instrument.mjs',
(createRunner, test) => {
test('emits one span each for beta.messages.stream() and the eager streaming tool runner', async () => {
await createRunner()
.expect({
span: container => {
expect(container.items.find(span => span.is_segment && span.name === 'main')).toBeDefined();
for (const id of ['msg_beta_stream', 'msg_tool_runner_eager']) {
const spans = container.items.filter(span => span.attributes[GEN_AI_RESPONSE_ID]?.value === id);
expect(spans, id).toHaveLength(1);
expect(spans[0]!.attributes['sentry.op']).toEqual({ type: 'string', value: 'gen_ai.chat' });
expect(spans[0]!.attributes[SENTRY_OP]).toEqual({ type: 'string', value: 'gen_ai.chat' });
}
},
})
Expand Down
Loading