fix: prevent page.evaluate promises from being collected - #2751
Open
Oxygen56 wants to merge 2 commits into
Open
Conversation
🦋 Changeset detectedLatest commit: d1f487c The changes in this PR will be included in the next version bump. This PR includes changesets to release 11 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
2 issues found across 5 files
Confidence score: 3/5
- In
packages/extension/understudy/frame.ts(Frame.evaluate), throwing a genericErrorwith raw exception details can leak internal/runtime messages throughPage.evaluate, which creates a user-facing error-sanitization risk — switch to a typed error and strip/redact the raw exception text before rethrowing. - In
packages/extension/understudy/frame.ts(Frame.evaluate), the newRuntime.awaitPromise/Runtime.callFunctionOnmaterialization path runs outside the existing retry guard for context-loss errors, so transient context invalidation may now surface as flaky evaluation failures instead of being retried — extend the same "Cannot find context with specified id" retry handling to this post-evaluate step.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/extension/understudy/frame.ts">
<violation number="1" location="packages/extension/understudy/frame.ts:181">
P2: The new materialization step (`Runtime.awaitPromise`/`Runtime.callFunctionOn`) runs against the retained `objectId` outside the try/catch that retries `Runtime.evaluate` on "Cannot find context with specified id". When page churn recreates the context between the evaluate and this follow-up call, the follow-up fails with no retry, so the whole `evaluate` rejects. Consider wrapping the materialize step with the same context-recreation retry (re-running evaluate with a fresh context id) or documenting why a retry is intentionally omitted for the non-evaluate round-trips.</violation>
<violation number="2" location="packages/extension/understudy/frame.ts:183">
P2: Custom agent: **Exception and error message sanitization**
`Frame.evaluate` throws a generic `new Error(...)` that bubbles to users through `Page.evaluate`. Replace it with a typed error class and remove the raw `exceptionDetails.exception?.description` fallback, which surfaces unsanitized JavaScript exception text to the user.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| returnByValue: false, | ||
| }); | ||
| } | ||
| res = await this.materializeEvaluationResult(res); |
Contributor
There was a problem hiding this comment.
P2: The new materialization step (Runtime.awaitPromise/Runtime.callFunctionOn) runs against the retained objectId outside the try/catch that retries Runtime.evaluate on "Cannot find context with specified id". When page churn recreates the context between the evaluate and this follow-up call, the follow-up fails with no retry, so the whole evaluate rejects. Consider wrapping the materialize step with the same context-recreation retry (re-running evaluate with a fresh context id) or documenting why a retry is intentionally omitted for the non-evaluate round-trips.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/extension/understudy/frame.ts, line 181:
<comment>The new materialization step (`Runtime.awaitPromise`/`Runtime.callFunctionOn`) runs against the retained `objectId` outside the try/catch that retries `Runtime.evaluate` on "Cannot find context with specified id". When page churn recreates the context between the evaluate and this follow-up call, the follow-up fails with no retry, so the whole `evaluate` rejects. Consider wrapping the materialize step with the same context-recreation retry (re-running evaluate with a fresh context id) or documenting why a retry is intentionally omitted for the non-evaluate round-trips.</comment>
<file context>
@@ -170,16 +174,47 @@ export class Frame implements FrameManager {
+ returnByValue: false,
});
}
+ res = await this.materializeEvaluationResult(res);
if (res.exceptionDetails) {
- throw new Error(res.exceptionDetails.text ?? "Evaluation failed");
</file context>
| res = await this.materializeEvaluationResult(res); | ||
| if (res.exceptionDetails) { | ||
| throw new Error(res.exceptionDetails.text ?? "Evaluation failed"); | ||
| throw new Error( |
Contributor
There was a problem hiding this comment.
P2: Custom agent: Exception and error message sanitization
Frame.evaluate throws a generic new Error(...) that bubbles to users through Page.evaluate. Replace it with a typed error class and remove the raw exceptionDetails.exception?.description fallback, which surfaces unsanitized JavaScript exception text to the user.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/extension/understudy/frame.ts, line 183:
<comment>`Frame.evaluate` throws a generic `new Error(...)` that bubbles to users through `Page.evaluate`. Replace it with a typed error class and remove the raw `exceptionDetails.exception?.description` fallback, which surfaces unsanitized JavaScript exception text to the user.</comment>
<file context>
@@ -170,16 +174,47 @@ export class Frame implements FrameManager {
+ res = await this.materializeEvaluationResult(res);
if (res.exceptionDetails) {
- throw new Error(res.exceptionDetails.text ?? "Evaluation failed");
+ throw new Error(
+ res.exceptionDetails.text ||
+ res.exceptionDetails.exception?.description ||
</file context>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
why
Closes #2745.
Runtime.evaluatepreviously awaited an evaluation result before retaining a remote object handle. During garbage collection and page churn, Chrome could collect that promise and fail the command withPromise was collected.what changed
test plan
Runtime.evaluate -32000 Promise was collected, while the retained-handle strategy resolves the same controllable promise to the expected value.git diff --check.Summary by cubic
Prevents Chrome from collecting
Page.evaluateresults during GC by retaining the remote handle before awaiting. Old behavior awaited inRuntime.evaluateand sometimes failed with “Promise was collected”; new behavior evaluates by reference, then materializes via the retained object, reducing flakiness.Runtime.awaitPromiseon the returnedobjectId, thenRuntime.releaseObject.Runtime.callFunctionOnto return-by-value, thenRuntime.releaseObject.exception.descriptionwhen present.Page.evaluateto delegate to frame-level evaluation for consistency.@browserbasehq/stagehand-extensionarchive and adds patch changesets for@browserbasehq/stagehand,@browserbasehq/stagehand-python, and@browserbasehq/stagehand-goto ship the fix.Written for commit d1f487c. Summary will update on new commits.