Skip to content

Fix TypeBuilder bug with repeated uses of a type - #3571

Closed
tlively wants to merge 1 commit into
WebAssembly:mainfrom
tlively:fix-type-builder
Closed

tlively wants to merge 1 commit into
WebAssembly:mainfrom
tlively:fix-type-builder

Conversation

@tlively

@tlively tlively commented Feb 17, 2021

Copy link
Copy Markdown
Member

Previously TypeBuilder would fail to properly replace all temporary types with
globally canonical types when a type was used more than once. Consider this
temporary type:

(struct (field (ref (struct)) (ref (struct))))

This heap type contains two (ref (struct)) each of which contains an empty
(struct) heap type. The logic for avoiding duplicate scanning meant that the
first inner (struct) was scanned, but the second one was not. The expectation
was that this would be fine because both occurrences of (ref (struct)) are
backed by the same temporary TypeInfo, so replacing the inner type of one
would update both at once. However, because scanning is depth-first, the second
(ref (struct)) is scanned after and therefore canonicalized before the first
inner (struct). Since (ref (struct)) was canonicalized before its inner
(struct), the temporary (struct) heap-type was accidentally leaked into the
global canonicalization of the type. This caused the global canonicalization to
be incorrect and also caused use-after-free bugs once the TypeBuilder was
disposed of.

One solution would have been to calculate a topological sort of the types to
determine the canonicalization order, but a simpler solution is to simply not
worry about duplicate work, scanning and canonicalizing every occurrence of
every type in the TypeBuilder's entries. This PR implements the simpler
solution.

Previously TypeBuilder would fail to properly replace all temporary types with
globally canonical types when a type was used more than once. Consider this
temporary type:

```
(struct (field (ref (struct)) (ref (struct))))
```

This heap type contains two `(ref (struct))` each of which contains an empty
`(struct)` heap type. The logic for avoiding duplicate scanning meant that the
first inner `(struct)` was scanned, but the second one was not. The expectation
was that this would be fine because both occurrences of `(ref (struct))` are
backed by the same temporary `TypeInfo`, so replacing the inner type of one
would update both at once. However, because scanning is depth-first, the second
`(ref (struct))` is scanned after and therefore canonicalized before the first
inner `(struct)`. Since `(ref (struct))` was canonicalized before its inner
`(struct)`, the temporary `(struct)` heap-type was accidentally leaked into the
global canonicalization of the type. This caused the global canonicalization to
be incorrect and also caused use-after-free bugs once the TypeBuilder was
disposed of.

One solution would have been to calculate a topological sort of the types to
determine the canonicalization order, but a simpler solution is to simply not
worry about duplicate work, scanning and canonicalizing every occurrence of
every type in the TypeBuilder's entries. This PR implements the simpler
solution.
@tlively
tlively requested a review from kripken February 17, 2021 07:21

@kripken kripken left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay for the simple solution!

@tlively

tlively commented Feb 18, 2021

Copy link
Copy Markdown
Member Author

Unfortunately it turns out that the simple solution is insufficient :'( For more complex patterns of types, a similar thing happens where a type ends up being canonicalized before its children. The next simple solution would be to entirely remove the logic for avoiding visiting a node more than once, but that wouldn't work once we want to support recursive types. I think I need to do a topological sort on the types after all. Thankfully the code already collects a dependency graph, so this won't be too much extra code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants