2.5gb on small project

I have a small project of Natural Language Processing and there are 2 files with 37k lines, which contains the datasets. The problem occurs even when i'm not using those files. Not even opening the files. I have to use other editor to run "node index.js" -my main fail.

I dont know how the problem starts, but when i open the vscode editor, it goes up to 2.5gb of ram on the startup. I dont know what is happening.

/needsMoreInfo

What is the file type of the data files?

Thanks for creating this issue! We figured it's missing some basic information or in some other way doesn't follow our issue reporting guidelines. Please take the time to review these and update the issue.

Happy Coding!

/needsMoreInfo

What is the file type of the data files?

They are all javascript and json files.
Vscode takes a while to shoot up the memory. When it does, the ram usage goes up to 2.5gb and starts to freeze some features.

Does it reproduce when starting with code --disable-extensions?

Yes, it does. I found out that the process “ts/js language features” is the responsible for the 2.5gb usage. When i manually disabled the feature, it starts to work normally. But only without JS intellisense features, and thats sad :/ Em ter, 13 de nov de 2018 às 07:33, Christof Marti <notifications@github.com> escreveu:

…

Does it reproduce when starting with code --disable-extensions? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/Microsoft/vscode/issues/62922#issuecomment-438198065>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQ_i6VoFW1gVR_7fKcElraQrdlp9Rbyzks5uupH7gaJpZM4YYn5e> .

Eduardo Souza (@edunsouza)

Eduardo Souza (@edunsouza) Can you share those source files?

Matt Bierner (@mjbvz)

Matt Bierner (@mjbvz) of course I can! And I'll be happy if I can help to find out the problem.

The files are available on the repository: https://github.com/edunsouza/communicator

Tested using typescript@3.2.0-dev.20181114

Switching between the js files in dataset/ I can still easily get up to over a GB. Seems mostly to happen after switching to another js file not under dataset.

Logs:
tsserver.log

Nathan Shively-Sanders (@sandersn)

Nathan Shively-Sanders (@sandersn) and Anders Hejlsberg (@ahejlsberg), any ideas on how we could do better here?

The issue looks to be with the dataset/afinn/afinn.js file. It contains an array literal with 30K+ elements specified as object literals:

module.exports = [
    {texto: ";)", sentimento: 1, ";)": 1},
    {texto: ";@", sentimento: -1, ";@": -1},
    {texto: ";*", sentimento: -1, ";*": -1},
    {texto: ";**", sentimento: -1, ";**": -1},
    {texto: ";~", sentimento: -1, ";~": -1},
    ...
];

The reverse mapping pattern means that each object literal has a unique type. To make matters worse, object literal normalization (#19513) causes us to expand into 30K+ unique types with 30K+ properties each (!!!). We then attempt to perform subtype reduction on the set of types in order to make a union type. This ends up generating enormous amounts of work. If I reduce down to 2500 unique types we still take close to 30 seconds to type check, so with 30K+ types we'll effectively take forever.

I'm thinking we should consider a governor on union subtype reduction. Given that is an O(n^2) operation, perhaps we limit subtype reduction to unions with 500 or less constituents.

Oops, meant to close the old PR, not the issue.

Navigation Menu

Repository navigation

2.5gb on small project #28540

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

2.5gb on small project #28540

Description

Activity

chrmarti commented on Nov 12, 2018

vscodebot commented on Nov 12, 2018

edunsouza commented on Nov 13, 2018

chrmarti commented on Nov 13, 2018

edunsouza commented on Nov 13, 2018

mjbvz commented on Nov 14, 2018

edunsouza commented on Nov 14, 2018

mjbvz commented on Nov 15, 2018

DanielRosenwasser commented on Nov 15, 2018

ahejlsberg commented on Nov 16, 2018

ahejlsberg commented on Jan 28, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions