Repository navigation
Discussion: Soliciting LTS input from Node user community #8
Description
As an ongoing effort, we need to be actively soliciting input on LTS requirements from the node user base. Let's use this issue to track the effort.
(Carried over from nodejs/dev-policy#67)
Activity
jasnell commentedon May 18, 2015
1.What is a reasonable LTS release schedule? (2 per year, 1 per year, 1 per month, etc)
2.What is a reasonable LTS support timeframe (6 months, 1 year, 5 years, etc)
3. What is a reasonable LTS maintenance policy (fixes only, etc)
4. Is there anything in the proposed dev-policy that raises red flags as far as LTS support is concerned?
5. What's missing / What haven't we thought about yet ?
jomo commentedon May 18, 2015
I totally agree with @mzdv. 👍
drewfish commentedon May 18, 2015
For "2. What is a reasonable LTS support timeframe" does it make sense to distinguish a "normal" level of support versus a "long-tail" support when only severe security bugs are fixed/backported?
domenic commentedon May 18, 2015
FWIW here is Firefox's LTS stuff: https://www.mozilla.org/en-US/firefox/organizations/faq/
One year support, with a 2-month overlap of support. So, I think that comes down to
Raynos commentedon May 18, 2015
Production and system debuggability stability is one of the reasons we would want a LTS.
This means having a fixed version of v8. With a fixed version of v8 any dtrace or system tap tooling that is hard coupled to the way the v8 stack is layed out in memory will just work. With a fixed version of v8 any post mortem debugging tooling that is hard coupled to the way v8 lays out the heap in memory will just work.
Pay very careful attention to any internal changes in v8 patch versions that might break system debugging tools. With all of the new ES6 features being implemented I'm worried about the patch velocity of v8.
trevnorris commentedon May 19, 2015
@domenic Being, officially or unofficially, the node/v8 liaison, you were keen on getting io.js to keep pace with v8 for a variety of reasons. Including the fact that v8 doesn't offer any LTS, and because their patch rate is at a near ridiculous speed. So in your opinion how would this be any different then the previous issue Node.js had? I guess the whole idea is that companies who care about LTS will be the ones contributing to its maintenance without any support from the v8 team?
(this is a clarifying question for anyone else who participates in this thread)
domenic commentedon May 19, 2015
Yes, I don't see a way for Node.js to do LTS releases without effectively taking on the maintenance burden of doing LTS V8 releases as well. Given how many security bugs alone are found in each 6-week release cycle, this is quite a lot of work. There's a reason the V8 team in particular, and the Chrome team in general, has eschewed LTS in favor of auto-updating: the security team strongly believes that's the only viable option.
domenic commentedon May 19, 2015
Which is to say, Google will not be supporting or supportive of any LTS releases that include outdated V8 versions. I doubt that changes the fact that Node.js LTS will probably want to try to pin to a V8 version, but it is what it is.
domenic commentedon May 19, 2015
Sorry to spam. "Google will not" is a bit strong. "The Chrome/V8 teams will not" is more accurate. I know the Google Cloud team has some nascent interest in being able to better support Node.js in this area, and I don't speak for them.
jasnell commentedon May 19, 2015
That's a fundamental issue really. It simply needs to be recognized that
LTS node releases will have to deal with a backlog of V8 issues. So long as
there is a process for moving things along, it's going to have to ok for
it not to be a perfect process.
On May 18, 2015 6:45 PM, "Domenic Denicola" notifications@github.com
wrote:
lfortin commentedon May 19, 2015
My $0.02:
1- 2 releases per year for new LTS branches, 1 release per year for older LTS branches
2- as long as resources are available to support them(could be 5+ years)
3- bug fixes, security fixes
4- no idea
5- no idea
bnoordhuis commentedon May 19, 2015
It's a pretty critical and fundamental issue though.
Maintaining an LTS version of V8 could easily turn into a full-time job. I know of only a handful of people outside the Googleplex who are up to the task and I don't think any of them will be volunteering anytime soon.
It's not the kind of code base that you pick up easily either. It's complex, massive (over 1.3 million lines of code if you include the test suite) and full of undocumented institutional knowledge. You can't just hire a random guy off the street and tell him to start maintaining it.
trevnorris commentedon May 19, 2015
Is there a bug describing what the LTS is looking for? Besides not upgrading V8 for a year.
I'd like to say the likelihood of being able to back port every fix for 1 year (which is ~8 releases) and never break API/ABI is very low. So instead it might be beneficial to entertain solutions like providing a compatibility later. Which would allow more to happen under the hood while still not requiring native addons to be recompiled.
Just food for thought. What I mean by bringing this up is that we should consider thinking outside the conventional box as to what an LTS could be.
EDIT: To clarify my first question. It does differ from @jasnell's question of wanting a "reasonable LTS maintenance policy". Instead I'm looking for a discussion about what are the technical reasons wanting to stay on the same V8 version.
Raynos commentedon May 20, 2015
@trevnorris
At Uber we care about post mortem debugging. We care about being to introspect core files from production issues.
This means we want to write tools to analyze the stack and the heap and we do not want them to break when we change versions of node;
This has worked for us when we upgraded from node 0.10.26 -> 0.10.32 -> 0.10.36
I don't need the same V8 versions; I just need a core file analyze tool that works; 100% of the time.
drewfish commentedon May 20, 2015
tl;dr: ABI is stable within a major.minor version.
At Yahoo we have a lot of addons to glue our internal c/c++ libraries into javascript. We have CI/CD setup for all of these so rebuilding isn't too hard. We almost have them all on nan so the API isn't too much of an issue. (There are still things that nan doesn't quite cover, for which we have a few conditional macros.)
We're happy to build different versions of each addon for different ABIs. We currently do this for node 0.10 and 0.12 for example. These built addons get published to a different package system (something like a yum repo). So for example, for addon
foo@1.2.3we buildnodejs_foo@0.10-1.2.3andnodejs_foo@0.12-1.2.3. When we setup a machine (or docker image, etc) for a node 0.10 runtime we install all the appropriatenodejs_*@0.10-*packages.The trick is that we don't want to be building too many different versions of every addon. If the ABI is stable within a major.minor version that would be great. We don't intend to use every major.minor versions, just a select few.
(And by major.minor I'm completely ignoring 0.x since according to semver "anything may change at any time".)
3 remaining items
trevnorris commentedon May 20, 2015
@Raynos @drewfish Thank you both for the feedback.
Pulling information from here and others I've spoken with, the only reason I've seen to keep the same major.minor is for tooling or compiling native addons, and I have not personally seen any business requirements requiring a dependency of node not be upgraded. Is this a fair assumption?
@domenic @bnoordhuis What is the likelihood that patches from 8 or 12 releases could be back ported without ever breaking API/ABI? (I realize this is very hard to estimate, but will also guide the direction of an LTS release)
@rvagg Pulling you in.
For everyone: What if we redirected the technical efforts of maintaining an LTS towards creating a stable API/ABI so native modules don't need to be recompiled every 6 weeks, and also toward maintaining the necessary tooling? As Ben mentioned, it will be difficult to find someone that could do this type of work. Also the workload of maintaining an LTS the alternative way could be more easily spread out among many contributors.
bjanakiraman commentedon May 20, 2015
I'd like to add to what @domenic said. I've talked a few of you via email in the past (and have talked with Domenic as well).
I'm part of the Google Cloud team and manage some of Google's node.js efforts, and I see the pain for node users. We are just getting started here, with a couple of people working on node. But the hope is that we'll be able to help with both supporting older versions of v8 better, and increased/better communication with the v8 team.
Before reading too much into this, please note it takes time and effort to build up expertise, and it may be a while before we have concrete progress.
bnoordhuis commentedon May 22, 2015
Hard to say. I won't say never but, given enough effort, you can probably almost always avoid it.
I think a bigger issue is that with the ever-growing delta between upstream and downstream, you eventually get to a point where back-porting patches no longer works, and where it's not even clear if a change is conceptually relevant because the code bases have diverged too much.
V8 has massive amounts of churn at times so it's better to assume you'll hit that point sooner rather than later.
ChALkeR commentedon May 23, 2015
Reasoning:
3 months of each year there will be two «bugfixes» LTS releases (one of them stabilizing) and one «security» release.
Disclaimer: an outsiders point of view, without binding to v8 releases and such.
trevnorris commentedon May 24, 2015
@bnoordhuis Thanks for the insight.
So it seems we would rapidly arrive at the point where we'd have to start writing our own bug fixes from scratch. And from what I've gathered in this thread there would be two concurrent LTS's we'd need to do that for.
So I'd like to ask me biggest concern. Is this feasible? As Ben has already pointed out, finding someone with the knowledge of being able to help maintain V8 is difficult.
If we have a concrete plan of how to address this issue then I'll shut up and go my way, but I think that making plans for how LTS should be supported without at the same time addressing how it can be achieved only sets us up for failure.
mikeal commentedon May 24, 2015
One thing I'd like to point out is that we're continuing to operate under the historical assumption that we'll have to source the resources for any v8 work that needs to be done on behalf of node. This has been pretty true for a long time but is starting to change.
Currently, more people from Google/v8 are working directly on node than ever before. Google Cloud Platform is invested in providing a great node experience and once that support has been in place for a while we should expect GCP to have the same concerns about LTS on behalf of their customers as we see here from other node hosting services.
There are no guarantees but I wouldn't be so quick to dismiss the idea of existing Google/v8 resources being invested in LTS some time in the near future if the need arises.
nmabhinandan commentedon May 25, 2015
1.What is a reasonable LTS release schedule? (2 per year, 1 per year, 1 per month, etc)
1 per year2.What is a reasonable LTS support timeframe (6 months, 1 year, 5 years, etc)
2 years3. What is a reasonable LTS maintenance policy (fixes only, etc)
bug/security fixes, obvious optimizations4. Is there anything in the proposed dev-policy that raises red flags as far as LTS support is concerned?
nope5. What's missing / What haven't we thought about yet ?
nopemikeal commentedon May 27, 2015
Some thoughts on adding features in LTS releases.
The assumption here seems to be that taking features in to LTS releases opens us up to more harm than good. That might be true most of the time but there are actually cases where not taking a feature means more breakage than taking it. Adding features to current releases opens up the possibility of causing forward incompatible breaks in the ecosystem. Even with node's historically modest feature additions and long cycles we've already seen this play out with npm's
~vs.^where newly published modules were breaking in all prior releases of node that had not upgraded npm independently of the one that ships with node.When it comes to adding features in LTS releases we should stay away from hard rules like "no features or API additions" and instead try to weight the cost of not adding them on a case-by-case basis.
trevnorris commentedon May 27, 2015
@mikeal
It seems one of the biggest desires for the LTS is that native modules will never have to be recompiled against the same LTS branch. Adding or changing any native API will require recompiling.
mikeal commentedon May 27, 2015
@trevnorris good point. I think we already have a policy that states that a major will never break ABI except in the case of some unavoidable security update.
rvagg commentedon Jun 29, 2015
Perhaps we should consider the option of maintaining a fork of V8 here in the nodejs org with the help of Google, perhaps we can convince them, with the help of @domenic and the Google Could team, to start considering an LTS model for V8 within their current development roadmap, even if it's just throwing issues over the wall where they concern security and stability that we can deal with over here without having to be deeply involved in the cutting-edge work.
What are the chances of organising a V8 Summit of some kind with representatives from Google and here to work this through?
jasnell commentedon Jun 29, 2015
That's a very good idea.
On Jun 28, 2015 8:44 PM, "Rod Vagg" notifications@github.com wrote:
trevnorris commentedon Jun 29, 2015
If we simultaneously tracked V8 issues, since they're labeled and all, and had their team ping us whenever a relevant commit landed that should be back ported it could help lighten the load of combining through and finding applicable commits.
We still need to assess feasibility. @bnoordhuis did a quick analysis and found over one million LOC changed in a 18 month period. Even if a fraction of those are stability and security related, that's a lot of work.
rvagg commentedon Jul 7, 2015
Suggesting we move this discussion to a new issue now that we have a concrete proposal in place.