Emit does not preserve empty lines

Hi,

TS Version: 1.1

Given

function foo() {

    var x = 10;

    var y = 11;
}

We used to get

function foo() {
    var x = 10;

    var y = 11;
}

In the new compiler the line break is missing

function foo() {
    var x = 10;
    var y = 11;
}

(Both compilers removed the first empty line, but the new compiler has gone a step further.)

This can affect the experience when debugging JavaScript in the browser.

Noel Abrahams (@NoelAbrahams)

Adding some background info... The reason the new compiler removes all blank lines is that this is the only thing we can do consistently. Blank line preservation is always going to be a hit or miss thing when it comes to constructs that are subject to rewriting, e.g. class and module declarations. We face exactly the same issues with comment preservation. So, while I'm sympathetic to resolving this, it's not an easy thing to do.

Noel Abrahams (@NoelAbrahams) I'm curious what issues you see when debugging?

Anders Hejlsberg (@ahejlsberg)

Anders Hejlsberg (@ahejlsberg) Noel Abrahams (@NoelAbrahams) I created a prototype emitted in the original CodePlex project that actually did a terrific job with comment and newline preservation. Even on constructs that got heavily rewritten (like arrow-functions to function expressions) it would almost always do what you intuitively expected it to do.

Line preservation mostly a function of just reusing the line breaks when preserving old code, and using relative spacing when doing transformations. i.e. if you have:

module M {
}
module N {
}

Then when you're emitting the IIFE for 'N' you say "we should keep the trivia between the module we're rewriting and the previous syntactic element".

Here is the before/after for how my emitter prototype worked that preserved comments/newlines with high levels of fidelity:

https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter2/ecmascript5/Parser.ts
https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter2/ecmascript5/Parser.ts.expected

You can see a lot of examples here as well:
https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter/ecmascript5/

The only feature i didn't implement was 'alignment'. i.e. if you had code that was aligned in some way in the original (very common with parameter declarations), we'd want the emitted code to preserve that as well.

But it would have been very trivial to do.

That said, conversion of contructs from TS to JS did try to preserve indentation. You can see that here:
https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter/ecmascript5/ClassDeclaration/ClassDeclaration2.ts
https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter/ecmascript5/ClassDeclaration/ClassDeclaration2.ts.expected

Note how the statements are properly indendented (even when they spread over multiple lines) even after we convert the nested modules and class into IIFEs

Anders Hejlsberg (@ahejlsberg)

Anders Hejlsberg (@ahejlsberg), there are no significant issues when debugging in the browser. It just makes it easer to navigate the JavaScript code and locate lines for setting breakpoints when there is an exact correspondence with the actual TypeScript source code.

I could personally live without the empty lines, but since TS has gone to such lengths to preserve and emit _beautiful_ JavaScript (:smile:) it seems only natural that this is implemented.

Anders Hejlsberg (@ahejlsberg)

Anders Hejlsberg (@ahejlsberg) Noel Abrahams (@NoelAbrahams) There is one issue that exists with debugging in the browser that is slightly related to this conversation. When using setter/getter chains (like with jquery) or chaining promises, the new line feeds are lost during translation. That being said, it is a huge pain point when working with Arrow functions.

As an example:

(<any> x).a('#test')
    .b('test')
    .c(() => 'foo')
    .d(() => 'bar')
    .e(() => 5)
    .f(() => 6);

Becomes:

x.a('#test').b('test').c(function () { return 'foo'; }).d(function () { return 'bar'; }).e(function () { return 5; }).f(function () { return 6; });

Using Chrome and sourceMaps, the breakpoints are still skipped.

http://www.typescriptlang.org/Playground#src=(%3Cany%3E%20x).a('%23test')%0A%20%20%20%20.b('test')%0A%09.c(()%20%3D%3E%20'foo')%0A%09.d(()%20%3D%3E%20'bar')%0A%09.e(()%20%3D%3E%205)%0A%09.f(()%20%3D%3E%206)%3B

Matt Traynham (@mtraynham)

Matt Traynham (@mtraynham), actually I think the problem you highlight is a slightly different one.

In previous versions the body of an inline function was always emitted on new lines:

// TS
var x = () => 'foo';

// JS - old
var x = function () { 
             return 'foo'; 
       };

// JS - new
var x = function () { return 'foo'; };

I too have found this to be a problem - having to occasionally go back and create a function so that I can set a breakpoint when debugging in the browser.

Noel Abrahams (@NoelAbrahams)

Noel Abrahams (@NoelAbrahams) Ahh yes, I have been using this exact same temp solution... I wasn't sure if this was an appropriate bug to contribute this problem to (erasure of line feeds), or should I open another?

I created #2259 to the separate issue.

As an engineering director exploring moving our javascript development community to typescript the new line capability would really help. One of the primary appeal of typescript was maintaining the readability and structure of the code created in typescript generatedJavascript.

Great to see comments are kept in the recent update and addition of the “--removeComments" command line directive.

In regard to empty lines, would it be possible - for now - to have them emitted, even if they are hit or miss? Such experimental feature could be asked with a “--keepEmptyLines" option. It would be not so much for having nice JS, but for having more readable JS.

In regard to chained function calls, would it be possible to have one call for line, to let users set breakpoints? Again, this feature could be asked with a "--oneCallForLine" option, if it is another "hit or miss" thing.

Thanks for your attention.

Actually, chained function calls could be split by a source code beautifier, therefore it doesn't make sense to embed such feature in TypeScript.

This shouldn't be that hard, shouldn't there just be an option in tsconfig.json

preserveWhitespace: true/false

?

Is there a way I could contribute to fixing this? If I could get some guidance around what an acceptable fix might look like, I'd be willing to take a stab at it.

Do not include js files in TS compilation

Rafał (@CarabusX)

Rafał (@CarabusX) I took slightly minimalistic approach, but very similar to yours:

        function preserveWhitespace(content) {
          return content.replace(/(\s)/g, (m) => {
            return `/**_P${m.charCodeAt(0)}**/`;
          });
        }
        function restoreWhitespace(content) {
          return content.replace(/\/\*\*_P(\d+)\*\*\//g, (_, m) => {
            return String.fromCharCode(m);
          });
        }
        content = preserveWhitespace(content);
        const output = ts.transpileModule(content, {
          reportDiagnostics: false,
          compilerOptions: {
            target: ts.ScriptTarget.ESNext,
            importsNotUsedAsValues: ts.ImportsNotUsedAsValues.Preserve,
            sourceMap: true,
            removeComments: false,
          },
        });
        output.outputText = restoreWhitespace(output.outputText.replace(/\s+/g, ''));

however it doesn't work well with imports

I tried many suggestions from this thread but none worked for me. After a while, however, I managed to come up with my own solution.

Before I share the code, I should note that this code relies on Prettier for formatting to get the JS code consistent with the TS code. I already use Prettier in all my projects, so I just copied the config I am using anyway.

One final note, as my use case for this was the Docusaurus website for https://github.com/sapphiredev, I have also decided to publish this conversion and formatting as an NPM package: https://www.npmjs.com/package/@sapphire/docusaurus-plugin-ts2esm2cjs

Now on to the code. This transpiles TS into ESM and then further converts that ESM into CJS.

const { runTransform } = require('esm-to-cjs');
const ts = require('typescript');
const prettier = require('prettier');
const sapphirePrettierConfig = require('@sapphire/prettier-config');

const documentationPrettierConfig = {
	...sapphirePrettierConfig,
	tabWidth: 2,
	useTabs: false,
	printWidth: 120,
	parser: 'babel'
};

/** @type {import('ts').CompilerOptions} */
const tsCompilerOptions = {
	module: ts.ModuleKind.ESNext,
	newLine: ts.NewLineKind.LineFeed,
	moduleResolution: ts.ModuleResolutionKind.NodeJs,
	target: ts.ScriptTarget.ESNext,
	removeComments: false,
	esModuleInterop: true,
	pretty: true
};

/**
 * Transpiles input TypeScript code to ESM code.
 * @param {string} code The code to transpile
 * @returns {{ outputText: string; diagnostics: unknown[]; sourceMapText?: string }} Input code transpiled to ESM
 */
const tsToEsm = (code) => ts.transpileModule(code, { reportDiagnostics: false, compilerOptions: tsCompilerOptions });

/**
 * Transforms input ESM code to CJS code.
 * @param {string} code The code to transform
 * @returns Input code transformed to CommonJS
 */
const esmToCjs = (code) => runTransform(code, { quote: 'single' });

/**
 * Escaped new lines in code with block comments so they can be restored by {@link restoreNewLines}
 * @param {string} code The code to escape new lines in
 * @returns The same code but with new lines escaped using block comments
 */
const escapeNewLines = (code) => code.replace(/\n\n/g, '\n/* :newline: */');

/**
 * Reverses {@link escapeNewLines} and restores new lines
 * @param {string} code The code with escaped new lines
 * @returns The same code with new lines restored
 */
const restoreNewLines = (code) => code.replace(/\/\* :newline: \*\//g, '\n');

/**
 * Formats the code using Prettier
 * @param {string} code The code to prettier format
 * @returns Prettier formatted code
 */
const prettierFormatCode = (code) => prettier.format(code, documentationPrettierConfig).slice(0, -1);

/**
 * The example code that we'll be transpiling and formatting.
 * Of note here is that it also has a type only import that will be removed during transpilation,
 * and it needs to be accounted for with the restoration of new lines.
 */
const codeStr = `import { Command, Args } from '@sapphire/framework';
import type { Message } from 'discord.js';

export class EchoCommand extends Command {
  public constructor(context: Command.Context, options: Command.Options) {
    super(context, {
      ...options,
      aliases: ['parrot', 'copy'],
      description: 'Replies with the text you provide'
    });
  }

  public async messageRun(message: Message, args: Args) {
    // ...
  }
}`;

const emptyLineEscapedTsCode = escapeNewLines(codeStr);

const esmCode = tsToEsm(emptyLineEscapedTsCode).outputText;
const cjsCode = esmToCjs(esmCode);

I would also like to see this implemented, with a few specific reasons in mind:

Consistency with general design philosophy per NoelAbrahams' comment above
Ciantic's comment about needing to deliver JS elsewhere
Being able to cleanly compare JS generated through TS and JS that existed before, where demonstrating a close match is important to adoption
Not having long lines requiring horizontal scrolling in the emitted code, where the source included line breaks to make things neat and beautiful and meeting linter/style specs (also commented on above)
Being able to continue working with transpiled JS if all these overhead costs of TypeScript become too much; anticipating difficulty here creates lock-in costs that raise barriers to adoption in the first place.

This PR, discussed in the 3.9 release notes, show code actions preserving newlines; why can't the emitted code do something similar?

Are there any updates for almost 10 years?
I would like to be able to configure preserving new lines

I'm changing this to Won't Fix. Reasons:

This has obviously not been a key blocker for anyone
Of all the "keep trivia" problems we have, this is surely the lowest-priority among them
At this point, many alternative transpilers with different configurable behaviors around comments/whitespace exist; if you need truly need newline preservation, you can use one of those
Performance remains top-of-mind for us and anything that makes emit slower needs to be carrying a lot more value than newline preservation.

At this point, many alternative transpilers with different configurable behaviors around comments/whitespace exist

Does anyone know of a good option for this?

At this point, many alternative transpilers with different configurable behaviors around comments/whitespace exist

Does anyone know of a good option for this?

It depends on your exact needs. There are several suggestions in this issue and a list of other transpilers is as follows. Note that this list hasn't been sanitized for the specific requirement:

tsup (uses esbuild or swc depending on config)
esbuild
swc
babel (with @babel/plugin-typescript)
rollup (with @rollup/plugin-typescript or rollup-plugin-typescript2)
turbopack

spike: TypeScript migration comparison

Navigation Menu

Repository navigation

Emit does not preserve empty lines #843

27 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Emit does not preserve empty lines #843

Description

Activity

ahejlsberg commented on Oct 9, 2014

CyrusNajmabadi commented on Oct 9, 2014

CyrusNajmabadi commented on Oct 9, 2014

NoelAbrahams commented on Oct 9, 2014

mtraynham commented on Mar 3, 2015

NoelAbrahams commented on Mar 4, 2015

mtraynham commented on Mar 5, 2015

timjmartel commented on Feb 24, 2016

fehmud commented on Jul 20, 2016

fehmud commented on Jul 21, 2016

ORESoftware commented on Feb 19, 2017

27 remaining items

edsrzf commented on Apr 29, 2021

sdwvit commented on Jun 25, 2021

favna commented on Nov 26, 2021

wbt commented on Feb 16, 2022

devope commented on Jan 3, 2024

RyanCavanaugh commented on Jan 3, 2024

rattrayalex commented on Jan 4, 2024

favna commented on Jan 4, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions