Improve module expression parsing/printing

[JLHwung]

Q	A
Fixed Issues?	See below
Patch: Bug Fix?	Yes
Major: Breaking Change?
Minor: New Feature?
Tests Added + Pass?	Yes
Documentation PR Link
Any Dependency Changes?
License	MIT

This PR fixes several module expression related parsing / printing issues

• The program of a module expression should not end at }, instead it should end before } because } is not part of the embedded program
• Print directives / comments of the embedded program
• Print inner comments of a module expression, i.e. the block comments between module and {
• Ensure no line terminators between module and {

[babel-bot]

Build successful! You can test your changes in the REPL here: https://babeljs.io/repl/build/53063/

[JLHwung]

Before this PR we parses this input as if it were

// note the extra required `{` after `module`
module {
  {| foo: 42 |}
}

This is now fixed by expecting { after module.

[nicolo-ribaudo]

Why do we need to eat it?

[JLHwung]

tt.eof is eaten when we are parsing top level program. If it is not eaten, the tt.eof token won't be pushed to this.tokens.

[nicolo-ribaudo]

Can we move it as tt.expect in parseTopLevel, to assert that we actually consumed all the tokens?

[JLHwung]

If we eat tt.eof in parseTopLevel, the Program will end earlier than it should be and as a result its trailing comments won't append. I revises it a bit and now it only consume when end is tt.eof, which must be invoked by parseTopLevel.

[liuxingbaoyu]

It seems that currently our generator is not allowed to throw exceptions, I personally prefer not to have this try.

[JLHwung]

The source map is updated because we now generate two sections module{ and } and map them respectively for the empty module expression module{}. Before this PR we map module{} as module{}.

[nicolo-ribaudo]

Maybe we could optimize this in the future (directly or in one of @jridgewell's packages) by merging source map segments that are adjacent both in the input and in the output.

[liuxingbaoyu]

This shouldn't be technically difficult, but I'm not sure that's better. In the last PR I marked the sourcemap for every ";".

[jridgewell]

This has to be done in Babel, because the sourcemap packages will not know whether this marking is important or not. Eg, function foo() { return bar; }, we should generate a marking for bar and } (the } is required for Chrome's debug stepping). I can't make the source map package ignore this module {} case without also ignoring that case.

[jlowcs]

Following this change, one of my TS projects is not building anymore. I'm seeing:

Module build failed (from ./node_modules/babel-loader/lib/index.js):
TypeError: /tmp/workspace/x/y/z/myFile.ts: Cannot read properties of undefined (reading 'start')
    at Generator._printComments (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:954:48)
    at Generator._printLeadingComments (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:780:10)
    at Generator.print (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:629:10)
    at Generator.printJoin (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:733:12)
    at Generator.printList (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:804:17)
    at Generator.cb [as CallExpression] (/tmp/workspace/node_modules/@babel/generator/src/generators/expressions.ts:212:8)
    at Buffer.exactSource (/tmp/workspace/node_modules/@babel/generator/src/buffer.ts:380:5)
    at Generator.exactSource (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:306:15)
    at Generator.print (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:633:10)
    at Generator.printJoin (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:733:12)
    at Generator.printList (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:804:17)
    at Generator.cb [as CallExpression] (/tmp/workspace/node_modules/@babel/generator/src/generators/expressions.ts:212:8)
    at Generator.exactSource (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:302:22)
    at Generator.print (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:633:10)
    at Generator.printTerminatorless (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:557:12)
    at printStatementAfterKeyword (/tmp/workspace/node_modules/@babel/generator/src/generators/statements.ts:139:13)
    at Generator.cb [as ReturnStatement] (/tmp/workspace/node_modules/@babel/generator/src/generators/statements.ts:157:3)
    at Generator.exactSource (/tmp/workspace/node_modules/@babel/generator/src/printer.ts:302:22)

The files that are failing to compile are using comments inside dynamic imports:

loadable(async () => import(/* webpackChunkName: "pages/psf_comment" */ 'src/app/pages/psf/comment'), {
	exportName: 'PsfComment',
	fallback: Loading,
})

[liuxingbaoyu]

@jlowcs Could be this #15031, a fix should be released soon, thanks for the report.
ref: #14979