new treesitter syntax highlighting fails to correctly handle indented heredoc limitstring when redirection operator is <<- #127

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Reproduced the problem in Safe Mode: http://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
  • Followed all applicable steps in the debugging guide: http://flight-manual.atom.io/hacking-atom/sections/debugging/
  • Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
  • Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description

I think there's been a regression. I'm not sure when it started, since I've been doing pretty much only python for quite a few months, but I just opened a shell script in atom-1.32.0-beta0 and immediately noticed the syntax highlighting was off. Someone else mentioned to me that the change might have been due to the new "Tree-sitter parsing system".

As the subject says, syntax highlighting fails to allow a tab-indented "limit string" (commonly something like EOF) when the here-document redirection operator is <<-. The BASH man page makes it crystal clear this should be allowed.

$ man bash | grep -A2 'If the redirection operator is <<-'
   If the redirection operator is <<-, then all leading tab characters are stripped
   from input lines and the line containing delimiter. This allows here-documents
   within shell scripts to be indented in a natural fashion.

Steps to Reproduce

1. Start a heredoc with <<-EOF
2. Finish it off with a tab-indented EOF

Expected behavior: [What you expect to happen]

When a heredoc uses a dash in its redirection operator (i.e., <<-), the limit string must be allowed to be tab-indented.

Actual behavior: [What actually happens]

When a heredoc uses a dash in its redirection operator (i.e., <<-), atom currently requires the limit string to be at the beginning of a line.

Reproduces how often: [What percentage of the time does it reproduce?]

100%

Versions

1.32.0-beta1 on Fedora 28

Additional Information

The following example code with an indented limit string (and only << redir operator) is invalid. Notice that Github's colorization correctly sees it as a problem (as does vim and atom).

#!/bin/bash
usage() {
	cat <<EOF
		Usage: ${0##*/} FOO BAR
	EOF
}
[[ $# -eq 2 ]] && echo $2 $1 || usage

The next code block has it fixed, by adding the -. Note that this is a very common idiom for bash scripts -- the following script will execute just fine. Both github and vim's colorization handle this perfectly; however, atom freaks out and thinks everything after the 2nd EOF should still be quoted -- basically, it acts as if the - isn't there at all.

#!/bin/bash
usage() {
	cat <<-EOF
		Usage: ${0##*/} FOO BAR
	EOF
}
[[ $# -eq 2 ]] && echo $2 $1 || usage