New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 682860 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Oct 23
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Feature

Blocked on:
issue 844166



Sign in to add a comment

cros lint: add a linter for shell scripts

Project Member Reported by vapier@chromium.org, Jan 19 2017

Issue description

in the ideal world, we'd run things against `shellcheck`.  but due to that being written in haskell, we'll have to settle for `bash -n` for now.

there's a slight issue in that we might run into shell fragments (i.e. something like foo.in which needs to be processed into foo).  for those cases, maybe we just ignore files that end in ".in" based on the assumption that they are fragments.

we could try running #!/bin/sh scripts through `checkbashisms`.  but we'll need to use the version in the chroot as we've fixed a number of issues in Gentoo that aren't in upstream.

we probably also want to update chromite's ImageTest phase to run `bash -n` on all installed scripts in $PATH (and maybe a few more known locations) in the rootfs.  these shouldn't be fragments, so we can be more strict about what's in there.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Jan 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/f011af9aaea4003bccccf493ed37ec3a04e04485

commit f011af9aaea4003bccccf493ed37ec3a04e04485
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Jan 20 00:03:05 2017

cros: lint: support linting of files based on content

Not all files use extensions (e.g. foo.py or blah.sh).  In fact, our style
guides encourage omitting them when they get installed as executables so
that people don't have to worry about the underlying language.

Update the linter to deal with this scenario.  We read the first 128 bytes
out in order to process the shebang.  Then we use that to determine if it
is a python or shell script and add them to the right linting paths.

For shell scripts, we initially just make sure it parses at a basic level.
There is opportunity for future work here, but will require more effort to
pull off.

BUG= chromium:682860 
TEST=`cros lint` on a broken shell script fails, but works on a good one

Change-Id: I5d46d66f7f56cbb74c2fa21880eff61d0e4728f3
Reviewed-on: https://chromium-review.googlesource.com/430930
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/f011af9aaea4003bccccf493ed37ec3a04e04485/cli/cros/cros_lint.py

Comment 2 by sjg@chromium.org, May 24 2017

Cc: sjg@chromium.org
What should we do here?
- Write a shellcheck in Python?
- Find some other existing tool and build on it?

Also I'd really like to have a linter for ebuilds. Is that contemplated here?

Comment 3 by vapier@chromium.org, May 24 2017

linting for ebuilds would be better in a sep bug

at this point, we should implement the ImageTest check, and `checkbashisms` after that.

Comment 4 Deleted

Comment 5 by vapier@chromium.org, May 24 2018

Cc: bmgordon@chromium.org
Components: Infra>Client>ChromeOS>Build
Labels: -Hotlist-Recharge-Cold
Owner: bmgordon@chromium.org
Status: Started (was: Untriaged)
Project Member

Comment 6 by bugdroid1@chromium.org, May 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/a29e19d9f75dbfcf53d2bc3d2d9989dc503257b9

commit a29e19d9f75dbfcf53d2bc3d2d9989dc503257b9
Author: Benjamin Gordon <bmgordon@chromium.org>
Date: Sat May 26 07:21:43 2018

cros lint: Run shellcheck if it is available

shellcheck is available outside the chroot through apt-get or dnf.  As
of crrev.com/c/1065029, it is also available in the chroot if the user
has emerged it.  Make cros lint try to run it for shell files so we can
start collecting data about what it finds.  For now, both the presence
of shellcheck and any errors it produces are considered non-blocking.

BUG= chromium:844166 ,  chromium:682860 
TEST=cros lint .../dev-util/shellcheck/shellcheck-0.4.7.ebuild

Change-Id: I332ff3fec34f6dd9bcf35125d84aaf4f7fa9bf44
Reviewed-on: https://chromium-review.googlesource.com/1070410
Commit-Ready: Benjamin Gordon <bmgordon@chromium.org>
Tested-by: Benjamin Gordon <bmgordon@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/a29e19d9f75dbfcf53d2bc3d2d9989dc503257b9/cli/cros/cros_lint.py

Project Member

Comment 7 by bugdroid1@chromium.org, May 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4b00245a305aa1273434d2a67b24b5498fafea72

commit 4b00245a305aa1273434d2a67b24b5498fafea72
Author: chromite-chromium-autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Sat May 26 10:11:54 2018

Roll src/third_party/chromite/ 046952c65..a29e19d9f (2 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/046952c65549..a29e19d9f75d

$ git log 046952c65..a29e19d9f --date=short --no-merges --format='%ad %ae %s'
2018-05-23 bmgordon cros lint: Run shellcheck if it is available
2018-05-24 bmgordon cros lint: Call shell linter based on extensions

Created with:
  roll-dep src/third_party/chromite
BUG= chromium:844166 , chromium:682860 , chromium:844166 


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: Ibdb4b4bce29bd0440f4d111914856e148ab8b31f
Reviewed-on: https://chromium-review.googlesource.com/1074492
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#562122}
[modify] https://crrev.com/4b00245a305aa1273434d2a67b24b5498fafea72/DEPS

Blockedon: 844166
Unless there are objections, I'll close this bug.  We can file more specific bugs for any enhancements we need (like crbug.com/891826).
Missing context from the previous comment: shellcheck went in earlier today and should be showing up in chroots whenever people run update_chroot.
Status: Fixed (was: Started)

Sign in to add a comment