New issue
Advanced search Search tips

Issue 910746 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Dec 4
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Eslint analyzer fails when the number of warnings is too large

Project Member Reported by qyears...@chromium.org, Nov 30

Issue description

Example:
https://tricium-dev.appspot.com/run/6289191864369152
https://chromium-swarm.appspot.com/task?id=417f71b96f686910

Log contains:

   Command ...
   Failed to read command output: bufio.Scanner: token too long

This is an error that happened in scanner.Scan here:
https://cs.chromium.org/chromium/infra/go/src/infra/tricium/functions/eslint/eslint_parser.go?l=131

Looks like Scanner is not mean to handle long lines. In this case what we actually want is the whole output, which may be large, and which incidentally is all on one line. Perhaps either Reader.Read or Reader.ReadLine could be used (https://golang.org/pkg/bufio/#Reader.Read).
 
Note, I think this may also be an opportunity to refactor the eslint_parser a little bit given that it doesn't parse output line by line, it only looks at one line (the whole JSON output of eslint).
Labels: -Pri-3 Pri-2
Owner: qyears...@chromium.org
Status: Started (was: Available)
Uploaded https://chromium-review.googlesource.com/c/infra/infra/+/1356853.
Project Member

Comment 3 by bugdroid1@chromium.org, Dec 3

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/2aed9f213a20963e0bb5609d9a83922f35eaaa5e

commit 2aed9f213a20963e0bb5609d9a83922f35eaaa5e
Author: Quinten Yearsley <qyearsley@chromium.org>
Date: Mon Dec 03 21:36:14 2018

[tricium eslint] Support reading larger eslint output

Previously we used bufio.Scanner to read the command output,
since this is how the pylint analyzer was written, because
in that case we read one line at a time, and lines are not
very large.

However in this case we want to read the whole command output
at once, not line by line, since eslint has JSON output, and
the JSON output is one big JSON blob.

Bug:  910746 
Change-Id: Ibd86923b107c74c647f88c7e3ae44c60efa78921
Reviewed-on: https://chromium-review.googlesource.com/c/1356853
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Cr-Commit-Position: refs/heads/master@{#19298}
[modify] https://crrev.com/2aed9f213a20963e0bb5609d9a83922f35eaaa5e/go/src/infra/tricium/functions/eslint/eslint_parser_test.go
[modify] https://crrev.com/2aed9f213a20963e0bb5609d9a83922f35eaaa5e/go/src/infra/tricium/functions/eslint/eslint_parser.go
[modify] https://crrev.com/2aed9f213a20963e0bb5609d9a83922f35eaaa5e/go/src/infra/tricium/functions/eslint/cipd.yaml

Status: Fixed (was: Started)

Sign in to add a comment