New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 820743 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 836548
Owner: ----
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Spacey fails when processing binary files (bufio.Scanner: token too long)

Project Member Reported by qyears...@chromium.org, Mar 10 2018

Issue description

I (unintentionally) uploaded a CL with a binary file, and Spacey tried to analyze it, resulting in a "token too long" error.

There are a couple issues with this:
 * Spacey shouldn't try to analyze binary files.
 * To process input with lines that are potentially long, bufio.Reader.ReadLine is preferred over bufio.Scanner.

Example task:
https://chromium-swarm.appspot.com/task?id=3c2628ef0575ef10&refresh=10&show_raw=1

Related discussion about bufio.Scanner:
https://stackoverflow.com/questions/21124327/how-to-read-a-text-file-line-by-line-in-go-when-some-lines-are-long-enough-to-ca
 
Note: According to https://twitter.com/davecheney/status/604837853344989184?lang=en, it seems like Scanner is preferred for normal text file cases, and it's simpler. What we want is just for Spacey to ignore binary files.

Most (but maybe not all) analyzers will probably want to ignore binary files.
Mergedinto: 836548
Status: Duplicate (was: Available)

Sign in to add a comment