New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 878386 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

my_activity.py gives inconsistent results

Project Member Reported by pkl@chromium.org, Aug 28

Issue description

I ran my_activity.py 5 times like this:

for i in 1 2 3 4 5 ; do
  ~/depot_tools/my_activity.py -b 3/1/18 -e 8/31/18 -u pkl@chromium.org > pkl-$i.txt
done

This gives 5 output files:
-rw-r--r--  1 pkl  eng      0 Aug 28 07:18 pkl-1.txt
-rw-r--r--  1 pkl  eng  28011 Aug 28 07:19 pkl-2.txt
-rw-r--r--  1 pkl  eng  24017 Aug 28 07:19 pkl-3.txt
-rw-r--r--  1 pkl  eng  21366 Aug 28 07:19 pkl-4.txt
-rw-r--r--  1 pkl  eng  21366 Aug 28 07:19 pkl-5.txt

I do not expect 5 consecutive runs would give 5 different output.

The first file is 0-length because it had an exception like this:
WARNING:root:Consider installing python-keyring
Traceback (most recent call last):
  File "/Users/pkl/depot_tools/my_activity.py", line 1096, in <module>
    sys.exit(main())
  File "/Users/pkl/depot_tools/my_activity.py", line 1051, in main
    my_activity.get_issues()
  File "/Users/pkl/depot_tools/my_activity.py", line 711, in get_issues
    self.monorail_issue_search, monorail_projects.keys())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 567, in get
    raise self._value
ssl.SSLError: ('The read operation timed out',)

The other 4 runs have warnings like this (which I think are expected):
WARNING:root:Consider installing python-keyring
ERROR:root:Access Errors:
ERROR:root:Looking up 'review.coreboot.org': Bad Request: User pkl not found
ERROR:root:Looking up 'pdfium-review.googlesource.com': Bad Request: User pkl not found


 
Cc: serg...@chromium.org tandrii@chromium.org
Looking at the git history, http://crrev/c/966563 which added ThreadPool may be related.
Labels: -Pri-2 Pri-1
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1194306 adds a --single-thread command line option to use only a single thread. This makes the script runs slowly, but correctly. I would consider making this the default until the ThreadPool code is fixed.

1. So, what's the diff in outputs?
2. With your change, have you re-run the experiment?
When I ran the above test, I was at home on my MacBook laptop.
I re-ran my experiment (also when I was on the MBP) and all 5 runs gave the same result.

Now I'm in the office on my MacPro desktop. I cannot reproduce this on my MacPro desktop.

Any suggestions as to what the difference may be? Can it be python version difference? More cores on my MacPro than on the MBP?
no idea. my_activity isn't a sturdy script I'd rely on -- iirc there were places where it silently ignores bad data.
If you are in the mood to debug this more, you can add a logging for each item being fetched from Gerrit and enable verbosity.
Just tried again with my MacBook Pro over my home connection. Still getting inconsistent results. The differences are in the crbug/ section. Some later runs are just getting fewer lines in the crbug/ section.

I'm going to take my laptop with me to the office and run it over Google-A to see if network makes a difference. I would be real surprise if it is.
Status: Available (was: Untriaged)
Cc: -serg...@chromium.org

Sign in to add a comment