my_activity.py gives inconsistent results |
||||
Issue descriptionI ran my_activity.py 5 times like this: for i in 1 2 3 4 5 ; do ~/depot_tools/my_activity.py -b 3/1/18 -e 8/31/18 -u pkl@chromium.org > pkl-$i.txt done This gives 5 output files: -rw-r--r-- 1 pkl eng 0 Aug 28 07:18 pkl-1.txt -rw-r--r-- 1 pkl eng 28011 Aug 28 07:19 pkl-2.txt -rw-r--r-- 1 pkl eng 24017 Aug 28 07:19 pkl-3.txt -rw-r--r-- 1 pkl eng 21366 Aug 28 07:19 pkl-4.txt -rw-r--r-- 1 pkl eng 21366 Aug 28 07:19 pkl-5.txt I do not expect 5 consecutive runs would give 5 different output. The first file is 0-length because it had an exception like this: WARNING:root:Consider installing python-keyring Traceback (most recent call last): File "/Users/pkl/depot_tools/my_activity.py", line 1096, in <module> sys.exit(main()) File "/Users/pkl/depot_tools/my_activity.py", line 1051, in main my_activity.get_issues() File "/Users/pkl/depot_tools/my_activity.py", line 711, in get_issues self.monorail_issue_search, monorail_projects.keys()) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 251, in map return self.map_async(func, iterable, chunksize).get() File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value ssl.SSLError: ('The read operation timed out',) The other 4 runs have warnings like this (which I think are expected): WARNING:root:Consider installing python-keyring ERROR:root:Access Errors: ERROR:root:Looking up 'review.coreboot.org': Bad Request: User pkl not found ERROR:root:Looking up 'pdfium-review.googlesource.com': Bad Request: User pkl not found
,
Aug 28
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1194306 adds a --single-thread command line option to use only a single thread. This makes the script runs slowly, but correctly. I would consider making this the default until the ThreadPool code is fixed.
,
Aug 28
1. So, what's the diff in outputs? 2. With your change, have you re-run the experiment?
,
Aug 28
When I ran the above test, I was at home on my MacBook laptop. I re-ran my experiment (also when I was on the MBP) and all 5 runs gave the same result. Now I'm in the office on my MacPro desktop. I cannot reproduce this on my MacPro desktop. Any suggestions as to what the difference may be? Can it be python version difference? More cores on my MacPro than on the MBP?
,
Aug 28
no idea. my_activity isn't a sturdy script I'd rely on -- iirc there were places where it silently ignores bad data. If you are in the mood to debug this more, you can add a logging for each item being fetched from Gerrit and enable verbosity.
,
Aug 29
Just tried again with my MacBook Pro over my home connection. Still getting inconsistent results. The differences are in the crbug/ section. Some later runs are just getting fewer lines in the crbug/ section. I'm going to take my laptop with me to the office and run it over Google-A to see if network makes a difference. I would be real surprise if it is.
,
Nov 30
,
Dec 3
|
||||
►
Sign in to add a comment |
||||
Comment 1 by pkl@chromium.org
, Aug 28