build a gerrit_info -> commit metadata database for analysis |
||||
Issue descriptionPer "Bayesian CL Rejection" ddoc https://docs.google.com/document/d/1wmaWX-I59fEtEROn8MKj2wgHJ_HfzeuiMwnGvzUQ4PY/edit There are a few ideas for how to proceed here, but the first limitation for any of them is that: - Our bad CL information, and CL information in general in cidb, is indexed by (gerrit_host, change_number) - But that doesn't contain any other CL information (such as repo path, author, commit size, etc). In principle, all of that 2nd bullet information is available in Gerrit. But if we are going to be doing some experimental number crunching, it's going to be very inefficient to make a whole bunch of Gerrit rpcs to retrieve it all the time. And we have all the information on disk in our repo checkout. Therefore, build a small tool to churn through the on disk checkouts and construction a (gerrit_host -> change_num) database.
,
Jan 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/infra_virtualenv/+/6a1c8c67d8712ed4ac9c03f2eef6c7f2fe5d0d22 commit 6a1c8c67d8712ed4ac9c03f2eef6c7f2fe5d0d22 Author: Aviv Keshet <akeshet@chromium.org> Date: Mon Dec 19 23:00:33 2016 pip_packages: add GitPython and its dependencies These are used by a new git history examining script in chromite. BUG= chromium:675793 TEST=None Change-Id: I9a35d3c696786e864a59380ad3d4401ac91054ed Reviewed-on: https://chromium-review.googlesource.com/422448 Commit-Ready: Aviv Keshet <akeshet@chromium.org> Tested-by: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> Reviewed-by: Aviv Keshet <akeshet@chromium.org> [add] https://crrev.com/6a1c8c67d8712ed4ac9c03f2eef6c7f2fe5d0d22/pip_packages/smmap2-2.0.1-py2.py3-none-any.whl [add] https://crrev.com/6a1c8c67d8712ed4ac9c03f2eef6c7f2fe5d0d22/pip_packages/gitdb2-2.0.0-py2.py3-none-any.whl [add] https://crrev.com/6a1c8c67d8712ed4ac9c03f2eef6c7f2fe5d0d22/pip_packages/GitPython-2.1.1-py2.py3-none-any.whl
,
Jan 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/f151d84ebb811d5ef5fa763782f12bb11f0bb5d3 commit f151d84ebb811d5ef5fa763782f12bb11f0bb5d3 Author: Aviv Keshet <akeshet@chromium.org> Date: Mon Dec 19 23:01:24 2016 extract_gerrit_history: a script to build a gerrit # to commit database This is the scaffolding for the script. Real functionality (iterating over multiple repositories, cross-referencing with good/bad CL information from cidb, consolidating to single database) coming later. CQ-DEPEND=CL:422448 BUG= chromium:675793 TEST=New unit test Change-Id: I72934ab59d62bbdda785b29b2cdbf82276216607 Reviewed-on: https://chromium-review.googlesource.com/422449 Commit-Ready: Aviv Keshet <akeshet@chromium.org> Tested-by: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> [modify] https://crrev.com/f151d84ebb811d5ef5fa763782f12bb11f0bb5d3/lib/clactions_unittest.py [modify] https://crrev.com/f151d84ebb811d5ef5fa763782f12bb11f0bb5d3/lib/clactions.py [add] https://crrev.com/f151d84ebb811d5ef5fa763782f12bb11f0bb5d3/scripts/extract_gerrit_history_unittest [add] https://crrev.com/f151d84ebb811d5ef5fa763782f12bb11f0bb5d3/bin/extract_gerrit_history [add] https://crrev.com/f151d84ebb811d5ef5fa763782f12bb11f0bb5d3/scripts/extract_gerrit_history.py [add] https://crrev.com/f151d84ebb811d5ef5fa763782f12bb11f0bb5d3/scripts/extract_gerrit_history_unittest.py [modify] https://crrev.com/f151d84ebb811d5ef5fa763782f12bb11f0bb5d3/venv/requirements.txt
,
Jan 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/16e270120904e921f70f62d486069e0b81c0ad55 commit 16e270120904e921f70f62d486069e0b81c0ad55 Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Dec 20 23:43:16 2016 extract_gerrit_history: crawl entire checkout to build database At the moment, this writes the database to a pickle file. In the long term, I'm not a fan of pickles, but this is a fast and easy solution for interactive / exploratory use for now. BUG= chromium:675793 TEST=manual local test Change-Id: Ic3d79da1108b668f9e1ae4871c5f38b3bf0ac752 Reviewed-on: https://chromium-review.googlesource.com/422578 Commit-Ready: Aviv Keshet <akeshet@chromium.org> Tested-by: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> [modify] https://crrev.com/16e270120904e921f70f62d486069e0b81c0ad55/scripts/extract_gerrit_history.py [modify] https://crrev.com/16e270120904e921f70f62d486069e0b81c0ad55/lib/clactions.py
,
Jan 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/559b72aea4ffbd1069a52442e96dfb8547edef7b commit 559b72aea4ffbd1069a52442e96dfb8547edef7b Author: Aviv Keshet <akeshet@chromium.org> Date: Wed Dec 21 22:17:05 2016 cidb: add method for fetching actions since a given time BUG= chromium:675793 TEST=manual local test Change-Id: I673f5b7d6928e5b38ee5f94098c1d06b4a6c0ee2 Reviewed-on: https://chromium-review.googlesource.com/422961 Commit-Ready: Aviv Keshet <akeshet@chromium.org> Tested-by: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> [modify] https://crrev.com/559b72aea4ffbd1069a52442e96dfb8547edef7b/lib/cidb.py
,
Apr 18 2017
Not actively working on this now. Might be part of xixuan's intern's project. Unowning and reducing P.
,
Mar 19 2018
,
Aug 13
,
Aug 21
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/593c9ba0618731607a819896d3cec1e4bb8b4e53 commit 593c9ba0618731607a819896d3cec1e4bb8b4e53 Author: Lann Martin <lannm@chromium.org> Date: Tue Aug 21 22:03:00 2018 Remove scripts/extract_gerrit_history This was written for a project that has since been abandoned. BUG= chromium:675793 TEST=none Change-Id: Ia020011295bfe14b163a12f862160c9dcea057ea Reviewed-on: https://chromium-review.googlesource.com/1173316 Commit-Ready: Lann Martin <lannm@chromium.org> Tested-by: Lann Martin <lannm@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [delete] https://crrev.com/e71a8489a09b57e76527e3da3209cf23d07c0d36/scripts/extract_gerrit_history_unittest.py [delete] https://crrev.com/e71a8489a09b57e76527e3da3209cf23d07c0d36/scripts/extract_gerrit_history_unittest [delete] https://crrev.com/e71a8489a09b57e76527e3da3209cf23d07c0d36/scripts/extract_gerrit_history.py |
||||
►
Sign in to add a comment |
||||
Comment 1 by autumn@chromium.org
, Jan 3 2017