This is meant to track errors in the datastore due to data exceeding the limits of the datastore
https://pantheon.corp.google.com/errors/CKTa_4OhyMSPswE?service&version&time=P7D&filter&refresh=auto&project=findit-for-me https://pantheon.corp.google.com/errors/CM6sovP8vdLglgE?service&version&time=P7D&filter&refresh=auto&project=findit-for-me
https://pantheon.corp.google.com/errors/CI_e0MLg2NyMLQ?service&version&time=P7D&filter&refresh=auto&project=findit-for-me
We got a lot of data writes that exceed the 1MB cap of an entity. https://pantheon.corp.google.com/errors/CKTa_4OhyMSPswE?time=P1D&project=findit-for-me Chan, any ETA to fix this? It has been around for a long while.
"BadRequestError: entity is too big" should be fixed now https://pantheon.corp.google.com/errors/CI_e0MLg2NyMLQ?service&version&time=P30D&project=findit-for-me The root cause is a big legacy property MasterFlakeAnalysis.swarming_rerun_results And I've used remote api to remove this legacy property.
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/a7cf11cfb901eb465596d17cf740512827b4c5ca commit a7cf11cfb901eb465596d17cf740512827b4c5ca Author: Chan <chanli@chromium.org> Date: Mon Jul 16 18:03:32 2018 [Findit] Don't store log_data in step if it's too large. This change will properly handles the case that log_data of a step is too large and cannot be stored in data store. Although after this change, we may need to request isolate for the failure_logs again when to use the log. I also refactored functions in ci_test_failure a bit to make it easier to understand and use. Also removed some data file for unittests. Bug: 806406 Change-Id: I570cd3dfa9a30809764e80e22e60504ff2a2a482 Reviewed-on: https://chromium-review.googlesource.com/1100545 Commit-Queue: Chan Li <chanli@chromium.org> Reviewed-by: Shuotao Gao <stgao@chromium.org> [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/services/extract_signal.py [delete] https://crrev.com/400e4df27e87a1df6cabb63c667bfb4b14fc6b4c/appengine/findit/services/test/data/m_b_223_abc_test_flaky.json [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/services/test_failure/ci_test_failure.py [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/libs/test_results/gtest_test_results.py [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/services/test_failure/extract_test_signal.py [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/libs/test_results/test/gtest_test_results_test.py [delete] https://crrev.com/400e4df27e87a1df6cabb63c667bfb4b14fc6b4c/appengine/findit/services/test/data/m_b_222_abc_test.json [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/services/constants.py [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/services/test/extract_signal_test.py [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/services/test_failure/test/ci_test_failure_test.py [modify] https://crrev.com/a7cf11cfb901eb465596d17cf740512827b4c5ca/appengine/findit/model/wf_step.py
close this bug for now. Feel free to reopen it if run into large data again.
Comment 1 by robert...@chromium.org
, Jan 26 2018