Modify Findit's built-in http client to examine response codes and exceptions and update a tsmon metric. The metric could have two fields: -HTTP status code(int, e.g. 200, 404, 500) -Exception type(string, e.g. "None", "ValueError", etc.)
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/27107f1edc7d1b4c17000383a5fbef805eb59ef9 commit 27107f1edc7d1b4c17000383a5fbef805eb59ef9 Author: Roberto Carrillo <robertocn@google.com> Date: Tue Oct 03 23:41:49 2017 [Findit] HttpInterceptor This change introduces the concept of an optional interceptor to be bound to an http client at init time. If set, the interceptor's OnRequest, OnResponse and OnException methods will be called and the return value of those methods will be used by the client as the request, response and exception respectively. The authenticating mechanism has been refactored to be provided by the interceptor, via header injection. The client will call the interceptor's GetAuthenticationHeaders method, and apply them accordingly to the request. Note that this is required for an upcoming change where we capture metrics about http requests made to external services via an interceptor. R=stgao,chanli,lijeffrey,wylieb,katesonia BUG= 768912 Change-Id: I4ad690148fab3b13e326d925ddc31c3192819f1c Reviewed-on: https://chromium-review.googlesource.com/692841 Commit-Queue: Roberto Carrillo <robertocn@chromium.org> Reviewed-by: Shuotao Gao <stgao@chromium.org> [modify] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/libs/http/test/retry_http_client_test.py [add] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/libs/http/test/interceptor_test.py [modify] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/gae_libs/http/test/auth_util_test.py [modify] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/gae_libs/http/test/http_client_appengine_test.py [modify] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/gae_libs/http/http_client_appengine.py [modify] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/libs/http/retry_http_client.py [modify] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/waterfall/test/swarming_util_test.py [add] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/libs/http/interceptor.py [modify] https://crrev.com/27107f1edc7d1b4c17000383a5fbef805eb59ef9/appengine/findit/gae_libs/http/auth_util.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/f8108ea323f7ca06bf065715c1e3dd69f95571ed commit f8108ea323f7ca06bf065715c1e3dd69f95571ed Author: Roberto Carrillo <robertocn@google.com> Date: Tue Oct 10 18:14:27 2017 [Findit] Only use logging.exception for non-retriable exceptions. The interceptor currently uses logging.exception() even for exceptions that will be retried (Such as ConnectionClosedException) and this generates noise in the GCP Errors page. Instead this change makes it so that we log a warning when we know we will retry the call. R=stgao,chanli BUG= 768912 Change-Id: I5f1809ad898fa15c00d1837fb983f6a9e5e9e3f3 Reviewed-on: https://chromium-review.googlesource.com/707800 Reviewed-by: Shuotao Gao <stgao@chromium.org> Commit-Queue: Roberto Carrillo <robertocn@chromium.org> [modify] https://crrev.com/f8108ea323f7ca06bf065715c1e3dd69f95571ed/appengine/findit/libs/http/interceptor.py [modify] https://crrev.com/f8108ea323f7ca06bf065715c1e3dd69f95571ed/appengine/findit/libs/http/test/interceptor_test.py [modify] https://crrev.com/f8108ea323f7ca06bf065715c1e3dd69f95571ed/appengine/findit/libs/http/test/retry_http_client_test.py [modify] https://crrev.com/f8108ea323f7ca06bf065715c1e3dd69f95571ed/appengine/findit/libs/http/retry_http_client.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/0767c1bc24ac4574d0410d7f8ca6ac40b2750917 commit 0767c1bc24ac4574d0410d7f8ca6ac40b2750917 Author: Roberto Carrillo <robertocn@google.com> Date: Wed Oct 11 20:10:13 2017 [Findit] Intercept http requests to external services and metric-ify them. This change wraps HttpClientAppengine in FinditHttpClient, which by default records exceptions and status codes in ts_mon metrics using an interceptor; and replaces its invocations throughout the project. This change also moves the metrics to common/ to avoid having a dependency of the form common/ --(depends on)--> waterfall/. R=chanli,lijeffrey,wylieb,stgao BUG= 768912 Change-Id: Ibad9a9ba48e7aac521ad75dcc5c8b82672c77265 Reviewed-on: https://chromium-review.googlesource.com/710802 Reviewed-by: Chan Li <chanli@chromium.org> Reviewed-by: Shuotao Gao <stgao@chromium.org> Commit-Queue: Roberto Carrillo <robertocn@chromium.org> [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/pipelines/compile_failure/test/extract_signal_for_compile_pipeline_test.py [rename] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/common/monitoring.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/libs/http/test/interceptor_test.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/trigger_swarming_task_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/revert_and_notify_culprit_pipeline.py [add] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/common/findit_http_client.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/process_flake_try_job_result_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/update_flake_analysis_data_points_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/infra_api_clients/codereview/gerrit.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/services/test/try_job_test.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/update_flake_bug_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/initialize_flake_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/initialize_flake_try_job_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/common/rotations.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/step_mapper.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/services/build_failure_analysis.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/services/try_job.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/identify_try_job_culprit_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/trigger_flake_swarming_task_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/test/swarming_util_test.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/services/revert.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/infra_api_clients/codereview/rietveld.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/recursive_flake_try_job_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/flake_analysis_service.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/pipelines/test_failure/extract_signal_for_test_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/build_util.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/swarming_util.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/finish_build_analysis_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/services/test_failure/ci_test_failure.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/determine_true_pass_rate_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/trigger_flake_swarming_task_service_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/regression_range_analysis_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/handlers/help_triage.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/common/waterfall/buildbucket_client.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/trigger_base_swarming_task_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/process_base_swarming_task_result_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/handlers/check_trybot_mapping.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/schedule_try_job_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/get_test_location_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/libs/http/interceptor.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/services/test/deps_test.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/services/test_failure/test/ci_test_failure_test.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/pipelines/test_failure/test/extract_signal_for_test_pipeline_test.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/util_scripts/remote_queries/bottleneck_query.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/test/step_mapper_test.py [add] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/common/test/findit_http_client_test.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/services/ci_failure.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/handlers/collect_tree_closures.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/monitor_try_job_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/handlers/test/collect_tree_closures_test.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/suspected_cl_util.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/extract_signal_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/infra_api_clients/codereview/codereview_util.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/pipelines/compile_failure/extract_signal_for_compile_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/waterfall/flake/identify_suspected_revisions_pipeline.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/handlers/periodic_bot_update.py [modify] https://crrev.com/0767c1bc24ac4574d0410d7f8ca6ac40b2750917/appengine/findit/handlers/step_by_step_comparison.py
Comment 1 by robert...@chromium.org
, Sep 29 2017