machenbach: have you experienced anything like this?

There was an endpoint somewhere where you could see which cq.cfg the prod CQ is actually using. But I don't find anymore how it's called right now...

https://apis-explorer.appspot.com/apis-explorer/?base=https://luci-config.appspot.com/_ah/api#p/config/v1/config.get_config_sets?config_set=projects%252Fwebrtc%252Frefs%252Fheads%252Fmaster&include_last_import_attempt=true&_h=13&

the above shows what luci-config would offer to CQ during next CQ restart, which is every 30 mins.

Well, that looks correct. Could you try to upload a fresh CL (e.g. whitespace) and see if the problem also reproduces there?

Hmm, so it seems this only hits our autoroller bot. A recent dev-created CL (https://codereview.webrtc.org/1955413003/) shows the right config is being used, but a recent autoroll-CL (https://codereview.webrtc.org/1964273003/) still uses the bad config.

Those CLs are uploaded by a bot like this: It goes like this:
1. bot_update runs
2. git checkout master
3. git clean -ffd
4. https://chromium.googlesource.com/external/webrtc/+/master/tools/autoroller/roll_chromium_revision.py runs and:
4.1. Deletes previous local branch (named 'special_roll_branch') if existing
4.2. Creates new local branch 'special_roll_branch'
4.3. Does local modifications for the roll + a local commit.
4.4. Uploads the CL: "git cl upload"
4.5. Sends it to CQ: "git cl set_commit"

Since the checkout is always synced before each new roll attempt with bot_update, it should always have the latest cq.config, no?
Could it be messed up by the fact that it always is on a local branch when bot_update runs?

See https://build.chromium.org/p/client.webrtc.fyi/builders/Auto-roll%20-%20WebRTC%20DEPS/builds/678 for logs.

Local stuff shouldn't matter AFAIK.

I'm getting multiple reports from team members that the trybot is still triggered for them as well. Example: https://codereview.webrtc.org/1968853002/ which was created from an up-to-date master.

What's going on here? Does the CQ take into account what trybots ran previously, so if android_compile_x86_dbg ran once for a CL, it's always re-run later?

No need to add me manually. Infra>CQ component is enough for us to triage it.

CL to log current CQ config: https://chromereviews.googleplex.com/427847013/.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/infra_internal.git/+/7fe7358b6d5de10cf322da11d92357ed9e7e5c8e

commit 7fe7358b6d5de10cf322da11d92357ed9e7e5c8e
Author: sergiyb <sergiyb@google.com>
Date: Thu May 12 13:32:36 2016

When I was manually restarting CQ to take my changes into account, I've realized auto_deploy.py was broken (due to a merge conflict) and didn't restart any CQs for a while. I've resolved the conflict and restarted all CQs, so new config should be used now. Please let me know if this is still an issue.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/infra_internal.git/+/b892e38ebc775ca2825f2434111c11a03e20ab5b

commit b892e38ebc775ca2825f2434111c11a03e20ab5b
Author: sergiyb <sergiyb@google.com>
Date: Sun May 15 22:27:17 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/infra_internal.git/+/9e99a2a6c83669aa9b0092bb8729291aae17267a

commit 9e99a2a6c83669aa9b0092bb8729291aae17267a
Author: sergiyb <sergiyb@google.com>
Date: Mon May 16 12:25:43 2016