Moving this nonessential bug to the next milestone.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

This is happening a lot as of 8/30 - assigning to myself to take a look.

https://wmatrix.googleplex.com/failures/unfiltered?suites=wifi_matfunc&tests=network_WiFi_HiddenScan&releases=tot

RCA
pyshark capture is failing for these devices ..

asuka auron_paine auron_yuna banon candy caroline cyan kefka lars lulu rikku terra ultima winky

No capture frames are returned from the pyshark lib 
 capture = pyshark.FileCapture(
        input_file=pcap_path, display_filter=display_filter)
    capture.load_packets(timeout=PYSHARK_LOAD_TIMEOUT)

Affected test cases 
network_WiFi_VisibleScan
network_WiFi_HiddenScan

The packet capture requires a second router to be placed in the wificell and be pingable. 
If the DUT hostname is foo, 'foo-router' has to exist for all tests, and 'foo-pcap' has to be exist. Both foo-router and foo-pcap are jetstream family, typically whirlwind. 
Can you check if the pcap routers in these cases were present? If not, does the autotest mention a way of specifying a capability? 'wificell' is a capability, there should be one for the pcap too, and we should skip the test if its not present. 

The PCAP for one of the failing cells 

               host chromeos9-row4-rack5-host1-pcap {
                        hardware ethernet ;
                        fixed-address 100.115.96.68;
                        ddns-hostname "chromeos9-row4-rack5-host1-pcap";
                        option host-name "chromeos9-row4-rack5-host1-pcap";
               }
Probably  we need to map the MAC for the router in promiscuous for the capture.
I need to confirm the other failing devices also 

The autotest logs should tell you the pcap status (whether a pcap device was detected, whether the packet started and stopped correctly and where to look for the capture file). I typically scp over the capture file and look at it on wireshark on my desktop. That may be one way to bisect where the failure is. 

yes it does report Error 

02/24 19:35:15.641 DEBUG|        base_utils:0185| Running 'ping -c 3 -i 0.500000 chromeos9-row4-rack5-host1-pcap'
02/24 19:35:15.693 WARNI|       ping_runner:0338| Ping command returned no output; stderr was ping: unknown host chromeos9-row4-rack5-host1-pcap
.
02/24 19:35:15.694 DEBUG|        base_utils:0185| Running 'ping -c 3 -i 0.500000 chromeos9-row4-rack5-host1-attenuator'
02/24 19:35:15.860 WARNI|       ping_runner:0338| Ping command returned no output; stderr was ping: unknown host chromeos9-row4-rack5-host1-attenuator

I didnot find auto test to have "pcap" as DEPENDENCY/capability  in contro.py to add Can you point out any ? or we need to introduce an new ?

We may need to introduce it. I don't know the details in this particular case, but your way to get historical context is to search the crbug.com (OS>Systems>Network if you will) for 'packet capture' or alternatively, look at 'git blame' output in the autotest files that do a packet capture and see if there was any reference to adding this as a capability when it was first added. bmahadev@ added these I believe, so if you can ask her in-person quickly, that also works. 

Looks like we have a label called packet_capture but its only used for interop testing; i.e. devices in pool:chaos

debayan@ - you can look up all the devices in pool:wificell that have a valid mac for *-pcap in dhcpd and add packet_capture label to all those hosts.

https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/master/puppet/modules/lab/files/dhcp-server/dhcpd.conf

Once this is done, you can add packet_capture as a dependency for these 2 ("network_WiFi_HiddenScan" and "network_WiFi_VisibleScan") tests.

1) Added packet_capture attribute to host having pcap in cautotest .
2) Added packet_capture DEPENDENCY in control , waiting to commit

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/6ec92f385530cd71789c4ecba6773c65be00c19a

commit 6ec92f385530cd71789c4ecba6773c65be00c19a
Author: Debayan <debayanb@google.com>
Date: Thu Mar 02 21:20:47 2017

Adding DEPENDENCY packet_capture

Devices having packet_capture field in cautotest , are having pcap
devices or having capability to capture packet and thus those only should run these
test cases.

BUG=chromium:627552
TEST=None

Change-Id: I5e72fdf2d8dc0b03315e08adace956013f9e7ccd
Reviewed-on: https://chromium-review.googlesource.com/447820
Commit-Ready: Debayan Banerjee <debayanb@chromium.org>
Tested-by: Debayan Banerjee <debayanb@chromium.org>
Reviewed-by: Harpreet Grewal <harpreet@chromium.org>

[modify] https://crrev.com/6ec92f385530cd71789c4ecba6773c65be00c19a/server/site_tests/network_WiFi_VisibleScan/control
[modify] https://crrev.com/6ec92f385530cd71789c4ecba6773c65be00c19a/server/site_tests/network_WiFi_HiddenScan/control

AFAICT, this is not actually fixed. There are still several issues:

(1) you do *not* actually need a separate pcap device for this test, AFAICT
(2) this test is broken in other ways (reporting the same error), and is continuing to fail (see [1])

Regarding (1): running this in my dev chamber with jetstream + kevin, I see the test configures jetstream to use one of its radios in monitor mode. This is perfectly legal and functional, and I see there is a reasonable pcap file stashed in the debug/ logs after running this test. So we should audit the usage of DEPENDENCY here to make sure it's accurate and precise.

Regarding (2): I see a similar pattern on both my Kevin and on the a sample Wizpig failure in the lab; the test is looking for Probe Requests coming from a specific MAC (the MAC from 'iw dev' on the DUT), but the DUT is actually performing probe requests on a different one.

For example:
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/110765770-chromeos-test/chromeos9-row4-rack6-host2/network_WiFi_HiddenScan/debug/

'iw dev' reported the DUT's wlan0 with address '7c:b0:c2:XX:XX:74', in the captured router.0.pcap, I see the appropriate Probe Requests coming from '7c:b0:c2:YY:YY:bd'.

I think item (2) is a fundamental problem in this test, and it needs to be addressed (haha, pun!). Apparently the Wifi MAC is free to do some sort of address randomization for these scans?


[1] The following still shows several failures, including caroline, kevin, kefka, etc.:
https://wmatrix.googleplex.com/failures/unfiltered?suites=wifi_matfunc&tests=network_WiFi_HiddenScan&releases=tot

re: item #2, that might be a function of MAC address randomization

OK, so I've fixed the MAC randomization issues (CLs in flight). mwifiex had broken randomization handling, and we also need to disable randomization for this test.

That still leaves item (1) from above. I'm not sure I'm going to bother with that one (I'll assign back to dbayanb to figure that one out when I'm done).

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f5474e4183a7001bc8abe3e5d6532025cb40a03c

commit f5474e4183a7001bc8abe3e5d6532025cb40a03c
Author: Brian Norris <briannorris@chromium.org>
Date: Thu Apr 06 17:56:28 2017

wifi_client: add mac_address_randomization()

Will be used in a few places, but I need to configure it for
network_WiFi_HiddenScan and network_WiFi_VisibleScan, which will fail if
MAC address randomization is enabled.

BUG=chromium:627552
TEST=none

Change-Id: Idc78a6c0cfac5a4d48e0107344930d5dcae320fd
Signed-off-by: Brian Norris <briannorris@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/469248
Reviewed-by: Eric Caruso <ejcaruso@chromium.org>

[modify] https://crrev.com/f5474e4183a7001bc8abe3e5d6532025cb40a03c/server/cros/network/wifi_client.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f5474e4183a7001bc8abe3e5d6532025cb40a03c

commit f5474e4183a7001bc8abe3e5d6532025cb40a03c
Author: Brian Norris <briannorris@chromium.org>
Date: Thu Apr 06 17:56:28 2017

wifi_client: add mac_address_randomization()

Will be used in a few places, but I need to configure it for
network_WiFi_HiddenScan and network_WiFi_VisibleScan, which will fail if
MAC address randomization is enabled.

BUG=chromium:627552
TEST=none

Change-Id: Idc78a6c0cfac5a4d48e0107344930d5dcae320fd
Signed-off-by: Brian Norris <briannorris@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/469248
Reviewed-by: Eric Caruso <ejcaruso@chromium.org>

[modify] https://crrev.com/f5474e4183a7001bc8abe3e5d6532025cb40a03c/server/cros/network/wifi_client.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/1d24456396027887b9a954b25a55424e3fd471d1

commit 1d24456396027887b9a954b25a55424e3fd471d1
Author: Brian Norris <briannorris@chromium.org>
Date: Thu Apr 06 17:56:28 2017

network_WiFi_{Hidden,Visible}Scan: disable MAC randomization

These tests are looking for the MAC address of the DUT in the packet
capture. Don't let randomization screw this up.

BUG=chromium:627552
TEST=network_WiFi_HiddenScan and network_WiFi_VisibleScan on devices
     which support MAC randomization (e.g., mwifiex on 4.4)

Change-Id: I7e6cd6013249e527a5094c342211b55ab375ba55
Signed-off-by: Brian Norris <briannorris@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/469249
Reviewed-by: Eric Caruso <ejcaruso@chromium.org>

[modify] https://crrev.com/1d24456396027887b9a954b25a55424e3fd471d1/server/site_tests/network_WiFi_HiddenScan/network_WiFi_HiddenScan.py
[modify] https://crrev.com/1d24456396027887b9a954b25a55424e3fd471d1/server/site_tests/network_WiFi_VisibleScan/network_WiFi_VisibleScan.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/1d24456396027887b9a954b25a55424e3fd471d1

commit 1d24456396027887b9a954b25a55424e3fd471d1
Author: Brian Norris <briannorris@chromium.org>
Date: Thu Apr 06 17:56:28 2017

network_WiFi_{Hidden,Visible}Scan: disable MAC randomization

These tests are looking for the MAC address of the DUT in the packet
capture. Don't let randomization screw this up.

BUG=chromium:627552
TEST=network_WiFi_HiddenScan and network_WiFi_VisibleScan on devices
     which support MAC randomization (e.g., mwifiex on 4.4)

Change-Id: I7e6cd6013249e527a5094c342211b55ab375ba55
Signed-off-by: Brian Norris <briannorris@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/469249
Reviewed-by: Eric Caruso <ejcaruso@chromium.org>

[modify] https://crrev.com/1d24456396027887b9a954b25a55424e3fd471d1/server/site_tests/network_WiFi_HiddenScan/network_WiFi_HiddenScan.py
[modify] https://crrev.com/1d24456396027887b9a954b25a55424e3fd471d1/server/site_tests/network_WiFi_VisibleScan/network_WiFi_VisibleScan.py

OK, MAC randomization shouldn't be causing problems now [1]. Assigning back to Debayan.

Debayan, can you please review (1) from comment #19? I'm not 100% convinced your change was correct, since I can clearly run this test successfully without a separate packet capture device -- I only need the AP to support monitor mode:

https://chromium-review.googlesource.com/#/c/447820/

--

[1] Except for Kevin and family, where we're waiting for this to land:

https://chromium-review.googlesource.com/#/c/469072/

I also didn't review, e.g., Intel MAC randomization, but presumably it's not broken like mwifiex is.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/kernel/+/7ddcb73a572a185895cb6946ed76daf9aae23daf

commit 7ddcb73a572a185895cb6946ed76daf9aae23daf
Author: Brian Norris <briannorris@chromium.org>
Date: Fri Apr 07 07:17:46 2017

FROMLIST: mwifiex: MAC randomization should not be persistent

nl80211 provides the NL80211_SCAN_FLAG_RANDOM_ADDR for every scan
request that should be randomized; the absence of such a flag means we
should not randomize. However, mwifiex was stashing the latest
randomization request and *always* using it for future scans, even those
that didn't set the flag.

Let's zero out the randomization info whenever we get a scan request
without NL80211_SCAN_FLAG_RANDOM_ADDR. I'd prefer to remove
priv->random_mac entirely (and plumb the randomization MAC properly
through the call sequence), but the spaghetti is a little difficult to
unravel here for me.

Fixes: c2a8f0ff9c6c ("mwifiex: support random MAC address for scanning")
Signed-off-by: Brian Norris <briannorris@chromium.org>
(am from https://patchwork.kernel.org/patch/9665815/)

BUG=chromium:627552, b:35573298
TEST=manual; also, disable randomization and run
     `test_that ... network_WiFi_HiddenScan`, checking that we no longer
     scan w/ random MAC (and can therefore pass the test)

Change-Id: Ib01529e661536541babf5e2a699cb7f44f6163a8
Reviewed-on: https://chromium-review.googlesource.com/469072
Commit-Ready: Brian Norris <briannorris@chromium.org>
Tested-by: Brian Norris <briannorris@chromium.org>
Reviewed-by: Kirtika Ruchandani <kirtika@chromium.org>
Reviewed-by: Eric Caruso <ejcaruso@chromium.org>

[modify] https://crrev.com/7ddcb73a572a185895cb6946ed76daf9aae23daf/drivers/net/wireless/marvell/mwifiex/cfg80211.c

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/kernel/+/7ddcb73a572a185895cb6946ed76daf9aae23daf

commit 7ddcb73a572a185895cb6946ed76daf9aae23daf
Author: Brian Norris <briannorris@chromium.org>
Date: Fri Apr 07 07:17:46 2017

FROMLIST: mwifiex: MAC randomization should not be persistent

nl80211 provides the NL80211_SCAN_FLAG_RANDOM_ADDR for every scan
request that should be randomized; the absence of such a flag means we
should not randomize. However, mwifiex was stashing the latest
randomization request and *always* using it for future scans, even those
that didn't set the flag.

Let's zero out the randomization info whenever we get a scan request
without NL80211_SCAN_FLAG_RANDOM_ADDR. I'd prefer to remove
priv->random_mac entirely (and plumb the randomization MAC properly
through the call sequence), but the spaghetti is a little difficult to
unravel here for me.

Fixes: c2a8f0ff9c6c ("mwifiex: support random MAC address for scanning")
Signed-off-by: Brian Norris <briannorris@chromium.org>
(am from https://patchwork.kernel.org/patch/9665815/)

BUG=chromium:627552, b:35573298
TEST=manual; also, disable randomization and run
     `test_that ... network_WiFi_HiddenScan`, checking that we no longer
     scan w/ random MAC (and can therefore pass the test)

Change-Id: Ib01529e661536541babf5e2a699cb7f44f6163a8
Reviewed-on: https://chromium-review.googlesource.com/469072
Commit-Ready: Brian Norris <briannorris@chromium.org>
Tested-by: Brian Norris <briannorris@chromium.org>
Reviewed-by: Kirtika Ruchandani <kirtika@chromium.org>
Reviewed-by: Eric Caruso <ejcaruso@chromium.org>

[modify] https://crrev.com/7ddcb73a572a185895cb6946ed76daf9aae23daf/drivers/net/wireless/marvell/mwifiex/cfg80211.c

@Brian

after mac randomization some of the DUT is giving error "Could not set property
"

both the test cases have similar failure in a bunch of devices after 9439 .
https://wmatrix.googleplex.com/wifi_matfunc?hide_missing=True&tests=network_WiFi_HiddenScan&days_back=20

https://wmatrix.googleplex.com/wifi_matfunc?hide_missing=True&tests=network_WiFi_VisibleScan&days_back=20


a sneak peak to the auto test logs gives this
  File "/usr/local/autotest/server/site_tests/network_WiFi_VisibleScan/network_WiFi_VisibleScan.py", line 44, in run_once
    with self.context.client.mac_address_randomization(False):
  File "/usr/local/autotest/server/cros/network/wifi_client.py", line 1411, in __enter__
    raise error.TestFail('Could not set property')

Can you please have a look into it .

Eric, is that expected? I thought you said that setting to False would not be a problem on DUTs that don't support randomization.

This works well enough:

https://chromium-review.googlesource.com/c/476110/

Not sure if that qualifies as a hack; perhaps we'd rather export a proper method to check whether a property is supported?

On a related note: why do we have a 'GetProperties' dbus method for shill, but not a 'GetProperty'? If we had the latter, it might be easier to return some kind of formalized "not implemented" value. (I suppose we could do that with 'GetProperties' too...)

The following revision refers to this bug:
  https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/d24d4800e53ef6a404e6d28bb5e74006220f8c5a

commit d24d4800e53ef6a404e6d28bb5e74006220f8c5a
Author: Brian Norris <briannorris@chromium.org>
Date: Sat Apr 22 04:57:48 2017

shill: wifi: don't complain about "unsupported" when not changing

If we're keeping the randomization setting the same (i.e., setting it to
"false" when it's already "false"), don't complain about it. This is
a nice feature for some types of users, where we only care that MAC
randomization is *disabled*, not whether it's supported.

BUG=chromium:627552
TEST=`test_that DUT network_WiFi_HiddenScan network_WiFi_VisibleScan` on
     a device without MAC randomization support

Change-Id: I2e5db4e22a2572069043fbf90669cf425c43d05a
Signed-off-by: Brian Norris <briannorris@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/476110
Reviewed-by: Kirtika Ruchandani <kirtika@chromium.org>

[modify] https://crrev.com/d24d4800e53ef6a404e6d28bb5e74006220f8c5a/wifi/wifi.cc

OK, I think the problems in #29 should be fixed now.

Assigning to Harpreet to address the issue in #26. (Or close, if my concern is invalid.)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/2a182bd29097c0c333f96965e0a54925e5721a9f

commit 2a182bd29097c0c333f96965e0a54925e5721a9f
Author: harpreet <harpreet@google.com>
Date: Wed May 24 06:59:19 2017

Revert adding packet_capture dependency

No need for separate pcap since we can get the capture from AP.

BUG=chromium:627552
TEST=Tested against a device in the lab with pcap

Change-Id: I28dcaff6fd41257e9b803e92d01dec8563916dc3
Reviewed-on: https://chromium-review.googlesource.com/513124
Commit-Ready: Harpreet Grewal <harpreet@chromium.org>
Tested-by: Harpreet Grewal <harpreet@chromium.org>
Reviewed-by: Brian Norris <briannorris@chromium.org>

[modify] https://crrev.com/2a182bd29097c0c333f96965e0a54925e5721a9f/server/site_tests/network_WiFi_VisibleScan/control
[modify] https://crrev.com/2a182bd29097c0c333f96965e0a54925e5721a9f/server/site_tests/network_WiFi_HiddenScan/control

Hmm, I'd close this as "Fixed", but then I still see quite a few similar errors on the wmatrix dashboard recently. Perhaps another bug? Would need someone to investigate.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/b7326a0b384a9b6afccbb76b9e0afbacac73414d

commit b7326a0b384a9b6afccbb76b9e0afbacac73414d
Author: Brian Norris <briannorris@chromium.org>
Date: Tue Feb 06 12:56:02 2018

network_WiFi_{Hidden,Visible}Scan: more precise probe request capture

For a visible network, there are various reasons a station might not
need to send a probe request before authenticating with an AP (e.g.,
passive scanning). So network_WiFi_VisibleScan shouldn't be requiring a
non-zero number of probe requests in the packet capture.

Also, we should start our packet captures before we start configuring
the AP here, if we want to see all activity. For one, we could miss an
in-progress scan done by the station (which may find the AP) if we wait
too long to start the capture.

This should help decrease the number of (likely false) failures with
messages like:

  Expected exactly two SSIDs, but got frozenset([])

And we remove this failure case from network_WiFi_VisibleScan entirely:

  Expected SSIDs (['']), but got frozenset([])

BUG=chromium:627552
TEST=$subject tests, on lab devices with both whirlwind and stumpy APs

Change-Id: Ie4a1647d902402b1ff9fb686e705051f3947bc30
Signed-off-by: Brian Norris <briannorris@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/861200
Reviewed-by: Kirtika Ruchandani <kirtika@chromium.org>

[modify] https://crrev.com/b7326a0b384a9b6afccbb76b9e0afbacac73414d/server/site_tests/network_WiFi_HiddenScan/network_WiFi_HiddenScan.py
[modify] https://crrev.com/b7326a0b384a9b6afccbb76b9e0afbacac73414d/server/site_tests/network_WiFi_VisibleScan/network_WiFi_VisibleScan.py