Various "important" Tast tests fail on guado_moblab-release: http://stainless/search?view=matrix&row=test&col=build&first_date=2018-11-25&last_date=2018-12-01&builder_name=%5Eguado_moblab-release%24&branch=%5Emaster%24&test=%5Etast%5C..*%5C.&exclude_cts=false&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false

security.Firewall fails consistently due to presumably-intentional iptables differences:

Missing iptables rule "-P INPUT DROP"
Missing iptables rule "-P OUTPUT DROP"
Missing iptables rule "-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT"
Missing iptables rule "-A INPUT -i lo -j ACCEPT"
Missing iptables rule "-A INPUT -p icmp -j ACCEPT"
Missing iptables rule "-A INPUT -d 224.0.0.251/32 -p udp -m udp --dport 5353 -j ACCEPT"
Missing iptables rule "-A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT"
Missing iptables rule "-A OUTPUT -o lo -j ACCEPT"
Missing ip6tables rule "-P INPUT DROP"
Missing ip6tables rule "-P OUTPUT DROP"
Missing ip6tables rule "-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT"
Missing ip6tables rule "-A INPUT -i lo -j ACCEPT"
Missing ip6tables rule "-A INPUT -p ipv6-icmp -j ACCEPT"
Missing ip6tables rule "-A INPUT -d ff02::fb/128 -p udp -m udp --dport 5353 -j ACCEPT" 
Missing ip6tables rule "-A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT"
Missing ip6tables rule "-A OUTPUT -o lo -j ACCEPT"
Missing ip6tables rule "-A OUTPUT -p ipv6-icmp -j ACCEPT"

security.NetworkListenersNonARC failed in the past (it's no longer run here, since it's informational) due to Moblab-specific services:

/usr/bin/python2.7 is listening at 0.0.0.0:9991
/usr/sbin/mysqld is listening at 127.0.0.1:3306
/usr/bin/python2.7 is listening at :::8080
/usr/sbin/apache2 is listening at :::80

I could add a "firewall" software dependency that's unset for moblab devices so that security.Firewall could be skipped there, and I could probably add a security.NetworkListenersMoblab test to handle the second case, but I'm wondering if we should even be running Tast tests on guado_moblab-release (or fizz-moblab-release, or any other Moblab release builders).

I suspect there are many more informational Tast tests that would fail on Moblab as well, but we don't have visibility into that since guado_moblab-release doesn't run bvt-perbuild:

guado_moblab-release: guado_moblab
  HWTest [moblab]
  HWTest [bvt-inline]
  HWTest [bvt-tast-cq]
  HWTest [bvt-installer]

The corresponding paladin builder looks like it doesn't run any of these tests:

guado_moblab-paladin: guado_moblab
  HWTest [moblab_quick]

So even if I fix these failures, I suspect that they could easily break again in the future since these tests don't run in the CQ.

Keith, what do you think makes the most sense here? Do you look at the guado_moblab-release results when performing releases (and chase down any test failures that have crept in), or would it be better to not run bvt-tast-cq (or bvt-inline?) on these builders?

(As a side mystery: the Autotest versions of these tests, security_Firewall and security_NetworkListeners, are both part of bvt-inline, but it doesn't look like they're run on guado_moblab-release. I have no idea how they're being skipped there.)