That's the expected behavior. Different boards may include different tests. For example:
https://cs.corp.google.com/chromeos_public/src/third_party/chromiumos-overlay/chromeos-base/autotest-tests-power/autotest-tests-power-9999.ebuild?type=cs&q=autotest-tests-power-9999.ebuild&l=1

That ebuild decides which tests to be included in a build, conditional include like:
shill? ( +tests_power_Resume )
If a board doesn't use shill, that test will not be included.

Also, some tests have special DEPENDENCIES, if lab can't find any duts to satisfy the required dependencies, the test will be skipped.

But it is surprising that a board has different number of bvt-inline tests builds produced in the same week.

e.g. Stout
R53-8530.25.0: 3
R53-8530.24.0: 25
R53-8530.23.0: 30
R53-8530.21.0: 10

Most of bvt-inline tests don't have special dependencies and do we know why would the ebuild decide to include 10 tests today and tomorrow 30 tests?

This can be chalked up as infra flake, most likely. Stout definitely seems to have "missing" tests more often.

I see 60-61 test results from the most recent builds. https://wmatrix.googleplex.com/unfiltered?releases=53&suites=bvt-inline&platforms=link|stout|parrot_ivb&days_back=90

This bug might be worthwhile to keep open for monitoring.

Please compare two suite runs and see what tests are missing, are the missing test has special label dependency? I'm wondering if this can be a label detection issue. 

+richard, does this looks like DUT issue ?

+vsuley, is this still happening ?

This should be tracked by the current morning planner :-)

Assigning to dhaddock@ as morning planning 

This is still happening. Today we are verifying M54 8743.35.0

on bvt-inline:
chell had 23 test run 
butterfly had 37 

Both usually have around 60. 

https://wmatrix.googleplex.com/unfiltered?suites=bvt-inline&releases=54&days_back=15

I'm the morning planner this week, will collect data best I can and update this issue. 

This appears to have improved greatly, starting Oct 1st (M54-8743.50.). Most builds and board are running ~60 bvt-inline tests. Dan, do you know what changed, and whether this is now fixed?

Assign to deputy.

I think Kevin fixed bunch of label detection code.

Stable in 59.