It happens every time!

Apparently it's related to this code:

func addTabFromList(ctx context.Context, s *testing.State,
        cr *chrome.Chrome, quiescenceCode string) (*renderer, error) {
        tab, err := addTab(ctx, s, cr, nextURL())
        if err != nil {
                return nil, err
        }
        err = tab.conn.Exec(ctx, quiescenceCode)
        if err != nil {
                return nil, errors.Wrap(err, "cannot inject quiescence code")
        }
        notQuiescedMsg := "tab did not quiesce within timeout"
        notQuiescedError := errors.New(notQuiescedMsg)
        err = testing.Poll(ctx, func(ctx context.Context) error {
                done := false
                err := evalPromiseBody(ctx, s, tab.conn,
                        "resolve(window.__telemetry_testHasReachedNetworkQuiescence())", &done)
                if err != nil {
                        return err
                }
                if !done {
                        return notQuiescedError
                }
                return nil
        }, &testing.PollOptions{
                Timeout:  15 * time.Second,
                Interval: 100 * time.Millisecond,
        })
        if err.Error() == notQuiescedMsg {
                s.Log(notQuiescedMsg)
                err = nil
        }
        return tab, err
}

I was trying to compare err with notQuiescedError but apparently I cannot do that: one is an "error" and the other is a *E.  I tried casting the *E into error but that was not possible.

Uh I don't know why I could not make the comparison earlier.  Now this code works:

func addTabFromList(ctx context.Context, s *testing.State,
        cr *chrome.Chrome, quiescenceCode string) (*renderer, error) {
        tab, err := addTab(ctx, s, cr, nextURL())
        if err != nil {
                return nil, err
        }
        err = tab.conn.Exec(ctx, quiescenceCode)
        if err != nil {
                return nil, errors.Wrap(err, "cannot inject quiescence code")
        }
        notQuiescedMsg := "tab did not quiesce within timeout"
        notQuiescedError := errors.New(notQuiescedMsg)
        err = testing.Poll(ctx, func(ctx context.Context) error {
                done := false
                err := evalPromiseBody(ctx, s, tab.conn,
                        "resolve(window.__telemetry_testHasReachedNetworkQuiescence())", &done)
                if err != nil {
                        return err
                }
                if !done {
                        return notQuiescedError
                }
                return nil
        }, &testing.PollOptions{
                Timeout:  15 * time.Second,
                Interval: 100 * time.Millisecond,
        })
        if err == notQuiescedError {
                s.Log(notQuiescedMsg)
                err = nil
        }
        return tab, err
}

In any case this looks like a compiler or run-time library error.

Sorry, the code in #2 is wrong, because unfortunately testing.Poll wraps the notQuiescedError, so the comparison always fails.

I tried returning the error in a variable like this:

pollError := errors.New(notQuiescedMsg)
...
inside the poll function:
    if err != nil {
        pollError = err
        return err
    }

and then outside testing.Poll, check if pollError is no longer notQuiescedError.  But the assignment pollError = err fails:

./platform/tast-tests/src/chromiumos/tast/local/bundles/cros/platform/memory_pressure.go:187:14: cannot use err (type error) as type *"chromiumos/tast/errors".E in assignment: need type assertion

Full code:

       notQuiescedMsg := "tab did not quiesce within timeout"
        notQuiescedError := errors.New(notQuiescedMsg)
        pollError := notQuiescedError
        err = testing.Poll(ctx, func(ctx context.Context) error {
                done := false
                err := evalPromiseBody(ctx, s, tab.conn,
                        "resolve(window.__telemetry_testHasReachedNetworkQuiescence())", &done)
                if err != nil {
                        pollError = err
                        return err
                }
                if !done {
                        return notQuiescedError
                }
                return nil
        }, &testing.PollOptions{
                Timeout:  15 * time.Second,
                Interval: 100 * time.Millisecond,
        })
        if pollError == notQuiescedError {
                s.Log(notQuiescedMsg)
                err = nil
        }
        return tab, err
}

Oh wait, it works with

                        pollError = err.(*errors.E)

Interesting.

You're mixing error and *errors.E, which is often tricky. In the code in #c4, notQuiescedError and pollError have type *errors.E, but they should be error. Could you try:

 var notQuiescedError error = errors.New(notQuiescedMsg)

So forget about comments #2 and #4, they really have nothing to do with this bug.  The run-time error should not happen in any case.

Since it's a Go bug, not one of ours, I'll figure out where to report it instead, then close this.

:/ I lost the ability to reproduce this quickly.

I don't think that this is a Go bug. It's perfectly normal for Go to panic at runtime when code tries to dereference a nil pointer (and Tast handled this as expected by recovering from the panic and just reporting the test as failing).