New issue
Advanced search Search tips

Issue 679560 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

[WPT Dashboard] Show test result previews on directory pages

Project Member Reported by jeffcarp@chromium.org, Jan 10 2017

Issue description

I'm imagining something like:

------------------------------------
web-platform-tests / 2dcontext / drawing-rectangles-to-the-canvas /
Directory contents:
------------------------------------

2d.clearRect.basic.html
Chrome 50% green  | FF 75% green  | Edge 100% green

2d.clearRect.clip.html
Chrome 50% green  | FF 75% green  | Edge 100% green

2d.clearRect.globalalpha.html
Chrome 90% green  | FF 100% green | Edge 75% green

2d.clearRect.globalcomposite.html
Chrome 100% green | FF 75% green  | Edge 50% green

2d.clearRect.negative.html
Chrome 100% green | FF 75% green  | Edge 50% green
 
Blocking: -647861
Cc: foolip@chromium.org rbyers@chromium.org qyears...@chromium.org
foolip & rbyers & qyearsley - would it be useful to show these percentages to give people a better sense of the test results when scrolling through a directory, or does it conflict too much with our goal of not exposing 'gameable' metrics?

Comment 2 by foolip@chromium.org, Jan 25 2017

I think that's fine certainly on individual tests. You could also use a pass/total notation and make it green only if 100% to get basically the same effect. It would also show curious cases where different browsers are getting different numbers of tests, which is possible in principle at least.

I think people would get more nervous if we start to aggregate it into per-directory or total scores, so I suppose avoid that for now.
Status: Fixed (was: Assigned)
I've updated the dashboard to show results on directory pages! Check it out, here's a good example page:
https://wptdashboard.appspot.com/t/dom/events

(the full results are re-uploading now so some directories might not exist yet)

Comment 4 by foolip@chromium.org, Jan 27 2017

Awesomesauce! One improvement you might make is to make 99% visually quite distinct from 100%, perhaps by fading from red to white or something?

Comment 5 by foolip@chromium.org, Jan 27 2017

I also think it would be fine to omit the percentages, not because they are inappropriate, but they don't add much and make the results look more cluttered.

Comment 6 by foolip@chromium.org, Jan 27 2017

If the title could be made "Web Platform Tests Dashboard" that would also be great.

Comment 7 by foolip@chromium.org, Jan 27 2017

Hmm, how come only some tests say test type JS? In https://wptdashboard.appspot.com/t/dom/events I would expect all tests to be the same type.

Comment 8 by foolip@chromium.org, Jan 27 2017

Also, this really looks quite nice, we've all seen dashboard that are a jumble of different font sizes and weird borders, but this is just clean.
> One improvement you might make is to make 99% visually quite distinct from 100%, perhaps by fading from red to white or something?

I'm not sure I understand completely - do you mean something like this? (scale is not linear)

% Passing:
0%        50%       99%     100%
|----------|---------|-------|
100%      50%        |      green
red       red      0% red 
                  (white)

> I also think it would be fine to omit the percentages

Agreed, done

> Hmm, how come only some tests say test type JS?

This is because I only started assigning test types recently on test runs, so not all results have a test_type yet. I was planning on waiting until we did another full test run for everything to be filled in, but I can also write a backfill script to populate it now if it's confusing.

> If the title could be made "Web Platform Tests Dashboard" that would also be great.

Done
Yep, that red-white-green scale is what I had in mind. But any scheme where any failure can immediately be seen by color would do, like red-yellow-green, or just a binary red/green.
Components: Blink>Infra>Ecosystem
Components: -Blink>Infra>Predictability

Sign in to add a comment