What I'm imagining:

A very simple App Engine app that serves two pages:

/              The index page with a login button if the user is not logged in.
               If the user is logged in and is an admin  (we could add everyone 
               in the chrome-infra MDB group as an admin), there'd be a giant 
               ON/OFF button
/control.json  A JSON blob of the following:

{
  "dry_run": true // controls whether
}

On the WPT Exporter side, we could add a flag --remote-control-url to tell the Exporter to read in configuration options from that URL. e.g.
./wpt-exporter --gh-user=... --gh-token=... --remote-control-url=https://wpt-exporter.appspot.com/control.json

For documentation, we could add "if the WPT Exporter is misbehaving, the first thing to do is go to the WPT Exporter control app and flip it to OFF."
 

What I'm imagining:

A very simple App Engine app that serves two pages:

/              The index page with a login button if the user is not logged in.
               If the user is logged in and is an admin  (we could add everyone 
               in the chrome-infra MDB group as an admin), there'd be a giant 
               ON/OFF button
/control.json  A JSON blob of the following:

{
  "dry_run": true // controls whether --dry-run is on or off
}

On the WPT Exporter side, we could add a flag --remote-control-url to tell the Exporter to read in configuration options from that URL. e.g.
./wpt-exporter --gh-user=... --gh-token=... --remote-control-url=https://wpt-exporter.appspot.com/control.json

For documentation, we could add "if the WPT Exporter is misbehaving, the first thing to do is go to the WPT Exporter control app and flip it to OFF."
 

If the exporter starts misbehaving, anyone on Chrome Infra should be able to turn it off quickly.

----------

(keeping previous description around)

What I'm imagining:

A very simple App Engine app that serves two pages:

/              The index page with a login button if the user is not logged in.
               If the user is logged in and is an admin  (we could add everyone 
               in the chrome-infra MDB group as an admin), there'd be a giant 
               ON/OFF button
/control.json  A JSON blob of the following:

{
  "dry_run": true // controls whether --dry-run is on or off
}

On the WPT Exporter side, we could add a flag --remote-control-url to tell the Exporter to read in configuration options from that URL. e.g.
./wpt-exporter --gh-user=... --gh-token=... --remote-control-url=https://wpt-exporter.appspot.com/control.json

For documentation, we could add "if the WPT Exporter is misbehaving, the first thing to do is go to the WPT Exporter control app and flip it to OFF."