chromeos2-devserver9 database entry set up incompletely
Reported by
jrbarnette@chromium.org,
Feb 7 2018
|
|||||||
Issue description
From the server database, you find this:
$ atest server list chromeos2-devserver9.cros.corp.google.com
Hostname : chromeos2-devserver9.cros.corp.google.com
Status : primary
Roles : devserver
Attributes : {}
Date Created : 2018-02-05 13:51:57
Date Modified: 2018-02-05 13:51:57
Note : None
The devserver attributes are empty. That shouldn't be: Without
properly set up attributes, the devserver can't be entered into
shadow_config.ini, and without that, it can't be used.
We need to figure out what went wrong adding the server, and
fix the tool (preferable) or process (if it can't be isolated
inside a tool) so that it doesn't happen again.
,
Feb 7 2018
Technically that is a "beta" devserver to test new the equipment. Currently being used by davidriley for some devserver work It is not suppose to be used for production use yet.
,
Feb 7 2018
,
Feb 7 2018
I've fixed the server attributes:
$ HOST=chromeos2-devserver9
$ atest server modify -a port -e 8082 $HOST.cros.corp.google.com
Attribute `port` of server chromeos2-devserver9.cros.corp.google.com is set to 8082.
Server chromeos2-devserver9.cros.corp.google.com is modified successfully.
Hostname : chromeos2-devserver9.cros.corp.google.com
Status : primary
Roles : devserver
Attributes : {u'port': u'8082'}
Date Created : 2018-02-05 13:51:57
Date Modified: 2018-02-05 13:51:57
Note : None
$ atest server modify -a restricted_subnet -e 100.115.224.0/19 $HOST.cros.corp.google.com
Attribute `restricted_subnet` of server chromeos2-devserver9.cros.corp.google.com is set to 100.115.224.0/19.
Server chromeos2-devserver9.cros.corp.google.com is modified successfully.
Hostname : chromeos2-devserver9.cros.corp.google.com
Status : primary
Roles : devserver
Attributes : {u'restricted_subnet': u'100.115.224.0/19', u'port': u'8082'}
Date Created : 2018-02-05 13:51:57
Date Modified: 2018-02-05 13:51:57
Note : None
,
Feb 7 2018
If it's not in prod, it should probably not be in the prod server database. We also use that for filtering metrics and other things.
,
Feb 7 2018
> Technically that is a "beta" devserver to test new the equipment. Currently being used by davidriley for some devserver work > > It is not suppose to be used for production use yet. OK. The database entry as it was created problems, so I'd rather not return to the status quo ante. davidriley@ - can you describe the current use and why/whether/how the devserver needs to be in the database at all?
,
Feb 7 2018
I'm no longer needing exclusive use for testing and I asked xixuan@ (current deputy) for it to be added to the production set of devservers.
,
Feb 7 2018
> [ ... ] I asked [ ... ] for it to be added to the production set of devservers. It should have become eligible for use in production the moment the attributes were fixed in comment#4. That said, I'd still like to clean up our process here: This server sat around unused for some time, and that was undesirable. Even if we figured out how to avoid that for the next time, there are two points on which I'd like to be convinced: * That the current manual procedure has up-to-date documentation. * That current procedure is sufficiently automated.
,
Feb 7 2018
Re #5 & #8, Where is the documentation for 'manually add one worked devserver to production'?
,
Feb 7 2018
On a different note I would have preferred the devserver be return back to continue some optimizations but oh well.
Can we also test changing this
Attributes : {u'restricted_subnet': u'100.115.224.0/19', u'port': u'8082'}
to
Attributes : {u'restricted_subnet': u'100.115.128.0/17', u'port': u'8082'}
,
Feb 7 2018
Re c#10: What optimizations are we talking about? It's reasonably performant as-is, so I'm not sure if it's a good use of time doing further optimization. I'd much rather have the time spent by bringing up the other similarly specced machines and turning off the 2x1Gbps and 2x2Gb machines in favour of these new ones.
,
Feb 8 2018
Server side Comparison of read cache settings on the RAID card Network side Comparison between cut-through vs store-and-forward buffering performance at max load on the network switch It's one of those once I get it right I can duplicate across the labs. I'm sure it's fast enough now but if I can squeeze a but more out of it it'd be nice :) Also need to document the new setup for the rest of the englab-sys-cros team. Although given jrbarnette's other work (network bandwidth management) can be made moot with an expedited manual rollout so plans are underway already.
,
Mar 8 2018
Is this server finally 100% setup correctly? If yes I would like to change the restricted subnet to 100.115.128.0/17
,
Mar 8 2018
pprabhu@pprabhu:infra_internal$ atest server modify chromeos2-devserver9.cros.corp.google.com -a restricted_subnet -e 100.115.128.0/17 Attribute `restricted_subnet` of server chromeos2-devserver9.cros.corp.google.com is changed from 100.115.224.0/19 to 100.115.128.0/17. Server chromeos2-devserver9.cros.corp.google.com is modified successfully. Hostname : chromeos2-devserver9.cros.corp.google.com Status : primary Roles : devserver Attributes : {u'restricted_subnet': u'100.115.128.0/17', u'port': u'8082'} Date Created : 2018-02-05 13:51:57 Date Modified: 2018-02-05 13:51:57 Note : None That devserver now has a legit entry. server_management_library does not have a task to add a devserver to prod. This means that we're currently setting up the server-db entries manually. The instructions are here: https://sites.google.com/a/google.com/chromeos/for-team-members/infrastructure/add-devserver-to-prod and are linked from the admin tasks reference page (after a few clicks) Filed issue 820197 for automating that. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by jrbarnette@chromium.org
, Feb 7 2018