Those steps seem right to me, and I was able to get the expected results a few times last week. I also had a case or two where it didn't work as expected.

Adding Pierre-Alexandre since I think he is the most recent person on the team to have worked with secure settings, in case he has any ideas.

@pbommana was this test on a corp machine? The proteus settings hardening feature is not enabled on enterprise domain-joined Windows. 

Ahh, Yes this was on Corp machine. 

I just tried the same on non corp machine and below are my observations want to double check the behavior :

Scenario 1 :
1. Launch Chrome and from Chrome settings --> Enable Show home button --> Set it as "https://play.google.com"
2. Exit Chrome
3. Navigate to "C:\Users\pbommana\AppData\Local\Google\Chrome\User Data\Default\Secure preferences"
4. Change Home page from "https://play.google.com" to "https://www.dell.com"
5. relaunch Chrome 

Observed behavior :
1. Under Chrome Settings I still see homepage set to "https://play.google.com" and there wasn't any bubble stating that settings were changed.


Scenario 2 :
1. Launch Chrome and from Chrome settings --> Enable Show home button --> Set it as "https://play.google.com"
2. Exit Chrome
3. Navigate to "C:\Users\pbommana\AppData\Local\Google\Chrome\User Data\Default\Secure preferences"
4. Change Home page from "https://play.google.com" to "https://www.dell.com"
5. Navigate to "C:\Users\pbommana\AppData\Local\Google\Chrome\User Data\Default\Preferences"
6. Change Home page from "https://play.google.com" to "https://www.dell.com"
7. Relaunch Chrome

Observed behavior :
1. There wasn't any bubble on the page stating that my homepage or Chromesettings were compromised.
2. Once navigate to "Chrome://settings" I see the dialog "Some Settings were reset"( I was expecting the bubble right after launch of Chrome from hotdog menu instead of going into Chrome://settings" please correct me if the feature has been changed)

Tried reproducing on Chrome M65 (65.0.3325.181) on a VM. For Scenario #2 Step #5-6, I did not see the home button URL in the regular Preferences file - only in the Secure Preferences file.

For Scenario 1, I observed the same behavior as you did for Scenario 2:
1. There wasn't any bubble on the page stating that my homepage or Chrome settings were compromised.
2. Once navigate to "Chrome://settings" I see the dialog "Some Settings were reset"

Note to self: maybe related to https://chromium-review.googlesource.com/c/chromium/src/+/985110?

I can't seem to find the code responsible for showing the bubble right after the launch of Chrome. It's possible it was removed some time ago (@gab do you know if this was done during the mojo-ification?). 

@pbommana Can you link me to the test case which documents the expected behavior? I might be able to find it easier with the expected strings. 

Apologize for the confusion where I mixed with CRX validation with Settings hardening bubble.

There shouldn't be any bubble on launch it's only when user navigates to "Chrome://settings".

No problem.

This only leaves the unexplained Scenario 1 from Comment #4. Could you try it again if you have the time? It's weird that the homepage stayed at the good value before the modification. Maybe the secure preferences file wasn't saved properly after the modification?

Thanks!

Tested this on multiple machines and can confirm that I was able to see "Some settings were reset" dialog when homepage is modified under "Secure Preferences" file. 

 proberge@ you are right this might be the file wasn't saved properly. Thank you again.

Thanks for the update. Marking as WontFix - intended behavior.