As an attempt to asses severity, I ran a server-side analysis for sync-ed Chrome Suggestions. TL;DR: nothing fishy there.

Server-side, 93% of all NTP tiles pointing to this particular URL (mobile.twitter.com/home) use the title "Twitter". The next most popular titles contain the same name in different languages (e.g. ~2% is "تويتر"), adding up to at least 99.8% and probably almost 100% (which I can't verify due to aggregation thresholds).

The fact that we don't reproduce the bug for server-side suggestions indicates that we could have an actual bug client-side (i.e. TopSites), which is what the original report was about.

I have some spare cycles and I will look into this.

I tried to look into the TopSites dataflow and could not see any issue.

I inspected the following parts (the data "flows" from bottom to top):

- TopSites clients use <async> TopSitesImpl::GetMostVisitedURLs 
- TopSitesCache::top_sites
- TopSitesCache::SetTopSites
- TopSitesImpl::SetTopSites
- TopSitesImpl::OnTopSitesAvailableFromHistory
- <async> TopSitesImpl::StartQueryForMostVisited
- HistoryService::QueryMostVisitedURLs
- <async> HistoryBackend::QueryMostVisitedURLs
- VisitSegmentDatabase::QuerySegmentUsage (through HistoryDatabase) from "urls" table;
- That table is populated by URLDatabase::AddURLInternal and URLDatabase::InsertOrUpdateURLRowByID.

Note that URL redirects are obtained in HistoryBackend::QueryMostVisitedURLs independently to URL title. Thus, unless TopSites overwrite the title based on redirects, this is not a culprit.

The only suspicious thing I observed is that each URL is added to URLDatabase via 2 calls and first call never has title. It may be possible that something happens in between the calls leading to a wrong title. Also occasionally leaf content has strange title, e.g. "Twitter" for some tweet.

I am going to continue the investigation, but chances are getting slim.

Now I went from TopSitesImpl up to UI.

Summary: the code is simple, the data is separated in the bridge, but in a trivial way. I haven't seen anything redirects related. I would be surprised if the culprit was actually here. 

I inspected the following parts (the data "flows" from top to bottom):

- TopSites clients use <async> TopSitesImpl::GetMostVisitedURLs 
- it is called from MostVisitedSites::InitiateTopSitesQuery()
- and the result is processed in MostVisitedSites::OnMostVisitedURLsAvailable
- MostVisitedSites::InitiateNotificationForNewTiles
- MostVisitedSites::SaveTilesAndNotify
- MostVisitedSites::Observer::OnMostVisitedURLsAvailable
- MostVisitedSitesBridge::JavaObserver::OnMostVisitedURLsAvailable (data is being separated into multiple arrays here, but in a trivial way)
- (java side)
- MostVisitedSitesBridge wrappedObserver onMostVisitedURLsAvailable
- TileGroup.onMostVisitedURLsAvailable (data is being combined again, but in a trivial way)
- NewTabPageView.onTileDataChanged
- TileGroup.renderTileViews
- TileView.renderTile

This time I went deeper from URLDatabase::AddURLInternal and URLDatabase::InsertOrUpdateURLRowByID to see why the title is set in the second call.

The first call goes like this (only url without a title):
- HistoryBackend::AddPageVisit
- HistoryBackend::NotifyURLVisited
- HistoryService::BackendDelegate::NotifyURLVisited
- InMemoryHistoryBackend::OnURLVisited
- InMemoryHistoryBackend::OnURLVisitedOrModified
- URLDatabase::InsertOrUpdateURLRowByID

The second call is (both title and url):
- WebContentsImpl::UpdateTitleForEntry
- HistoryTabHelper::TitleWasSet
- HistoryTabHelper::UpdateHistoryPageTitle
- HistoryService::SetPageTitle
- HistoryBackend::SetPageTitle
- HistoryBackend::NotifyURLsModified 
- HistoryService::BackendDelegate::NotifyURLsModified 
- HistoryService::NotifyURLsModified 
- InMemoryHistoryBackend::OnURLsModified
- InMemoryHistoryBackend::OnURLVisitedOrModified
- URLDatabase::InsertOrUpdateURLRowByID

Overall, I did not see anything immediately wrong. However, I investigated only one case of typing a ULR in omnibox (which is not in history).

Unfortunately, I am still not able to reproduce the bug and I have to conclude that there is not enough information.

Please reopen if you disagree or like to add something.

Here is a patch with my logging.

Deleted: logging.patch 27.5 KB

logging.patch 27.5 KB Download

Let's keep the bug open to track this further.
Apparently, there's some sequence of action possible on twitter which leads to that state of associating the wrong title with the twitter URL.

Vitalii, can you quickly describe what actions with the site you exercised?

With Darin's hint to use the Twitter Lite PWA I managed to reproduce the problem.
If you install the PWA (open mobile twitter and select "add to homescreen" from the menu), and run Twitter Lite, there are some interesting effects happening in history.

In the attached video, you can see that links within twitter appear twice in history with the same title. The interesting difference is that the first entry has the URL mobile.twitter.com/home while the second one has the link to the leaf page. Both have the same title, e.g. "Raquel Willis on Twitter".

Another interesting observation is timing: As you can see the second entry in history appeared later when I re-opened the history view although I didn't trigger any visits to that site. It appears there's some race condition. 

I think I also observed at times that some sort of clean-up happened in my history view, where such duplicates got removed -- but I'm not 100% sure.

Here's the repro: https://drive.google.com/open?id=0B2aULhDdpumqRkczbVhWYVB5WHc

Deleted: twitter-tiles-repro.mp4 28.6 MB

twitter-tiles-repro.mp4 28.6 MB Download

Thanks Tim, that's very useful.

As a next step, I recommend understanding well why sometimes two history entries show up, and assuming there's a client-side "redirect", narrow down how that is exactly implemented on the site (e.g. history.replaceState(), location override, refresh metatag, etc.).

Unfortunately I cannot reproduce (both ToT and Canary 61.0.3155.0)

My steps:
1) Open Twitter Lite PWA
2) Open some tweet A
3) Open Clank and go to History
4) Observe tweet A as the last item
5) Go back to PWA
6) Go back to tweet stream in PWA
7) Open a different tweet B
8) Go back to Clank and reopen History

I observe:
- tweet B
- tweet stream with title "Twitter"
- tweet A

On video it is:
- tweet B
- tweet stream with title "tweet A"
- tweet A

What am I missing? 

Yesterday, when testing together with tschumann@ on different devices, only he could reproduce the issue, although we were apparently following the same steps (modulo actual clicked tweet etc.). I wouldn't rule out a race condition that is more reproducible in certain devices.  

the only special thing about my usage might be that I tend to have many tabs open (20-30+). Maybe that makes race conditions more likely?

I tried to reproduce this multiple times.

- Without multiple open tabs, I tried on 3 devices (Nexus 6P, Samsung and Moto E). No result. I even checked my logging on Nexus and haven't seen anything suspicious.

- With multiple open tabs (15 news, 15 twitter), I checked on Nexus 6P. Again no result.

Hey vitalii, is there anything else we can do here? Should we grab Tim's very same hardware?

Indeed that looks like the only option so far.

ping - any news?

This issue might be a collection of two:
issue 751023 - which might reduce duplicates and
an upcoming effort to use short titles in TopSites/MostLikely (details TBD)

In either case, this won't happen in M-62 or M-63.

I don't think this bug is related to the two issues you mentioned. 

Here, it looks like somehow we screw up connected titles. While I couldn't reproduce the exact issue today (M61, stable), there's an interesting observation around changing titles in the history view:
-- Open Twitter, click on a post
-- Open history from preferences and you see an entry  with a title like "tsop! on Twitter: <tweet title>" for mobile.twitter.com.
-- If you click that entry, it takes you to the actual tweet (in my reproduction of the bug, this was not the case)
-- If you open history again, the title is now different -- it's "Twitter"

So something is disagreeing what the actual title should be. Maybe that leads us closer to the bug?

sorry, but I did not actively work on this bug since my investigation. Unfortunately, I could not find anything wrong in the code and I am not familiar with it, so I am not sure how to continue.

fhorschig@, could you reassign to someone from site suggestions team?

I can repro the scenario in #19, but it's not clear to me whether it's related, since the observation is the opposite of the original (i.e. root title used for leaf content in history).

Here's how this could be different: twitter seems to use javascript history pushState() to react to a click on a tweet (which doesn't actually issue a http request to the server with the leaf URL) and then presumably use JS to update the content of the page, including the title.

When you click on history, you do actually request that particular URL to the server, and this could be a totally different flow (*presumably* it replies with HTML with title 'Twitter' and later updates the title, which could be ignored by history https://cs.chromium.org/chromium/src/chrome/browser/history/history_tab_helper.cc?type=cs&l=165).

Unless you object, I'll close this bug as can't repro and get back to darin@.

thanks for clarification. It's interesting to see how these different sources play together. So it sounds like one possible explanation would be that either we don't keep things nicely in sync, or actually the twitter JS had a bug.

Closing the bug until we can repro sounds good to me -- please get back to darin.