I came to the conclusion that we have got an issue when reading the following statements of different specs:

https://drafts.csswg.org/css-align-3/#baseline-export
Note: The edges used to synthesize baselines from a box depend on their formatting context: inline-level boxes synthesize from their margin edges [CSS-INLINE-3], table cells synthesize from their content edges [CSS2], and grid and flex items synthesize from their border edges [CSS-GRID-1] [CSS-FLEXBOX-1].

https://drafts.csswg.org/css-inline/#baseline-synthesis-replaced
Note: Authors can use margins (positive or negative) to adjust the alignment of replaced content within a line.

https://www.w3.org/TR/css-writing-modes-3/#replaced-baselines
If an atomic inline (such as an inline-block, inline-table, or replaced inline element) does not have a baseline, then the UA synthesizes a baseline table thus:
- alphabetic
    The alphabetic baseline is assumed to be at the under margin edge. 
- central
    The central baseline is assumed to be halfway between the under and over margin edges of the box. 

https://github.com/w3c/csswg-drafts/issues/638#issuecomment-262734994
E.g. grids and flexboxes synthesize using the border edges of an item, whereas inline layout synthesizes using the margin edges of an inline box. We can't change the latter behavior due to compat. So we're restricted to inline-block need to use the margin edges. In that case, either all block containers use margin edges, or all inline-level boxes do, or everything does. Since treating empty block containers differently from empty grid or flex containers didn't seem reasonable, we narrowed down to either inline-level boxes use margins and flex/grid-level boxes use borders, or all boxes use margins. The discussion concluded on the former: https://lists.w3.org/Archives/Public/www-style/2016Sep/0000.html

I've got already a patch, relatively simple, to fix the issue. However, I'd like someone to confirm the issue before going further on my implementation.

What other browsers do? 

I believe there are a few tests regarding synthesized baselines in imported/csswg, and iirc vertical-rl were failing. I'll check when I'm back on my PC.

s/rl/lr/

Regarding other browsers, I think Safari shares the issue with Chrome because it still uses the same codebase. Regarding Firefox, I filed the following bug: 

https://bugzilla.mozilla.org/show_bug.cgi?id=1322986

I understand that FF acknowledges the issue and it's working on it. In any case, I'll implement that what other browsers do. In that regard, I'll file a bug in WK to see how they think about this issue.

As it was discussed and agreed in the same bug [1] reported for Firefox, the current implemented behavior seems correct. So I'll close this issue as WONTFIX.