This is not a major issue but it'd be nice to behave more consistently for users with disabilities who rely on the keyboard.

Steps to repro:

1. Load this document:

data:text/html,<textarea cols=6 rows=10>a-b-c-d-e-f-g-h-i-j-k-l-m-n-o-p-q-r-s-t-u-v-w-x-y-z</textarea>

2. Start with the cursor at the beginning of the first line

3. Press the End key to move the cursor to the end of the first line, now affinity is upstream.

4. Now press shift+right or down to extend the selection

5. Then press left arrow to collapse the selection back to a cursor

Expected result:
The cursor goes back to the end of the first line (affinity upstream)

Actual result:
The cursor is at the start of the second line (affinity downstream)

Note that if there is an explicit line break between the first and second lines, this behavior doesn't occur.

It seems to me like a selection should just keep track of the anchor affinity, that would solve the issue.