Some updates:
With a chromebook with an actual search button instead of caps-lock this is working now (at least on current dev channel; may have been fixed through crbug/782572). However, it’s important to set search-button behavior to search-button instead of caps-lock in the keyboard settings.

With an external keyboard that has a caps-lock key, this is still broken and caps-lock is always activated after caps-lock key has been pressed.

Also lshift+rshift doesn’t trigger caps-lock in any variant, but that’s a minor problem.

In Neo2 layout, Hitting Caps Lock key generate an event with dom_code=CapsLock and dom_key=AltGraph. An remapped key will be generated here [1]. Later, not sure whether it's a regression, the remapped key is overwritten [2]. After I replace the line 726-728 with:

if (!remapped_key) {
  characteristic_flag = ui::EF_CAPS_LOCK_ON;
  remapped_key =
      GetRemappedKey(prefs::kLanguageRemapCapsLockKeyTo, delegate_);
}

The reported issue is resolved.

[1] https://cs.chromium.org/chromium/src/ui/chromeos/events/event_rewriter_chromeos.cc?dr&l=693
[2] https://cs.chromium.org/chromium/src/ui/chromeos/events/event_rewriter_chromeos.cc?dr&l=727

I tested the proposed fix with ToT, but Hitting Caps Lock key seems to generate an event with dom_code=META_LEFT and dom_key=AltGraph, so it doesn't work well.
I'll investigate in more detail in next week.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/09cf341567b00dc10d651eb5ccc96b6d0210a39b

commit 09cf341567b00dc10d651eb5ccc96b6d0210a39b
Author: Yuichiro Hanada <yhanada@chromium.org>
Date: Sat Feb 24 00:01:34 2018

Make Caps-Lock work as Mod3 on Neo2 keyboard layout.

This CL fixes the issue when connecting an external keyboard that has
a Caps-Lock key.
The issue happening when mapping Search key to Caps-Lock key on the
internal keyboard will be addressed later.

Bug:  747823 
Test: Manual. Confirm that Caps-Lock + Q generates @ and Caps-Lock is
      not enabled after that.

Change-Id: I96c378d85ca85ce0d78bbc1ebd57bd44fb5f1a5f
Reviewed-on: https://chromium-review.googlesource.com/933667
Reviewed-by: Dan Erat <derat@chromium.org>
Commit-Queue: Yuichiro Hanada <yhanada@chromium.org>
Cr-Commit-Position: refs/heads/master@{#538952}
[modify] https://crrev.com/09cf341567b00dc10d651eb5ccc96b6d0210a39b/ui/chromeos/events/event_rewriter_chromeos.cc

Is this still open?

No, this has been fixed. I verified that it’s working on current builds.
Thank you!

Oops, I forgot to update this issue. Let me close it now.

I opened:

https://crbug.com/893336

For some remaining problems with the neo layout (layer 6 and compose still don't work)

 Issue 495277  has been merged into this issue.