IDMap is very odd. A simple map for id-based objects perhaps has use, but IDMap is for pointer values only:

  T* Lookup(KeyType id) const {
    DCHECK(sequence_checker_.CalledOnValidSequence());
    typename HashTable::const_iterator i = data_.find(id);
    if (i == data_.end())
      return nullptr;
    return &*i->second;
  }

From what I can tell, the majority of uses of IDMap is to hold unique_ptr, so by allowing the objects to live directly in the container, that would simplify and speed things up.

The IDMap::Iterator class could also use some cleanup to allow range-for use.