NDB has a rule that 
  given an async func F1 calling sync func F2
F2 may not make RPCs.

auth lib violates this rule, especially in tests: auth.is_group_member may fetch authdb while being called synchronously from an async func, i.e. there is no protection. It happens to work because in prod we prefetch db before calling application code. This caused a deadlock in buildbucket tests and 2-3h of head-scratching and bisection-by-commenting-parts-of-code. The workaround was to call auth_testing.clear_local_state() and auth.warmup() in setUp().