New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 772110 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Oct 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 765834



Sign in to add a comment

Predator exception of too many entity

Project Member Reported by infe...@chromium.org, Oct 5 2017

Issue description

entity is too big (/base/data/home/apps/s~google.com:findit-for-me/katesonia.404594034428766767/first_party/gae_libs/handlers/base_handler.py:191)
Traceback (most recent call last):
  File "/base/data/home/apps/s~google.com:findit-for-me/katesonia.404594034428766767/first_party/gae_libs/handlers/base_handler.py", line 183, in _Handle
    result = handler_func() or {}
  File "/base/data/home/apps/s~google.com:findit-for-me/katesonia.404594034428766767/frontend/handlers/crash_handler.py", line 77, in HandlePost
    need_analysis, crash_data = NeedNewAnalysis(json_crash_data)
  File "/base/data/home/apps/s~google.com:findit-for-me/katesonia.404594034428766767/frontend/handlers/crash_handler.py", line 104, in NeedNewAnalysis
    return predator_client.NeedsNewAnalysis(crash_data), crash_data
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/utils.py", line 197, in inner_wrapper
    return wrapped_decorator(func, args, kwds, **options)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3841, in transactional
    func, args, kwds, **options).get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 430, in _help_tasklet_along
    value = gen.send(val)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 1014, in transaction
    result = callback()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3849, in <lambda>
    return transaction_async(lambda: func(*args, **kwds), **options)
  File "/base/data/home/apps/s~google.com:findit-for-me/katesonia.404594034428766767/common/predator_app.py", line 210, in NeedsNewAnalysis
    model.put()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3458, in _put
    return self._put_async(**ctx_options).get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 824, in put
    key = yield self._put_batcher.add(entity, options)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 358, in _put_tasklet
    keys = yield self._conn.async_put(options, datastore_entities)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
    result = rpc.get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1893, in __put_hook
    self.check_rpc_success(rpc)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1385, in check_rpc_success
    raise _ToDatastoreError(err)
BadRequestError: entity is too big
Project Member

Comment 2 by bugdroid1@chromium.org, Oct 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/4f37541d34af04b8d91b5c8e2062f1455cadb967

commit 4f37541d34af04b8d91b5c8e2062f1455cadb967
Author: Sharu Jiang <katesonia@google.com>
Date: Tue Oct 10 17:41:17 2017

[Predator] Decrease the maximum stack_trace size.

For some entities, even though every property doesn't exceed the maximum size, but it is still possible that the entity is too big. The is probably because the stack_trace is too big(even when it is less than 1048487 bytes).

So we can decrease the maximum stack_trace size that we think it's ok to put to datastore.

Bug:  772110 
Change-Id: Ibc382dbad21a5ea90d7f1e73a8bc00744dca66be
Reviewed-on: https://chromium-review.googlesource.com/704001
Commit-Queue: Sharu Jiang <katesonia@chromium.org>
Reviewed-by: Shuotao Gao <stgao@chromium.org>

[modify] https://crrev.com/4f37541d34af04b8d91b5c8e2062f1455cadb967/appengine/predator/app/common/model/crash_analysis.py

Status: Verified (was: Assigned)
not seeing this exception any more.

Sign in to add a comment