ArtifactResults object may overwrite artifacts |
|||
Issue descriptionTrying to reason about why it got really tricky to deal with a tempfile to later add as an artifact here: [1]: https://chromium-review.googlesource.com/c/catapult/+/772831/29..34/telemetry/telemetry/internal/story_runner.py I noticed there is actually a bug on AddArtifact as currently implemented. Say we create a tempfile: /tmp/test_ZNsuLC.log And then we do results.AddArtifact( 'test1', 'logs', '/tmp/test_ZNsuLC.log') Which will move 'test_ZNsuLC.log' to the artifacts_dir and register it as a log of test1. Many tests later, since that name no longer exists on the tempdir, by pure chance the name might be reused. So we are unlucky and create again: /tmp/test_ZNsuLC.log And then we do results.AddArtifact( 'test42', 'logs', '/tmp/test_ZNsuLC.log') Which will move 'test_ZNsuLC.log' to the artifacts_dir, overwriting the logs from test1. Now even more I think having the ArtifactResults object moving files into the artifacts_dir is problematic (because we'll always be liable to this sort of collisions) and we should try to avoid that as much as possible. Instead we should try to create the files _within_ the artifacts_dir to begin with. For this particular case, my proposal would be to have some, say, CreateArtifact method that would: - automatically create a new file name _within_ the artifacts_dir (e.g. using the `dir` option of tempfile.mkstemp or tempfile.NamedTemporaryFile) - ensure the file is added to the artifacts list *even in the case of failure* since particularly in case of failure is when we want to ensure that logs are preserved. Then, rather than the current tmpfile dance in [1], we could simply do: with results.CreateArtifact('test42', 'logs') as log_file: with logging_util.CaptureLogs(log_file): logging.info('Add this to the logs')
,
Dec 7 2017
I think a create artifact method would be great. Didn't think of that when I was writing the CL. Thanks for the idea! I think that'll simplify the code a lot, and mean less file system operations and stuff. I'll make a CL to do this. I don't think the name collision will happen that much; quick math shows that there are 3,521,614,606,208 (7 ^ 62. 52 characters, 10 digits) possible names. If we create 1000 temp files, the chance we have a collision is 1 - (3,521,614,606,207 / 3,521,614,606,208) ^ 1000, which google says is 2.8399505e-10. I could be doing the math wrong though.
,
Dec 7 2017
Actually the math for the second part is wrong. It should really be 1 - (3,521,614,606,207 / 3,521,614,606,208) * (3,521,614,606,206 / 3,521,614,606,208) * ..., but I don't want to write that out 1000 times.
,
Dec 7 2017
Actually I don't even know if that math is right (comment 3). But regardless of if a collision would happen or not, I'll still do the change.
,
Dec 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/1e7ecd60c6727b05aeed749ada4c963f4bbde719 commit 1e7ecd60c6727b05aeed749ada4c963f4bbde719 Author: Stephen Martinis <martiniss@chromium.org> Date: Mon Dec 18 21:11:33 2017 Add CreateArtifact to page test results This also modifies story_runner to use this. Bug: chromium:792389 Change-Id: I9aecbae098c549a81f7e905c930bf11f826a99a8 Reviewed-on: https://chromium-review.googlesource.com/812429 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> [modify] https://crrev.com/1e7ecd60c6727b05aeed749ada4c963f4bbde719/common/py_utils/py_utils/logging_util.py [modify] https://crrev.com/1e7ecd60c6727b05aeed749ada4c963f4bbde719/telemetry/telemetry/internal/results/page_test_results.py [modify] https://crrev.com/1e7ecd60c6727b05aeed749ada4c963f4bbde719/common/py_utils/py_utils/logging_util_unittest.py [modify] https://crrev.com/1e7ecd60c6727b05aeed749ada4c963f4bbde719/telemetry/telemetry/internal/results/artifact_results_unittest.py [modify] https://crrev.com/1e7ecd60c6727b05aeed749ada4c963f4bbde719/telemetry/telemetry/internal/story_runner.py [modify] https://crrev.com/1e7ecd60c6727b05aeed749ada4c963f4bbde719/telemetry/telemetry/internal/story_runner_unittest.py [modify] https://crrev.com/1e7ecd60c6727b05aeed749ada4c963f4bbde719/telemetry/telemetry/internal/results/results_options.py [modify] https://crrev.com/1e7ecd60c6727b05aeed749ada4c963f4bbde719/telemetry/telemetry/internal/results/artifact_results.py
,
Dec 28 2017
|
|||
►
Sign in to add a comment |
|||
Comment 1 by perezju@chromium.org
, Dec 6 2017