I see that your benchmarking logic is in trace_event_perftest.cc. Does that enable testing tracing performance when there are multiple processes & threads? I don't see any logic for creating multiple processes/threads before running test.

If this a micro benchmark, why is this not using the same infrastructure for cc_perf_test? The fact that you need to update the tools/perf/generate_perf_json.py seems strange to me. 

>  I don't see any logic for creating multiple processes/threads before running test.
the plan is to follow up with more coverage (including multithreading) once the bulk skeleton lands.
Not sure instead about multiprocess, all tracing is process local % flushing.

Primiano: I thought part of tracing v2 is having a dedicated tracing process & have all other processes dumb trace logs to that process? That is to fix the problem of when a process dies, its trace log are lost?

If you think cc micro benchmark can cover all the main use cases of tracingV2, I have no more concern about the approach. Though we should figure out why this is not reusing the existing cc micro benchmark bot setup.

> Primiano: I thought part of tracing v2 is having a dedicated tracing process & have all other processes dumb trace logs to that process? That is to fix the problem of when a process dies, its trace log are lost?

Ok good point. Right now we don't have that multiprocess part (And won't have it until we get there, likely EOQ) but will have it at some point as you say.
Until then, nothing will change cross-process, and getting coverage of the basic macros has higher prio.
We should definitely setup a test to cover that as well, perhaps inspired to memory_tracing_browsertest.cc which just fakes at the IPC boundary.

I suggest to have that though after reshuffling the tracing codebase as we are planning in this week.

> Though we should figure out why this is not reusing the existing cc micro benchmark bot setup.
Not sure I follow this. From what I see in https://codereview.chromium.org/2418983003/, this is doing that.
The "cc micro benchmark bot setup" seems to me gtest_perf_test.py, which is what we are using there as well?

Oh, I realize that Emily also put other cc perf tests there as well, but I am not sure that tools/perf/generate_perf_json.py is ready to use in production mode yet. 

+eyaich@ would know the status of swarming cc micro benchmarks

generate_perf_json.py is currently used in production for the non-telemetry based perf tests, ie cc_perftests, as it generates the src side json.  The only thing you would need to change here is if you are planning on running the tests on an additional bot, you would add the json in this file, similar to https://codereview.chromium.org/2418983003/.

I am actively working on swarming cc micro benchmarks, but that is a WIP.  I hope to get through that this week though.

eyaich@ so kraynov is introducing a new test which is very similar in nature to cc_perftests. What does he have to do? Is changing both the py and the .json wrong? What is the best way forward?
Thanks

You need to change the generate_perf_json.py script then run it to generate the new json and check both in. 

In generate_perf_json, you need to add an entry for each bot that you are running the new test on, similar to https://codereview.chromium.org/2418983003/.  I would be happy to review it.

but https://codereview.chromium.org/2418983003/ is precisely the CL in question :)
can you please review it?

sorry!  wasn't paying attention and thought that one was already checked in.  done.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c07975895dccd5899b528949fd82750a90387a91

commit c07975895dccd5899b528949fd82750a90387a91
Author: kraynov <kraynov@chromium.org>
Date: Tue Oct 25 16:45:06 2016

Tracing: Enable tracing_perftests on bots.

There is an intention to track TRACE_EVENT macros performance
without browser and it's designed exactly like cc_perftests.
See bug entry for details.

BUG= 656029 

Review-Url: https://codereview.chromium.org/2418983003
Cr-Commit-Position: refs/heads/master@{#427379}

[modify] https://crrev.com/c07975895dccd5899b528949fd82750a90387a91/BUILD.gn
[modify] https://crrev.com/c07975895dccd5899b528949fd82750a90387a91/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/c07975895dccd5899b528949fd82750a90387a91/tools/perf/generate_perf_json.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fe26f22a88e179d2edfa2f3d9ed37c108c6db405

commit fe26f22a88e179d2edfa2f3d9ed37c108c6db405
Author: kraynov <kraynov@chromium.org>
Date: Tue Nov 15 01:49:39 2016

Performance tests for tracing macros.

Cases introduced:
- Long-running TRACE_EVENT0
- Using TracedValue
- Tracing from 4 concurrent threads

BUG= 656029 

Review-Url: https://codereview.chromium.org/2450953003
Cr-Commit-Position: refs/heads/master@{#432051}

[modify] https://crrev.com/fe26f22a88e179d2edfa2f3d9ed37c108c6db405/components/tracing/test/trace_event_perftest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/209cf4279b5ffd543f5b2fd885544a9c7345db41

commit 209cf4279b5ffd543f5b2fd885544a9c7345db41
Author: kraynov <kraynov@chromium.org>
Date: Tue Nov 15 20:41:34 2016

Enable tracing perftests on Android bots.

BUG= 656029 

Review-Url: https://codereview.chromium.org/2498113002
Cr-Commit-Position: refs/heads/master@{#432245}

[modify] https://crrev.com/209cf4279b5ffd543f5b2fd885544a9c7345db41/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/209cf4279b5ffd543f5b2fd885544a9c7345db41/tools/perf/generate_perf_json.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/822db0de74c2a00f24869a50b78e461c42ac9495

commit 822db0de74c2a00f24869a50b78e461c42ac9495
Author: charliea <charliea@chromium.org>
Date: Fri Nov 18 23:11:59 2016

Revert "Enable tracing perftests on Android bots."

This reverts commit 209cf4279b5ffd543f5b2fd885544a9c7345db41.

The test is failing on all Android bots.

TBR=sullivan,dtu,eyaich1,kraynov
BUG= 656029 , 666828 

Review-Url: https://codereview.chromium.org/2513923003
Cr-Commit-Position: refs/heads/master@{#433326}

[modify] https://crrev.com/822db0de74c2a00f24869a50b78e461c42ac9495/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/822db0de74c2a00f24869a50b78e461c42ac9495/tools/perf/generate_perf_json.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1e9ba1e05266a90a6c2e71c9f362aa2fd54256ba

commit 1e9ba1e05266a90a6c2e71c9f362aa2fd54256ba
Author: kraynov <kraynov@chromium.org>
Date: Fri Dec 09 15:07:36 2016

Enable tracing perftests on Android bots.

Re-landing https://crrev.com/2498113002 CL which was reverted because
bot's builder didn't build tracing_perftests target.
It has been fixed https://chromium-review.googlesource.com/c/417166 .

BUG= 656029 , 666828 

Review-Url: https://codereview.chromium.org/2562013002
Cr-Commit-Position: refs/heads/master@{#437537}

[modify] https://crrev.com/1e9ba1e05266a90a6c2e71c9f362aa2fd54256ba/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/1e9ba1e05266a90a6c2e71c9f362aa2fd54256ba/tools/perf/generate_perf_json.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4dcd14614295f37dfee188a3d198c02db751d2dc

commit 4dcd14614295f37dfee188a3d198c02db751d2dc
Author: kraynov <kraynov@chromium.org>
Date: Fri Jan 27 17:12:35 2017

tracing_perftests: add coverage to TRACE_TASK_EXECUTION.

BUG= 656029 

Review-Url: https://codereview.chromium.org/2592093003
Cr-Commit-Position: refs/heads/master@{#446700}

[modify] https://crrev.com/4dcd14614295f37dfee188a3d198c02db751d2dc/components/tracing/test/trace_event_perftest.cc

Graphs:
go/tracing-perftests-android
go/tracing-perftests-linux