bqschemaupdater: -dry-run output is not useful |
|||||
Issue descriptioncontext: https://chromium-review.googlesource.com/c/chromium/tools/build/+/888179 When running bqschemaupdater -dry-run -table cr-buildbucket-dev.test.x \ -message devtools_goma.CompileEvent \ -I ~/infra/goma-client/lib/ in CWD = local of https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/goma The output is ============ Running getTableMetadata for datasetID test tableID x googleapi: Error 404: Not found: Dataset cr-buildbucket-dev:test, notFound Would run createTable with datasetID test, tableID x, metadata &{Name: Description:This is BigQuery schema for goma-related events stored in chrome-infra. NEXT ID TO USE: 6 Schema:[0xc42008da90 0xc42008dae0 0xc42008db30 0xc4202865f0 0xc420286640] ViewQuery: UseLegacySQL:false UseStandardSQL:false TimePartitioning:0xc42039a080 ExpirationTime:0001-01-01 00:00:00 +0000 UTC Labels:map[] ExternalDataConfig:<nil> FullID: Type: CreationTime:0001-01-01 00:00:00 +0000 UTC LastModifiedTime:0001-01-01 00:00:00 +0000 UTC NumBytes:0 NumRows:0 StreamingBuffer:<nil> ETag:} 2018/01/26 09:34:36 Finished updating table. ============ This is hard to read. Further, let's imagine we have a table X with 100 fields and we are adding one more field to the proto. Dry run will print 101 fields and that would be hard to read and generally hard to understand what exactly is being changed.
,
Jan 31 2018
,
Jan 31 2018
Thanks for documenting this nodir!
,
Jan 31 2018
WDYT about the proposal in comment 1?
,
Feb 22 2018
going to do this because going to use this tool myself for a large schema
,
Feb 22 2018
ty! Also, sorry for not responding to #4. SGTM.
,
Feb 22 2018
,
Feb 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/20ceecc8c85438ccf082e2ae2e624f6aa8e9d843 commit 20ceecc8c85438ccf082e2ae2e624f6aa8e9d843 Author: Nodir Turakulov <nodir@google.com> Date: Fri Feb 23 19:38:01 2018 [descutil] Fix panic in descutil.IndexSourceCodeInfo descutil.At can return a slice and a slice cannot be used as a map key. Bug: 806292 Change-Id: Ifbbc60b2801a47aa643552fea1cf4a7081de53fe Reviewed-on: https://chromium-review.googlesource.com/932753 Reviewed-by: Katie Thomas <katthomas@google.com> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/20ceecc8c85438ccf082e2ae2e624f6aa8e9d843/common/proto/google/descutil/util.go [modify] https://crrev.com/20ceecc8c85438ccf082e2ae2e624f6aa8e9d843/common/proto/google/descutil/util_test.desc [modify] https://crrev.com/20ceecc8c85438ccf082e2ae2e624f6aa8e9d843/common/proto/google/descutil/util_test.pb_test.go [modify] https://crrev.com/20ceecc8c85438ccf082e2ae2e624f6aa8e9d843/common/proto/google/descutil/util_test.proto
,
Feb 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/586804aac3fc3c0f0617edea9c5948ec092bf693 commit 586804aac3fc3c0f0617edea9c5948ec092bf693 Author: Nodir Turakulov <nodir@google.com> Date: Fri Feb 23 20:44:41 2018 [bqschemaupdater] make schema text more compact Currenly textual representation of a schema is verbose for large schemas. Make it more compact. The new format is inspired by protobufs. Example: https://gist.githubusercontent.com/nodirt/0592b57be00f2b3dcd20e4e73a427f51/raw/af52b28910b6b93b5e49960ba3af255fbe57641d/schema.txt Bug: 806292 Change-Id: I6a53fb06d411a8d7c93cd4cdcb70cf271ec1000c Reviewed-on: https://chromium-review.googlesource.com/933221 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Katie Thomas <katthomas@google.com> [modify] https://crrev.com/586804aac3fc3c0f0617edea9c5948ec092bf693/common/bq/eventupload.go [modify] https://crrev.com/586804aac3fc3c0f0617edea9c5948ec092bf693/common/bq/testdata/testmessage.pb.go [modify] https://crrev.com/586804aac3fc3c0f0617edea9c5948ec092bf693/common/bq/testdata/testmessage.proto [modify] https://crrev.com/586804aac3fc3c0f0617edea9c5948ec092bf693/tools/cmd/bqschemaupdater/dry_run_table_store.go
,
Feb 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/934d5f0d282a9be41f0ef85bc3d5dda31cd58eb0 commit 934d5f0d282a9be41f0ef85bc3d5dda31cd58eb0 Author: Nodir Turakulov <nodir@google.com> Date: Fri Feb 23 21:43:12 2018 [bqschemaupdater] add confirmation and diff When creating a new table, print schema. When updating an existing table, print unified diff of the schema. In both cases, ask for a user confirmation before making actual changes. Use fmt.Print* functions directly instead of log.Print* functions. This change makes -dry-run flag much less useful. It will be deleted in a separate CL. Bug: 806292 Change-Id: I67c8f9655ad6ee55b2adb36ac3e4c60c6facba0a Reviewed-on: https://chromium-review.googlesource.com/933302 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Katie Thomas <katthomas@google.com> [modify] https://crrev.com/934d5f0d282a9be41f0ef85bc3d5dda31cd58eb0/tools/cmd/bqschemaupdater/dry_run_table_store.go [modify] https://crrev.com/934d5f0d282a9be41f0ef85bc3d5dda31cd58eb0/tools/cmd/bqschemaupdater/main.go [modify] https://crrev.com/934d5f0d282a9be41f0ef85bc3d5dda31cd58eb0/tools/cmd/bqschemaupdater/main_test.go [modify] https://crrev.com/934d5f0d282a9be41f0ef85bc3d5dda31cd58eb0/tools/cmd/bqschemaupdater/schema.go
,
Feb 23 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by no...@chromium.org
, Jan 26 2018