Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ddl: speed up tests #12888

Merged
merged 7 commits into from
Oct 23, 2019
Merged

ddl: speed up tests #12888

merged 7 commits into from
Oct 23, 2019

Conversation

zimulala
Copy link
Contributor

@zimulala zimulala commented Oct 23, 2019

What problem does this PR solve?

Speed up testing time.

What is changed and how it works?

  • TestDropColumn tests drop column, so I will add the required columns when I create the table.

  • Change multiple insert statements to one.

go test  -v -vet=off -timeout 10m  -race -count 1

Before this PR:

Local
ok      github.com/pingcap/tidb/ddl    78.583s

CI
2m 23s(https://internal.pingcap.net/idc-jenkins/blue/organizations/jenkins/tidb_ghpr_unit_test/detail/tidb_ghpr_unit_test/11707/pipeline/75/) 

After this PR:

Local
ok      github.com/pingcap/tidb/ddl    67.245s

CI
1m 38s(https://internal.pingcap.net/idc-jenkins/blue/organizations/jenkins/tidb_ghpr_unit_test/detail/tidb_ghpr_unit_test/11811/pipeline/75)

@zimulala
Copy link
Contributor Author

/run-all-tests

@zimulala
Copy link
Contributor Author

/run-unit-test

@codecov
Copy link

codecov bot commented Oct 23, 2019

Codecov Report

Merging #12888 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #12888   +/-   ##
===========================================
  Coverage   79.9757%   79.9757%           
===========================================
  Files           465        465           
  Lines        107095     107095           
===========================================
  Hits          85650      85650           
  Misses        14989      14989           
  Partials       6456       6456

Copy link
Member

@bb7133 bb7133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about set @@tidb_record_plan_in_slow_log = 0 for more test suite in ddl package?

@zimulala
Copy link
Contributor Author

@bb7133 I think it is enough to change two. Most of the other tests are internal tests, not using MustExec.

@bb7133 bb7133 added status/LGT1 Indicates that a PR has LGTM 1. and removed status/DNM labels Oct 23, 2019
Copy link
Member

@bb7133 bb7133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

for i := start; i < num; i++ {
sql := fmt.Sprintf("insert into test_add_index values (%d, %d, %d)", i, i, i)
tk.MustExec(sql)
for i := start; i < 4090; i += 100 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually 4100 records are inserted, does it matter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start is -10, so I think 4090 is OK.

Copy link
Contributor

@AilinKid AilinKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

multiDDL := make([]string, 0, num)
sql := "create table t2 (c1 int, c2 int, c3 int, "
for i := 4; i < 4+num; i++ {
multiDDL = append(multiDDL, fmt.Sprintf("alter table t2 drop column c%d", i))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that means dropping the column sequentially, maybe more disorderedly.

@zimulala
Copy link
Contributor Author

PTAL @djshow832

Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 23, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Oct 23, 2019

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Oct 23, 2019

@zimulala merge failed.

@zimulala
Copy link
Contributor Author

/run-ntegration-common-test

@zz-jason zz-jason merged commit 9a2aa5e into pingcap:master Oct 23, 2019
@zimulala zimulala deleted the speedup-tests branch October 23, 2019 23:47
lfkdsk added a commit to JustProject/tidb that referenced this pull request Oct 26, 2019
…ect/tidb into feature-add-udf-support

* 'feature-add-udf-support' of https://github.com/JustProject/tidb: (26 commits)
  *: fix bug that the kill command doesn't work when the killed session is waiting for the pessimistic lock (pingcap#12852)
  executor: fix the projection upon the indexLookUp in indexLookUpJoin can't get result. (pingcap#12889)
  planner, executor: support create view on union (pingcap#12595)
  planner/cascades: introduce TransformationID in cascades planner (pingcap#12879)
  executor: fix data race in test (pingcap#12910)
  executor: reuse chunk row for insert on duplicate update (pingcap#12847)
  ddl: speed up tests (pingcap#12888)
  executor: speed up test (pingcap#12896)
  expression: implement vectorized evaluation for `builtinSecondSig` (pingcap#12886)
  expression: implement vectorized evaluation for `builtinJSONObjectSig` (pingcap#12663)
  expression: speed up builtinRepeatSig by using MergeNulls (pingcap#12674)
  expression: speed up unit tests under the expression package (pingcap#12887)
  store,kv: snapshot doesn't cache the non-exists kv entries lead to poor 'insert ignore' performance (pingcap#12872)
  executor: fix data race in `GetDirtyTable()` (pingcap#12767)
  domain: increase TTL to reduce the occurrence of reporting min startTS errors (pingcap#12578)
  executor: split test for speed up (pingcap#12881)
  executor: fix inconsistent of grants privileges with MySQL when executing `grant all on ...` (pingcap#12330)
  expression: implement vectorized evaluation for `builtinJSONUnquoteSig` (pingcap#12841)
  tune grpc connection count between tidb and tikv (pingcap#12884)
  Makefile: change test parallel to 8 (pingcap#12885)
  ...
XiaTianliang pushed a commit to XiaTianliang/tidb that referenced this pull request Dec 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/test status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants