Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

restore: merge tidb-tools/pkg/restore-util #146

Merged
merged 18 commits into from
Jan 22, 2020
Merged

restore: merge tidb-tools/pkg/restore-util #146

merged 18 commits into from
Jan 22, 2020

Commits on Oct 21, 2019

  1. restore-util: Implement split/scatter (pingcap#274) 

    * implement split/scatter

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* init test

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* redesign output/input of the lib

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* update dependency

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* add commments and more tests

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* add ScanRegions interface to Client

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix potential data race

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* Apply suggestions from code review

Co-Authored-By: kennytm <kennytm@gmail.com>

* Update pkg/restore-util/client.go

Co-Authored-By: kennytm <kennytm@gmail.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* update dependency

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* resolve conflicts

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix prefix rewrite

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* add RewriteRule/skip failed scatter region/retry the SplitRegion

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix test

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* check if region has peer

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* more logs

Signed-off-by: 5kbpers <tangminghua@pingcap.com>
    @5kbpers
    5kbpers committed Oct 21, 2019
    Configuration menu
    Copy the full SHA
    564a9ab View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2019

  1. restore-util: add split retry interval (pingcap#277) 

    * reset dependencies to release-3.1

* add split retry interval

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix go.sum

Signed-off-by: 5kbpers <tangminghua@pingcap.com>
    @5kbpers
    5kbpers committed Oct 30, 2019
    Configuration menu
    Copy the full SHA
    ee6e218 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2019

  1. restore-util: wait for scatter region sequentially (pingcap#279) 

    * wait for scatter region sequentially

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>
    @5kbpers
    5kbpers committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    97ba30e View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2019

  1. restore-util: add on split hook (pingcap#281) 

    * restore-util: add on split hook

Signed-off-by: Neil Shen <overvenus@gmail.com>

* Nil check onSplit

Co-Authored-By: kennytm <kennytm@gmail.com>
    @overvenus @kennytm @5kbpers
    2 people authored and 5kbpers committed Nov 6, 2019
    Configuration menu
    Copy the full SHA
    d5b0016 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2019

  1. restore-util: fix returned new region is nil (pingcap#283) 

    * restore-util: fix returned new region is nil

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* more logs

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* *: gofmt

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* Apply suggestions from code review

Co-Authored-By: kennytm <kennytm@gmail.com>

* fix log

Signed-off-by: 5kbpers <tangminghua@pingcap.com>
    @5kbpers
    5kbpers committed Nov 8, 2019
    Configuration menu
    Copy the full SHA
    64e57b3 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2019

  1. restore-util: call onSplit on splitByRewriteRules (pingcap#285) 

    Signed-off-by: Neil Shen <overvenus@gmail.com>
    @overvenus @kennytm
    overvenus authored and kennytm committed Nov 12, 2019
    Configuration menu
    Copy the full SHA
    a32debb View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2019

  1. restore-util: fix overlapped error message (pingcap#293) 

    * restore-util: fix overlapped error message

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix log message

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* reduce error trace

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix test

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>
    @5kbpers @kennytm
    5kbpers authored and kennytm committed Nov 26, 2019
    Configuration menu
    Copy the full SHA
    72ddd22 View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2019

  1. restore-util: log warning when cannot find matched rewrite rule (ping… 

    …cap#299)
    @5kbpers
    5kbpers committed Dec 9, 2019
    Configuration menu
    Copy the full SHA
    17315e7 View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2019

  1. restore-util: add method to set placement rules and store labels (pin… 

    …gcap#301)

* restore-util: add method to set placement rules and store labels

Signed-off-by: disksing <i@disksing.com>

* minor fix

Signed-off-by: disksing <i@disksing.com>

* address comment

Signed-off-by: disksing <i@disksing.com>

* add GetPlacementRules

Signed-off-by: disksing <i@disksing.com>

* fix test

Signed-off-by: disksing <i@disksing.com>
    @disksing @overvenus
    disksing authored and overvenus committed Dec 17, 2019
    Configuration menu
    Copy the full SHA
    9f9465b View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2019

  1. restore-util: support batch split (pingcap#300) 

    * restore-util: support batch split

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* go fmt

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* Apply suggestions from code review

Co-Authored-By: kennytm <kennytm@gmail.com>

* address commits

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* Update pkg/restore-util/split.go

Co-Authored-By: kennytm <kennytm@gmail.com>

* add onSplit callback

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix test

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>
    @5kbpers @overvenus
    5kbpers authored and overvenus committed Dec 18, 2019
    Configuration menu
    Copy the full SHA
    b9ac458 View commit details
    Browse the repository at this point in the history

Commits on Dec 23, 2019

  1. restore-util: add upper bound time for waiting for scatter (pingcap#305) 

    * restore: fix scatter regions failed

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* add log

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* stop waiting for scatter after 3min

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>
    @5kbpers
    5kbpers committed Dec 23, 2019
    Configuration menu
    Copy the full SHA
    d1462ac View commit details
    Browse the repository at this point in the history

Commits on Dec 25, 2019

  1. restore-util: fix wrong url (pingcap#306) 

    Signed-off-by: disksing <i@disksing.com>
    @disksing @overvenus
    disksing authored and overvenus committed Dec 25, 2019
    Configuration menu
    Copy the full SHA
    bd126c4 View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2020

  1. restore-util: add warning about unmatched table id (pingcap#313) 

    * restore-util: support table partition

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix log

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* warn table id does not match

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* add unit tests

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* Apply suggestions from code review

Co-Authored-By: Neil Shen <overvenus@gmail.com>

* fix compile error

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix test

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

Co-authored-by: Ian <ArGregoryIan@gmail.com>
Co-authored-by: Neil Shen <overvenus@gmail.com>
    @5kbpers @IANTHEREAL @overvenus
    3 people committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    e8ab896 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2020

  1. Add 'pkg/restoreutil/' from commit 'e8ab896effc0960a789c5e45eb425b435… 

    …1dabc6f'

git-subtree-dir: pkg/restoreutil
git-subtree-mainline: 0c1dea4
git-subtree-split: e8ab896
    @overvenus
    overvenus committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    5b5f660 View commit details
    Browse the repository at this point in the history

  2. *: prune tidb-tools 

    Signed-off-by: Neil Shen <overvenus@gmail.com>
    @overvenus
    overvenus committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    6087860 View commit details
    Browse the repository at this point in the history

  3. restore: address linters suggestions 

    Signed-off-by: Neil Shen <overvenus@gmail.com>
    @overvenus
    overvenus committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    23aa6f7 View commit details
    Browse the repository at this point in the history

  4. restore: merge restoreutil into restore 

    Signed-off-by: Neil Shen <overvenus@gmail.com>
    @overvenus
    overvenus committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    5f5a239 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2020

  1. address comment 

    Signed-off-by: Neil Shen <overvenus@gmail.com>
    @overvenus
    overvenus committed Jan 19, 2020
    Configuration menu
    Copy the full SHA
    bafcbc1 View commit details
    Browse the repository at this point in the history