FT-701 keep all FT files consistency before next checkpoint #323
Conversation
[summary] When we do hot backup like below: 1) take checkpoint lock 2) lock table for read 3) wirte binary log postion 4) copy redo log to backup dir 5) unlock tables 6) start to copy datas files 6.1) write data by another client 6.2) copy file1 to backup dir 7) release checkpoint lock Between 6.1) and 6.2) the FT may does ft_close and update FT header to new, the new writes are visible in the backup instance, this broke the consistency. It would be better for TokuDB to update FT header only in checkpoint, keep all FT files consistency. Copyright (c) 2015, BohuTANG All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
This is a very nice idea. Thank you for the contribution. I need a little time to analyze and review the idea before merging. |
|
Clever. Assuming that works, maybe we can avoid cloning nodes during Bradley C Kuszmaul - via snartphone
|
|
I guess I don't see how to use this idea to reduce checkpoint variability. It is still a clever idea that seems like it can make traditional backup One difference that does show up today (since we haven't implemented it) is Bradley C Kuszmaul - via snartphone
|
|
Hi, On Wednesday, September 16, 2015, Bradley C. Kuszmaul <
BohuTANG "The great artist is the simplifier."--- Vincent Van Gogh |
|
I'd like to better understand what toku hot backup cannot do. Doesn't it Bradley C Kuszmaul - via snartphone
|
|
If the master instance is very busy, how TokuDB hotbackup create On Wednesday, September 16, 2015, Bradley C. Kuszmaul <
BohuTANG "The great artist is the simplifier."--- Vincent Van Gogh |
|
Hotbackup can backup multiple directories. If one backs up the data dirs
|
|
Rich, On Wednesday, September 16, 2015, Rik Prohaska [email protected]
BohuTANG "The great artist is the simplifier."--- Vincent Van Gogh |
|
Why? When Tokutek hot backup finishes, one has a consistent snapshot of the On Wednesday, September 16, 2015, Rik Prohaska [email protected]
BohuTANG "The great artist is the simplifier."--- Vincent Van Gogh — |
|
Looks good to me, I will have a try. On Thursday, September 17, 2015, Rik Prohaska [email protected]
BohuTANG "The great artist is the simplifier."--- Vincent Van Gogh |
|
This patch has some bugs when table create and close less than a checkpoint interval, will fixed. |
|
Moved to #331 |
4 participants
[summary]
When we do hot backup like below:
6.1) write data by another client
6.2) copy file1 to backup dir
Between 6.1) and 6.2) the FT may does ft_close and update FT header to new, the new writes are visible in the backup instance, this broke the consistency.
It would be better for TokuDB to update FT header only in checkpoint, keep all FT files consistency.
Copyright (c) 2015, BohuTANG
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.