[Feature](partition) Support new partition recycle mechanism #57013

zclllyybb · 2025-10-15T13:52:16Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

now support:

create table auto_recycle(
    k0 datetime(6) not null
)
auto partition by range (date_trunc(k0, 'day')) ()
DISTRIBUTED BY HASH(`k0`) BUCKETS 1
properties(
    "replication_num" = "1",
    "partition.retention_count" = "3"
);

which means only keep the latest 3 partition in history partitions.

ccr will be tested in selectdb/ccr-syncer#646

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes. [Doc] Describe new partition lifecycle behaviours doris-website#2992

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-10-15T13:52:22Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zclllyybb · 2025-10-15T13:52:32Z

run buildall

doris-robot · 2025-10-15T14:30:16Z

TPC-DS: Total hot run time: 190623 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3261f0649b07b2ee820ba5442aff1a4fc65ab3c2, data reload: false

query1	1107	453	415	415
query2	6584	1668	1642	1642
query3	6754	220	224	220
query4	26350	23672	23301	23301
query5	5869	657	493	493
query6	350	249	212	212
query7	4661	498	311	311
query8	321	270	256	256
query9	8728	2622	2591	2591
query10	571	374	317	317
query11	16073	15415	14955	14955
query12	187	121	120	120
query13	1695	579	446	446
query14	12630	9198	9200	9198
query15	259	195	187	187
query16	7835	680	503	503
query17	1622	898	747	747
query18	2117	473	340	340
query19	364	200	179	179
query20	143	140	146	140
query21	231	151	117	117
query22	5089	4891	4673	4673
query23	34825	33956	34196	33956
query24	8203	2500	2572	2500
query25	583	557	460	460
query26	1253	298	166	166
query27	2876	514	364	364
query28	4415	2275	2238	2238
query29	854	615	515	515
query30	313	241	220	220
query31	939	848	794	794
query32	87	84	79	79
query33	627	383	340	340
query34	829	942	514	514
query35	835	861	807	807
query36	1046	1046	928	928
query37	134	113	95	95
query38	3603	3806	3493	3493
query39	1479	1430	1405	1405
query40	221	133	117	117
query41	67	57	102	57
query42	125	114	104	104
query43	474	517	474	474
query44	1329	833	825	825
query45	188	179	175	175
query46	843	1030	639	639
query47	1810	1876	1770	1770
query48	402	429	320	320
query49	775	511	414	414
query50	640	681	403	403
query51	3988	3870	3969	3870
query52	110	108	109	108
query53	255	273	197	197
query54	620	583	525	525
query55	95	86	85	85
query56	377	307	304	304
query57	1204	1202	1143	1143
query58	292	281	279	279
query59	2610	2718	2558	2558
query60	347	334	329	329
query61	150	156	159	156
query62	807	752	686	686
query63	232	192	195	192
query64	4428	1149	821	821
query65	4037	3959	3951	3951
query66	1027	431	331	331
query67	15533	15433	15409	15409
query68	6877	937	593	593
query69	493	322	295	295
query70	1377	1299	1351	1299
query71	509	356	322	322
query72	5927	4968	4880	4880
query73	576	592	364	364
query74	8882	8943	8636	8636
query75	4296	3298	2721	2721
query76	4716	1152	740	740
query77	938	390	311	311
query78	9696	10027	8938	8938
query79	2219	818	583	583
query80	688	557	509	509
query81	485	263	230	230
query82	438	163	134	134
query83	297	270	241	241
query84	308	114	93	93
query85	892	474	423	423
query86	353	323	306	306
query87	3788	3718	3633	3633
query88	3350	2296	2223	2223
query89	398	323	303	303
query90	2036	222	215	215
query91	163	161	133	133
query92	79	74	66	66
query93	1155	989	646	646
query94	685	445	342	342
query95	388	321	322	321
query96	491	582	279	279
query97	2996	2972	2863	2863
query98	243	222	213	213
query99	1475	1415	1289	1289
Total cold run time: 282641 ms
Total hot run time: 190623 ms

doris-robot · 2025-10-15T14:35:46Z

ClickBench: Total hot run time: 30.64 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3261f0649b07b2ee820ba5442aff1a4fc65ab3c2, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.06
query3	0.26	0.08	0.08
query4	1.60	0.11	0.12
query5	0.28	0.27	0.25
query6	1.17	0.65	0.64
query7	0.03	0.03	0.02
query8	0.06	0.04	0.05
query9	0.61	0.54	0.52
query10	0.60	0.57	0.58
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.63	0.61	0.61
query14	1.03	1.03	1.03
query15	0.87	0.87	0.85
query16	0.40	0.39	0.40
query17	1.04	1.04	1.04
query18	0.22	0.20	0.21
query19	1.97	1.85	1.87
query20	0.02	0.02	0.01
query21	15.48	0.92	0.57
query22	0.75	1.08	0.61
query23	15.09	1.39	0.61
query24	6.45	1.43	1.16
query25	0.48	0.08	0.20
query26	0.68	0.15	0.13
query27	0.07	0.06	0.06
query28	9.53	1.41	0.93
query29	12.57	4.08	3.26
query30	0.28	0.14	0.12
query31	2.84	0.60	0.39
query32	3.24	0.57	0.48
query33	3.00	3.14	3.12
query34	16.17	5.41	4.83
query35	4.90	4.87	4.88
query36	0.71	0.50	0.51
query37	0.10	0.07	0.08
query38	0.08	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 104.07 s
Total hot run time: 30.64 s

zclllyybb · 2025-10-15T15:11:45Z

run buildall

doris-robot · 2025-10-15T15:54:02Z

TPC-DS: Total hot run time: 189945 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a6ce988ffb0e319d45b8b7f55959bed78ab9742, data reload: false

query1	1048	432	395	395
query2	6550	1790	1717	1717
query3	6758	243	225	225
query4	26148	23771	22932	22932
query5	5136	657	503	503
query6	347	257	249	249
query7	4667	493	308	308
query8	331	273	272	272
query9	8700	2590	2582	2582
query10	549	373	286	286
query11	15377	15020	14724	14724
query12	201	122	115	115
query13	1685	562	450	450
query14	11542	9301	9351	9301
query15	251	189	173	173
query16	8391	690	569	569
query17	1746	827	642	642
query18	2124	460	385	385
query19	278	226	181	181
query20	141	164	132	132
query21	263	144	123	123
query22	4637	4851	4726	4726
query23	34892	33531	34096	33531
query24	8899	2445	2481	2445
query25	581	534	472	472
query26	1225	288	169	169
query27	2821	520	383	383
query28	4335	2327	2199	2199
query29	777	647	514	514
query30	316	253	220	220
query31	970	879	786	786
query32	89	79	70	70
query33	597	374	359	359
query34	802	846	534	534
query35	804	835	756	756
query36	964	1009	901	901
query37	122	112	83	83
query38	3534	3571	3496	3496
query39	1474	1427	1403	1403
query40	229	124	113	113
query41	64	58	59	58
query42	126	111	108	108
query43	484	505	478	478
query44	1340	842	832	832
query45	189	188	173	173
query46	848	994	628	628
query47	1772	1808	1727	1727
query48	411	426	306	306
query49	751	507	421	421
query50	648	689	413	413
query51	3916	3987	3878	3878
query52	108	118	101	101
query53	237	267	202	202
query54	602	600	530	530
query55	90	86	85	85
query56	346	319	309	309
query57	1177	1175	1134	1134
query58	297	277	287	277
query59	2581	2650	2489	2489
query60	353	361	355	355
query61	188	187	217	187
query62	798	731	694	694
query63	235	203	215	203
query64	4391	1167	848	848
query65	4034	3962	3936	3936
query66	1068	432	332	332
query67	15422	15038	15108	15038
query68	8035	953	617	617
query69	485	329	284	284
query70	1379	1314	1309	1309
query71	502	334	317	317
query72	6064	4904	4776	4776
query73	708	573	360	360
query74	8813	9008	9006	9006
query75	3966	3472	2887	2887
query76	3733	1180	738	738
query77	814	423	332	332
query78	9519	9798	8968	8968
query79	2434	857	590	590
query80	694	595	515	515
query81	495	269	226	226
query82	449	163	131	131
query83	292	275	251	251
query84	312	112	102	102
query85	865	465	436	436
query86	341	309	286	286
query87	3812	3812	3652	3652
query88	2894	2285	2282	2282
query89	412	327	303	303
query90	2010	218	220	218
query91	182	161	140	140
query92	80	72	64	64
query93	1132	980	656	656
query94	696	443	332	332
query95	402	316	318	316
query96	495	578	289	289
query97	2961	2973	2863	2863
query98	240	222	224	222
query99	1500	1412	1311	1311
Total cold run time: 279493 ms
Total hot run time: 189945 ms

doris-robot · 2025-10-15T15:59:33Z

ClickBench: Total hot run time: 30.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4a6ce988ffb0e319d45b8b7f55959bed78ab9742, data reload: false

query1	0.05	0.04	0.05
query2	0.09	0.05	0.06
query3	0.26	0.09	0.09
query4	1.60	0.11	0.11
query5	0.27	0.27	0.25
query6	1.18	0.65	0.64
query7	0.03	0.03	0.03
query8	0.05	0.07	0.04
query9	0.63	0.53	0.51
query10	0.58	0.58	0.57
query11	0.17	0.11	0.12
query12	0.16	0.12	0.12
query13	0.64	0.62	0.62
query14	1.02	1.04	1.03
query15	0.86	0.89	0.86
query16	0.41	0.41	0.40
query17	1.05	1.04	1.06
query18	0.22	0.20	0.20
query19	1.90	1.83	1.81
query20	0.02	0.02	0.01
query21	15.45	0.94	0.61
query22	0.77	1.20	0.78
query23	14.83	1.40	0.62
query24	7.16	2.84	0.41
query25	0.30	0.13	0.07
query26	0.44	0.17	0.13
query27	0.08	0.04	0.05
query28	9.31	1.35	0.93
query29	12.71	3.94	3.28
query30	0.30	0.14	0.11
query31	2.84	0.59	0.39
query32	3.26	0.55	0.48
query33	3.14	3.07	3.20
query34	16.12	5.45	4.85
query35	4.89	4.91	4.94
query36	0.69	0.54	0.50
query37	0.11	0.08	0.07
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.14	0.14
query41	0.09	0.03	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 104.06 s
Total hot run time: 30.08 s

hello-stephen · 2025-10-15T16:11:55Z

FE UT Coverage Report

Increment line coverage 18.45% (19/103) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-10-15T18:19:49Z

FE Regression Coverage Report

Increment line coverage 70.87% (73/103) 🎉
Increment coverage report
Complete coverage report

fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java

dataroaring · 2025-10-16T06:38:32Z

what about using retention_count, e.g. "partition.retention_count" = "3"?

regression-test/suites/partition_p0/auto_partition/test_auto_new_recycle.groovy

zclllyybb · 2025-10-16T10:20:42Z

run buildall

doris-robot · 2025-10-16T10:52:18Z

TPC-DS: Total hot run time: 190908 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a98480b2dc299fe5aa3809adf79db5a59d2aae31, data reload: false

query1	1065	441	408	408
query2	6572	1725	1709	1709
query3	6752	223	222	222
query4	26387	23807	23492	23492
query5	5268	650	493	493
query6	333	247	227	227
query7	4656	496	299	299
query8	314	270	247	247
query9	8726	2579	2572	2572
query10	524	367	311	311
query11	15302	15274	14886	14886
query12	198	134	114	114
query13	1696	554	437	437
query14	11431	9340	9368	9340
query15	201	193	184	184
query16	7672	679	528	528
query17	1309	753	615	615
query18	2175	454	377	377
query19	215	224	221	221
query20	141	155	145	145
query21	267	134	123	123
query22	4910	4762	4526	4526
query23	34920	33881	33792	33792
query24	8552	2493	2604	2493
query25	641	601	506	506
query26	1351	303	196	196
query27	2736	521	383	383
query28	4556	2251	2242	2242
query29	856	684	536	536
query30	332	250	212	212
query31	1006	892	794	794
query32	87	80	67	67
query33	614	401	367	367
query34	843	928	557	557
query35	880	855	776	776
query36	974	1127	951	951
query37	132	127	86	86
query38	3760	3692	3631	3631
query39	1539	1402	1397	1397
query40	221	128	118	118
query41	60	61	56	56
query42	123	114	114	114
query43	494	507	481	481
query44	1396	827	822	822
query45	186	198	177	177
query46	844	998	633	633
query47	1788	1776	1769	1769
query48	392	422	331	331
query49	791	490	409	409
query50	668	689	417	417
query51	3971	3954	3863	3863
query52	112	108	97	97
query53	237	270	200	200
query54	628	592	539	539
query55	94	86	85	85
query56	309	316	321	316
query57	1206	1210	1110	1110
query58	294	275	284	275
query59	2616	2750	2580	2580
query60	364	340	327	327
query61	161	155	156	155
query62	820	728	666	666
query63	235	202	200	200
query64	4393	1178	894	894
query65	4150	3973	3973	3973
query66	1035	435	325	325
query67	15445	15169	15216	15169
query68	9388	955	593	593
query69	518	328	288	288
query70	1353	1275	1317	1275
query71	507	340	336	336
query72	5823	4947	4868	4868
query73	676	572	362	362
query74	9044	9233	8625	8625
query75	4560	3330	2822	2822
query76	3775	1157	746	746
query77	985	403	333	333
query78	9634	9816	8899	8899
query79	1948	835	607	607
query80	718	575	500	500
query81	490	259	300	259
query82	455	167	135	135
query83	303	270	251	251
query84	298	111	111	111
query85	900	497	434	434
query86	335	318	314	314
query87	3736	3776	3633	3633
query88	2932	2275	2245	2245
query89	419	334	290	290
query90	2046	224	231	224
query91	169	261	136	136
query92	85	70	67	67
query93	1300	1002	655	655
query94	699	464	335	335
query95	400	331	327	327
query96	503	578	279	279
query97	2987	2940	2884	2884
query98	236	214	213	213
query99	1433	1384	1299	1299
Total cold run time: 281581 ms
Total hot run time: 190908 ms

doris-robot · 2025-10-16T10:57:48Z

ClickBench: Total hot run time: 30.38 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a98480b2dc299fe5aa3809adf79db5a59d2aae31, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.06	0.06
query3	0.26	0.08	0.08
query4	1.61	0.12	0.12
query5	0.28	0.26	0.25
query6	1.17	0.66	0.65
query7	0.04	0.03	0.03
query8	0.06	0.05	0.04
query9	0.66	0.53	0.52
query10	0.59	0.58	0.58
query11	0.16	0.12	0.12
query12	0.16	0.12	0.13
query13	0.63	0.62	0.60
query14	1.03	1.03	1.04
query15	0.88	0.86	0.86
query16	0.41	0.40	0.40
query17	1.05	1.06	1.06
query18	0.22	0.21	0.20
query19	1.96	1.84	1.87
query20	0.02	0.01	0.01
query21	15.42	0.94	0.57
query22	0.75	1.06	0.64
query23	15.08	1.44	0.63
query24	6.98	1.52	0.69
query25	0.48	0.13	0.19
query26	0.67	0.16	0.13
query27	0.08	0.05	0.05
query28	9.61	1.43	0.96
query29	12.54	3.90	3.27
query30	0.28	0.14	0.14
query31	2.83	0.61	0.39
query32	3.24	0.55	0.48
query33	3.12	3.06	3.05
query34	16.20	5.48	4.84
query35	4.90	4.90	4.98
query36	0.70	0.52	0.54
query37	0.11	0.07	0.06
query38	0.07	0.05	0.05
query39	0.04	0.03	0.03
query40	0.19	0.16	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.81 s
Total hot run time: 30.38 s

hello-stephen · 2025-10-16T11:27:10Z

FE UT Coverage Report

Increment line coverage 18.27% (19/104) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-10-16T13:18:29Z

FE Regression Coverage Report

Increment line coverage 70.19% (73/104) 🎉
Increment coverage report
Complete coverage report

regression-test/suites/partition_p0/auto_partition/test_auto_new_recycle.groovy

zclllyybb · 2025-10-22T03:25:05Z

run buildall

doc of apache/doris#57060, apache/doris#57013 and fix of #2970 ## Versions - [x] dev - [x] 3.x - [ ] 2.1 - [ ] 2.0 ## Languages - [x] Chinese - [x] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built

github-actions · 2025-10-22T04:01:37Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-10-22T04:01:40Z

PR approved by anyone and no changes requested.

doris-robot · 2025-10-22T05:01:19Z

TPC-DS: Total hot run time: 190683 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f3001c37d8979855a73307b46bb28561e4cb6076, data reload: false

query1	1103	442	409	409
query2	6569	1698	1705	1698
query3	6752	221	226	221
query4	26508	23290	23856	23290
query5	5855	623	471	471
query6	331	262	223	223
query7	4652	490	294	294
query8	308	264	253	253
query9	8695	2586	2582	2582
query10	563	356	304	304
query11	16179	15101	14906	14906
query12	185	123	117	117
query13	1689	577	452	452
query14	12152	9486	9660	9486
query15	231	197	174	174
query16	7717	675	538	538
query17	1623	915	673	673
query18	2374	473	375	375
query19	282	239	193	193
query20	147	134	141	134
query21	229	161	137	137
query22	4695	4595	4659	4595
query23	34748	33735	34047	33735
query24	8496	2549	2582	2549
query25	579	560	514	514
query26	1488	289	174	174
query27	2799	522	395	395
query28	4617	2263	2206	2206
query29	790	633	516	516
query30	387	233	209	209
query31	1010	971	758	758
query32	91	76	70	70
query33	572	422	345	345
query34	844	883	535	535
query35	827	873	792	792
query36	924	1012	865	865
query37	137	106	87	87
query38	3543	3509	3476	3476
query39	1474	1414	1445	1414
query40	226	128	118	118
query41	62	61	58	58
query42	123	114	115	114
query43	490	502	469	469
query44	1264	745	731	731
query45	180	182	172	172
query46	896	983	653	653
query47	1755	1791	1692	1692
query48	398	429	331	331
query49	747	515	459	459
query50	677	682	414	414
query51	3914	3928	3857	3857
query52	112	117	103	103
query53	263	285	208	208
query54	606	615	534	534
query55	89	92	87	87
query56	335	339	313	313
query57	1175	1182	1118	1118
query58	292	280	275	275
query59	2509	2729	2494	2494
query60	352	345	346	345
query61	161	160	149	149
query62	820	706	660	660
query63	233	208	203	203
query64	3634	1161	888	888
query65	4053	3952	3966	3952
query66	844	491	333	333
query67	15539	15159	15048	15048
query68	9695	889	602	602
query69	530	330	284	284
query70	1367	1287	1239	1239
query71	530	357	331	331
query72	5805	4860	4844	4844
query73	682	601	361	361
query74	9099	9169	8979	8979
query75	4658	3368	2792	2792
query76	4817	1144	763	763
query77	1018	407	317	317
query78	9738	9752	8912	8912
query79	1616	840	635	635
query80	688	575	517	517
query81	511	263	235	235
query82	337	162	137	137
query83	306	276	261	261
query84	304	117	95	95
query85	867	468	419	419
query86	341	326	305	305
query87	3771	3732	3718	3718
query88	2946	2280	2270	2270
query89	402	324	308	308
query90	2018	306	224	224
query91	174	173	137	137
query92	87	68	65	65
query93	1145	997	656	656
query94	700	432	344	344
query95	395	317	327	317
query96	511	581	291	291
query97	2968	3008	2906	2906
query98	231	214	212	212
query99	1407	1417	1264	1264
Total cold run time: 283578 ms
Total hot run time: 190683 ms

doris-robot · 2025-10-22T05:06:30Z

ClickBench: Total hot run time: 28.49 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f3001c37d8979855a73307b46bb28561e4cb6076, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.05
query3	0.26	0.09	0.08
query4	1.60	0.12	0.12
query5	0.27	0.26	0.26
query6	1.16	0.67	0.65
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.63	0.54	0.51
query10	0.58	0.58	0.58
query11	0.18	0.12	0.12
query12	0.17	0.12	0.12
query13	0.65	0.61	0.61
query14	1.04	1.05	1.05
query15	0.88	0.86	0.87
query16	0.40	0.40	0.40
query17	1.04	1.09	1.03
query18	0.22	0.20	0.21
query19	2.01	1.82	1.79
query20	0.02	0.02	0.01
query21	15.42	0.18	0.12
query22	5.10	0.06	0.06
query23	15.65	0.26	0.11
query24	3.01	0.71	0.40
query25	0.08	0.05	0.07
query26	0.15	0.12	0.14
query27	0.07	0.06	0.05
query28	3.82	1.13	0.93
query29	12.56	3.93	3.30
query30	0.28	0.14	0.11
query31	2.83	0.60	0.39
query32	3.24	0.56	0.49
query33	3.03	3.20	3.09
query34	16.18	5.40	4.88
query35	4.90	4.94	4.95
query36	0.70	0.50	0.50
query37	0.10	0.08	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.16	0.15
query41	0.08	0.04	0.04
query42	0.04	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 98.89 s
Total hot run time: 28.49 s

fe/fe-core/src/main/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandler.java

fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java

yujun777

LGTM

dataroaring

LGTM

now support: ```sql create table auto_recycle( k0 datetime(6) not null ) auto partition by range (date_trunc(k0, 'day')) () DISTRIBUTED BY HASH(`k0`) BUCKETS 1 properties( "replication_num" = "1", "partition.retention_count" = "3" ); ``` which means only keep the latest 3 partition in history partitions. ccr will be tested in selectdb/ccr-syncer#646 ### Release note Resolve the problem of different name format when need to recycle auto partition

…57013) now support: ```sql create table auto_recycle( k0 datetime(6) not null ) auto partition by range (date_trunc(k0, 'day')) () DISTRIBUTED BY HASH(`k0`) BUCKETS 1 properties( "replication_num" = "1", "partition.retention_count" = "3" ); ``` which means only keep the latest 3 partition in history partitions. ccr will be tested in selectdb/ccr-syncer#646 ### Release note Resolve the problem of different name format when need to recycle auto partition

…ism #57013 (#57939) Cherry-picked from #57013 Co-authored-by: zclllyybb <[email protected]>

zclllyybb requested review from CalvinKirs, dataroaring, gavinchou, morningman and w41ter as code owners October 15, 2025 13:52

zclllyybb force-pushed the new_partition branch from 3261f06 to 4a6ce98 Compare October 15, 2025 15:11

dataroaring reviewed Oct 16, 2025

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java Outdated Show resolved Hide resolved

dataroaring added the dev/4.0.x label Oct 16, 2025

dataroaring reviewed Oct 16, 2025

View reviewed changes

regression-test/suites/partition_p0/auto_partition/test_auto_new_recycle.groovy Show resolved Hide resolved

zclllyybb mentioned this pull request Oct 20, 2025

[Doc] Describe new partition lifecycle behaviours apache/doris-website#2992

Merged

8 tasks

dataroaring reviewed Oct 21, 2025

View reviewed changes

regression-test/suites/partition_p0/auto_partition/test_auto_new_recycle.groovy Show resolved Hide resolved

zclllyybb added 4 commits October 21, 2025 16:13

base_code

03c5b0d

fix create table fail

35939f1

fix

41b0f74

change property meaning and add backup restore case

0e50a50

fix syncer regression util

f3001c3

zclllyybb force-pushed the new_partition branch 2 times, most recently from 84fd8f6 to f3001c3 Compare October 22, 2025 03:22

w41ter approved these changes Oct 22, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 22, 2025

github-actions bot added the reviewed label Oct 22, 2025

zclllyybb mentioned this pull request Oct 22, 2025

Add test of table property partition.retention_count selectdb/ccr-syncer#646

Merged

zclllyybb requested review from HappenLee and dataroaring October 23, 2025 18:04

deardeng reviewed Nov 11, 2025

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandler.java Show resolved Hide resolved

yujun777 reviewed Nov 12, 2025

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java Show resolved Hide resolved

yujun777 approved these changes Nov 12, 2025

View reviewed changes

dataroaring approved these changes Nov 12, 2025

View reviewed changes

zclllyybb merged commit 4058843 into apache:master Nov 12, 2025
42 of 47 checks passed

zclllyybb deleted the new_partition branch November 12, 2025 04:04

github-actions bot mentioned this pull request Nov 12, 2025

branch-4.0: [Feature](partition) Support new partition recycle mechanism #57013 #57939

Merged

yiguolei pushed a commit that referenced this pull request Nov 13, 2025

branch-4.0: [Feature](partition) Support new partition recycle mechan…

8d40c7a

…ism #57013 (#57939) Cherry-picked from #57013 Co-authored-by: zclllyybb <[email protected]>

yiguolei added dev/4.0.2-merged and removed dev/4.0.x labels Nov 13, 2025

[Feature](partition) Support new partition recycle mechanism #57013

[Feature](partition) Support new partition recycle mechanism #57013

Uh oh!

Conversation

zclllyybb commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Oct 15, 2025

Uh oh!

zclllyybb commented Oct 15, 2025

Uh oh!

doris-robot commented Oct 15, 2025

Uh oh!

doris-robot commented Oct 15, 2025

Uh oh!

zclllyybb commented Oct 15, 2025

Uh oh!

doris-robot commented Oct 15, 2025

Uh oh!

doris-robot commented Oct 15, 2025

Uh oh!

hello-stephen commented Oct 15, 2025

FE UT Coverage Report

Uh oh!

hello-stephen commented Oct 15, 2025

FE Regression Coverage Report

Uh oh!

Uh oh!

dataroaring commented Oct 16, 2025

Uh oh!

Uh oh!

zclllyybb commented Oct 16, 2025

Uh oh!

doris-robot commented Oct 16, 2025

Uh oh!

doris-robot commented Oct 16, 2025

Uh oh!

hello-stephen commented Oct 16, 2025

FE UT Coverage Report

Uh oh!

hello-stephen commented Oct 16, 2025

FE Regression Coverage Report

Uh oh!

Uh oh!

zclllyybb commented Oct 22, 2025

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

doris-robot commented Oct 22, 2025

Uh oh!

doris-robot commented Oct 22, 2025

Uh oh!

Uh oh!

Uh oh!

yujun777 left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

zclllyybb commented Oct 15, 2025 •

edited

Loading