Skip to content

Conversation

a2659802
Copy link
Contributor

@a2659802 a2659802 commented Aug 16, 2025

PR Details

Description

Added WriteToBufferCustomZIP method, which allows using a custom ZIP library (e.g., github.com/klauspost/compress/zip) for better performance while keeping the original WriteToBuffer method backward-compatible.

  • Introduced ZipWriter interface to abstract standard and custom ZIP writers.
  • WriteToBuffer now calls WriteToBufferCustomZIP with the standard library zip.NewWriter by default.
  • writeToZip method updated to accept ZipWriter interface.

Related Issue

#2199

Motivation and Context

To improve in-memory write performance and support high-performance third-party ZIP libraries while maintaining compatibility with the standard library.

How Has This Been Tested

write a benchmark test for testing performance

package excel

import (
	"archive/zip"
	"bytes"
	"fmt"
	"io"
	"testing"

	kzip "github.com/klauspost/compress/zip"

	"github.com/stretchr/testify/require"
	"github.com/xuri/excelize/v2"
)

const (
	columnNum = 10
	columnLen = 100
	rowNum    = 50000
)

var data [rowNum][columnNum]string
var columnName []string
var columnNameAny []any

func init() {
	for r := range rowNum {
		for col := range columnNum {
			data[r][col] = string(bytes.Repeat([]byte{'a'}, columnLen))
		}
	}
	for range columnNum {
		columnName = append(columnName, "name")
		columnNameAny = append(columnNameAny, "name")
	}
}

func libExcelizeStreamImpl(t testing.TB, factory func(io.Writer) excelize.ZipWriter) []byte {
	file := excelize.NewFile()
	defer file.Close()
	sw, err := file.NewStreamWriter("Sheet1")
	require.NoError(t, err)

	style := &excelize.Style{Font: &excelize.Font{
		Bold: true,
	}}
	styleId, err := file.NewStyle(style)
	require.NoError(t, err)
	err = sw.SetRow("A1", columnNameAny, excelize.RowOpts{
		StyleID: styleId,
	})
	require.NoError(t, err)

	for i := range data {
		row := fmt.Sprintf("A%d", i+2)
		err := sw.SetRow(row, []any{data[i][0], data[i][1], data[i][2],
			data[i][3], data[i][4], data[i][5], data[i][6], data[i][7], data[i][8], data[i][9]})
		require.NoError(t, err)
	}

	require.NoError(t, sw.Flush())
	// buffer, err := file.WriteToBuffer()
	buffer, err := file.WriteToBufferCustomZIP(factory)
	require.NoError(t, err)
	return buffer.Bytes()
}

func BenchmarkExcelizeStreamStd(b *testing.B) {
	for b.Loop() {
		_ = libExcelizeStreamImpl(b, func(w io.Writer) excelize.ZipWriter { return zip.NewWriter(w) })
	}
}

func BenchmarkExcelizeStreamKzip(b *testing.B) {
	for b.Loop() {
		_ = libExcelizeStreamImpl(b, func(w io.Writer) excelize.ZipWriter { return kzip.NewWriter(w) })
	}
}
Running tool: D:\soft\Go125\bin\go.exe test -benchmem -run=^$ -bench ^(BenchmarkExcelizeStreamStd|BenchmarkExcelizeStreamKzip)$ github.com/a2659802/go-example/pkg/excel -count=1

goos: windows
goarch: amd64
pkg: github.com/a2659802/go-example/pkg/excel
cpu: 12th Gen Intel(R) Core(TM) i5-12500
=== RUN   BenchmarkExcelizeStreamStd
BenchmarkExcelizeStreamStd
BenchmarkExcelizeStreamStd-12
       2         599544500 ns/op        272403848 B/op   4153804 allocs/op
=== RUN   BenchmarkExcelizeStreamKzip
BenchmarkExcelizeStreamKzip
BenchmarkExcelizeStreamKzip-12
       3         468739967 ns/op        272623032 B/op   4152989 allocs/op
PASS
ok      github.com/a2659802/go-example/pkg/excel        3.077s

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@xuri xuri added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Aug 16, 2025
Copy link
Member

@xuri xuri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR. I've left some comments.

Copy link
Member

@xuri xuri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll made some changes based on your branch.

- Add new ZipWriter data type
- Add new field ZipWriter in the File data type
- Made the CharsetTranscoder function parameter not use the internal data type charsetTranscoderFn
- Update unit tests
Copy link
Member

@xuri xuri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the third-party archive package kzip you mentioned reduce 25% time cost, but the size of generated workbook will be increase over 100%.

@xuri xuri added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 19, 2025
@xuri xuri moved this to Features in Excelize v2.10.0 Aug 19, 2025
@xuri xuri linked an issue Aug 19, 2025 that may be closed by this pull request
1 task
Copy link

codecov bot commented Aug 19, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.24%. Comparing base (b372bd3) to head (d1ca080).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2200   +/-   ##
=======================================
  Coverage   99.24%   99.24%           
=======================================
  Files          32       32           
  Lines       30571    30574    +3     
=======================================
+ Hits        30340    30343    +3     
  Misses        153      153           
  Partials       78       78           
Flag Coverage Δ
unittests 99.24% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@xuri xuri merged commit 845a274 into qax-os:master Aug 19, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
Status: Features
Development

Successfully merging this pull request may close these issues.

Use github.com/klauspost/compress/zip to improve export performance
2 participants