webpack-dev-server crashes with "Cannot remove headers after they are sent to the client" #2150
Description
Bug report
My company (Canva) has an extremely large web app that we build with Webpack. When we turn on experiments.lazyCompilation with devServer.hot: true during "initial" page load the dev server will sometimes crash with the following error:
Error: Cannot remove headers after they are sent to the client
at ServerResponse.removeHeader (node:_http_outgoing:804:11)
at removeResponseHeader (/<omitted>/node_modules/.pnpm/[email protected][email protected]/node_modules/webpack-dev-middleware/dist/utils/compatibleAPI.js:131:7)
at sendError (/<omitted>/node_modules/.pnpm/[email protected][email protected]/node_modules/webpack-dev-middleware/dist/middleware.js:175:9)
at errorHandler (/<omitted>/node_modules/.pnpm/[email protected][email protected]/node_modules/webpack-dev-middleware/dist/middleware.js:218:11)
at Volume.ReadStream.<anonymous> (/<omitted>/node_modules/.pnpm/[email protected][email protected]/node_modules/webpack-dev-middleware/dist/middleware.js:662:9)
at Volume.ReadStream.emit (node:events:518:28)
at Volume.ReadStream.emit (node:domain:489:12)
at onread (/<omitted>/node_modules/.pnpm/[email protected]/node_modules/memfs/src/volume.ts:2366:12)
at Immediate.<anonymous> (/<omitted>/node_modules/.pnpm/[email protected]/node_modules/memfs/src/volume.ts:902:9)
at processImmediate (node:internal/timers:483:21) {
code: 'ERR_HTTP_HEADERS_SENT'
}
Expected Behavior
The dev server not to crash. Naively this can be by wrapping this function call in a res.headersSent check but I'm not sure whether that is logically correct.
How Do We Reproduce?
TBC
npx webpack-cli info
System:
OS: Linux 6.8 Ubuntu 22.04.5 LTS 22.04.5 LTS (Jammy Jellyfish)
CPU: (16) x64 Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
Memory: 92.18 GB / 123.81 GB
Binaries:
Node: 20.19.4 - /usr/bin/node
npm: 10.8.2 - /usr/bin/npm
Packages:
babel-loader: ^8.0.6 => 8.2.4
css-loader: ^5.2.4 => 5.2.6
ejs-loader: ^0.5.0 => 0.5.0
file-loader: ^5.0.2 => 5.0.2
html-webpack-plugin: ~5.5.0 => 5.5.3
loader-runner: ^4.1.0 => 4.2.0
loader-utils: 3.3.1 => 3.3.1
postcss-loader: ^5.2.0 => 5.2.0
raw-loader: ^4.0.0 => 4.0.2
svg-url-loader: ^3.0.3 => 3.0.3
svgo-loader: ^3.0.0 => 3.0.0
swc-loader: 0.2.6 => 0.2.6
terser-webpack-plugin: ^5.2.4 => 5.3.9
thread-loader: ^4.0.2 => 4.0.2
ts-loader: ^9.4.4 => 9.4.4
url-loader: ^4.1.1 => 4.1.1
webfontloader: ^1.6.28 => 1.6.28
webpack: 5.99.9 => 5.99.9
webpack-bundle-analyzer: 4.10.2 => 4.10.2
webpack-cli: ^5.1.4 => 5.1.4
webpack-dev-server: 5.2.2 => 5.2.2
webpackbar: ^5.0.0-3 => 5.0.0-3