View compiler - Temporary file collision issue

[expertsender-marcinsynak]

We've been using Spark since forever for email templating purposes.

Our environment is highly multi-threaded. Sometimes we compile many different views in parallel and use them to generate HTML emails.

Randomly we get an error during view compilation, like this:

System.IO.IOException: The file 'C:\Users\esservice\AppData\Local\Temp\iq30upji.out' already exists.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at System.CodeDom.Compiler.Executor.CreateInheritedFile(String file)
   at System.CodeDom.Compiler.Executor.ExecWaitWithCaptureUnimpersonated(SafeUserTokenHandle userToken, String cmd, String currentDir, TempFileCollection tempFiles, String& outputName, String& errorName, String trueCmdLine)
   at System.CodeDom.Compiler.Executor.ExecWaitWithCapture(SafeUserTokenHandle userToken, String cmd, String currentDir, TempFileCollection tempFiles, String& outputName, String& errorName, String trueCmdLine)
   at Microsoft.CSharp.CSharpCodeGenerator.Compile(CompilerParameters options, String compilerDirectory, String compilerExe, String arguments, String& outputFile, Int32& nativeReturnValue, String trueArgs)
   at Microsoft.CSharp.CSharpCodeGenerator.FromFileBatch(CompilerParameters options, String[] fileNames)
   at Microsoft.CSharp.CSharpCodeGenerator.FromSourceBatch(CompilerParameters options, String[] sources)
   at Microsoft.CSharp.CSharpCodeGenerator.System.CodeDom.Compiler.ICodeCompiler.CompileAssemblyFromSourceBatch(CompilerParameters options, String[] sources)
   at Spark.Compiler.BatchCompiler.Compile(Boolean debug, String languageOrExtension, String[] sourceCode) in C:\Users\marcin.synak.3XR\Downloads\Biblioteki\Spark\Spark-1.7.40017.0-source\src\Spark\Compiler\BatchCompiler.cs:line 126
   at Spark.Compiler.CSharp.CSharpViewCompiler.CompileView(IEnumerable`1 viewTemplates, IEnumerable`1 allResources) in C:\Users\marcin.synak.3XR\Downloads\Biblioteki\Spark\Spark-1.7.40017.0-source\src\Spark\Compiler\CSharp\CSharpViewCompiler.cs:line 31

Yes, I know this is a prehistoric 1.7 version :) Compiled manually, as well.

At first, I thought that tons of old files in my temp dir may be responsible for filename collisions. But specified file was not there. So I can only assume that two threads tried to do a view compilation at the same time and chose the same name for temp file. One thread did its job, the other failed.

My questions are:

1. Is there any way to influence the way temporary *.out files are named? I could implement my own name generator that would be multithread-proof.
2. Is there a way to specify different temp dir for Spark compiler? Using settings or otherwise? Separating these files from the rest of the buzz could be helpful.
3. Could an upgrade to the latest Spark version help with any of this?

Many thanks for keeping this project alive. I can see that migrating to .NET Core is now an option, too. Wow :)

[RobertTheGrey]

Any thoughts on this @bounav from your recent experimentation on multi-threading?

[bounav]

Interesting use-case.

Is there any way to influence the way temporary *.out files are named? I could implement my own name generator that would be multithread-proof

We'd have to investigate how the file names are generated. It would be useful to have a way of controlling what the filenames are in an extensible way.

Is there a way to specify different temp dir for Spark compiler?

Same here, we'll have to look into it. Maybe a setting could be leveraged?

Could an upgrade to the latest Spark version help with any of this?

I'd say definitely maybe? In all seriously, being up to date with the latest is a good idea. Not sure if it's going to help, but I don't think anybody fancy fixing stuff in an old version.
If you want to experiment with the most recent work we've done (including supporting .net core), a pre-release on nuget is available.

If it's still an issue with the new version what would be helpful would be if you could isolate the issue into a separate vanilla project that doesn't include anything sensitive and upload it on github?

[expertsender-marcinsynak]

Thanks for the answer.

I tried to find out some info by looking at the source. But I did not find anything. Spark uses CodeDom and temp file settings may be completely internal.

The issue is not critical, just annoying. I get maybe dozen of such errors daily with hundreds of thousands of generated emails. It's completely random.

Until a better solution is found, I'm considering catching this particular exception and doing a retry.