F# Compiler Server: plan

This is a first-party design for dotnet/fsharp. It is based on main @ 99406265, SDK 11.0.100-rc.1, and Apple M-series hardware (11 cores, 18 GB). Written 2026-10-18. Implementation: branch compiler_server (diff); measured results: report.

Summary

How fsc runs today

On main, Fsc is a ToolTask marked [MSBuildMultiThreadableTask] (#20506). That attribute only lets the task run on a thread inside a multi-threaded MSBuild node, using TaskEnvironment for the working directory and environment. Fsc.ExecuteTool still calls ToolTask.ExecuteTool, which starts dotnet fsc.dll @rsp once per project (Fsc.fs:758). Every compilation therefore pays runtime startup, assembly loading and JIT. Phase 1 measured that JIT is the main reason the JIT flags help.

Architecture

Why this shape

• No extra client process. The Fsc task is already loaded and JIT-warm in MSBuild nodes, and FSharp.Build.dll is the library call. An AOT thin client would get the child-process environment for free, but it would mean shipping a native binary per platform from dotnet/fsharp. We can get the exact same inputs in-process by calling ToolTask.GetProcessStartInfo, which is protected virtual. That call returns the exact working directory and environment the stock child would receive, including in -mt mode.
• No broker. A broker exists to pick an idle worker and to start and stop workers. Per-slot exclusive file locks do the first. The slot owner starting its own worker does the second. Locks are released by the kernel when a process dies, so a crashed client can't wedge the pool.
• Worker = fsc.dll. This keeps everything that already ships with the compiler: Server GC runtimeconfig, R2R images, deps.json, FSharp.Core binding, and the reference resolver. The SDK's FSharp/ folder gains no files. The Fsc task already knows the host (DOTNET_HOST_PATH) and DotnetFscCompilerPath.
• Processes, not threads. The working directory, Console, environment variables and many compiler statics are process-wide. One compilation per process keeps fidelity simple. Compiling inside MSBuild nodes was rejected for the same reasons, and also because a compiler crash would take down the node and type providers would load into MSBuild.

Request lifecycle

Fidelity: the worker must look like a fresh dotnet fsc.dll @rsp

Process-state audit: what a second compilation would otherwise inherit

I searched src/Compiler for module-level mutable state and caches. The worker captures a snapshot after startup and restores it before every compilation (CaptureProcessState / RestoreProcessState in fsc.fs). Each module gets a small internal reset helper.

Type providers and compiler extensions

• Explicit hooks. A process-wide flag is set wherever the compiler loads third-party code into its own process:
  • getTypeProviderAssembly → Assembly.UnsafeLoadFrom (CompilerLocation.fs:183), for type-provider design-time assemblies
  • DependencyProvider → Assembly.LoadFrom (DependencyProvider.fs:518), for --compilertool and dependency managers
  • the AssemblyResolve / native resolve handler installation
• Backstop. The worker subscribes to AppDomain.AssemblyLoad. Any load from outside the compiler directory or the shared framework directory marks the worker for retirement, which catches future extension points nobody has hooked yet. This is the third-party heuristic, kept only as a safety net.
• Retirement. The worker stops listening before it replies, sends "retire":true with its pid, and exits. The client waits for that pid to exit before it releases the slot. That ensures nothing holds the provider DLL open on Windows when the next build step copies it.
• The cost is one worker restart per project that uses type providers. Every other compilation in the build still uses a warm worker.

Enabling and fallback

<!-- Microsoft.FSharp.Targets -->
<_FSharpUseSharedCompilation Condition="'$(FSharpUseSharedCompilation)' == 'true' and '$(UseSharedCompilation)' != 'false'">true</_FSharpUseSharedCompilation>
<Fsc … UseSharedCompilation="$(_FSharpUseSharedCompilation)" />

The SDK sets UseSharedCompilation=true by default (Microsoft.NET.Sdk.targets:1172 and Microsoft.Managed.Core.targets:57). Reusing that property would turn the F# server on for everyone. So the opt-in is a new property, and the existing one can only veto.

The server path is skipped (stock behaviour) when any of these hold:

• a HostObject is set (VS in-process compile)
• DotnetFscCompilerPath is empty (.NET Framework fsc.exe)
• MSBuild runs on .NET Framework
• --pause is used
• SkipCompilerExecution is set

The task falls back to stock when:

• no slot is free
• connecting or spawning fails or times out
• the connection drops before a response arrives (for example, a worker crash or stack overflow)

Cancellation (Ctrl+C) kills the worker the task is talking to, so a cancelled compilation can't keep writing outputs that race with the next build. A new worker is started on demand.

Pool, slots and lifetime

Protocol

JSONL over a named pipe (a Unix domain socket on macOS and Linux): one request line, then one response line per connection, written with System.Text.Json's Utf8JsonWriter and Utf8JsonReader. The STJ source generator only works with C#, and both ends are F#. Hand-written reader and writer code for two small message types is a few dozen lines with no reflection, so startup cost is low. FSharp.Build (netstandard2.0) references STJ compile-only (ExcludeAssets="runtime"). The server path only runs on .NET Core MSBuild, where STJ is part of the shared framework, so no new DLL ships.

→ {"v":1,"type":"compile","cwd":"/src/proj","args":["@/tmp/MSBuildTemp/tmp1a2b.rsp"]}
← {"v":1,"exitCode":0,"retire":false,"pid":48211,"lines":[[1,"warning FS3370: …"],[0,"STATS: #ByteArrayFile = …"]]}
→ {"v":1,"type":"shutdown"}

Stop CLI

• dotnet <sdk>/FSharp/fsc.dll --compiler-server-shutdown visits every pool dir of the current user. For each slot it takes the lock, waiting for any running compilation to finish, then sends shutdown. Workers that don't respond are killed using the pid file the worker writes.
• It uses the same DLL as the worker and the client protocol code, so there's nothing extra to install. This works now by full path.
• The natural home later is dotnet build-server shutdown, which already stops MSBuild nodes, VBCSCompiler and Razor. That needs a small dotnet/sdk change to call the command above.

Code changes (dotnet/fsharp branch compiler_server)

The SDK side needs nothing: a new FSharp.Build plus fsc.dll ship in sdk/<v>/FSharp/ as usual.

Test and benchmark plan

Correctness: byte-identical to stock

1. Build FSharp.Compiler.Service.slnx (17 compilations: FSharp.Core ×3 TFMs, FCS ×2, …) with the stock R2R layout of main. Snapshot every obj/**/*.dll|pdb|xml and refint assembly, and keep the MSBuild warnings and messages.
2. Rebuild with the server layout and FSharpUseSharedCompilation=true, cold and then warm with workers reused. Compare every output byte for byte, plus the diagnostics text in order.
3. Stress reuse ordering: force a single worker (FSharpCompilerServerMaxWorkers=1, a test knob) so each worker compiles many projects in sequence. Run with -m:1 and a reversed project order.
4. Spot checks: a compilation with errors (exit code and messages), --consolecolors-, a type-provider project (worker retires), a non-English LANG, and killing a worker mid-compilation (fallback).

Performance: whole dotnet build of the slnx

For each config: one warm-up build, then two measured builds, as requested. Each run is dotnet build -c Release --no-restore --no-incremental with MSBuild node reuse off, so the only state that survives between builds is the server's workers. F# workers are stopped between configs. I'll report wall and CPU time per build, plus the server's cold first build separately, since it includes worker startup and JIT. Two runs on a desktop machine with roughly 3–5% noise can only show large effects.

Risks and open items

• Hidden statics. The audit is a grep, so something it missed could still leak between compilations. The byte-identical tests over 17 compilations in several orders are the check; anything they find goes into RestoreProcessState.
• Windows console encoding. Without --utf8output, stock output passes through the OEM code page, which can turn characters into ?. The server sends exact strings. To be identical, the worker would have to round-trip text through Console.OutputEncoding. That's planned, but untested on this Mac.
• Memory. Up to 11 idle workers each hold a Server-GC heap. Mitigations: the compacting GC, an idle timeout, and possibly a lower N later.
• Static state that depends on the working directory or env (for example BinFolderOfDefaultFSharpCompiler) is computed from the compiler location, which is the same for every request. Other environment variables are ignored, as agreed.

Reliability later (designed for, not built now)

• Health and version handshake; a worker log file; structured telemetry of fallbacks.
• Memory caps and recycling after K compilations or M GB.
• Opt-in cache reuse (IL readers for immutable framework and NuGet references) for extra speed, as a separate measured experiment.
• dotnet build-server shutdown integration; multi-user /tmp hardening on Linux.