Performance
Principle: no performance advantage is claimed without a measurement. A difference within measurement noise is called neither an "improvement" nor a "regression".
Measurement 1 — Game test server start-to-stop time (2026-10-19)
| Item | Value |
|---|---|
| Command | :poly-bootstrap:startupBenchmark |
| Scenario | Mojang game test server: create a new world → minecraft:always_pass → save → stop |
| Comparison | Vanilla (game jar on the class path, run without Poly Mod Loader) vs Poly Mod Loader (4 development mods, 4 transform rules, warmed transform cache) |
| Repetitions | 5 each, 1 warm-up run discarded. Vanilla and Poly Mod Loader run alternately, in a new directory each run |
| Environment | Windows 11, JDK 25.0.3, Minecraft 26.2, Poly Mod Loader 0.1.0-alpha.1 |
| Variant | Median (s) | Standard deviation (s) | Individual runs (s) |
|---|---|---|---|
| Vanilla | 7.137 | 0.122 | 6.99, 7.25, 6.96, 7.16, 7.14 |
| Poly Mod Loader | 7.225 | 0.312 | 7.84, 7.03, 7.23, 7.17, 7.33 |
Conclusion: The median difference is +0.09 s (+1.2%), which is within the Poly Mod Loader standard deviation (0.31 s). In this scenario, the cost of Poly Mod Loader bootstrap, discovery, and transform planning cannot be told apart from measurement noise. No performance improvement is claimed.
The raw data is in docs/benchmarks/2026-10-19-startup.json in the Poly Mod Loader repository.
Measurement 2 — Server tick time, heap, and GC under the same load (2026-10-21)
| Item | Value |
|---|---|
| Command | :poly-bootstrap:tickBenchmark (about 7 minutes, no window) |
| Scenario | On the game test server, while a data pack test waits 1,200 ticks: keep 40 pigs alive, and fill and clear a 16×16×16 block cube every tick. It is a data pack, so vanilla does the same work |
| Comparison | Vanilla vs Poly Mod Loader (5 development mods, 32 transform rules, 1 access widening) |
| Measurement tools | The agent tools/tick-probe, which times every tick, and Java Flight Recorder, attached identically to both. The first 200 ticks of each run are discarded |
| Repetitions | 5 each, 1 warm-up run discarded, run alternately |
| Environment | Intel Core i9-14900HX, 32 GB RAM, Windows 11, JDK 25.0.3 (G1, default heap), Minecraft 26.2 |
Median of 5 runs (standard deviation between runs in parentheses):
| Metric | Vanilla | Poly Mod Loader |
|---|---|---|
| Tick p50 (ms) | 18.13 (±0.97) | 18.20 (±2.25) |
| Tick p95 (ms) | 22.10 (±1.48) | 21.55 (±1.73) |
| Tick p99 (ms) | 53.1 (±16.0) | 46.6 (±26.5) |
| Ticks over 50 ms (%) | 1.10 | 0.99 |
| Peak heap usage (MiB) | 1,386 (±1,141) | 1,621 (±420) |
| Total GC pause (ms) | 1,263 | 1,227 |
Conclusion: The difference in every metric is smaller than the standard deviation between runs, so there is "no significant difference": neither an improvement nor a regression. Peak heap varied from 642 to 3,597 MiB between runs because of G1's heap sizing decisions, so this measurement cannot judge a memory difference.
The raw data is in docs/benchmarks/2026-10-21-ticks.json in the Poly Mod Loader repository.
Measurement 3 — Server tick time, heap, and GC with a fixed heap (2026-10-21)
| Item | Value |
|---|---|
| Command | :poly-bootstrap:tickBenchmark -PtickBenchmarkHeap=2g |
| Scenario | Same as measurement 2, but the cube is filled with glass. When filled with stone, the lighting calculation queue grew throughout the run, so the heap numbers measured the lighting engine, not the loader |
| Heap | -Xms2g -Xmx2g on both sides. The maximum heap left right after GC is recorded too |
| Everything else | Same as measurement 2 |
Median of 5 runs (standard deviation between runs in parentheses):
| Metric | Vanilla | Poly Mod Loader |
|---|---|---|
| Tick p50 (ms) | 12.97 (±2.23) | 12.25 (±1.44) |
| Tick p95 (ms) | 15.60 (±2.94) | 14.48 (±1.01) |
| Tick p99 (ms) | 18.12 (±3.25) | 15.85 (±0.94) |
| Maximum heap right after GC (MiB) | 158.1 (±12.7) | 157.4 (±10.7) |
| GC count | 18 | 18 |
| Total GC pause (ms) | 365.3 (±62.5) | 368.3 (±56.6) |
Conclusion: With the heap fixed, memory became comparable. The heap right after GC, GC count, and pause time are all indistinguishable from vanilla. The Poly Mod Loader median tick time is slightly lower, but it is within the vanilla standard deviation, so no improvement is claimed.
The raw data is in docs/benchmarks/2026-10-21-ticks-fixed-heap.json in the Poly Mod Loader repository.
Measurement 4 — Client frame time (2026-10-21)
frameBenchmark: opens a saved world with the client and times every frame (Minecraft.runTick) for 30 seconds. vsync off, no frame limit, 5 runs each. The Poly Mod Loader side includes a development mod that does per-frame drawing work (HUD, drawing in the world, shaders).
| Metric | Vanilla | Poly Mod Loader + development mod |
|---|---|---|
| Average FPS | 1,331 (±163) | 1,163 (±170) |
| Frame p50 (ms) | 0.656 (±0.052) | 0.800 (±0.075) |
| Frame p95 (ms) | 1.545 (±0.457) | 1.083 (±0.693) |
| Frame p99 (ms) | 2.070 (±0.579) | 1.962 (±0.747) |
Conclusion: p50 is 0.14 ms longer, recorded as a measured regression (0.9% of a 60 FPS frame budget). This measurement cannot separate the loader cost from the mod's drawing cost. The differences in p95, p99, and average FPS are within the variation between runs.
The raw data is in docs/benchmarks/2026-10-21-frames.json in the Poly Mod Loader repository.
Measurement 5 — A real Fabric mod pack: startup and server ticks (2026-10-22)
| Item | Value |
|---|---|
| Commands | :poly-bootstrap:startupBenchmarkFabric, :poly-bootstrap:tickBenchmarkFabric -PtickBenchmarkHeap=4g |
| Scenario | Startup as in measurement 1; ticks as in measurement 3 (glass cube, fixed heap of 4 GiB on both sides) |
| Comparison | Vanilla (no mods) vs Poly Mod Loader with the Fabric mods of the 26.2 compatibility set: 41 jars, which load 97 Fabric mods on this server (nested jars included) with 71 Mixin configurations |
| Not compared | The same mods under Fabric Loader. This measurement does not say whether Poly Mod Loader is faster or slower than Fabric Loader. It shows how far the mod pack, run through Poly Mod Loader, is from vanilla |
| Metric | Vanilla | Poly Mod Loader + Fabric mods |
|---|---|---|
| Startup, median (s) | 6.03 (±0.23) | 15.01 (±0.64) |
| Tick p50 (ms) | 7.91 (±0.32) | 4.04 (±0.09) |
| Tick p95 (ms) | 13.29 (±0.56) | 6.83 (±0.10) |
| Tick p99 (ms) | 15.16 (±0.61) | 8.81 (±0.62) |
| Maximum heap right after GC (MiB) | 141.6 (±6.5) | 182.9 (±1.3) |
| GC count | 12 | 16 |
| Total GC pause (ms) | 168.8 (±7.9) | 250.9 (±16.3) |
Conclusion:
- Startup takes 9.0 s longer with the mod pack, and memory after GC is 41 MiB higher. Both differences are far larger than the variation between runs, so they are recorded as measured costs. They include the initialization of 97 mods, the link check, and applying 71 Mixin configurations. This measurement cannot split them between the loader and the mods.
- Tick times are about half of vanilla's. This is not a Poly Mod Loader improvement. The pack includes Lithium, a server optimization mod, and FerriteCore. Their Mixin configurations loaded on this server and no Mixin failure was logged. No variant without Lithium was run, so the difference is not attributed to a specific mod. What the data shows: optimization mods such as Lithium apply under Poly Mod Loader, and with them the ticks in this workload were shorter than vanilla's.
The raw data is in docs/benchmarks/2026-10-22-startup-fabric.json and docs/benchmarks/2026-10-22-ticks-fabric.json in the Poly Mod Loader repository.
Measurement 6 — A real Forge mod pack: startup and server ticks (2026-10-23)
| Item | Value |
|---|---|
| Commands | :poly-bootstrap:startupBenchmarkForge, :poly-bootstrap:tickBenchmarkForge -PtickBenchmarkHeap=4g |
| Scenario | As in measurement 5: startup as in measurement 1; ticks as in measurement 3 (glass cube, fixed heap of 4 GiB on both sides) |
| Comparison | Vanilla (no mods) vs Poly Mod Loader with Forge 65.1.5 and the Forge mods of the 26.2 compatibility set: the jars corpusForge puts on the game test server, of which 32 of 38 run (the other 6 are client-only on a server, POLY-COMPAT-209). Every run logged "Forge mod loading, version 65.1.5" |
| Not compared | The same mods under Forge's own launcher. This measurement does not say whether Poly Mod Loader is faster or slower than Forge. It shows how far Forge and the mod pack, run through Poly Mod Loader, are from vanilla |
| Runs, tick measurement, environment | As in measurement 3 (Poly Mod Loader 0.1.0-alpha.18, after commit e1756cd, with libIPN pinned) |
Startup (5 runs, seconds):
| Variant | Median | Standard deviation | Individual runs |
|---|---|---|---|
| Vanilla | 6.281 | 0.090 | 6.18, 6.38, 6.40, 6.28, 6.24 |
| Poly Mod Loader + Forge mods | 22.864 | 0.865 | 21.36, 23.25, 22.86, 23.33, 21.93 |
Server ticks (median of 5 runs; standard deviation between runs in parentheses):
| Metric | Vanilla | Poly Mod Loader + Forge mods |
|---|---|---|
| Ticks measured per run | 1,006 | 1,007 |
| Tick p50 (ms) | 8.13 (±0.21) | 12.49 (±0.31) |
| Tick p95 (ms) | 9.81 (±0.40) | 15.08 (±4.21) |
| Tick p99 (ms) | 12.79 (±0.93) | 20.93 (±9.10) |
| Longest tick (ms) | 47.4 | 60.9 |
| Ticks over 50 ms (%) | 0.00 | 0.70 |
| Maximum heap right after GC (MiB) | 139.2 (±2.2) | 267.3 (±2.4) |
| GC count | 12 | 21 |
| Total GC pause (ms) | 185.0 (±17.9) | 564.8 (±138.5) |
| Longest GC pause (ms) | 39.0 | 61.2 |
Conclusion:
- Startup takes 16.6 s (+264%) longer. Every Poly Mod Loader run is more than 15 s longer than every vanilla run, so it is recorded as a measured cost. It includes checking Forge's files, the patched game jar, FML's mod discovery and layers, Mixin, and constructing the 32 mods and writing their config files. This measurement cannot split it between Poly Mod Loader, Forge, and the mods. In the same session,
startupBenchmark(development mods only) took 6.24 s for vanilla and 6.98 s for Poly Mod Loader. - Memory: the heap right after GC is 128 MiB (+92%) higher, with 9 more GCs and 380 ms more total pause. The differences are far larger than either side's standard deviation, so they are recorded as the memory cost of Forge and the mod pack.
- Tick time: p50 is 4.4 ms (+54%) longer. In all five runs Poly Mod Loader was at 12.25–13.06 ms and vanilla at 7.83–8.36 ms, with no overlap, so it is recorded as a measured cost. However, in the same session this PC's runs split into two groups, at about 8 ms and about 12.3 ms p50. The same session's
tickBenchmark(development mods only, same 4 GiB heap) gave 8.38, 8.92, 8.82, 8.59 and 12.30 ms for vanilla and 7.70, 12.47, 7.43, 12.53 and 12.29 ms for Poly Mod Loader. So this measurement cannot tell whether all Forge runs falling into the slow group comes from machine state. The cost of Forge's event bus and of the mods' tick work was not measured separately. - p95 and p99 have a large standard deviation on the Poly Mod Loader side (one run's longest tick was 118 ms, and that run's longest GC pause 99.9 ms), so only the medians are compared.
The raw data is in docs/benchmarks/2026-10-23-startup-forge.json and docs/benchmarks/2026-10-23-ticks-forge.json in the Poly Mod Loader repository.
Measurement 7 — A real Quilt mod pack: startup and server ticks (2026-10-24)
| Item | Value |
|---|---|
| Commands | :poly-bootstrap:startupBenchmarkQuilt, :poly-bootstrap:tickBenchmarkQuilt -PtickBenchmarkHeap=4g |
| Scenario | As in measurements 5 and 6: startup as in measurement 1; ticks as in measurement 3 (glass cube, fixed heap of 4 GiB on both sides) |
| Comparison | Vanilla (no mods) vs Poly Mod Loader with the 26.2 Quilt set (114 Quilt-only builds and 3 required mods: the jars corpusQuilt puts on the game test server) and a test harness. 132 external mods load (77 Quilt, 55 Fabric, nested mods included) |
| Left out of the set (benchmark only) | Craftable Bucket of Sulfur Cube and its Easy version add a function sulfur:tick that does not exist to minecraft:tick, so the game rejects the whole minecraft:tick tag and the benchmark data pack's tick function does not run either (vanilla TagLoader). Omni: a common Mixin checks the client class LocalPlayer with instanceof, so the server stops with NoClassDefFoundError the moment a mob jumps (checked in the bytecode; the same under Quilt Loader). The first run still had these three, so the tick function did not run (Poly Mod Loader p50 0.06 ms); that result was discarded |
| Not compared | The same mods under real Quilt Loader. This measurement does not say whether Poly Mod Loader is faster or slower than Quilt Loader |
| Runs, tick measurement, environment | As in measurement 3 (Poly Mod Loader 0.2.0-beta.2 plus that work's changes). Another job (a NeoForge game run) was running on the same PC. Vanilla and Poly Mod Loader run alternately, so both had the same conditions, but the absolute numbers may vary more than usual |
Startup (5 runs, seconds):
| Variant | Median | Standard deviation | Individual runs |
|---|---|---|---|
| Vanilla | 6.539 | 0.450 | 6.56, 6.28, 6.54, 7.31, 6.15 |
| Poly Mod Loader + Quilt set | 12.873 | 0.757 | 14.26, 12.36, 13.07, 12.87, 12.49 |
Server ticks (median of 5 runs; standard deviation between runs in parentheses):
| Metric | Vanilla | Poly Mod Loader + Quilt set |
|---|---|---|
| Ticks measured per run | 1,006 | 1,006 |
| Tick p50 (ms) | 8.40 (±0.35) | 7.67 (±0.05) |
| Tick p95 (ms) | 10.05 (±0.75) | 9.27 (±0.13) |
| Tick p99 (ms) | 13.52 (±1.28) | 12.37 (±1.27) |
| Longest tick (ms) | 47.3 | 69.6 |
| Ticks over 50 ms (%) | 0.00 | 0.30 |
| Maximum heap right after GC (MiB) | 139.4 (±1.1) | 193.3 (±1.5) |
| GC count | 12 | 15 |
| Total GC pause (ms) | 198.9 (±15.7) | 291.3 (±20.8) |
| Longest GC pause (ms) | 38.9 | 60.7 |
Conclusion:
- Startup takes 6.3 s (+97%) longer. Every Poly Mod Loader run is more than 5 s longer than every vanilla run, so it is recorded as a measured cost. It includes judging and link-checking 132 mods, Mixin configurations, Fabric API's initialization, and reading more than 65 data packs; it was not split further.
- Memory: the heap right after GC is 54 MiB (+39%) higher, with 3 more GCs and 92 ms more total pause. The differences are far larger than the standard deviation, so they are recorded as the memory cost of the Quilt set.
- Tick time: p50 is 0.73 ms shorter on the Poly Mod Loader side. In all five runs Poly Mod Loader was at 7.61–7.75 ms and vanilla at 7.86–8.80 ms. This is not claimed as an improvement. As in measurement 6, this PC's runs split into groups within a session, and this time another game run was going on at the same time. Which mods in the set (RailOptimization and others) shorten ticks was not measured separately. The longest tick and the ticks over 50 ms are higher on the Poly Mod Loader side, together with a longer longest GC pause.
The raw data is in docs/benchmarks/2026-10-24-startup-quilt.json and docs/benchmarks/2026-10-24-ticks-quilt.json in the Poly Mod Loader repository.
Measurement 8 — A real NeoForge mod pack: startup and server ticks (2026-10-24)
| Item | Value |
|---|---|
| Commands | :poly-bootstrap:startupBenchmarkNeoForge, :poly-bootstrap:tickBenchmarkNeoForge -PtickBenchmarkHeap=4g |
| Scenario | As in measurement 6: startup as in measurement 1; ticks as in measurement 3 (glass cube, fixed heap of 4 GiB on both sides) |
| Comparison | Vanilla (no mods) vs Poly Mod Loader with NeoForge 26.2.0.93 and the NeoForge mods of the 26.2 compatibility set: the jars corpusNeoForge puts on the game test server (36 from the set plus a game test harness; FML runs 49 mods). The Poly Mod Loader side starts with the Poly Mod Loader Java agent, as an installed profile does |
| Not compared | The same mods under NeoForge's own launcher. This measurement does not say whether Poly Mod Loader is faster or slower than NeoForge. It shows how far NeoForge and the mod pack, run through Poly Mod Loader, are from vanilla |
| Runs, tick measurement, environment | As in measurement 3 (commit 2ee8d3a). Other jobs may have been running on the same PC |
Startup (5 runs, seconds):
| Variant | Median | Standard deviation | Individual runs |
|---|---|---|---|
| Vanilla | 7.577 | 0.637 | 6.51, 6.48, 7.58, 7.69, 7.70 |
| Poly Mod Loader + NeoForge mods | 21.288 | 1.891 | 17.92, 18.11, 21.29, 21.55, 21.54 |
Server ticks (median of 5 runs; standard deviation between runs in parentheses):
| Metric | Vanilla | Poly Mod Loader + NeoForge mods |
|---|---|---|
| Ticks measured per run | 1,007 | 1,007 |
| Tick p50 (ms) | 9.67 (±0.28) | 4.80 (±0.86) |
| Tick p95 (ms) | 16.36 (±2.28) | 6.36 (±1.55) |
| Tick p99 (ms) | 18.55 (±2.26) | 8.98 (±1.86) |
| Longest tick (ms) | 42.8 | 63.2 |
| Ticks over 50 ms (%) | 0.00 | 0.10 |
| Maximum heap right after GC (MiB) | 141.3 (±3.8) | 225.3 (±1.0) |
| GC count | 12 | 14 |
| Total GC pause (ms) | 173.7 (±7.1) | 282.6 (±11.9) |
| Longest GC pause (ms) | 30.3 | 57.0 |
Conclusion:
- Startup takes 13.7 s (+181%) longer. Every Poly Mod Loader run is more than 10 s longer than every vanilla run, so it is recorded as a measured cost. It includes checking NeoForge's files and the patched game jar, FML's mod discovery, layers and processors, Mixin, and constructing the 49 mods and writing their config files, so it cannot be split. Both sides split into two groups by run order (vanilla about 6.5 s and 7.6 s, Poly Mod Loader about 18 s and 21.5 s). This looks like machine state, but this measurement cannot tell.
- Tick time: p50 is 4.9 ms (−50%) shorter on the Poly Mod Loader side. In all five runs Poly Mod Loader was at 4.51–6.62 ms and vanilla at 9.46–10.10 ms, with no overlap. The set includes Lithium and FerriteCore, which shorten server ticks, so this difference is taken as those mods' effect and is not claimed as a performance advantage of Poly Mod Loader or NeoForge. It was not measured mod by mod.
- Memory: the heap right after GC is 84 MiB (+59%) higher, with 2 more GCs and 109 ms more total pause. The differences are far larger than either side's standard deviation, so they are recorded as the memory cost of NeoForge and the mod pack.
- The longest tick and the longest GC pause are longer on the Poly Mod Loader side (52–66 ms per run). They look like single GC pauses, so only the medians are compared.
- In the first tick measurement attempt (the same day), the last Poly Mod Loader run stopped while constructing mods and timed out. The same stop came back later in
corpusNeoForge(26.1.2), which gave a thread dump: a deadlock between the game class loader and FML's plugin class loader when two Kotlin mods were constructed in parallel. It is fixed. The numbers above were measured before the fix; the fixed code holds a lock for less time while looking up classes but does the same work.
The raw data is in docs/benchmarks/2026-10-24-startup-neoforge.json and docs/benchmarks/2026-10-24-ticks-neoforge.json in the Poly Mod Loader repository.
Measurement 9 — The same Fabric mods: Poly Mod Loader vs Sinytra Connector (2026-10-24)
Unlike measurements 5 to 8, this is the first measurement compared directly with another loader stack. The same Fabric mods were run on Poly Mod Loader and on Sinytra Connector on real NeoForge.
| Item | Value |
|---|---|
| Commands | :poly-bootstrap:startupBenchmarkConnector -PtickBenchmarkHeap=4g, :poly-bootstrap:tickBenchmarkConnector -PtickBenchmarkHeap=4g |
| Variants | ① Vanilla (no mods) ② Poly Mod Loader + the common Fabric mods + Fabric API ③ NeoForge 26.2.0.93 + Sinytra Connector + the common Fabric mods + Forgified Fabric API. The three variants run alternately in the same session (1 warm-up run each discarded, 5 measured runs, a new directory each run) |
| Connector side setup | A dedicated server made with the NeoForge installer's --installServer (neoforge-26.2.0.93-installer.jar, SHA-1 97205829a020fc9389f504a77f590b54157c8dde). Poly Mod Loader is not involved. The mods Connector's README lists as required are downloaded from Modrinth, pinned by SHA-512: Connector 3.1.0-beta.3+26.2, Forgified Fabric API 0.161.0+26.2+4.0.2, Launchpad 1.10.2+26.2. The optional mod Connector Extras was not included |
| How it starts | The Connector side uses the argument files NeoForge installed (user_jvm_args.txt, win_args.txt) as they are; only the entry point is net.neoforged.fml.startup.GameTestServer (Mojang's game test server) from the same jar instead of FML's net.neoforged.fml.startup.Server. It does not ask about the EULA, and the EULA was not accepted |
| Kept the same | The same JDK, the same heap (-Xms4g -Xmx4g, for startup too), the same tick measurement agent, JFR settings and data pack (measurement 3's glass cube, 1,200 ticks, first 200 discarded), and the same log4j settings (NeoForge's default settings write an extra debug log file, so they were replaced). On its first start, Connector converts the Fabric jars and keeps them in the game folder's .cache/. The warm-up run's .cache/ is copied into each measured run, so it is measured as a second start, the way Poly Mod Loader uses its shared transform cache |
| Each side's counterparts | Fabric API (Poly Mod Loader) ↔ Forgified Fabric API (Connector). The game test harnesses are each side's own: fabric-gametest-harness on the Poly side (Fabric API freezes registries only in the dedicated server's main, so the harness does the same), neoforge-gametest-harness on the Connector side (real NeoForge does not tick game tests in production) |
| Startup metrics | Start to stop (as in measurement 1), and start to ready: when the output first shows vanilla GameTestRunner's "Running test environment" (after world creation, before the first test) |
| Environment | Intel Core i9-14900HX, 31.7 GiB RAM, Windows 11 Pro 10.0.26200, JDK 25.0.3, G1, Minecraft 26.2, Poly Mod Loader 1.0.0 (commit 2f212b5), Fabric API 0.161.0+26.2. Before and during the measurement, other game JVMs were checked for every 15 seconds and there never was one (14 checks during startup, 29 during ticks, all quiet) |
The common mod set. First the whole set of Fabric jars from the 26.2 compatibility set (except the two Xaero mods, which are left out on the game test server) was run on both sides, and only the mods that actually loaded on both were kept. The 19 common mods: AppleSkin, Cloth Config, Collective, EntityCulling, Fabric Language Kotlin, FancyMenu, FerriteCore, GeckoLib, Jade, JEI, Konkrete, Melody, MezzConfig, NotEnoughAnimations, Placeholder API, 3D Skin Layers, Veinminer, Simple Voice Chat, YetAnotherConfigLib. In every run, Poly Mod Loader must log "Foreign mods: 89 loaded" (nested jars included) and FML must list all of these mods, or the task fails.
| Left out | Left out by | Reason |
|---|---|---|
| Architectury 21.1.12 | Connector | The Mixin MixinBaseSpawner.checkSpawnerSpawn fails its injection check (a method NeoForge patched). The server stops |
| Balm 26.2.0.9 | Connector | The Mixin LivingEntityMixin.clearAllEffects fails its injection check. The server stops |
| Lithium 0.25.3 | Connector | The Mixin block.hopper.LevelMixin fails its injection check. The server stops |
| Puzzles Lib 26.2.5 | Connector | Connector's jar conversion stops on one Mixin class ("Error parsing property 'slice'", ClassCastException) |
| Forge Config API Port 26.2.1 (Fabric build) | Connector | The net.neoforged.fml.config classes inside the jar clash with NeoForge's own: LinkageError (loader constraint violation) |
| Zoomify 2.17.0 | Connector | NeoForge reads the neoforge.mods.toml in the multi-loader jar and requires the language provider klf, which is not there. A client mod |
| LambDynamicLights 4.12.4 (+ nested libraries) | Poly Mod Loader | environment: client, so it is not loaded on a server (POLY-COMPAT-108, as in Fabric Loader). Connector loads it on a server too |
| 13 client-only mods such as Sodium, Iris and Continuity | Both | Not loaded on a server (Connector does load some shared libraries nested in Dynamic FPS and MoreCulling) |
The six mods above load on Poly Mod Loader (measurement 5), so the mods Connector cannot run are a compatibility difference, not a speed result of this measurement. In particular, Lithium is left out, so measurement 5's shorter ticks are not part of this one.
Startup (5 runs, seconds):
| Variant | To stop, median (±standard deviation) | Individual runs | To ready, median (±standard deviation) | Individual runs |
|---|---|---|---|---|
| Vanilla | 6.562 (±0.172) | 6.30, 6.56, 6.77, 6.56, 6.63 | 5.295 (±0.067) | 5.24, 5.28, 5.39, 5.30, 5.38 |
| Poly Mod Loader + common mods | 12.154 (±0.147) | 12.33, 11.99, 12.15, 11.99, 12.19 | 10.722 (±0.140) | 10.96, 10.72, 10.83, 10.61, 10.66 |
| NeoForge + Connector + common mods | 11.826 (±0.653) | 10.93, 11.83, 11.17, 12.37, 12.31 | 10.232 (±0.622) | 9.46, 10.23, 9.63, 10.75, 10.81 |
Server ticks (median of 5 runs; standard deviation between runs in parentheses):
| Metric | Vanilla | Poly Mod Loader + common mods | NeoForge + Connector + common mods |
|---|---|---|---|
| Ticks measured per run | 1,007 | 1,007 | 1,007 |
| Tick p50 (ms) | 8.63 (±0.18) | 8.47 (±0.26) | 8.48 (±0.17) |
| Tick p95 (ms) | 10.57 (±0.45) | 10.91 (±0.34) | 9.94 (±1.02) |
| Tick p99 (ms) | 13.16 (±0.48) | 13.97 (±0.97) | 12.51 (±2.84) |
| Longest tick (ms) | 57.2 | 61.8 | 57.3 |
| Ticks over 50 ms (%) | 0.10 | 0.20 | 0.10 |
| Maximum heap right after GC (MiB) | 138.4 (±0.9) | 173.1 (±1.1) | 203.0 (±2.7) |
| GC count (whole run, startup included) | 12 | 16 | 15 |
| Total GC pause (ms) | 192.9 (±17.7) | 282.3 (±34.0) | 293.5 (±36.7) |
| Longest GC pause (ms) | 45.2 | 49.5 | 78.9 |
Conclusion:
- Memory: Poly Mod Loader uses less. The heap right after GC is 29.9 MiB (−15%) lower than Connector's. In all five runs Poly Mod Loader was at 171–174 MiB and Connector at 201–207 MiB, with no overlap, and the difference is far larger than the standard deviation, so it is recorded as a measured difference. Above vanilla, Poly Mod Loader adds 34.7 MiB and Connector 64.6 MiB. The Connector side includes NeoForge itself, and this difference was not split between "the Connector layer" and "NeoForge". The longest GC pause is also longer on the Connector side (61–114 ms per run vs 44–60 ms).
- Tick p50: no difference. 8.47 vs 8.48 ms, the same, and within the standard deviation of vanilla (8.63 ms) too.
- Tick p95 and p99: Poly Mod Loader did not win. Connector's p95 median is 0.98 ms lower, and four of Connector's five runs (9.67–10.02 ms) were lower than every Poly Mod Loader run (10.18–11.02 ms). But one Connector run (12.12 ms) makes its standard deviation (1.02 ms) larger than the difference, so by the rule it is no significant difference. The same goes for p99 (a difference of 1.46 ms, Connector standard deviation 2.84 ms). Poly Mod Loader's p95 (10.91 ms) is also higher than vanilla's (10.57 ms), but within the standard deviation. Recorded as something to improve: with this set, Poly Mod Loader's tail latency is not lower than vanilla's or Connector's. The cause was not measured here (see measurement 10).
- GC count: Poly Mod Loader has 1 more (16 vs 15, the same in every run on both sides; vanilla 12). The JFR recording starts with the JVM, so GCs during startup are included. The total pause is within the standard deviation, so no difference is claimed.
- Startup (second start): Poly Mod Loader did not win. Connector's median is 0.49 s shorter to ready and 0.33 s shorter to stop. Three of Connector's five runs are shorter than every Poly Mod Loader run, and two fall within the range of the Poly Mod Loader runs (both to stop and to ready). The difference is within Connector's standard deviation (0.62–0.65 s), so it is no significant difference. The Poly Mod Loader side varies far less between runs (±0.14 s). Above vanilla, to ready, Poly Mod Loader adds 5.4 s and Connector 4.9 s.
- First start (the warm-up runs, one each, not used for conclusions): Connector took 15.2 s (13.8 s to ready) because it converts the Fabric jars, and Poly Mod Loader 12.0 s (10.6 s to ready). But Poly Mod Loader's transform cache had already been warmed by test runs before this task, so this is not a first-start comparison under the same conditions.
The raw data, with the jar lists of both variants, is in docs/benchmarks/2026-10-24-startup-connector.json and docs/benchmarks/2026-10-24-ticks-connector.json in the Poly Mod Loader repository. The pinned Connector downloads are listed in poly-bootstrap/compat/connector-reference-26.2.json.
Measurement 10 — Why measurement 9's tick tail is higher, and the ticks measured again (2026-10-24)
In measurement 9, Poly Mod Loader's tick p95 and p99 were higher than Connector's, and its p95 was higher than vanilla's too. The cause was looked for with a profiler, to be fixed if it was Poly Mod Loader's, and then only the ticks were measured again with the same settings. In short: no cost that Poly Mod Loader adds inside a tick was found, so the loader code was not changed. The p95 difference is concentrated at the front of the measured range (ticks 200–399), while the JIT is still settling, and about half of the p99 difference comes from one GC more falling inside the measured range.
How it was profiled
- The 15 raw JFR recordings of measurement 9 (default settings, Java samples every 20 ms) were analyzed again. The measured range was taken from the first time
GameTestServer.tickServerwas sampled on the server thread. - To separate slow ticks from normal ones, opt-in options were added to the measurement tools (off by default):
-PtickBenchmarkTickEvents=truemakes the tick probe write a JFR eventpoly.TickProbe.Call(duration, sequence number) for every tick, and-PtickBenchmarkJfrSettings=<default|profile|.jfc>changes the recording settings. They apply to all three variants the same way. - Profile runs:
tickBenchmarkConnector -PtickBenchmarkHeap=4g -PtickBenchmarkRuns=2 -PtickBenchmarkTickEvents=truewith a.jfcbased onprofile(Java samples every 5 ms,jdk.ClassLoadon,jdk.Compilationthreshold 0 ms, safepoints and VM operations, monitor 1 ms and park 2 ms thresholds). The first measured run overlapped with another job's game JVM and was discarded; for each variant only the warm-up run and the second run were used. The numbers from profile runs are not used as measurement results (they include the profiler's cost). - Slow ticks = at or above each run's p95 after tick 200; normal ticks = p40 to p60. Samples, allocations, GCs, class loads, deoptimizations, compilations and lock waits inside each group were counted.
What was found
| Suspect | Evidence | Verdict |
|---|---|---|
| Poly Mod Loader code (event dispatch, Fabric API event wiring, entry points) | Of 2,363 server thread samples in measurement 9's measured range, 4 (0.2%) have a dev.polymodloader frame and 3 (0.1%) a net.fabricmc frame. In the profile runs' slow ticks, both are 0. For comparison, on the Connector side NeoForge frames (EventBus.post, EventHooks.onNeighborNotify and others) are 1.8% of normal-tick samples | No |
| Total work on the server thread | Server thread CPU time from the end-of-run thread dump (measurement 9, median of 5): vanilla 12.23 s, Poly Mod Loader 12.28 s, Connector 12.17 s | No difference |
| Class loading and transforms during ticks | After tick 200, 0.9 class loads and 0.08 ms on the server thread per slow tick (vanilla 1.3 and 0.08 ms, Connector 1.2 and 0.10 ms). Measurement 9's Poly runs have one transform log line in the tick range | No difference |
| Logging | The only Poly Mod Loader log line during ticks is that one transform line | No |
| Allocation and GC count | Allocation samples in the measured range: vanilla 7.84 GiB, Poly Mod Loader 7.86 GiB, Connector 7.24 GiB (per run). The top allocation sites are all game code (BlockPos, CollectingNeighborUpdater). After the first tick, all three variants have 4 GCs in all 7 runs (measurement 10). The difference in GC count (12, 16, 15) is all in startup: Metadata GC Threshold 8 for vanilla, 10 for Poly and Connector; Evacuation Pauses during startup 0 for vanilla, 1 for Connector, 2 for Poly | Not about ticks |
| Where GCs fall in the measured range | The interval between GCs during ticks is about 320–360 ticks for all three. Connector's first GC comes at tick 0 (the discarded range) and the next at ticks 300–340, so 3 GCs fall in the measured range. Poly Mod Loader's first GC during ticks comes at ticks 240–255 (inside the measured range), so 4 fall in it (vanilla's first is at ticks 195–220, so 3 or 4). Which tick a GC falls on is set by how much startup allocated after its last GC. Ticks over 30 ms per run (measurement 10): Poly Mod Loader 3–4, vanilla 3–4, Connector 2–3. Without GC ticks (over 30 ms), the p99 difference from Connector shrinks from 1.11 to 0.73 ms (measurement 10) and from 1.46 to 0.76 ms (measurement 9); p95 stays about the same (measurement 10: 10.54 vs 10.02 ms) | A phase difference, not a cost of Poly code |
| Lock contention | No JavaMonitorEnter (1 ms or more) or ThreadPark (2 ms or more) on the server thread in the measured range, for all three | No |
| The tick probe | The same jar, the same arguments and the same target method (GameTestServer.tickServer) for all three, and the task checks the [tick-probe] timing line in every run. On the Poly and NeoForge sides the probe wraps the method after each side's own transforms, so each side's hooks in tickServer (poly:server_tick_events, NeoForge's ServerTickEvent) are all inside the measurement | Treated the same |
| JIT compilation | Compilations per tick range (profile runs, 15 compiler threads): in ticks 200–399, vanilla 236 and 275, Poly 247 and 230, Connector 241 and 252. In the 3 seconds before the start, 9–15 C2 compilations of Poly Mod Loader's own methods | No difference |
The answer was where the slow ticks are. Of the 102 slow ticks (p95 or above) in the profile runs, those in ticks 200–399 are 61 for Poly Mod Loader, 62 for vanilla and 48 for Connector. Apart from the few ticks with a GC, those ticks have no class loads and no loader frames; the same game methods as in normal ticks just run slower (the top sampled methods are the same as in normal ticks). In the medians of 25-tick groups, all three variants settle only around ticks 300–400. In ticks 100–200, Connector (NeoForge) is the fastest; Poly Mod Loader is the slowest at ticks 225–250 (its first GC during ticks comes around then) and the fastest from tick 275 on (measurement 10, median of 7 runs, ms):
| Tick | 0 | 100 | 150 | 200 | 225 | 250 | 275 | 300 | 400 | 475 |
|---|---|---|---|---|---|---|---|---|---|---|
| Vanilla | 20.2 | 14.1 | 11.6 | 10.8 | 9.5 | 9.5 | 9.5 | 9.5 | 8.9 | 8.8 |
| Poly Mod Loader | 20.6 | 13.6 | 11.9 | 10.9 | 10.8 | 9.9 | 8.9 | 8.8 | 8.3 | 8.3 |
| Connector | 20.2 | 12.4 | 10.8 | 10.0 | 9.7 | 9.5 | 9.3 | 9.0 | 8.6 | 8.4 |
The benchmark discards only the first 200 ticks, so this settling range goes into p95 and p99. Splitting the same raw ticks into ranges (median of per-run values, ±standard deviation, ms):
| Range | Metric | Vanilla | Poly Mod Loader | Connector |
|---|---|---|---|---|
| Measurement 9, ticks 200–399 | p95 / p99 | 11.55 (±0.69) / 14.66 | 12.38 (±0.28) / 16.70 | 10.38 (±3.51) / 13.06 |
| Measurement 9, tick 400 to the end | p95 / p99 | 9.79 (±0.30) / 12.50 | 9.58 (±0.41) / 11.63 | 9.61 (±0.26) / 11.85 |
| Measurement 10, ticks 200–399 | p95 / p99 | 11.86 (±0.59) / 16.09 | 11.63 (±1.52) / 17.02 | 10.77 (±1.86) / 15.08 |
| Measurement 10, tick 400 to the end | p95 / p99 | 9.77 (±2.10) / 12.06 | 9.20 (±0.68) / 12.63 | 9.61 (±2.26) / 11.75 |
After tick 400, Poly Mod Loader's p95 is the same as or lower than Connector's and vanilla's (within the standard deviation, so no advantage is claimed). The average over ticks 200–399 is the same for Poly Mod Loader and vanilla (median of the 12 runs of measurements 9 and 10: 10.02 vs 9.99 ms), and lower for Connector (9.46 ms). So in this range Poly Mod Loader does not do more work than vanilla; the NeoForge side warms up faster. Connector also allocates about 8% less per tick (allocation samples: BlockPos$MutableBlockPos.immutable 1.9 GiB per run vs 2.8 GiB for Poly Mod Loader and 2.9 GiB for vanilla). Less allocation in the same game code suggests the JIT's escape analysis turned out differently. Why was not found. Candidates: the shape of the game code NeoForge patched (Level.markAndNotifyBlock shows up separately only on the Connector side), the different Mixins of Forgified Fabric API and Fabric API (even with the same Fabric mods, self-time distributions differ: ServerChunkCache.getChunk self time 16% for Connector, 11% for vanilla, 6% for Poly Mod Loader), and run order (always vanilla → Poly Mod Loader → Connector).
One allocation showed up only on the Poly Mod Loader side: Collective's (Fabric build) Mixin Level_updateNeighborsAt creates CallbackInfo objects, about 35 MiB per run (0.4% of all allocation). That is the mod's own cost, not something for Poly Mod Loader to change, and by its size not the cause of the tail latency.
What was changed
The loader code was not changed, because the profile found no cost of Poly Mod Loader's own. Only the measurement tools changed (the profile options above, off by default and not used in the measurement below).
Ticks measured again
| Item | Value |
|---|---|
| Command | :poly-bootstrap:tickBenchmarkConnector -PtickBenchmarkHeap=4g -PtickBenchmarkRuns=7 |
| Settings | As in measurement 9 (the same three variants, mods, JDK, heap, data pack, log settings and default JFR settings; 1,200 ticks, first 200 discarded; 1 warm-up run each discarded). The difference: 7 measured runs (measurement 9 had 5) |
| Code | Poly Mod Loader 1.0.0. The loader code is the same as in measurement 9; only the measurement tools are at commit b7d71fd |
| Environment | The same PC as measurement 9 (Intel Core i9-14900HX, 31.7 GiB RAM, Windows 11 Pro 10.0.26200, JDK 25.0.3, G1, Minecraft 26.2). Before and during the measurement, other game JVMs were checked for every 15 seconds, and all 38 checks were quiet |
Server ticks (median of 7 runs; standard deviation between runs in parentheses). The last column is measurement 9's Poly Mod Loader, for comparison:
| Metric | Vanilla | Poly Mod Loader | NeoForge + Connector | Measurement 9 Poly Mod Loader |
|---|---|---|---|---|
| Ticks measured per run | 1,006 | 1,007 | 1,006 | 1,007 |
| Tick p50 (ms) | 8.74 (±0.51) | 8.24 (±0.16) | 8.49 (±0.51) | 8.47 (±0.26) |
| Tick p95 (ms) | 10.58 (±1.61) | 10.73 (±0.66) | 10.07 (±2.09) | 10.91 (±0.34) |
| Tick p99 (ms) | 13.47 (±1.47) | 14.18 (±1.53) | 13.07 (±2.36) | 13.97 (±0.97) |
| Longest tick (ms) | 51.3 | 61.8 | 55.2 | 61.8 |
| Ticks over 50 ms (%) | 0.10 | 0.10 | 0.10 | 0.20 |
| Maximum heap right after GC (MiB) | 141.5 (±3.5) | 173.1 (±4.0) | 203.8 (±2.3) | 173.1 (±1.1) |
| GC count (whole run, startup included) | 12 | 16 | 15 | 16 |
| Total GC pause (ms) | 189.6 (±7.7) | 267.6 (±21.6) | 283.2 (±17.6) | 282.3 (±34.0) |
| Longest GC pause (ms) | 41.6 | 48.7 | 65.6 | 49.5 |
p95 per run (ms): vanilla 10.58, 14.77, 10.23, 10.51, 10.65, 11.53, 10.44 / Poly Mod Loader 10.61, 11.04, 12.01, 11.94, 10.73, 10.45, 10.56 / Connector 13.61, 9.88, 9.91, 10.04, 14.84, 10.07, 10.10.
Conclusion:
- Before and after: no improvement. The loader code was not changed, so this is a re-measurement. Poly Mod Loader's p95 went 10.91 → 10.73 ms and p99 13.97 → 14.18 ms, within the standard deviation; it reproduces measurement 9.
- Tick p95 and p99: Poly Mod Loader still did not win. Connector's p95 median is 0.66 ms lower, and five of Connector's seven runs (9.88–10.10 ms) are lower than every Poly Mod Loader run (10.45–12.01 ms). The other two (13.61 and 14.84 ms) are higher than every Poly Mod Loader run. The standard deviation (2.09 ms) is larger than the difference, so by the rule it is no significant difference, but the shape, with Connector lower in an ordinary run, is the same as in measurement 9. The same goes for p99 (a difference of 1.11 ms, Connector standard deviation 2.36 ms).
- Cause: JIT settling at the front of the measured range, and where the GC falls. As the range table shows, the p95 difference is in ticks 200–399 and not after tick 400. In that range Poly Mod Loader warms up at the same speed as vanilla, and the NeoForge side warms up faster. About half of the p99 difference comes from Connector's first GC during ticks falling in the discarded range (tick 0), which leaves one GC fewer in its measured range. It is not caused by work Poly Mod Loader does every tick.
- Against vanilla: p95 10.73 vs 10.58 ms and p99 14.18 vs 13.47 ms, within the standard deviation: no significant difference. Poly Mod Loader's p50 is 0.50 ms lower, but within vanilla's standard deviation (0.51 ms), so it is not claimed.
- Variation: Poly Mod Loader varies least between runs of the three variants (p95 standard deviation 0.66 ms vs 1.61 ms for vanilla and 2.09 ms for Connector). One vanilla run and two Connector runs were slow throughout (p50 9.3–9.6 ms).
- Memory: the heap right after GC is the same as in measurement 9 (Poly Mod Loader 173.1 MiB, Connector 203.8 MiB, with no overlap in the five and the seven runs).
The raw data is in docs/benchmarks/2026-10-24-ticks-connector-rerun.json, and the per-range numbers for every run of measurements 9 and 10 are in docs/benchmarks/2026-10-24-tick-ranges-connector.json, in the Poly Mod Loader repository.
What is not measured
- Client frame time with Poly Mod Loader only and no mods (separating loader cost from mod cost)
- Allocation rate, chunk generation time
- The same Fabric mod pack under Fabric Loader (measurement 5), and a variant without Lithium
- Client frame time with a Fabric mod pack, and a Forge mod pack
- The same Forge mod pack under Forge's own launcher (measurement 6), and the cost of Forge's event bus and of each mod's tick work
- The same NeoForge mod pack under NeoForge's own launcher (measurement 8), and a NeoForge set without Lithium
- The same Quilt mod pack under Quilt Loader (measurement 7), and a Quilt set on the 26.1 line
- In the Connector comparison (measurement 9): client frame time (FPS); the mods only one side loads (Architectury, Balm, Lithium, Puzzles Lib, Forge Config API Port and Zoomify, which Connector cannot run, and LambDynamicLights, which Poly Mod Loader leaves out on a server); a first start under the same conditions (empty caches); a variant with NeoForge alone (separating NeoForge's share from Connector's); Connector Extras; the 26.1 line; a dedicated server (it stops at the EULA check, so the game test server was used)
- In measurement 10: why the NeoForge side warms up faster in ticks 200–399 and allocates less per tick (comparing JIT inlining and escape analysis, with
-XX:+PrintCompilation,PrintInliningand so on); a variant with a different run order (always vanilla → Poly Mod Loader → Connector); a variant that discards the first 400 ticks (measurement 10's range table splits the same raw ticks, it is not a separate measurement); CPU time sampling (jdk.CPUTimeSample, Linux only) - JMH microbenchmarks (event dispatch, transform planning)
- Other hardware
Until these are measured, performance goals (such as a 5% better frame p95) are only goals, not results.