Extract installed chunks mapping's variable for improved long-term caching

[huangshuwei]

Hi:
Even if the hash of assets information is extracted from the manifest.js file by html inline. but manifest.js file changes when assets change every time.

this part of the manifest.js file changes every time:

/******/ 	// objects to store loaded and loading chunks
/******/ 	var installedChunks = {
/******/ 		10: 0
/******/ 	};

Unless this part is also extracted from the manifest.js file, cache problem will be solved.

[tonyzheng121]

@huangshuwei do you have some examples? when i use, it throw error

[huangshuwei]

@lomotony
This config file is based on webpack3
https://github.com/huangshuwei/webpack3-vue2-demo/blob/master/webpack.config.js

[ufologist]

try webpack.NamedChunksPlugin change numerical chunk id to chunk name

more detail: Predictable long term caching with Webpack

[jouni-kantola]

As of closing PR #18 this can now be worked on. I'd much appreciate a PR. Currently, I don't really have the time to solving this one. 👶 👶 require a lot of 😍 .

[NinoFloris]

Yes this is bugging us too, webpack generates the hashes of files before starting the compilation.

This means our final webpack runtime chunk is byte for byte equal to a version with a different chunkhash because the file at the start used to be different (it was hashed with the manifest still in there).
As we extracted the manifest and use named chunks it's 100% equal but the name would still invalidate caches, so we're using a small extra compiler plugin that just overrides the renderedhash on the chunk to be a constant.

@jouni-kantola
Providing this is a configurable option (as when your runtime is embedded in vendor you definitely don't want this) is this something you'd accept as a pr?

[jouni-kantola]

Accidentally closed when typing 😁

What I was going to write was:

1. Could you please add a repro setup?
2. I'd gladly take PRs.
3. Maybe there are other events to hook up to in the webpack compilation, to prevent the issue described. I didn't fully understand if the initial issue is about the same as yours, @NinoFloris.

[NinoFloris]

Related by outcome for sure ^_^

After tuning and reading the webpack source I have something that also works if the runtime code is inside another chunk!
Got it working with this snippet to give a stable hash to the final output of the runtime chunk.

  plugins:  [
    {
        apply(compiler) {
            compiler.plugin("this-compilation", compilation => {
                const { hashFunction, hashDigest, hashDigestLength, hashSalt } = compilation.outputOptions;

                compilation.mainTemplate.plugin("require-ensure", (source, chunk) => {
                    const chunkHash = crypto.createHash(hashFunction);
                    if (hashSalt)
                        chunkHash.update(hashSalt);
                    chunk.updateHash(chunkHash);
                    chunk.hash = chunkHash.digest(hashDigest);
                    chunk.renderedHash = chunk.hash.substr(0, hashDigestLength);
                })
            });
        }
    }
]

Basically the new content for the chunkHash is added by the chunk itself and we only do this to the runtime chunk, the hashoptions set by the user in output are neatly honored.

This keeps the runtime chunk stable by re-examining what the actual content that would be written to disk would be, and as there is nothing identifiable/volatile in there it stays constant.

The hash will update when:

• The runtime code changes (e.g. webpack update)
• Chunk modules besides runtime get added/changed/removed

As we don't have to recursively change hashes because everything stays stable or is externalized into the manifest just updating this single chunk's hash is enough.

See link for related code in webpack
https://github.com/webpack/webpack/blob/master/lib/Compilation.js#L1284

[jouni-kantola]

@NinoFloris:
The initial issue is about another part of webpack's runtime being variable, not only the chunk manifest, but contents of installedChunks within the webpack runtime (AKA manifest/bootstrapping code). You mentioned

is byte for byte equal to a version with a different chunkhash because the file at the start used to be different

which to me reads that your runtime asset has same content after extracting the chunk manifest, but changes hash anyways. If I understood you correctly, then please create a new issue for this.

[NinoFloris]

Yes which is variable due to not using named chunks, as the only initial entry will ever be the runtime chunk, once you do that you get the next issue which is what I've posted here. all roads lead to this problem

[jouni-kantola]

I agree your issue should be fixed, @NinoFloris, and I'd happily take a PR, but it's not the same problem as the issue here (I'll rename this issue). The manifest asset could be made even stickier. You can repro by e.g. trying updating a static import to a dynamic import.

I've created a new issue where we can continue this discussion: #22

[NinoFloris]

Thanks for creating the other issue :)

Although I have to add, as I said, the installedChunks variable always has an initial value of

{ #runtimechunkid#: 0 }

In the normal case this is a number and it can/will change based on how many chunks you have, whereas in my case its says "runtime" because we use named chunks and it's therefore stable.

This means as long as the runtime chunk code doesn't change, I'll have exactly the same output every build.

If @huangshuwei would add

    new webpack.NamedModulesPlugin(), //optional-ish
    new webpack.NamedChunksPlugin(),

To his plugins like @ufologist linked to, then he'll be in the same boat as where I was

more detail: Predictable long term caching with Webpack

[jouni-kantola]

@NinoFloris: Thank you for your perseverance 🥇 NamedChunksPlugin had gone me by completely. That works great. I was so stuck with the old patterns of using only HashedModuleIdsPlugin and NamedModulesPlugin.

But still, I'm not sure NamedChunksPlugin would be needed if this issue was fixed, i.e. to extract the runtime's chunk ID (like with the chunk manifest).

[NinoFloris]

Basically that happens here https://github.com/webpack/webpack/blob/8b0a2ad2b3298372bf09ec22017e373625f5b06a/lib/JsonpMainTemplatePlugin.js#L12 I'm not sure though if you can just remove that entry, something has to bootstrap the bootstrapper basically in this case, that's why the initial entry is the runtime chunk.