Project always builds because of RequireJS.json

[gregveres]

Has anybody noticed that their project is always out of date and rebuilds every time you do something like run a unit test?

I have a project that is doing this. it takes 18 seconds just sitting there figuring out what to do before doing it, which really disrupts my flow. I got tired of it today and decided to look into why it is happening. If you go into VS and select Tools / Options then select Projects and Solutions and pick Build and Run, you can change the MSBuild project build output verbosity to Diagnostic.

Then the next time you build the first line that will show up in the output window will tell you why the project is building. This is what I get as the first line of my project that includes RequireJSDotNet:

1>Project 'XXX' is not up to date. Project item 'C:\Users\gregv...\RequireJS.json' has 'Copy to Output Directory' attribute set to 'Copy always'.

Ok, I guess when something is flagged as Copy Always, it means that the build system is going to do a build pass all the time.

so I decided to change Copy to Output Directory to "Copy if newer". Then I started to get:

1>Project 'XXX' is not up to date. Copy to output file 'C:\Users\gregv...\bin\RequireJS.json' missing from output location.

no matter how many times I run the build I always get this. I have the Build Action set to RequireJsNetConfig. I assume this build action does not copy the .json file to the output directory.

Do I have something configured wrong? Or is this just an oversight and a bug?

Thanks
Greg

[gregveres]

If I change the copy to output to "only if newer" and then go to the file system and manually copy the file from the source directory to the output directory and do another build (no file changes so there is nothing to build), the build finishes immediately, which is what I am looking for.

[gregveres]

Any thoughts here?
I have set the file's properties to copy if newer, but now it doesn't copy the file at all and it doesn't delete the file during a build clean. But I have to keep it there or the project is never up to date and always builds. I couldn't see in the project where the build action code is located. Any suggestions on where to look?

[gregveres]

@aquamoth

can somebody point me to where the build action of RequireJsNetConfig code is located? I really need to fix this issue. Right now the project always builds or it never copies the changed RequireJS.json file to the output.

Thanks

[aquamoth]

RequireJsNet.Compressor.RequireCompressorTask is the entrypoint you are looking for I believe.
If is defined as a task in the project file by <UsingTask ... /> and then activated by the target named "RequireJsNetMinify" with project-specific settings.

I have moved away from using the compressor myself in favor of calling requirejs/bin/r.js from a gulp-task. The compressor gives me too much problems and the scripted approach fits my evolved build process better. Still using RequireJS.Net for bootstrapping the runtime though.

So, good luck fixing the compressor task.

[gregveres]

Hi @aquamoth

I am about to do the same thing. I found that recently our compression stopped working and when I went to fix it, I was getting very random behaviour. Without changing the source code, building the project once would get everything working, then building the second time (no changes, just build again) would cause it to fail. I was finding the 'paths' section of the require config would contain large lists of my modules with no path specified. require.js gets angry with this for good reason. But then if I built again, it would work and all the modules would have valid paths again.

So I am also going to move to gulp and r.js. Do you have a particular tutorial that you followed that worked well? I am working through this one https://www.atlascode.com/blog/advanced-requirejs-bundling-typescript/ but it is not MVC based. I am hoping that doesn't matter.

[gregveres]

@aquamoth
After spending the day looking at gulp and the various options there, I think I have come down to creating a common bundle and then a bundle per page (ie, per controller action). I think I am going to have gulp start with the files in Controllers/Root//.ts. If I give r.js each one of those files, I should get out a minimized version of .ts that includes all the dependencies. I think I just need to load that minimized file because it will contain all of the dependent modules.

But this is where RequireJSNet gets in my way. RequireJSNet outputs something like this:

<script>
var requireConfig = ...
var require = ...
</script>
<script src="path to require.js">
<script>
require(["controllers/root/controller/action"]);
</script>

That's great. I need all of that and it works very well. But I think I need it to have two script tags inserted in the middle:

<script>
var requireConfig = ...
var require = ...
</script>
<script src="path to require.js"></script>
<script src="path to common bundle"></script>
<script src="path to minimized controller/action file"></script>
<script>
require(["controllers/root/controller/action"]);
</script>

Did you approach it this way? If so, how did you get those two script tags inserted?
Or are you approaching it another way?

I guess another though would be to have the bundler of RequireJSNet some how insert a path component to where to find the controller/action file and have it find the minimized file. I will think about how I could do that... I see there is an extension point for the require options. Maybe I can insert it there. If I could get, say jquery mapped to the common file and the controller/action mapped to the minimized controller/action file in the requires = line, that might do just the trick.

[aquamoth]

I've pulled all my build logic into a Gist so you can look at it.

It's a lot of stuff and some is just not useful for your usecase, but I think I've included everything relevant for the compression and requirejs loading to work.

I started by showing how I get the correct gulp-task to fire automatically after each build, by adding a post-build command.

In gulpfile.js you probably want to checkout the task "js:bundle". That's the one that initiates all bundling.

Finally I add the PartialView in "Views__Shared___RequireJS.cshtml" to my layout file, in which I call into RequireJS.Net for each page load. RequireJS.Net then makes sure the correct bundles gets loaded by loading the RequireJS.json and RequireJS.override.json.

Let me know what parts are most confusing and I'll try to help you out where I can.

[gregveres]

Thank you @aquamoth. I am looking at it now.

I spent half of today trying to get gulp to bundle but I couldn't get it to find the files I was requiring and in my digging around I got persuaded to move on to webpack. But now that I am digging in there, I am not sure that it will straightforward to move to webpack. My app is brandable per customer and webpack doesn't seem to handle that well so I would not be using webpack for CSS but all the examples assume you are using it for css.

Anyway, I am off to read your gist. Thank you!

[gregveres]

This is how you specify what goes into your bundles right - with the contents of the imports in these files:

scripts__bundle_roots__layout_f6.ts
scripts__bundle_roots__stocks_bundle.ts

Do you ever set the bundling enabled app setting that gets put into the LoadOverrides config of the RequireJSNet? Never mind, I see that you are using gulp to write out the overrides yourself. See my note below about this...

And finally, are you bundling just the common files? if I am following the flow properly you have:

Gulp runs bundle on rjs_optimize_runner. That does a start on three bundles, layout_f6, auctions and stock, with two further bundles commented out.

Now for the note about the overrides file. After I discovered that our bundling had stopped working, I started digging into it more. Another developer did the initial work earlier this year to get bundling going for us so I didn't know the details. But one thing that jumped out at me was the size of the overrides file. It is huge because each and every TS file in our project is listed in that file specifying which bundle it belongs to. The size of our override file was 14KB alone.

Is it really needed? I thought that requireJS built up an internal list of modules that have been defined, so as long as you include the bundles before you request a module from the bundle then requireJS knows where to get the module. Am I wrong on that?

Anyway, thank you for sharing, that was super helpful. I might come back to this if webpack falls through. At this point, I think webpack will produce a better solution than requireJS as long as I can keep it contained to just the TS files and not include all the css as well.

[aquamoth]

I really want to move into webpack land myself one day, but am unwilling to put in the time right now.As you say, the override file only really needs to point to a bundle for the first bundled file to load; loading the bundle automatically fills the hashmap with loaded scripts.I had even forgot I write out the override file myself. I wrote that because I didnt get things to work any other way.. I feel my solution is a hack, but its been stable for a long time now and handles added files pretty well.Good luck though!Den 3 dec. 2017 21:31 skrev gregveres <[email protected]>:This is how you specify what goes into your bundles right - with the contents of the imports in these files: scripts__bundle_roots__layout_f6.ts scripts__bundle_roots__stocks_bundle.ts Do you ever set the bundling enabled app setting that gets put into the LoadOverrides config of the RequireJSNet? Never mind, I see that you are using gulp to write out the overrides yourself. See my note below about this... And finally, are you bundling just the common files? if I am following the flow properly you have: Gulp runs bundle on rjs_optimize_runner. That does a start on three bundles, layout_f6, auctions and stock, with two further bundles commented out. Now for the note about the overrides file. After I discovered that our bundling had stopped working, I started digging into it more. Another developer did the initial work earlier this year to get bundling going for us so I didn't know the details. But one thing that jumped out at me was the size of the overrides file. It is huge because each and every TS file in our project is listed in that file specifying which bundle it belongs to. The size of our override file was 14KB alone. Is it really needed? I thought that requireJS built up an internal list of modules that have been defined, so as long as you include the bundles before you request a module from the bundle then requireJS knows where to get the module. Am I wrong on that? Anyway, thank you for sharing, that was super helpful. I might come back to this if webpack falls through. At this point, I think webpack will produce a better solution than requireJS as long as I can keep it contained to just the TS files and not include all the css as well. —You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

[gregveres]

I see a path forward moving off of RequireJS and on to webpack at this point. The thing that gives me great hope is that one of the documentation pages for webpack talks about the use case of a common bundle and then a bundle per view.
https://webpack.js.org/concepts/entry-points/

I have decided that I am going to start small with a purely manual configuration and once i get that going well, I will live with it for a while to see how things evolve. I think this is how it is going to turn out:

the page options are going to turn into a script tag at the bottom of the view rather than being declared in C# at the top of the view. That's not really a big change.
Webpack will have a list of view entry points in the config file and I will need to add a script tag at the bottom of the view.
At that point I think things would be working.

If you are interested here are some of the articles I am reading and following:
HowTo-gist (general knowledge)
https://github.com/petehunt/webpack-howto

using webpack with .Net-core. I am not using .Net-core but most of it is still relevant. https://dotnetcore.gaprogman.com/2017/01/05/bundling-in-net-core-mvc-applications-with-webpack/

I started with this one but found that it seemed overly complicated without hitting my use case, but there were some good things form it:
https://medium.com/@jonjam/combining-webpack-with-asp-net-mvc-5-a5bd07c49d0b

I ended up using his github project as my test bed to explore the concepts. This was where I found that you can use [chunkhash] as part of the output file name to get the cache busting strategy. I liked your strategy where the version portion of the request changed each request. There are so many times I have to use the version of refresh that wipes out the cache.

This showed me how to include the bundled files that have the hash on them.
http://michaco.net/blog/Angular4GettingHashedWebpackBundlesWorkingInASPNETCoreMVC
or this one that uses some extra bits to get the exact file name. This looks like it might a "down the road" addition:
https://scottaddie.com/2015/12/14/a-practical-approach-to-cache-busting-with-webpack-and-asp-net-5/

There is a way that webpack can modify your _layout file by calling it _layout_template.cshtml, but I found that it included all the bundles on every page for some reason, which isn't at all the right thing. And this happened automatically when the layout file had the _template added to it. So using the above mechanism of wild carding out the hash was interesting.

this one was interesting to read because you will want to setup webpack-notifier
https://developer.telerik.com/featured/webpack-for-visual-studio-developers/

I think I have enough research done now that I can try this out. If you want, I can create gist with my results if and when I get it working.

[aquamoth]

Thanks for the info. I’ll put it in my backlog for now, but I’m grateful for all the links. If you feel up to it when you are done, a gist of a working concept is always helpful. It seems we have similar requirements. 😊 Finally, I think your idea of starting small and manual and then grow the configuration is spot on. Also, I really try to stay away from any automagic fixing, so I’ll steer very clear of that _template “feature” if I can. 😊 Från: gregveres [mailto:[email protected]] Skickat: den 4 december 2017 06:21 Till: vtfuture/RequireJSDotNet <[email protected]> Kopia: Mattias Åslund <[email protected]>; Mention <[email protected]> Ämne: Re: [vtfuture/RequireJSDotNet] Project always builds because of RequireJS.json (#98) I see a path forward moving off of RequireJS and on to webpack at this point. The thing that gives me great hope is that one of the documentation pages for webpack talks about the use case of a common bundle and then a bundle per view. https://webpack.js.org/concepts/entry-points/ I have decided that I am going to start small with a purely manual configuration and once i get that going well, I will live with it for a while to see how things evolve. I think this is how it is going to turn out: the page options are going to turn into a script tag at the bottom of the view rather than being declared in C# at the top of the view. That's not really a big change. Webpack will have a list of view entry points in the config file and I will need to add a script tag at the bottom of the view. At that point I think things would be working. If you are interested here are some of the articles I am reading and following: HowTo-gist (general knowledge) https://github.com/petehunt/webpack-howto using webpack with .Net-core. I am not using .Net-core but most of it is still relevant. https://dotnetcore.gaprogman.com/2017/01/05/bundling-in-net-core-mvc-applications-with-webpack/ I started with this one but found that it seemed overly complicated without hitting my use case, but there were some good things form it: https://medium.com/@jonjam/combining-webpack-with-asp-net-mvc-5-a5bd07c49d0b I ended up using his github project as my test bed to explore the concepts. This was where I found that you can use [chunkhash] as part of the output file name to get the cache busting strategy. I liked your strategy where the version portion of the request changed each request. There are so many times I have to use the version of refresh that wipes out the cache. This showed me how to include the bundled files that have the hash on them. http://michaco.net/blog/Angular4GettingHashedWebpackBundlesWorkingInASPNETCoreMVC or this one that uses some extra bits to get the exact file name. This looks like it might a "down the road" addition: https://scottaddie.com/2015/12/14/a-practical-approach-to-cache-busting-with-webpack-and-asp-net-5/ There is a way that webpack can modify your _layout file by calling it _layout_template.cshtml, but I found that it included all the bundles on every page for some reason, which isn't at all the right thing. And this happened automatically when the layout file had the _template added to it. So using the above mechanism of wild carding out the hash was interesting. this one was interesting to read because you will want to setup webpack-notifier https://developer.telerik.com/featured/webpack-for-visual-studio-developers/ I think I have enough research done now that I can try this out. If you want, I can create gist with my results if and when I get it working. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#98 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AFNKN6dhBnSwHqe0NTxgVx1wdYiqIxIMks5s84FTgaJpZM4PH2Ox> . <https://github.com/notifications/beacon/AFNKNzQ9KBlsYh6d2Diwi07eFCb0Ze6jks5s84FTgaJpZM4PH2Ox.gif>

[gregveres]

Well, I am now a day into implementation and I can tell you that getting the config right is only half the battle. I am moving everything from NuGet packages over to npm packages to make webpack happy and it is a mess.

I use knockout and I am currently fighting with a couple knockout binding packages that are defined correctly. And I see that signalR is going to be a problem.

I also use typescript and since I had to switch from the classic moduleResovler to the node resolver, I had to touch over 250 files to change the paths to files they import. If that was the worst of it, I would be a happy guy. I have been documenting everything I am doing and will share it when I am done.

[gregveres]

@aquamoth

Well 7 days later and I have 5 pages converted over to use webpack and the solution builds and deploys via VSTS. Whew. That was 7 days of doing nothing but this. I think it is worth it though - I didn't have a bundling solution that worked. Our RequireJSNet solution stopped working and I didn't like the way it bundled so this was time well spent.

I know have a good pattern to use for my new pages and a backlog task to convert over the 40ish pages that haven't been converted.

I documented everything I did in OneNote as I went, including links to articles where I found information. If you want I can send it to you. I just need to know how to get it to you. I would also bundle up the scripts that I came up with.

I