node run and build take a lot of memory

[fnoop]

yarn build on a 2gb VM quickly exhausts the memory and crashes the VM.
yarn run uses 1Gb+ rss in usage.

yarn build produces a 38Mb output.

Need to reduce all.

[fnoop]

[dev] [mav@maverick-ubuntuvm ~/code/maverick-web]$ yarn build
yarn run v1.22.4
$ vue-cli-service build

⠋  Building for production...To enable GQL files in ESLint, set the `pluginOptions.apollo.lintGQL` project option to `true` in `vue.config.js`. Put `false` to hide this message.
You also need to install `eslint-plugin-graphql` and enable it in your ESLint configuration.
⠙  Building for production...

 WARNING  Compiled with 3 warnings                                                                                                                                       1:42:05 PM

 warning

asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
  html.worker.js (531 KiB)
  css.worker.js (809 KiB)
  ts.worker.js (3.48 MiB)
  js/app.49dd4eba.js (406 KiB)
  css/chunk-vendors.6ad3efae.css (425 KiB)
  js/chunk-vendors.89771c94.js (2.42 MiB)

 warning

entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  app (3.23 MiB)
      css/chunk-vendors.6ad3efae.css
      js/chunk-vendors.89771c94.js
      css/app.25f78e49.css
      js/app.49dd4eba.js


 warning

webpack performance recommendations:
You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application.
For more info visit https://webpack.js.org/guides/code-splitting/

  File                                   Size              Gzipped

  dist/ts.worker.js                      3560.79 KiB       807.00 KiB
  dist/js/chunk-vendors.89771c94.js      2478.62 KiB       680.47 KiB
  dist/css.worker.js                     808.92 KiB        173.63 KiB
  dist/html.worker.js                    531.22 KiB        131.95 KiB
  dist/js/app.49dd4eba.js                406.37 KiB        78.69 KiB
  dist/json.worker.js                    231.95 KiB        65.83 KiB
  dist/editor.worker.js                  124.35 KiB        39.11 KiB
  dist/css/chunk-vendors.6ad3efae.css    424.82 KiB        48.92 KiB
  dist/css/app.25f78e49.css              1.87 KiB          0.70 KiB

  Images and other types of assets omitted.

 DONE  Build complete. The dist directory is ready to be deployed.
 INFO  Check out deployment instructions at https://cli.vuejs.org/guide/deployment.html

Done in 98.28s.

[fnoop]

After disabling monaco editor:

[dev] [mav@maverick-ubuntuvm ~/code/maverick-web]$ yarn build
yarn run v1.22.4
$ vue-cli-service build

⠋  Building for production...To enable GQL files in ESLint, set the `pluginOptions.apollo.lintGQL` project option to `true` in `vue.config.js`. Put `false` to hide this message.
You also need to install `eslint-plugin-graphql` and enable it in your ESLint configuration.
⠏  Building for production...

 WARNING  Compiled with 3 warnings                                                                                                                                       1:58:57 PM

 warning

asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
  js/app.e7546c08.js (406 KiB)
  css/chunk-vendors.6ad3efae.css (425 KiB)
  js/chunk-vendors.89771c94.js (2.42 MiB)

 warning

entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  app (3.23 MiB)
      css/chunk-vendors.6ad3efae.css
      js/chunk-vendors.89771c94.js
      css/app.25f78e49.css
      js/app.e7546c08.js


 warning

webpack performance recommendations:
You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application.
For more info visit https://webpack.js.org/guides/code-splitting/

  File                                   Size              Gzipped

  dist/js/chunk-vendors.89771c94.js      2478.62 KiB       680.47 KiB
  dist/js/app.e7546c08.js                406.37 KiB        78.69 KiB
  dist/css/chunk-vendors.6ad3efae.css    424.82 KiB        48.92 KiB
  dist/css/app.25f78e49.css              1.87 KiB          0.70 KiB

  Images and other types of assets omitted.

 DONE  Build complete. The dist directory is ready to be deployed.
 INFO  Check out deployment instructions at https://cli.vuejs.org/guide/deployment.html

Done in 60.54s.

Massive difference

[fnoop]

Disabling xtermjs, cesium and layers reduces significantly further.

[dev] [mav@maverick-ubuntuvm ~/code/maverick-web]$ yarn build
yarn run v1.22.4
$ vue-cli-service build

⠋  Building for production...To enable GQL files in ESLint, set the `pluginOptions.apollo.lintGQL` project option to `true` in `vue.config.js`. Put `false` to hide this message.
You also need to install `eslint-plugin-graphql` and enable it in your ESLint configuration.
⠸  Building for production...

 WARNING  Compiled with 3 warnings                                                                                                                                       2:29:48 PM

 warning

asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
  js/app.ffeb3ad4.js (391 KiB)
  css/chunk-vendors.e5386d39.css (421 KiB)
  js/chunk-vendors.d5948740.js (1.05 MiB)

 warning

entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  app (1.85 MiB)
      css/chunk-vendors.e5386d39.css
      js/chunk-vendors.d5948740.js
      css/app.a4e06123.css
      js/app.ffeb3ad4.js


 warning

webpack performance recommendations:
You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application.
For more info visit https://webpack.js.org/guides/code-splitting/

  File                                   Size              Gzipped

  dist/js/chunk-vendors.d5948740.js      1078.97 KiB       321.25 KiB
  dist/js/app.ffeb3ad4.js                391.36 KiB        75.21 KiB
  dist/css/chunk-vendors.e5386d39.css    420.99 KiB        47.75 KiB
  dist/css/app.a4e06123.css              1.00 KiB          0.47 KiB

  Images and other types of assets omitted.

 DONE  Build complete. The dist directory is ready to be deployed.
 INFO  Check out deployment instructions at https://cli.vuejs.org/guide/deployment.html

Done in 48.60s.

[fnoop]

Disabling store/mdoules/schema.js and plugins/mavlink/mavlinkEnumData.json reduces the main app.js a massive amount:

[dev] [mav@maverick-ubuntuvm ~/code/maverick-web]$ yarn build
yarn run v1.22.4
$ vue-cli-service build

⠋  Building for production...To enable GQL files in ESLint, set the `pluginOptions.apollo.lintGQL` project option to `true` in `vue.config.js`. Put `false` to hide this message.
You also need to install `eslint-plugin-graphql` and enable it in your ESLint configuration.
⠼  Building for production...

 WARNING  Compiled with 3 warnings                                                                                                                                                                                            2:51:36 PM

 warning

asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
  css/chunk-vendors.25d3568b.css (421 KiB)
  js/chunk-vendors.d5948740.js (1.05 MiB)

 warning

entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  app (1.64 MiB)
      css/chunk-vendors.25d3568b.css
      js/chunk-vendors.d5948740.js
      css/app.a4e06123.css
      js/app.b43587e9.js


 warning

webpack performance recommendations:
You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application.
For more info visit https://webpack.js.org/guides/code-splitting/

  File                                   Size              Gzipped

  dist/js/chunk-vendors.d5948740.js      1078.97 KiB       321.25 KiB
  dist/js/app.b43587e9.js                182.45 KiB        30.89 KiB
  dist/css/chunk-vendors.25d3568b.css    420.99 KiB        47.75 KiB
  dist/css/app.a4e06123.css              1.00 KiB          0.47 KiB

  Images and other types of assets omitted.

 DONE  Build complete. The dist directory is ready to be deployed.
 INFO  Check out deployment instructions at https://cli.vuejs.org/guide/deployment.html

Done in 36.29s.

[fnoop]

Removed vuelayers (still being called by Planner2d component)

[dev] [mav@maverick-ubuntuvm ~/code/maverick-web]$ yarn build
yarn run v1.22.4
$ vue-cli-service build

⠋  Building for production...To enable GQL files in ESLint, set the `pluginOptions.apollo.lintGQL` project option to `true` in `vue.config.js`. Put `false` to hide this message.
You also need to install `eslint-plugin-graphql` and enable it in your ESLint configuration.
⠋  Building for production...

 WARNING  Compiled with 3 warnings                                                                                                                                                                                            3:01:25 PM

 warning

asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
  css/chunk-vendors.70c99fb0.css (421 KiB)
  js/chunk-vendors.658436e2.js (920 KiB)

 warning

entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  app (1.47 MiB)
      css/chunk-vendors.70c99fb0.css
      js/chunk-vendors.658436e2.js
      css/app.7f98b729.css
      js/app.a6cde628.js


 warning

webpack performance recommendations:
You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application.
For more info visit https://webpack.js.org/guides/code-splitting/

  File                                   Size              Gzipped

  dist/js/chunk-vendors.658436e2.js      920.08 KiB        275.91 KiB
  dist/js/app.a6cde628.js                163.09 KiB        26.57 KiB
  dist/css/chunk-vendors.70c99fb0.css    420.99 KiB        47.72 KiB
  dist/css/app.7f98b729.css              0.58 KiB          0.32 KiB

  Images and other types of assets omitted.

 DONE  Build complete. The dist directory is ready to be deployed.
 INFO  Check out deployment instructions at https://cli.vuejs.org/guide/deployment.html

Done in 36.98s.
[dev] [mav@maverick-ubuntuvm ~/code/maverick-web]$

[fnoop]

@SamuelDudley - i'm going to permanently remove monaco from the main vue.config.js because it increases the size of the build by about 10x!
We can put it back by looking at lazy loading or at a vue component eg:
https://github.com/egoist/vue-monaco
The component approach will hopefully automatically include it in a more efficient manner.

[SamuelDudley]

@SamuelDudley - i'm going to permanently remove monaco from the main vue.config.js because it increases the size of the build by about 10x!
We can put it back by looking at lazy loading or at a vue component eg:
https://github.com/egoist/vue-monaco
The component approach will hopefully automatically include it in a more efficient manner.

So considering -web will be served mainly over a local network, others are not likely going to use -web rather -web-dist and we have a way of reliability building -web-dist from -web do you think the bundle size actually matters?

[SamuelDudley]

I should add I'm okay with removing stuff, but I want to maintain a branch with it included so that the hours of work to date are not lost to bitrot.

[SamuelDudley]

Example chunking and removal of unused features
microsoft/monaco-editor-webpack-plugin#97

[SamuelDudley]

Links to some tools to help with bundle analysis and code splitting https://github.com/kristerkari/awesome-bundle-size

https://github.com/webpack-contrib/webpack-bundle-analyzer
https://github.com/robertknight/webpack-bundle-size-analyzer
https://github.com/samccone/bundle-buddy

[cglusky]

Rollup and parcel both support Vue and tree shaking.

[fnoop]

So considering -web will be served mainly over a local network, others are not likely going to use -web rather -web-dist and we have a way of reliability building -web-dist from -web do you think the bundle size actually matters?

When you're talking about a factor of 5-10 then I think it really matters - it's not just that you're loading 8-10mb javascript over the network, it's that your poor browser has to execute that 8-10mb javascript. It's why the ttfp was so slow. This is just as much about the size and speed of -web-dist than memory usage of -web.

I'm absolutely not talking about losing any of the work that either of us have done, just disabling it all except for the basics so we can see what is actually necessary. What's in master now is really the basic core of what we can get away with - vue, vuex, vue-router, vuetify, apollo, and we're down to about 1mb js. This can be further improved with treeshaking and chunking - certainly vue and vuetify can be improved significantly.

I've disabled the mavlink-related components (cockpit, planner in particular) which then removes the likes of vuelayers/openlayers, pixi.js, cesium, and disabled the more advanced config features like xtermjs and monaco. All of this is simply disabled btw, none of the code is gone.

Then, we can start to re-enable carefully, watching the bundle size and ttfp as we go along. I suggest that for the 1.2 release we keep all the mavlink stuff disabled and just concentrate on the config, maverick and video modules.

[SamuelDudley]

@cglusky I haven't used either before, certainly worth a look long term.

@fnoop Taking a step back, my thoughts are that -web is very different to a normal webpage, e.g. a web store that is used to sell mountain bikes.

Using the bike store example, loading and execution times are very important as delays in getting pictures rendered might be the difference in a sale vs a user giving up and looking somewhere else. In the case of maverick-web users are much more heavily invested in the whole maverick / autonomous system ecosystem and will happily deal with much longer start-up times.
Additionally many of the use cases are over local networks where throughput is much less of an issue. Lastly, once its all cached in the browser, the network issue is now gone.

I agree with what you are doing and your plan. I just wanted to ensure that we don't limit ourselves for the sake of a small bundle size and super fast ttfp.

On the ttfp: In vue-vanilla-cesium I enabled chunking of cesium (https://github.com/SamuelDudley/vue-vanilla-cesium/blob/master/vue.config.js#L81) and lazy loading of the web endpoint that used cesium in the vue router (https://github.com/SamuelDudley/vue-vanilla-cesium/blob/master/src/router/index.ts#L12)

The combination of those two optimizations means that the application loads the login screen instantly and then starts parallel background requests to pull down the cesium js bundle as the user types into the login form.
We could look into doing something similar for the maverick-web landing page and prioritizing what chunks / pages are loaded based on what a user is likely to first interact with.

[fnoop]

Using the bike store example, loading and execution times are very important as delays in getting pictures rendered might be the difference in a sale vs a user giving up and looking somewhere else. In the case of maverick-web users are much more heavily invested in the whole maverick / autonomous system ecosystem and will happily deal with much longer start-up times.

Yes I completely agree. I just think it had got a bit out of hand and wanted to get back to basics and get that right before going forward!

[SamuelDudley]

see https://github.com/goodrobots/maverick-web/tree/ttfp for changes to reduce ttfp

edit: we will need to update the icon names as this requires moving to material design icons:
https://material.io/resources/icons/?icon=error_outline&style=outline

[SamuelDudley]

most of the delay in ttfp when viewing the webpage via webdev is the hot reload server itself. When viewing the dist ttfp is quite snappy.

[SamuelDudley]

have a look at https://goodrobots.github.io/maverick-web/ to get a feel for the current page load speed.
With the code splitting in place it would be interesting to re-enable some of the features and see if it has helped.

[SamuelDudley]

Some testing with a new vue-cli repo leads me to believe that vuetify is already being tree shaken.

[SamuelDudley]

looks like all the css is bundled

[fnoop]

Before enabling xterm, a yarn build takes about max 700mb memory, size of build:
app (1.61 MiB)

Built web page definitely much faster to load. It has now chunked into dozens of files which means lots of http connections. Also lots of warnings in the build process so needs tidying up a bit.

[fnoop]

After xterm, yarn build still takes about 700mb, size of build: app (1.61 MiB). Odd.

[SamuelDudley]

I think my chunking strategy is a bit aggressive currently. We can tweak it to make less, larger chunks.

[fnoop]

There is a brilliant report built and pushed to -web-dist:
http://maverick-raspberrylite.local/report.html

The chunking is working great - it's much better than without. If we can reduce the core chunking in the future that would be great.

[SamuelDudley]

Sorry, I thought you were aware of that report feature! As you can see its chunking each npm package individually. We can keep this logic in place and add a few hand crafted rules to merge a lot of the smaller bits of common code.