How to speed up builds and tests in your CI environment

• What is taking most of the time in your builds? Cloning, building, testing?
• Improve cloning times by
  • Use a reference repository (under Advanced Clone Behaviors) so that you can point to a local base repo instead of copying the entire history for every project, you may also check out @larsxschneider's cached clone script that also caches submodules and Git LFS objects
  • Narrow refspec by selecting "Honor refspec on initial clone" in Advanced Clone Behaviors and specfying an appropriate refspec
  • Sparse checkout only the directories that are needed for the operation
  • reduced repository sizes by moving large binary files to Git LFS using bfg-repo cleaner
  • push based build triggering, + use ngrok or open a port if needed
  • get version control system server and artifact repository closer to the build server (maybe by hosting it yourself or moving the build server closer to the rest)
• Improve build times by
  • only building components once and transport them to all downstream build jobs and machines
  • using an artifact repository for shared components
  • SSDs, more cores and more memory
  • Optimized compiler switches - mvn client VM, parallel build options, memory options
  • use a build system that is caching build results globally and does build parallelization automatically
• Improve test times by
  • identifying which tests are still testing used functionality and disable the rest
  • rewriting integration and system tests to use more mocking (one process without IPC and file system interaction preferred)
  • analyze bottlenecks and silent exceptions using New Relic, Dynatrace, OverOps or just VisualVM
  • Break monolythic applications done into multiple components that can be tested in isolation (and only retested if they changed)
  • dedicate a person per team to improve test execution time
  • test partitioning across multiple machines using Jenkins Parallel Test Executor Plugin (this repository)
  • SSDs, more cores and more memory
  • learn more about Jenkins Pipelines, its job resuming functionalities and its GitHub Integration
  • if you like to use a pre-setup environment using Docker that integrates with GitHub.com, you can follow this tutorial.
  • pre-heating of integration test systems (e.g. by unfreezing memory snapshots of already fully up and running application server and database VMs)
  • only do smoke tests in build, all tests in nightly build (CI result is now also shown in code browser out of the pull request)
  • optimized compiler switches, parallel test runs (Maven 3 now supports this better), compile options like -XX:+TieredCompilation -XX:TieredStopAtLevel=1 for short running mvn invocations
  • use different commit status contexts for different tests, only make smoke test status checks mandatory
  • find the best tradeoff of number of tests/test duration and code coverage
  • use a build system that is automatically caching test results parallelizing tests (if code tested and test have not changed)

Interesting links

• Tools for Continuous Integration at Google Scale
• Facebook's Build Environment
• Speeding up builds with buck
• Culture as code
• Inner Source