-
Notifications
You must be signed in to change notification settings - Fork 37
Setting up a development Environment
We support Mac OS X, Windows and Linux. You need to be comfortable with command-line tools.
- Install dependencies:
git clone https://github.com/overview/overview-server.git
cd overview-server
./dev
That last command will take a long time the first time you run it -- maybe hours -- as it downloads and compiles components. When the text stops, it's ready.
You should also set up integration tests. Install Node LTS so you can run those. (Test that node --version
is greater than 8
and that npm --version
does not produce an error.)
You can set the OVERVIEW_MULTI_USER flag to false (see configuration) to bypass all login and user management. Or you can use the default admin login, [email protected]
/[email protected]
. Or you can register a new user. Overview won't actually email you, but you can read the email it would have sent on the console where you ran ./dev
.
A fresh development install does not have plugins configured (Word cloud, Entities, etc.) To get these to appear in the Add View menu, you need to do two things:
- Add the plugins from the Admin menu, giving a URL and description for each. For a normal development environment, the url will be
http://localhost:[port]
where port matches the numbers in plugins.yml - Then start the plugins running with
docker-compose -f plugins.yml up
Overview is a hive of servers. Here they are:
- PostgreSQL: stores our data about documents. (Actual documents are stored elsewhere.)
- Redis: caches stuff.
-
overview-server: a Play web server that listens at http://localhost:9000, with source code in
web/
. If Play is running while you edit the source code (inweb/
), Play will automatically recompile the next time you load a page. -
worker: a plain Java program, with source code and tests in
worker/
. It won't restart when you edit code -- at least, not by default. -
plugins: Each plugin is in fact a tiny web server. To start them all,
docker-compose -f plugins.yml up
If you're going to fiddle, you should probably run each command in its own console.
-
./sbt web/test
: Compiles and unit-tests the code inweb/
. -
./sbt '; project web; ~testQuick'
: Useful when coding inweb/
. Recompiles and re-tests, every time a file changes. It's equally great for theworker
andcommon
projects.- Beware: if you run
~testQuick
whiledev
is also hosting a web server, you can hit a race:dev
will recompile when you refresh the web page, and~testQuick
will recompile when you save a file. They'll step on each other's toes and one will crash. Train yourself to wait for./sbt ~testQuick
to finish its work before you refresh the page.
- Beware: if you run
-
./sbt db-evolution-applier/run
: runsdb-evolution-applier/src/main/resources/migration/*.sql
on the development database. For the test database, use./sbt test-db-evolution-applier/run
. -
./auto/test-coffee.sh
: continually re-runs JavaScript unit tests inweb/test/js/
. -
./auto/test-coffee-once.sh
: just runs the tests once -
./build overview-server.zip
: creates a build,overview-server.zip
. -
./auto/psql.sh
and./auto/psql.sh overview-test
: log in to theoverview-dev
andoverview-test
databases, respectively. PostgreSQL SQL Documentation
All these commands will transparently run inside a Docker container. That means you won't need to install any dependencies.
Everything goes in Docker volumes. You can see those volumes in docker-compose.yml
. Here are the Docker volumes we use:
- Document Sets and document metadata go in
database
(Postgres) - File contents (raw uploaded files and ready-to-view PDFs) go in
blob-storage
(flat files; in production it's S3) - Search indexes (one per document set) go in
search
(flat files) - Build stuff goes in
homedir
To make Overview stop, hit Ctrl+C
. If you want everything destroyed, add the scary option -v
to the non-scary command, docker-compose down
. The -v
option will delete all Overview data. You've been warned.
-
docker ps
shows running Docker containers. (A "container" is basically a Linux process. Even if you're on Windows or Mac.) -
docker ps -a
shows running and "stopped" Docker containers. -
auto/docker-exec.sh bash
will runbash
inside the already-running "dev" container (which you run by callingdocker-compose up
) -
auto/docker-run.sh bash
will runbash
inside a fresh "dev" container. This will have the same persisted data (built files, search indexes, etc) as the running "dev" container, but it won't be able to "see" the running "dev" container.
To run just the web server, run ./sbt run
. It will restart every time you change a file in app/
, test/
, or common/src/main
and then reload.
To run unit tests every time you change a file in app/
or test
/, run ./sbt '; project web; ~testQuick'
.
Beware: if you run both commands simultaneously, sbt
will destroy its own files, leading to errors. Train yourself to wait for ~testQuick
to finish its work before you refresh the page.
To run just the worker, run ./sbt worker/re-start
. It will restart every time you change a file in worker/src/main
or common/src/main
. (You'll probably want to have docker-compose up -d database redis
running. You might not want the web server running.)
To run unit tests every time you change a file in worker/src
or common/src/main
, run ./sbt '; project worker; ~testQuick'
.
Beware: if you run worker/re-start
and ; project worker; ~testQuick
simultaneously, sbt
will destroy its own files, leading to errors. Run one and then the other. Or don't auto-restart the worker; just run ./sbt worker/run
instead.
To run unit tests every time you change a file in common/src
, run ./sbt '; project common; ~testQuick'
.
Run auto/test-coffee.sh
to auto-test all CoffeeScript/JavaScript code any time it changes. The code is in app/assets/javascripts
and the tests are in test/assets/javascripts
.
Run integration-tests/run
to test everything. This expects ./dev
to be running.