Skip to content
This repository has been archived by the owner on Aug 13, 2024. It is now read-only.

Latest commit

 

History

History
76 lines (54 loc) · 2.51 KB

README.md

File metadata and controls

76 lines (54 loc) · 2.51 KB
Raw

License Coverage Status Test Docker

Snowplow ClickHouse Loader

Quickstart

Assuming Docker is installed:

  1. Run the ClickHouse server
$ docker run -d -p 8123:8123 \
    --name some-clickhouse-server \
    --ulimit nofile=262144:262144 \
    --volume=$HOME/clickhouse_db_vol:/var/lib/clickhouse yandex/clickhouse-server
  1. Start the client shell:
$ docker run -it \
    --rm \
    --link some-clickhouse-server:clickhouse-server yandex/clickhouse-client \
    --host clickhouse-server
  1. Make sure your database is created (tutorial in this example). You can keep working in this session to check the loaded data:
:) CREATE DATBASE IF NOT EXISTS tutorial
:) USE tutorial
  1. Run the Loader (config implies tutoral DB and some enriched data on local FS):
$ sbt
> run --config config/config.local.minimal.hocon --resolver config/resolver.json

It will automatically load the data from your local filesystem in batches of 1000 (configurable in the code)

Copyright and License

Snowplow ClickHouse Loader is copyright 2021 Snowplow Analytics Ltd.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.