Add couchbase govk

[peczenyj]

This PR add supports for couchbase using gocb api version 2

To be possible run the tests you have two options

1. starts couchbase and set the following env vars:

• COUCHBASE_SERVER must contains the connection string example couchbase://localhost
• COUCHBASE_BUCKET must contains the name of the bucket to test
• COUCHBASE_USERNAME & COUCHBASE_PASSWORD must contains the user and pass needed to access the couchbase

This can be done following this procedure using docker

2. by running the tests with a bigger timeout (most of the time) and using the integration tag like in the example below. This integration tests starts a local container using github.com/testcontainers/testcontainers-go (you can check here and here how it works.

$ go test -v -timeout=600s -tags=integration ./...
2024/05/29 14:34:30 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 26.1.3
  API Version: 1.44
  Operating System: Ubuntu 24.04 LTS
  Total Memory: 15806 MB
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
  Test SessionID: 272a8f169a1a0cac2aa669bf15c7affb5f0c82e8c53b6e656244dcda14340428
  Test ProcessID: 1a1ef910-978e-44d5-8557-2c6a73ac2160
2024/05/29 14:34:30 🐳 Creating container for image testcontainers/ryuk:0.7.0
2024/05/29 14:34:30 ✅ Container created: 84ddc12dec3d
2024/05/29 14:34:30 🐳 Starting container: 84ddc12dec3d
2024/05/29 14:34:30 ✅ Container started: 84ddc12dec3d
2024/05/29 14:34:30 🚧 Waiting for container id 84ddc12dec3d image: testcontainers/ryuk:0.7.0. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms}
2024/05/29 14:34:30 🔔 Container is ready: 84ddc12dec3d
2024/05/29 14:34:30 🐳 Creating container for image couchbase:community-7.1.1
2024/05/29 14:34:30 ✅ Container created: 54dda7abeb27
2024/05/29 14:34:30 🐳 Starting container: 54dda7abeb27
2024/05/29 14:34:31 ✅ Container started: 54dda7abeb27
2024/05/29 14:34:31 🔔 Container is ready: 54dda7abeb27
=== RUN   TestClient
    couchbase_test.go:20: checking for couchbase connection string on env var "COUCHBASE_SERVER"
=== RUN   TestClient/Test_client_with_codec_JSON
=== RUN   TestClient/Test_client_with_codec_Gob
--- PASS: TestClient (0.02s)
    --- PASS: TestClient/Test_client_with_codec_JSON (0.01s)
    --- PASS: TestClient/Test_client_with_codec_Gob (0.01s)
PASS
2024/05/29 14:34:57 🐳 Terminating container: 54dda7abeb27
2024/05/29 14:34:57 🚫 Container terminated: 54dda7abeb27
ok  	github.com/philippgille/gokv/couchbase	27.600s

if there is no env var COUCHBASE_SERVER set, the tests will be skipped.

[philippgille]

Hi 👋 , I had looked at testcontainers for gokv in the past (and seen it used elsewhere before that), and I don't remember what exactly it was, but for gokv there was some deficiency and I chose to go with this instead:

gokv/magefiles/test.go

Lines 41 to 135 in c954c54

	switch impl {
	case "cockroachdb":
	dockerImage = "cockroachdb/cockroach"
	dockerCmd += `cockroachdb -p 26257:26257 --health-cmd='curl -f http://localhost:8080/health?ready=1' --health-interval 1s ` + dockerImage + ` start-single-node --insecure`
	setup = func() error {
	var out string
	out, err = script.Exec(`docker exec gokv-cockroachdb bash -c './cockroach sql --insecure --execute="create database gokv;"'`).String()
	if err != nil {
	// Print the output here, as it could be more info than what's in err.
	fmt.Println(out)
	return err
	}
	return nil
	}
	case "consul":
	dockerImage = "bitnami/consul"
	dockerCmd += `consul -e CONSUL_LOCAL_CONFIG='{"limits":{"http_max_conns_per_client":1000}}' -p 8500:8500 --health-cmd='curl http://localhost:8500/v1/health/state/passing | grep serfHealth' --health-interval 1s ` + dockerImage
	healthExec = `docker exec -u root gokv-consul sh -c 'apt update && apt install --yes --no-install-recommends curl'`
	case "datastore": // Google Cloud Datastore via "Cloud Datastore Emulator"
	// Using the ":slim" or ":alpine" tag would require the emulator to be installed manually.
	// Both ways seem to be okay for setting the project: `-e CLOUDSDK_CORE_PROJECT=gokv` and CLI parameter `--project=gokv`
	// `--host-port` is required because otherwise the server only listens on localhost IN the container.
	dockerImage = "google/cloud-sdk"
	dockerCmd += `datastore -p 8081:8081 ` + dockerImage + ` gcloud beta emulators datastore start --no-store-on-disk --project=gokv --host-port=0.0.0.0:8081`
	case "dynamodb": // DynamoDB via "DynamoDB local"
	dockerImage = "amazon/dynamodb-local"
	dockerCmd += `dynamodb-local -p 8000:8000 ` + dockerImage
	case "etcd":
	dockerImage = "bitnami/etcd"
	dockerCmd += `etcd -p 2379:2379 --env ALLOW_NONE_AUTHENTICATION=yes --health-cmd='etcdctl endpoint health' --health-interval 1s ` + dockerImage
	case "hazelcast":
	dockerImage = "hazelcast/hazelcast"
	dockerCmd += `hazelcast -p 5701:5701 --health-cmd='curl -f http://localhost:5701/hazelcast/health/node-state' --health-interval 1s ` + dockerImage
	case "ignite":
	dockerImage = "apacheignite/ignite"
	dockerCmd += `ignite -p 10800:10800 --health-cmd='${IGNITE_HOME}/bin/control.sh --baseline | grep "Cluster state: active"' --health-interval 1s ` + dockerImage
	case "memcached":
	dockerImage = "memcached"
	dockerCmd += `memcached -p 11211:11211 --health-cmd='echo stats | nc -w 1 localhost 11211' --health-interval 1s ` + dockerImage
	healthExec = `docker exec -u root gokv-memcached sh -c 'apt update && apt install --yes --no-install-recommends netcat-openbsd'`
	case "mongodb":
	dockerImage = "mongo"
	dockerCmd += `mongodb -p 27017:27017 --health-cmd='echo "db.runCommand({ ping: 1 }).ok" | mongosh localhost:27017/test --quiet' --health-interval 1s ` + dockerImage
	case "mysql":
	dockerImage = "mysql"
	dockerCmd += `mysql -e MYSQL_ALLOW_EMPTY_PASSWORD=true -p 3306:3306 --health-cmd='mysqladmin ping -h localhost' --health-interval 1s ` + dockerImage
	case "postgresql":
	dockerImage = "postgres:alpine"
	dockerCmd += `postgres -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=gokv -p 5432:5432 --health-cmd='pg_isready -U postgres' --health-interval 1s ` + dockerImage
	case "redis":
	dockerImage = "redis"
	dockerCmd += `redis -p 6379:6379 --health-cmd='redis-cli ping' --health-interval 1s ` + dockerImage
	case "s3": // Amazon S3 via Minio
	dockerImage = "minio/minio"
	dockerCmd += `s3 -e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" -e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" -p 9000:9000 --health-cmd='mc ready local' --health-interval 1s ` + dockerImage + ` server /data`
	case "tablestorage": // Tablestorage via Azurite
	// In the past there was this problem: https://github.com/Azure/Azurite/issues/121
	// With this Docker image:
	// docker run -d --rm --name azurite -e executable=table -p 10002:10002 arafato/azurite
	// Now with the official image it still doesn't work. // TODO: Investigate / create GitHub issue.
	// docker run -d --rm --name azurite -p 10002:10002 mcr.microsoft.com/azure-storage/azurite azurite-table
	case "tablestore":
	// Currently no emulator exists for Alibaba Cloud Table Store.
	case "zookeeper":
	dockerImage = "zookeeper"
	dockerCmd += `zookeeper -p 2181:2181 -e ZOO_4LW_COMMANDS_WHITELIST=ruok --health-cmd='echo ruok | timeout 2 nc -w 2 localhost 2181 | grep imok' --health-interval 1s ` + dockerImage
	default:
	return errors.New("unknown `gokv.Store` implementation")
	}
	// TODO: until docker images for windows appear, skip those test for windows
	if dockerImage != "" && runtime.GOOS == "windows" {
	return nil
	}
	// For some implementations there's no way to test with a Docker container yet.
	// For them we skip the Docker stuff but still execute the tests, which can skip on connection error and we can see the skips in the test results.
	if dockerImage != "" {
	// Pull Docker image
	fmt.Printf("Pulling Docker image %s...", dockerImage)
	var out string
	out, err = script.Exec("docker pull " + dockerImage).String()
	if err != nil {
	// Depending on the error, printing the output could be interesting, as it could be more info than what's in err.
	fmt.Println(out)
	return err
	}
	// Start Docker container
	fmt.Println("Starting Docker container...")
	fmt.Println(dockerCmd)
	out, err = script.Exec(dockerCmd).String()
	if err != nil {
	fmt.Println(out)
	return err
	}

And the container stop at the end is also here:

gokv/magefiles/test.go

Lines 139 to 150 in c954c54

	defer func() {
	out, err2 := script.Exec("docker stop " + containerID).String()
	if err2 != nil {
	// Set err for returning, but only if it's not set yet.
	// ⚠️ Make sure all below errors are set to `err` with `err = ` and not shadowed with `:=`.
	if err == nil {
	err = err2
	}
	// Depending on the error, printing the output could be interesting, as it could be more info than what's in err.
	fmt.Println(out)
	}
	}()

Maybe the deficiency doesn't exist anymore, but even then I'd like the testing to be consistent across store implementations. So either all other implementations should switch to testcontainers as well, or probably easier to switch the PR to start the container with that Magefile.

It will require at least the addition of couchbase as a line in this file as well: https://github.com/philippgille/gokv/blob/c954c5435f359149e66112d0b99efe7f76ea2f3b/build/implementations.

[peczenyj]

ok I will fix it

[peczenyj]

@philippgille I think I find how to prepare the couchbase image for this test.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.64%. Comparing base (9d4aa6e) to head (98a49d3).
Report is 2 commits behind head on master.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #193   +/-   ##
=======================================
  Coverage   63.64%   63.64%           
=======================================
  Files          25       25           
  Lines        2107     2107           
=======================================
  Hits         1341     1341           
  Misses        651      651           
  Partials      115      115           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.