Skip to content
This repository has been archived by the owner on Nov 6, 2018. It is now read-only.

Gerrit is broken #684

Open
jstrachan opened this issue Nov 28, 2016 · 3 comments
Open

Gerrit is broken #684

jstrachan opened this issue Nov 28, 2016 · 3 comments

Comments

@jstrachan
Copy link
Contributor

moved issue from: fabric8io/gofabric8#304

I installed Gerrit using the one click deploy. Out of the box, it does not work because it is missing the "gerrit-admin-ssh" and "gerrit-users-ssh-keys". So it fails to start. But I think overall it is broken. I tried messing with the yml file as well, to tweak it as the namespace is wrong and I wanted different default credentials.

So I went and generated a priv/pub key pair and created the secrets manually for "gerrit-admin-ssh" and "gerrit-users-ssh-keys". I just used the same keys for both for testing purposes.

I removed the Gerrit app runtime and then ran my yml again to install it.

Still no luck. The container is crashing but the errors are not clear as why it might be happening.

kubectl -n app describe pods gerrit-1386751604-ld9wy
Name:		gerrit-1386751604-ld9wy
Namespace:	app
Node:		gke-forge-paas-default-pool-0b253c51-2yqj/10.128.0.3
Start Time:	Sat, 26 Nov 2016 05:59:50 -0700
Labels:		group=io.fabric8.devops.apps
		pod-template-hash=1386751604
		project=gerrit
		provider=fabric8
		version=2.2.297
Status:		Running
IP:		10.0.1.52
Controllers:	ReplicaSet/gerrit-1386751604
Containers:
  gerrit:
    Container ID:	docker://0903e458cfdb447ea512af466872587c9d5edffb987d183e571443ec251b891a
    Image:		fabric8/gerrit:2.2.297
    Image ID:		docker://sha256:11b359a11bc5e0510b49543bdee449b9c0581c96864fb6b59077a142a5774e7d
    Port:		8080/TCP
    Limits:
      cpu:	0
      memory:	0
    Requests:
      cpu:		0
      memory:		0
    State:		Waiting
      Reason:		CrashLoopBackOff
    Last State:		Terminated
      Reason:		Error
      Exit Code:	1
      Started:		Sat, 26 Nov 2016 06:02:07 -0700
      Finished:		Sat, 26 Nov 2016 06:02:15 -0700
    Ready:		False
    Restart Count:	4
    Volume Mounts:
      /home/gerrit/ssh-keys from gerrit-users-ssh-keys (rw)
      /root/.ssh from gerrit-admin-ssh-key (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from gerrit-token-ieunl (ro)
    Environment Variables:
      GERRIT_PUBLIC_KEYS_PATH:		/home/gerrit/ssh-keys
      GIT_SERVER_PASSWORD:		RedHat$1
      GERRIT_ADMIN_FULLNAME:		Administrator
      GERRIT_ACCOUNTS:			jenkins,jenkins,[email protected],secret,Non-Interactive Users:Administrators;sonar,sonar,[email protected],secret,Non-Interactive Users
      GERRIT_ADMIN_USER:		<set to the key 'gerrit-admin-user' of config map 'gerrit'>
      GIT_SERVER_IP:			gogs.app.svc.cluster.local
      GERRIT_ADMIN_PRIVATE_KEY:		/root/.ssh/id_rsa
      GERRIT_REPO_DESCRIPTION:		<set to the key 'gerrit-repo-description' of config map 'gerrit'>
      GERRIT_SSH_PATH:			/root/.ssh
      GERRIT_ADMIN_EMAIL:		[email protected]
      GERRIT_GIT_REMOTEPATH:		ssh://admin@localhost:29418/All-Projects
      GIT_SERVER_PROJ_ROOT:		gogsadmin
      GERRIT_ADMIN_PWD:			<set to the key 'gerrit-admin-pwd' of config map 'gerrit'>
      GERRIT_GIT_PROJECT_CONFIG:	/home/gerrit/configs/project.config
      GIT_SERVER_PORT:			80
      GERRIT_INITIAL_COMMIT:		<set to the key 'gerrit-initial-commit' of config map 'gerrit'>
      GERRIT_USER_PUBLIC_KEY_SUFFIX:	-rsa.pub
      GIT_SERVER_USER:			gogsadmin
      GERRIT_GIT_LOCALPATH:		/home/gerrit/git
      GERRIT_USER_PUBLIC_KEY_PREFIX:	id-
      AUTH_TYPE:			DEVELOPMENT_BECOME_ANY_ACCOUNT
      KUBERNETES_NAMESPACE:		app (v1:metadata.namespace)
Conditions:
  Type		Status
  Initialized 	True 
  Ready 	False 
  PodScheduled 	True 
Volumes:
  gerrit-admin-ssh-key:
    Type:	Secret (a volume populated by a Secret)
    SecretName:	gerrit-admin-ssh
  gerrit-users-ssh-keys:
    Type:	Secret (a volume populated by a Secret)
    SecretName:	gerrit-users-ssh-keys
  gerrit-token-ieunl:
    Type:	Secret (a volume populated by a Secret)
    SecretName:	gerrit-token-ieunl
QoS Class:	BestEffort
Tolerations:	<none>
Events:
  FirstSeen	LastSeen	Count	From							SubobjectPath		Type		Reason		Message
  ---------	--------	-----	----							-------------		--------	------		-------
  2m		2m		1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal		Pulling		pulling image "fabric8/gerrit:2.2.297"
  2m		2m		1	{default-scheduler }								Normal		Scheduled	Successfully assigned gerrit-1386751604-ld9wy to gke-forge-paas-default-pool-0b253c51-2yqj
  2m		2m		1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal		Pulled		Successfully pulled image "fabric8/gerrit:2.2.297"
  2m		2m		1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal		Created		Created container with docker id 4216ed0d1bd8; Security:[seccomp=unconfined]
  2m		2m		1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal		Started		Started container with docker id 4216ed0d1bd8
  2m		2m		1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal		Started		Started container with docker id 5d6468df51c9
  2m		2m		1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal		Created		Created container with docker id 5d6468df51c9; Security:[seccomp=unconfined]
  2m		2m		1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}				Warning		FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "gerrit" with CrashLoopBackOff: "Back-off 10s restarting failed container=gerrit pod=gerrit-1386751604-ld9wy_app(37500b26-b3d8-11e6-bf41-42010a8000de)"

  2m	2m	1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal	Created		Created container with docker id 7074e385b8ac; Security:[seccomp=unconfined]
  2m	2m	1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal	Started		Started container with docker id 7074e385b8ac
  1m	1m	2	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}				Warning	FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "gerrit" with CrashLoopBackOff: "Back-off 20s restarting failed container=gerrit pod=gerrit-1386751604-ld9wy_app(37500b26-b3d8-11e6-bf41-42010a8000de)"

  1m	1m	1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal	Started		Started container with docker id 530d5fd4c789
  1m	1m	1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal	Created		Created container with docker id 530d5fd4c789; Security:[seccomp=unconfined]
  1m	50s	3	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}				Warning	FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "gerrit" with CrashLoopBackOff: "Back-off 40s restarting failed container=gerrit pod=gerrit-1386751604-ld9wy_app(37500b26-b3d8-11e6-bf41-42010a8000de)"

  2m	37s	4	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal	Pulled		Container image "fabric8/gerrit:2.2.297" already present on machine
  37s	37s	1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal	Created		Created container with docker id 0903e458cfdb; Security:[seccomp=unconfined]
  37s	37s	1	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Normal	Started		Started container with docker id 0903e458cfdb
  2m	13s	8	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}	spec.containers{gerrit}	Warning	BackOff		Back-off restarting failed docker container
  28s	13s	2	{kubelet gke-forge-paas-default-pool-0b253c51-2yqj}				Warning	FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "gerrit" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=gerrit pod=gerrit-1386751604-ld9wy_app(37500b26-b3d8-11e6-bf41-42010a8000de)"

Here is my yml:

---
apiVersion: "v1"
kind: "List"
items:
- apiVersion: "v1"
  kind: "ServiceAccount"
  metadata:
    annotations:
      fabric8.io/iconUrl: "https://cdn.rawgit.com/fabric8io/fabric8-devops/master/gerrit/src/main/fabric8/icon.png"
    finalizers: []
    labels:
      provider: "fabric8"
      project: "gerrit"
      version: "2.2.297"
      group: "io.fabric8.devops.apps"
    name: "gerrit"
    ownerReferences: []
  imagePullSecrets: []
  secrets:
  - name: "gerrit-admin-ssh"
  - name: "gerrit-users-ssh-keys"
- apiVersion: "v1"
  kind: "Service"
  metadata:
    annotations:
      fabric8.io/app-menu: "development"
      fabric8.io/iconUrl: "https://cdn.rawgit.com/fabric8io/fabric8-devops/master/gerrit/src/main/fabric8/icon.png"
      fabric8.io/git-commit: "0958d61020835b9707536cc0525344fc30a764dd"
      fabric8.io/git-branch: "release-v2.2.297"
      fabric8.io/build-id: "4"
      fabric8.io/git-url: "http://gogs.ux.fabric8.io/gogsadmin/oss-parent/commit/0958d61020835b9707536cc0525344fc30a764dd"
      fabric8.io/build-url: "http://jenkins.ux.fabric8.io/job/oss-parent/4"
    finalizers: []
    labels:
      project: "gerrit"
      provider: "fabric8"
      expose: "true"
      version: "2.2.297"
      group: "io.fabric8.devops.apps"
    name: "gerrit"
    ownerReferences: []
  spec:
    deprecatedPublicIPs: []
    externalIPs: []
    loadBalancerSourceRanges: []
    ports:
    - port: 80
      protocol: "TCP"
      targetPort: 8080
    selector:
      project: "gerrit"
      provider: "fabric8"
      group: "io.fabric8.devops.apps"
- apiVersion: "v1"
  kind: "Service"
  metadata:
    annotations:
      fabric8.io/git-commit: "0958d61020835b9707536cc0525344fc30a764dd"
      fabric8.io/iconUrl: "https://cdn.rawgit.com/fabric8io/fabric8-devops/master/gerrit/src/main/fabric8/icon.png"
      fabric8.io/git-branch: "release-v2.2.297"
      fabric8.io/build-id: "4"
      fabric8.io/git-url: "http://gogs.ux.fabric8.io/gogsadmin/oss-parent/commit/0958d61020835b9707536cc0525344fc30a764dd"
      fabric8.io/build-url: "http://jenkins.ux.fabric8.io/job/oss-parent/4"
    finalizers: []
    labels:
      project: "gerrit"
      provider: "fabric8"
      expose: "true"
      version: "2.2.297"
      group: "io.fabric8.devops.apps"
    name: "gerrit-ssh"
    ownerReferences: []
  spec:
    deprecatedPublicIPs: []
    externalIPs: []
    loadBalancerSourceRanges: []
    ports:
    - port: 29418
      protocol: "TCP"
      targetPort: 29418
    selector:
      project: "gerrit"
      provider: "fabric8"
      group: "io.fabric8.devops.apps"
- apiVersion: "v1"
  kind: "ConfigMap"
  metadata:
    annotations:
      fabric8.gerrit/summary: "[Gerrit](https://code.google.com/p/gerrit/) Web based\
        \ code review and project management for Git based projects"
      description: "Gerrit is a web based code review system, facilitating online\
        \ code reviews for projects using the Git version control system.\n\nGerrit\
        \ makes reviews easier by showing changes in a side-by-side display, and allowing\
        \ inline comments to be added by any reviewer.\n\nGerrit simplifies Git based\
        \ project maintainership by permitting any authorized user to submit changes\
        \ to the master Git repository, rather than requiring all approved changes\
        \ to be merged in by hand by the project maintainer. This functionality enables\
        \ a more centralized usage of Git.\n\n[https://www.gerritcodereview.com/](https://www.gerritcodereview.com/)"
      fabric8.gerrit/iconUrl: "https://cdn.rawgit.com/fabric8io/fabric8-devops/master/gerrit/src/main/fabric8/icon.png"
      fabric8.io/json-schema: "{\n  \"type\" : \"object\",\n  \"properties\" : {\n\
        \    \"GERRIT_ADMIN_PWD\" : {\n      \"type\" : \"string\",\n      \"description\"\
        \ : \"Admin password\",\n      \"default\" : \"secret\"\n    },\n    \"GERRIT_INITIAL_COMMIT\"\
        \ : {\n      \"type\" : \"string\",\n      \"description\" : \"Parameter used\
        \ when the Gerrit Git repo is created using the DevOpsConnector. By default,\
        \ it is false as commits have already been done within gogs repo\",\n    \
        \  \"default\" : \"false\"\n    },\n    \"GERRIT_ADMIN_USER\" : {\n      \"\
        type\" : \"string\",\n      \"description\" : \"Admin user used to perform\
        \ operations on gerrit\",\n      \"default\" : \"admin\"\n    },\n    \"GERRIT_REPO_DESCRIPTION\"\
        \ : {\n      \"type\" : \"string\",\n      \"description\" : \"Description\
        \ ot add to the Gerrit Git repo created when a CD Project is created\",\n\
        \      \"default\" : \"Description of the gerrit git repo\"\n    }\n  }\n}"
    finalizers: []
    labels:
      provider: "fabric8"
      project: "gerrit"
      version: "2.2.297"
      group: "io.fabric8.devops.apps"
    name: "gerrit"
    ownerReferences: []
  data:
    gerrit-initial-commit: "false"
    gerrit-admin-pwd: "RedHat$1"
    gerrit-admin-user: "gerritadmin"
    gerrit-repo-description: "Description of the gerrit git repo"
- apiVersion: "extensions/v1beta1"
  kind: "Deployment"
  metadata:
    annotations:
      fabric8.io/iconUrl: "https://cdn.rawgit.com/fabric8io/fabric8-devops/master/gerrit/src/main/fabric8/icon.png"
      fabric8.io/git-commit: "0958d61020835b9707536cc0525344fc30a764dd"
      fabric8.io/metrics-path: "dashboard/file/kubernetes-pods.json/?var-project=gerrit&var-version=2.2.297"
      fabric8.io/build-id: "4"
      fabric8.io/build-url: "http://jenkins.ux.fabric8.io/job/oss-parent/4"
      fabric8.io/git-branch: "release-v2.2.297"
      fabric8.io/git-url: "http://gogs.ux.fabric8.io/gogsadmin/oss-parent/commit/0958d61020835b9707536cc0525344fc30a764dd"
    finalizers: []
    labels:
      provider: "fabric8"
      project: "gerrit"
      version: "2.2.297"
      group: "io.fabric8.devops.apps"
    name: "gerrit"
    ownerReferences: []
  spec:
    replicas: 1
    selector:
      matchExpressions: []
      matchLabels:
        provider: "fabric8"
        project: "gerrit"
        group: "io.fabric8.devops.apps"
    template:
      metadata:
        annotations:
          fabric8.io/git-commit: "0958d61020835b9707536cc0525344fc30a764dd"
          fabric8.io/secret-ssh-key: "gerrit-admin-ssh"
          fabric8.io/metrics-path: "dashboard/file/kubernetes-pods.json/?var-project=gerrit&var-version=2.2.297"
          fabric8.io/build-id: "4"
          fabric8.io/build-url: "http://jenkins.ux.fabric8.io/job/oss-parent/4"
          fabric8.io/iconUrl: "https://cdn.rawgit.com/fabric8io/fabric8-devops/master/gerrit/src/main/fabric8/icon.png"
          fabric8.io/git-branch: "release-v2.2.297"
          fabric8.io/git-url: "http://gogs.ux.fabric8.io/gogsadmin/oss-parent/commit/0958d61020835b9707536cc0525344fc30a764dd"
          fabric8.io/secret-ssh-public-key: "gerrit-users-ssh-keys[id-jenkins-rsa.pub,id-sonar-rsa.pub]"
        finalizers: []
        labels:
          provider: "fabric8"
          project: "gerrit"
          version: "2.2.297"
          group: "io.fabric8.devops.apps"
        ownerReferences: []
      spec:
        containers:
        - args: []
          command: []
          env:
          - name: "GERRIT_PUBLIC_KEYS_PATH"
            value: "/home/gerrit/ssh-keys"
          - name: "GIT_SERVER_PASSWORD"
            value: "RedHat$1"
          - name: "GERRIT_ADMIN_FULLNAME"
            value: "Administrator"
          - name: "GERRIT_ACCOUNTS"
            value: "jenkins,jenkins,[email protected],secret,Non-Interactive Users:Administrators;sonar,sonar,[email protected],secret,Non-Interactive\
              \ Users"
          - name: "GERRIT_ADMIN_USER"
            valueFrom:
              configMapKeyRef:
                key: "gerrit-admin-user"
                name: "gerrit"
          - name: "GIT_SERVER_IP"
            value: "gogs.${namespace}.svc.cluster.local"
          - name: "GERRIT_ADMIN_PRIVATE_KEY"
            value: "/root/.ssh/id_rsa"
          - name: "GERRIT_REPO_DESCRIPTION"
            valueFrom:
              configMapKeyRef:
                key: "gerrit-repo-description"
                name: "gerrit"
          - name: "GERRIT_SSH_PATH"
            value: "/root/.ssh"
          - name: "GERRIT_ADMIN_EMAIL"
            value: "[email protected]"
          - name: "GERRIT_GIT_REMOTEPATH"
            value: "ssh://admin@localhost:29418/All-Projects"
          - name: "GIT_SERVER_PROJ_ROOT"
            value: "gogsadmin"
          - name: "GERRIT_ADMIN_PWD"
            valueFrom:
              configMapKeyRef:
                key: "gerrit-admin-pwd"
                name: "gerrit"
          - name: "GERRIT_GIT_PROJECT_CONFIG"
            value: "/home/gerrit/configs/project.config"
          - name: "GIT_SERVER_PORT"
            value: "80"
          - name: "GERRIT_INITIAL_COMMIT"
            valueFrom:
              configMapKeyRef:
                key: "gerrit-initial-commit"
                name: "gerrit"
          - name: "GERRIT_USER_PUBLIC_KEY_SUFFIX"
            value: "-rsa.pub"
          - name: "GIT_SERVER_USER"
            value: "gogsadmin"
          - name: "GERRIT_GIT_LOCALPATH"
            value: "/home/gerrit/git"
          - name: "GERRIT_USER_PUBLIC_KEY_PREFIX"
            value: "id-"
          - name: "AUTH_TYPE"
            value: "DEVELOPMENT_BECOME_ANY_ACCOUNT"
          - name: "KUBERNETES_NAMESPACE"
            valueFrom:
              fieldRef:
                fieldPath: "metadata.namespace"
          image: "fabric8/gerrit:2.2.297"
          imagePullPolicy: "IfNotPresent"
          name: "gerrit"
          ports:
          - containerPort: 8080
            name: "http"
          resources:
            limits:
              cpu: "0"
              memory: "0"
            requests:
              cpu: "0"
              memory: "0"
          volumeMounts:
          - mountPath: "/root/.ssh"
            name: "gerrit-admin-ssh-key"
            readOnly: false
          - mountPath: "/home/gerrit/ssh-keys"
            name: "gerrit-users-ssh-keys"
            readOnly: false
        imagePullSecrets: []
        nodeSelector: {}
        serviceAccountName: "gerrit"
        volumes:
        - name: "gerrit-admin-ssh-key"
          secret:
            items: []
            secretName: "gerrit-admin-ssh"
        - name: "gerrit-users-ssh-keys"
          secret:
            items: []
            secretName: "gerrit-users-ssh-keys"
@jstrachan jstrachan mentioned this issue Nov 28, 2016
Closed
@rawlingsj
Copy link
Contributor

The issue is that the gerrit ssh keys aren't being generated. We have a gofabric8 secrets command that will generate these keys on OpenShift https://github.com/fabric8io/gofabric8/blob/master/cmds/secrets.go#L106

The command looks at the installed OpenShift templates and checks for an annotation as to whether it should generate the keys. http://fabric8.io/guide/secretAnnotations.html#ssh-keys

I think we need to update gofabric8 secrets so that it runs on kubernetes and checks the installed catalog configmaps to see if we need to generate the secrets.

@jstrachan
Copy link
Contributor Author

wonder if we should look at using gofabric8 secrets as an init container for gerrit?

@rawlingsj
Copy link
Contributor

yeah thats a much better idea. I can have a go at that now.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jstrachan @rawlingsj