Skip to content

Commit

Permalink
[usecase, gctl]: showcase install/un-install of CRD (#28)
Browse files Browse the repository at this point in the history
This commit showcases how GenericController can be used to install
and uninstall a CRD when a particular namespace gets created and
deleted correspondingly.

This commit can be understood in two ways:
1/ Files in test/integration/generic/ shows how golang based
integration code can be written to verify this use-case.

2/ Files in examples/gctl/install-uninstall-crd/ shows how metac
controller & jsonnet files are used to verify this use-case.

NOTES:
1/ There are some changes to vendoring as well. Latest k8s.io
dependencies are set.

2/ Integration test case has been set to run only if test is not
run in short mode.

Signed-off-by: AmitKumarDas <[email protected]>
  • Loading branch information
Amit Kumar Das authored Oct 9, 2019
1 parent 5091d52 commit 41435e8
Show file tree
Hide file tree
Showing 18 changed files with 783 additions and 61 deletions.
10 changes: 9 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,12 @@ integration-dependencies:
# This can be run on one's laptop or Travis like CI environments.
.PHONY: integration-test
integration-test: generated_files integration-dependencies
@PATH="$(PWD)/hack/bin:$(PATH)" go test ./test/integration/... -v -timeout 5m -args --logtostderr -v=1
@PATH="$(PWD)/hack/bin:$(PATH)" \
go test ./test/integration/... -v -short -timeout 5m \
-args --logtostderr -v=1

.PHONY: integration-test-gctl
integration-test-gctl: generated_files integration-dependencies
@PATH="$(PWD)/hack/bin:$(PATH)" \
go test ./test/integration/generic/... -v -timeout 5m \
-args --logtostderr -v=1
46 changes: 24 additions & 22 deletions TODO
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
### TODO:
- test metac with changes to rbac

- gctl add integration tests:
- Test metac with changes to rbac
- Make all examples to work & then make it better
- Add integration tests w.r.t GenericController:
- check if finalizer works
- check if attachments not created by gctl are not updated
- check if attachments not created by gctl are not deleted
Expand All @@ -19,44 +19,46 @@
- operator - WatchIsOwner, WatchIsCreator, WatchIsUpdater
- operator - WatchInKey, WatchInValue, ValueContainsWatch, KeyContainsWatch

- gctl - read & review
### Blogs:
- Integration Tests
- go based integration tests vs. bash based vs GCtl based
- Blog talks only about GCtl tests & give links to how Go & Bash are done

### Learn:
- read & review & compare them against GenericController
- https://github.com/GoogleCloudPlatform/metacontroller/issues/98
- https://github.com/GoogleCloudPlatform/metacontroller/pull/143
- https://github.com/GoogleCloudPlatform/metacontroller/pull/168

### Few targeted usecases:
- ConformanceTest
- CStorConfigController
- DDP
- Install
- UnInstall
- Upgrade

- all controller enhancements
### Future actions:
- All controller enhancements
- https://github.com/GoogleCloudPlatform/metacontroller/issues/154
- make Metac run based on either of following:
- CustomResource, or # current mode
- Config # default mode
- make cctl, dctl & gctl work from config files vs. current way that depends on CRDs

- explore the one binary sidecars
- Explore the one binary sidecars
- jsonnetd service + hooks as a docker image

- gctl integration test
- code refactoring - one .go file per test
- proper logs, informative logs, proper log levels
- Use latest stable Kubernetes version & etcd for integration testing
- make all examples to work & then make it better
- restructure examples to enable community
- code to use klog
- test/unittest
- find bugs & fix

- targeted usecases
- ConformanceTest
- CStorConfigController
- DDP
- Install
- UnInstall
- Upgrade

### Future Actions:
- Make metacontroller watch resources from specific namespace if required
- Should Metac support UDS?
- https://eli.thegreenplace.net/2019/unix-domain-sockets-in-go/
- >>enisoc 2:08 AM
if the new client-go dynamic informer does everything metacontroller needs
(e.g. start/stop dynamically without process restart), it would be great to
rebase metacontroller on client-go's dynamic package

### Meeting Notes & Agenda
- https://docs.google.com/document/d/1HV_Fr0wIW9tr5OZwK_6oGux_OhcGtxxWWV6dCYJR9Cw/
Expand Down
19 changes: 12 additions & 7 deletions controller/common/manage_attachments.go
Original file line number Diff line number Diff line change
Expand Up @@ -206,9 +206,12 @@ func (m AttachmentManager) String() string {
return m.AttachmentExecuteBase.String()
}

// preApply prepares the manager before invoking the Apply
// This method sets various default options if applicable
func (m *AttachmentManager) preApply() {
// setDefaults prepares the attachment manager before invoking the
// Apply
//
// NOTE:
// This method sets various **default** options if applicable
func (m *AttachmentManager) setDefaults() {
if m.DeleteFn == nil {
m.errs = appendErrIfNotNil(
m.errs, anyAttachmentsDefaultDeleter()(m),
Expand All @@ -220,16 +223,18 @@ func (m *AttachmentManager) preApply() {
)
}
if m.IsWatchOwner == nil {
// defaults to set this watch as not the owner
// of the attachments
m.IsWatchOwner = kubernetes.BoolPtr(false)
// defaults to set this watch as the owner of the
// attachments, since this tunable is used only during
// creation of these attachment(s) while observing the
// watch resource
m.IsWatchOwner = kubernetes.BoolPtr(true)
}
}

// Apply executes create, delete or update operations against
// the child resources set against this manager instance
func (m *AttachmentManager) Apply() error {
m.preApply()
m.setDefaults()
if len(m.errs) != 0 {
return utilerrors.NewAggregate(m.errs)
}
Expand Down
59 changes: 59 additions & 0 deletions examples/gctl/install-uninstall-crd/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
## Install-Uninstall-CRD

This is an example of GenericController that adds a CRD for a given Namespace.
This also removes the CRD when the given Namespace is deleted.

### Prerequisites

* Kubernetes 1.8+ is recommended for its improved CRD support, especially garbage collection.
* Install Metac using yamls from manifests folder

```sh
# cd to metac's root folder

kubectl apply -f ./manifests/metacontroller-namespace.yaml
kubectl apply -f ./manifests/metacontroller-rbac.yaml
kubectl apply -f ./manifests/metacontroller.yaml

# verify if metac was installed properly
kubectl get crd
kubectl get sts -n metac
```

### Deploy the Controllers

```sh
# cd to examples/gctl/install-uninstall-crd/

# any change in any of the hooks should be accompanied
# by deleting 1/ configmap & 2/ operator & 3/ re-applying both
kubectl create configmap install-uninstall-crd -n metac --from-file=hooks
kubectl apply -f operator.yaml

# verify the deploy, svc & configmap of this example
kubectl get gctl
kubectl get cm -n metac
kubectl get deploy -n metac
kubectl get svc -n metac
```

### Create the Namespace

```sh
kubectl apply -f my-namespace.yaml
```

Watch for the CRD to get created:

```sh
kubectl get crds --watch

kubectl get crd storages.dao.amitd.io
```

Check that the CRD get cleaned up when this Namespace is deleted:

```sh
kubectl delete -f my-namespace.yaml
kubectl get crds --watch
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
function(request) {
// Delete the CRD since namespace is observed
// to being deleted
attachments: [],
// Mark as finalized once we observe this CRD are gone.
finalized: std.length(request.attachments['CustomResourceDefinition.apiextensions.k8s.io/v1beta1']) == 0
}
54 changes: 54 additions & 0 deletions examples/gctl/install-uninstall-crd/hooks/sync-crd.jsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
function(request) {
// Create a CRD for the namespace specified in GenericController
attachments: [
{
apiVersion: "apiextensions.k8s.io/v1beta1",
kind: "CustomResourceDefinition",
metadata: {
// name must match the spec fields below, and be in the form:
// <plural>.<group>
name: "storages.dao.amitd.io",
},
spec: {
// group name to use for REST API: /apis/<group>/<version>
group: "dao.amitd.io",
// version name to use for REST API: /apis/<group>/<version>
version: "v1alpha1",
// either Namespaced or Cluster
scope: "Namespaced",
names:{
// plural name to be used in the URL:
// i.e. /apis/<group>/<version>/<plural>
plural: "storages",
// # singular name to be used as an alias on the CLI and for display
singular: "storage",
// kind is normally the CamelCased singular type.
// Your resource manifests use this.
kind: "Storage",
# shortNames allow shorter string to match your resource on the CLI
shortNames: ["stor"],
},
additionalPrinterColumns: [
{
JSONPath: ".spec.capacity",
name: "Capacity",
description: "Capacity of the storage",
type: "string",
},
{
JSONPath: ".spec.nodeName",
name: "NodeName",
description: "Node where the storage gets attached",
type: "string",
},
{
JSONPath: ".status.phase",
name: "Status",
description: "Identifies the current status of the storage",
type: "string",
},
],
}
}
]
}
5 changes: 5 additions & 0 deletions examples/gctl/install-uninstall-crd/my-namespace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: v1
kind: Namespace
metadata:
name: amitd
---
71 changes: 71 additions & 0 deletions examples/gctl/install-uninstall-crd/operator.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
apiVersion: metac.openebs.io/v1alpha1
kind: GenericController
metadata:
name: install-un-crd
spec:
watch:
apiVersion: v1
resource: namespaces
nameSelector:
# we are interested in amitd namespace only
- amitd
attachments:
- apiVersion: apiextensions.k8s.io/v1beta1
resource: customresourcedefinitions
nameSelector:
# we are interested in storages CRD only
- storages.dao.amitd.io
hooks:
sync:
webhook:
url: http://jsonnetd.metac/sync-crd
finalize:
webhook:
url: http://jsonnetd.metac/finalize-crd
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: jsonnetd
namespace: metac
spec:
replicas: 1
selector:
matchLabels:
app: jsonnetd
template:
metadata:
labels:
app: jsonnetd
spec:
containers:
- name: hooks
# this deployment is all about exposing jsonnet as a webhook
# that understands metac request & response payload
image: metacontroller/jsonnetd:0.1
imagePullPolicy: Always
workingDir: /hooks
volumeMounts:
- name: hooks
mountPath: /hooks
volumes:
- name: hooks
configMap:
# this configmap provides the jsonnet files that
# get executed as webhooks
name: install-uninstall-crd
---
apiVersion: v1
kind: Service
metadata:
# this name is used to build the webhook url
name: jsonnetd
# this namespace is used to build the webhook url
namespace: metac
spec:
selector:
app: jsonnetd
ports:
- port: 80
targetPort: 8080
Loading

0 comments on commit 41435e8

Please sign in to comment.