Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add flink module #110

Merged
merged 8 commits into from
Sep 30, 2024
Merged

feat: add flink module #110

merged 8 commits into from
Sep 30, 2024

Conversation

ishanarya0
Copy link
Member

Adding a module named flink
This will be used by dagger to resolve the following details:

a. kube cluster
b. kube ns
c. influx instance
d. sink kafka stream name

A dagger would have only flink module as its dependency.

Note: The dependecy would look like this

                         -> flink 1 -> multiple daggers
         -> namespace A 
                         -> flink 2 -> multiple daggers
1 kube              
       
         -> namespace B  -> flink 3 -> multiple daggers

@mabdh
Copy link
Member

mabdh commented Sep 1, 2024

Can you please add test cases for this?

modules/flink/config.go Outdated Show resolved Hide resolved
Comment on lines +62 to +77
/*
s.Run("create flink with invalid config will return invalid error", func() {
_, err := s.resourceClient.CreateResource(s.ctx, &entropyv1beta1.CreateResourceRequest{
Resource: &entropyv1beta1.Resource{
Name: "test-flink",
Project: "test-project",
Kind: "flink",
Spec: &entropyv1beta1.ResourceSpec{
Configs: structpb.NewStringValue("{}"),
Dependencies: []*entropyv1beta1.ResourceDependency{},
},
},
})
s.Assert().Equal(codes.InvalidArgument, status.Convert(err).Code())
})
*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we clean this up if not needed?

ishanarya0 and others added 3 commits September 30, 2024 14:32
* feat: add dagger module

* feat: add flink dep

* feat: add transformations

* fix: read stream from config root

* feat: add Plan implementation

* fix: chart values

* fix: resolve TODOs and refactored

* fix: source sink base handling

* feat: Output to have CR details

* feat: handle status

* refactor: seperate contants by type

* refactor: kubeGetCRD function

* feat: add dagger update action

* fix: add Update action

* chore: change var name to sink_kafka_stream

* feat: merge consumer group ID if sink is same

---------

Co-authored-by: Ishan Arya <[email protected]>
* feat: add dagger module

* feat: add flink dep

* feat: add transformations

* fix: read stream from config root

* feat: add Plan implementation

* fix: chart values

* fix: resolve TODOs and refactored

* fix: source sink base handling

* feat: Output to have CR details

* feat: handle status

* refactor: seperate contants by type

* refactor: kubeGetCRD function

* feat: add dagger update action

* fix: add Update action

* chore: change var name to sink_kafka_stream

* feat: merge consumer group ID if sink is same

* feat: add start, stop and reset actions (#112)

* feat: add start & update action

* feat: add reset action

---------

Co-authored-by: Ishan Arya <[email protected]>

---------

Co-authored-by: Ishan Arya <[email protected]>
@ishanarya0 ishanarya0 changed the base branch from main to dagger September 30, 2024 09:47
@ishanarya0 ishanarya0 merged commit 033e751 into dagger Sep 30, 2024
6 checks passed
FemiNoviaLina pushed a commit that referenced this pull request Nov 14, 2024
* rebase

* test: add e2e-test

* test: fix TearDown

* test: fix test

* feat: change var name

* feat: add dagger module create & update action (#107)

* feat: add dagger module

* feat: add flink dep

* feat: add transformations

* fix: read stream from config root

* feat: add Plan implementation

* fix: chart values

* fix: resolve TODOs and refactored

* fix: source sink base handling

* feat: Output to have CR details

* feat: handle status

* refactor: seperate contants by type

* refactor: kubeGetCRD function

* feat: add dagger update action

* fix: add Update action

* chore: change var name to sink_kafka_stream

* feat: merge consumer group ID if sink is same

---------

Co-authored-by: Ishan Arya <[email protected]>

* feat: dagger actions (#114)

* feat: add dagger module

* feat: add flink dep

* feat: add transformations

* fix: read stream from config root

* feat: add Plan implementation

* fix: chart values

* fix: resolve TODOs and refactored

* fix: source sink base handling

* feat: Output to have CR details

* feat: handle status

* refactor: seperate contants by type

* refactor: kubeGetCRD function

* feat: add dagger update action

* fix: add Update action

* chore: change var name to sink_kafka_stream

* feat: merge consumer group ID if sink is same

* feat: add start, stop and reset actions (#112)

* feat: add start & update action

* feat: add reset action

---------

Co-authored-by: Ishan Arya <[email protected]>

---------

Co-authored-by: Ishan Arya <[email protected]>

* feat: add extra streams

---------

Co-authored-by: Ishan Arya <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants