Nussknacker and Flink Parallelism Configuration

[arjun-guptha]

Hello team, I'm a newcomer to the Nussknacker tool. I've managed to create a basic Kafka producer-consumer workflow and deployed it in Apache Flink version 1.16.2 using Nussknacker. My question pertains to how we can manage the parallelism of individual tasks within a Flink job. Currently, I've observed that all tasks in the Flink job have the same parallelism value of one. Can you please guide me on how to configure the parallelism for individual tasks within a Flink job using Nussknacker?

[arkadius]

Hello @arjun-guptha . Parallelism can be set in scenario properties. You can get there by clicking the button on the right panel inside the diagram editor. Here: https://demo.nussknacker.io/visualization/DetectLargeTransactions?nodeId=DetectLargeTransactions you can see the window with available options. Does this answer your question?

[arjun-guptha]

Thanks for your response.I've carefully examined your answer. Based on the solution you provided, it seems that parallelism can be configured for the entire scenario. Please correct me if I'm mistaken.
For instance, in my Flink job, I have a flow that goes like this: Kafka (source) ->-filter-> lookup -> Kafka (sink). In this scenario, I would like to set a parallelism of 3 for both the source and sink Kafka components and a parallelism of 6 for the lookup component. I'm curious to know how this flow can be configured in Nussknacker. I'm eagerly awaiting your response.

[arkadius]

You're right. We have such a limitation that parallelism can be configured only for the entire scenario. If you find this limitation annoying, you can submit a feature request for having the parallelism configurable for each node.

[arjun-guptha]

Thanks for the Response