Debugging doesn't work inside functions.

[mhussain-1]

Hi, I am new to nussknacker, and flink. Just learning how the streaming work. Please see the screenshot attached. The debugging doesn't work inside the sourceStream function. It works outside of this function. Please help me how to debug those areas as well. I am using IntelliJ editor

.

OR see the code below

class FilterPastVersions extends RichFlatMapFunction[(String, Long, ObjectNode, String), (String, ObjectNode)] {

  private var version: ValueState[Long] = _

  override def flatMap(input: (String, Long, ObjectNode, String), out: Collector[(String, ObjectNode)]): Unit = {

    // access the state value
    val tmpCurrentVersion = version.value()

    // If it hasn't been used before, it will be null
    if (tmpCurrentVersion == null || input._2 > tmpCurrentVersion){
      version.update(input._2)
      out.collect((input._4, input._3))
    }
  }

  override def open(parameters: Configuration): Unit = {
    val versionDesc = new ValueStateDescriptor[Long]("version", createTypeInformation[Long])
    versionDesc.setQueryable("version-state")

    version = getRuntimeContext.getState(versionDesc)
  }
}

If I put breakpoints in the flatmap function of FilterPastVersions, those breakpoints are never hit. The program does successfully print messages from Kafka.

[mproch]

Hi and welcome :)

If you are running scenarios in external Flink (e.g. in Docker container) the code of functions (sources, custom filters etc.) is not executed in the JVM of Nussknacker Designer, but is sent (as .jar files) to Flink JobManager JVM (there the sourceStream function is executed) and later to Flink TaskManagers JVMs (there the flatMap is executed when the process is running).

https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/concepts/flink-architecture/ - please have a look at this description (Nussknacker sends and deploys Flink job via REST API)

If you want to debug those functions you can either:

• attach debugger remotely to JobManager or TaskManager (see e.g. https://cwiki.apache.org/confluence/display/FLINK/Remote+Debugging+of+Flink+Clusters)
• use "Test from file" functionality in Nussknacker (https://docs.nussknacker.io/documentation/docs/scenarios_authoring/TestingAndDebugging) - then the scenario will run in embedded Flink cluster in Designer
• if you want to write own components, probably the best way is to start with some scalatest tests and debug there

Hope it helps,
maciek