URI / Location values in JSON 'can not be determined with webdriver.io' ?

[raumB]

Describe your question with as much detail as possible
Hello, being not a specialist in this subject I just wanted to understand the reason for https://github.com/webdriverio-community/wdio-cucumberjs-json-reporter/blob/v6/lib/reporter.js#L238 and https://github.com/webdriverio-community/wdio-cucumberjs-json-reporter/blob/v6/lib/reporter.js#L319. Is this due to some principle technical restriction and it never would be possible to have the URI / location parameters implemented? Thanks and kind regards.

If it's about a specific piece of code, try and include some of it to support your question.

            uri: 'Can not be determined',

(https://github.com/webdriverio-community/wdio-cucumberjs-json-reporter/blob/v6/lib/reporter.js#L238)

                location: 'can not be determined with webdriver.io'

(https://github.com/webdriverio-community/wdio-cucumberjs-json-reporter/blob/v6/lib/reporter.js#L319)

Environment (please complete the following information):

• Node.js version: 10.16.3
• NPM version:
• webdriver.io version: ~6.11.x
• @wdio/cucumber-framework version: ~6.11.1
• wdio-cucumberjs-json-reporter version: ~2.0.2

Additional context
Cucumber reporters for other technologies seem to to offer these parameters which could be used for forward navigation to the test case.

[wswebcreation]

Hi @raumB

WebdriverIO behaves in a different way regarding Cucumber reports in comparison to other frameworks that use CucumberJS. WebdriverIO has its own way of reporting and this module is translating that way back into JSON.
During the time I created this module there was no way to determine that, maybe there is now. That's why I added a label for people to help with this.

[christian-bromann]

This data should be available now as we updated the payload information with the last Cucumber release. @raumB you want to give us a hand and help us fix this?

[raumB]

Thanks a lot for the updates. I understand that in principle an enhancement with regard to the mentioned properties would be possible. With which exact versions of WDIO and Cucumber such an enhancement would work together? I'm asking since we are still using WDIO v6 for some other reason. Maybe we would need to go to v7 and maybe also switch in parallel to higher versions of other components? Thanks and kind regards.

[christian-bromann]

@raumB I would recommend to switch to v7 anyway. If you aren't a TypeScript user this update should be fairly simple. See more on this in our migration guide. Then we can fix this and release a new version of the reporter that you can then use in your project. Happy to help in this process.

[raumB]

We plan to evaluate v7 for our test setup.

[raumB]

@christian-bromann @wswebcreation
In an evaluation project we have now managed to migrate from WDIO v6 to v7 and would be able to test on v7 level.
To our understanding the URI parameter should contain some link or path to the corresponding feature file and the location parameter should contain some indicator for the specific corresponding test (for example, a line number from the feature file).
I assume there are some standards around about syntax and format of the cucumber.json file and what I'm suggesting for the two parameters should be checked against their common use in that file.
I myself could not contribute when it comes down to development activities, but I could try to test things, of course.
Thanks and kind regards.

[wswebcreation]

I've checked the output but the WDIO reporter is not giving back this information for the reporter. Will keep this one open, but this might first need to be fixed in the Cucumber framework adapter

[raumB]

Does also the WDIO v7 reporter does not give back this information?

[wswebcreation]

Nope, this is not given back, but maybe something Cucumber is giving back, but not passing it to the Cucumber framework adapter. Need to dive into this as this might be a 3rd party dependency, but if someone wants to take a look at it then be my guest

[wswebcreation]

Some info for the person who's going to pick this up

• the uri is of type string and contains the path to the file
• the location is of type Location and contains this ({path-to-step-file}:{column})

[wswebcreation]

@christian-bromann

I've looked into this again, I can determine the uri and the line for the feature file level, but the @wdio/cucumber-framework only gives back feature-file data, not the TS/JS-files date like the TS/JS-file file path and line/column from the specific step.

Do you happen to know how this could be added to the @wdio/cucumber-framework?

[christian-bromann]

Do you happen to know how this could be added to the @wdio/cucumber-framework?

Unfortunately the @wdio/reporter doesn't propagate any spec locations (e.g. row and column of scenario or step). This would be a nice addition.

[silke-vormbrock]

@wswebcreation
Sorry for not coming back to your comment for such a long time.
We finally got the confirmation from development that the uri and the line for the feature file level would be sufficient to ensure traceability. Your support in this topic would be very much appreciated!

[silke-vormbrock]

@wswebcreation
would it be ok to invite for a meeting to discuss possible next steps?

[wswebcreation]

Hi @silke-vormbrock

Sorry for not getting back to this. I have it on my to-do list, which is pretty long ATM :(, but will notify you when this is released

[silke-vormbrock]

Hi @wswebcreation
thank you so much for coming back to me. Do you have a rough idea when this release can be expected?
This missing piece would allow us to visualize fully automated traceability. Thanks!

[wswebcreation]

Hi @silke-vormbrock

I hope before the end of the month

[silke-vormbrock]

Hi @wswebcreation,
that would be brilliant! As Product Excellence Award Winners 2021 we are invited to the devX Quality Talk on Feb 8 and one of our topics is how we ensure traceability in every pipeline run. Thank you!

[wswebcreation]

@silke-vormbrock

A new version has been published, see https://github.com/webdriverio-community/wdio-cucumberjs-json-reporter/releases/tag/v4.2.0

This will now add the line and uri, see the example below

[
  {
    "keyword": "Feature",
    "type": "feature",
    "description": "",
    "line": 2,
    "name": "Sample",
    "uri": "/Users/Git/wdio-cucumber-v6-v7/v7/test/features/sample.feature",
    "tags": [
      {
        "location": {
          "line": 1,
          "column": 1
        },
        "name": "@sample",
        "id": "4"
      }
    ],
    "elements": [
      {
        "keyword": "Scenario",
        "type": "scenario",
        "description": "",
        "name": "cukes",
        "tags": [
          {
            "name": "@sample",
            "astNodeId": "4"
          }
        ],
        "id": "sample;cukes",
        "steps": [
          {
            "arguments": [],
            "keyword": "Before",
            "name": "",
            "result": {
              "status": "passed",
              "duration": 1000000
            },
            "line": null,
            "match": {
              "location": "can not be determined with webdriver.io"
            }
          },
          {
            "arguments": [],
            "keyword": "Before",
            "name": "",
            "result": {
              "status": "passed",
              "duration": 0
            },
            "line": null,
            "match": {
              "location": "can not be determined with webdriver.io"
            }
          },
          {
            "arguments": [],
            "keyword": "Given",
            "name": "I have a background",
            "result": {
              "status": "passed",
              "duration": 2000000
            },
            "line": null,
            "match": {
              "location": "can not be determined with webdriver.io"
            }
          },
          {
            "arguments": [],
            "keyword": "Given",
            "name": "I have 42 cukes in my belly",
            "result": {
              "status": "passed",
              "duration": 2905000000
            },
            "line": null,
            "match": {
              "location": "can not be determined with webdriver.io"
            }
          },
          {
            "arguments": [],
            "keyword": "After",
            "name": "",
            "result": {
              "status": "passed",
              "duration": 1000000
            },
            "line": null,
            "match": {
              "location": "can not be determined with webdriver.io"
            }
          }
        ]
      }
    ],
    "id": "sample",
    "metadata": {
      "browser": {
        "name": "chrome",
        "version": "97.0.4692.71"
      },
      "device": "Test Device",
      "platform": {
        "name": "ios",
        "version": "14"
      }
    }
  }
]

Can you please confirm and then close this issue if this fixes your needs

[silke-vormbrock]

Hi @wswebcreation,

thank you so much! I forwarded this information to development and hope to get feedback very quickly.

[silke-vormbrock]

Hi @wswebcreation,
our developer is checking the solution currently, we expect his results by the end of the week.

[wswebcreation]

Cool, thanks for the update

[silke-vormbrock]

Hi @wswebcreation,
unfortunately, we are not yet there where we want to be, but your update helped us to move into the right direction. I scheduled a meeting with our colleagues from Cumulus and will come up with a final update after this meeting. Thank you for your effort!

[wswebcreation]

Hi @silke-vormbrock

can not be determined with webdriver.io is being determined per step and not per test case/scenario. So I wonder how your reporting tool works. Can you explain that a bit more?

[silke-vormbrock]

Hi @wswebcreation,
this is the answer I got from the developer who supports us in this topic:
Hi Wim, you are correct at the moment each step information is taken to display the test case. This might change in the future to the scenario name. To drill down into the scenario it could be useful to link the individual steps to its implementation. Would this be possible via webdriver.io?

[wswebcreation]

Hi @silke-vormbrock

Thanks for the answer, officially the location is connected to the step file and its data. The problem is that this can't be determined in WDIO, see the above conversation.

If they want to have the step name they need to use the combination of keyword and name, like thin this step example

{
  "arguments": [],
  "keyword": "Given",
  "name": "I have 42 cukes in my belly",
  "result": { "status": "passed", "duration": 1322000000 },
  "line": null,
  "match": { "location": "can not be determined with webdriver.io" }
},

[silke-vormbrock]

Hi Wim,
thank you so much for your support. Our dev colleagues took over for the next steps.
Please allow me to keep this GitHub issue open until we have the final solution in place.
Thanks! Best regards, Silke

[wswebcreation]

Sure, no problem

[christian-bromann]

Any updates @silke-vormbrock ?