Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert FSH to FHIR and back using right click context menu in the explorer view #97

Open
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

hvamstel
Copy link

Description:
Convert a JSON file to FSH by right click on the file in the explorer view and select the convert option.
Convert a FSH file to JSON by right click on the file in the explorer view and select the convert option.

Testing Instructions:
Right click on a file and convert and then inspect the result.

Related Issue:
Convert FSH to FHIR and back using right click context menu in the explorer view #96

@hvamstel hvamstel marked this pull request as draft September 27, 2024 10:43
@hvamstel hvamstel marked this pull request as ready for review September 30, 2024 07:27
@cmoesel
Copy link
Member

cmoesel commented Sep 30, 2024

@hvamstel - Thank you for the very cool PR. I am reviewing it now and will provide more feedback later. Regarding the broken CI check -- if you run npm run check locally, it will perform the same checks as CI. You'll see that it fails the prettier formatting check. We use prettier to ensure consistent style across the project. It's easy to fix your violations; just run npm run prettier:fix and check in the changes.

@hvamstel
Copy link
Author

@cmoesel Thanks for looking into it. I did run
npm run prettier
But that didn't explain what needed to be changed and
npm run prettier --write
didn't seem to do anything.

Your suggestion of npm run prettier:fix works fine.

@cmoesel
Copy link
Member

cmoesel commented Oct 1, 2024

Hi @hvamstel. Thanks again for this contribution. It's a cool idea and I can see its usefulness. In my review of it, I did find a few limitations/questions I'd like to discuss:

  1. When doing FSH to FHIR, the FSH file is processed by itself and any other FSH files in the project / IDE are ignored. So if the FSH file refers to something from another FSH file in the project, SUSHI won't find that definition and there will be errors. This means it only works for FSH files that are fully self-contained.
  2. Similarly, when doing FHIR to FSH, the JSON file is processed by itself and any other JSON files in the project / IDE are ignored. This isn't as problematic as FSH to FHIR because in this case, it usually just creates aliases for the things it can't find. But it does mean the produced FSH might not be as good as it could be.
  3. Setting FHIR version and dependencies at the extension level can be a bit confusing to users because other parts of the extension ignore those settings and pull this info from sushi-config.yaml instead. It might be cool if this worked the same way -- pulling FHIR version and dependencies from sushi-config.yaml or ImplementationGuide-xyz.json if it finds one. If it can't find one, I wonder if it might be better to prompt the user for that info instead? (Although, TBH, I'm not sure how easy that is to do in VS Code). Extension-level config is kind of hidden from the user and also seems a bit more global in nature.
  4. You've added sushi and gofsh as dependencies, pinning them to specific versions. This means that if we want to use newer versions, we'll have to publish a new release of the extension just to bump the dependencies. For the already existing SUSHI Build action, we use the user's globally installed SUSHI rather than bundling one. Do you know if it's possible to use the globally installed NPM packages instead (like if you declared them as peerDependencies)? This would allow the user to decide what versions of SUSHI / GoFSH are used. (But I don't know if it is possible).
  5. When FSH to FHIR is invoked, the entire JSON result is put on a single line -- it would be nice if we could automatically format the JSON so it's easier to read.
  6. When either capability is invoked, it would be nice to give the new tab focus. I had a lot of tabs open and had to search for it.
  7. I noticed the the resulting tabs are editors -- showing the results as active changes and prompting me to save them when I close the tab. I expected it to be more like when I preview a markdown document, showing a read-only view. Is that something you considered?

I'm interested in your thoughts about these.

@hvamstel
Copy link
Author

hvamstel commented Oct 8, 2024

Hi Chris,

My main goal was to quickly convert a single file locally. Specifically, I wanted to convert a FHIR resource to FSH to understand how FHIR is represented in FSH and to create a good starting point for our FSH-based resource. Another goal was to convert from FSH to FHIR to demonstrate the equivalence between the two, proving that moving from FHIR to FSH doesn't result in any semantic changes. This ensures that existing FHIR projects can be safely converted to FSH.

1+2) The method(s) that I found in the gofsh and sushi packages allowed only the conversion of a single stand-alone file as far as I could see. If there are other methods, then it would be nice to have the conversion in the context of the sushi project.

  1. This approach seems promising. In our GIT repository, we have multiple Sushi projects, each in its own folder. To get the correct sushi-config, the FSH file to be converted should be in a subfolder of the folder containing the sushi-config. Are there any classes or methods in the plugin to find the sushi-config and its dependencies?

  2. The packages gofsh and sushi need to be present otherwise the methods are unknown, and the compilation/interpretation will fail to build the extension. I do not believe it is possible to work around this.

5,6,7) Are good ideas and should be in currently :)

@cmoesel
Copy link
Member

cmoesel commented Oct 10, 2024

Hi Hans,

Thanks for the response and the updates! The implementation of 5, 6, and 7 came out very nicely. I really like the way that works now. I hope you do too!

As for 1, the SUSHI API takes in a string that can have any number of FSH definitions in it. So, in theory, you could concatenate all of the FSH files in the project into one big FSH string and send it through. But... you will get an array of all the JSONs back -- and the tricky bit would be figuring out which ones pertain to the original file you invoked "FSH to FHIR" on. Maybe we should update the return object in the API to also include something similar to the fsh-index.json that the SUSHI CLI generates. The other tricky bit would be how you want to handle multiple results -- which is actually a problem that the current implementation already has if the FSH file you select has multiple definitions in it. Should it create multiple preview tabs? Or one preview tab with an array of all the relevant JSON results?

I'm also realizing now that by adding all the files, the process could take a good bit longer to run - so that's a tradeoff to consider. I wonder if we could extend the SUSHI API to also add a mode where you specify a primary FSH file and have it only compile the things needed to complete that file. That could speed things up -- but... I'm not sure where that would fit in our priorities.

As for 2, the GoFSH API takes an array of JSONs as input -- so you could pass in all the JSONs found in the project. Similar to above, however, the tricky bit would be displaying only the FSH that is relevant to the JSON file originally selected. You might be able to figure it out using the map style option -- but it's still not trivial. We could potentially make it easier by adding another style that returns FSH files in an array matching the original order of the passed in JSONs (although even this might not be perfect since aliases get their own file and if there are contained resources, they might also get their own files... so we'd need to think about that). As I noted originally though, not passing in the full context for "FHIR to FSH" is much less of a problem than it is for "FSH to FHIR".

Regarding 3, here is the code we currently have for finding a sushi-config and grabbing its fhirVersion and dependencies. It just grabs the first one it finds, but you could look at the list of all the ones it finds in the workspace and figure out which one is the closest to the FSH file and use that. Current code starts here (click through to get to all the code):

const configFiles = await workspace.findFiles('sushi-config.{yaml,yml}');

Regarding 4, I was hoping it might work if you made fsh-sushi and gofsh peerDependencies, but I've just experimented with this myself and it looks like it doesn't really make a difference. The dependencies get compiled into the plugin anyway. So perhaps that is not possible.

@hvamstel
Copy link
Author

Hi Chris

I noticed indeed that you can have multiple resources in one FSH file. Currently I open multiple previews with Json resource in each. This seems to work quite nicely with the caveat that it can be quite a lot of preview tabs when there are a lot of resources described in one FSH file.

I also rewrote the configuration part and now it will read from the sushi config. It will search for the sushi-config that is part of the same sushi project by matching the directory path. As there can be more than one sushi projects in the workspace as is in our situation. For this I reused some of the code that you mentioned and there are some refactoring opportunities.

Having the conversion in the context of the complete sushi project would be nice. The issue I see there is that concatenating all the resources together and retrieving the needed resources from the result can be complex and therefore error prone. Adopting the API of sushi and gofsh as you mentioned would help here.

Concatenating the FSH files will increase the time of the conversion having some kind of progress of activity indication could help. I was thinking of a panel in the status bar informing the user of a conversion in progress. Happy to know your thoughts and ideas on this.

Copy link
Member

@cmoesel cmoesel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started to review this but noticed that some of the changes you referred to (regarding opening multiple tabs, extracting information from sushi-config, etc) are not pushed up to this PR. Did you mean to push them up? Or were you waiting for further discussion?

I agree that supporting FSH to FHIR JSON and FHIR to FSH w/ full context would be easiest with the API changes I described -- but I'm not sure if/when we will be able to implement those API changes. Perhaps we should ask the FSH community (on Zulip) if they would support us releasing this feature with the single-file-only limitation for now. What do you think?

README.md Outdated Show resolved Hide resolved
@hvamstel
Copy link
Author

Hi Chris,

Indeed, I forget to push to origin sorry for that.
I did fix the typos in the README.md and moved the feature description to the appropriate place.

I also managed to implement running the FSH to JSON conversion in the project context and only show the converted resources that where requested.

Copy link
Member

@cmoesel cmoesel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, Hans. This is fantastic! I love it! I think people will really like this new feature!

I've provided some additional comments and suggestions, but they are mostly small and easy to do. Please let me know if you disagree with any of the suggestions I have made.

The one last thing I would like to ask you about is unit tests. As a general rule, we like to include unit tests for any new features that we introduce. This helps to ensure that as the extension evolves, we do not accidentally break anything. I know you have already invested a lot of time in this, but would you have a little more time to create a few basic unit tests to validate the core functionality? If not, then I will check to see if anyone here is able to add unit tests for you.

Thank you again for contributing this excellent feature!

@@ -92,6 +92,12 @@ To run the SUSHI Build task, use VS Code's Run Task feature. The "Run Task" feat

Note that you must have SUSHI installed locally in order for the task to run successfully. See [SUSHI Installation instructions](https://fshschool.org/docs/sushi/installation/) for help installing.

## Conversion from FHIR Json to FSH and from FSH to FHIR Json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Conversion from FHIR Json to FSH and from FSH to FHIR Json
## Conversion from FHIR JSON to FSH and from FSH to FHIR JSON

@@ -92,6 +92,12 @@ To run the SUSHI Build task, use VS Code's Run Task feature. The "Run Task" feat

Note that you must have SUSHI installed locally in order for the task to run successfully. See [SUSHI Installation instructions](https://fshschool.org/docs/sushi/installation/) for help installing.

## Conversion from FHIR Json to FSH and from FSH to FHIR Json

Right-clicking on a `.fsh` file and then selecting the option "FSH to FHIR JSON" from the context menu will convert this FSH file to a FHIR JSON file. The FHIR JSON file will be created in memory in a separate file tab.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Right-clicking on a `.fsh` file and then selecting the option "FSH to FHIR JSON" from the context menu will convert this FSH file to a FHIR JSON file. The FHIR JSON file will be created in memory in a separate file tab.
Right-clicking on a `.fsh` file and then selecting the option "FSH to FHIR JSON" from the context menu will convert this FSH file to one or more FHIR JSON files. The resulting FHIR JSON files will be created in memory and displayed in separate file tabs.


Right-clicking on a `.fsh` file and then selecting the option "FSH to FHIR JSON" from the context menu will convert this FSH file to a FHIR JSON file. The FHIR JSON file will be created in memory in a separate file tab.

Right-clicking on a `.json` file that contains a FHIR resource and then selecting the option "FHIR to FSH" from the context menu will convert this JSON FHIR file to a FSH file. The FSH file will be created in memory in a separate file tab.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Right-clicking on a `.json` file that contains a FHIR resource and then selecting the option "FHIR to FSH" from the context menu will convert this JSON FHIR file to a FSH file. The FSH file will be created in memory in a separate file tab.
Right-clicking on a `.json` file that contains a FHIR resource and then selecting the option "FHIR to FSH" from the context menu will convert this JSON FHIR file to a FSH file. The FSH file will be created in memory and displayed in a separate file tab.

Right-clicking on a `.fsh` file and then selecting the option "FSH to FHIR JSON" from the context menu will convert this FSH file to a FHIR JSON file. The FHIR JSON file will be created in memory in a separate file tab.

Right-clicking on a `.json` file that contains a FHIR resource and then selecting the option "FHIR to FSH" from the context menu will convert this JSON FHIR file to a FSH file. The FSH file will be created in memory in a separate file tab.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Note that the "FSH to FHIR JSON" and "FHIR to FSH" features use this extension's built-in versions of SUSHI and GoFSH. These may not be the most recently released versions and may not correspond to the versions you have locally installed.

Comment on lines +58 to +63
"command": "extension.fshToFHIRjson",
"title": "FSH to FHIR JSON",
"category": "vscode-fsh"
},
{
"command": "extension.FHIRtoFSH",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nitpicky, but.. I'd like for these commands to use consistent casing. The already existing command on line 58 is extension.openFhir. In order to remain consistent with casing, that means the new commands should probably be extension.fshToFhir and extension.fhirToFsh.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This, of course, pertains to all the locations that use these extension keys (not just the lines above).


const dependencies = getDependenciesFromSushiConfig(parsedConfig);
dependencies.forEach(dep => {
output.appendLine('Found dependency: ' + dep.packageId + '@' + dep.version);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though GoFSH uses @ to separate package id and version, the most popular way to do it in the FHIR world is with # (for example, you'll see it that way if you look in your .fhir cache). So I'd suggest that since this is displaying to the user, have the output message use # instead of @.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will change this, my thinking was in keeping it the same as in gofsh to avoid confusion.

return null;
}
})
.find(version => /current|4\.0\.1|4\.[1-9]\d*\.\d+|5\.\d+\.\d+/.test(version));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've added support for FHIR R6 preview in SUSHI now, so you might as well make this:

Suggested change
.find(version => /current|4\.0\.1|4\.[1-9]\d*\.\d+|5\.\d+\.\d+/.test(version));
.find(version => /current|4\.0\.1|4\.[1-9]\d*\.\d+|[56]\.\d+\.\d+/.test(version));

Comment on lines +173 to +174
} else if (line.startsWith('ConceptMap:')) {
ids.push(line.split(' ')[1].trim());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FSH doesn't have a ConceptMap: keyword. So you can delete this.

But... you should add Logical: and Resource:.

} else if (line.startsWith('Extension:')) {
ids.push(line.split(' ')[1].trim());
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think you can simplify the block above by using a regular expression. Something like:

const DECLARATION_REGEX = /^(Instance|Profile|Extension|ValueSet|CodeSystem|Logical|Resource):/;

and then in the loop:

if (DECLARATION_REGEX.test(line) {
  ids.push(line.split(' ')[1].trim());
}

conversionDependencies.push(dep.packageId + '@' + dep.version);
});
} else {
output.appendLine('No sushi-config.yaml file found.');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of a project w/ FHIR JSON (and not FSH), there probably is not a sushi-config.yaml. Instead, there might be an ImplementationGuide JSON file (e.g., ImplementationGuide-my.package.id.json). For example, if you download and unzip a package like https://www.hl7.org/fhir/us/core/package.tgz and then try to convert one of its JSON files.

So... you could consider looking for an ImplementationGuide file if a sushi-config.yaml file is not found. That's up to you. I think we will be very glad to accept this PR without support for ImplementationGuide configs, so if you don't have time (or interest) to do it, that's OK. In that case, however, perhaps you could put a comment here with a todo? (E.g., // TODO: Look for and use an ImplementationGuide JSON file if there is no sushi-config.yaml).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interesting I haven't thought on that one. For now I will add the TODO and then think about adding this feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants