Merge pull request #398 from JetBrains-Research/changelog-update-v0.3.0

Changelog update - `v0.3.0`

.github/workflows/build.yml

@@ -78,8 +78,6 @@ jobs:
           echo "changelog=$CHANGELOG" >> $GITHUB_OUTPUT
           echo "pluginVerifierHomeDir=~/.pluginVerifier" >> $GITHUB_OUTPUT
 
-          ./gradlew listProductsReleases # prepare list of IDEs for Plugin Verifier
-
 
       # Run tests
       - name: Run Tests
@@ -88,7 +86,7 @@ jobs:
       # Collect Tests Result of failed tests
       - name: Collect Tests Result
         if: ${{ failure() }}
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v4.4.0
         with:
           name: tests-result
           path: ${{ github.workspace }}/build/reports/tests
@@ -102,12 +100,12 @@ jobs:
 
       # Run Verify Plugin task and IntelliJ Plugin Verifier tool
       - name: Run Plugin Verification tasks
-        run: ./gradlew runPluginVerifier -Pplugin.verifier.home.dir=${{ steps.properties.outputs.pluginVerifierHomeDir }}
+        run: ./gradlew verifyPlugin
 
       # Collect Plugin Verifier Result
       - name: Collect Plugin Verifier Result
         if: ${{ always() }}
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v4.4.0
         with:
           name: pluginVerifier-result
           path: ${{ github.workspace }}/build/reports/pluginVerifier
@@ -129,7 +127,7 @@ jobs:
 
       # Store already-built plugin as an artifact for downloading
       - name: Upload artifact
-        uses: actions/upload-artifact@v2.2.4
+        uses: actions/upload-artifact@v4.4.0
         with:
           name: ${{ steps.artifact.outputs.filename }}
           path: ./build/distributions/content/*/*

.run/Build Plugin.run.xml

@@ -0,0 +1,24 @@
+<component name="ProjectRunConfigurationManager">
+  <configuration default="false" name="Build Plugin" type="GradleRunConfiguration" factoryName="Gradle">
+    <ExternalSystemSettings>
+      <option name="executionName" />
+      <option name="externalProjectPath" value="$PROJECT_DIR$" />
+      <option name="externalSystemIdString" value="GRADLE" />
+      <option name="scriptParameters" value="" />
+      <option name="taskDescriptions">
+        <list />
+      </option>
+      <option name="taskNames">
+        <list>
+          <option value="buildPlugin" />
+        </list>
+      </option>
+      <option name="vmOptions" />
+    </ExternalSystemSettings>
+    <ExternalSystemDebugServerProcess>true</ExternalSystemDebugServerProcess>
+    <ExternalSystemReattachDebugProcess>true</ExternalSystemReattachDebugProcess>
+    <DebugAllEnabled>false</DebugAllEnabled>
+    <RunAsTest>false</RunAsTest>
+    <method v="2" />
+  </configuration>
+</component>

.run/Run IDE for UI Tests.run.xml

@@ -5,18 +5,21 @@
       <option name="executionName" />
       <option name="externalProjectPath" value="$PROJECT_DIR$" />
       <option name="externalSystemIdString" value="GRADLE" />
-      <option name="scriptParameters" value="runIdeForUiTests" />
+      <option name="scriptParameters" value="--refresh-dependencies --stacktrace" />
       <option name="taskDescriptions">
         <list />
       </option>
       <option name="taskNames">
-        <list />
+        <list>
+          <option value=":runIde" />
+        </list>
       </option>
       <option name="vmOptions" />
     </ExternalSystemSettings>
     <ExternalSystemDebugServerProcess>true</ExternalSystemDebugServerProcess>
     <ExternalSystemReattachDebugProcess>true</ExternalSystemReattachDebugProcess>
     <DebugAllEnabled>false</DebugAllEnabled>
+    <RunAsTest>false</RunAsTest>
     <method v="2" />
   </configuration>
 </component>

CHANGELOG.md

@@ -4,6 +4,21 @@
 
 ## Unreleased
 
+## 0.3.0
+
+### Fixed
+- Multiple bugs in the UI.
+- Multiple bugs in the LLM-based test generation algorithm.
+
+### Changed
+- Major refactoring: removed most of the global services.
+- Migration to Gradle IntelliJ Plugin 2.x,
+
+### Added
+- Full LLM-based Kotlin test generation for line, method/function, and class.
+- Integration with HuggingFace.
+- Support IDEA `242.*`.
+
 ## 0.2.1
 
 ### Fixed

CONTRIBUTING.md

@@ -1,5 +1,15 @@
 # TestSpark
 
+## Table of contents
+- [Description](#description)
+- [Build project](#build-project)
+- [Run IDE for UI tests](#run-IDE-for-ui-tests)
+- [Plugin Configuration](#plugin-configuration-file)
+- [Language Support Documentation](#language-support-documentation)
+- [Classes](#classes)
+- [Tests](#tests)
+
+
 ## Description
 
 In this document you can find the overall structure of TestSpark plugin. The classes are listed and their purpose is described. This section is intended for developers and contributors to TestSpark plugin.
@@ -15,14 +25,147 @@ to include test generation using Grazie in the build process, you need to pass S
 
 ## Run IDE for UI tests
 In IDEA, run the `Run IDE for UI Tests` configuration. Alternatively, use the following command:
-`gradle runIdeForUiTests`.
+`gradle runIde`.
 
 ### Including Test generation using Grazie (for JetBrains employees only!)
 to include test generation using Grazie in the runIdeForUiTests process, you need to pass Space username and token as properties:
-`gradle runIdeForUiTests -Dspace.username=<USERNAME> -Dspace.pass=<TOKEN>`.
+`gradle runIde -Dspace.username=<USERNAME> -Dspace.pass=<TOKEN>`.
 
 `<TOKEN>` is generated by Space, which has access to Automatically generating unit tests maven packages.
 
+---
+
+## Language Support Documentation
+
+The TestSpark plugin supports automatic test generation for various programming languages (currently Java and Kotlin)
+and aims to support even more programming languages in the future.
+
+This document provides an overview of the existing implementation of Kotlin and Java support and guidelines for adding
+more programming languages.
+
+>How can I add support for a new programming language? 
+In brief, you need to extend all the necessary interfaces with implementations specific to the new language.
+Below, you will find a detailed guide divided into six key components of the entire pipeline with the most 
+important interfaces addressing this goal. 
+
+
+## Key Components
+
+### 1. PSI Parsers
+
+The first step is to enable the collection of the appropriate information for the code under test. This part is
+responsible for working with the PSI (Program Structure Interface) generated by IntelliJ IDEA. It helps parse the part
+where the cursor is located, provides a choice of the code elements that are available for testing at cursor's position.
+Then find all the needed dependencies to make the prompt complete with all the necessary knowledge about the code under
+test.
+
+This part is the most granular but complex at the same time.
+
+The main reason for this is to include dependencies only for the languages we need. This avoids errors if the user does
+not have some languages that our plugin supports. _For example, if we work with a Python project, we don't want to depend
+on Kotlin because it will cause an error if Kotlin isn't present._
+
+Additionally, we want to incrementally add dependencies on other languages for faster startup.
+_For example, we do not want to fetch the dependency on Java when we work with TypeScript._
+Other benefits include better organization, easier maintenance, and clearer separation of
+concerns. As a side-bonus, the addition of new languages will be easier.
+
+**Module Dependencies:**
+
+- **langwrappers**: This is a foundational module for language extensions.
+- **<Language>**: Depends on the `langwrappers` module to implement the `<Language>`-specific `PsiHelper`
+  and `PsiHelperProvider`.
+- **src/**: Depends on `langwrappers` because we want to use `PsiHelper` and other interfaces regardless of the current
+  language. Depends on `<Language>`, to make `plugin.xml` aware of the implementations of the Extension Point.
+
+**Plugin Dependencies:**
+
+- The main `plugin.xml` file declares the `psiHelperProvider` extension point using
+  the `com.intellij.lang.LanguageExtensionPoint` class.
+- The language-specific modules extend this extension point to register their implementations.
+- When the project is opened, we load the EPs needed to work with the current project. Then, using
+  the `PsiHelperProvider` interface, we can get the appropriate `<language>PsiHelper` class per file.
+
+**Implementation Details:**
+
+- **Common Module (`langwrappers`)**:
+    - Contains the `PsiHelper` interface, which provides the necessary methods to interact with `psiFile`.
+    - The `PsiHelperProvider` class includes a companion object to fetch the appropriate `PsiHelper` implementation
+      based on the file's language.
+
+- **<Language> Module**:
+    - Implements the `<Language>PsiHelper` and `<Language>PsiHelperProvider` classes, which provide <Language>-specific
+      logic.
+    - Declares the extension point in `testspark-<Language>.xml`.
+
+To add new languages, create a separate module for this language and register its implementation as an extension of
+the `psiHelperProvider` EP. Then follow the template provided above.
+
+### 2. Prompt Generation
+
+When we know how to parse the code, we need to construct the prompt.
+
+For each language, adjust the prompt that goes to the LLM. Ensure that the language, framework platform, and mocking
+framework are defined correctly in:
+
+```kotlin
+data class PromptConfiguration(
+    val desiredLanguage: String,
+    val desiredTestingPlatform: String,
+    val desiredMockingFramework: String,
+)
+```
+
+Additionally, check that all the dependencies (collected by `PsiHelper` for the current strategy) are passed
+properly. `PromptGenerator` and `PromptBuilder` are responsible for this job.
+
+### 3. Parsing LLM Response
+
+When the LLM response to our prompt is received, we have to parse it.
+
+We want to retrieve test case, all the test functions and additional information like imports or supporting functions
+from the response.
+
+The current structure of this part is located in:
+
+- `kotlin/org/jetbrains/research/testspark/core/test`
+- `kotlin/org/jetbrains/research/testspark/tools`
+
+It can be more easily understood with the following diagram:
+![](https://private-user-images.githubusercontent.com/70476032/349256986-dc7e1ff9-a9a5-4bd2-a51f-ecbfabeb6cba.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjIzNTEyOTAsIm5iZiI6MTcyMjM1MDk5MCwicGF0aCI6Ii83MDQ3NjAzMi8zNDkyNTY5ODYtZGM3ZTFmZjktYTlhNS00YmQyLWE1MWYtZWNiZmFiZWI2Y2JhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MzAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzMwVDE0NDk1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWJjMDg3MWM2ZDA4MDJlZGUwNzliMzNkNzA3YWI4YTcwM2RmYTFjMmE1MGM4MjM5NjJiOGI2ZjgxNTE2OTU2YjQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.8OfRa1wJhDfFq3QT6h5yIjBh1VqB9UrrQfZGp0_SLDo)
+
+- `TestsAssembler`: Assembler class for generating and organizing test cases from the LLM response.
+- `TestSuiteParser`: Extracts test cases from raw text and generates a test suite.
+- `TestBodyPrinter`: Generates the body of a test function as a string.
+
+### 4. Compilation
+
+Before showing the code to the user, it should be checked for compilation.
+
+- `TestCompiler`: Compiles a list of test cases and returns the compilation result.
+
+Here one should specify the appropriate compilation strategy for each language. With all the dependencies and build paths.
+
+### 5. UI Representation
+
+Once the code generated by the LLM is checked for the compilation, it should be presented in the UI.
+
+- `TestCaseDisplayService`: Service responsible for the representation of all the UI components.
+- `TestSuiteView`: Interface specific for working with buttons.
+- `TestClassCodeAnalyzer`: Interface for retrieving information from test class code.
+- `TestClassCodeGenerator`: Interface for generating and formatting test class code.
+
+### 6. Running and saving tests
+
+We should be able to run all the tests in the UI and then save them to the desired folder.
+
+- `TestPersistentStorage`: Interface representing a contract for saving generated tests to a specified file system location.
+
+For Kotlin and Java, the `TestProcessor` implementation also allows saving the JaCoCo report to see the code coverage of 
+the test that will be saved.
+
+---
+
 ## Plugin Configuration File
 
 The plugin configuration file is `plugin.xml` which can be found in `src/main/resources/META-INF` directory. All declarations (such as actions, services, listeners) are present in this file.
@@ -86,30 +229,7 @@ All the listener classes can be found in `listeners` directory.
 ### Services
 
 All the service classes can be found in `services` directory.
-
-- `CoverageSelectionToggleListener` is a topic interface for showing or hiding highlighting when the coverage is toggled for one test or many tests.
-- `CoverageToolWindowDisplayService` creates the *"Coverage visualisation"* tool window panel and the coverage table to display the test coverage data of the tests generated by EvoSuite.
-- `CoverageVisualisationService` visualises the coverage in the gutter and the editor (by colouring), injects the coverage data into the *"Coverage visualisation"* tool window tab.
-- `ErrorService`service class for handling error occurrences.
-- `QuickAccessParametersService` allows to load and get the state of the parameters in the *"Parameters"* tool window panel.
-- `RunnerService` is used to limit TestSpark to generate tests only once at a time.
-- `SettingsApplicationService` stores the application-level settings persistently. It uses `SettingsApplicationState` class for that.
-- `SettingsProjectService` stores the project-level settings persistently. It uses `SettingsProjectState` class for that.
-- `StaticInvalidationService` invalidates the cache statically.
-- `TestCaseCachingService` contains the data structure for caching the generated test cases and is responsible for adding, retrieving and removing (invalidating) the generated tests.
-- `TestCaseDisplayService` displays the tests generated by EvoSuite, in the *"Generated tests"* tool window panel.
-- `TestSparkTelemetryService` sends usage information to Intelligent Collaboration Tools Lab at JetBrains Research if the user has opted in.
-
-### Settings
-
-All the classes related to TestSpark `Settings/Preferences` page can be found in `settings` directory.
-
-- `SettingsApplicationState` is responsible for storing the values of the EvoSuite Settings entries.
-- `SettingsEvoSuiteComponent` displays and captures the changes to the values of the entries in the EvoSuite page of the Settings dialog.
-- `SettingsEvoSuiteConfigurable` allows to configure some EvoSuite settings via the EvoSuite page in the Settings dialog, observes the changes and manages the UI and state.
-- `SettingsPluginComponent` displays and captures the changes to the values of the entries in the TestSpark main page of the Settings dialog.
-- `SettingsPluginConfigurable` allows to configure some Plugin settings via the Plugin page in the Settings dialog, observes the changes and manages the UI and state.
-- `SettingsProjectState` is responsible for storing the values of the Plugin Settings entries.
+We currently have three services for managing EvoSuite settings (`EvoSuiteSettingsService`), the LLM-based approach/generation (`LLMSettingsService`), and general plugin settings (`PluginSettingsService`).
 
 ### Tools
 

README.md

@@ -21,14 +21,14 @@ TestSpark is a plugin for generating unit tests. TestSpark natively integrates d
 
 TestSpark currently supports two test generation strategies:
 <ul>
-        <li>LLM-based test generation (using <a href="https://openai.com">OpenAI</a> and JetBrains internal AI Assistant platform)</li>
+        <li>LLM-based test generation (using <a href="https://openai.com">OpenAI</a>, HuggingFace, and JetBrains internal AI Assistant platform)</li>
         <li>Local search-based test generation (using <a href="https://www.evosuite.org">EvoSuite</a>)</li>
 </ul>
 <h4>LLM-based test generation</h4>
     <p>For this type of test generation, TestSpark sends request to different Large Language Models. Also, it automatically checks if tests are valid before presenting it to users.</p>
-    <p>This feature needs a token from OpenAI platform or the AI Assistant platform.</p>
+    <p>This feature needs a token from OpenAI, HuggingFace, or the AI Assistant platform.</p>
     <ul>
-        <li>Supports Java (any version).</li>
+        <li>Supports Java (any version) and Kotlin (K2 mode should be disabled, checkout the Settings section on README).</li>
         <li>Generates unit tests for capturing failures.</li>
         <li>Generate tests for Java classes, methods, and single lines.</li>
     </ul>
@@ -70,6 +70,7 @@ If you are running the plugin for the first time, checkout the [Settings](#setti
 - [Coverage](#coverage)
 - [Integrating tests into the project](#integrating-tests-into-the-project)
 - [Settings](#settings)
+- [Disable K2 for Kotlin Test Generation](#disable-K2)
 - [Telemetry](#telemetry-opt-in)
 
 ### Generating Tests
@@ -229,7 +230,9 @@ Or to a new file:
 
 ![Tests adding to a new file](readme-images/gifs/AddingToANewFile.gif#gh-light-mode-only)
 ![Tests adding to a new file_dark](readme-images/gifs/AddingToANewFile_dark.gif#gh-dark-mode-only)
-
+### Disable K2
+For LLM-based Kotlin test generation, you need to disable the K2 mode for now.
+![Disable K2 mode](readme-images/pngs/k2-mode/disable-k2.png)
 ### Settings
 <!-- How can users configure the plugin to match their needs? -->
 The plugin is configured mainly through the Settings menu. The plugin settings can be found under <kbd>Settings</kbd> > <kbd>Tools</kbd> > <kbd>TestSpark</kbd>. Here, the user is able to select options for the plugin: