diff --git a/docs/contribute-to-openms/adding-new-tool-to-topp.md b/docs/contribute-to-openms/adding-new-tool-to-topp.md index fdb6b6e6..eeee0cb0 100644 --- a/docs/contribute-to-openms/adding-new-tool-to-topp.md +++ b/docs/contribute-to-openms/adding-new-tool-to-topp.md @@ -30,15 +30,6 @@ add `-test` to the call of your TOPP tool and also create the expected output th flag active. The flag ensures that UniqueId's, dates etc are equal no matter where and when the tool is run. ``` -## What do I have to do to add a new UTILS tool? - -- Add the code to `src/utils/` and register it in `src/utils/executables.cmake`. -- Add your tool to `getUtilList()` in `src/openms/source/APPLICATIONS/ToolHandler.cpp`. This creates a doxygen page with - the `–help` output of the tool (using `TOPPDocumenter`). This page must be included at the end of the doxygen - documentation of your tool (see other tools for an example). -- Add it to the UTILS docu page (in `doc/doxygen/public/UTILS.doxygen`) -- Write a test (this is optional for UTILS). See TOPP tools above and add the test to the bottom of `src/tests/topp/CMakeLists.txt`. - ## I want to implement a new file adapter. What is to be done? First, add a file adapter class to the `include/OpenMS/FORMAT/` and `source/FORMAT/` folders. The file adapter should diff --git a/docs/develop-with-openms/link-external-code-to-openms.md b/docs/develop-with-openms/link-external-code-to-openms.md index 112c214e..03174608 100644 --- a/docs/develop-with-openms/link-external-code-to-openms.md +++ b/docs/develop-with-openms/link-external-code-to-openms.md @@ -1,7 +1,7 @@ External Code using OpenMS ========================== -If OpenMS' TOPP and UTILS tools are not enough in a certain scenario, you can either request a change to OpenMS, if you +If OpenMS' TOPP tools are not enough in a certain scenario, you can either request a change to OpenMS, if you feel this functionality is useful for others as well, or modify/extend OpenMS privately. For the latter, there are multiple ways to do this: @@ -100,7 +100,7 @@ installer for your platform (see our internal FAQ which is built using "make doc machine. If you've used an external project and have a new executable (+ an optional new library), use the installer approach as -well, and manually copy the new executable to the `TOPP/UTILS` binary directory (e.g. on Windows this could be +well, and manually copy the new executable to the `TOPP` binary directory (e.g. on Windows this could be `c:/program files/OpenMS/bin`, on Linux it could be `/bin`. If you do not use the installer, copy all required files manually, plus a few extra steps, see below. What needs to be diff --git a/docs/index.rst b/docs/index.rst index 1ba4c9d2..6c633a8c 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -52,7 +52,6 @@ Contents openms-applications-and-tools/installation.md openms-applications-and-tools/openms-graphical-user-interfaces.md openms-applications-and-tools/topp-tools.md - openms-applications-and-tools/utils-tools.md openms-applications-and-tools/command-line-interface.md openms-applications-and-tools/visualize-with-openms.md diff --git a/docs/openms-applications-and-tools/openms-applications/swathwizard.md b/docs/openms-applications-and-tools/openms-applications/swathwizard.md index e0698241..a91bffca 100644 --- a/docs/openms-applications-and-tools/openms-applications/swathwizard.md +++ b/docs/openms-applications-and-tools/openms-applications/swathwizard.md @@ -7,7 +7,7 @@ SwathWizard SwathWizard is an assistant for Swath analysis. The Wizard takes the user through the whole analysis pipeline for SWATH proteomics data analysis, i.e. the -[TOPP Documentation: OpenSwathWorkflow](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_OpenSwathWorkflow.html) tool, including downstream tools such as [GitHub:PyProphet/pyProphet](https://github.com/PyProphet/pyprophet) and the [GitHub:msproteomicstools/TRIC alignment](https://github.com/msproteomicstools/msproteomicstools) tool. +[TOPP Documentation: OpenSwathWorkflow](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_OpenSwathWorkflow.html) tool, including downstream tools such as [GitHub:PyProphet/pyProphet](https://github.com/PyProphet/pyprophet) and the [GitHub:msproteomicstools/TRIC alignment](https://github.com/msproteomicstools/msproteomicstools) tool. Since the downstream tools require Python and the respective modules, the Wizard will check their proper installation status and warn the user if a component is missing. diff --git a/docs/openms-applications-and-tools/topp-tools/types-of-topp-tools.md b/docs/openms-applications-and-tools/topp-tools/types-of-topp-tools.md index 4b924a0d..94c93573 100644 --- a/docs/openms-applications-and-tools/topp-tools/types-of-topp-tools.md +++ b/docs/openms-applications-and-tools/topp-tools/types-of-topp-tools.md @@ -41,9 +41,7 @@ The following tools are offered: - **Protein/Peptide identification** - **Protein/Peptide processing** - **Targeted experiments and OpenSWATH** -- **Peptide property prediction** - For more information, view the [Peptide property prediction](types-of-topp-tools/peptide-property-prediction.md) section. - **Cross-linking** Cross-linking is a technique where substances are chemically treated to create covalent bonds between different molecules. The strength of the covalent bonds can be quantified to indicate the proximity of certain molecules within a 3D structure. diff --git a/docs/openms-applications-and-tools/topp-tools/types-of-topp-tools/peptide-property-prediction.md b/docs/openms-applications-and-tools/topp-tools/types-of-topp-tools/peptide-property-prediction.md deleted file mode 100644 index 2159fe7c..00000000 --- a/docs/openms-applications-and-tools/topp-tools/types-of-topp-tools/peptide-property-prediction.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -orphan: true ---- -Peptide Property Prediction -=========================== - -Train a model for retention time prediction as well as for the prediction of proteotypic peptides. - -Two applications has been described in the following publications: Nico Pfeifer, Andreas Leinenbach, Christian G. Huber -and Oliver Kohlbacher Statistical learning of peptide retention behavior in chromatographic separations: A new -kernel-based approach for computational proteomics. BMC Bioinformatics 2007, 8:468 Nico Pfeifer, Andreas Leinenbach, -Christian G. Huber and Oliver Kohlbacher Improving Peptide Identification in Proteome Analysis by a Two-Dimensional -Retention Time Filtering Approach J. Proteome Res. 2009, 8(8):4109-15. - -The predicted retention time can be used in `IDFilter` to filter out false identifications. For data from several -identification runs: - -1. Align the data using MapAligner. -2. Use the various identification wrappers like `MascotAdapter` to get the identifications. -3. To train a model using `RTModel` use `IDFilter` for one of the runs to get the high scoring identifications (40 to 200 - distinct peptides should be enough). -4. Use `RTModel` as described in the documentation to train a model for these spectra. With this model, use `RTPredict` - to predict the retention times for the remaining runs. The predicted retention times are stored in the idXML files. - These predicted retention times can then be used to filter out false identifications using the `IDFilter` tool. - -A typical sequence of TOPP tools would look like this: - -```bash -MapAligner -in Run1.mzML,...,Run4.mzML -out Run1_aligned.mzML,...,Run4_aligned.mzML -MascotAdapter -in Run1_aligned.mzML -out Run1_aligned.idXML -ini Mascot.ini -MascotAdapter -in Run2_aligned.mzML -out Run2_aligned.idXML -ini Mascot.ini -MascotAdapter -in Run3_aligned.mzML -out Run3_aligned.idXML -ini Mascot.ini -MascotAdapter -in Run4_aligned.mzML -out Run4_aligned.idXML -ini Mascot.ini -IDFilter -in Run1_aligned.idXML -out Run1_best_hits.idXML -pep_fraction 1 -best_hits -RTModel -in Run1_best_hits.idXML -out Run1.model -ini RT.ini -RTPredict -in Run2_aligned.idXML -out Run2_predicted.idXML -svm_model Run1.model -RTPredict -in Run3_aligned.idXML -out Run3_predicted.idXML -svm_model Run1.model -RTPredict -in Run4_aligned.idXML -out Run4_predicted.idXML -svm_model Run1.model -IDFilter -in Run2_predicted.mzML -out Run2_filtered.mzML -rt_filtering -IDFilter -in Run3_predicted.mzML -out Run3_filtered.mzML -rt_filtering -IDFilter -in Run4_predicted.mzML -out Run4_filtered.mzML -rt_filtering -``` - -For a file with certainly identified peptides used to train a model for RT prediction, use the IDs. Therefore, the file -has to have one peptide sequence together with the RT per line (separated by one tab or space). This can then be loaded -by `RTModel` using the `-textfile_input` flag: - -```bash -RTModel -in IDs_with_RTs.txt -out IDs_with_RTs.model -ini RT.ini -textfile_input -``` - -The likelihood of a peptide to be proteotypic can be predicted using `PTModel` and `PTPredict`. - -Assume we have a file `PT.idXML` which contains all proteotypic peptides of a set of proteins. Lets also assume, we have -a fasta file containing the amino acid sequences of these proteins called `mixture.fasta`. To be able to train `PTPredict`, -negative peptides (peptides, which are not proteotypic) are required. Therefore, one can use the Digestor, which is -located in the `APPLICATIONS/UTILS/` folder together with the `IDFilter`: - -```bash -Digestor -in mixture.fasta -out all.idXML -IDFilter -in all.idXML -out NonPT.idXML -exclusion_peptides_file PT.idXML - -``` - -In this example the proteins are digested in silico and the non proteotypic peptides set is created by subtracting all -proteotypic peptides from the set of all possible peptides. Then, train `PTModel`: - -`PTModel -in_positive PT.idXML -in_negative NonPT.idXML -out PT.model -ini PT.ini` diff --git a/docs/openms-applications-and-tools/utils-tools.md b/docs/openms-applications-and-tools/utils-tools.md deleted file mode 100644 index 9943d909..00000000 --- a/docs/openms-applications-and-tools/utils-tools.md +++ /dev/null @@ -1,17 +0,0 @@ -UTILS Tools -=========== - -There are also a number of tools in the beta stage called UTILS. They include but are not limited to: - -- Signal processing and preprocessing -- File handling -- Algorithm evaluation -- Protein/peptide identification -- Cross-linking -- Quantitation -- Metabolite identification -- Targeted experiments and OpenSWATH -- RNA -- Quality control - -Additional information for the [ProteomicsLFQ](../topp-and-utils/proteomicslfq.md) Quantitation UTILS tool is provided. For the full list of UTILS tools, visit the [API Reference](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_documentation.html) website. diff --git a/docs/quick-reference/contributor-faq.md b/docs/quick-reference/contributor-faq.md index eef99b8b..79fcf683 100644 --- a/docs/quick-reference/contributor-faq.md +++ b/docs/quick-reference/contributor-faq.md @@ -17,11 +17,6 @@ The following section provides general information to new contributors. * Subscribe to the [open-ms-general](https://sourceforge.net/projects/open-ms/lists/open-ms-general) or [contact us](/quick-reference/contact-us.md). -### What is the difference between an OpenMS tool and util? - -A tool starts its lifecycle in `UTILS` and may exist without being thoroughly tested. Tools may be promoted from `UTILS` -to `TOOLS` if they are stable enough, are fully tested, fully documented, and a test workflow exists. - ### I have written a class for OpenMS. What should I do? Follow the [OpenMS coding conventions](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/coding_conventions.html). @@ -167,7 +162,7 @@ Find a definition of the main page [here](https://github.com/OpenMS/OpenMS/edit/ Add a new module [here](https://github.com/OpenMS/OpenMS/edit/develop/doc/doxygen/public/Modules.doxygen). -### How is the command line documentation for TOPP/UTILS tools created? +### How is the command line documentation for TOPP tools created? The program `OpenMS/doc/doxygen/parameters/TOPPDocumenter.cpp` creates the command line documentation for all classes that are included in the static `ToolHandler.cpp` tools list. It can be included in the documentation using diff --git a/docs/quick-reference/developer-faq.md b/docs/quick-reference/developer-faq.md index 19437f60..4cdb2410 100644 --- a/docs/quick-reference/developer-faq.md +++ b/docs/quick-reference/developer-faq.md @@ -17,11 +17,6 @@ The following section provides general information to new contributors. * Subscribe to the [open-ms-general](https://sourceforge.net/projects/open-ms/lists/open-ms-general) or [contact-us](/quick-reference/contact-us.md). -### What is the difference between an OpenMS tool and util? - -A tool starts its lifecycle in `UTILS` and may exist without being thoroughly tested. Tools may be promoted from `UTILS` -to `TOOLS` if they are stable enough, are fully tested, fully documented, and a test workflow exists. - ### I have written a class for OpenMS. What should I do? Follow the [OpenMS coding conventions](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/coding_conventions.html). @@ -204,10 +199,10 @@ to learn more about contributing to releases. ## Working in Integrated Development Environments (IDEs) -### Why are there no `source/TEST` and `source/APPLICATIONS/TOPP|UTILS` folder? +### Why are there no `source/TEST` and `source/APPLICATIONS/TOPP` folder? All source files added to an IDE are associated with their targets. Find the source files for each test within -its own subproject. The same is true for the `TOPP` and `UTILS` classes. +its own subproject. The same is true for the `TOPP` classes. ### I'm getting the error "Error C2471: cannot update program database" @@ -347,22 +342,17 @@ generates a html table with the parameters. This table can then be included in t following `doxygen` command:`@htmlinclude OpenMS_.parameters`. ```{note} -Parameter documentation is automatically generated for `TOPP/UTILS` included in the static `ToolHandler.cpp` tools list. +Parameter documentation is automatically generated for `TOPP` included in the static `ToolHandler.cpp` tools list. ``` -To include TOPP/UTILS parameter documentation use following `doxygen` command: - +To include TOPP parameter documentation use following `doxygen` command: `@htmlinclude TOPP_.parameters` -or - -`@htmlinclude UTILS_.parameters` - Test if everything worked by calling `make doc_param_internal`. The parameters documentation is written to `OpenMS/doc/doxygen/parameters/output/`. -### How is the command line documentation for TOPP/UTILS tools created? +### How is the command line documentation for TOPP tools created? The program `OpenMS/doc/doxygen/parameters/TOPPDocumenter.cpp` creates the command line documentation for all classes that are included in the static `ToolHandler.cpp` tools list. It can be included in the documentation using the diff --git a/docs/quick-reference/glossary.md b/docs/quick-reference/glossary.md index 7fbc0048..f1edc165 100644 --- a/docs/quick-reference/glossary.md +++ b/docs/quick-reference/glossary.md @@ -88,9 +88,6 @@ pyOpenMS TOPP Tools OpenMS provides a number of functions that process mass spectrometry data called TOPP tools. More information on TOPP tools can be found in the [OpenMS API reference documentation](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_documentation.html). -UTILS - Besides {term}`TOPP`, OpenMS offers a range of other tools. They are not included in {term}`TOPP` as they are not part of typical analysis pipelines. More information is present in [OpenMS UTILS Documentation](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_documentation.html). - TOPPView TOPPView is a viewer for MS and HPLC-MS data. More information is available in [TOPPView documentation](../openms-applications-and-tools/visualize-with-openms.md). diff --git a/docs/research/whitepapers/OpenMS-2.0-kernel-whitepaper.md b/docs/research/whitepapers/OpenMS-2.0-kernel-whitepaper.md deleted file mode 100644 index 1032adc1..00000000 --- a/docs/research/whitepapers/OpenMS-2.0-kernel-whitepaper.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -orphan: true ---- -OpenMS 2.0 Kernel Whitepaper -=========================== - -## Introduction - -With the constantly increasing amount of data that needs to be handled in computational mass spectrometry we start to -reach the boundary of what is possible given the current design and implementation of the OpenMS kernel. Already in 2012 -the OpenMS core team discussed and identified the need for a redesign of the OpenMS kernel with respect to it's scalability -and ease of use. So far, only partial solutions have been proposed that lack the approval of the OpenMS core team as -well as the OpenMS community. - -This document summarises the discussion of the OpenMS core team, held at the OpenMS developer retreat in Izmir, -Turkey 2014. - -## Detailed problem description - -With modern mass spectrometry instruments and measurement techniques the amounts of data that need to be analysed in -computational mass spectrometry reach magnitudes that do not fit into the main memory of neither desktop nor clusters -machines. Therefore the OpenMS kernel has to be redesigned to decouple the algorithms from the actual implementation of -the data structures. - -Here we describe the interfaces and concepts we plan to implement for OpenMS 2.0. The OpenMS team has acknowledged the -fact, that this will only be possible in an incremental way, i.e., implementing those interfaces on top of the existing -data structures and then switch the individual classes to the new design, instead of a large redesign in a separate -development line. - -## Benchmarking the changes - -Before we start to implement individual changes we intend to create a benchmark suite. This benchmark suite should -enable us to assess the changes in memory consumption and runtime due to the new data structures. - -## Spectrum/Chromatogram interface - -The first decision reached during the discussion was to abandon the `Peak` data structure. It will be replaced by a pair -of `double`s such that a spectrum/chromatogram is a combination of two `std::vector`. - -```cpp -class RawDataArray -{ - std::vector intensities_; - std::vector positions_; -}; -``` - -with invariant `intensities.size() == positions.size()`. This data structure will be augmented with a container to hold -position-wise meta data (see for instance `FloatDataArray`). - -```cpp -class RawDataArray -{ - typedef std::vector MetaDataArray; - typedef std::string MetaDataID; - - std::vector intensities_; - std::vector positions_; - std::map meta_; -}; -``` - -Given this base class `Spectrum` and `Chromatogram` can be derived from this class, redefining `position_` to either -`m/z` or `rt`. Additionally the `Spectrum` and `Chromatogram` will add relevant members (e.g., MS-level for `Spectrum`) -that should not be stored as meta data. - -``` -class Spectrum : public RawDataArray -{ - int ms_level_; - Precursor precursor_; - double rt_; - int scan_id_; -}; -``` - -The overall design is summarised in the following figure. - -![](/images/research/whitepapers/summary.png) - -## MSRun (was MSExperiment) - -`MSExperiment` will be renamed to `MSRun`. On top of MSRun we will define an interface that hides the actual implementation -of `MSRun` from the user allowing the use of cached, on-disc, or full in-memory implementations. Direct access to -chromatograms will be hidden behind the `ISpectrumAccess` and `IChromatogramAccess` interfaces. - -```cpp -class ISpectrumAccess -{ -public: - int getNrSpectra() const; - getMetaSpectrum(int i) const; - const ISpectrum& getSpectrum(int i) const; - const ExpMeta& getExperimentMeta() const; - // iterators for spectra - SpectrumIterator spectrumBegin(); - SpectrumIterator spectrumEnd(); -}; - -class IChromatogramAccess -{ -public: - int getNrChromatograms() const; - getMetaSpectrum(int i) const; - const IChromatogram& getChromatogram(int i) const; - const ExpMeta& getExperimentMeta() const; - // iterators for chromatograms - ChromatogramIterator chromatogramBegin(); - ChromatogramIterator chromatogramEnd(); -}; -``` - -The overall design is summarised in the following figure. - -![](/images/research/whitepapers/design-summary.png) - -## Modularisation - -We decided that the OpenMS library has to be modularised. Starting with SuperHirn as a proof of concept we will identify -other parts of the library that should be an independent module and move them into a separate library. Additionally we -will add a new library `libOpenMSChemistry` that should implement light-weight alternatives to the existing chemistry -data structures. - -To avoid the direct integration of classes associated to UTILS into OpenMS, we will additionally support multi-file UTILS -compared to the existing TOPP like utils, that should consist of only one file. diff --git a/docs/run-workflows-with-openms-tools/openms-in-knime.md b/docs/run-workflows-with-openms-tools/openms-in-knime.md index 1e44cdc1..e718e1d0 100644 --- a/docs/run-workflows-with-openms-tools/openms-in-knime.md +++ b/docs/run-workflows-with-openms-tools/openms-in-knime.md @@ -1,7 +1,7 @@ OpenMS in KNIME =============== -KNIME is a free graphical application for creating and executing workflows. OpenMS provides a plugin, which exposes TOPP and UTILS tools to the KNIME interface. +KNIME is a free graphical application for creating and executing workflows. OpenMS provides a plugin, which exposes TOPP tools to the KNIME interface. ```{toctree} diff --git a/docs/topp-and-utils/swathwizard.md b/docs/topp-and-utils/swathwizard.md index 63969ca1..9b7e80d3 100644 --- a/docs/topp-and-utils/swathwizard.md +++ b/docs/topp-and-utils/swathwizard.md @@ -7,7 +7,7 @@ SwathWizard An assistant for Swath analysis. The Wizard takes the user through the whole analysis pipeline for SWATH proteomics data analysis, i.e. the -[TOPP Documentation: OpenSwathWorkflow](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_OpenSwathWorkflow.html) tool, including downstream tools such as [GitHub:PyProphet/pyProphet](https://github.com/PyProphet/pyprophet) and the [GitHub:msproteomicstools/TRIC alignment](https://github.com/msproteomicstools/msproteomicstools) tool. +[TOPP Documentation: OpenSwathWorkflow](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/TOPP_OpenSwathWorkflow.html) tool, including downstream tools such as [GitHub:PyProphet/pyProphet](https://github.com/PyProphet/pyprophet) and the [GitHub:msproteomicstools/TRIC alignment](https://github.com/msproteomicstools/msproteomicstools) tool. Since the downstream tools require Python and the respective modules, the Wizard will check their proper installation status and warn the user if a component is missing. diff --git a/docs/tutorials-and-quickstart-guides/guides/user-quickstart-guide.md b/docs/tutorials-and-quickstart-guides/guides/user-quickstart-guide.md index db5b3fdd..0a1d220b 100644 --- a/docs/tutorials-and-quickstart-guides/guides/user-quickstart-guide.md +++ b/docs/tutorials-and-quickstart-guides/guides/user-quickstart-guide.md @@ -18,14 +18,8 @@ Before using OpenMS, become familiar with the following terms: | Tool and Utilities | Description | |--------------------|-------------| |**TOPPView** |A tool that is used to view and explore {term}`LC-MS` data, alignments, groups, peptide identifications, and more.| -|**TOPPAS** |A graphical workflow design tool that is used to create pipelines from all {term}`TOPP tools` (and {term}`UTILS`).| +|**TOPPAS** |A graphical workflow design tool that is used to create pipelines from all {term}`TOPP tools`.| |**TOPP tools** |A set of command line tools. Each of these command line tools is a building block of an analysis pipeline and are chained together in a way that fits the requirements of the user. The {term}`TOPP tools` are accessible from a command prompt/shell or via {term}`TOPPAS`. | -|**UTILS** |Besides {term}`TOPP`, OpenMS offers range of other tools. They are not included in {term}`TOPP` as they are not part of typical analysis pipelines. A set of command line utilities, similar to {term}`TOPP tools`, mostly used during pipeline construction or parameter optimization. | - -```{seealso} - -[UTILS documentation](https://abibuilder.cs.uni-tuebingen.de/archive/openms/Documentation/nightly/html/UTILS_documentation.html)| -``` ```{important} Users can now use {term}`KNIME` in place of {term}`TOPPAS`; the later will deprecated with no support in near future.