Relevant Issues and Features - From RAVEN v1.1 to v2.0

[alfoa]

---

Issue Description

This issue is a floating tracking system for recording all the main defects and features addressed from a release to the other.
Base Release: RAVEN v.1.1
Next Release: RAVEN v.2.0

Addressed Defects:

• [DEFECT] Heavy Tests for Simulated Annealing timing out #1219
• [DEFECT] Fix failed heavy test  #1186
• [DEFECT] RELAP5 interface restart function failure #1185
• [DEFECT] Windows plugin tests failing #1176
• Windows doesn't find TestHarness directory for plugins #1162
• [DEFECT] CashFlow doesn't work in windows #1157
• [DEFECT] MOOSE pedantic check #1152
• [DEFECT] Relap5 Interface Minor Edit mis-collection #1143
• [DEFECT] Plugin installation procedure #1109
• [DEFECT] Cannot Sample Multiple RELAP5 Variables on the Same Line #1103
• [DEFECT] Input reader doesn't strip Out whitespace when reading array #1102
• [DEFECT] tensorflow_keras.tf_cnn1d test time out  #1101
• [DEFECT] Error starting on non-Windows platforms #1099
• [DEFECT] Heavy Test Skip #1089
• Coverage not running Topology UI #1085
• shutil.SameFileError while copy slave files #1080
• [DEFECT] Coverage broken #1076
• FileExistsError when making new folder #1070
• CIVET: 'Test masters' failure #1068
• METADATA in Interfaced PP - Missing  #1065
• Custom sampler do not take constant in Raven code interface #1057
• force matplotlib backend to Agg in unSupervisedLearning #1048
• Interactive plots don't close Python when closing on OSX #1043
• tests on linux-type machines failed due to bad python 3.7.4 build in conda #1036
• fix test in raven master branch #1031
• expr syntax error when build raven documents #1013
• Number type (int vs. double) in Sampler #1003
• Singke Sample in Optimizer crashes run. #1002
• RELAP5 code interface #995
• improvement of HistorySet MetaData Printing #986
• Fix some Win test #982
• Data Object error message #981
• Grid Sampler Crashed with Integer Grids #973
• PIL tests fail in Python3 #940
• batchSize = 1 #939
• EnsembleModel cannot be re-used #859
• Test failed due to PR#777 #823
• MOOSE Input Incorrect Parse of Multi-line String #794
• test framework/Optimizers.multitrajlevelWithModelHolding shows prematurely exit #763
• The calculation for the percentile in basicStatistics pp is inconsistent #756
• Code using input files in other directories not thread safe #753
• RrR inner run fail results in Optimizer crash #745
• If illegal XML named variables are present, the code crashes #733
• ensemble model pb weights for variables coming from functions #557
• MOOSE Input Parse Fails on Hanging Quote #540
• Regression test script runs plugin tests multiple times #475
• Unexpected exit in optimizer #269
• Disappearing Data on Plots in 3D #263
• Probability Weights on edge cases of Grid Sampler #185
• Pickling high-sample GaussianProcess ROM #134
• Possible bug in MAAP interface.  #95
• Why does OpenModelica pass the xsd test?  #76

New Features:

• Parallelization and Performance:
  • Completely restructured parallelism in RAVEN. Replacing the old Parallel Python system with the modern and powerful RAY library for parallel dispatching and HPC massive parallelism. New “RAVEN decorators” to ease the interaction with parallel systems has been deployed
  • Deployed a new system (lazy importing) for isolating the import of heavy libraries and algorithms if not requested by the user (i.e. all the main classes are now instantiated on demand, when they are used in an input file). This system drastically improved the start-up time (the time to instantiate the code), now being almost instantaneous.
• Documentation:
  • Updated RAVEN documentation (user manual, user guide, tests’ description) for a clearer explanation of the different features, such as Post Processors and Optimizers.
  • Updated workshop material:
    • Updated Introduction presentation (lighter and synchronized with the current development)
    • Added section (lecture) about hybrid modeling examples
    • Added section (lecture) about synthetic time series generation (e.g. ARMA, VARMA)
• Code interfaces:
  • New interfaces:
    • NEUTRINO: SPH-based highly accurate fluid solver, currently used simulate coastal inundation and flooding scenarios but extensible to support various fluid dynamic flows
  • Major updates:
    • RELAP5-3D: improved robustness of interface for restart mode. Addressed few issues caused by the previous Python3 upgrade. Added the feature to sample RELAP5-3D when cards are inputted in multiple lines.
    • MOOSE-based App: Extended the robustness of the interface to handle a wider variety of inputs, including: Multiline vectors, Scalar entries on different lines than their keys, ordered inputs, hanging vector opening/closing, Multiple "root" nodes in file. In addition, added the possibility to perturb external input files (with the Generic Code interface approach) in the interface; this capability is useful when the App-specific auxiliary files need to be perturbed (e.g. CSVs, XMLs, etc.).
• Samplers:
  • Improvement of Samplers’ performances
  • Improvement of the robustness of the Restart capabilities for Samplers (default tolerances and handling have been modified to allow for OS differences)
  • Improvement of Limit Surface search performance
  • Addition of a new Adaptive Sampler based on the convergence on the error on statistical moments (named, Adaptive Montecarlo)
• Optimizers:
  • New Optimization system with support for customizable APIs:
    • flexible brand-new API for the ease development of optimization algorithms (both internally and externally)
    • support for probability distributions (e.g. risk weighted optimization). Development of algorithms for fulfilling such use case is currently ongoing.
  • Development of two new optimization algorithms:
    • Conjugate Gradient optimization
    • Simulating Annealing Continuous optimization
• Surrogate Models:
  • Deep Learning:
    • Addition of the support for Deep Learning (Neural Networks) with the deployment of the interface for TensorFlow/Keras
  • ARMA/VARMA:
    • Adds the option for the ARMA to produce higher-dimensional data on request, representing the ARMA re-evaluated for consecutive years, potentially also applying a growth factor. These realizations must be placed into a DataSet to be used effectively.
    • The Clustering features accessible to the ARMA were extended to include hierarchal lists of cluster-able features that can be enabled a la carte by the user. The families of features are currently global, Fourier, ARMA, and peak. Additionally, ClusteredROM objects can be swapped from "full" to "truncated" mode (or vice versa) both at training time and at unpickling time.
    • Improvement of the unbiased sampling for the synthetic time series generation index (e.g. time) clustering
• Post-Processors:
  • Addition of the MCS Post-Processor:
    • Post-Processor aimed to import Minimal Cut-Sets (PRA) generated by an external PRA code (e.g. SAPHIRE) into RAVEN
  • Addition of Pareto Frontier algorithm:
    • Aimed to identify the points lying on the Pareto Frontier in a cost-value space
  • Limit Surface improvements:
    • Option to compute the bounding error of the limit surface (maximum error on the computed probability of failure)
  • Basic-Statistics improvements:
    • Fixed the discrepancy on quantile (percentile) calculation if no statistical weights or uniform weights were used (the should have been the same).
• Plotting:
  • Addressed the issue of multiple 3D plots on the same figure. Now all the plots are shown
• RAVEN Plug-ins:
  • Standardized the creation of new RAVEN plugins:
    • Regression testing
    • Config files (e.g. Plugin dependencies)
    • Documentation and NQ1 requirements
  • In addition to the already deployed plug-ins (CashFlow and PRA plugin), new supported plug-ins have been deployed:
    • LOGOS: integration of classical PRA analysis (SAPHIRE) with Dynamic-PRA (RAVEN) for financial optimization of maintenance. LOGOS uses the RAVEN CashFlow plugin for its financial analyses.
    • SR2ML: aimed to provide sets of reliability models designed to be interfaced. These models can be employed to perform both static and dynamic system risk analysis and determine risk importance of specific elements of the considered system.
    • SWAW: aimed to provide sets of advanced workflows and methods to be applied to plant health and asset management. These methods focus on maintenance and replacement optimization and system reliability/unavailability.
• Features' issues' tracking IDs:
  • [TASK] Limit Surface Integral Bounding Error #1216
  • [TASK] Adding Simulated Annealing #1214
  • [TASK] Add workshop material for ARMA #1208
  • [TASK] Conda should work when default channel not available #1207
  • [TASK] ROM Clustering improvement  #1199
  • [TASK] Pareto Frontier PP #1195
  • [TASK] Restructuring of the Optimizzation system #1193
  • [TASK] StringsThatMean simplification #1191
  • [TASK] SRAW Plugin #1190
  • [TASK] Add LOGOS Plugin #1180
  • [TASK] Extend Dependencies configure file to accept skip-check #1178
  • [TASK] Add SR2ML as RAVEN plugin #1172
  • [TASK] The Sampler Restart default tollerance should be modified #1169
  • [TASK] Creation of a MCS Post-Processor #1167
  • [TASK] Rook should error out if there are missing required parameters in testers #1154
  • [TASK] Adaptive sampler as base class #1149
  • [TASK] MOOSE interface to perturb external (not getpot) input files #1140
  • [TASK] New adaptive sampler with statistical convergence #1138
  • [TASK] Replace Parallel Python with RAY #1137
  • [TASK] Relocate CashFlow #1131
  • [TASK] RrR: conversion script should be relative to working dir #1127
  • [TASK] [plugins] add Testers with plugins #1121
  • [TASK] Reduce warnings verbosity #1117
  • [UNDER-DISCUSSION] The Plugin install script can only be executed from the RAVEN folder #1112
  • [TASK] Add an error if plugins not installed and tests are requested #1111
  • figuring out why coverage might not have GUI coverage #1104
  • [TASK] Basic samplers should handle no-variable sampling #1094
  • Change current SPSA gradient strategy #1090
  • [TASK] Plain-text Library Dependencies #1084
  • Reading input from paramInput for optimizer #1072
  • contribute and Issue template pages' update #1067
  • CONJUGATE Gradient  #1061
  • The RAVEN start time is too slow #1047
  • Add ability for rook to have a configuration file #1034
  • Make rook harder to kill #1019
  • Neutrino Interface #1018
  • ERROR IN CASE OF UNSYNC HISTORIES IN TIME DEP BASIC STATISTICS! #1011
  • UserGuide_ForwardSampler #1009
  • Need to support Pyside2 #1007
  • clean_raven should be able to remove all *pyc files  #1004
  • Rook requires psutils #1000
  • [rook] relative error in XMLdiff #993
  • Multi-year ARMA #991
  • Colors optional in ROOK #985
  • Relevant Issues and Features - From RAVEN v1.0 to v1.1 #984
  • Template comma list failure #979
  • Update Manuals INL Code reports #974
  • Update Manuals INL Code reports #974
  • post processor clean up #970
  • No submodule of pluginName in gitbub when I install raven on Windows. #955
  • Inconsistencies in User Guide Data Mining Example #952
  • Implementation and Development of Optimization Schemes for RIAM  #918
  • "Function" variables in Optimizer #867
  • Improvements of DataObject #839
  • Disable PostProcessor to accept the same input DataObjects as the output DataObjects #805
  • Optimizers' tollerances < in place of <= #791
  • check_py_coverage.sh must be parallelized #764
  • establish_conda_env thread safety #754
  • ARMA producing nans using the TRAIN_ARMA files in the Hybrid project repository from the September2017 milestone report. #752
  • Manual entry for HS2PS postprocessor  #737
  • Documentation for Interfaced Postprocessors is too minimal to be useful  #728
  • Optimizer paths not identical #624
  • Conda after version 4.4 changes behavior #616
  • ARMA reseeding not as expected #597
  • Implement Deep Neuron Network based on Keras interface of TensorFlow #574
  • Prefixes in sampler with runnig external code #383
  • Improved optimizer persistance #332
  • Check that the InputData is as specified. #311
  • Optimizer: TargetEvaluation outputs available in preconditioner #301
  • dry runs before optimization begins #299
  • Misleading error message when ROM subtype is not of allowable types. #261
  • Unique optimizer denoising options #244
  • Limit Surface PostProcessor does not report when the limit surface is empty (not found). #172
  • Preventing User Error in the Limit Surface Search #171
  • Manual Improvement / PostProcessors #169
  • Metadata XML writing is intensive #52
  • Adaptive and default parameters in Optimizer #39

---

For Change Control Board: Issue Review

This review should occur before any development is performed as a response to this issue.

• 1. Is it tagged with a type: defect or task?
• 2. Is it tagged with a priority: critical, normal or minor?
• 3. If it will impact requirements or requirements tests, is it tagged with requirements?
• 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
• 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

---

For Change Control Board: Issue Closure

This review should occur when the issue is imminently going to be closed.

• 1. If the issue is a defect, is the defect fixed?
• 2. If the issue is a defect, is the defect tested for in the regression test system? (If not explain why not.)
• 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
• 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
• 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

[alfoa]

RAVEN V.2.0 has been released.

This issue can be closed, and another one containing the reference of tasks/defects from RAVEN v.2.0 to RAVEN v.2.1 has been opened:

#1225