New representativity calculations post processor

doc/user_manual/PostProcessors/Validation.tex

@@ -9,12 +9,12 @@ \subsubsection{Validation PostProcessors}
 \begin{itemize}
   \item  \textbf{Probabilistic}, using probabilistic method for validation, can be used for both static and time-dependent problems.
   \item  \textbf{PPDSS}, using dynamic system scaling method for validation, can only be used for time-dependent problems.
-  % \item  \textbf{Representativity}
+  \item  \textbf{Representativity}, using represntativity (bias) factor for validation, currently, can be used for static data.
   \item  \textbf{PCM}, using Physics-guided Coverage Mapping method for validation, can only be used for static problems.
 \end{itemize}
 %
 
-The choices of the available metrics and acceptable data objects are specified in table \ref{tab:ValidationAlgorithms}.
+The choices of the available metrics and acceptable data objects are specified in table~\ref{tab:ValidationAlgorithms}.
 
 \begin{table}[]
 \caption{Validation Algorithms and respective available metrics and DataObjects}
@@ -23,6 +23,7 @@ \subsubsection{Validation PostProcessors}
 \hline
 \textbf{Validation Algorithm} & \textbf{DataObject}                                            & \textbf{Available Metrics}                                                   \\ \hline
 Probabilistic                 & \begin{tabular}[c]{@{}c@{}}PointSet \\ HistorySet\end{tabular} & \begin{tabular}[c]{@{}c@{}}CDFAreaDifference\\ \\ PDFCommonArea\end{tabular} \\ \hline
+Representativity              & \begin{tabular}[c]{@{}c@{}}PointSet \\ HistorySet\end{tabular} & \begin{tabular}[c]{@{}c@{}}\end{tabular} \\ \hline
 PPDSS                         & HistorySet                                                     & DSS                                                           \\ \hline
 PCM                           & PointSet                                                       & (not applicable)                                              \\ \hline
 \end{tabular}
@@ -105,7 +106,7 @@ \subsubsection{Validation PostProcessors}
       \item \xmlAttr{type}, \xmlDesc{required string attribute}, the sub-type of this Metric (e.g., SKL, Minkowski)
     \end{itemize}
     \nb The choice of the available metric is \xmlString{DSS}, please
-    refer to \ref{sec:Metrics} for detailed descriptions about this metric.
+    refer to~\ref{sec:Metrics} for detailed descriptions about this metric.
     \item \xmlNode{pivotParameterFeature}, \xmlDesc{string, required field}, specifies the pivotParameter for a feature <HistorySet>. The feature pivot parameter is the shared index of the output variables in the data object.
     \item \xmlNode{pivotParameterTarget}, \xmlDesc{string, required field}, specifies the pivotParameter for a target <HistorySet>. The target pivot parameter is the shared index of the output variables in the data object.
     \item \xmlNode{separateFeatureData}, \xmlDesc{string, optional field}, specifies the custom feature interval to apply DSS postprocessing. The string should contain three parts; start time, `|', and end time all in one. For example, 0.0|0.5.
@@ -247,7 +248,7 @@ \subsubsection{Validation PostProcessors}
     number of measurements should be equal to the number of features and in the same order as the features listed in \xmlNode{Features}.
 \end{itemize}
 
-The output of PCM is comma separated list of strings in the format of ``pri\textunderscore post\textunderscore stdReduct\textunderscore [targetName]'', 
+The output of PCM is comma separated list of strings in the format of ``pri\textunderscore post\textunderscore stdReduct\textunderscore [targetName]'',
 where [targetName] is the $VariableName$ specified in DataObject of \xmlNode{Targets}.
 
 
@@ -267,3 +268,84 @@ \subsubsection{Validation PostProcessors}
   ...
 <Simulation>
 \end{lstlisting}
+
+
+\paragraph{Representativity}
+The \textbf{Representativity} post-processor is one of three \textbf{Validation} post-processors, in fact there is a
+post-processor interface that acts as a gate for applying these validation algorithms
+(i.e., representativity, Physics-guided Convergence Mapping (PCM), and Dynamic System Scaling (DSS)).
+The post-processor is in charge of deploying a common infrastructure for the user of  \textbf{Validation} problems.
+The representativity theory was first founded in the Neutronics community~\cite{Gandini, palmiotti1, palmiotti2}, then lately, was transformed to the thermal hydraulics~\cite{Epiney1, Epiney2}.
+
+%
+\ppType{Representativity}{Representativity}
+%
+
+\begin{itemize}
+  \item \xmlNode{Features}, \xmlDesc{comma separated string, required field}, specifies the names of  the features, which can be the measuables/observables of the mock model. Reader should be warned that this nomenclature is different than the machine learning nomenclature.
+
+  \item \xmlNode{Targets}, \xmlDesc{comma separated string, required field}, contains a comma separated list of
+     targets. These are the Figures of merit (FOMs) in the target model against which the mock model is being validated.
+
+    \item \xmlNode{featureParameters}, \xmlDesc{comma separated string, required field}, specifies the names of  the parameters/inputs to the mock/prototype model.
+
+    \item \xmlNode{targetParameters}, \xmlDesc{comma separated string, required field}, specifies the names of the parameters/inputs to the target model.
+
+        \item \xmlNode{pivotParameter},  \xmlDesc{string, optional field}, ID of the temporal variable of the mock model. Default is ``time''.
+        \nb Used just in case the  \xmlNode{pivotValue}-based operation  is requested (i.e., time dependent validation).
+        \item \xmlNode{targetPivotParameter}, \xmlDesc{string, optional field}, ID of the temporal variable in the target model. Default is ``time''.
+        \nb Used just in case the  \xmlNode{pivotValue}-based operation  is requested (i.e., time dependent validation).
+Indeed, Both \textbf{PointSet} and \textbf{HistorySet} can be accepted by this post-processor.
+If the name of given variable to be compared is unique, it can be used directly, otherwise the variable can be specified
+with $DataObjectName|InputOrOutput|VariableName$ nomenclature.
+
+\textbf{Example:}
+\begin{lstlisting}[style=XML,morekeywords={subType}]
+<Simulation>
+...
+  <Steps>
+  <!--Multirun the prototype model-->
+  <MultiRun name="mcRunExp" re-seeding="20021986">
+    <Input class="DataObjects" type="PointSet">inputPlaceHolder2</Input>
+    <Model class="Models" type="ExternalModel">linModel</Model>
+    <Sampler class="Samplers" type="MonteCarlo">ExperimentMCSampler</Sampler>
+    <Output class="DataObjects" type="PointSet">outputDataMC1</Output>
+  </MultiRun>
+  <!--Multirun the target model-->
+  <MultiRun name="mcRunTar" re-seeding="68912002">
+    <Input class="DataObjects" type="PointSet">inputPlaceHolder2</Input>
+    <Model class="Models" type="ExternalModel">tarModel</Model>
+    <Sampler class="Samplers" type="MonteCarlo">TargetMCSampler</Sampler>
+    <Output class="DataObjects" type="PointSet">outputDataMC2</Output>
+  </MultiRun>
+  <!--Create the Representativity PostProcessor-->
+  <PostProcess name="PP1">
+    <Input class="DataObjects" type="PointSet">outputDataMC1</Input>
+    <Input class="DataObjects" type="PointSet">outputDataMC2</Input>
+    <Model class="Models" type="PostProcessor">pp1</Model>
+    <Output class="DataObjects" type="PointSet">pp1_metric</Output>
+    <Output class="OutStreams" type="Print">pp1_metric_dump</Output>
+  </PostProcess>
+  </Steps>
+...
+  <Models>
+    <ExternalModel ModuleToLoad="../../../AnalyticModels/expLinModel.py" name="linModel" subType="">
+      <inputs>p1, p2, e1, e2, e3, bE</inputs>
+      <outputs>F1, F2, F3</outputs>
+    </ExternalModel>
+    <ExternalModel ModuleToLoad="../../../AnalyticModels/tarLinModel.py" name="tarModel" subType="">
+      <inputs>p1, p2, o1, o2, o3, bT</inputs>
+      <outputs>FOM1, FOM2, FOM3</outputs>
+    </ExternalModel>
+    <PostProcessor name="pp1" subType="Representativity">
+      <Features>outputDataMC1|F1, outputDataMC1|F2, outputDataMC1|F3</Features>
+      <Targets>outputDataMC2|FOM1, outputDataMC2|FOM2, outputDataMC2|FOM3</Targets><!---->
+      <featureParameters>outputDataMC1|p1,outputDataMC1|p2</featureParameters>
+      <targetParameters>outputDataMC2|p1,outputDataMC2|p2</targetParameters>
+      <pivotParameter>outputDataMC1|time</pivotParameter>
+      <targetPivotParameter>outputDataMC2|time</targetPivotParameter>
+    </PostProcessor>
+  </Models>
+...
+<Simulation>
+\end{lstlisting}

doc/user_manual/raven_user_manual.bib

@@ -112,3 +112,49 @@ @TechReport{RAVENtheoryManual
   year = {2016},
   key = {INL/EXT-16-38178}
 }
+
+@book{Gandini,
+	title={Uncertainty analysis and experimental data transposition methods based on perturbation theory},
+	author={Gandini, A},
+	journal={Uncertainty Analysis},
+	pages={217--258},
+	year={1988},
+	publisher={CRC Press, Boca Raton, Fla, USA}
+}
+
+@article{palmiotti1,
+	title={A global approach to the physics validation of simulation codes for future nuclear systems},
+	author={Palmiotti, Giuseppe and Salvatores, Massimo and Aliberti, Gerardo and Hiruta, Hikarui and McKnight, R and Oblozinsky, P and Yang, WS},
+	journal={Annals of Nuclear Energy},
+	volume={36},
+	number={3},
+	pages={355--361},
+	year={2009},
+	publisher={Elsevier}
+}
+
+@article{palmiotti2,
+	title={The role of experiments and of sensitivity analysis in simulation validation strategies with emphasis on reactor physics},
+	author={Palmiotti, Giuseppe and Salvatores, Massimo},
+	journal={Annals of Nuclear Energy},
+	volume={52},
+	pages={10--21},
+	year={2013},
+	publisher={Elsevier}
+}
+
+@article{Epiney1,
+	title={A Systematic Approach to Inform Experiment Design Through Modern Modeling and Simulation Methods},
+	author={Epiney, A and Rabiti, C and Davis, C},
+	year={2019}
+}
+
+@inproceedings{Epiney2,
+	title={Representativity Analysis Applied to TREAT Water Loop LOCA Experiment Design},
+	author={Epiney, Aaron S and Woolstenhulme, Nicolas},
+	booktitle={International Conference on Nuclear Engineering},
+	volume={83785},
+	pages={V003T13A055},
+	year={2020},
+	organization={American Society of Mechanical Engineers}
+}

ravenframework/Models/PostProcessors/BasicStatistics.py

@@ -27,14 +27,14 @@
 #External Modules End-----------------------------------------------------------
 
 #Internal Modules---------------------------------------------------------------
-from .PostProcessorInterface import PostProcessorInterface
+from .PostProcessorReadyInterface import PostProcessorReadyInterface
 from ...utils import utils
 from ...utils import InputData, InputTypes
 from ...utils import mathUtils
 from ... import Files
 #Internal Modules End-----------------------------------------------------------
 
-class BasicStatistics(PostProcessorInterface):
+class BasicStatistics(PostProcessorReadyInterface):
   """
     BasicStatistics filter class. It computes all the most popular statistics
   """
@@ -164,92 +164,47 @@ def __init__(self):
     self.sampleSize     = None # number of sample size
     self.calculations   = {}
     self.validDataType  = ['PointSet', 'HistorySet', 'DataSet'] # The list of accepted types of DataObject
+    self.inputDataObjectName = None # name for input data object
+    self.setInputDataType('xrDataset')
 
-  def inputToInternal(self, currentInp):
+  def inputToInternal(self, inputIn):
     """
-      Method to convert an input object into the internal format that is
+      Method to select corresponding data from Data Objects and normalize the ProbabilityWeight of corresponding data
       understandable by this pp.
-      @ In, currentInp, object, an object that needs to be converted
+      @ In, inputIn, dict, a dictionary that contains the input Data Object information
       @ Out, (inputDataset, pbWeights), tuple, the dataset of inputs and the corresponding variable probability weight
     """
-    # The BasicStatistics postprocessor only accept DataObjects
-    self.dynamic = False
-    currentInput = currentInp [-1] if type(currentInp) == list else currentInp
-    if len(currentInput) == 0:
-      self.raiseAnError(IOError, "In post-processor " +self.name+" the input "+currentInput.name+" is empty.")
-
+    inpVars, outVars, dataSet = inputIn['Data'][0]
     pbWeights = None
-    if type(currentInput).__name__ == 'tuple':
-      return currentInput
-    # TODO: convert dict to dataset, I think this will be removed when DataSet is used by other entities that
-    # are currently using this Basic Statisitics PostProcessor.
-    if type(currentInput).__name__ == 'dict':
-      if 'targets' not in currentInput.keys():
-        self.raiseAnError(IOError, 'Did not find targets in the input dictionary')
-      inputDataset = xr.Dataset()
-      for var, val in currentInput['targets'].items():
-        inputDataset[var] = val
-      if 'metadata' in currentInput.keys():
-        metadata = currentInput['metadata']
-        self.pbPresent = True if 'ProbabilityWeight' in metadata else False
-        if self.pbPresent:
-          pbWeights = xr.Dataset()
-          self.realizationWeight = xr.Dataset()
-          self.realizationWeight['ProbabilityWeight'] = metadata['ProbabilityWeight']/metadata['ProbabilityWeight'].sum()
-          for target in self.parameters['targets']:
-            pbName = 'ProbabilityWeight-' + target
-            if pbName in metadata:
-              pbWeights[target] = metadata[pbName]/metadata[pbName].sum()
-            elif self.pbPresent:
-              pbWeights[target] = self.realizationWeight['ProbabilityWeight']
-        else:
-          self.raiseAWarning('BasicStatistics postprocessor did not detect ProbabilityWeights! Assuming unit weights instead...')
-      else:
-        self.raiseAWarning('BasicStatistics postprocessor did not detect ProbabilityWeights! Assuming unit weights instead...')
-      if 'RAVEN_sample_ID' not in inputDataset.sizes.keys():
-        self.raiseAWarning('BasicStatisitics postprocessor did not detect RAVEN_sample_ID! Assuming the first dimension of given data...')
-        self.sampleTag = utils.first(inputDataset.sizes.keys())
-      return inputDataset, pbWeights
-
-    if currentInput.type not in ['PointSet','HistorySet']:
-      self.raiseAnError(IOError, self, 'BasicStatistics postprocessor accepts PointSet and HistorySet only! Got ' + currentInput.type)
 
-    # extract all required data from input DataObjects, an input dataset is constructed
-    dataSet = currentInput.asDataset()
     try:
       inputDataset = dataSet[self.parameters['targets']]
     except KeyError:
       missing = [var for var in self.parameters['targets'] if var not in dataSet]
-      self.raiseAnError(KeyError, "Variables: '{}' missing from dataset '{}'!".format(", ".join(missing),currentInput.name))
-    self.sampleTag = currentInput.sampleTag
+      self.raiseAnError(KeyError, "Variables: '{}' missing from dataset '{}'!".format(", ".join(missing),self.inputDataObjectName))
+    self.sampleTag = utils.first(dataSet.dims)
 
-    if currentInput.type == 'HistorySet':
+    if self.dynamic:
       dims = inputDataset.sizes.keys()
       if self.pivotParameter is None:
-        if len(dims) > 1:
-          self.raiseAnError(IOError, self, 'Time-dependent statistics is requested (HistorySet) but no pivotParameter \
-                got inputted!')
+        self.raiseAnError(IOError, self, 'Time-dependent statistics is requested (HistorySet) but no pivotParameter \
+              got inputted!')
       elif self.pivotParameter not in dims:
         self.raiseAnError(IOError, self, 'Pivot parameter', self.pivotParameter, 'is not the associated index for \
                 requested variables', ','.join(self.parameters['targets']))
-      else:
-        self.dynamic = True
-        if not currentInput.checkIndexAlignment(indexesToCheck=self.pivotParameter):
-          self.raiseAnError(IOError, "The data provided by the data objects", currentInput.name, "is not synchronized!")
-        self.pivotValue = inputDataset[self.pivotParameter].values
-        if self.pivotValue.size != len(inputDataset.groupby(self.pivotParameter)):
-          msg = "Duplicated values were identified in pivot parameter, please use the 'HistorySetSync'" + \
-          " PostProcessor to syncronize your data before running 'BasicStatistics' PostProcessor."
-          self.raiseAnError(IOError, msg)
+      self.pivotValue = dataSet[self.pivotParameter].values
+      if self.pivotValue.size != len(dataSet.groupby(self.pivotParameter)):
+        msg = "Duplicated values were identified in pivot parameter, please use the 'HistorySetSync'" + \
+        " PostProcessor to syncronize your data before running 'BasicStatistics' PostProcessor."
+        self.raiseAnError(IOError, msg)
     # extract all required meta data
-    metaVars = currentInput.getVars('meta')
-    self.pbPresent = True if 'ProbabilityWeight' in metaVars else False
+    self.pbPresent = 'ProbabilityWeight' in dataSet
     if self.pbPresent:
       pbWeights = xr.Dataset()
       self.realizationWeight = dataSet[['ProbabilityWeight']]/dataSet[['ProbabilityWeight']].sum()
       for target in self.parameters['targets']:
         pbName = 'ProbabilityWeight-' + target
-        if pbName in metaVars:
+        if pbName in dataSet:
           pbWeights[target] = dataSet[pbName]/dataSet[pbName].sum()
         elif self.pbPresent:
           pbWeights[target] = self.realizationWeight['ProbabilityWeight']
@@ -267,6 +222,9 @@ def initialize(self, runInfo, inputs, initDict):
       @ In, initDict, dict, dictionary with initialization options
       @ Out, None
     """
+    if len(inputs)>1:
+      self.raiseAnError(IOError, 'Post-Processor', self.name, 'accepts only one DataObject')
+    self.inputDataObjectName = inputs[-1].name
     #construct a list of all the parameters that have requested values into self.allUsedParams
     self.allUsedParams = set()
     for metricName in self.scalarVals + self.vectorVals:
@@ -284,6 +242,8 @@ def initialize(self, runInfo, inputs, initDict):
     inputObj = inputs[-1] if type(inputs) == list else inputs
     if inputObj.type == 'HistorySet':
       self.dynamic = True
+      if not inputObj.checkIndexAlignment(indexesToCheck=self.pivotParameter):
+        self.raiseAnError(IOError, "The data provided by the input data object is not synchronized!")
     inputMetaKeys = []
     outputMetaKeys = []
     for metric, infos in self.toDo.items():
@@ -1544,6 +1504,21 @@ def spearmanCorrelation(self, featVars, targVars, featSamples, targSamples, pbWe
     da = xr.DataArray(spearmanMat, dims=('targets','features'), coords={'targets':targVars,'features':featVars})
     return da
 
+  def _runLegacy(self, inputIn):
+    """
+      This method executes the postprocessor action with the old data format. In this case, it computes all the requested statistical FOMs
+      @ In,  inputIn, object, object contained the data to process. (inputToInternal output)
+      @ Out, outputSet, xarray.Dataset or dictionary, dataset or dictionary containing the results
+    """
+    if type(inputIn).__name__ == 'PointSet':
+      merged = inputIn.asDataset()
+    elif 'metadata' in inputIn:
+      merged = xr.merge([inputIn['metadata'],inputIn['targets']])
+    else:
+      merged = xr.merge([inputIn['targets']])
+    newInputIn = {'Data':[[None,None,merged]]}
+    return self.run(newInputIn)
+
   def run(self, inputIn):
     """
       This method executes the postprocessor action. In this case, it computes all the requested statistical FOMs

ravenframework/Models/PostProcessors/Factory.py

@@ -37,6 +37,7 @@
 from .EconomicRatio import EconomicRatio
 from .ValidationBase import ValidationBase
 from .Validations import Probabilistic
+from .Validations import Representativity
 from .Validations import PPDSS
 from .Validations import PhysicsGuidedCoverageMapping
 from .TSACharacterizer import TSACharacterizer