Required actions are written in bold, other actions are recommended. Not all actions might apply to every research project.
- Draw conclusions while considering dataset and experiment limitations
- Document task adequacy, representativeness and pre-processing
- Split the data such as to avoid spurious correlations
- Perform exploratory data analyses to ensure quality and correctness
- Publish the dataset accessibly & indicate changes
- Be aware of the score differences required to claim significance
- Use a code repository (e.g. git) with proper documentation including licensing to distribu tecode for replicability
- Report all details about hyperparameter search and model training
- Publish model predictions and evaluation scripts
- In the paper, specify the parameters used for replicability as well as the hardware requirements
- Use efficient hyperparameter optimization methods
- Use of model cards
- Report mean & standard deviation over multiple runs
- Perform significance testing or Bayesian analysis and motivate your choice of method
- Carefully reflect on the amount of evidence regarding your initial hypotheses
- Guarantee the replicability of experiments
- Discuss the potential ethical & social impact
- Avoid citing pre-prints (if applicable)
- Prioritize computational efficiency
- Include an Ethics Statement