WiSe 23/24: Data analysis and data management
Laurenz Rettig
Kommentar
Research data in physics are growing more and more rich and complex, and novel tools such as AI-based analysis approaches promise huge advances in knowledge gain. Simultaneously, these developments pose substantial challenges to students and researchers, and require developing novel data analysis approaches, and data management solutions.
In this lecture, I will discuss various data analysis approaches found in physics, e.g. statistical analysis, nonlinear and multivariate fitting analysis, global analysis, generation and management of large datasets, annotation and F.A.I.R. readiness of research data, etc. The lecture will be accompanied by an exercise class with a hands-on programming course in Python, where examples from the lecture will be further deepened.
Topics include:
- Random processes and description as stochastic events, statistical description as probability distribution functions, central moments of distribution functions and their properties
- Statistical hypothesis testing, confidence intervals, the chi2 and student-t distribution
- Definition of a measurement, measurement errors, precision and accuracy, measurement models and comparison with predictions
- Types of measurement errors, instrument response, resolution, influence and types of noise in measurements, detection limit, quantitation limit
- Calculation and progression of measurement errors, correlated errors, covariance, counting experiments, shot noise
- Data management: formal requirements for documentation of experiments and annotation of data, electronic laboratory notebooks, meta data management, FAIR data management
- Graphic visualization of research data, linear regression, interpretation and statistical analysis of results
- Nonlinear regression of data, statistical discussion of results, bootstrap/Monte Carlo approach, chi2 maps
- Correlated uncertainties, multidimensional fits, multivariate nonlinear regression
- Handling and analysis of highdimensional research data, structured and integrated data acquisition, single-event data streams, data binning, data reduction
- Manipulation of large data sets: strategies, lazy computing frameworks etc.
- Nonprobabilistic analysis approaches: singular value decomposition, Fourier analysis, etc.
- Outlook: machine learning and AIbased data analysis
16 Termine
Regelmäßige Termine der Lehrveranstaltung