Français
Scaling of scoring rules
Virtualconference
Authors : Wallin, Jonas (Author of the conference)
CIRM (Publisher )
60G25 - Prediction theory, See also {62M20}
62C25 - Compound decision problems
62M30 - Statistics of spatial processes
Additional resources :
https://www.cirm-math.fr/RepOrga/2146/Slides/Wallin.pdf
CIRM (Publisher )
Loading the player...
|
Abstract :
Averages of proper scoring rules are often used to rank probabilistic forecasts. In many cases, the individual observations and their predictive distributions in these averages have variable scale (variance). I will show that some of the most popular proper scoring rules, such as the continuous ranked probability score (CRPS), up-weight observations with large uncertainty which can lead to unintuitive rankings. We have developed a new scoring rule, scaled CRPS (SCRPS), this new proper scoring rule is locally scale invariant and therefore works in the case of varying uncertainty.
I will demonstrate this how this affects model selection through parameter estimation in spatial statitics.
Keywords : Proper soring rules; model selection; prediction
MSC Codes : I will demonstrate this how this affects model selection through parameter estimation in spatial statitics.
Keywords : Proper soring rules; model selection; prediction
60G25 - Prediction theory, See also {62M20}
62C25 - Compound decision problems
62M30 - Statistics of spatial processes
Additional resources :
https://www.cirm-math.fr/RepOrga/2146/Slides/Wallin.pdf
|
Information on the Video
Film maker : Hennenfent, GuillaumeLanguage : English Available date : 15/06/2020 Conference Date : 02/06/2020 Subseries : Research talks arXiv category : Machine Learning ; Statistics Theory Mathematical Area(s) : Probability & Statistics Format : MP4 (.mp4) - HD Video Time : 00:35:09 Targeted Audience : Researchers Download : https://videos.cirm-math.fr/2020-06-02_Wallin.mp4 
|
Information on the Event
Event Title : Mathematical Methods of Modern Statistics 2 / Méthodes mathématiques en statistiques modernes 2Event Organizers : Bogdan, Malgorzata ; Graczyk, Piotr ; Panloup, Fabien ; Proïa, Frédéric ; Roquain, Etienne Dates : 15/06/2020 - 19/06/2020 Event Year : 2020 Event URL : https://www.cirm-math.com/cirm-virtual-e...
Citation Data
DOI : 10.24350/CIRM.V.19644103Cite this video as: Wallin, Jonas (2020). Scaling of scoring rules. CIRM. Audiovisual resource. doi:10.24350/CIRM.V.19644103 URI : http://dx.doi.org/10.24350/CIRM.V.19644103 |
See Also
- [Virtualconference] Experimenting in equilibrium / Author of the conference Wager, Stefan.
- [Virtualconference] Structure learning for CTBN's / Author of the conference Miasojedow, Błażej.
- [Virtualconference] The price of competition: effect size heterogeneity matters in high dimensions! / Author of the conference Wang, Hua.
- [Virtualconference] Hierarchical bayes modeling for large-scale inference / Author of the conference Yekutieli, Daniel.
- [Virtualconference] Change: detection, estimation, segmentation / Author of the conference Siegmund, David.
- [Virtualconference] High-dimensional, multiscale online changepoint detection / Author of the conference Samworth, Richard.
- [Virtualconference] The smoothed multivariate square-root Lasso: an optimization lens on concomitant estimation / Author of the conference Salmon, Joseph.
- [Virtualconference] Knockoff genotypes: value in counterfeit / Author of the conference Sabatti, Chiara.
- [Virtualconference] Optimal and maximin procedures for multiple testing problems / Author of the conference Rosset, Saharon.
- [Virtualconference] Sparse multiple testing: can one estimate the null distribution ? / Author of the conference Roquain, Etienne.
- [Virtualconference] Bayesian spatial adaptation / Author of the conference Rockova, Veronika.
- [Virtualconference] Universal inference using the split likelihood ratio test / Author of the conference Ramdas, Aaditya K..
- [Virtualconference] How to estimate a density on a spider web ? / Author of the conference Picard, Dominique.
- [Virtualconference] Post hoc bounds on false positives using reference families / Author of the conference Neuvial, Pierre.
- [Virtualconference] Quasi logistic distributions and Gaussian scale mixing / Author of the conference Letac, Gerard.
- [Virtualconference] Shrinkage estimation of mean for complex multivariate normal distribution with unknown covariance when p > n / Author of the conference Konno, Yoshihiko.
- [Virtualconference] Treatment effect estimation with missing attributes / Author of the conference Josse, Julie.
- [Virtualconference] Floodgate: inference for model-free variable importance / Author of the conference Janson, Lucas.
- [Virtualconference] On Cholesky structures on real symmetric matrices and their applications / Author of the conference Ishi, Hideyuki.
- [Virtualconference] Optimal control of false discovery criteria in the general two-group model / Author of the conference Heller, Ruth.
- [Virtualconference] Isotonic Distributional Regression (IDR) - leveraging monotonicity, uniquely so! / Author of the conference Gneiting, Tilmann.
- [Virtualconference] De-biasing arbitrary convex regularizers and asymptotic normality / Author of the conference Bellec, Pierre C..
- [Virtualconference] Consistent model selection criteria and goodness-of-fit test for common time series models / Author of the conference Bardet, Jean-Marc.
- [Virtualconference] High-dimensional classification by sparse logistic regression / Author of the conference Abramovich, Felix.
Bibliography
- BOLIN, David et WALLIN, Jonas. Scale invariant proper scoring rules Scale dependence: Why the average CRPS often is inappropriate for ranking probabilistic forecasts. arXiv preprint arXiv:1912.05642, 2019. - https://arxiv.org/abs/1912.05642
