Reliable Decision-Making Under Uncertainty Through The Lens of Statistics and Optimization