Interpretable Approaches to Opening Up Black-Box Models