Stat8056 Home Page
STAT8056/PUB8475: Advanced Topics on Machine Learning and Data Science
Spring 2024(1/17/24-04/29/24)
Instructors:
STAT8056: Shen Xiaotong (Stat, xshen@stat.umn.edu)
xshen
PUB8475: Pan Wei (Biostat, panxx014@umn.edu)
panxx014
Course Website: http://www.stat.umn.edu/~xshen/stat8056.htm
Lecture Meeting: 09:45am-11:00am, MW, Health Sciences Edu Ctr 2-132
Office Hour: 11:00am-12:00pm, MW, or by appointment, Ford 391
Syllabus
No textbook. Slides and published research papers will be shared.
Topics to be covered:
-
Optimization
-
Data exploration and data science
-
Optimization for machine learning
-
High-dimensional analysis: prediction and inference
-
Deep neural network learning: basics (FNN, CNN, RNN/LSTM); advanced topics (Transformers, Diffusion models, etc)
-
Recommender systems: personalized prediction
-
Undirected and directed graphical models
-
Unstructured data and text mining: Numerical embedding and language models
-
Causal Machine Learning
Data Exploration, Optimization, and High-dimensional learning
1. Download
Breiman, L. and Cutler, A. (1993). A Deterministic algorithm for
global optimization. Mathematical Programming, 58, 179--1993.
2.
Download
Horst, R., and Thoai, N. V. (1999). DC programming: Overview. Journal
of Optimization Theory and Application. 103, 1-43.
3. Download
Li, C., Shen, X., and Pan, W. (2021). Inference for a large directed graphical model with
interventions. Preprint.
4. Download
Chen, Y., Ye, Y., and Wang, M. (2018). Approximation hardness for a class of sparse optimization
problems. Journal of Machine Learning Research, 20, 1-27.
5. Download
Mazumder, R. and Hastie, T. (2012). The graphical lasso: New insights and
alternatives. Electronic Journal of Statistics, 6, 2125-2149.
Deep learning, language models, and generaative models
1. Download
Sohl-Dickstein, J. and Weiss, E., Maheswaranathan, N., and Ganguli, S. (2015).
Deep unsupervised learning using nonequilibrium thermodynamics.
International Conference on Machine Learning. 2256--2265.
2. Download
Dinh, L. and Sohl-Dickstein, J. and Bengio, S. (2016).
Density estimation using real nvp. arXiv preprint arXiv:1605.08803.
3. Download
Dai, B., Shen, X., and Pan, W. (2022). Significance tests of feature
relevance for a black-box learner. IEEE Transactions on Neural Networks
and Learning Systems. 1-14.
Python library dnn-inference
https://dnn-inference.readthedocs.io/en/latest/
4. Download
Dai, B., Shen, X., and Wang, J. (2022). Embedding learning. Journal of American Statistical Association. 116,
1--36.
5. Download
Kotelnikov, A., Baranchuk, D., Rubachev, I., and Babenko, A. (2023).
TabDDPM: Modelling tabular data with diffusion models. International Conference on Machine Learning.
6. Download
Zheng, S and Charoenphakdee, N. (2022). Diffusion models for missing value imputation in tabular data.
arXiv preprint arXiv:2210.17128.
7. Download
Yang, Y., and Shen, X. (2023). Boosting summarization with normalizing flows and optimized
training. The proceeding of the 2023 Conference on Empirical Methods in Natural Language Processing. 2727-2751.
Week 8 (March 4-8): Spring break
Recommender systems and graphical models
1. Download
Guo, J., Levina, E., Michailidis, G. and Zhu, J. (2010). Joint estimation of
multiple graphical models. Biometrika, 98, 1-15.
2. Download
Zhu, Y., Shen, X. and Ye, C. (2016).
Personalized prediction and sparsity pursuit in latent factor models.
Journal of American Statistical Association, 111, 241-252.
3. Download
Bi, X., Qu, A., Wang, J., and Shen, X. (2017). A group-specific recommender system.
Journal of American Statistical Association, 112, 1344-1353.
4. Download
Dai, B., Wang, J, Shen, X., and Qu, P. (2020).
Smooth neighborhood recommender systems. The Journal of Machine Learning Research. 20(16),1-24.
5. Download
Mazumder, R., Hastie, T, and Tibshirani, R. (2010). Spectral regularization
algorithms for learning large incomplete matrices. Journal of Machine Learning
Research, 11, 2287-2322.
6. Download
Zhu, Y., Shen, X. and Pan, W. (2014). Structural pursuit
over multiple undirected graphs. Journal of American Statistical
Association, 109, 1683-1696.
7. Download
Wu, C., Xu, G., Shen, X., and Pan, W. (2020). A regularization-based adaptive test for high-dimensional
generalized linear models. Journal of Machine Learning Research}. 21(128), 1-67.
8.
Download Yuan, Y., Shen, X., Pan, W., and Wang, Z. (2019). Reconstruction of a directed acyclic
Gaussian graph. Biometrika. 106, 109-125. R-package: TLPDAG Download.
9. Download
Li, C., Shen, X., and Pan, W. (2020). Likelihood inference
for a large causal network. Journal of American Statistical Association. 113, 1--16.
R-package: clrdag Download
HWK due on
04/15/24 in class
Kaggle Data link
Final project report due on
04/29/24 in class