참고문헌
제1고지: 미분 자동 계산
[1] Todd Young, Martin J. Mohlenkamp, “Introduction to Numerical Methods and Matlab Programming for Engineers — Lecture 27: Numerical Differentiation,” http://www.ohiouniversityfaculty.com/youngt/IntNumMeth/lecture27.pdf (2019. 11)
[2] Wengert, Robert Edwin, “A simple automatic derivative evaluation program,” Communications of the ACM 7.8 (1964): 463-464.
[3] William W. Cohen, “Automatic Reverse-Mode Differentiation: Lecture Notes,” http://www.cs.cmu.edu/~wcohen/10-605/notes/autodiff.pdf (2019. 11)
[4] Paszke, Adam, et al, “Automatic differentiation in pytorch,” (2017).
[5] Fei-Fei Li, et al, “CS231n: Convolutional Neural Networks for Visual Recognition,” http://cs231n.github.io/neural-networks-3/ (2019. 11)
[6] Baydin, Atilim Gunes, et al, “Automatic differentiation in machine learning: a survey,” Journal of machine learning research 18.153 (2018).
[7] Maclaurin, Dougal, “Modeling, inference and optimization with composable differentiable procedures,” Doctoral dissertation, Harvard University, Graduate School of Arts & Sciences (2016).
[8] 파이썬 문서, “unittest - 단위 테스트 프레임워크” https://docs.python.org/ko/3/library/unittest.html (2019. 11)
[9] Travis CI, https://travis-ci.org/ (2019. 11)
참고문헌 539
제2고지: 자연스러운 코드로
[10] Hertz, Matthew, and Emery D. Berger. “Quantifying the performance of garbage collection vs. explicit memory management,” Proceedings of the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications. 2005.
[11] PyPI, “Memory Profiler” https://pypi.org/project/memory-profiler/ (2019. 11)
[12] 파이썬 문서, “contextlib” https://docs.python.org/3/library/contextlib.html (2019. 11)
[13] 위키백과, “Test functions for optimization” https://en.wikipedia.org/wiki/Test_functions_for_optimization (2019. 11)
[14] Christopher Olah, “Neural Networks, Types, and Functional Programming” http://colah.github.io/posts/2015-09-NN-Types-FP/ (2019. 11)
[15] Yann LeCun, “Differentiable Programming” https://www.facebook.com/yann.lecun/posts/10155003011462143 (2019. 11)
[16] 파이토치 문서, “TORCHSCRIPT” https://pytorch.org/docs/stable/jit.html (2019. 11)
[17] 텐서플로용 스위프트, https://www.tensorflow.org/swift (2019. 11)
제3고지: 고차 미분 계산
[18] Graphviz - Graph Visualization Software, http://www.graphviz.org/ (2019. 11)
[19] 위키백과, “Rosenbrock function” https://en.wikipedia.org/wiki/Rosenbrock_function (2019. 11)
[20] 파이토치 문서, “torch.optim.LBFGS” https://pytorch.org/docs/stable/optim.html?highlight=l%20bfgs#torch.optim.LBFGS (2019. 11)
[21] Gulrajani, Ishaan, et al, “Improved training of wasserstein gans,” Advances in neural information processing systems, 2017.
[22] Finn, Chelsea, Pieter Abbeel, and Sergey Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” Proceedings of the 34th International Conference on Machine Learning—Volume 70. JMLR.org, 2017.
[23] Schulman, John, et al, “Trust region policy optimization,” International conference on machine learning, 2015.
제4고지: 신경망 만들기
[24] Seiya Tokui, “Aggressive Buffer Release” https://docs.google.com/document/d/1CxNS2xg2bLT9LoUe6rPSMuluqt8Gbkz156LTPJPT3BE/ (2019. 11)
[25] LeCun, Yann A., et al, “Efficient backprop.” Neural networks: Tricks of the trade, Springer, Berlin, Heidelberg, 2012. 9-48.
[26] Pascanu, Razvan, Tomas Mikolov, and Yoshua Bengio, “On the difficulty of training recurrent neural networks,” International conference on machine learning, 2013.
[27] Duchi, John, Elad Hazan, and Yoram Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” Journal of Machine Learning Research 12.Jul (2011): 2121-2159.
참고문헌 541
[28] Zeiler, Matthew D, “ADADELTA: an adaptive learning rate method,” arXiv preprint arXiv:1212.5701 (2012).
[29] Loshchilov, Ilya, and Frank Hutter, “Fixing weight decay regularization in adam,” arXiv preprint arXiv:1711.05101 (2017).
[30] 체이너 MNIST 예시, https://github.com/chainer/chainer/blob/v6/examples/mnist/train_mnist_custom_loop.py (2019. 11)
[31] 파이토치 MNIST 예시, https://github.com/pytorch/examples/blob/master/mnist/main.py (2019. 11)
[32] 체이너 문서, “Link and Chains” https://docs.chainer.org/en/stable/reference/links.html (2019. 11)
[33] 텐서플로 API 문서, “Module: tf.keras.optimizers” https://www.tensorflow.org/api_docs/python/tf/keras/optimizers (2019. 11)
제5고지: DeZero의 도전
[34] Srivastava, Nitish, et al, “Dropout: a simple way to prevent neural networks from overfitting,” The journal of machine learning research 15.1 (2014): 1929-1958.
[35] Ioffe, Sergey, and Christian Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167 (2015).
[36] Simonyan, Karen, and Andrew Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 (2014).
[37] He, Kaiming, et al, “Deep residual learning for image recognition,” Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
[38] Iandola, Forrest N., et al, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size,” arXiv preprint arXiv:1602.07360 (2016).
[39] SPHINX, https://www.sphinx-doc.org/en/master/ (2019. 11)
[40] ONNX, https://onnx.ai/ (2019. 11)
[41] Goodfellow, Ian, et al, “Generative adversarial nets,” Advances in neural information processing systems, 2014.
[42] Kingma, Diederik P., and Max Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114 (2013).
[43] Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge, “Image style transfer using convolutional neural networks,” Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
참고문헌 543