참고문헌

제1고지: 미분 자동 계산

[1] Todd Young, Martin J. Mohlenkamp, “Introduction to Numerical Methods and Matlab Programming for Engineers — Lecture 27: Numerical Differentiation,” http://www.ohiouniversityfaculty.com/youngt/IntNumMeth/lecture27.pdf (2019. 11)

[2] Wengert, Robert Edwin, “A simple automatic derivative evaluation program,” Communications of the ACM 7.8 (1964): 463-464.

[3] William W. Cohen, “Automatic Reverse-Mode Differentiation: Lecture Notes,” http://www.cs.cmu.edu/~wcohen/10-605/notes/autodiff.pdf (2019. 11)

[4] Paszke, Adam, et al, “Automatic differentiation in pytorch,” (2017).

[5] Fei-Fei Li, et al, “CS231n: Convolutional Neural Networks for Visual Recognition,” http://cs231n.github.io/neural-networks-3/ (2019. 11)

[6] Baydin, Atilim Gunes, et al, “Automatic differentiation in machine learning: a survey,” Journal of machine learning research 18.153 (2018).

[7] Maclaurin, Dougal, “Modeling, inference and optimization with composable differentiable procedures,” Doctoral dissertation, Harvard University, Graduate School of Arts & Sciences (2016).

[8] 파이썬 문서, “unittest - 단위 테스트 프레임워크” https://docs.python.org/ko/3/library/unittest.html (2019. 11)

[9] Travis CI, https://travis-ci.org/ (2019. 11)

참고문헌 539

제2고지: 자연스러운 코드로

[10] Hertz, Matthew, and Emery D. Berger. “Quantifying the performance of garbage collection vs. explicit memory management,” Proceedings of the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications. 2005.

[11] PyPI, “Memory Profiler” https://pypi.org/project/memory-profiler/ (2019. 11)

[12] 파이썬 문서, “contextlib” https://docs.python.org/3/library/contextlib.html (2019. 11)

[13] 위키백과, “Test functions for optimization” https://en.wikipedia.org/wiki/Test_functions_for_optimization (2019. 11)

[14] Christopher Olah, “Neural Networks, Types, and Functional Programming” http://colah.github.io/posts/2015-09-NN-Types-FP/ (2019. 11)

[15] Yann LeCun, “Differentiable Programming” https://www.facebook.com/yann.lecun/posts/10155003011462143 (2019. 11)

[16] 파이토치 문서, “TORCHSCRIPT” https://pytorch.org/docs/stable/jit.html (2019. 11)

[17] 텐서플로용 스위프트, https://www.tensorflow.org/swift (2019. 11)

제3고지: 고차 미분 계산

[18] Graphviz - Graph Visualization Software, http://www.graphviz.org/ (2019. 11)

[19] 위키백과, “Rosenbrock function” https://en.wikipedia.org/wiki/Rosenbrock_function (2019. 11)

[20] 파이토치 문서, “torch.optim.LBFGS” https://pytorch.org/docs/stable/optim.html?highlight=l%20bfgs#torch.optim.LBFGS (2019. 11)

[21] Gulrajani, Ishaan, et al, “Improved training of wasserstein gans,” Advances in neural information processing systems, 2017.

[22] Finn, Chelsea, Pieter Abbeel, and Sergey Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” Proceedings of the 34th International Conference on Machine Learning—Volume 70. JMLR.org, 2017.

[23] Schulman, John, et al, “Trust region policy optimization,” International conference on machine learning, 2015.

제4고지: 신경망 만들기

[24] Seiya Tokui, “Aggressive Buffer Release” https://docs.google.com/document/d/1CxNS2xg2bLT9LoUe6rPSMuluqt8Gbkz156LTPJPT3BE/ (2019. 11)

[25] LeCun, Yann A., et al, “Efficient backprop.” Neural networks: Tricks of the trade, Springer, Berlin, Heidelberg, 2012. 9-48.

[26] Pascanu, Razvan, Tomas Mikolov, and Yoshua Bengio, “On the difficulty of training recurrent neural networks,” International conference on machine learning, 2013.

[27] Duchi, John, Elad Hazan, and Yoram Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” Journal of Machine Learning Research 12.Jul (2011): 2121-2159.

참고문헌 541

[28] Zeiler, Matthew D, “ADADELTA: an adaptive learning rate method,” arXiv preprint arXiv:1212.5701 (2012).

[29] Loshchilov, Ilya, and Frank Hutter, “Fixing weight decay regularization in adam,” arXiv preprint arXiv:1711.05101 (2017).

[30] 체이너 MNIST 예시, https://github.com/chainer/chainer/blob/v6/examples/mnist/train_mnist_custom_loop.py (2019. 11)

[31] 파이토치 MNIST 예시, https://github.com/pytorch/examples/blob/master/mnist/main.py (2019. 11)

[32] 체이너 문서, “Link and Chains” https://docs.chainer.org/en/stable/reference/links.html (2019. 11)

[33] 텐서플로 API 문서, “Module: tf.keras.optimizers” https://www.tensorflow.org/api_docs/python/tf/keras/optimizers (2019. 11)

제5고지: DeZero의 도전

[34] Srivastava, Nitish, et al, “Dropout: a simple way to prevent neural networks from overfitting,” The journal of machine learning research 15.1 (2014): 1929-1958.

[35] Ioffe, Sergey, and Christian Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167 (2015).

[36] Simonyan, Karen, and Andrew Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 (2014).

[37] He, Kaiming, et al, “Deep residual learning for image recognition,” Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.

[38] Iandola, Forrest N., et al, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size,” arXiv preprint arXiv:1602.07360 (2016).

[39] SPHINX, https://www.sphinx-doc.org/en/master/ (2019. 11)

[40] ONNX, https://onnx.ai/ (2019. 11)

[41] Goodfellow, Ian, et al, “Generative adversarial nets,” Advances in neural information processing systems, 2014.

[42] Kingma, Diederik P., and Max Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114 (2013).

[43] Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge, “Image style transfer using convolutional neural networks,” Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.

참고문헌 543

서브목차