-
Notifications
You must be signed in to change notification settings - Fork 2
/
TODO
38 lines (29 loc) · 1.97 KB
/
TODO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
Feature requests:
- Add further levels of verbosity printing
- Add handling of convolution method/type
- Add padding to convolution and maxpool methods, instead of for input data
- Set up loss to apply normaliser by default if softmax layer not present (similar to keras)
USEFUL BACKGROUND READING - MACHINE LEARNING
============================================
DropBlock:
- https://arxiv.org/pdf/1810.12890.pdf
Batch Normalisation:
- https://kevinzakka.github.io/2016/09/14/batch_normalization/
Other adaptive learning rates:
- https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8563135/
- http://almostconvergent.blogs.rice.edu/2020/12/07/momentumrnn-integrating-momentum-into-recurrent-neural-networks/
AdamW:
- https://towardsdatascience.com/why-adamw-matters-736223f31b5d
Data viewing:
- https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=4,2&seed=0.62319&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false
Convolutional neural networks:
- https://www.analyticsvidhya.com/blog/2018/12/guide-convolutional-neural-network-cnn/
- https://blog.roboflow.com/what-is-a-convolutional-neural-network/#:~:text=Activation%20Function%20Layer&text=It%20takes%20in%20the%20weighted,Sigmoid
- https://arxiv.org/pdf/1603.07285.pdf
Convolutional layer including stride:
- https://medium.com/@mayank.utexas/backpropagation-for-convolution-with-strides-8137e4fc2710
USEFUL BACKGROUND READING - FORTRAN
===================================
assumed rank: https://community.intel.com/t5/Intel-Fortran-Compiler/Passing-negative-indexed-arrays-to-a-subroutine/m-p/1131949
assumed shape: https://pages.mtu.edu/~shene/COURSES/cs201/NOTES/chap08/assumed.html
https://wandb.ai/byyoung3/ml-news/reports/New-Method-For-LLM-Quantization--VmlldzozOTU1NTgz