dl : MPI with fortran binary (24jan022)
dl.f95 : I started here; uses MPI (18jan022)
hello.cu : learning to use CUDA (18jan022)
nn : the CUDA binary (24jan022)
nn.cu : Let's write a neural network code using CUDA (18jan022)
nn.h : Let's put symbols into a header file (18jan022)
--------------------------------------------------------------------------------
2^NBITS-2 numbers
for each number:

NBITS => NNeurons layer 1 neurons => layer 2 neuron -> output

On GPU: (eventually) each block gets one data element
   	entire network forward and back done in one block
   	each thread gets one neuron

(could vary architecture or hyperparameters across MPI processes)

<0 ==> composite
>0 ==> prime
truth is +/- 1 if got it wrong

Let's use truth-output as error function.
Slope is +/- 1.
Depends only on sign of output.
Sign is opposite that of output.

But I think that error function does not work well.  Seeks output > truth.  Which can be true even when getting wrong answers.  Somehow I am confusing myself.  I'll do it LDL's way.

For now call the entire kernel for each datapoint.  (Can't distribute
data among blocks until they are independent.)
Yes this is really stupid for now!

I think I can decompose the inner products in each layer across threads,
though their size varies with layer.
Let's just use the larger and waste some threads.

--------------------------------------------------------------------------------
HOST:
y[0..1<<NBITS-1]	ground truth


GPU:
ni 0..NNEURONS	ni=0 does not appear in a1[] on host -- need to subtract 1
ti 0..NBITS    

in[0..NBITS]   in[0] = 1. (in[] on host is array of input for all data)

a1[0..NNEURONS][0..NBITS]  a1[0][] has no meaning on CPU -- just to pad same as on GPU
			   a1[ni][0] is bias of neuron ni
			      a1[ni][i>0] is weight i of neuron ni
z[0..NNEURONS]	z[0] = 1.
a2[0..NNEURONS] a2[0] = bias of layer-2 neuron