First NN implementation attempt

[roopekole]

Summary of the experiments with this implementation:

• tried to train a fully connected neural network
• used stabilizers and destabilizers flattened into a vector to predict Y matrix turned into a vector (and later back to matrix)
• completely ignored the masking matrix as discussed with the client

The problem is that the model does not learn - loss on the validation data remains high even with crazy number of epochs (like +20 000 which is abnormally high number and takes a long time to train). I tried with different learning rates and schedulers to somehow generalize to new data, but no results.

Also tried with different model complexities (1-5 hidden layers with varying neuron amounts) but the results did not improve. The story remains the same: training loss gets small but the validation does not.

The best outcome I achieved was 3 layers with 64 neurons passing through a few thousand epochs after which the validation stopped converging and overfitting became a problem (and early stopping was applied). The learning progression is attached as an image. The evaluation fails to beat the random pivot picker:

method	n_rep	num_qubits	h	s	cx	depth
nn	9.5	4.0	12.75	16.55	10.45	22.75
normal_heuristic	9.5	4.0	12.3	16.45	7.9	20.7
optimum	9.5	4.0	10.2	15.7	5.95	17.45
random	9.5	4.0	11.15	14.65	10.4	21.05