This project is a Regression models that take any Logic Gate as a data and tries to change its weight according to Logic Gate.
If we provide like data = [[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1]] where data[i][0] and data[i][1] are inputs/switches and data[i][2] is output which represent OR Logic Gate, Then the model will predict the weights of that logic gate
logic_gates_regression.py: Implementation of single Neural Layerlgrm.py: Implementation of single Neural Layer but with Matrix
After running logic_gates_regression.py:
0 0 | 0 = 0.08078284864022509
0 1 | 1 = 0.9493994252904192
1 0 | 1 = 0.9494216592658261
1 1 | 1 = 0.999750537874312
which is pretty accurate as given the data of OR
you can change the logic gate data of different logic gates like AND or NAND in the file
logic_gates_regression.pyand observer what output models give.
It is a single neuron that accepts two input and give one output
Since it is only one Neuron it can not predict the weights for XOR because XOR need both NAND, AND and OR Logic Gates.
XOR: data = [[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 0]]
- To solve this we need more than one layors.
To fit a Neural Network for the XOR logic gate, we need more than one layer. For previous logic gates, we only needed one layer because they were simple functions like AND or OR. In those cases, we could use just two layers: one input layer and one output layer.
But XOR is different. It cannot be solved with a single straight line or a single layer of computation. Instead, XOR can be thought of as a combination of simpler logic operations like NAND, AND, and OR working together.
Because of this, we need three layers: an input layer, a hidden layer, and an output layer. The hidden layer is important because it learns intermediate patterns. Intuitively, you can think of it as the part that figures out helpful combinations like NAND and AND, and then the output layer uses those results to produce the final XOR output.
xor/xor.py: this is an implementation of xor model thats uses finite differencexor/matrix.py: this code is stripped from my other project minimath
after the fitting we get something like this
0.2332609678734387
0.18858506748101467
0.1491174201443169
0.044203639047194376
0.015088221216178404
0.008234421546795847
0.005497796884104048
0.00407331671599689
--- Result ---
0^0 = 0.05877567308130995
0^1 = 0.9457659678597024
1^0 = 0.945406620914334
1^1 = 0.058965208246468497
as you can see it's pretty accurate.
you can build xor model without matrix too, but using matrix will help us later, when we will use backpropagation method
No one uses Finite Difference in large real world models, the reason is it's very very computationally expensive, because we need to compute the cost by slightly changing every single parameter in the model one by one.
For real world models we use backpropagation that instead of checking each parameter separately, it computes the gradient of all parameters in one efficient pass. It uses the chain rule to reuse intermediate results from the forward pass and calculate gradients for every parameter that affects the output, without recomputing the whole model again and again.
In future we will also discuss about backpropagation.