Neural Networks #1: Basics

less than 1 minute read

Feed Forward Neural Networks

Neural networks are made of layers of nodes connected to eachother with weights. At each layer the nodes apply an activation function (non-linear) to the previous layer and pass the output times some weight to the next layer.

  • The input layer is a vector of the input $x$ values and a bias
  • Each hidden layer is a vector of nodes that act on the output function of the previous layer
  • The output layer is a single node that calculates the final output value

Code

A neural net can be trained using the sklearn MLP Classifier module

from sklearn.neural_network import MLPClassifier

nn = MLPClassifier(hidden_layer_sizes=(35,),
                   activation='relu',
                   alpha=1e-4,
                   
                   # Optimization params
                   max_iter=10,    # Number of epochs
                   solver='sgd',   # Stochastic gradient descent
                   tol=1e-4,
                   learning_rate_init=.1,
                   
                   # Other
                   random_state=1,
                   verbose=True)

Fitting and predictions are done in the normal way

nn.fit(X_train, y_train)

y_test_pred = nn.predict(X_test)

Recurrent Neural Networks

RNNs are particularly good at language modelling, machine translation, & text classification.

Convolutional Neural Networks

CNNs are the basis for the most advanced imaged-based machine learning applications such as object detection & image classification.

Updated: