DAM101 Unit One (Deep Learning Fundamentals)

Take Aways from unit one.

What is AI: Artificial Intelligence (AI) can be simply defined as an imitation of human behaviors by a machine. AI came into existence in 1957 when a programmer wrote code that can play checkers against him, and this process of creating a bot that can play checkers gradually became a technology known as Artificial Intelligence.

Difference between Deep Learning and Machine Learning:

Machine Learning (ML) :

• AI is the big idea of making computers smart enough to think like a human, in which machine learning is a method to accomplish those goals. Therefore, machine learning is also called a subset of AI.
• It can train on a small set of data.
• Can train on a CPU (central processing unit).
• Makes simple and linear correlations.

Deep Learning:

• Machine learning is a big toolbox to make a computer think smart like a human, which needs human intervention to learn and to correct. But deep learning is the next evolution which can learn by itself by reflecting on past results; therefore, it’s called a subset of machine learning.

• It requires a large set of data.
• Needs computational resources like GPU (graphics processing unit) to train.
• Makes non-linear and complex correlations.

How does a neural network learn?

Training a neural network is akin to teaching a toddler to speak for the first time. At the outset, the network begins with no prior knowledge, and through a process of repetition and reinforcement, it adjusts its internal parameters. As the training progresses, the network improves its ability to predict outcomes.

Artificial neurons

Artificial neurons are connection points in an artificial neural network. Artificial neural networks, like biological neural networks, have a layered architecture, and each node has the capability of processing the input data and forwarding the output data on the network.

A neural network is composed of five things

1. Nodes and Layers * Input Layer: This layer receives input from the users.
   • Hidden Layer: An intermediate layer between the input and output layers where all the computations happen.
   • Output Layer: Gives the results of the computations done.:

2. Linear Regression Model:

   • Each artificial neuron has its own linear regression model.

3. Nodes and Layers Organization:

   • Nodes are called the multiprocessing units.
   • Nodes are organized in layers.
   • Layers are connected via weights.
   • The number of layers depends on where the layer resides and the network architecture design.

4. Output Values and Weights:

   • Each node has an output value that is the result of the operation that occurred with the data passed to it as an input, along with the weight that connects the inputs to the nodes.
   • Weights between the connections determine how much influence each input has on its output.

5. Data Passing and Training:

   • Data is passed from one layer to another through a feedforward network.
   • Neural networks rely on training data.

Artificial neural networks are all inspired by biological neurons, but they omit all the basic functionalities of a biological neuron

Biological neurons :

• Information processing : Neurons receive impulses from other neurons through dendrites, integrate all the impulses, and pass them down through the axon.
• Communication : Neurons communicate with each other through synapses, which are the junctions between two neurons.

Artificial neurons:

• Information processing : They take input from a neuron layer that precedes them, with some weight associated with every input. They find the weighted sum by adding biases, and a mathematical function called an activation function decides whether to send the information to the next neuron layer based on its own recruitment.
• Communication : Neurons in the artificial neural network communicate through connections, each with individual weights determining its influence on the outcome. With training, all those weights are adjusted to perform tasks with the highest accuracy.

Single Layered Perceptron

A single-layer perceptron is the simplest form of a feedforward neural network, consisting of only one layer of perceptron . In this architecture, there are no hidden layers between the input and output layers.

Quick rundown of the single-layer perceptron:

Structure:

• A single-layer perceptron consists of an input layer and an output layer.
• The input layer contains neurons representing input features. Unlike multi-layer perceptrons, single-layer perceptrons do not have a hidden layer. Each input neuron is directly connected to the output neurons.

Functionality:

• Each neuron in the output layer computes a weighted sum of the input features.
• This weighted sum is then passed through an activation function, often a step function or a sigmoid function, to produce an output.
• The output is typically binary (0 or 1), or in the case of continuous output, it ranges between 0 and 1.

• A single-layer perceptron is the simplest type of artificial neural network and can only classify linearly separable cases with a binary target.

Training:

• Single-layer perceptrons are trained using supervised learning algorithms, in which the model is trained on a dataset consisting of input-output pairs.

• The goal is for the model to learn a mapping from input data to output labels.

• During training, the weights of connections between the input and output neurons are adjusted to minimize the error between the predicted output and the actual output.

Limitations:

• Single-layer perceptrons can only solve linearly separable problems, meaning they can only learn decision boundaries that are linear.

• They are not capable of learning complex patterns or relationships; therefore, they do not have the ability to solve real-world problems with nonlinear data distributions.