A Beginner’s Guide to TensorFlow: Building Machine Learning Model

The “divide and conquer” rule can make solving big and complicated tasks a breeze. Imagine we have a big puzzle and access to good friends who are willing to help. Each friend can solve a small part of the puzzle, and these small parts are combined once finished. This approach solves the problem in smaller steps to obtain the final result. In terms of machine learning, solving a large task is quite a resource-consuming and time-taking task. TensorFlow breaks a complex machine learning task into smaller tasks with each task processed in parallel. This makes the overall process efficient.

TensorFlow is an open-source framework for implementing machine learning models. Additionally, the advent of deep learning-based neural networks has made TensorFlow more in demand in various domains like natural language processing, image, and audio recognition, etc. If you’re new to TensorFlow and want to train and evaluate ML and DNN models, you’re in the right place. This blog provides a step-by-step guide to TensorFlow on building machine learning (ML) and deep neural network (DNN) models using this powerful framework.

Getting started with TensorFlow#

A tensor is a multi-dimensional data structure representing the input, output, and intermediate data in TensorFlow computations. In TensorFlow, a graph represents a computation with nodes and edges. A node represents operations, and edges represent the data flow between these operations.

Building an ML model#

A machine learning model consists of a neural network with interconnected nodes organized into layers. Each node takes input from multiple nodes from the previous layer. Let’s take an example of a simple graph with an “add” operation as shown below:

In our simple example that depicts the “add” operation, layer one consists of two constants, a and b, and layer two consists of a single operation node c. The edges represent a flow of data between nodes. The first layer’s output becomes the second layer’s input, and this process continues until the final layer, which produces the network’s output.

Let’s see how we can create this simple graph in TensorFlow:

• Line 1: Imports the tensorflow library.
• Line 4: Creates a graph using the tf.Graph() function.
• Lines 9–11: Defines two nodes a and b using the tf.constant() function to hold constant values of 5 and 10, respectively.
• Line 11: Creates the c node using the tf.add() function to add values of a and b.
• Line 14: Creates a session using tf.Session(graph=graph).
• Line 16: Runs the session to execute the operations within the graph.

Creating a simple neural network#

Let’s build a simple neural network to implement an XOR logic gate with the following structure:

• An input layer with two nodes representing the two inputs of the XOR gate.
• A hidden layer with two nodes. This layer is called the hidden layer since it is not directly observable or accessible in terms of the input or output of the network.
• An output layer with one node representing the network output.

The input-output relation of an XOR logic gate is depicted in the table below:

Let’s build our neural network in TensorFlow:

• Lines 5–6: Defines the input-output relation for the XOR logic gate.
• Lines 9–11: Defines the number of nodes for input, hidden, and output layers.

The resulting graph looks as follows:

Assigning weights and bias#

Now that we have a basic structure of the XOR neural network, we start by assigning weights and biases to the network. Weights are numerical values associated with the graph edges and determine the weightage of that connection in influencing the output. Biases are parameters associated with each node, excluding the nodes in the input layer. These introduce an offset to the weighted sum of inputs to a node.

Let’s assign weights and biases to our neural network:

• Lines 6–7: Defines weights and biases of the hidden layer.

• Lines 10–11: Defines weights and biases of the output layer.

Note: The initial weights assigned here are randomly chosen from a normal distribution, and all biases are zeros. We will see later how to update the values of weights and biases as we train our model.

The resultant network with weights and biases looks as follows:

As an example, the output of the first node in the hidden layer will be calculated as follows:

hidden 1=input 1×w1+input 2×w2+b1\text{hidden \ 1} = \text{input\ 1} \times w1 + \text{input\ 2} \times w2 + b_1

Applying the activation function#

An activation function is applied to the weighted sum of inputs and biases of the node to produce an output of the node. An XOR logic gate is a non-linear operation, and we must introduce non-linearity into the model to make the network learn a non-linear XOR logic. Hence, we use an activation function to introduce non-linearity.

For the hidden layer, we will use a sigmoid function as an activation function that provides a smooth and continuous non-linear transformation, and it is mathematically written as follows:

S(hidden 1)=11+e−(hidden 1)S(\text{hidden \ 1}) = \frac{1}{1 + e^{-(\text{hidden \ 1})}}

For the output layer, we perform a linear transformation of the hidden layer outputs, weights, and bias as follows:

output=hidden 1×w5+hidden 2×w6+b3\text{output} = \text{hidden\ 1} \times w5 + \text{hidden\ 2} \times w6 + b_3

Let’s apply these two activation functions:

• Line 4: Applies the sigmoid activation function to the nodes in the hidden layer.

• Line 6: Applies the linear transformation to calculate the network output.

Defining the loss function and optimizer#

Now that we have completed one pass of the network, we need to check if the network’s output matches the desired output. To do so, we define a loss function. A loss function measures the dissimilarity between the predicted and expected output. This helps us to quantify the model’s performance. We will use a mean squared error (MSE) as a loss function that calculates the average squared difference between the predicted and expected output values.

Based on the output of the loss function, we now need to update the model’s parameters, like weights and biases. We will use gradient descent optimization algorithms to find the optimum values of weights and biases. A gradient descent tunes the parameters in the direction of the steepest descent of the loss function.

Let’s implement the loss function and the optimization algorithm into our machine learning model:

• Line 2: Defines the loss function to reduce the mean squared error between the output_layer and y. Here y is the expected output and output_layer is the predicted output.

• Lines 5–6: Defines the optimization algorithm with parameter learning_rate. This defines the step size taken during each parameter update. A larger value of learning_rate can result in faster convergence but also risks overshooting the optimal value. Similarly, a smaller value of learning_rate results in slow convergence but is expected to provide a more precise result.

Training the model#

Now that everything is in place, it’s time to train our model using the training date defined in x_train and y_train:

• Lines 2–3: Defines the training data that represents the input-output relation for the XOR logic gate.

• Lines 7–8: Defines the training parameters num_epochs and batch_size.

• Lines 13–17: Performs the training using training data by passing the data in batches of size batch_size to our model. This procedure is performed num_epochs times.

Let’s visualize the training procedure using the following illustration:

Validating the model#

Now is the time to validate our model to check if the predicted output is the same as the expected output. We can input once again our training data and compare the model’s output to the expected output:

The predicted output shows that our neural network has converged to the expected output of an XOR logic gate. The first and last predicted outputs are close to 0 and the second and third predicted outputs are close to 1.

Next steps#

This blog has briefly introduced TensorFlow and how we can build a machine learning model using tensors.

We encourage you to explore more activation and loss functions and practice building more complex machine learning models. You can also check out the following courses on Educative to learn machine learning: