Javascript required
Skip to content Skip to sidebar Skip to footer
Home / Pycolab Ascii Art Arrays to Stack Must Be Passed as a Sequence Type Such as List or Tuple

Pycolab Ascii Art Arrays to Stack Must Be Passed as a Sequence Type Such as List or Tuple

Your First Deep Learning Projection in Python with Keras Step-By-Step

Concluding Updated on October thirteen, 2021

Keras is a powerful and easy-to-employ costless open up source Python library for developing and evaluating deep learning models .

It wraps the efficient numerical computation libraries Theano and TensorFlow and allows you to define and train neural network models in only a few lines of lawmaking.

In this tutorial, you lot will notice how to create your get-go deep learning neural network model in Python using Keras.

Kick-showtime your projection with my new book Deep Learning With Python, including step-by-pace tutorials and the Python source code files for all examples.

Allow'southward become started.

  • Update Feb/2017: Updated prediction example so rounding works in Python ii and 3.
  • Update Mar/2017: Updated example for the latest versions of Keras and TensorFlow.
  • Update Mar/2018: Added alternate link to download the dataset.
  • Update Jul/2019: Expanded and added more useful resource.
  • Update Sep/2019: Updated for Keras v2.2.5 API.
  • Update Oct/2019: Updated for Keras v2.3.0 API and TensorFlow v2.0.0.
  • Update Aug/2020: Updated for Keras v2.4.three and TensorFlow v2.3.
  • Update Oct/2021: Deprecated predict_class syntax

Tour of Deep Learning Algorithms

Develop Your First Neural Network in Python With Keras Footstep-By-Step
Photograph by Phil Whitehouse, some rights reserved.

Keras Tutorial Overview

There is not a lot of lawmaking required, but we are going to step over information technology slowly so that y'all will know how to create your own models in the futurity.

The steps you are going to comprehend in this tutorial are as follows:

  1. Load Information.
  2. Ascertain Keras Model.
  3. Compile Keras Model.
  4. Fit Keras Model.
  5. Evaluate Keras Model.
  6. Tie It All Together.
  7. Make Predictions

This Keras tutorial has a few requirements:

  1. You lot accept Python 2 or 3 installed and configured.
  2. You have SciPy (including NumPy) installed and configured.
  3. You have Keras and a backend (Theano or TensorFlow) installed and configured.

If y'all need help with your surroundings, run across the tutorial:

  • How to Setup a Python Environment for Deep Learning

Create a new file called keras_first_network.py and blazon or copy-and-paste the code into the file equally you lot go.

Need help with Deep Learning in Python?

Have my gratis 2-week electronic mail grade and discover MLPs, CNNs and LSTMs (with code).

Click to sign-upwardly at present and as well get a costless PDF Ebook version of the form.

1. Load Data

The get-go footstep is to define the functions and classes nosotros intend to use in this tutorial.

We will utilize the NumPy library to load our dataset and we will use two classes from the Keras library to define our model.

The imports required are listed below.

We can now load our dataset.

In this Keras tutorial, nosotros are going to employ the Pima Indians onset of diabetes dataset. This is a standard motorcar learning dataset from the UCI Machine Learning repository. It describes patient medical tape data for Pima Indians and whether they had an onset of diabetes within five years.

As such, it is a binary classification trouble (onset of diabetes as 1 or not as 0). All of the input variables that describe each patient are numerical. This makes it easy to employ directly with neural networks that wait numerical input and output values, and ideal for our offset neural network in Keras.

The dataset is available from here:

  • Dataset CSV File (pima-indians-diabetes.csv)
  • Dataset Details

Download the dataset and place it in your local working directory, the aforementioned location as your python file.

Relieve it with the filename:

Have a look within the file, yous should see rows of information like the following:

We can now load the file as a matrix of numbers using the NumPy function loadtxt().

There are 8 input variables and one output variable (the last column). Nosotros volition be learning a model to map rows of input variables (X) to an output variable (y), which we frequently summarize every bit y = f(10).

The variables tin be summarized as follows:

Input Variables (X):

  1. Number of times pregnant
  2. Plasma glucose concentration a 2 hours in an oral glucose tolerance test
  3. Diastolic blood force per unit area (mm Hg)
  4. Triceps skin fold thickness (mm)
  5. 2-Hour serum insulin (mu U/ml)
  6. Body mass alphabetize (weight in kg/(top in one thousand)^two)
  7. Diabetes pedigree function
  8. Age (years)

Output Variables (y):

  1. Class variable (0 or 1)

Once the CSV file is loaded into retention, we can split the columns of data into input and output variables.

The data will be stored in a 2d array where the showtime dimension is rows and the second dimension is columns, east.g. [rows, columns].

We can split the array into two arrays past selecting subsets of columns using the standard NumPy slice operator or ":" We can select the get-go 8 columns from alphabetize 0 to index 7 via the piece 0:8. Nosotros tin and so select the output column (the ninth variable) via alphabetize viii.

Nosotros are now ready to define our neural network model.

Notation, the dataset has 9 columns and the range 0:8 volition select columns from 0 to 7, stopping before index viii. If this is new to you, then you tin can learn more near array slicing and ranges in this post:

  • How to Index, Slice and Reshape NumPy Arrays for Machine Learning in Python

2. Define Keras Model

Models in Keras are divers as a sequence of layers.

Nosotros create a Sequential model and add together layers one at a time until we are happy with our network architecture.

The first thing to go right is to ensure the input layer has the correct number of input features. This can exist specified when creating the beginning layer with the input_dim statement and setting it to 8 for the viii input variables.

How do nosotros know the number of layers and their types?

This is a very hard question. There are heuristics that we tin use and ofttimes the all-time network structure is found through a procedure of trial and error experimentation (I explain more about this hither). By and large, you lot demand a network large plenty to capture the structure of the problem.

In this case, we will utilise a fully-connected network construction with three layers.

Fully connected layers are divers using the Dense class. We tin specify the number of neurons or nodes in the layer as the first argument, and specify the activation function using the activation argument.

We volition use the rectified linear unit activation part referred to as ReLU on the first two layers and the Sigmoid role in the output layer.

It used to be the instance that Sigmoid and Tanh activation functions were preferred for all layers. These days, better operation is achieved using the ReLU activation part. Nosotros use a sigmoid on the output layer to ensure our network output is betwixt 0 and 1 and piece of cake to map to either a probability of class 1 or snap to a difficult classification of either class with a default threshold of 0.5.

We can piece it all together by adding each layer:

  • The model expects rows of data with eight variables (the input_dim=viii argument)
  • The beginning hidden layer has 12 nodes and uses the relu activation function.
  • The second hidden layer has eight nodes and uses the relu activation function.
  • The output layer has one node and uses the sigmoid activation function.

Note, the most disruptive thing hither is that the shape of the input to the model is defined as an argument on the first hidden layer. This means that the line of code that adds the first Dense layer is doing two things, defining the input or visible layer and the first hidden layer.

3. Compile Keras Model

Now that the model is defined, we can compile it.

Compiling the model uses the efficient numerical libraries under the covers (the so-called backend) such as Theano or TensorFlow. The backend automatically chooses the best way to represent the network for training and making predictions to run on your hardware, such every bit CPU or GPU or even distributed.

When compiling, we must specify some additional properties required when training the network. Remember training a network ways finding the all-time gear up of weights to map inputs to outputs in our dataset.

We must specify the loss function to use to evaluate a ready of weights, the optimizer is used to search through different weights for the network and any optional metrics we would like to collect and report during training.

In this case, we volition apply cross entropy as the loss statement. This loss is for a binary nomenclature problems and is defined in Keras as "binary_crossentropy". You lot can learn more nearly choosing loss functions based on your trouble hither:

  • How to Choose Loss Functions When Grooming Deep Learning Neural Networks

Nosotros volition define the optimizer as the efficient stochastic slope descent algorithm "adam". This is a popular version of gradient descent because it automatically tunes itself and gives good results in a broad range of bug. To learn more about the Adam version of stochastic gradient descent see the post:

  • Gentle Introduction to the Adam Optimization Algorithm for Deep Learning

Finally, because information technology is a classification problem, nosotros volition collect and study the classification accurateness, divers via the metrics argument.

four. Fit Keras Model

We have defined our model and compiled it gear up for efficient ciphering.

Now it is fourth dimension to execute the model on some information.

Nosotros tin railroad train or fit our model on our loaded information by calling the fit() part on the model.

Training occurs over epochs and each epoch is split into batches.

  • Epoch: One laissez passer through all of the rows in the training dataset.
  • Batch: One or more samples considered by the model within an epoch before weights are updated.

One epoch is comprised of 1 or more batches, based on the called batch size and the model is fit for many epochs. For more on the departure between epochs and batches, see the post:

  • What is the Difference Betwixt a Batch and an Epoch in a Neural Network?

The training process volition run for a fixed number of iterations through the dataset called epochs, that we must specify using the epochs argument. We must also set the number of dataset rows that are considered before the model weights are updated within each epoch, called the batch size and ready using the batch_size argument.

For this problem, we volition run for a small number of epochs (150) and use a relatively small-scale batch size of 10.

These configurations can be called experimentally by trial and error. We want to train the model plenty then that it learns a skillful (or good plenty) mapping of rows of input data to the output nomenclature. The model will always have some error, but the amount of mistake will level out after some signal for a given model configuration. This is called model convergence.

This is where the work happens on your CPU or GPU.

No GPU is required for this case, just if you're interested in how to run large models on GPU hardware cheaply in the cloud, see this post:

  • How to Setup Amazon AWS EC2 GPUs to Train Keras Deep Learning Models

5. Evaluate Keras Model

We have trained our neural network on the entire dataset and nosotros can evaluate the functioning of the network on the aforementioned dataset.

This will but give us an idea of how well we take modeled the dataset (e.g. train accuracy), simply no idea of how well the algorithm might perform on new information. We have done this for simplicity, but ideally, you could separate your data into train and test datasets for preparation and evaluation of your model.

You lot can evaluate your model on your training dataset using the evaluate() function on your model and pass it the same input and output used to train the model.

This will generate a prediction for each input and output pair and collect scores, including the boilerplate loss and any metrics you have configured, such as accuracy.

The evaluate() role volition return a listing with two values. The starting time volition exist the loss of the model on the dataset and the second volition be the accuracy of the model on the dataset. Nosotros are just interested in reporting the accuracy, and then we will ignore the loss value.

half dozen. Tie It All Together

You have just seen how you tin easily create your get-go neural network model in Keras.

Let's tie it all together into a complete code example.

Yous can copy all of the code into your Python file and salvage it as "keras_first_network.py" in the same directory as your information file "pima-indians-diabetes.csv". You tin then run the Python file every bit a script from your command line (command prompt) equally follows:

Running this example, you lot should meet a message for each of the 150 epochs printing the loss and accurateness, followed by the concluding evaluation of the trained model on the training dataset.

It takes nearly ten seconds to execute on my workstation running on the CPU.

Ideally, we would similar the loss to go to zero and accuracy to get to one.0 (due east.g. 100%). This is not possible for whatever only the most little machine learning problems. Instead, we volition e'er have some mistake in our model. The goal is to choose a model configuration and training configuration that accomplish the lowest loss and highest accuracy possible for a given dataset.

Annotation, if you effort running this example in an IPython or Jupyter notebook you may get an error.

The reason is the output progress bars during training. Yous tin can easily turn these off by setting verbose=0 in the call to the fit() and evaluate() functions, for example:

Annotation: Your results may vary given the stochastic nature of the algorithm or evaluation process, or differences in numerical precision. Consider running the example a few times and compare the boilerplate outcome.

What score did you go?
Post your results in the comments beneath.

Neural networks are a stochastic algorithm, meaning that the same algorithm on the same data can railroad train a different model with dissimilar skill each time the code is run. This is a characteristic, not a bug. You tin can acquire more nigh this in the post:

  • Embrace Randomness in Auto Learning

The variance in the functioning of the model means that to become a reasonable approximation of how well your model is performing, y'all may need to fit it many times and calculate the average of the accuracy scores. For more than on this approach to evaluating neural networks, see the post:

  • How to Evaluate the Skill of Deep Learning Models

For example, below are the accuracy scores from re-running the case 5 times:

We tin see that all accuracy scores are effectually 77% and the average is 76.924%.

7. Make Predictions

The number one question I get asked is:

Afterward I train my model, how can I apply information technology to make predictions on new information?

Great question.

We can adapt the above example and use information technology to generate predictions on the training dataset, pretending it is a new dataset we have not seen before.

Making predictions is as like shooting fish in a barrel as calling the predict() role on the model. Nosotros are using a sigmoid activation function on the output layer, so the predictions volition be a probability in the range between 0 and 1. We can easily convert them into a crisp binary prediction for this classification job by rounding them.

For example:

Alternately, we can catechumen the probability into 0 or i to predict crisp classes direct, for instance:

The complete case below makes predictions for each example in the dataset, then prints the input data, predicted class and expected class for the first 5 examples in the dataset.

Running the case does non testify the progress bar as before as we have set the verbose argument to 0.

After the model is fit, predictions are made for all examples in the dataset, and the input rows and predicted class value for the first five examples is printed and compared to the expected form value.

Nosotros tin encounter that most rows are correctly predicted. In fact, nosotros would expect about 76.9% of the rows to be correctly predicted based on our estimated performance of the model in the previous department.

If y'all would like to know more well-nigh how to make predictions with Keras models, see the mail:

  • How to Make Predictions with Keras

Keras Tutorial Summary

In this post, you discovered how to create your outset neural network model using the powerful Keras Python library for deep learning.

Specifically, you learned the six key steps in using Keras to create a neural network or deep learning model, step-by-step including:

  1. How to load data.
  2. How to define a neural network in Keras.
  3. How to compile a Keras model using the efficient numerical backend.
  4. How to railroad train a model on information.
  5. How to evaluate a model on data.
  6. How to make predictions with the model.

Do y'all take any questions virtually Keras or about this tutorial?
Ask your question in the comments and I volition practise my best to reply.

Keras Tutorial Extensions

Well washed, you have successfully developed your first neural network using the Keras deep learning library in Python.

This section provides some extensions to this tutorial that you might want to explore.

  • Melody the Model. Modify the configuration of the model or preparation process and come across if you can improve the performance of the model, east.chiliad. reach ameliorate than 76% accurateness.
  • Salve the Model. Update the tutorial to salvage the model to file, and so load it later on and apply it to make predictions (see this tutorial).
  • Summarize the Model. Update the tutorial to summarize the model and create a plot of model layers (see this tutorial).
  • Separate Train and Exam Datasets. Split the loaded dataset into a railroad train and test fix (dissever based on rows) and use 1 fix to train the model and the other set to judge the performance of the model on new data.
  • Plot Learning Curves. The fit() part returns a history object that summarizes the loss and accuracy at the finish of each epoch. Create line plots of this data, called learning curves (run across this tutorial).
  • Learn a New Dataset. Update the tutorial to utilize a different tabular dataset, peradventure from the UCI Car Learning Repository.
  • Use Functional API. Update the tutorial to apply the Keras Functional API for defining the model (encounter this tutorial).

Further Reading

Are you looking for some more than Deep Learning tutorials with Python and Keras?

Accept a wait at some of these:

Related Tutorials

  • 5 Stride Life-Cycle for Neural Network Models in Keras
  • Multi-Class Classification Tutorial with the Keras Deep Learning Library
  • Regression Tutorial with the Keras Deep Learning Library in Python
  • How to Filigree Search Hyperparameters for Deep Learning Models in Python With Keras

Books

  • Deep Learning (Textbook), 2016.
  • Deep Learning with Python (my book).

APIs

  • Keras Deep Learning Library Homepage
  • Keras API Documentation

How did y'all become? Practise you lot have whatsoever questions almost deep learning?
Post your questions in the comments below and I will do my all-time to help.

Develop Deep Learning Projects with Python!

Deep Learning with Python

 What If Yous Could Develop A Network in Minutes

...with just a few lines of Python

Detect how in my new Ebook:
Deep Learning With Python

It covers end-to-cease projects on topics like:
Multilayer Perceptrons,Convolutional Nets andRecurrent Neural Nets, and more...

Finally Bring Deep Learning To
Your Ain Projects

Skip the Academics. Just Results.

Meet What's Inside

oggandle1987.blogspot.com

Source: https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/