Tuesday, February 7, 2023
HomeArtificial IntelligencePicture Augmentation with Keras Preprocessing Layers and tf.picture

Picture Augmentation with Keras Preprocessing Layers and tf.picture

Final Up to date on August 6, 2022

If you work on a machine studying drawback associated to pictures, not solely do you’ll want to accumulate some photographs as coaching information, however you additionally must make use of augmentation to create variations within the picture. It’s very true for extra advanced object recognition issues.

There are various methods for picture augmentation. You could use some exterior libraries or write your individual capabilities for that. There are some modules in TensorFlow and Keras for augmentation too.

On this submit, you’ll uncover how you should utilize the Keras preprocessing layer in addition to the tf.picture module in TensorFlow for picture augmentation.

After studying this submit, you’ll know:

  • What are the Keras preprocessing layers, and methods to use them
  • What are the capabilities offered by the tf.picture module for picture augmentation
  • The way to use augmentation along with the tf.information dataset

Let’s get began.

Picture augmentation with Keras preprocessing layers and tf.picture.
Photograph by Steven Kamenar. Some rights reserved.


This text is split into 5 sections; they’re:

  • Getting Photographs
  • Visualizing the Photographs
  • Keras Preprocessing Layers
  • Utilizing tf.picture API for Augmentation
  • Utilizing Preprocessing Layers in Neural Networks

Getting Photographs

Earlier than you see how you are able to do augmentation, you’ll want to get the photographs. Finally, you want the photographs to be represented as arrays, for instance, in HxWx3 in 8-bit integers for the RGB pixel worth. There are various methods to get the photographs. Some could be downloaded as a ZIP file. In case you’re utilizing TensorFlow, you might get some picture datasets from the tensorflow_datasets library.

On this tutorial, you’ll use the citrus leaves photographs, which is a small dataset of lower than 100MB. It may be downloaded from tensorflow_datasets as follows:

Working this code the primary time will obtain the picture dataset into your laptop with the next output:

The operate above returns the photographs as a tf.information dataset object and the metadata. This can be a classification dataset. You possibly can print the coaching labels with the next:

This prints:

In case you run this code once more at a later time, you’ll reuse the downloaded picture. However the different method to load the downloaded photographs right into a tf.information dataset is to make use of the image_dataset_from_directory() operate.

As you possibly can see from the display output above, the dataset is downloaded into the listing ~/tensorflow_datasets. In case you have a look at the listing, you see the listing construction as follows:

The directories are the labels, and the photographs are recordsdata saved underneath their corresponding listing. You possibly can let the operate to learn the listing recursively right into a dataset:

You could wish to set batch_size=None if you do not need the dataset to be batched. Often, you need the dataset to be batched for coaching a neural community mannequin.

Visualizing the Photographs

It is very important visualize the augmentation consequence, so you possibly can confirm the augmentation result’s what we would like it to be. You need to use matplotlib for this.

In matplotlib, you may have the imshow() operate to show a picture. Nevertheless, for the picture to be displayed appropriately, the picture ought to be offered as an array of 8-bit unsigned integers (uint8).

Given that you’ve a dataset created utilizing image_dataset_from_directory()You will get the primary batch (of 32 photographs) and show a number of of them utilizing imshow(), as follows:

Right here, you see a show of 9 photographs in a grid, labeled with their corresponding classification label, utilizing ds.class_names. The pictures ought to be transformed to NumPy array in uint8 for show. This code shows a picture like the next:

The whole code from loading the picture to show is as follows:

Notice that in case you’re utilizing tensorflow_datasets to get the picture, the samples are offered as a dictionary as an alternative of a tuple of (picture,label). You need to change your code barely to the next:

For the remainder of this submit, assume the dataset is created utilizing image_dataset_from_directory(). You could must tweak the code barely in case your dataset is created in a different way.

Keras Preprocessing Layers

Keras comes with many neural community layers, akin to convolution layers, that you’ll want to prepare. There are additionally layers with no parameters to coach, akin to flatten layers to transform an array like a picture right into a vector.

The preprocessing layers in Keras are particularly designed to make use of within the early levels of a neural community. You need to use them for picture preprocessing, akin to to resize or rotate the picture or modify the brightness and distinction. Whereas the preprocessing layers are alleged to be half of a bigger neural community, you may also use them as capabilities. Under is how you should utilize the resizing layer as a operate to rework some photographs and show them side-by-side with the unique:

The pictures are in 256×256 pixels, and the resizing layer will make them into 256×128 pixels. The output of the above code is as follows:

For the reason that resizing layer is a operate, you possibly can chain them to the dataset itself. For instance,

The dataset ds has samples within the type of (picture, label). Therefore you created a operate that takes in such tuple and preprocesses the picture with the resizing layer. You then assigned this operate as an argument for the map() within the dataset. If you draw a pattern from the brand new dataset created with the map() operate, the picture will probably be a reworked one.

There are extra preprocessing layers accessible. Some are demonstrated beneath.

As you noticed above, you possibly can resize the picture. You may also randomly enlarge or shrink the peak or width of a picture. Equally, you possibly can zoom in or zoom out on a picture. Under is an instance of manipulating the picture measurement in varied methods for a most of 30% improve or lower:

This code reveals photographs as follows:

When you specified a hard and fast dimension in resize, you may have a random quantity of manipulation in different augmentations.

You may also do flipping, rotation, cropping, and geometric translation utilizing preprocessing layers:

This code reveals the next photographs:

And eventually, you are able to do augmentations on coloration changes as effectively:

This reveals the photographs as follows:

For completeness, beneath is the code to show the results of varied augmentations:

Lastly, it is very important level out that the majority neural community fashions can work higher if the enter photographs are scaled. Whereas we often use an 8-bit unsigned integer for the pixel values in a picture (e.g., for show utilizing imshow() as above), a neural community prefers the pixel values to be between 0 and 1 or between -1 and +1. This may be finished with preprocessing layers too. Under is how one can replace one of many examples above so as to add the scaling layer into the augmentation:

Utilizing tf.picture API for Augmentation

In addition to the preprocessing layer, the tf.picture module additionally supplies some capabilities for augmentation. Not like the preprocessing layer, these capabilities are meant for use in a user-defined operate and assigned to a dataset utilizing map() as we noticed above.

The capabilities offered by the tf.picture aren’t duplicates of the preprocessing layers, though there’s some overlap. Under is an instance of utilizing the tf.picture capabilities to resize and crop photographs:

Under is the output of the above code:

Whereas the show of photographs matches what you would possibly count on from the code, the usage of tf.picture capabilities is kind of totally different from that of the preprocessing layers. Each tf.picture operate is totally different. Due to this fact, you possibly can see the crop_to_bounding_box() operate takes pixel coordinates, however the central_crop() operate assumes a fraction ratio because the argument.

These capabilities are additionally totally different in the way in which randomness is dealt with. A few of these capabilities don’t assume random habits. Due to this fact, the random resize ought to have the precise output measurement generated utilizing a random quantity generator individually earlier than calling the resize operate. Another capabilities, akin to stateless_random_crop(), can do augmentation randomly, however a pair of random seeds within the int32 must be specified explicitly.

To proceed the instance, there are the capabilities for flipping a picture and extracting the Sobel edges:

This reveals the next:

And the next are the capabilities to control the brightness, distinction, and colours:

This code reveals the next:

Under is the entire code to show all the above:

These augmentation capabilities ought to be sufficient for many makes use of. However when you have some particular concepts on augmentation, you’ll in all probability want a greater picture processing library. OpenCV and Pillow are widespread however highly effective libraries that mean you can rework photographs higher.

Utilizing Preprocessing Layers in Neural Networks

You used the Keras preprocessing layers as capabilities within the examples above. However they will also be used as layers in a neural community. It’s trivial to make use of. Under is an instance of how one can incorporate a preprocessing layer right into a classification community and prepare it utilizing a dataset:

Working this code provides the next output:

Within the code above, you created the dataset with cache() and prefetch(). This can be a efficiency method to permit the dataset to arrange information asynchronously whereas the neural community is educated. This is able to be vital if the dataset has another augmentation assigned utilizing the map() operate.

You will note some enchancment in accuracy in case you take away the RandomFlip and RandomRotation layers since you make the issue simpler. Nevertheless, as you need the community to foretell effectively on a large variation of picture high quality and properties, utilizing augmentation might help your ensuing community grow to be extra highly effective.

Additional Studying

Under is a few documentation from TensorFlow that’s associated to the examples above:


On this submit, you may have seen how you should utilize the tf.information dataset with picture augmentation capabilities from Keras and TensorFlow.

Particularly, you discovered:

  • The way to use the preprocessing layers from Keras, each as a operate and as a part of a neural community
  • The way to create your individual picture augmentation operate and apply it to the dataset utilizing the map() operate
  • The way to use the capabilities offered by the tf.picture module for picture augmentation

Develop Deep Studying Initiatives with Python!

Deep Learning with Python

 What If You Might Develop A Community in Minutes

…with just some strains of Python

Uncover how in my new Book:

Deep Studying With Python

It covers end-to-end initiatives on subjects like:

Multilayer PerceptronsConvolutional Nets and Recurrent Neural Nets, and extra…

Lastly Carry Deep Studying To

Your Personal Initiatives

Skip the Lecturers. Simply Outcomes.

See What’s Inside



Please enter your comment!
Please enter your name here

Most Popular

Recent Comments