=================================================================================

Graphical Processing Units (GPUs) are well-suited for the parallel processing required by many deep learning tasks. GPUs have many cores that can perform parallel computations simultaneously, which accelerates the training of neural networks. This parallelization significantly reduces the time required to train deep learning models compared to using only CPUs.

Fast GPUs can help you get from experiment to result as quickly as possible.The Keras handles the problem in a modular way as shown in Figure 4150.

Figure 4150. The deep-learning software and hardware stack in Keras process. [1]

To use TensorFlow, you don't need to explicitly require the use of a GPU. However, TensorFlow will automatically try to use it if you have a GPU. On the other hand, if you have more than one GPU, you must assign operations to each GPU explicitly, or only the first one will be used. To do this, you simply need to type the line as following:
         with tensorflow.device("/gpu:1"):
              "/cpu:0": the main CPU of your machine
              "/gpu:0": the first GPU of your machine, if one exists
              "/gpu:1": the second GPU of your machine, if a second exists
              "/gpu:2": the third GPU of your machine, if a third exists, and so on

PyTorch is a deep learning library based on Torch and developed by Facebook. It is relatively easy to use, and thus has recently gained a lot of popularity. It will automatically select a GPU, if one is available, reverting to the CPU otherwise.

============================================

[1] François Chollet, Deep Learning with Python, 2018.