[FIXED] Can Horovod with TensorFlow work on non-GPU instances in Amazon SageMaker?

Issue

I want to perform distributed training on Amazon SageMaker. The code is written with TensorFlow and similar to the following code where I think CPU instance should be enough:  https://github.com/horovod/horovod/blob/master/examples/tensorflow_word2vec.py

Can Horovod with TensorFlow work on non-GPU instances in Amazon SageMaker?

Solution

Yeah you should be able to use both CPU's and GPU's with Horovod on Amazon SageMaker. Please follow the below example for the same

https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-python-sdk/tensorflow_script_mode_horovod/tensorflow_script_mode_horovod.ipynb

Answered By - Arun Lokanatha