Issue
I am a noob in python and needed to train a model on a dataset.I found both the notebook and dataset at the same place and made appropriate changes to notebook to run the data from storage.The code fails at the training stage after 1 epoch completes with 'graph execution error'
Here is my jupyter notebook:https://github.com/Megahedron69/wasteSegregationmodel
Here is the dataset:https://www.kaggle.com/datasets/aashidutt3/waste-segregation-image-dataset
Here is the original notebook:https://www.kaggle.com/code/gpiosenka/waste-f1-score-97
exact error location in notebook:
Solution
Thanks to @Dr. Snoopy answer.There were corrupt images in my data set so used a simple python script in my root directory to remove truncated images
#pip install pillow
from PIL import Image
import os
def find_truncated_images(dataset_dir):
truncated_images = []
for root, _, files in os.walk(dataset_dir):
for filename in files:
file_path = os.path.join(root, filename)
try:
with Image.open(file_path) as img:
img.load()
except (IOError, OSError) as e:
# Log the file path if it's a truncated image
print(f"Truncated image: {file_path}")
truncated_images.append(file_path)
return truncated_images
def remove_truncated_images(truncated_images):
for file_path in truncated_images:
try:
os.remove(file_path)
print(f"Removed: {file_path}")
except OSError as e:
print(f"Error removing {file_path}: {e}")
if __name__ == "__main__":
dataset_dir = "." # Set the path to your dataset directory
truncated_images = find_truncated_images(dataset_dir)
remove_truncated_images(truncated_images)
Answered By - Kartic Joshi
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.