Cassava tree disease recognition based on vgg16 image classification

---

Preface

Identification and model integration of cassava leaf disease
There are many hills and dry lands in Guangxi , Natural conditions are suitable for cassava production . Guangxi has been planting cassava on a large scale , Keep the cassava of the whole country
In the forefront of production . Cassava is not only the main source of farmers' income , And its processing products are rich , The industrial chain is long , the
The economy has a wide range of influences . However , The diagnosis and control of cassava diseases and insect pests have brought great challenges to the production of cassava farmers .
The main problems are as follows ：
（1） Offline consultation of agricultural experts , There is a problem of low timeliness ;
（2） Online consultation of agricultural experts , There is a problem that the pictures taken by farmers are not professional , It affects the accuracy of experts' online diagnosis to distinguish cassava diseases and pests .
For the above problems , be based on VGG-16 Deep learning neural network is designed to identify four diseases 、 Cassava health
And non cassava leaves （ Six categories in total ） Model of , Through big data technology 、 With the help of cloud computing technology , Running successfully
Model recognition classifies more than 15000 cassava pictures .

---

One 、 Dataset analysis ：

The original dataset contains two folders
1 With all pictures data Folder
2 Contains the names and categories of all the pictures csv file , common 6 Categories

Two 、 Data preprocessing method ：

1. Add the folder needed to partition the validation set

The code is as follows （ Example ）：

os.makedirs('./data-raw/train/0')
os.makedirs('./data-raw/train/1')
os.makedirs('./data-raw/train/2')
os.makedirs('./data-raw/train/3')
os.makedirs('./data-raw/train/4')
os.makedirs('./data-raw/train/5')

os.makedirs('./data-raw/vel/0')
os.makedirs('./data-raw/vel/1')
os.makedirs('./data-raw/vel/2')
os.makedirs('./data-raw/vel/3')
os.makedirs('./data-raw/vel/4')
os.makedirs('./data-raw/vel/5')

---

2. Read in the data

Read train.csv file
take image_id Save as list to data_img_name
take label Save as list to label
Take care to remove the first line

The code is as follows （ Example ）：

import csv
f_csv = open('./data/raw/2021hwjx/train.csv', "r")
read = csv.reader(f_csv)
a = list(read)
data_img_name = []
label = []
for i in a:
    data_img_name.append(i[0])
    label.append(i[1])
data_img_name.remove('image_id')
label.remove('label')

---

3. Dataset splitting function :

Will list full_list In proportion ratio（ Random ） Divided into 2 Sublist sublist_1 And sublist_2
:param full_list: Data list
:param ratio: Child list 1
:param shuffle: Child list 2
:return:

The code is as follows （ Example ）：

def data_split(full_list, ratio, shuffle=False):

    n_total = len(full_list)
    offset = int(n_total * ratio)
    if n_total == 0 or offset < 1:
        return [], full_list
    if shuffle:
        random.shuffle(full_list)
    sublist_1 = full_list[:offset]
    sublist_2 = full_list[offset:]
    return sublist_1, sublist_2

---

4. Partition validation set

Slice and classify data sets , In proportion to Verification set ： Training set =2：8
Select randomly for each category 80% Then merge into a list
（ Directly from csv The table shows the row to which each category ends ）
The code is as follows （ Example ）：

train_0 = data_img_name[:978]
test_data0, train_data0 = data_split(train_0, ratio=0.2)
test_label_0 = [0]*len(test_data0)
train_label_0 = [0]*len(train_data0)

train_1 = data_img_name[978:2948]
test_data1, train_data1 = data_split(train_1, ratio=0.2)
test_label_1 = [1]*len(test_data1)
train_label_1 = [1]*len(train_data1)

train_2 = data_img_name[2948:5095]
test_data2, train_data2 = data_split(train_2, ratio=0.2)
test_label_2 = [2]*len(test_data2)
train_label_2 = [2]*len(train_data2)

train_3 = data_img_name[5095:10637]
test_data3, train_data3 = data_split(train_3, ratio=0.2)
test_label_3 = [3]*len(test_data3)
train_label_3 = [3]*len(train_data3)

train_4 = data_img_name[10637:12956]
test_data4, train_data4 = data_split(train_4, ratio=0.2)
test_label_4 = [4]*len(test_data4)
train_label_4 = [4]*len(train_data4)

train_5 = data_img_name[12956:]
test_data5, train_data5 = data_split(train_5, ratio=0.2)
test_label_5 = [5]*len(test_data5)
train_label_5 = [5]*len(train_data5)

#  Training set 
train_data = train_data0 + train_data1 + train_data2 + train_data3 + train_data4 + train_data5
train_label = train_label_0 + train_label_1 + train_label_2 + train_label_3 + train_label_4 + train_label_5

#  Verification set 
test_data = test_data0 + test_data1 + test_data2 + test_data3 + test_data4 + test_data5
test_label = test_label_0 + test_label_1 + test_label_2 + test_label_3 + test_label_4 + test_label_5

---

5. Classify and save the divided pictures

Copy the original picture to the folder with the category

The code is as follows （ Example ）：

import shutil

root_dir = './data/raw/2021hwjx/train_img'

a = 6
train_range = []

for j in range(a):
    if j == 0:
        train_range = train_data0
    if j == 1:
        train_range = train_data1
    if j == 2:
        train_range = train_data2
    if j == 3:
        train_range = train_data3
    if j == 4:
        train_range = train_data4
    if j == 5:
        train_range = train_data5
    for i in range(len(train_range)):
        img_dir = os.path.join(root_dir, train_range[i])
        shutil.move(img_dir, './data-raw/train/{}'.format(j))


for j in range(a):
    if j == 0:
        test_range = test_data0
    if j == 1:
        test_range = test_data1
    if j == 2:
        test_range = test_data2
    if j == 3:
        test_range = test_data3
    if j == 4:
        test_range = test_data4
    if j == 5:
        test_range = test_data5
    for i in range(len(test_range)):
        img_dir = os.path.join(root_dir, test_range[i])
        shutil.move(img_dir, './data-raw/vel/{}'.format(j))

---

6. Original image preprocessing :

Transform the original
Including tailoring
normalization
take PIL image Turn into VGG16 Needed tensor type

The code is as follows （ Example ）：

from torch.utils.data import DataLoader
from torchvision import datasets, transforms
import torchvision

batch_size = 16

train_transforms = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize((.5, .5, .5), (.5, .5, .5))
])
val_transforms = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize((.5, .5, .5), (.5, .5, .5))
])

train_dir = './data-raw/train'
train_datasets = datasets.ImageFolder(train_dir, transform=train_transforms)
train_data_loader = torch.utils.data.DataLoader(train_datasets, batch_size=batch_size, shuffle=True)

val_dir = './data-raw/vel'
val_datasets = datasets.ImageFolder(val_dir, transform=val_transforms)
val_data_loader = torch.utils.data.DataLoader(val_datasets, batch_size=batch_size, shuffle=True)

---

3、 ... and 、 Create a network model ：

download VGG16 Pre training model
Modify the model
Add a layer of 1000 Input 6 Output linear layer , Make it meet our classification needs

The code is as follows （ Example ）：

import torchvision
from torch import nn

vgg16_true = torchvision.models.vgg16(pretrained=True)
vgg16_true.classifier.add_module('add_liner', nn.Linear(1000, 6))

---

Four 、 Set model parameters ：

Including whether it supports GPU Speed up
Learning rate

The code is as follows （ Example ）：

if torch.cuda.is_available():
    vgg16_true = vgg16_true.cuda()
loss_fn = nn.CrossEntropyLoss()

if torch.cuda.is_available():
    loss_fn = loss_fn.cuda()
    
learning_rate = 0.0001   #  Learning rate 
optimizer = torch.optim.SGD(vgg16_true.parameters(), lr=learning_rate)

---

5、 ... and 、 model training ：

The code is as follows （ Example ）：

from torch.utils.tensorboard import SummaryWriter
#  Record the number of workouts 
total_train_step = 0
#  Record the number of tests 
total_test_step = 0
#  Number of training rounds 
epoch = 100
accuracy_bass = 0

train_data_size = len(train_datasets)
test_data_size = len(val_datasets)

#  add to tensorboard
writer = SummaryWriter("./logs")

for i in range(epoch):
    print("------- The first  {}  Round of training begins -------".format(i+1))

    #  The training steps begin 
    vgg16_true.train()
    for data in train_data_loader:
        imgs, targets = data
        imgs = imgs.cuda()
        targets = targets.cuda()
        outputs = vgg16_true(imgs)
        loss = loss_fn(outputs, targets)

        #  Optimizer optimization model 
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        total_train_step = total_train_step + 1
        if total_train_step % 100 == 0:
            print(" Training times ：{}, Loss: {}".format(total_train_step, loss.item()))
            writer.add_scalar("train_loss", loss.item(), total_train_step)

    #  The test step begins 
    vgg16_true.eval()
    total_test_loss = 0
    total_accuracy = 0
    with torch.no_grad():
        for data in val_data_loader:
            imgs, targets = data
            imgs = imgs.cuda()
            targets = targets.cuda()
            outputs = vgg16_true(imgs)
            loss = loss_fn(outputs, targets)
            total_test_loss = total_test_loss + loss.item()
            accuracy = (outputs.argmax(1) == targets).sum()
            total_accuracy = total_accuracy + accuracy

    print(" On the overall test set Loss: {}".format(total_test_loss))
    total = total_accuracy/test_data_size
    print(" Accuracy on the overall test set : {}".format(total))
    writer.add_scalar("test_loss", total_test_loss, total_test_step)
    writer.add_scalar("test_accuracy", total, total_test_step)
    total_test_step = total_test_step + 1

    torch.save(vgg16_true, "./model_zoo/vgg16_true_{}.pth".format(i))
    
    
    print(" Model saved ")

writer.close()

---

Tips ： The correct rate here is to record the correct number on the test set , Not divided by total number of test sets ：

6、 ... and 、 Picture prediction classification ：

Put the picture to be predicted into docs Under the document , Start here when you need to re predict the picture

The code is as follows （ Example ）：

#  Put the picture to be predicted into docs Under the document , Start here when you need to re predict the picture 
import torchvision
from PIL import Image
import os

imge_name = "58780369.jpg"  #  The name of the picture to be predicted 

root_dir = "./docs"
imge_path = os.path.join(root_dir, imge_name)
image_0 = Image.open(imge_path)
print(image_0)

imge_transforms = torchvision.transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize((.5, .5, .5), (.5, .5, .5))
])

image = imge_transforms(image_0)
print(image.shape)

image = torch.reshape(image, (-1, 3, 224, 224))
model = vgg16_true
model = torch.load("./model_zoo/vgg16_true_45.pth", map_location=torch.device('cpu'))

model.eval()
with torch.no_grad():
    output = model(image)
print(output.argmax(1))

---

summary

nothing