Milkfish Freshness Classification Using Convolutional Neural Networks Based on Resnet50 Architecture

. Milkfish (Chanos chanos) had become the main commodity in three major cities in Indonesia, contributed at least 77 thousand tons of aquaculture production in 2021. The quality of fish is determined based on the level of freshness carried out in the sorting process, the sorting process is generally done by evaluating physical characteristics of the fish. However, this method is still considered less efficient and economical because the ability to classify the freshness level of fish can vary for each individual. In this study, by utilizing deep learning, a classification method for milkfish freshness level classification with ResNet50 architecture is proposed, the proposed method is purposed to overcome the previously stated problems, thus creating an efficient and economical system. By creating an efficient system, milkfish sorting process can be carried out quicker and more accurately. Using personal dataset divided into four different classes, the proposed method produces excellent result.


Introduction
Milkfish is considered as one of Indonesia's main food commodities, this is shown based on data provided by Badan Pusat Statistik Indonesia, in 2021 Indonesia had produced a total of 14.648.360tons of farmed fish from multiple kind [1], at the same year, milkfish had become one of five fisheries production with counting at least 77 thousand tons [2], while on the other hand Indonesia had sold at least 2.527.632tons of marine fish (from various sources) at the fish auction site [3].Based on the provided data this conclude that milkfish plays important role and has significant part in contribution to Indonesia's food commodity, therefore this resource needs to be utilized as well as possible.A fish quality and economical value can be determined by its freshness, sensory evaluation is one of the methods of assessing fish freshness by observing the fish's characteristics such as appearance, color, Hanifa, Ramadhan, Husna, Widiyono, et al [8] 2023 CNN algorithm with MobileNetV2 architecture Testing accuracy of 97%, 94%, and 93% for tuna, milkfish, and mackerel respectively.Based on the provided literature study, it is possible to utilize Convolution Neural Network (CNN) and other machine learning algorithm such as SVM (Support Vector Machine) for classifying fish freshness level, it is also considered that CNN can be used for integrated feature extraction and classification in one system [7], therefore it is also can be concluded that the use of CNN for classification purpose is still relevant to this date.This research is build based on existing methods and algorithm, yet this research tried to improve of what prior works has obtained, for example compared to [6] this research classifies fish freshness into four classes instead of two classes, and compared to [7] this research focused on ResNet50 architecture while using private dataset with larger quantity image instead of using combination of transfer learning method and smaller quantity image dataset.
This research specifically aims to create milkfish freshness classification system using CNN algorithm with ResNet50 architecture and Adam optimizer.The classification is divided into four class consisting of fresh, less fresh, starting to rot, and rotten class, dataset used in this research is private dataset consisting of 1803 images of milkfish taken in 16 hours of time.The purpose of the proposed model of milkfish freshness classification is to overcome challenges in milkfish freshness classification process such as inefficiency due to misclassification or human error, and uneconomical factors due to excessive man-power needed to perform large-scale work load.It is also aimed that by eliminating these challenges, this research can help various individuals in working with milkfish freshness classification (for example, milkfish seller who needs to classify the fishes based on its freshness in a largescale unit or capacity, or a human individual who doesn't have the ability to classify or determine fresh milkfish, will be helped by this system in determining fresh milkfish.)

Methodology 2.1 Dataset
Dataset used in this research is a private dataset containing 1804 images of milkfish taken in 16 hours of time, the images were taken with Canon 800D DSLR camera with the resolution of 1980 x 1980 pixels.Each milkfish was taken fourteen times within interval of one hour and total duration of 16 hours, while the milkfish itself was purchased randomly from several different fish seller in Sayung Market, Demak Regency, Indonesia.Currently this dataset is not available to the public and considered to be the author's private archive.The sample images taken from the dataset is shown in the following figure.

Figure 1. Dataset Sample from Each Class
During the 1 st through 4 th hour the milkfish is classified into fresh class, during the 5 th through 8 th hour it is classified into less fresh class, during the 9 th through 12 th hour it is classified into starting to rot class, and lastly during the 13 th through 16 th hour it is classified into rotten class.This is based on the changes of physical appearance of the fish as the time goes by, there are sign of rotting such as the eye membrane that is turning murky, the body turning into pale color, and bleeding mouth.

Convolution Neural Network (CNN)
CNN is a more advanced development of Artificial Neural Network that consisted of neural network that carries out the weight, bias, and activation function, furthermore, most common layers in CNN is convolution, pooling, and fully connected [11].In the convolution layer, features of the image is extracted by utilizing specific Kernel, then activation process is performed by mapping negative value into 0 value and preserving positive value in order to achieve fast and effective training process.In pooling layer, feature map reduction is performed resulting in reduced parameter that needs to be learn by the network.Lastly the fully connected layer is performed resulting in vector with K dimension where K is the number of classes that can be predicted [11], [12].

CNN Layer with ResNet50 Architecture
There are several architectures in CNN algorithm in the case of computer vision field, one of them is ResNet50 (Residual Network), this architecture relies on large dataset for its training process with various category of class [13].In this research, ResNet50 architecture is used because it's considered effective to work with large dataset with various class in it.Previous work done in [7]

Research Workflow
In this research, tool used is Matlab R2020a to develop and running the code for this research, the steps in this research are shown in figure 3 and explained as follows,

Image Preprocessing
In order to improve the quality of the image it is needed to perform image preprocessing, enhancement used in this step is firstly applying Wiener Filter with the purpose of filter existing signal noises using spectral properties from the existing signal and noises, by considering both of them as a stochastic process with linear property [14].The use of Wiener Filter is because some of the images in the dataset appears to be blurry due to external factors such as the camera shaking while taking the pictures, by applying Wiener Filter it appears to reduce the blur thus enhancing the image quality.
Second enhancement is image adjustment function from Matlab to fix the brightness and contrast of the image.The use of image adjustment is because the background used has almost the same colors of the milkfish body color appearance, in order to distinguish the background and the object (milkfish) image adjustment is performed, it appears that the milkfish's body after the preprocessing appears to be darker than the background thus making the milkfish more distinctive than the background.The enhancement result is shown in figure 4.

GLCM Feature Extraction
GLCM or Gray Level Co-occurrence Matrix is a statistic method used to obtain texture feature by obtaining the correlation value of two pixel in a specific range and angle [15].The extracted features in this research are contrast, energy, homogeneity, and correlation as described in the following equation 1, 2, 3, and 4, the result of GLCM feature extraction from sample dataset is provided in Table 3. Contrast is a parameter that measure the contrast value of a pixel with other adjacent pixels [16].Energy is a feature that represent the number of square elements in a GLCM matrix through a homogenous area to inhomogeneous area [16].Homogeneity refers to a similarity between pixel where the value of GLCM matrix of a homogenous picture is 1 [16].Correlation is used to measure linear dependency of a grayscale color in an image [16].
The variable p (i,j) represents the value of the matrix elements in a row (i) and column (j); μi,μj represents the average value of the elements in the matrix row and column; σi σj represents the standard deviation value in the matrix row and column [17].In this research, GLCM feature extraction is used to obtain GLCM features from each class in order to know the differences of GLCM features value of each class, for example based on table 3 it is shown that the contrast and energy value of sample from fresh class is significantly lower than sample from rotten class, while on the other hand the homogeneity value of sample from fresh class is significantly higher than sample from rotten class, while for the correlation value it appears that the value is decreasing from fresh all the way to rotten class.Based on the obtained value of GLCM features, it can be concluded that the image from each class had different characteristics, therefore the dataset in this research can be used in milkfish freshness classification.Furthermore, based on the research workflow in figure 3, although GLCM features extraction in this research doesn't directly contribute to the classification process, it contributed indirectly by indicating that each class had different and variative value of GLCM features.

Training Data and Confusion Matrix
In this research, dataset used contained 1803 images of milkfish classified into four classes being fresh, less fresh, starting to rot, and rotten class, furthermore, the training data portion for the training process is 80%.In the training process, there are three optimizers used being Adam, SGDM (Stochastic Gradient Descent with Momentum), and RMSProp (Root Mean Square Propagation).Adam optimizer is considered to work well even with small resources thus considered effective and can replace traditional stochastic gradient algorithm [18].In this research, three of the optimizers is compared with at least three times training run in order to find the best optimizer to be used, additional training options used are epoch number of 16, mini batch size of 32, validation frequency of 30, and false verbose value.The training result is shown in table 4 as well as figure 5 shows the accuracy and loss graphics for the last training run of Adam optimizer.

Conclusion
In this research a classification of milkfish freshness level was done using Convolution Neural Network (CNN).In this research a dataset containing 1803 milkfish images taken within 16 hours of time and divided into four classes being fresh, less fresh, starting to rot, and rotten class.In this research the CNN architecture used is ResNet50 with image size of 250 x 250 x 3 and utilizing Adam optimizer with 16 epochs, moreover image preprocessing technique such as applying Wiener Filter and image adjustment is also performed in order to enhance the quality of the images.The result obtained for training accuracy is 99,35% of accuracy score from at least three training runs.The result obtained for testing accuracy is 100% of accuracy score from at least three testing runs, with the model being able to correctly classified images into its class.With the conclusions obtained from this research, it is hope that this research can overcome challenges in milkfish freshness classification process such as inefficiency and uneconomical factors It is also hoped that by eliminating these challenges, this research can help various individuals in working with milkfish freshness classification.
research, however the dataset used is relatively small (234 images with three classes) compared to this research (1803 images with four classes).Further explanation of ResNet50 architecture used in this research is explained in the following figure and table.

Table 3 .
Feature Extraction Result from Sample Dataset

Table 4 .
Training Run Result

No. Optimizer 1 st Training 2 nd Training 3 rd Training
Accuracy and Loss Graphic milkfish freshness based on CNN algorithm therefore this research can be used as a proof of concept.Furthermore, although this research is focused and limited in classifying milkfish freshness, it's still possible for the further development of the proposed method to be used into similar cases for example Tilapia freshness classification, or Seabass freshness classification etc.