Escolar Documentos
Profissional Documentos
Cultura Documentos
A R T I C LE I N FO A B S T R A C T
Keywords: White button mushrooms, one of the most commonly and widely consumed mushrooms worldwide, require
Online sorting grading before entering the market. The diameter of the pileus is an important factor that affects the selling
White button mushroom price. The aim of this study is to develop an automatic sorting system for fresh white button mushrooms based on
Image processing image processing according to the pileus diameter. First, an automatic sorting hardware system is designed that
Automatic control
consists of a conveying mechanism, an image acquisition system, a control module and an actuator. Second, we
present an image algorithm based on the watershed method, Canny operator, OR operation, and closed op-
eration to determine the pileus diameter of fresh white button mushrooms. The algorithm eliminates the in-
fluence of the shadow and petiole on the image. Third, a precise control strategy is introduced based on the
conveyor speed, distance between the trigger and the flap piece, trigger time and algorithm processing time.
Finally, we present analysis and control software based on OpenCV 2.4.10 and Visual Studio 2010. In the ex-
perimental portion, a prototype system was tested to validate its applicability and reliability by the grading
performance and results. The experimental results showed that the average maximum grading speed was
102.41 mushrooms/minute, the accuracy of grading was 97.42%, the damage rate was 0.05%, and the un-
detected rate was 0.96%. Relative to manual grading, the grading speed was improved by 38.86%, and the
accuracy was improved by 6.84%. The system can operate stably and continuously. For long-duration grading in
particular, the advantage of the automatic sorting system is apparent because it eliminates labourer fatigue. The
system achieves online automatic sorting and grading of fresh white button mushrooms with minimal de-
struction.
⁎
Corresponding author.
E-mail address: 34941269@qq.com (W. Feng).
https://doi.org/10.1016/j.compag.2018.06.022
Received 12 January 2018; Received in revised form 14 April 2018; Accepted 10 June 2018
0168-1699/ © 2018 Published by Elsevier B.V.
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
417
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
White button Pressurize to set Adjust light seven different threshold segmentation algorithms (shown in Fig. 3).
mushroom Conveyor start value source controller As shown in Fig. 3, only one of the methods could not effectively
remove the shadow and petiole while maintaining the pileus. The factor
Trigger that affected the maximum diameter was the petiole.
Accordingly, an image algorithm was designed to determine the
1st Electromagnetic 1st pneumatic
valve act actuator move 1st Container diameter of the white button mushroom based on the watershed
Line scan method, Canny operator, OR operation and closed operation. First, the
first watershed algorithm (FWS) combining the global threshold seg-
2nd Electromagnetic 2nd pneumatic
Acquire, analyze valve act actuator move 2nd Container mentation method and maximum entropy threshold segmentation
and process method was used to remove the shadow from the image. Then, the
second watershed algorithm (SWS) based on the Canny operator, the
No Electromagnetic No pneumatic OR operator and the closed operation was used to remove the petiole
Digital output valve act actuator move 3rd Container
noise. Finally, the minimum enclosing rectangle method was used to
Fig. 2. Workflow. determine the diameter of the white button mushroom.
(a) Original image; (b) BGM; (c) Iteration threshold method; (d) OTSU method; (e) Self-adaption
threshold method; (f) Top and bottom threshold method; (g) Manual set threshold value 71; (h)
MEM
Fig. 3. Comparison of seven image-processing effects.
418
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
i=0
Pn ⎝ Pn ⎠ (4) 3.3. Minimum enclosing rectangle method
L−1
Pi P
H (B ) = − ∑ ln ⎛ i ⎞
⎜ ⎟ Extract the silhouette for watershed2, and remove the larger or
1−Pn ⎝ 1−Pn ⎠ (5)
i=t+1 smaller silhouette to obtain the second silhouette picture contours2. Use
In formula (4) and (5), H(T) and H(B) are the entropies related to the Graham algorithm to scan the convex hull of the white button
the probability and density for the T and B distributions, respectively. mushroom, and use the even interval rotation method to identify the
Define the entropy function φ(t) as follows: enclosing rectangle with the minimum area to obtain the final picture
result2.
φ (t ) = H (T ) + H (B ) (6)
The optimal threshold value T2 is the grey degree grade t when the (1) Find an enclosing rectangle M1 of silhouette picture contours2 and
value of φ(t) is maximised. ROI_image should be converted into a compute its area S1. Let SMIN = S1 and MMIN = M1 and set the po-
greyscale image before binarisation processing. The grey degree value sition angle α = β = 0.
of a pixel larger T2 is assigned to 128 (grey) and less than or equal to T2 (2) Rotate the silhouette clockwise with an angle θ, identify its en-
is assigned to 0 (black). Accordingly, ROI_image is converted into a closing rectangle M2, and compute its area S2. The new position
binary image with a 0 or 128 grey level and designated image2. angle is α = α + θ.
(3) Compare S2 with SMIN. If S2 < SMIN, let SMIN = S2, β = α, and
3.1.4. First watershed algorithm MMIN = M2.
Use the foreground of image1 as the foreground of image marker1, ( 90
)
(4) Iterative N N = θ times for (2) and (3). Rectangle MMIN is the
which has a grey level marked as 255 (white). Use the background of required minimum enclosing rectangle. Rotate the silhouette and
image2 as the background of image marker1, which has a grey level enclosing rectangle anticlockwise by an angle β.
marked as 128 (grey). The grey levels of the other pixels in image
marker1 are marked as 0 (black). Set marker1 as the marked image of the The longer side of the minimum enclosing rectangle is the required
watershed to identify the certain foreground region, certain background diameter of the white button mushroom. Variations in the mushroom
region and uncertain region. Then, use the watershed algorithm to position affect the shorter side of the minimum enclosing rectangle with
process ROI_image to obtain the resulting image watershed1 of the FWS. no change to the longer side due to the circular symmetry of the pileus.
419
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
T1 T2
Binarization Binarization
image1 image2
Combination
marker1
Canny operator
FSD
contours1 watershed1
OR operation
result1
420
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
shown in Table 1. The software system is based on OpenCV 2.4.10 and Visual Studio
The container farthest away from the photoelectric sensor was de- 2010 to acquire, analyse and process images and output control in-
signed for the grade of mushroom with the highest quantity; these structions to control the valve using the USB-4761 module. The algo-
mushrooms enter the container immediately and do not require elec- rithm flow diagram of the image processing is shown in Fig. 6.
tromagnetic valve action. The container farther from the photoelectric The control algorithm flow diagram is shown in Fig. 7. We set DO as
sensor is designed for the grade of mushroom with a lower quantity; a control variable byte. Grade-1 corresponds to the grade of mushroom
these mushrooms are sorted using the second actuator (4 in Fig. 1). This with the highest quantity and two valves without actuation. Grade-2
actuator is farther from the photoelectric sensor, which results in a corresponds to the grade of mushroom with a lower quantity and the
longer delay for its valve action. The container nearest to the photo- second actuator (4 in Fig. 1) controlled by D0 and D1 bits of DO. Grade-
electric sensor is for the grade of mushroom with the lowest quantity to 3 corresponds to the grade of mushroom with the lowest quantity and
reduce the frequency of the action of the first actuator (3 in Fig. 1). This the first actuator (3 in Fig. 1) controlled by D2 and D3 bits of DO.
421
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
Table 2
Static experimental data and analysis.
No. 1 2 3 4 5 6 7 8 9 10
AV (mm) 49.00 59.80 37.50 49.80 44.00 45.80 54.60 35.40 46.70 44.40
EV (mm) 49.20 60.45 37.00 50.20 44.50 45.05 54.70 35.65 46.74 44.10
Error (%) 0.41 1.08 1.35 0.80 1.12 1.66 0.18 0.70 0.09 0.68
No. 11 12 13 14 15 16 17 18 19 20
AV (mm) 47.50 35.40 46.00 45.10 46.30 37.00 46.00 49.40 40.00 47.60
EV (mm) 48.00 35.80 46.10 45.50 47.80 37.55 46.25 49.85 39.55 47.00
Error (%) 1.04 1.12 0.22 0.88 3.14 1.46 0.54 0.90 1.14 1.28
No. 21 22 23 24 25 26 27 28 29 30
AV (mm) 40.00 37.00 40.30 45.80 36.80 54.00 48.00 43.00 47.00 35.70
EV (mm) 39.60 36.65 40.90 45.98 36.30 54.30 47.35 42.55 47.65 34.75
Error (%) 1.01 0.95 1.47 0.39 1.38 0.55 1.37 1.06 1.36 2.73
No. 31 32 33 34 35 36 37 38 39 40
AV (mm) 34.90 45.40 42.00 35.80 38.10 39.50 42.00 43.10 42.10 56.00
EV (mm) 34.65 45.80 42.15 36.20 38.55 39.85 42.15 43.45 42.40 55.70
Error (%) 0.72 0.87 0.36 1.10 1.17 0.88 0.36 0.81 0.71 0.54
probability of the t-test for the AV and EV values was 0.9405, which is
larger than 0.05. The results of the t-test show that there was no sig-
nificant difference between the AV and EV values. The above analysis
shows that the image-processing algorithm is appropriate for static
measurement.
422
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
Table 3
Original experiment data.
No. Automatic grading Manual grading
1 63 System 48 47 0 63
Manual 48 47 0 52 42 1
By calliper 47 48 0 48 47 0
Damaged 0 0 0 0 0 0
2 61 System 48 54 0 78
Manual 49 55 0 68 35 1
By calliper 49 55 0 48 56 0
Damaged 0 0 0 0 0 0
3 66 System 50 61 2 91
Manual 53 61 1 40 73 2
By calliper 48 67 0 49 66 0
Damaged 0 0 0 0 0 0
4 67 System 55 58 1 93
Manual 61 54 1 31 82 0
By calliper 58 57 1 55 58 0
Damaged 0 0 0 0 2 0
5 61 System 58 46 0 89
Manual 58 46 0 43 61 0
By calliper 57 47 0 58 46 0
Damaged 0 0 0 0 0 0
6 60 System 48 60 1 84
Manual 49 60 1 32 77 0
By calliper 51 58 1 36 73 0
Damaged 0 0 0 0 0 0
8 60 System 71 30 1 95
Manual 71 30 1 45 57 0
By calliper 65 36 1 46 55 1
Damaged 0 0 0 0 0 0
9 61 System 79 23 0 101
Manual 79 23 0 52 50 0
By calliper 76 26 0 59 43 0
Damaged 0 0 0 0 0 0
10 60 System 77 26 0 86
Manual 77 26 0 54 49 0
By calliper 72 31 0 56 47 0
Damaged 0 0 0 0 0 0
Note:
(1) The first line of each record is the automatic statistic quantity using the system.
(2) The second line of each record was counted by a person to compute the undetected rate. “Manual” in the rows refers to counting by a person. “Manual” in the
columns refers to grading by a person.
(3) The third line of each record is the quantity measured using the calliper to compute the grading accuracy.
Ha1 + Ha2 + Ha3 efficiency, accuracy and damaged rate were similar to the method for
Ea = × 100%
Ma (10) automatic classification. The analysis of the data in Table 3 is shown in
Table 4.
Max {|Ha1−Ca1|, |Ha2−Ca2|, |Ha3−Ca3 |} ⎞ Table 4 shows that the average automatic grading speed was 102.41
Aa = ⎛1−⎜ × 100%
⎟
⎝ Ha1 + Ha2 + Ha3 ⎠ (11) mushrooms per minute, the average automatic accuracy was 97.42, the
average automatic damage rate was 0.05% and the average automatic
Da1 + Da2 + Da3
Da = × 100% undetected rate was 0.96%. Relative to manual classification, the effi-
Ha1 + Ha2 + Ha3 (12) ciency improved by 38.86%, and the accuracy improved by 6.84%. For
classification over a long duration, the advantage of the automatic
|(Ga1 + Ga2 + Ga3)−(Ha1 + Ha2 + Ha3 )|
La = × 100% classification is clear. In actual production, the daily manual classifi-
Ha1 + Ha2 + Ha3 (13)
cation (for eight hours) capability is from 60 kg to 100 kg depending on
The average value was computed using ten records. For manual the sorting individual’s working efficiency. For the automatic system,
classification, undetected cases did not exist, and the computations for the average efficiency is 102.41 mushrooms per minute, equivalent to
423
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
Table 4 developed by Chen and Ting (2004) for dried shiitake mushrooms. The
Data analysis. present research improved the grading efficiency relative to the grading
No. Type Efficiency Accuracy (%) Damaged Undetected cycle of 4.8 s in that research. An important factor is the efficiency.
(quantity/ rate (%) rate (%) Although the daily automatic efficiency improved five times over
minute) manual efficiency, the manual feeding mode limited the commercial
application. Therefore, we plan to address the feeding mechanism in
1 Automatic 90.48 98.95% 0.00% 0.00%
Manual 90.48 93.68% 0.00%
future studies.
2 Automatic 102.30 100.00% 0.00% 1.92%
Manual 80.00 78.85% 0.00% 7. Conclusions
3 Automatic 104.55 94.78% 0.00% 1.74%
Manual 75.82 92.17% 0.00%
In this study, an automatic sorting system for fresh white button
4 Automatic 103.88 97.41% 0.00% 1.72%
Manual 74.84 79.31% 0.00% mushrooms was designed, realised and tested. One important devel-
5 Automatic 102.30 99.04% 0.00% 0.00% opment was the image-processing model and the algorithm based on
Manual 70.11 85.58% 0.00% the watershed method, Canny operator, OR operation and closed op-
6 Automatic 110.00 98.18% 0.00% 0.91%
eration to determine the pileus diameter of fresh white button mush-
Manual 78.57 96.36% 1.82%
7 Automatic 103.64 99.52% 0.48% 3.35%
rooms. Another development was a precise control strategy based on
Manual 70.85 97.61% 0.00% the conveyor speed, distance between the trigger and flap piece, trigger
8 Automatic 102.00 94.12% 0.00% 0.00% time and algorithm processing time. The hardware consisted of me-
Manual 64.42 97.06% 0.00% chanical parts, including a conveyor, roller, cleaner guide rod, and
9 Automatic 102.00 97.06% 0.00% 0.00%
slider. Additional hardware included electrical and electronic parts,
Manual 60.59 93.14% 0.00%
10 Automatic 103.00 95.15% 0.00% 0.00% such as a line-scan camera, motors, photoelectric sensor, light source
Manual 71.86 98.06% 0.00% and its controller, electromagnetic valve, USB relay control module,
11 Automatic 98.86 99.04% 0.39% 1.35% IPC, and image capture card. The software function consisted of the
Manual 65.97 97.30% 0.96% camera configuration, parameter configuration, image acquisition,
Average Automatic 102.41 97.42% 0.05% 0.96%
value Manual 73.75 91.18% 0.18%
image processing, output control, data display and statistical analysis.
Our goal was to design a system that improves the commercial pro-
duction capacity. The proposed system can process more than 600 kg of
approximately 1.29 kg per minute is approximately 619.31 kg for eight mushrooms in eight working hours. The system can be used for other
hours. Thus, the daily automatic efficiency is improved five times over fruits and vegetables classified by their size.
the manual efficiency.
Acknowledgements
6.3. Discussion This research was funded by the Department of Science &
Technology of Shandong Province, China, project no. 2016GNC11008,
The high efficiency is clearly shown through the experiment de- and by the Shandong Academy of Agricultural Sciences, China, project
scribed above. The maximum action frequency of the valve is five times no. CXGC2017B04. The authors declare that they have no conflict of
per second. In theory, one valve can sort 300 mushrooms per minute, interest.
and the efficiency is doubled for two valves. In our experiment, the
efficiency was 102.41 mushrooms per minute and was a function of the Appendix A. Supplementary material
mushroom feeding mode and the image-processing speed. A vibration
disc was designed to guide the mushrooms individually onto the con- Supplementary data associated with this article can be found, in the
veyor along the guide plate. However, the fresh mushrooms were often online version, at https://doi.org/10.1016/j.compag.2018.06.022.
blocked, overlapped, or damaged due to their high moisture content.
The manual feeding mode was adopted in the dynamic test. The image- References
processing speed can be improved by adding cameras.
The accuracy depends on the image-processing algorithm and the Abbasgholipour, M., Omid, M., Keyhani, A., Mohtasebi, S.S., 2011. Color image seg-
control algorithm. We found in the static test that the average accuracy mentation with genetic algorithm in a raisin sorting system based on machine vision
in variable conditions. Expert Syst. Appl. 38 (4), 3671–3678.
for the image-processing algorithm was 99.01%. When there were Abera, K.A., Manahiloh, K.N., Nejad, M.M., 2017. The effectiveness of global thresholding
brown spots or scars leading to large connection areas on the image, the techniques in segmenting two-phase porous media. Constr. Build. Mater. 142,
accuracy of the algorithm was affected. However, the occurrence of 256–267.
Ambrose, A., Kandpal, L.M., Kim, M.S., Lee, W.H., Cho, B.K., 2016. High speed mea-
brown spots or scars is unusual for fresh white button mushrooms. surement of corn seed viability using hyperspectral imaging. Inf. Phys. Technol. 75,
Therefore, the accuracy is predominantly affected by the control algo- 173–179.
rithm, which can be improved using variable air pressure with the size Arakeri, M.P., Lakshmana, 2016. Computer vision based fruit grading system for quality
evaluation of tomato in agriculture industry. Procedia Comput. Sci. 79, 426–433.
of mushroom and an exact variable time delay with the IPC processing Cappelli, A., 1984. Fungi Europaei: Agaricus (in Italian). Giovanna Biella, Saronno, Italy,
speed. pp. 123–125.
Any damage caused by the system on the product arose through Chen, H.H., Ting, C.H., 2004. The development of a machine vision system for shiitake
grading. J. Food Qual. 27 (5), 352–365.
improper air pressure and the lack of buffering between the container
Duan, F., Wang, Y., Liu, H., Li, Y., 2007. A machine vision inspector for beer bottle. Eng.
and the actuator. The damage rate was reduced by adjusting the air Appl. Artif. Intell. 20 (7), 1013–1021.
pressure. The mushroom was directly pushed into the container without Elmasry, G., Cubero, S., Moltó, E., Blasco, J., 2012. In-line sorting of irregular potatoes by
a buffering slideway in the hardware design, which was a shortcoming using automated computer-based machine vision system. J. Food Eng. 112 (1–2),
60–68.
of this design. Any damage done by manual classification arose through Jarimopas, B., Jaisin, N., 2008. An experimental machine vision system for sorting sweet
unsuitable sorting force. tamarind. J. Food Eng. 89 (3), 291–297.
For certain conveyor speeds, the distance between two adjacent Jian, C., Gao, J., Ao, Y., 2017. Automatic surface defect detection for mobile phone screen
glass based on machine vision. Appl. Soft Comput. 52, 348–358.
image-processed mushrooms had a minimum value. If the distance was Jiang, W., Zhou, H., Shen, Y., Liu, B., Fu, Z., 2015. Image segmentation with pulse-cou-
less than the minimum value, the mushroom was undetected. pled neural network and canny operators. Comput. Electron. Agric. 46 (C), 528–538.
A similar research study was conducted on the grading system Kurtulmuş, F., Kavdir, İ., 2014. Detecting corn tassels using computer vision and support
424
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425
vector machines. Expert Syst. Appl. 41 (16), 7390–7397. emerging technologies: automatic mushroom harvester development. J. Agric. Eng.
Mattila, P., Suonpää, K., Piironen, V., 2000. Functional properties of edible mushrooms. J. Res. 78 (1), 15–23.
Nutr. 16 (7–8), 694–696. Sun, R.Y., Zhao, C.Y., 2015. Restoration of space object images by using a maximum
Medina-Carnicer, R., Madrid-Cuevas, F.J., Carmona-Poyato, A., 2005. Evaluation of entropy method. Chin. Astron. Astrophy 39 (1), 89–99.
global thresholding techniques in non-contextual edge detection. Pattern Recognit. Tang, Z., Huang, L., Zhang, X., Lao, H., 2016. Robust image hashing based on color vector
Lett. 26 (10), 1423–1434. angle and canny operator. AEU – AEU Int. J. Electron. Commun. 70 (6), 833–841.
Mizushima, A., Lu, R., 2013. An image segmentation method for apple sorting and Tillett, R.D., Batchelor, B.G., 1991. An algorithm for locating mushrooms in a growing
grading using support vector machine and Otsu's method. Comput. Electron. Agric. bed. Comput. Electron. Agric. 6 (3), 191–200.
94, 29–37. Vı́Zhányó, T., Felföldi, J., 2000. Enhancing colour differences in images of diseased
Nashat, S., Abdullah, A., Abdullah, M.Z., 2014. Machine vision for crack inspection of mushrooms. Comput. Electron. Agric. 26 (2), 187–198.
biscuits featuring pyramid detection scheme. J. Food Eng. 120, 233–247. Vooren, J.G.V.D., Polder, G., van der Heijden, G.W.A.M., 1991. Application of image
NY/T 1790–2009, Grades and specification of Agaricus bisporus. analysis for variety testing of mushroom. Euphytica 57 (3), 245–250.
Pasban, A., Mohebbi, M., Pourazarang, H., Varidi, M., 2014. Effects of endemic hydro- Yang, Y., Zha, Z., Gao, M., He, Z., 2016. A robust vision inspection system for detecting
colloids and xanthan gum on foaming properties of white button mushroom puree surface defects of film capacitors. Signal Process. 124, 54–62.
studied by cluster analysis: a comparative study. J. Taibah Univ. Sci. 8 (1), 31–38. Yorulmaz, O., Pearson, T., Çetin, A., 2012. Detection of fungal damaged popcorn using
Pourdarbani, R., Ghassemzadeh, H.R., Seyedarabi, H., Nahandi, F.Z., Vahed, M.M., 2015. image property covariance features. Comput. Electron. Agric. 84 (2), 47–52.
Study on an automatic sorting system for date fruits. J. Saudi Soc. Agric. Sci. 14 (1), Yu, G.H., Zhao, Y., Li, G., Shi, H., 2005. Algorithm for locating individual mushroom and
83–90. description of its contour using machine vision. Trans. CSAE 21 (6), 101–104.
Reed, J.N., Tillett, R.D., 1994. Initial experiments in robotic mushroom harvesting. Zhou, Y.S., Li, Q., Li, H.Y., Wang, R.B., He, W.B., 1995. Application of computer vision in
Mechatron 4 (3), 265–279. mushroom picking robot. Trans. CSAE 11 (4), 27–32.
Reed, J.N., Miles, S.J., Butler, J., Baldwin, M., Noble, R., 2001. Ae-automation and
425