Você está na página 1de 6


2, MARCH 2015


Vein Visualization Using a Smart Phone

With Multispectral Wiener Estimation
for Point-of-Care Applications
Jae Hee Song, Choye Kim, and Yangmo Yoo

AbstractEffective vein visualization is clinically important for

various point-of-care applications, such as needle insertion. It can
be achieved by utilizing ultrasound imaging or by applying infrared laser excitation and monitoring its absorption. However,
while these approaches can be used for vein visualization, they
are not suitable for point-of-care applications because of their
cost, time, and accessibility. In this paper, a new vein visualization method based on multispectral Wiener estimation is proposed
and its real-time implementation on a smart phone is presented. In
the proposed method, a conventional RGB camera on a commercial smart phone (i.e., Galaxy Note 2, Samsung Electronics Inc.,
Suwon, Korea) is used to acquire reflectance information from
veins. Wiener estimation is then applied to extract the multispectral information from the veins. To evaluate the performance of
the proposed method, an experiment was conducted using a color
calibration chart (ColorChecker Classic, X-rite, Grand Rapids,
MI, USA) and an average root-mean-square error of 12.0% was
obtained. In addition, an in vivo subcutaneous vein imaging experiment was performed to explore the clinical performance of the
smart phone-based Wiener estimation. From the in vivo experiment, the veins at various sites were successfully localized using
the reconstructed multispectral images and these results were confirmed by ultrasound B-mode and color Doppler images. These results indicate that the presented multispectral Wiener estimation
method can be used for visualizing veins using a commercial smart
phone for point-of-care applications (e.g., vein puncture guidance).
Index TermsColor image, multispectral image, RGB image,
smart phone, smart phone camera, vein visualization, wiener estimation.

Manuscript received August 29, 2013; revised December 2, 2013 and January 22, 2014; accepted March 18, 2014. Date of publication March 21, 2014;
date of current version March 2, 2015. This work was supported in part by
the Ministry of Science, ICT & Future Planning, Korea, under the Convergence
Information Technology Research Center support program (NIPA-2013-H040113-1007) supervised by the National IT Industry Promotion Agency, the Sogang
University Research Grant of 2013 (SRF-201314003.01), and the Leading Foreign Research Institute Recruitment Program through the National Research
Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future
Planning (MSIP) (2013K1A4A3055268).
J. H. Song is with the Medical Solutions Institute, Sogang Institute of
Advanced Technology, Sogang University, Seoul 121-742, Korea (e-mail:
C. Kim and Y. Yoo are with the Department of Electronic Engineering, Sogang University, Seoul 121-742, Korea (e-mail: choye@sogang.ac.kr;
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JBHI.2014.2313145

FFECTIVE vein visualization is clinically important for
various point-of-care applications, such as needle insertion [1], [2]. Needle insertion has been widely used for blood
sample acquisition and intravenous infusion, but successful insertion rates are not high, especially for obese patients and
children [3]. In addition, insertion failure may lead to several
adverse problems, such as blood clotting, allergic reactions and
bone and nerve damage [4]. As such, ultrasound imaging, which
visualizes the location of veins at various depths, has been utilized to improve the success rate of vascular accessing [5]. Due
to its superior guidance performance [6], ultrasound imaging
substantially decreases the rate of vascular access failure, particularly when used by less experienced clinicians [7]. However, ultrasound imaging may not be optimal for point-of-care
applications because of its high cost, time required and low
For subcutaneous veins, infrared vein visualization can also
be used as a guidance tool. Infrared vein visualization illuminates and detects infrared lights in order to differentiate veins
from other soft tissues based on their spectroscopic optical absorption coefficients [1]. Using this technique, clinicians can
achieve a higher success rate for vascular accessing [8], [9].
However, this infrared imaging guidance method suffers from
the high cost of the equipment required and the safety concerns
associated with the usage of lasers.
In this paper, a new vein visualization method based on multispectral Wiener estimation is proposed and its real-time implementation on a smart phone is presented. In the proposed
method, the RGB color image is acquired via the built-in camera on a commercial smart phone. The reflectance spectrum
information for the vein visualization is then achieved by utilizing a multispectral Wiener estimation method. In addition, post
image processing algorithms, such as histogram stretching and
average/median filtering, are applied to improve image contrast
while reducing noise. The performance of the developed system
was evaluated using color calibration chart experiments. Moreover, in vivo imaging experiments were conducted to confirm
the feasibility of the proposed vein visualization method by implementing it on a commercial smart phone (Galaxy Note 2,
Samsung Electronics Inc., Suwon, Korea) as an Android
This paper is organized as follows. In Section II, the method
used to estimate the reflectance spectrum information from
RGB images is described. In Sections III and IV, the results

2168-2194 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.



Fig. 1. Schematic diagram illustrating the reflectance spectrum estimation

process. Photographs of each instrument used in the experiments such as the
color calibration chart, white reference tile, lighting booth, smart phone, and
spectrometer are displayed in the diagram.

Fig. 2. Vein visualization images obtained by the proposed method and their
histograms (a) before and (b) after post image processing. After histogram
stretching, image contrast improved considerably. The yellow boxes in the
middle of the images indicate the ROI used in the postprocessing.

from the performance evaluation and the in vivo imaging experiments are presented, respectively. In Section V, the advantages
and possible applications and limitations of the developed system are discussed, and finally in Section VI, we present the

flectance spectrum est . It can be obtained by

A. Reflectance Spectrum Estimation From a Color Image
Using Wiener Estimation
Fig. 1 illustrates the entire procedure for estimating the reflectance spectrum from the obtained RGB (red, green, and blue)
data. In the proposed method, the Wiener estimation method,
known as one of the simplest low- to high-dimension estimators,
which also boasts good performance, was applied [10][12]. As
shown in Fig. 1, in Wiener estimation, an RGB pixel value
from the built-in smart phone camera vi (i = R, G, B) can be
represented by

vi = fi () E () S () () d, i = R, G, B (1)
where fi () is the filter response for each red, green, and blue
wavelengths, E () is the spectrum of the light source, S () is
the camera sensitivity, and () is the targets real reflectance
spectrum [10], [11]. By utilizing matrix forms, (1) can be rewritten as
vi = Fit E S


where t indicates the matrix transposition. The estimated reflectance spectrum est can be obtained from (2) if the inverse
matrix H = G1 is as follows:
est = H .


The estimation matrix H can be calculated by minimizing

| est | where  denotes the ensemble average. H is the
error between real reflectance spectrum and estimated re-

H = r v 1


where r v is the correlation matrix between the camera output

v and the reflectance spectrum , and v v is the autocorrelation
matrix of v. It must be noted that and v can be obtained
using a spectrometer and the smart phone camera utilized in the
experiments, respectively.
The substitution of the obtained estimation matrix H using
(4) for (3) leads to

h2,B R
2 h2,R
. .

. = .
. G .
. .

k 1
k 1,R
k 1,G
k 1,B
hk ,R
hk ,G
hk ,B
From (5), the reflectance spectrum rest = [1 2
k 1 k ]t can be estimated from the camera output v =
[R G B]t . The subscript k is an index representing specific
wavelengths which are to be estimated. The wavelength values
used in this paper are listed in Table II.
To estimate the spectral intensity of the kth wavelength, the
vector [hk ,R hk ,G hk ,B ] composed of Wiener estimator obtained by (4) is multiplied by the camera input vector [R G B]t .
Of the estimated wavelengths, 620 (620 nm) was selected and
its intensity was normalized so as to set the maximum value
to 255 (i.e., gray scale) for vein visualization. Note that the
light source, camera sensitivity, and filter responses are already
included in the estimation matrix H. As such, the reflectance
spectrum of any RGB images can be obtained using the predetermined estimation matrix if RGB images are taken under the
same circumstances, which is critical for real-time performance.
B. Postprocessing for Multispectral Images From Wiener
Use as shown in the left panel of Fig. 2(a), the low contrast
of a multispectral image from Wiener estimation may not allow


for the distinct visual observation of subcutaneous veins. Thus,

a proper post image processing method for enhanced contrast
must be employed. To ensure that only skin and blood signals
are contained in the reconstructed multispectral image, it is
necessary to choose the appropriate region for extracting input
signals. For this purpose, the region of interest (ROI) is displayed
in the middle of the reconstructed multispectral image so the
user may conveniently focus on the target region (e.g., palm or
forearm) in ROI. The size of the ROI is determined by dividing
the screen horizontally and vertically into five equal parts based
on the data acquired and the user settings.
The right panel of Fig. 2(a) shows the histogram obtained
from the ROI in the left panel. As shown in the image, signals are
gathered within a narrow region, leading to low image contrast.
To enhance image contrast and to control image brightness,
a histogram stretching algorithm is applied to the estimated
multispectral image as follows:

Pin minin
(maxout minout ) + minout (6)
maxin minin

where Pout and Pin are the pixel values after and before histogram stretching, respectively, and maxin and minin are the
maximum and minimum values in the ROI before histogram
stretching, respectively. Since the intensity of the input signal can vary depending on image acquisition conditions and
surrounding environmental factors, such as light source, background light, and illuminating angle between the camera and
the target, both maxin and minin are determined on a frame-byframe basis using the input signals within the ROI. In addition, a
margin of 40% is given to minin to reduce the dark areas caused
by shadows and weak illumination. maxin and minin which
represent the maximum and minimum values after histogram
stretching are set to be 180 and 0, respectively.
With histogram stretching, as shown in Fig. 2(b), the distribution of the histogram is adjusted to improve image contrast. This
results in better visualization of blood vessels. Since noise can
be also emphasized by histogram stretching, 33 2-D average
and 55 median filters are sequentially applied to the histogram
stretched image.
A. Color Calibration Chart Experiments
The color calibration chart experiments were conducted to obtain the estimation matrix G1 and to evaluate the performance
of the procedure. As shown in Fig. 1, a color calibration chart
(ColorChecker Classic, X-rite, Grand Rapids, MI, USA) consists of various color samples for color adjustment. For increased
measurement accuracy, all experiments were conducted in a D65
illumination (i.e., standard daylight) environment created using
a lighting booth (SpectraLight III, X-rite, Grand Rapids, MI,
USA). To calculate the estimation matrix H [see Fig. 1 and (4)],
the camera output v and the real reflectance spectrum were
measured using the built-in smart phone camera (Galaxy Note
2, Samsung Electronics Inc., Suwon, Korea) and the spectrometer (Avaspect ULS2048 16, Avantes Inc., Broomfield, CO,
USA), respectively.


In order to access the real reflectance spectrum , the spectrum of the light source was first measured using a white reference tile (WS-2 Reference Tiles, Avantes Inc.), which reflected
98% of light within 3501800 nm in wavelength. Each sample
color spectrum in the color calibration chart was calculated by
dividing the measured sample color spectra by the light source
spectrum. The camera output v was measured by taking a photograph of the color calibration chart. In the performance evaluation, values were measured five separate times and then averaged
To evaluate the obtained H, the estimated spectra were compared to the measured spectra from the spectrometer. For quantitative analysis, the root-mean-square error was calculated by

i=1 (xm sr,i xest,i )
where N was the number of samples, and xm sr,i and xest,i
denoted ith measured and estimated values, respectively.
B. In Vivo Imaging Experiments Using the Developed Vein
Visualization System
The proposed reflectance spectrum estimation method was
implemented on the smart phone as an Android application.
Using the developed system, in vivo imaging experiments were
conducted to demonstrate the clinical feasibility of the application. The eclipse SDK 2.72 was used to develop the Android
application. The built-in flash light was programmed to be kept
on continuously so that a sufficiently bright light would illuminate the targets. The screen was split in half in order to show
reference and vein images simultaneously. In the reference image placed on the left panel, the ROI, indicated by a yellow box,
was developed to allow users to place targets within the ROI.
With the developed system and the obtained estimation matrix
H, in vivo RGB and vein images were obtained from various
locations around the human body including the palm, wrist,
back of the hand, and inside elbow, which are the most common
locations for subcutaneous vein visualization applications. In
addition, to confirm that the patterns shown in the vein images
are real blood vessels, the locations of the veins were compared
to those in the ultrasound color Doppler images obtained from a
commercial ultrasound imaging system (V10, Samsung Medison, Seoul, Korea), which is a sensitive imaging method for
detecting blood vessels.
A. Color Calibration Chart Experiments
Table I and Fig. 3 show the averaged camera output v and the
measured spectra values gathered from the spectrometer from
the color calibration chart, respectively. With v and , the estimation matrix H can be calculated by using (4) and the obtained
values of H are summarized in Table II. To evaluate the performance of Wiener estimation, the estimated spectra in Fig. 3 were
compared to the measured spectra from the spectrometer. In addition, for quantitative evaluation, the RMSE values between
the measured and estimated spectra were calculated by using




Fig. 4. Photographs of the operation of the developed mobile device-based

multispectral imaging system for vein visualization system: (a) color calibration
chart imaging and (b) in vivo imaging of the back of the hand.

Fig. 5. In vivo RGB and multispectral vein images obtained from elbows,
palms, wrists and backs of the hand of multiple volunteers. It can be seen that
veins are more clearly visualized on the multispectral images.
Fig. 3. Reflectance spectra obtained from each sample color in the color
calibration chart using the spectrometer.

(7) and summarized in Fig. 3. As listed in Fig. 3, the maximum

and minimum RMSEs were 34.4% and 4.3%, respectively, and
the average RMSE value was 12.0%.
B. In Vivo Imaging Experiments
Fig. 4 is a photograph of the vein visualization system operating on the Galaxy Note 2 smart phone as an Android application.
The color calibration chart and the images of the back of the
hand were produced using the application. As shown in Fig. 4,
the RGB and multispectral images were respectively displayed
on the left and right panels in the split screen. In the middle of

the RGB image (left panel), a yellow ROI box is displayed. All
in vivo images shown in the results section were obtained using
the developed system.
Fig. 5 shows the RGB and relevant vein images of the palm,
wrist, back of the hand and inside elbow from multiple volunteers. It can be observed that veins were more clearly distinguishable in the multispectral images than in the original RGB
image. Even in several cases, veins invisible in the RGB image
were successfully revealed in the multispectral images. Consistently successful results were also seen in the palm, wrist, back
of the hand, and inside elbow as shown in Fig. 5.
To verify the accuracy of the proposed method, the locations
of the veins shown in the multispectral images were compared
to those shown in the ultrasound color Doppler images. Since
multispectral images exhibit a coronal plane whereas ultrasound
images show a transverse plane, three positions, in which ultrasound color Doppler images were to be obtained, were selected
and marked on skin in the vein image. Fig. 6 shows the multispectral vein image indicated by three marks. Fig. 6 also shows
the corresponding ultrasound color Doppler images obtained
from a commercial ultrasound imaging system (V10, Samsung
Medison, Seoul, Korea). As shown in Fig. 6(a) and (b), a single
thick vein (first mark), a single thin vein (second mark) and two
thinner veins (third mark) which were separated from the upper
vein were also shown in the ultrasound color Doppler images.
As such, we believe that patterns shown in the multispectral
images were revealing the positions of real blood vessels. For
further verification, ultrasound B-mode images were obtained to



Fig. 8. In vivo RGB and multispectral forearm vein images from two high
BMI volunteers when an interior light was turned (a) and (b) on and (c) and
(d) off, respectively. The BMI values of the two volunteers were 29.4 and 28.1,

Fig. 6. Multispectral vein image and three ultrasound color flow images obtained at three marked positions. By comparing locations of blood vessels on
multispectral and ultrasound color Doppler images, it is concluded that the dark
pattern in the multispectral image visualized real blood vessels.

Fig. 7. Consecutive ultrasound B-mode images obtained at the second moving

toward the third marked positions (clockwise direction) in Fig. 6. It can be seen
that another blood vessel is splitting from the initial single blood vessel. Blood
vessels were indicated by yellow arrows.

observe the bifurcation pattern of blood vessels from the second

mark to the third mark. Fig. 7 shows the obtained ultrasound
B-mode images along the blood vessels. After assessing the
ultrasound B-mode images in Fig. 7 in a clockwise direction,
the starting point of the vessels bifurcation and the distance
between the separated vessels were found to be increasing. This
coincides with the results shown in Fig. 6(a). Note that the difference in vessel depths between the ultrasound color Doppler and
the B-mode images were caused by the coupling space between
the ultrasound transducer and the skin.
Most modern smart phones have a built-in RGB camera and
they typically have enough computational power to incorporate the presented reflectance spectrum estimation technique.
As such, multispectral images were obtained by running the developed Android application based on Wiener estimation. It has
successfully demonstrated that subcutaneous veins can be suf-

ficiently visualized. Because of its easy and simple application

and its outstanding portability, we expect the proposed vein visualization system to be of great use in clinical and commercial
One conceivable application for the developed system is to
provide clearer guidance images for vein puncture, which is
necessary for effectively performing blood sample acquisition
and intravenous infusion. However, it has been well known that
vein puncture does not have a high rate of success [3]. Infants
and neonates in particular suffer from vein puncture failure due
to their small vessels and limited puncture sites. Thus, guidance
images based on infrared vein visualization systems have been
used to allow for higher success rates of vascular access [8], [9].
To investigate the utility of the proposed system, in vivo experiments on 10 volunteers were conducted. The average body
mass index (BMI) of the volunteers was 23.67 and the variance
was 3.59. Therefore, volunteers were within the normal range
group [13] and it can be stated that they have typical properties.
Using the developed system, subcutaneous veins were successfully visualized in most cases. Experiments on people with high
BMIs were also conducted to mimic the results in overweight
patients. As shown in Fig. 8(a) and (b), it can be seen that
the image contrast of the forearm veins improved substantially
even for those with BMIs of 29.4 and 28.1. Fig. 8(c) and (d)
shows additional vein images obtained from the same high BMI
volunteers with interior light turned off. Due to the flashlight
capabilities of the smart phone, the veins could be still distinguished on multispectral images when the lights were turned
off. As such, we expected that the developed system may be utilized in locations outside of the hospital such as in ambulances
or accident areas.
Compared to conventional infrared vein imaging systems, the
developed system boasts a smaller size, lighter weight, and lower
cost since any extra light sources and photo-detectors outside
of the built-in smart phone camera are not necessary. However,
although the developed system can show veins in most cases
thus far, further investigation must be pursued in consideration
of various factors such as skin color, obesity, vein depth, age,



and gender before the developed system may be applied as a

clinical tool for vein puncture navigation.
Furthermore, although target veins were successfully detected
using the reconstructed multispectral vein images, there exist
several limitations in the presented study. Due to the real-time
performance requirement, a simple model, e.g., Wiener, was
adopted instead of more accurate methods based on complex
modeling, e.g., KubelkaMunk theory [14][16]. As shown in
Figs. 7 and 8, while the locations of the target veins were confirmed using the ultrasound B-mode and color Doppler images,
a more in-depth performance evaluation of the proposed Wiener
estimation method, in terms of detection accuracy, would be required as a further study.
In this paper, a multispectral imaging method for subcutaneous vein visualization and its implementation on a commercial
smart phone are presented. To extract reflectance spectra from
the RGB images, we utilized the Wiener estimation in the proposed method. The performance and accuracy of the developed
system were evaluated through the color calibration chart and in
vivo location comparison experiments. From these experiments,
it was confirmed that the patterns shown in multispectral image
were successful in revealing subcutaneous veins. Therefore, the
developed system has the potential for vein puncture guidance
and smart phone authentication although further investigations
are required to verify the feasibility of its applications.

[10] P. Stigell, K. Miyata, and M. Hauta-Kasari, Wiener estimation method

in estimating of spectral reflectance from RGB images, Pattern Recogn.
Image Anal., vol. 17, pp. 233242, 2007.
[11] N. Tsumura, H. Sato, T. Hasegawa, H. Haneishi, and Y. Miyake, Limitation of color samples for spectral estimation from sensor responses in fine
art painting, Opt. Rev., vol. 6, pp. 5761, 1999.
[12] T. Yoshida, H. Inoue, S. Usui, H. Satodate, N. Fukami, and S. E. Kudo,
Narrow-band imaging system with magnifying endoscopy for superficial
esophageal lesions, Gastrointest. Endosc., vol. 59, pp. 288295, 2004.
[13] Global database on body mass index, BMI classification, WHO, 2006.
[14] E. Claridge, S. Cotton, P. Hall, and M. Moncrieff, From colour to tissue histology: Physics-based interpretation of images of pigmented skin
lesions, Med. Image Anal., vol. 7, pp. 489502, 2003.
[15] C. Tang, A. W. K. Kong, and N. Craft, Uncovering vein patterns from
color skin images for forensic analysis, in Proc. IEEE Conf. Computer
Vision Pattern Recog., 2011, pp. 665672.
[16] H. Zhang, C. Tang, A. W. K. Kong, and N. Craft, Matching vein patterns from color images for forensic investigation, in Proc. IEEE Conf.
Biometrics: Theory, Appl. Syst., 2012, pp. 7784.

Jae Hee Song received the B.S. degree in computer

science, the M.S. degree in electronic engineering,
and the Ph.D. degree in interdisciplinary program of
integrated biotechnology from the Sogang University,
Seoul, Korea, in 2004, 2006, and 2012, respectively.
He is currently a Postdoctoral Research Fellow of
Medical Solutions Institute, Sogang Institute of Advanced Technology, Seoul, Korea. His research interests include high-intensity focused ultrasound, elastography, portable ultrasound imaging system and
photoacoustic imaging and its clinical applications.

[1] V. P. Zharov, S. Ferguson, J. F. Eidt, P. C. Howard, L. M. Fink, and
M. Waner, Infrared imaging of subcutaneous veins, Lasers Surg. Med.,
vol. 34, pp. 5661, 2004.
[2] H. D. Zeman, G. lovhoiden, C. Vrancken, and R. K. Danish, Prototype
vein contrast enhance, Opt. Eng., vol. 44, 086401, pp. 19, 2005.
[3] T. Maecken and T. Grau, Ultrasound imaging in vascular access, Crit.
Care Med., vol. 35, pp. S178S185, 2007.
[4] M. Mansoor, S. N. Sravani, S. Z. Naqvi, I. Badshah, and M. Saleem,
Real-time low cost infrared vein imaging system, in Proc. Int. Conf.
Signal Process., Image Process. Pattern Recog., Coimbatore, India, 2013,
pp. 14.
[5] The American Institute of Ultrasound in Medicine. (Aug. 20, 2013). Use
of ultrasound to guide vascular access procedures. [Online]. Available:
[6] T. J. Milling, J. Rose, W. M. Briggs, R. Birkhahn, T J. Gaeta, J. J. Bove,
and L. A. Melniker, Randomized, controlled clinical trial of point-ofcare limited ultrasonography assistance of central venous cannulation:
The third sonography outcomes assessment program (SOAP-3) trial,
Crit. Care. Med., vol. 33, pp. 17641769, 2005.
[7] A. G. Randolph, D. J. Cook, C. A. Gonzales, and C. G. Pribble, Ultrasound guidance for placement of central venous catheters: a meta-analysis
of the literature, Crit. Care. Med., vol. 24, pp. 20532058, 1996.
[8] H. A. Hess, A biomedical device to improve pediatric vascular access
success, Pediatr. Nurs., vol. 36, pp. 259263, 2010.
[9] M. J. Kim, J. M. Park, N. Rhee, S. M. Je, S. H. Hong, Y. M. Lee,
S. P. Chung, and S. H. Kim, Efficacy of vein viewer in pediatric peripheral intravenous access: A randomized controlled trial, Eur. J. Pediatr.,
vol. 171, pp. 11211125, 2012.

Choye Kim received the B.S. and M.S. degrees in electronic engineering from
the Sogang University, Seoul, Korea, in 2004 and 2012, respectively.
She is currently a Researcher at LG Electronics.

Yangmo Yoo received the B.S. and M.S. degrees

in electronics engineering from Sogang University,
Seoul, Korea, in 1999 and 2001, respectively, and the
Ph.D. degree in bioengineering from the University
of Washington, Seattle, WA, USA, in 2007.
From 2007 to 2009, he was a Systems Design Engineer with Philips Healthcare, Bothell, WA, USA.
He is currently an Associate Professor of Electronics
Engineering and the Interdisciplinary Program of Integrated Biotechnology at Sogang University, Seoul,
Korea. His current research interests include medical
ultrasound imaging and its clinical applications in diagnostics and therapy.