Marker Encoded Fringe Projection Profilometry for

Efficient 3D Model Acquisition

Budianto,1 Daniel P.K. Lun, 1,* Tai-Chiu Hsung2
1

Centre for Signal Processing, Department of Electronic and Information Engineering, the Hong Kong Polytechnic University,
Hung Hom, Kowloon, Hong Kong
2 The Discipline of Oral and Maxillofacial Surgery, Faculty of Dentistry, The University of Hong Kong, Hong Kong
*Corresponding author: enpklun@polyu.edu.hk
Received Month X, XXXX; revised Month X, XXXX; accepted Month X,
XXXX; posted Month X, XXXX (Doc. ID XXXXX); published Month X, XXXX

This paper presents a novel marker encoded fringe projection profilometry (FPP) scheme for efficient 3-dimensional (3D) model
acquisition. Traditional FPP schemes can introduce large error to the reconstructed 3D model when the target object has an abruptly
changing height profile. For the proposed scheme, markers are encoded in the projected fringe pattern to resolve the ambiguities in
the fringe images due to that problem. Using the analytic complex wavelet transform, the marker cue information can be extracted
from the fringe image, and is used to restore the order of the fringes. A series of simulations and experiments have been carried out to
verify the proposed scheme. They show that the proposed method can greatly improve the accuracy over the traditional FPP schemes
when reconstructing the 3D model of objects with abruptly changing height profile. Since the scheme works directly in our recently
proposed complex wavelet FPP framework, it enjoys the same properties that it can be used in real time applications for color
objects. 2014 Optical Society of America
OCIS codes: (150.6910) Three-dimensional sensing; (100.2650) Fringe analysis; (100.5070) Phase retrieval; (100.5088) Phase
unwrapping; (110.6880) Three-dimensional image acquisition; (100.7410) Wavelet.
http://dx.doi/org/10.1364/AO.99.099999

1. Introduction
Due to the relatively low cost and high efficiency, fringe
projection profilometry (FPP) has been employed for 3dimensional (3D) model acquisition in various applications
such as quality control system [1], real world 3D scene
reconstruction [2,3], and medical tomography [4]. For FPP,
fringe patterns are first projected onto the target object. Due
to the height profile of the object, the fringe patterns as
shown on the object surface are deformed as compared to the
projected ones. Thus by measuring the amount of
deformation, the objects 3D model can be readily
reconstructed.
Traditional FPP methods can be further divided into two
categories: temporal multiplexing [1,513] and frequency
multiplexing [4,1418]. Temporal multiplexing methods
make use of two or more fringe patterns projecting onto the
target object sequentially. It allows accurate reconstruction of
the 3D model of an object but can have severe distortion if
the object is moving during the image capturing process.
Hence they are not suitable to dynamic 3D model
reconstruction applications.
The frequency multiplexing methods on the other hand

only needs to project one fringe pattern to the target object in

order to reconstruct its 3D model. One of the important
frequency multiplexing FPP methods is the Fourier transform
profilometry (FTP) [4,14]. For FTP, a high frequency blackand-white stripe pattern is projected onto the object and the
deformed fringe pattern is captured by a camera [14]. The
deformation due to the object height profile can be evaluated
by measuring the phase shift of the stripes in the captured
fringe image. Similar to many phase detection problems
[1,511], the phase analysis process of the FTP method
gives only the wrapped phase data that has to be unwrapped
to obtain the true phase shift information usable for 3D
model reconstruction [14]. Traditional phase unwrapping
algorithms estimate the true phase shift by integrating the
wrapped phase differences [1922]. However, due to the
various artifacts in the fringe images, some of the wrapped
phase data can be missing. The same can also happen when
the target object has a sharp change in height. Directly
carrying out the unwrapping process with such erroneous
data will lead to severe distortion to the final reconstructed
3D model. Recent quality-guided phase unwrapping
methods, such as [21], will also fail particularly when the

Fig. 1. FPP setup using the FTP scheme in crossed optical axes geometry
[14] .

fringes have phase discontinuity along the object boundary

with respect to the reference background. It is the case when
the object does not have a direct contact with the reference
background or the object itself has a curvature (such as a
bowl) such that some parts of it cannot be seen from the
angle of the camera.
In fact, the above problem stems from Itohs analysis [19]
that assumes the phase can be estimated by integrating the
wrapped phase difference. The estimated true phase shift will
thus have much error if some of the phase data are missing.
To solve this problem, recent approaches try to embed period
order information in the projected fringe patterns. By period
order, we refer to the number of 2 jumps in the phase angle
that is hidden in the wrapped phase data. If the period order
is known, phase unwrapping can be achieved even when
some of the wrapped phase data are missing. Traditional
approaches embed phase order information to the fringe
patterns using, for instance, multiple cameras [2] [12], multiwavelength fringe pattern [23], fringe pattern with additional
information, such as colors [24], speckles [25] [26], and
markers [5,27,28], multiple frequencies [29], multiple
patterns [30] [31], and gray coded fringe patterns [1], [9],
etc. However, the approaches in [2], [12], and [23] require
additional hardware systems. The performance of the
approach in [24,28,29] can be seriously affected by the color
pattern of the object, whereas the approaches in [5] and [27]
can only be applied to a simple scene (e.g., a single object).
The approaches in [1,9], [30], and [31] require additional
fringe projections. They are not suitable to dynamic
applications similar to the temporal multiplexing methods
mentioned above. In addition to the above approaches, it was
recently reported in [32] that a dual frequency scheme can
be used to embed the period order information to the fringe
patterns in a phase measurement profilometry process (PMPDF). In such method, high frequency fringe patterns are
generated to encode the objects height information as the
traditional phase shifting profilometry (PSP) approaches [6].
In addition to that, low frequency signals are added to the
fringe patterns to encode the period order information.
Similar to other PSP methods, it requires at least 5 fringe
patterns to obtain both the wrapped phase data and the period
order information. Hence it is also not suitable to dynamic
applications. Besides, our experiment shows that the method
is highly sensitive to the quality of the fringe images. It will
be exemplified in Section V.

In this paper, we propose a new marker encoding and

detection algorithm using the Dual Tree Complex Wavelet
Transform (DTCWT) [33]. The objective of the proposed
algorithm is to provide an efficient method for estimating the
phase order information of the fringes in order to assist the
phase unwrapping process when working in the FTP based
FPP environment. We have shown in [18,34] that the
DTCWT is an effective tool for denoising and removing the
bias of fringe images [35,36]. For the proposed algorithm,
we add to the original DTCWT FPP framework the markers
encoding and detection algorithm that allows the period order
information to be embedded into the projected fringe pattern
and extracted from the captured fringe image to assist in the
phase unwrapping process [37]. As different from the
traditional marker encoding schemes, the proposed approach
is applicable to objects with color texture. Furthermore, this
approach is more localizable compared to the approaches
using single strip marker or single point marker. It can
provide the period order information not only at the center
but along the fringe pattern. Thus it is applicable to the scene
that contains several objects with sudden jumps in height.
Unlike the approaches that use multiple frame patterns, the
proposed scheme is applicable to dynamic applications
because it requires only a single fringe pattern for the entire
operation. Finally, the algorithm is not sensitive to the quality
of the fringe image. As different from the PMP-DF, the
period order information can be detected accurately with
noisy fringe images or images with abnormal brightness.
With the period order information, we can reconstruct the 3D
model of the object even when some of the fringe data are
missing due to the artifacts of the fringe image or the
irregularity of the object shape.
This paper is organized as follows. In Section II, the basic
principle of the FTP using the DTCWT is introduced. We
then describe in Section III and IV, respectively, the
proposed marker encoding and detection algorithm. The
simulation and experiment results are discussed in Section V.
Finally, the conclusions of this work are presented in Section
VI.
2. Background
A. Principle of the FTP Method
In this section, we first outline the basic concept of the
conventional FTP method [14]. The typical setup of the FTP
method is shown in Fig.1. In the figure, a projector at E p
projects a fringe pattern and a camera captures the deformed
fringe pattern as shown on the object surface at point Ec .
The point E p and Ec are placed at a distance l0 from a
reference plane R and they are at a distance of d 0 in the
direction parallel to the reference plane R. The deformed
fringe pattern captured by the camera at Ec can be modeled
mathematically as a sinusoidal function as follows [38]:

Fpp
Fpq
g
Fqp
Fqq

Hpp,
Vpp,
Dpp
Hpq,
Vpq,
Dpq

Hqp,
Vqp,
Dqp
Hqq,
Vqq,
Dqq

Level 1

Fhh
Fhg

Fgh
Fgg

Hhh,
Vhh,
Dhh
Hhg,
Vhg,
Dhg

Fhh

...

...

...

Fig. 3. The DTCWT FPP framework with the proposed period order
estimation algorithm

Hgh,
Vgh,
Dgh
Hgg,
Vgg,
Dgg

y ( x) y (0) x 1 y ( x)
m

...

where y ( x) y ( x) y ( x 1) and y ( x) . However,

(3) is valid only if y ( x) is known for all x. If for a particular

Level 2

point x such that y ( x ') is estimated with error or even

Fig. 2. Realization of the DTCWT using 4 DWT trees

g ( x, y ) a( x, y ) b( x, y )cos ( x, y ) n( x, y )

(1)

In (1), g ( x, y) represents the pixels, in x and y directions,

of the fringe image captured by the camera. a( x, y ) is the
bias caused by the surface illumination of the object; b( x, y)
is the local amplitude of the fringe; and n( x, y ) is the noise
generated
in
the
image
capture
process.
( x, y) 2 f0 x ( x, y) is the phase angle in which f 0 is the
carrier frequency, and ( x, y ) is the phase shift of the fringe.
( x, y ) has a close relationship with the object height profile
h( x, y ) as follows:

h ( x, y )

l0
( x, y )
2 f 0 d0

(2)

In (2), ( x, y ) ( x, y ) o ( x, y ) , where o ( x, y ) is the

( x, y ) when there is no object. It is assumed to be known in
the initial calibration process. Hence if ( x, y) is known,
( x, y ) and also ( x, y ) can be determined. Then the object
height profile and in turn the 3D model of the object can be
reconstructed.
B. The Need of Phase Unwrapping
However, any trigonometric methods that directly retrieve
( x, y) from g ( x, y) will end up with only the wrapped
version of ( x, y) , which is always in the range of to . It
is caused by the fact that, as shown in (1), all ( x, y )
separated by 2 will have the same g ( x, y ) . Let us denote
the wrapped version of ( x, y) be ( x, y) . A phase
unwrapping process is needed for retrieving ( x, y ) from
( x, y) . Many of the existing unwrapping algorithms [20]
are based on Itohs analysis [19], which suggests to integrate
the wrapped phase differences in order to calculate the
desired phase ( x, y ) . Denote y ( x) ( x, y) and

y ( x) ( x, y) . The phase unwrapping process based on

Itohs analysis can be described as follows:

(3)

missing, the unwrapped y ( x) will have error for all x x.

Such situation unfortunately is common in typical FPP setups
due to the artifacts of the captured fringe images or the
irregularity of the object shape.
C. Fringe Image Enhancement Based on the DTCWT
As mentioned above, the estimation of y ( x) is erroneous
in practice due to the artifacts such as noises and bias in the
fringe images. To reduce the artifacts, we have shown in
[18] [34] [39] an efficient image enhancement algorithm
based on the DTCWT [33]. As shown in Fig. 2, the 2DDTCWT is realized by four 2D discrete wavelet transform
(DWT) trees. The outputs of the transforms are combined to
generate wavelet coefficients of 6 orientations at different
resolutions [40]. Besides, due to the specially designed
wavelet functions, the transform is approximately analytic.
That is, it gives nearly zero negative frequency response.
Based on this special property, it is shown in [18] that
normal fringe images have a piecewise smooth magnitude
response in the DTCWT domain, which is much different
from that of noises. Besides, we show in [35] that we can
also detect and remove the bias in the fringe image in the
DTCWT domain. They lead to an improved FPP algorithm as
shown in the bottom 4 blocks of Fig. 3. In the figure, the
noisy fringe image is first transformed to the DTCWT
domain where the bias removal and denoising operations are
carried out [35]. The processed DTCWT coefficients are
then transformed back to the spatial domain. Traditional
phase unwrapping algorithm is then used to recover the
original phase information and the 3D model of the object
can be reconstructed. The above fringe image enhancement
method can effectively reduce the estimation error of y ( x) .
However in case that some of the y ( x) are missing due to
occlusion or other reasons, such enhancement method still
cannot avoid the severe distortion appeared in the final
reconstructed 3D model. In this paper, we propose an
improved FPP algorithm which also adopts the DTCWT
based fringe image enhancement method in [18]. In addition,
a period order estimation process is added (as shown in Fig.

Fig. 5. A fringe pattern with markers located at different phase angles of

the sinusoidal fringes.

angle y (k y ) M k y

Fig. 4. The original sinusoidal fringe projection pattern (for a particular

row) and the markers (top). Resulting fringe projection pattern with markers
embedded (bottom).

3) to facilitate the correct implementation of phase

unwrapping even when some of the phase information is
missing. Details are shown in Section III and IV.
3. Proposed Marker Coding for Consistent Phase
Unwrapping
In fact, the relationship between y ( x) and y ( x) can be
written as follows [19]:

y ( x) y ( x) k y ( x)2

(4)

computed even if some of the wrapped phase information is

missing. In this paper, we propose to embed a set of
structured markers into the projected fringe pattern to
facilitate the estimation of the period order k y ( x ) from the
fringe image. The marker encoded fringe projection pattern is
defined as follows:
pym ( x) py ( x) my ( x)
(5)
where p y ( x) is the original sinusoidal fringe projection
pattern and my ( x) is the marker signal added to the fringe
projection pattern. As illustrated in Fig. 4, the markers are
realized as a sequence of impulses (dash line) added to the
original sinusoidal fringe projection pattern (solid line) at
different phase angles. These phase angles are carefully
selected such that they encode the order number of the
sinusoidal period. Ideally, for every sinusoidal period in p y

, where

refers to a modulo b.

In our experiment, we choose To = 36 and Tm = 4. Hence, 9

unique markers can be inserted into 9 different sinusoidal
periods respectively. The set of markers will be repeated for
the next 9 sinusoidal periods. Such arrangement allows phase
unwrapping using (4) when up to 8 consecutive sinusoidal
fringe periods are missing due to whatever reasons. This
resolvability is sufficient in normal applications of the FPP.
To facilitate the detection of the markers, the mapping
function M(.) should be designed to maximize the difference
of y (k y ) between two neighboring markers. A natural choice
is as follows:

y ky M ky

Nm

ky

Nm

Nm 1
2

.
Nm

2
Nm

(6)

2
.
Nm N m

N 1
ky. m
2

where k y ( x) is an integer. It is the so-called period order that

determines the number of 2 jumps required to unwrap the
wrapped phase. If k y ( x) is known, y ( x) can always be

Nm

Here we assume Nm is an odd number. It is shown in

Appendix A that the mapping function in (6) ensures
neighboring markers will have a difference in y (k y ) with
value at least

N m 1 ,

which is about the maximum

Nm
possible value (i.e. ). An example of the marker encoded
fringe projection pattern is shown in Fig. 5. In the figure, the
thick black and white columns are the sinusoidal fringe
projection pattern. The markers are characterized by the
sharp black and white lines. As mentioned above, we select
Nm = 9 in our experiment. Following (6), 9 markers are
inserted into 9 consecutive sinusoidal periods at 9 different
phase angles 0,5 ,1 ,6 ,2 ,7 ,3 ,8 ,4 , where 2 / 9 ,
respectively. With this arrangement, any 2 neighboring
markers are separated by at least 4 . Such arrangement
maximizes the difference in y (k y ) of neighboring markers. It

with period order number k y , a marker should be added to it

will improve the performance of the later marker detection

process. Based on (6), we can also define my ( x) of (5) as

mapping function. It is however difficult to achieve in

practice since each sinusoidal period is represented by a
limited number of pixels of the fringe projection pattern. It is
not possible to have many different y (k ) . That is, there can

follows:

at phase angle y (k y ) M k y , where M(.) is a unique

only be a limited number of unique markers. Suppose that

every sinusoidal period is represented by To pixels of the
fringe projection pattern and every marker has a size of Tm
pixels. Then at most Nm To / Tm unique markers can be
made. Here we assume To is an integer multiple of Tm. In this
case, a marker will be added to a sinusoidal period at phase

my ( x) f k y x *
ky

(7)

where

k y N m 1
To . k y k y
1 x Tm .
2
f k y ( x)
Nm

otherwise
0

Nm

(8)

In (7), the symbol * denotes linear convolution and can be

any short support impulsive function. In our experiment, we
set to be the first derivative of an impulse function. Each
marker is represented by 4 pixels in the projected fringe
pattern as mentioned above.
4. Proposed Period Order Detection Algorithm
In this section, we discuss how the embedded markers and
the period order information are detected from the fringe
image captured by the camera. As mentioned in Section II,
the captured fringe image with markers embedded will be
processed based on a DTCWT FPP framework as shown in
Fig. 3. Then the wavelet coefficients of 6 orientations at
different levels will be generated. Recall that the markers are
signals of sharp changes in magnitude. They induce strong
wavelet coefficients particularly in the first few levels. On
the other hand, normal fringe pattern usually does not have
high frequency contents. Hence their wavelet coefficients can
often be found in higher levels of the wavelet transform. For
the proposed algorithm, we examine the first 2 levels of the
wavelet transform to detect the positions of the markers. Note
also that the markers are added row-wise to the fringe
pattern, they will not introduce wavelet coefficients of all 6
orientations. To be more specific, only the subbands of 45 o,
75o, 105o, and 135o will contain significant wavelet
coefficients of the markers.
Those wavelet coefficients will be sent to the Period Order
Estimation function block as shown in Fig. 3. Denote the
wavelet coefficients at level j and orientation subband m as
d ( j, m) . The marker cue information Q is first computed in

(a)
(b)
Fig. 6. Marker detection results: (a) using only the maxima of Q; (b) the
zoom in version which shows many falsely detected markers (circled).

the Period Order Estimation function block using the

following formulation,

2
j

Q j | d ( j , m) |
(9)
o
j i
m {45
, 75o ,

105o , 135o }

where d ( j, m) is the magnitude of the complex wavelet

coefficients. Parameter and are used to control the
contribution of wavelet coefficients to the marker cue
function. We empirically select = 1 and = 1, although our
experiments show that the final result is not sensitive to their
selection. In (9), j () is an interpolation function (e.g.,
bilinear interpolation) applied to the accumulation results of
each level such that they have the same size as the original
fringe image. Due to the shift invariance property of the
DTCWT [40], the singularities that characterize the markers
in the fringe image will generate strong coefficients at similar
positions at all levels in the DTCWT domain. The Gaussianlike magnitude response of the wavelet functions [40] will
also ensure that each marker will have only one maximum of
Q. Thus the position of the maxima in Q is strongly related to
the position of the markers.
However, the first two levels of the wavelet transform are
also swamped by the coefficients of noises. It means that the
maxima in Q can also be contributed by noises. Hence in
practical setting, detecting markers by using only the maxima
of Q will get false detection results, as illustrated in Fig. 6.
To identify the maxima of the markers, we threshold both the
magnitude and phase of the complex wavelet coefficients at
the positions where the maxima of Q are found. The
magnitude of the markers wavelet coefficients in general
should be much higher than that of noises. So a thresholding
operation to the magnitude of the wavelet coefficients can be
performed first to remove the maxima contributed by noises
of small magnitude. Given the wavelet coefficients d ( j, m)
at j = 1 and 2 and m = {45o, 75o, 105o, and 135o}, the
following operation is carried out:
d ( j, m)
if d ( j, m) x , y j ,m
x, y
d j, m x , y
(10)
0
otherwise

where d ( j, m) is the d ( j, m) at position {x, y} and

x, y

j ,m nj ,m 2log N j ,m
and nj ,m median

d( j,m ) / 0.6745

(11)

In (11), is the so-called universal threshold [41] used in

many wavelet denoising applications [4143]; N j ,m is the

(a)
(b)
Fig. 7. A comparison of local phase and local relative phase: (a) local phase;
(b) local relative phase.

number of coefficients at level j and orientation subband m.

n is the standard deviation of noise which is estimated
based on the robust statistics [44].
To further improve the detection of the maxima of the
markers, we consider the local relative phase of the complex
wavelet coefficients [4547]. By definition, the local relative

phase ( j, m) of the complex wavelet coefficients d ( j, m)

is given by [45] [47],

( j, m) d ( j, m) x , y d ( j, m) x 1, y

(12)

where d ( j, m) is the local phase angle at position {x, y}.

x, y
While the local phase of the complex wavelet is known to be
arbitrary irrespective to the structure of the image, there is a
strong relationship between the local relative phase and the
orientation of the edges in natural images [47]. Our
experiment shows that it also provides a good description of
the markers. Since all markers have the same structure, they
incur complex wavelet coefficients of similar local relative
phase. It however is not case for those of noises. An example
is shown in Fig. 7. As it is seen in Fig. 7(a), the local phases
(arrows) around the markers maxima have unpredictable
directions. However, the local relative phases (arrows) as
shown in Fig. 7(b) around the markers maxima have a
similar horizontal direction, while it is not the case for those
of noises. For the proposed algorithm, we first compute the
mean relative phase of the complex wavelet coefficients.
More specifically, we apply a 2D rectangular mask of 3x3
centered at every complex wavelet coefficient d ( j, m) .
Then the mean of the relative phase
index

from the set i for

i 1,..., n is computed by,

i 1

i 1

arctan cos i / sin i

(13)

where n is the number of maxima in a particular mask.

Finally, the following thresholding procedure with respect to
is carried out:

d ( j, m)
x, y
d j, m x , y
0

if ( j, m)

x, y

(14)

otherwise

where ( j, m)
is the mean relative phase at level j and
x, y
orientation subband m at position {x, y}; and is a very small
real number.
The thresholded wavelet coefficients are then used in (9)
for computing the marker information cue Q and in turn
detecting the position of the markers. The detection accuracy
is greatly improved by using the abovementioned
thresholding techniques based on the magnitude and relative
phase of the complex wavelet coefficients. An example of the
end result is shown in Fig. 8. It can be seen that almost all of
the maxima of noises are removed. The maxima retained
clearly show the positions of the markers in the fringe image.
When the maxima of the markers are identified, the next
step is to determine the period order ky of each marker by
identifying y of the marker from the fringe image (see (6)
for the relationship between ky and y ). To do so, we first use
the flood fill algorithm [21] to find the regions in the
wrapped phase map where the phase difference is bounded
by 2. Hence within the region, the period order should be

(a)
(b)
Fig. 8. (a) Marker detection results after using the proposed thresholding
method; (b) the zoom in version.

the same. Let us define Y j to be the set of all y in such

regions with j as the region index. An exhaustive search is
then performed to estimate the period order ky based on the
maxima detected in region j as follows:
1
i min
y i '
i'
N j yY j
(15)

k yY j M 1 i

where is defined in Section 3; Nj is the total number of

maxima that can be detected in region j; and y is the phase
angle of the maxima in row y, which is obtained directly by
inspecting the fringe image based on the maxima position
indicated in Q. However, due to the various artifacts in the
fringe image, the y obtained is always slightly different
from the true y . (15) thus helps to identify the correct i
based on y . Another problem when implementing (15) is
that the flood fill algorithm may accidentally include the
maxima from neighboring regions into the computation. It is
particularly the case when the fringe image is of low quality.
Recall that when embedding the markers, arrangement has
been made to maximize the difference in y of neighboring
markers. So in practice before implementing (15), we first
carry out a screening process to all y such that those having
large difference from the rest will be ignored. Once we get
the period order ky from (15), the phase unwrapping problem
can be solved using (4) and the 3D model of the object can
be readily reconstructed.
5. Simulation and Experiment
To evaluate the computational efficiency and accuracy of
the proposed algorithm, a simulation using a computer
generated fringe pattern was first carried out. Fig. 9(a) and
(b) show a computer generated cone object and the deformed
marker encoded fringe pattern, respectively, which were used
in the simulation. They serve as the ground truth for the
evaluation. To simulate the real working environment, white
Gaussian noise (variance 1.0) is added to the fringe pattern.
The simulation code is written in MATLAB running on a
personal computer at 3.4 GHz. The resolution of the testing

(a)
(b)
Fig. 9. The object used in the simulation. (a) A computer generated 3D cone
(ground truth); (b) the deformed fringe pattern

fringe image is 20482048 pixels.

We compare the proposed algorithm with the traditional
Window Fourier Filtering (WFF) method [48] [49] and the
DTCWT method without markers embedded [18] [35]. For
DTCWT, we use the filter coefficient proposed in [50]. The
length of the wavelet filters used is 12, which is typical for
wavelet filter. For both approaches, the phase unwrapping is
done by using the Goldstein algorithm [22]. All algorithms
are implemented in MATLAB. Table 1 shows the
comparison results in terms of the execution time and SNR at
different noise levels. As shown in the table, the proposed
algorithm is faster by approximately 10 times than the
WFF+Goldstein method with similar, if not better, SNR.
Compared to the DTCWT+Goldstein method (without
markers), the proposed algorithm gives a similar performance
both in the execution time and SNR. They show that the use
of markers does not introduce much burden to the
computation of the algorithm.
The real advantage of using markers is that it allows
correct phase unwrapping even when some of the phase
information is seriously corrupted or even missing. To
demonstrate it, we conducted a series of experiments using
real objects. More specifically, we implemented our
proposed algorithm with an FPP hardware setup that contains
a DLP projector and a digital SLR camera. The projector has
a 2000:1 contrast ratio with light output of 3300 ANSI
lumens and the camera has a 22.2 x 14.8mm CMOS sensor
and a 17-50mm lens. Both devices are connected to the
computer with a 3.4GHz CPU and 16GB RAM for image
processing. They are placed at a distance of 700mm-1200mm
from the object.
In our experiment, a marker encoded fringe pattern at the

(a)

(b)

(c)

(d)

(e)
(f)
Fig. 10. Comparison of the proposed algorithm and the traditional phase
unwrapping method. (a) texture image; (b) fringe pattern illumination;
(c) reconstructed 3D shape with texture using the proposed method;
(d) reconstructed 3D shape with height profile using the proposed method;
(e) reconstructed 3D shape with texture using the traditional
DTCWT+Goldstein; and (f) reconstructed 3D shape with height profile using
the traditional DTCWT+Goldstein

(a)

(b)

Table 1. Comparison between the proposed method, the conventional

DTCWT+Goldstein method, and the WFF+Goldstein in terms of execution time and SNR

Proposed

0.2
0.4
0.6
0.8
1

Time (s)
2.27
2.26
2.27
2.28
2.28

SNR
39.55
38.40
36.95
35.49
33.86

Conventional
DTCWT
Time (s)
2.13
2.12
2.12
2.13
2.12

SNR
39.37
38.22
36.79
35.35
33.99

WFF
Time (s)
33.39
33.14
33.11
33.53
33.42

SNR
43.12
38.13
34.84
32.32
30.29

(c)
Fig. 11. (a) Texture images (b) markers encoded fringe image; (c) one of the
PMP-DF fringe images.

(a)

(f)

(b)

(g)

(c)

(h)

(d)

(i)

(e)

(j)

Fig. 12. The first column is the comparison of the proposed algorithm
and the PMP-DF for images captured with ISO 100 and shutter speed
1/15s: (a) a texture image, (b) height profile by the proposed method, (c)
3D shape with texture by the proposed method, (d) height profile by the
PMP-DF, (e) 3D shape with texture by the PMP-DF.
The second column is the comparison of the proposed method and the
PMP-DF for images captured with the ISO 100 and shutter speed 1/30s:
(f) a texture image, (g) height profile by the proposed method, (h) 3D
shape with texture by the proposed method, (i) height profile by the PMPDF, (j) 3D shape with texture by the PMP-DF.

resolution of 12801024 is generated and projected onto the

target object. The fringe pattern consists of about 35
sinusoids in x-direction; each has a length of 36 pixels. A
marker is embedded to each sinusoid with 4 pixels width.
There are 9 unique markers and repeated in every 9
sinusoids.

(a)

(f)

(b)

(g)

(c)

(h)

(d)

(i)

(e)

(j)

Fig. 13. The first column is the comparison of the proposed algorithm
and the PMP-DF for images captured with ISO 1600 and shutter speed
1/80s: (a) a texture image, (b) height profile by the proposed method, (c)
3D shape with texture by the proposed method, (d) height profile by the
PMP-DF, (e) 3D shape with texture by the PMP-DF.
The second column is the comparison of the proposed method and the
PMP-DF for images captured with the ISO 1600 and shutter speed
1/125s: (f) a texture image, (g) height profile by the proposed method,
(h) 3D shape with texture by the proposed method, (i) height profile by
the PMP-DF, (j) 3D shape with texture by the PMP-DF.

In the first experiment, we compare the performance of the

proposed algorithm (using markers) with the conventional
DTCWT+Goldstein method (without markers) in the
situation that there are phase jumps in the fringe image. To
create such testing environment, a paper plane and two small
boxes of different height are used, as illustrated in Fig. 10a.

Basically no fringe patterns can be found on the edges of the

boxes, as shown in Fig. 10b. Thus phase jumps are
introduced to the fringe images. As shown in Fig. 10c-f, both
approaches can correctly reconstruct the paper plane since
they both use the DTCWT FPP framework as shown in Fig.
3. However, only the proposed method can make a correct
estimation of the height of the boxes. It is expected since the
period order information obtained from the markers allows us
to restore the unwrapped phase even when there are phase
jumps in the fringe image.
In the second experiment, we used the same objects as in
the first experiment but comparing the proposed algorithm
with the PMP-DF [32]. As it is mentioned in Section I, the
PMP-DF method uses a low frequency signal added to the
high frequency fringes for embedding period order
information. Thus it serves similar to the markers of the
proposed algorithm. As required by the PMP-DF, 6 frames of
image with phase shifted sinusoidal fringes are used for the
reconstruction of one 3D model. It is in contrast to the
proposed algorithm which requires only 1 fringe image due
to the use of the DTCWT FPP framework. Fig. 11 shows a
scene captured by the camera, the marker encoded fringe
image used in the proposed algorithm and one of the fringe
images used in the PMP-DF. Fig. 12 and Fig. 13 show the
results at different ISOs and shutter speeds when capturing
the fringe images. When the ISO value is high (e.g. ISO 1600
for our camera), noise will be introduced to the fringe
images. By changing the shutter speed, the brightness of the
fringe images will be changed. These experiments show the
robustness of both methods. It can be seen in the figures that
both approaches can perform satisfactorily in the cases of
ISO 100 and shutter speed 1/15s. When the image becomes
darker (shutter speed is changed to 1/30s), the height profile
reconstructed by the PMP-DF is seriously distorted. It is also
the case when the ISO value is changed to 1600. The PMPDF has problem in both cases (shutter speed 1/80s and
1/125s), while the proposed algorithm performs normally. It
can be observed that the PMP-DF is rather sensitive to the
quality of the fringe images. It is because if the image is too
bright or too dark, the detected low frequency signal will
have its magnitude decreased or may even be distorted.
Particularly when the image is noisy, the detected period
order information based on the low frequency signal becomes
very unreliable. On contrary, the detection of the markers is
not affected by the brightness of the fringe image.
In the final experiment, we used a free form object, a
human hand. Reconstructing the 3D model of free form
objects like human hands is very challenging because of the
abruptly changing surface and the discontinuity around the
edges. Nevertheless the proposed algorithm is able to
reconstruct the 3D model satisfactorily (see Fig. 14).

(a)

(b)

(c)

(d)

(e)

6. Conclusion
In this paper, a new marker encoding and detection
algorithm for the fringe projection profilometry (FPP) is
proposed. Based on our previously developed dual tree

(f)
Fig. 14. 3D model reconstruction of a human hand. (a) texture image; (b) the
fringe image; (c) wrapped phase of the hand; (d) the detected markers; (e) the
reconstructed 3D model with texture image; (f) the reconstructed 3D model.

complex wavelet (DTCWT) FPP framework, the proposed

system can reconstruct the 3D model of an object using only
one projection fringe image. The system can also handle the
bias and noise problem in the image. In this paper, a marker
encoding scheme is developed to embed the period order
information to facilitate phase unwrapping even when there
are phase jumps in the image. Based on our proposed
algorithm, the marker cue information can be extracted and
the period order information can be estimated accurately with
parsimonious. The system can be built with merely a
conventional projector and a camera with no additional
hardware requirement. Experimental results show that the
proposed algorithm is robust to the quality of the fringe
image and does not introduce much burden computationally
to the original DTCWT FPP framework. Future work will be
focused on achieving high quality 3D model reconstruction
of more complex objects.
Appendix A

Prove: Given Nm is an odd integer, show that the

mapping function in (6) ensures neighboring markers
will have a difference in y (k y ) with value at least

2
2
a b
a b
Nm
Nm

Proof: It is given in (6) that

y ky M ky
ky.

Nm

Nm 1
2

.
Nm

Nm 1
2

1 .

2 N m 1
Nm
2

k . N 2 1

Nm

Nm 1
N 1
k y . m
2
2

1 .

2
2
ba
b a
Nm
Nm

Nm

2
.
Nm N m

2
Nm

Nm

Nm

2 N m 1 N m 1

Nm 2
Nm

Nm

2
Nm
2
Nm

Nm

k . N 2 1

Nm

k . N 2 1 k
m

N 1
2
m
Nm
2

Nm

1 .

1 .

Nm 1
2

Nm 1
2

Nm

Nm

N 1
2
Nm m
Nm
2

Nm

2 N m 1 N m 1

Nm 2
Nm
N 1
N 1
Since m m , the statement is proved.

2
Nm

Nm

Nm

(Q.E.D.)
Hence the neighboring marker for the period ky+ 1 will
have,
N 1
y k y 1 k y 1 . m
2

This work is fully supported by the Hong Kong Research

Grant Council under research grant B-Q38N

2
.
Nm N m

References

Let

1.

1 .

Nm 1
2

a and
Nm

k . N 2 1

Nm

Since Nm is odd, a must be either greater than or

smaller than b.

2.

3.

4.

If a b then N m a b 0 .
Then

Nm

If b a then N m b a 0 . Then

N m 1 .

N 1
ky . m
Nm
2

2
Nm

Nm

5.

Z. Song, R. Chung, and X.-T. Zhang, "An Accurate and Robust

Strip-Edge-Based Structured Light Means for Shiny Surface
Micromeasurement in 3-D," IEEE Trans. Ind. Electron. 60, 1023
1032 (2013).
R. R. Garcia and A. Zakhor, "Consistent Stereo-Assisted Absolute
Phase Unwrapping Methods for Structured Light Systems," IEEE
J. Sel. Top. Signal Process. 6, 411424 (2012).
F. Sadlo, T. Weyrich, R. Peikert, and M. Gross, "A practical
structured light acquisition system for point-based geometry and
texture," in Eurographics/IEEE VGTC Symposium Proceedings in
Point-Based Graphics (2005), pp. 89145.
B. Shi, B. Zhang, F. Liu, J. Luo, and J. Bai, "360 Fourier
Transform Profilometry in Surface Reconstruction for
Fluorescence Molecular Tomography," IEEE J. Biomed. Heal.
Inform. 17, 681689 (2013).
S. Zhang and S.-T. Yau, "High-resolution, real-time 3D absolute
coordinate measurement based on a phase-shifting method," Opt.
Express 14, 2644 (2006).

Nm

6.
7.

8.

9.

10.

11.

12.

13.
14.

15.

16.

17.
18.

19.
20.
21.

22.

23.

24.

25.

26.
27.

28.
29.

30.

P. S. Huang and S. Zhang, "Fast three-step phase-shifting

algorithm," Appl. Opt. 45, 50865091 (2006).
F. Yang and X. He, "Two-step phase-shifting fringe projection
profilometry: intensity derivative approach," Appl. Opt. 46, 7172
7178 (2007).
S. Zhang and S.-T. Yau, "High-speed three-dimensional shape
measurement system using a modified two-plus-one phase-shifting
algorithm," Opt. Eng. 46, 136031136036 (2007).
G. Sansoni, M. Carocci, and R. Rodella, "Three-Dimensional
Vision Based on a Combination of Gray-Code and Phase-Shift
Light Projection: Analysis and Compensation of the Systematic
Errors," Appl. Opt. 38, 6565 (1999).
Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook,
"Period Coded Phase Shifting Strategy for Real-time 3-D
Structured Light Illumination," IEEE Trans. Image Process. 20,
30013013 (2011).
T.-W. Hui and G. K.-H. Pang, "3-D Measurement of Solder Paste
Using Two-Step Phase Shift Profilometry," IEEE Trans. Electron.
Packag. Manuf. 31, 306315 (2008).
Y. Wang, K. Liu, Q. Hao, X. Wang, D. L. Lau, and L. G.
Hassebrook, "Robust Active Stereo Vision Using Kullback-Leibler
Divergence," IEEE Trans. Pattern Anal. Mach. Intell. 34, 548563
(2012).
M. Gupta, Q. Yin, and S. K. Nayar, "Structured Light in Sunlight,"
Proc. IEEE Int. Conf. Comput. Vis. (2013).
M. Takeda and K. Mutoh, "Fourier transform profilometry for the
automatic measurement of 3-D object shapes," Appl. Opt. 22,
39773982 (1983).
D. Ali, . Serhat, and F. N. Ecevit, "Continuous wavelet transform
analysis of projected fringe patterns," Meas. Sci. Technol. 15,
1768 (2004).
A. Z. Abid, M. A. Gdeisat, D. R. Burton, M. J. Lalor, and F.
Lilley, "Spatial fringe pattern analysis using the two-dimensional
continuous wavelet transform employing a cost function," Appl.
Opt. 46, 6120 (2007).
C. Quan, C. J. Tay, and L. Chen, "Fringe-density estimation by
continuous wavelet transform," Appl. Opt. 44, 23592365 (2005).
T.-C. Hsung, D. Pak-Kong Lun, and W. W. L. Ng, "Efficient
fringe image enhancement based on dual-tree complex wavelet
transform," Appl. Opt. 50, 39733986 (2011).
K. Itoh, "Analysis of the phase unwrapping algorithm," Appl. Opt.
21, 2470 (1982).
D. C. G. and M. D. Pritt, Two-Dimensional Phase Unwrapping:
Theory, Algorithms, and Software (John Wiley & Sons, 1998).
S. Zhang, X. Li, and S.-T. Yau, "Multilevel quality-guided phase
unwrapping algorithm for real-time three-dimensional shape
reconstruction.," Appl. Opt. 46, 507 (2007).
R. M. Goldstein, H. A. Zebker, and C. L. Werner, "Satellite radar
interferometry: Two-dimensional phase unwrapping," Radio Sci.
23, 713720 (1988).
J. Gass, A. Dakoff, and M. K. Kim, "Phase imaging without 2
ambiguity by multiwavelength digital holography," Opt. Lett. 28,
1141 (2003).
Y. Wang, S. Yang, and X. Gou, "Modified Fourier transform
method for 3D profile measurement without phase unwrapping,"
Opt. Lett. 35, 790 (2010).
Y. Zhang, Z. Xiong, Z. Yang, and F. Wu, "Real-Time Scalable
Depth Sensing With Hybrid Structured Light Illumination," IEEE
Trans. Image Process. 23, 97109 (2014).
Y. Zhang, Z. Xiong, and F. Wu, "Unambiguous 3D measurement
from speckle-embedded fringe," Appl. Opt. 52, 77977805 (2013).
H. Cui, W. Liao, N. Dai, and X. Cheng, "A flexible phase-shifting
method with absolute phase marker retrieval," Measurement 45,
101108 (2012).
S. Gai and F. Da, "A novel phase-shifting method based on strip
marker," Opt. Lasers Eng. 48, 205211 (2010).
W.-H. Su and H. Liu, "Calibration-based two-frequency projected
fringe profilometry: a robust, accurate, and single-shot
measurement for objects with large depth discontinuities," Opt.
Express 14, 9178 (2006).
J. M. Huntley and H. Saldner, "Temporal phase-unwrapping
algorithm for automated interferogram analysis.," Appl. Opt. 32,
304752 (1993).

31.
32.

33.

34.

35.

36.

37.

38.
39.

40.

41.
42.

43.

44.

45.

46.

47.

48.

49.

50.

Y. Wang and S. Zhang, "Novel phase-coding method for absolute

phase retrieval," Opt. Lett. 37, 2067 (2012).
K. Liu, Y. Wang, D. L. Lau, Q. Hao, and L. G. Hassebrook,
"Dual-frequency pattern scheme for high-speed 3-D shape
measurement," Opt. Express 18, 52295244 (2010).
I. W. Selesnick, R. G. Baraniuk, and N. C. Kingsbury, "The dualtree complex wavelet transform," IEEE Signal Process. Mag. 22,
123151 (2005).
T.-C. Hsung, D. P. Lun, and W. W. L. Ng, "Zero spectrum
removal using joint bilateral filter for Fourier transform
profilometry," Vis. Commun. Image Process. (VCIP), 2011 IEEE
14 (2011).
W. W.-L. Ng and D. P.-K. Lun, "Effective bias removal for fringe
projection profilometry using the dual-tree complex wavelet
transform," Appl. Opt. 51, 59095916 (2012).
W. W.-L. Ng and D. P.-K. Lun, "Image enhancement for fringe
projection profilometry," Circuits Syst. (ISCAS), 2013 IEEE Int.
Symp. 729732 (2013).
B. Budianto and D. P. K. Lun, "Efficient 3-dimensional model
reconstruction based on marker encoded fringe projection
profilometry," Acoust. Speech Signal Process. (ICASSP), 2014
IEEE Int. Conf. 574578 (2014).
X. Su and W. Chen, "Fourier transform profilometry: a review,"
Opt. Lasers Eng. 35, 263284 (2001).
T.-C. Hsung and D. P.-K. Lun, "On optical phase shift
profilometry based on dual tree complex wavelet transform,"
Image Process. (ICIP), 2010 17th IEEE Int. Conf. 337340 (2010).
N. Kingsbury, "A dual-tree complex wavelet transform with
improved orthogonality and symmetry properties," in Proc. Int.
Conf. Image Process. (2000), Vol. 2, pp. 375378 vol.2.
D. L. Donoho, "De-noising by soft-thresholding," IEEE Trans. Inf.
Theory 41, 613627 (1995).
L. Sendur and I. W. Selesnick, "Bivariate shrinkage with local
variance estimation," IEEE Signal Process. Lett. 9, 438441
(2002).
L. Sendur and I. W. Selesnick, "Bivariate shrinkage functions for
wavelet-based denoising exploiting interscale dependency," IEEE
Trans. Signal Process. 50, 27442756 (2002).
D. L. Donoho and I. M. Johnstone, "Adapting to Unknown
Smoothness via Wavelet Shrinkage," J. Am. Stat. Assoc. 90,
12001224 (1995).
A. Vo and S. Oraintara, "On the distributions of the relative phase
of complex wavelet coefficients," in Proc. Int. Symp. Circuits
Systems (2009), pp. 529532.
J. Fauqueur, N. Kingsbury, and R. Anderson, "Multiscale
Keypoint Detection using the Dual-Tree Complex Wavelet
Transform," in Proc. IEEE Int. Conf. Image Process. (IEEE,
2006), pp. 16251628.
A. Vo and S. Oraintara, "A study of relative phase in complex
wavelet domain: Property, statistics and applications in texture
image retrieval and segmentation," Signal Process. Image
Commun. 25, 2846 (2010).
Q. Kemao, "Two-dimensional windowed Fourier transform for
fringe
pattern
analysis:
Principles,
applications
and
implementations," Opt. Lasers Eng. 45, 304317 (2007).
Q. Kemao, H. Wang, and W. Gao, "Windowed Fourier transform
for fringe pattern analysis: theoretical analyses," Appl. Opt. 47,
54085419 (2008).
H. Shi, B. Hu, and J. Zhang, "A Novel Scheme for the Design of
Approximate Hilbert Transform Pairs of Orthonormal Wavelet
Bases," IEEE Trans. Signal Process. 56, 22892297 (2008).