Automation in Construction 10 2001.

527540
www.elsevier.comrlocaterautcon

Assessment of a camera pose algorithm using images of

brick sewers
D. Cooper a,) , T.P. Pridmore b, N. Taylor c
a
b

School of Computing and Management Sciences, Sheffield Hallam Uniersity, City Campus, Sheffield, S1 1WB, UK
School of Computer Science and Information Technology, Uniersity of Nottingham, Jubilee Campus, Wollaton Road,
Nottingham, NG8 1BB, UK
c
School of Enironment and Deelopment, Sheffield Hallam Uniersity, City Campus, Sheffield, S1 1WB, UK
Accepted 31 October 2000

Abstract
Further developments in an ongoing multi-disciplinary research programme concerning the automation of sewer surveys
are reported. Previous papers have suggested a theoretical model for use in establishing a frame of reference for closed
circuit television CCTV. camera images of non-man-entry NME. brick sewers, in order to enable quantitative observations
to be automatically acquired employing computer vision techniques. Herein is discussed a simulation testing strategy
designed to determine the models predictive accuracy and thereby assess the corresponding assumptions made. Test results
are presented which display robust model characteristics. Further developments are then considered in the context of
in-service practice. q 2001 Elsevier Science B.V. All rights reserved.
Keywords: Closed-circuit-television image; Non-man-entry brick sewer; Computer vision

1. Motivation and previous work

Although they are no longer being built, tens of
thousands of kilometres of brick sewer are still in
use in the United Kingdom. Many of these pipes are
in poor condition. The majority are classified as
non-man-entry or NME less than 1 m diameter.,
primarily for health and safety reasons. These NME

)
Corresponding author. Tel.: q44-114-225-3798r3171; fax:
q44-114-225-3161.
E-mail address: D.Cooper@shu.ac.uk D. Cooper..

sewers are typically inspected or surveyed by propelling a closed circuit television CCTV. camera
together with a light source through the pipe concerned using some form of remote-controlled tractor.
Over the years, sewer surveys of this type have
generated huge libraries of records in the form of
many thousands of hours of video. Interpretation of
these images by human operative is slow, laborious,
subjective and prone to error. Some computer-based
tools are now available which support the operative
in classifying images on-line. This approach eases,
but does not remove, the drawbacks noted above.
Hence, there is an urgent need to find a method of
processing images automatically using computer-

0926-5805r01r$ - see front matter q 2001 Elsevier Science B.V. All rights reserved.
PII: S 0 9 2 6 - 5 8 0 5 0 0 . 0 0 1 0 9 - 6

528

D. Cooper et al.r Automation in Construction 10 (2001) 527540

based technology, thereby improving objectivity, to

identify the sewers most in need of repair or replacement. More details of the related problems and possibilities are given in Refs. w1,46,8,10x.
A first step towards providing for the automatic
acquisition of quantitative data from respective
CCTV camera images requires knowledge of the
cameras pose i.e., position and orientation or attitude. relative to the pipe. Ideally, the tractor would
track directly along the sewers invert. However,
tractors invariably wander up the sidewalls owing to
slippage, obstruction and wear-and-tear; this complicates image assessment. Knowledge of camera pose
would enable measurements of sewer pipe features
to be made directly from the CCTV images. For
example, the location, length and width of a pipe
wall crack could be assessed from its captured image
employing digital computer vision techniques w3,8x.
Whilst chainage data, i.e., position along the sewer
pipe, are usually available by employing a digitally
monitored wire-dispensing drum also mounted on
the tractor, camera location in the cross-section and
its orientation relative to the pipes centreline. are
not and image analysis is a largely subjective human
task.

Brick sewers date from the Victorian era and,

being the oldest, constitute a particular problem with
respect to general sewer maintenance. One compensating feature is available in the present context in
that a key feature of brick sewers is the pattern of
mortar lines running along the pipe walls, roughly
parallel to each other and to the axis of the pipe, as
shown in Fig. 1; the upper courses are highlighted
see below.. Lines that are parallel in space generate
image lines which converge at a point known as the
vanishing point for a given direction, as indicated in
Fig. 1. The vanishing point provides for an initial
frame of reference w8x. In a locally straight run of
brick sewer with the camera and light source pointing roughly along the pipe axis, longitudinal mortar
lines converge in the image to a vanishing point in a
dark region near the centre of the image. The dark
region represents parts of the pipe too distant from
the light source for them to appear clearly on the
image. When the vanishing point has been located,
the angle g between the pipe and camera axes can be
found, provided that the focal length f of the camera
is known.
In Ref. w8x, approaches to the vanishing point
detection problem were reviewed, a method for the

Fig. 1. A typical brick sewer image, with edge detection and further processing used to highlight the mortar lines in the crown.

D. Cooper et al.r Automation in Construction 10 (2001) 527540

automatic detection of vanishing points in images of

brick sewers was presented and the success of the
technique in recovering partial survey camera orientation evaluated. Subsequent work w3x, employing a
rigorous mathematical model of the mortar lines,
raised the possibility of recovering further information about camera orientation and additional positional parameters from the pattern of orientations of
images of longitudinal mortar lines. That is, the use
of computer vision to find the vanishing point, and
thereby to recover the mortar line orientations, provides input to the mathematical model; this gives a
procedure to provide knowledge of camera pose
from the image. Initial findings suggested that the
procedure was capable of recovering quantitative
information from reasonable quality images, although specific assumptions about the shape of the
sewer and the cameras orientation had to be made in
order to achieve this result see Section 2.. Model
predictions remained unconfirmed by experimental
testing. Importantly, experimental field testing of the
model employing elderly NME brick sewers is not
readily available, owing to the associated prohibition
of human access. Laboratory testing as employed in
related studies of otherwise unused concrete NME
sewer pipes w1,10x would fail to replicate the aged
nature encountered in field conditions. The ageing
process is too complex and variable to be realistically modelled to an acceptable level of confidence.
Rather than construct a new. brick sewer in the
laboratory at some cost and then simulate the wear
and tear of a century of use involving noxious agents
noting the health issues involved., data set simulation provided a cost-effective option. The present
paper therefore reports on ensuing work to provide
an alternative means of assessing the mathematical
modelling discussed in Ref. w3x.
Section 2 provides a brief overview of the modelling used. Emphasis is placed upon the assumptions made, whilst mathematical detail is minimised
given its availability elsewhere w3,8x. Section 3 introduces the use of artificial data sets to provide for the
means of testing the model in the absence of a
physical prototype. Simulation techniques are extremely useful to engineers in circumstances where
physical testing is effectively unavailable. Variations
in model performance under differing conditions are
studied. The findings are tabulated and considered.

529

As a result of these findings, an enhancement has

been made to the vanishing point determination
method, and the pose recovery procedure has been
applied to in-service VHS video images relating to
sewers in, first, good physical condition and, second,
poor physical condition. This is described in Section
4. Finally, conclusions are drawn in Section 5.

2. Recovering camera pose from visible mortar

lines
Brick sewers generally take one of three sectional
geometries, circular, egg where the invert possesses
the greater curvature to provide for relatively deeper
flow and structural strength, given that earth pressure
is greatest here. and U-profile. The last identified
consists of brick sides and invert with a slab top and
does not figure in the present study. With camera
and light source mounted on top of the tractor, the
sewer crown is a well-defined region in CCTV video
images. Further, as both the circular and egg section
cases ideally possess crowns of constant finite curvature, these cases may be termed, in the manner of
masonry arches, as possessing segmental crown geometry w9x. Consequently, the brick sewer study
programme w1,3,8,10x is concerned with consideration of segmental crown associated CCTV image
mortar lines, acquired employing edge detection w2x;
recall Fig. 1.
The key assumptions in the reported camera pose
modelling w3x are:
v

the sewer possesses a circular segmental crown;

brick and mortar courses are regular;
the angle g between the camera and sewer axes
is effectively zero.

Clearly, constructional tolerances and ageing under ground pressure and variable surface loading
result in imperfect segmental crown geometry; mortar lines will vary in thickness and lie, this feature
being influenced by the bricks varying in size andror
condition. Indeed, despite lateral earth pressure tending to restrain the pipe, crown profiles can flatten,
threatening collapse. With the camera axis inevitably

530

D. Cooper et al.r Automation in Construction 10 (2001) 527540

A calculation based on Eq. 1. for consecutive

mortar lines m and m q 1 shows that the angle
between them in the image is given by:
p 2 sinh y2 psin
y1

umq 1 y um sh ytan

h
2

1q p 2 cosh y2 pcos

cos l y mh .
.

h
2

cos l y mh .

2.

Fig. 2. Upper cross section through pipe, showing angles j , h , n

and l and distance pR.

non-parallel to the sewer axis g / 0., it is clear that

any modelling for camera pose must be robust. It is
to be noted that the modelling, and its assumptions,
are of three-dimensional form.
The essential pose parameters additional to g ,
which is of longitudinal form by nature and noting
that chainage is to be separately acquired., are shown
in Fig. 2. A plane X, Y . coordinate system is
chosen for a cross-section of the pipe, with origin on
the pipe axis. Let R be the segment radius. Suppose
that the distance of the camera centre from the centre
line of the pipe is pR, and n is the angle, measured
anti-clockwise, between the X-axis and the line from
the pipe centre to the camera centre. Let h be the
angle subtended at the pipe centre by a typical brick,
and j the angle between the X-axis and the first
brick boundary encountered anticlockwise; write l
for the angular distance between the mid-point of the
first brick and the camera centre. Clearly n s j q
hr2 q l. Definition of camera pose requires knowledge of g , together with p, h and l.
The parameters shown in Fig. 2 can be related to
an angle um , the angle which mortar line number m
makes with the horizontal in the image. Examples of
such lines are highlighted in Fig. 1. Under the assumptions noted above, including g s 0, it can be
shown w3x that,

tan um s

sin j q mh . y psin n
cos j q mh . y pcos n

1.

Radial line angles in an actual image are found by

grouping the line segments, which were acquired by
employing edge detection w2x. Firstly, segments of
lines that do not pass near to the vanishing point are
rejected. Secondly, for each remaining line segment,
its orientation is calculated as the angle between the
image horizontal and a line from the vanishing point
to the segment mid-point. Thirdly, the line segments
are grouped into classes according to their orientations, with each class covering a range of size 28.
Then the number of line segments in each class,
weighted according to their lengths, is found, and
each local maximum above a threshold is assumed to
arise from a radial line; the mean orientation of the
segments in the class is used as an estimate of the
orientation of the radial line. Further details can be
found in Ref. w3x. The 28 spread was empirically
found to be large enough to identify most mortar
lines but small enough to give an accurate estimate
of the mortar line orientation "18..
Observed radial angle differences umq 1 y um from
an image give Eq. 2. for a sequence of values of m.
Because only the sewer crown is being considered,
only radial lines above the image vanishing point are
used, those where the angle lies between 08 and
1808. A least-squares technique now enables the
determination of values of the three variables, p, l
and h , which minimise the root mean square error
when the observed radial angle differences are compared with calculated values. Clearly, at least four
mortar line angles are needed to determine the three
unknowns.
The initial values for the iterative process used to
implement the least-squares technique were p s 0.4
and l s 90, with h approximated by the average
difference between the radial angles. It has been
found previously w3x that choosing different initial
values can affect the convergence of the process, but
that it is unlikely to affect the final values when the
process does converge.

D. Cooper et al.r Automation in Construction 10 (2001) 527540

3. Model testing
3.1. Generation of artificial test data
Given the impossibility, as noted previously, of
comparing the pose parameters calculated from
images using Eq. 2. with actual physical data, a
simulation exercise was carried out. The simulation
generates artificial data incorporating practical imperfections such as lack of sectional circularity and a
non-zero angle between camera and pipe axes. The
purpose is to show that the differences between the
assumptions in the analysis and a practical situation
do not give rise to disproportionately large numerical
errors in the results.
To evaluate the robustness of the method therefore, an ellipse is used to model a deformed or
inaccurately constructed circular segmental crown.
The assumption of circularity is relaxed, and it is
assumed only that the pipe has a constant elliptical
upper half cross-section. This is so that the equations
can be used to create artificial data for use when
testing the extent to which results are distorted by
considering the pipe to be circular.
Further, during a normal sewer survey the camera
is generally intended to point directly down the pipe,
though as the vehicle carrying the camera is driven
or pulled along there will obviously be some variation in the direction of the line of sight. The angle g
between the axis of the pipe and the cameras principal axis, directed along the line of sight, is therefore
allowed to be non-zero.
As in Section 2, a plane X, Y . coordinate system
is used for a cross-section of the pipe, with origin on
the pipe axis. Let A and B be the major and minor
radii of the ellipse, respectively. Let R be the root
mean square radius, so that 2 R 2 s A2 q B 2 . Let b
be the angle, measured anti-clockwise, between the
X-axis and the major axis of the ellipse. The parameter p is now the distance of the camera centre from
the centre line of the pipe as a proportion of the
average radius R. As before, n is the angle, measured anti-clockwise, between the X-axis and the
line in a cross-section. from the pipe centre to the
camera centre.
Eq. 1. will now be generalised. By using the
standard parametric equations, X s A cos t and

531

Y s B sin t , for an ellipse and rotating through b , it

can be shown w3x that for any mortar line there is an
angle t , such that the angular position u of a line in
the image relative to the line from the image centre
to the vanishing point. is given by,
tan u s

A cost sin b q B sint cos b y Rp sin n . cosg

A cost cos b y B sint sin b y Rp cos n

3.

The parameter t has no obvious geometrical significance. However, Eq. 3. may be used to create
artificial series of mortar line angles for given values
of p, h , b , g , n and the radii. Note that h does not
occur in Eq. 3.; the value of h was used to determine the brick size or, more precisely, the combined
size of a brick and mortar joint., and values for t
were calculated to make distances between adjacent
mortar lines equal to this size throughout the model.
It can be seen that only the ratio of the radii B and
A affects the angles produced, not their actual values.
In the tables of results below, the distortion is shown
by giving AB as a percentage of A. Calculations
were done with brick size calculated from a circle of
radius 1, and values of B less than 1 were chosen to
give distortions varying from 0% to 5%. Inspection
of images suggests that distortions of up to 5% will
cover typical cases.
For each value of B, the value of A was calculated to allow the correct number of bricks of the
calculated size to fit around the ellipse. Then a series
of values of t was calculated to make the distance
between the points on the ellipse corresponding to
consecutive values of t equal to the brick size. The
values of t for these mortar lines were then used in
Eq. 3. to generate a series of values of u which
correspond to the values um in Eq. 1..
The robustness of the system may then be evaluated by finding the values for p, h and l which are
obtained by applying the method of Section 2, under
the assumptions that A s B s R and g s 0. That is,
Eq. 3. is used to artificially generate imperfect data
in place of image-generated data such as could be
acquired from Fig. 1; the ability of calculations
based on Eq. 2. to handle these data is tested. The
results can be evaluated because the input is known.
Values assumed in the simulation were p s 0.24,
h s 12.86 corresponding to 28 brick courses around
a full circle. and n s 140. These are means of values

D. Cooper et al.r Automation in Construction 10 (2001) 527540

532

found from experiments using nine images, reported

in Ref. w3x, and appear to be realistic image values.
The parameter b was set to 0.
3.2. Ealuation of the effect of the assumption that
the camera points directly down the pipe
It was shown in Ref. w8x that, assuming that the
pipe has constant cross section, the distance from the
image centre to the vanishing point for longitudinal
lines is f tan g , where f is the focal length of the
camera. The positions of the vanishing points found
and knowledge of possible values for the focal length
f show that g is less than 3.5 for the data considered
here. It should be noted that the images used were
provided by an experienced survey contractor and
arose from typical sewer surveys.
Hence, cos g ) 0.998, and Eq. 3. shows that the
error introduced into the value of tan u by the
assumption that g s 0 is less than 0.2%, whatever
the values of the other parameters. Such an error in
tan u results in an error less than 0.001 rad, or 0.06,
in u . The angles of the mortar lines in the poor
quality images produced by sewer surveys cannot be
measured without errors at least as big as this, so it
would be expected that the assumption g s 0 would
have a negligible effect on the results.
With equal values for the major and minor radii
A and B no distortion., and values for p, h and n
as above, artificial series of mortar line angles in the
upper half of images were created with values for g ,
the deviation of the camera axis from the pipe axis,
of 08, 18, 28, 38, 3.58 and 48. The recovered values of
p, h and l when these are analysed under the
assumption g s 0 are shown in Table 1. As expected
Table 1
Recovered values of p, h and l from artificial data, with a
circular pipe section and varying g

0
1
2
3
3.5
4

0.240
0.240
0.241
0.241
0.242
0.242

12.86
12.86
12.85
12.84
12.83
12.82

121.6
121.6
121.3
121.0
120.8
120.5

The first line of the table also shows the values used to generate
the data. Angles are measured in degrees.

Table 2
Recovered values of p, h and l from artificial data, with an
elliptical pipe section and g s 0
Distortion %.

0
1
2
3
4
5

0.240
0.246
0.253
0.260
0.267
0.274

12.86
12.75
12.65
12.54
12.43
12.33

122
119
116
114
111
109

The first line of the table also shows the values used to generate
the data. Angles are measured in degrees.

from the discussion above, the errors introduced are

small, suggesting that the model is robust.
3.3. Ealuation of the effect of the assumption of
circularity
To test the effect of the eccentricity of the elliptical cross-section on the results obtained by assuming
that it is circular, artificial series of mortar line
angles in the upper half of images were created with
distortions 05%. This was done by varying the
minor radius from 1 to 0.9742 in normalised units.,
with the major radius adjusted in each case to give
the correct number of brick courses around the ellipse. The tests were carried out twice, with g s 0
the value assumed in the recovery process. and with
g s 3.58 as far as could be expected from the value
assumed in the recovery process.. The results are
shown in Tables 2 and 3.
Tables 2 and 3 show that the assumption of
circularity does have an effect on the results. With a
5% distortion, the recovered distance of the camera

Table 3
Recovered values of p, h and l from artificial data, with an
elliptical pipe section and g s 3.58
Distortion %.

0
1
2
3
4
5

0.242
0.248
0.255
0.262
0.269
0.277

12.83
12.72
12.62
12.51
12.40
12.30

121
118
115
113
110
108

Angles are measured in degrees.

D. Cooper et al.r Automation in Construction 10 (2001) 527540

from the pipe centre was 0.27 or 0.28 times the

radius, instead of 0.24 times. For l, the error initially
seems quite large, up to 148, indicating that the
angular position of the camera will be hard to estimate. However, to put this in perspective, the combined effect of the errors in p and l in the worst
case 5% distortion, g s 3.58. is to misplace the
camera by 3.5 cm with a 1 m diameter pipe, the
largest classed as non-man-entry in the UK. The
largest error in h represents a recovered assumption
of about 29 bricks around the pipe instead of 28.
Furthermore, the similarity between Tables 2 and
3 confirms that the approximation of g by 0 does not
have a major effect on the size of the errors.
Again, satisfactory model robustness is considered to be displayed.
3.4. Ealuation of the effect of inaccuracies in measuring the radial angles
The tests were repeated with a further perturbation, to simulate the problem that the angles of the
mortar lines u . cannot in practice be measured with
complete accuracy. Each line was derived by grouping shorter line segments into classes of angles lying
within a spread of 2 see Section 2.. In the simulation, after calculation of the angles of a series of
mortar lines, each angle was replaced by a randomly
generated value having a uniform distribution with a
range 1 each side of the calculated value, to model
the errors expected within classes. A hundred randomly generated cases were input to the model, and
the mean recovered values of p, h and l, and their
standard deviations, were calculated. The results, for
distortions up to 5% and g s 3.58, are shown in
Table 4.
Table 4
Recovered values of p, h and l from artificial data, with an
elliptical pipe section and g s 3.58, with angles perturbed up to 18
Distortion mean p "s.d.. mean h "s.d.. mean l "s.d..
%.
0
1
2
3
4
5

0.245 "0.017.
0.249 "0.017.
0.254 "0.018.
0.261 "0.017.
0.270 "0.018.
0.276 "0.017.

12.82 "0.35.
12.73 "0.35.
12.63 "0.32.
12.53 "0.30.
12.40 "0.32.
12.33 "0.30.

Angles are measured in degrees.

121 "8.
118 "8.
116 "7.
114 "7.
111 "6.
109 "6.

533

Table 5
Recovered values of p, h and l from artificial data, with an
elliptical pipe section and g s 3.58, with angles perturbed up to 28
Distortion mean p "s.d.. mean h "s.d.. mean l "s.d..
%.
0
1
2
3
4
5

0.250 "0.030.
0.247 "0.032.
0.256 "0.034.
0.269 "0.033.
0.271 "0.038.
0.271 "0.032.

12.76 "0.60.
12.82 "0.65.
12.67 "0.72.
12.44 "0.57.
12.43 "0.70.
12.40 "0.56.

121 "14.
122 "16.
119 "16.
113 "12.
113 "15.
111 "11.

Angles are measured in degrees.

Further, the angles of the line segments which are

part of a mortar line in the image inevitably show
some variation from the true orientation of the mortar line, and may contribute to the count of segments
in an incorrect class. This may result in some mortar
lines being assigned to classes adjacent to their
correct classes. To simulate the effect of these errors,
a more demanding perturbation of up to 28 was
generated, 100 cases again being assessed. The results, again with distortion 05% and g s 3.58, are
shown in Table 5.
In none of the 200 simulations did the perturbation of the angles cause a failure in convergence. The
mean values of the parameters recovered, in Tables 4
and 5, are similar to those found in Table 3, showing
that the perturbation does not generally increase the
error in the recovered values. The standard deviations are relatively small, showing that, although the
perturbation can occasionally produce quite large
variations in the results, these are rare.
The results of this simulation again demonstrate
the stability of the pose recovery method.

4. The pose recovery procedure applied to real

image sequences
4.1. A reised anishing point technique
The results reported in Section 3 lend confidence
to the method used to recover camera pose from lists
of mortar line angles. An improved vanishing point
technique allows further analysis of the image sequences considered in Ref. w8x, to determine mortar

534

D. Cooper et al.r Automation in Construction 10 (2001) 527540

lines and recover camera pose. Two sections of

video were used, one showing a sewer in reasonably
good condition, the other a sewer in such poor
condition that the mortar joints are obscured andror
do not form straight lines between courses. Each

section was sampled to give images at intervals of 1

s and again to give images at intervals of 1r12 s.
The images employed here were acquired from
industrially sourced sewer survey videos obtained
using standard survey equipment. The image analysis

Fig. 3. Two images captured at an interval of 1r12 s from a sewer in reasonably good condition images 6 and 7., with the automatically
identified mortar lines superimposed.

D. Cooper et al.r Automation in Construction 10 (2001) 527540

software was written in the C language, and most

was embedded in the TINA vision research environment w7x. Technical specifications of the camera used
are not available; images were reacquired from a
VHS video cassette recorder using a Data Translation Fidelity 100 frame grabber and digitised to
produce a 768 = 576 array of picture elements or
pixels.. To remove high-frequency noise and reduce
processing time, each image was then resampled to
384 = 288 pixels. Two hundred fifty-six grey levels
are distinguished 0 s black, 255 s white.. Survey
images are acquired using physically robust but often
optically low quality cameras in wet, misty pipes and
are frequently noisy and blurred as a result. The
strong directional light sources in common use also
tend to introduce glare and specular reflections. The
poor quality of the images shown is typical of images produced in these conditions. Quality is further
reduced when survey images stored on VHS tape, an
inherently lossy medium, are reacquired, as is the
case here.
The first step in identifying the vanishing point is
application of a light intensity threshold to the image, to determine the extent of the dark region in
which the vanishing point lies w8x; recall Fig. 1. A
simple fixed threshold, specified by the user, is
applied. All pixels whose intensity values are below
this threshold are considered to be part of the dark
region, all others are not. The thresholding technique
has been altered to reduce the likelihood of dark

535

Table 7
Recovered values of p, h and l from real images of a good pipe,
with 1 s interval between images
Image

Error

0
1
2
3
5
7
8
9

0.48
0.58
0.20
0.07
0.09
0.09
0.00
0.10

15.8
17.8
14.4
13.6
13.8
12.9
13.2
12.5

203
234
226
218
252
32
177
28

1.4
1.8
1.6
1.3
1.1
0.7
1.4
1.4

Angles are measured in degrees. Errors are measured as in Table

6.

regions arising from the pipe floor or invert pulling

the initial vanishing point estimate towards the lower
edge of the image. As mentioned in Section 2, the
light source is on top of the tractor, exaggerating this
bias in illumination. The modification made is simple but effective; instead of applying a threshold to
the entire image and finding the centroid of the
largest dark region, it is applied only to an area,
centred on the image centre, 3r4 of the size of the
full image. The reduced area was chosen to be
rectangular because the full image is rectangular.
This reduction neatly removes the nearby pipe floor
from the process, and represents an application of
knowledge-based engineering by recognising image
bias.
4.2. Results from a sewer in good condition

Table 6
Recovered values of p, h and l from real images of a good pipe,
with 1r12 s interval between images
Image

Error

0
2
3
4
5
6
7
8
9

0.40
0.58
0.47
0.49
0.39
0.34
0.29
0.30
0.38

15.2
18.0
18.6
17.7
16.0
15.4
15.4
14.9
16.1

192
233
226
231
201
192
199
192
209

1.3
1.9
2.0
1.8
1.4
1.1
1.4
1.0
1.7

Angles are measured in degrees. The error column shows the root
mean square of the differences between the measured mortar line
angles and the angles calculated from the recovered values of the
parameters.

Mortar lines detected in two consecutive images

from the sequence acquired from the good sewer at
1r12 s intervals are shown in Fig. 3. In all cases, as
noted in Section 2 and illustrated in Fig. 1, analysis
was restricted automatically to lines above the horizontal. Where a visual check showed an obvious
mortar line which had been missed, for example near
the top right of the images in Fig. 3, an additional
line was inserted manually halfway between the lines
on each side of the gap; work continues in providing
for this to be undertaken automatically see Section
5.. On further examination of the mortar lines which
were identified in the images of this sequence, it was
found that images 3, 4 and 7 all had a spurious line
in the first brick course above the horizontal at the

536

D. Cooper et al.r Automation in Construction 10 (2001) 527540

left-hand side see Fig. 3b.. This was removed;

again, work in is progress to achieve the automatic
detection of such lines. The iterative process converged for nine of the 10 images in this sequence.
The recovered values of p, h and l obtained from
these images are shown in Table 6. The table also

shows a measure of the error in the results, namely

the root mean square of the differences between the
radial angles measured and angles calculated from
the recovered values. The largest error is 2.08, which
is of the same order as the accuracy with which the
angles can be measured. There is consistency in the

Fig. 4. Four images captured at intervals of 1 s from a sewer in reasonably good condition images 03., showing camera rotation.

D. Cooper et al.r Automation in Construction 10 (2001) 527540

537

Fig. 4 continued ..

values for consecutive images, as one would expect

when the time interval between them is relatively
small.
When the analysis was carried out without removing the spurious line from images 3, 4 and 7, values
close to p s 0.32, h s 118 and l s 808 were ob-

tained. There is no obvious inconsistency between

these values of p and those obtained from the other
images; however, the values of h and l were consistent with each other but inconsistent with the values
from the other images. Widely differing values from
images close together in time provide extra evidence

538

D. Cooper et al.r Automation in Construction 10 (2001) 527540

which can help a knowledge-based system to detect

spurious mortar lines.
The recovered values of p, h and l which were
found for the images in the sequence taken from the
good sewer at 1 s intervals are shown in Table 7.

The iterative process converged for eight of the 10

images of this sequence. The r.m.s. errors are less
than 28. The beginning of the sequence shows considerable variation in the values of p and l; this
variation is consistent, from inspection of the images

Fig. 5. An image captured from a sewer in poor condition, and the same image with the automatically identified mortar lines superimposed.

D. Cooper et al.r Automation in Construction 10 (2001) 527540

in Fig. 4, with the actual camera movement at this

time. For images later in the sequence, the values of
p are small, indicating that the camera is near the
centre of the pipe. There is considerable variation in
the values of the angle l; for small values of p, a
small camera movement can indeed generate large
changes in l and so these values must be less
reliable.. It should also be noted that the angle l is
measured relative to the first brick course above the
horizontal at the right hand side of the image, and
this may not be the same course in successive images.
As anticipated, differences in the recovered parameters between consecutive images are smaller in
the case of intervals of 1r12 s than in the case of 1 s
intervals.
4.3. Results from a sewer in poor condition
When the technique was applied to images captured at 1 s intervals from a sewer in poor condition,
convergence was achieved in only four cases out of
eight, and in all of these the root mean square error
was at least 38. The best of these results, with an
error of 3.38, was from image 5, which is shown in
Fig. 5; here p was recovered as 0.78 and h as 6.08;
this very small value for h , suggesting a large number of brick courses, is an indication that the result is
inaccurate.
The results were slightly better for the sequence
of images at 1r12 s intervals from the poor sewer;
for seven of the 10 images estimates were obtained
with error values less than 48 Table 8.. Here there
are wide variations in the values of the camera pose
parameters. A sewer in as poor a condition as this
will have a highly uneven floor, where earth pressure
Table 8
Recovered values of p, h and l from real images of a poor pipe,
with 1r12 s interval between images
Image

Error

0
3
4
6
7
8
9

0.36
0.27
0.35
0.13
0.54
0.42
0.52

9.4
10.9
10.2
13.7
9.9
9.7
19.4

74
53
47
107
78
43
319

3.2
2.3
2.9
2.0
3.2
3.7
2.4

Angles are measured in degrees.

539

is greatest recall Section 2., causing considerable

camera movement.
The lack of consistency in the results in the case
of this section of video is not surprising in view of
the extremely poor quality of the images. However,
the convergence problems and the wide variation in
results from adjacent images do provide a warning of
the dire sewer condition.
5. Discussion and conclusion
A previously reported but effectively untested
theoretical model for recovering camera pose from
CCTV video surveys of NME sewers has been subjected to assessment by employing simulation techniques, in the absence of availability and accessibility of physical prototypes. The key issues relate to
the viability of the models underlying assumptions
and its stability when used on imperfect data. The
use of input data artificially generated with variations from the assumptions lack of camera and
sewer co-directionality, non-circular sewer crowns
and brickrmortar course irregularities. has shown
the model to be able to provide useful output data
under a range of imperfect conditions. That is, the
model is reliably robust. The limits of the variation
in output as a result of the variation in input have
been found.
Ensuing application to actual CCTV sewer videos
confirms useful pose output to be available for sewers in good, but not ideal, condition. Poor condition
sewers result in the procedure largely failing to
provide direct pose output. However, this is to be
expected given the lack of suitable information in the
images in this case, and, indeed, the method remains
useful in that this situation automatically informs of
the dire state of the sewer concerned.
Proposed developments include more use of
knowledge-based engineering concepts. Whilst the
work reported in this paper includes an improved
vanishing point detection procedure to compensate
for illumination bias, more can be done in exploiting
the expected brickrmortar coursing patterns. For
example, the mortar line identification technique,
which is central to the models application, should be
able to distinguish reliably between actual lines and
spurious lines which are artifacts of the edge detection method. The missing of courses should be made

540

D. Cooper et al.r Automation in Construction 10 (2001) 527540

obvious by exploiting knowledge of the structure of

brick sewers, and their location hunted by refinement
of thresholds or the use of image enhancement.
Knowledge of camera pose will lead to automatic
quantitative assessment of physical sewer features.
Automation of non-man-entry sewer surveys appears
to be a logical undertaking for the construction industry, and the findings reported here confirm the
current methods promise and encourage further
study.
Notation
CCTV Closed circuit television
NME Non-man-entry
g
angle between pipe and camera axes
f
focal length of the camera
R
radius of a circular segment, or root mean
square radius of an elliptical segment
p
distance of camera centre from centre line
of the pipe, as a proportion of radius R
n
angle between X-axis and line from pipe
centre to the camera centre.
h
angle subtended at pipe centre by a typical
brick
j
angle between X-axis and first brick boundary encountered anticlockwise
l
angular distance between mid-point of first
brick and camera centre.
angle which mortar line number m makes
um
with the horizontal in image
A
major radius of elliptical segment
B
minor radius of elliptical segment
b
angle between X-axis and major axis of
ellipse
t
parameter for parametric equation of ellipse

Acknowledgements
This research continues work which was supported by EPSRC Standard Research Grant GRr

J49112. The TINA vision research environment was

supplied by the Artificial Intelligence Vision Research Unit, University of Sheffield. The authors
would like to thank Mr. Neil Bunting of Subtech,
Sheffield, for supplying the sewer survey videos
used, and Mr. Simon Broadhurst, Assistant Engineer,
Main Drainage Section, City of Salford, for further
industrially sourced advice. We are also grateful to
the anonymous referees who made suggestions for
improving the clarity of parts of the paper.

References
w1x S. Broadhurst, G. Cockerham, N. Taylor, T.P. Pridmore,
Automatic task modelling for sewer studies, Autom. Constr.
5 1996. 6171.
w2x J. Canny, Finding edges and lines in images. MIT-AI-TR-720,
Massachusetts Institute of Technology, 1983.
w3x D. Cooper, T.P. Pridmore, N. Taylor, Towards the recovery
of extrinsic camera parameters from video records of sewer
surveys, Mach. Vision Appl. 11 1998. 5363.
w4x National Water Council, Manual of Sewer Condition Classification, NWC, London, 1980.
w5x X. Pan, T.A. Clarke, T.J. Ellis, The detection and tracking of
pipe joints in noisy images, Videometrics III vol. 2350,
SPIE, Boston, 1994, pp. 136147, Nov.
w6x X. Pan, T.J. Ellis, T.A. Clarke, Robust tracking of circular
features, in: D. Pycock Ed.., Proc. British Machine Vision
Conference, Birmingham, UK, BMVA Press, London, 1995,
pp. 553562.
w7x J. Porrill, S.B. Pollard, T.P. Pridmore, J. Bowen, J.E.W.
Mayhew, J.P. Frisby, TINA: the Sheffield AIVRU vision
system,Proc. 10th International Joint Conference on Artificial
Intelligence, Milan, Italy, Morgan Kaufmann, San Francisco,
CA, 1987.
w8x T.P. Pridmore, D. Cooper, N. Taylor, Estimating camera
orientation from vanishing point location during sewer surveys, Autom. Constr. 5 1997. 407419.
w9x N. Taylor, P.A. Mallinder, The brittle hinge in masonry arch
mechanisms, Struct. Eng. 71 1993. 359366.
w10x N. Taylor, T.P. Pridmore, S. Fu, Automatic visual detection
of lateral junctions in sewers, Proceedings of the Institution
of Civil Engineers: Water, Maritime and Energy 130 1998.
5669.