DATA QUALITY AND ITS ROLE IN DECISION MAKING

TONY GILL
GradCertPubSecMgmt, GAICD

Department of Lands and Planning

GPO Box 1680, Darwin NT 0801
Tel: +61 8 8995 5317
Email: tony.gill@nt.gov.au

Abstract
As more organisations, government and non-government, embrace spatial
information and technologies, greater attention needs to be placed on the role and
importance of authoritative data sources. The NT Government’s Spatially Enabling
Government (SEG) initiative will further drive the discussion within Government.
However, awareness and the significance of the issue must be brought into the
public domain.
This paper highlights the need for greater focus on the role of authoritative spatial
datasets in decision making by government, industry and the public and argues the
need for data custodians of authoritative spatial data sources to place greater
attention on transparent quality assurance practices. Questions posed by this
paper and their implications include; what is the role of quality assurance in the
creation and maintenance of spatial data, should we be demanding it of spatial
data suppliers, what value will it add and what are the risks in not achieving quality
assurance certification?
With the emergence of a range of alternative spatial data sources, the case for a
dataset certification regime is put forward as a mechanism whereby custodians of
authoritative spatial data sources can stand out in the growing information market
place.

Spatially Enabling Government

Spatially Enabling Government (SEG) is a Cabinet endorsed initiative of the NT
Government with the objective of providing a whole-of-government (or as
equated by Althaus, C & Tiernan (2005) as joined-up-government) approach for
the enhancement of Government services through the provision of underlying
spatial information and associated services.
SEG includes a range of activities, not all fully funded, including;
• Establishing location and street addressing services
• Improved coverage, currency and quality of imagery and mapping
• Provision of satellite-based positioning infrastructure and facilities
• Assistance to agencies to spatially enable their information and business
operations, and
• Provision of a governance framework for the realisation of SEG benefits

1
The vision of SEG is to generate significant opportunities to deliver integrated
services to the community, but to also become an enabling component of
Joined-up Government.
The NTG’s ICT Strategic Intent 2010-15 document (NTG, 2009) highlighted the
significance of information in the decision making process when it stated
“Decisions are based on information and knowledge” and as a consequence
“information must be easy to access securely, accurate, reliable, timely and
usable”. Without achieving these attributes, there is a risk in policy and
operational decisions being made that are not only flawed and costly, but could
put lives at risk.
Government, industry and the community are in a spatially or location enabling
phase. There is recognition that everything happens somewhere and as Tony
Maber (MDS, 2011) states “Organisations not using the location element of
retained or acquired business information are missing out on a valuable
ingredient when making decisions”.
Maber claims “A picture paints a thousand words, but a map tells the whole
story!” (MDS, 2011). However, explorer Captain James Cook, found time and
again that maps of his time were created based on myth, misinformation or just
plain poor information rather than fact or carefully observed measurement (Film
Australia, 2007). Without Captain Cook correcting the maps of the day many
decisions of the time may have had disastrous consequences for mariners and
business men.
In a whole-of-government approach, solving complex issues like Closing The
Gap of indigenous disadvantage require sound sources of information that span
agencies and levels of government. Key to this is ensuring we start from a
sound base of good quality authoritative information.

Origins of quality and its links to spatial information

Dr. Joseph M. Juran’s book, Managerial Breakthrough in 1964 led to quality
management and improvement systems like Lean and Six Sigma (Juran, 2011).
The American Society for Quality (ASQ) was formed in 1946 as the American
Society for Quality Control and now self-describes itself as the “champion of the
quality movement” (ASQ, 2011), of which Juran was made an Honorary
Member in 1968.
While ISO 9000 has its roots from British standards (Wikipedia, 2011b), in the
early 1980’s in the USA, Government and industry, saw the need to focus on
quality as part of doing business in an expanding competitive global
marketplace (NIST, 2010) and enacted legislation, The Malcolm Baldrige
National Quality Improvement Act of 1987 that established the Malcolm Baldrige
National Quality Award “to promote awareness of performance excellence as an
important element in competitiveness … that would help U.S. companies
achieve worldclass quality” (NIST, 2010). In 2001, the U.S. National Institute of
Standards and Technology (NIST) commissioned an evaluation of the net
benefits of the Baldrige Program and extrapolated entire economy benefits of
USD $24.65 billion (year 2000 present value) based on USD $2.17 billion (year
2000 present value) for members of the ASQ (NIST, 2001). The evaluation has

2
demonstrated linkages between the implementation of quality initiatives to the
organisation and the nation.
The ISO 19000 series of standards for Geographic Information contains a range
of standards that relate directly to the field of spatial data quality. This includes,
among others, ISO 19113:2004 (Quality Principles), ISO 19114:2005 (Quality
Evaluation Procedures), ISO 19138:2008 (Data Quality Measures), ISO
19115:2005 (Metadata). This represents a relatively comprehensive set of
standards for the management of spatial information and the recording and
measurement of its quality at the dataset/product level.
The draft ISO 19158 (Quality Assurance of Data Supply) standard is currently
being worked on and is expected to be published in April/May 2011 (Body,
2011). It recognises the role of ISO 9000:2005 as establishing the principles of
quality management and other ISO 19000 series standards for the data quality
reporting and evaluation.
The Open Geospatial Consortium (OGC) is a consensus-based organisation of
industry, government agencies and universities with the aim of developing
interface standards to support interoperable vendor independent geo-enabled
solutions (OGC, 2011). OGC have recognised the need to ensure that these
interoperable systems are capable of exchanging information on data quality
and have formed a Data Quality Working Group (OGC, 2011b).

What defines quality?

The Government of British Columbia defines data quality as being “The state of
completeness, validity, consistency, timeliness and accuracy that makes data
appropriate for a specific use” (Wikipedia 2011) and aligns well with the NTG
ICT Strategic Intent.
The standard on Geographic Information – Quality Principles (AS/NZS ISO
19113:2004) raised a key issue that challenges the specific use qualifier, when
it acknowledges the increasing tendency for geographic datasets to be “shared,
integrated and used for purposes other than their producer’s intended ones”.
As such, quality becomes a subjective assessment by the consumer of the data
as to its fitness for use for a particular purpose that may be outside the original
intended use of the data by the producer.

The significance of Authoritativeness in quality

Authoritative is defined as “arising or originating from a figure of authority” or
“highly accurate or definitive” (Wikitionary, 2011) and authoritativeness is
defined as “the quality of possessing authority” or “the quality of trustworthiness
and reliability” (Wikitionary, 2011b).
All of these descriptors could easily be applied to cadastral datasets produced
and overseen by the respective Surveyors-General in each State and Territory
jurisdiction of Australia. This comes to being because of the Torrens Title
System whereby the Government, through the Registrars-General and
Surveyors-General guarantee indefeasibility of title. This provides a high level of
confidence to the public, financial institutions and industry in the accuracy and

3
reliability of the cadastre. That is not to say that the cadastre does not and will
not be “inaccurate” relative to other measures at times and the consequences
that government must deal with in these circumstances.
However, it is not always as clear for many other datasets. Take for instance
topographic data. What is it that makes one topographic dataset more
authoritative than another? Both may well have observed the XYZ of a specific
physical feature on the earth’s surface, but have recorded different values both
within acceptance tolerances of the techniques employed. Contours produced
from a DEM of LIDAR points with 1.0m spacing may well have individual Z
accuracies of +/- 0.15m will not concur with contours produced from a more
sparsely populated DEM with observed breaklines from photogrammetry with Z
accuracies of +/- 0.30m. Which dataset is more accurate is open for debate as it
really depends on the intended use and whether the advantages and limitations
of one technique override the other.

Trust – Is it real or imagined?

David Schell, then Chairman and CEO of Open Geospatial Consortium, stated
“there is a tremendous need to pay attention to liabilities, that is the risks
associated with providing data and services that may figure prominently in
important decisions affecting lives and property” and that “digital spatial
information is a matter of public trust and we need to recognise this” (GIM,
2008). Trustworthiness and reliability are at the heart of authoritativeness and
for many government produced datasets this is assumed by the community of
users.
Unlike private enterprise, Governments are enduring entities and as Schell
points out, liability must be managed and as a result a level of trust is inherently
placed in government produced datasets. This attention to liabilities makes
government slow at responding to user requirements or paradigm shifts
compared to their private enterprise counterparts.
Private enterprise however, trades in its “Reputation Capital” (Van der Vlugt,
2011). The purpose of a private enterprise organisation is to be profitable to its
owners and not necessarily to be enduring, although company structures
enable it to be so. For private enterprises whose stock in trade is services, be it
professional or information, reputation is everything. It is the fundamental asset
to the organisation and is most recognisable through goodwill on a company’s
balance sheet. A change in reputation has direct linkages back to the underlying
value of the enterprise. The goodwill associated with a company makes it
attractive to other companies and individuals and provides an exit strategy for
its owners. Therefore, maintaining the reputation capital is vital to the owner’s
interests. Rebuilding a damaged reputation is costly and sometimes terminal to
the organisation such as the case of Pan Pharmaceuticals Limited’s eventual
collapse as a consequence of the Therapeutic Goods Administration’s
suspension of its license “because of serious concerns about the quality and
safety of products manufactured by the company” (TGA, 2003).

Elements of decision making

Kohl (2010), defines the decision making process as having six steps;

4
 1. Identify and define the problem
2. Gather and analyse the information
3. Develop alternative solutions
4. Choose the best alternative
5. Take Action
6. Evaluate the decision.
Importantly, Kohl emphasises the critical need to “have accurate information to
solve issues” in the decision making process and that if you “short-change this
part of the process, you can create unnecessary delays and unintended
results”.

What is the role of quality assurance in the creation and maintenance of

spatial data?
The substitution of unauthoritative or unknown quality data can lead to poor
decisions that at best may be an inconvenience and at worst life threatening to
those relying on the subsequent decision.
In the case of the Gretley Mine Disaster (DPI, 1998), his Honour Acting Judge
Staunton, in his Summary of Findings (DPI, 1998b, pg. 41), identified a root
cause of the water in-rush that killed 4 miners as a failure by the Department to
properly interpret a plan in compiling two other plans, which were subsequently
relied upon by others in the compilation of other documents such as the mine
plan.
The Government’s response to the inquiry, draws on evidence from the inquiry
“that the surveyors who worked at the Gretley mine implicitly believed the plans
of the old workings were accurate because they were held by the Department”
(DPI, 1998c, pg. 3), even though the Department maintained the view it was
“solely the repository for records”.
While these tragic consequences should be few and far between, it highlights
the significance of quality assurance practices in the management of
information as well as understanding the lineage of information that is compiled
from other sources that are subsequently relied upon.

Should we be demanding quality assurance of spatial data suppliers?

NT Government tenders request tenderers to indicate if they have a
documented Performance Plan and on which standard is it based (e.g. ISO
9000:2000, or other). The purpose behind this is to understand if the tenderer
has sufficient processes in place to ensure that the requested product or service
is delivered to specification and where it is not then there are processes in place
to correct the defect and minimise its future occurrence, thus reducing the
potential risk to the acquirer of the products and services.
In comparison, Government in it liability management modus operandi, attempts
to use disclaimers to absolve itself from anything but a duty of care through
statements like “does not warrant this product or any part of it is correct or

5
complete and will not be liable for any loss, damage or injury suffered by any
person as a result of its inaccuracy or incompleteness” (DLP, 2011).
If quality assurance practices are something government demands of its
suppliers, why is it that Government cannot embrace the same standards when
it is a provider of spatial product and services?

What value will it add and what are the risks in not achieving quality
assurance certification?
Public trust and reputation capital are crucial elements in an end-users ability to
do their day-to-day work without regard to the underlying soundness of the
information on which they rely to make decisions. The Gretley mine disaster
brings home how fragile and false this blind trust can be.
A functioning and effective quality management system is both an insurance
policy on an organisation’s reputation capital and as evaluated by NIST adds
value to the organisation.
When procuring products and services, government aims to get value for
money. This is not a simple equation of cost versus time, but recognition that
quality is part of the equation. An organisation that has effective quality
management systems is more than likely able to contain the cost and time
elements while producing a quality outcome.

Google embraces User Generated Content (Crowd Sourcing)

9 November 2010 was a significant date in the spatial world as this is when
Google announced (Google, 2010) they were moving from the widely accepted
authoritative PSMA Australia (PSMA) data source for cadastral and addressing
information. The PSMA data is aggregated on a quarterly basis from all of the
jurisdictions in Australia, Australia Post and the Australian Electoral
Commission to produce CADLite™ and GNAF™ that is released to value added
resellers and on-licensed to organisations such as Google for use in a range of
web-based map and location services. Google have licensed a one-off copy of
non-authoritative datasets from the Telstra subsidiary Sensis (SpatialSource,
2011).
Interestingly, numerous places in the Northern Territory can be found to have
inaccuracies in Google Maps, some of which end-users would not even know
were problematic. Examples include non-current parcels in Tennant Creek
(Google 2011b) to a non-authoritative cadastre misaligning with road
centrelines (Google, 2011c). It is questionable how many users will be even
bothered to report the problem using the “Report a problem” link that requires
the user to go through a registration process and for Google to fix it.
This is a very interesting development for Google. They have built a reputation
of trust in delivering the search results that their user base expects within the
first page. When their user base starts to question the accuracy
(trustworthiness) of their map data, then Google runs the risk of taking a hit on
their reputation capital. As most internet based businesses rely on visitation
rates, as the user base declines, so does the market value and revenue
generation capacity of the organisation.

6
In 2007, Edelman questioned the end of MySpace (Edelman, 2007). In 2011
MySpace announced it was closing operations in Australia, UK and Germany as
it faces stiff competition for its user base from companies like Facebook
(TechGeek, 2011). Edelman made an interesting point that MySpace had “built
on an unsustainable foundation. They’ve made the classic gamble that short-
term gain will trump long-term stability … MySpace is headed for a big, clumsy
fall”.
As Edwin Haverkamp of MapData Sciences, explains “spatial elements in data
are the glue that binds information together … You need good quality and
frequently updated mapping data to make critical business decisions and every
day planning” (MDS, 2011). Time will tell whether Google are able to persist
with User Generated Content or whether they return to using an authoritative
and regularly maintained source of data that their user base can be confident in
using to base decisions on.

Authoritative data sources – standing out from the crowd

The question remains. How do authoritative data sources stand out among the
ever crowded marketplace for spatial information?
As more and more web-based mapping services (e.g. Google Maps, Microsoft
Bing Maps) and various mashups appear, it is becoming harder for the end user
to get a sense of the quality of the underlying data.
ISO 9000 Certification is proudly emblazoned across company letterheads and
documents for a reason. It demonstrates to it clients, current and prospective,
as wells as its competitors, that it takes quality seriously and that its makes
business sense.
The time is rapidly approaching whereby spatial data suppliers achieve ISO
9000 certification as well as full compliance with the ISO 19000 standards
relating to quality to not only insure their reputation capital but also as a
mechanism to manage their liabilities.
There is even a case for the spatial data suppliers to embrace a certification
regime that is able to recognise the authoritativeness of a dataset as one of the
qualities that define it. Once this is achieved, the OGC Data Quality Working
Group can then provide a mechanism by which online mapping and mashups
can signify the authoritativeness of the various datasets being displayed in a
similar way that web page certificates are issued and are able to be queried by
the end-user.

Conclusion
Data quality is a key component of the decision making process. Without
understanding data quality poor, costly and sometimes tragic consequences
can result. Organisations that do not embrace a quality management system
are likely to suffer from a loss of reputation and as a result may not survive in
the longer-term. Users need to know whether they can trust the spatial
information they are using and a mechanism needs to be found that provides
them with a level of confidence that decisions they are about to make using
spatial information are foundered on good quality reliable data.

7
References
Althaus, C & Tiernan, A 2005, Managing Out: The Public Sector in the
Community, PSM Program 2005, Australian Public Service Commission,
Canberra
ASQ 2011, Who We Are, American Society for Quality, viewed 27 Jan 2011,
<http://asq.org/about-asq/who-we-are/index.html>
Body, C 2011, Geospatial Standards Architect and Chair IT-004 Geographic
Information Committee, Standards Australia, pers. comm., Jan 2011
DLP 2011, Standard Map Product Disclaimer, Land Information Division,
Department of Lands and Planning, Northern Territory Government
DPI 1998, Mine Safety - Gretley Inquiry Summary, Department of Primary
Industry, New South Wales Government, viewed 10 Jan 2011,
<http://www.dpi.nsw.gov.au/__data/assets/pdf_file/0004/87160/Gretley-
Inquiry-summary.pdf
DPI 1998b, Mine Safety - Gretley Inquiry, Department of Primary Industry, New
South Wales Government, viewed 10 Jan 2011,
<http://www.dpi.nsw.gov.au/minerals/safety/mine-safety-initiatives/gretley-
inquiry>
DPI 1998c, Mine Safety – Government Response, Department of Primary
Industry, New South Wales Government, viewed 10 Jan 2011,
<http://www.dpi.nsw.gov.au/__data/assets/pdf_file/0003/87159/Gretley-
Inquiry-Govt-response.pdf>
Edelman, D 2007, The End of MySpace, viewed 2 Feb 2011,
<http://www.davidlouisedelman.com/technology/end-of-myspace/>
Film Australia 2007, Captain Cook: Obsession and Discovery, Film Australia
GIM 2008, A New Age II, GIM International, February 2008
Google 2010, Changing World, Changing Maps, Google, viewed 4 Jan 2011,
<http://google-latlong.blogspot.com/2010/11/changing-world-changing-
maps.html>
Google 2010b, Google Maps – Tennant Creek, Google, viewed 2 Feb 2011, <
http://maps.google.com.au/maps?f=q&source=s_q&hl=en&geocode=&q=t
ennant+creek&sll=-
25.335448,135.745076&sspn=65.081593,60.029297&ie=UTF8&hq=&hne
ar=Tennant+Creek+Northern+Territory&ll=-
19.654935,134.196432&spn=0.017318,0.014656&z=16>
Google 2010c, Google Maps – Katherine, Google, viewed 2 Feb 2011, <
http://maps.google.com.au/maps?f=q&source=s_q&hl=en&geocode=&q=k
atherine&aq=&sll=-
14.231523,129.522028&sspn=0.071299,0.058622&ie=UTF8&hq=&hnear=
Katherine,+Northern+Territory&ll=-
14.468796,132.292452&spn=0.002226,0.001832&z=19>

8
Juran 2011, About Juran – Our Founder, Juran Institute, viewed 27 Jan 2011,
<http://www.juran.com/about_juran_institute_our_founder.html>
Kohl, Tammy A.S. 2010, The Principles of Decision Making, viewed 31 Jan
2011, <http://www.goodezines.com/ezinearticles/the-principles-of-
decision-making/>
MDS 2011, What’s Happening in Mapping – Issue #48, MapData Sciences,
viewed 31 Jan 2011,
<http://www.mapds.com.au/Newsletters/newsletter/1101/1101.asp>
NIST 2001, Economic Evaluation of the Baldrige National Quality Program,
National Institute of Standards and Technology, viewed 27 Jan 2011,
<http://www.nist.gov/director/planning/upload/report01-3.pdf>
NIST 2010, Baldrige FAQs, Baldrige Performance Excellence Program,
Frequently Asked Questions, National Institute of Standards and
Technology, viewed 27 Jan 2011,
<http://www.nist.gov/baldrige/about/baldrige_faqs.cfm>
NIST 2010b, The Malcolm Baldrige National Quality Improvement Act of 1987 –
Public Law 100-107, National Institute of Standards and Technology,
viewed 31 Jan 2011,
<http://www.nist.gov/baldrige/about/improvement_act.cfm>
NTG 2009, NTG Information Communication Technology – Strategic Intent
2010-2015, June 2009, Version 0.7, ICT Policy and Strategy Branch,
Department of Business and Employment, Northern Territory Government,
viewed 27 Jan 2011,
<http://www.territory2030.nt.gov.au/doc/feedback/45NTGIMC_Attachment
1.pdf>
OGC 2011, About OGC, Open Geospatial Consortium, viewed 1 Feb 2011,
<http://www.opengeospatial.org/ogc>
OGC 2011b, Data Quality Working Group, Open Geospatial Consortium,
viewed 1 Feb 2011,
<http://www.opengeospatial.org/projects/groups/dqwg>
SpatialSource 2011, Sensis comments on Google Development, SpatialSource,
viewed 2 Feb 2011,
<http://www.spatialsource.com.au/2011/01/25/article/Sensis-comments-
on-Google-development/YLZVBBGXGV>
TechGeek 2011, MySpace to close operations in Australia, axes 500 jobs,
TechGeek.com.au, viewed 2 Feb 2011,
<http://techgeek.com.au/2011/01/12/myspace-to-close-operations-in-
australia-axes-500-jobs/>
TGA 2003, Pan Pharmaceuticals Limited – Regulatory Action and Product
Recall Information, Therapeutic Goods Administration, Department of
Health and Aging, Australian Government, viewed 31 Jan 2011,
<http://www.tga.gov.au/recalls/2003/pan.htm>

9
Van de Vlugt, Maurits, 2011, LinkedIn Poll – How do you know if a data source
you are using is authoritative?, LinkedIn Poll Comment,
<http://polls.linkedin.com/vote/116805/pfdby>
Wikipedia 2011, Data Quality, viewed 27 Jan 2011,
<http://en.wikipedia.org/wiki/Data_quality>
Wikipedia 2011b, ISO9000, viewed 31 Jan 2011,
<http://en.wikipedia.org/wiki/ISO_9000>
Wikitionary 2011, Authoritative, viewed 27 Jan 2011,
<http://en.wiktionary.org/wiki/authoritative>
Wikitionary 2011b, Authoritativeness, viewed 27 Jan 2011,
<http://en.wiktionary.org/wiki/authoritativeness>

10