Escolar Documentos
Profissional Documentos
Cultura Documentos
Research Paper
ABSTRACT1
KEYWORDS
Content audit is an evaluation method used to identify, Content audit, Digital Information Space, Content Assessment,
describe, quantify, and assess content quality of a website or of a Information Architecture, Typology.
larger information ecosystem (social media, web application,
newsletter, intranet, etc.). Despite the growing popularity of the
method in the last years, very little research has been conducted ACM Reference format:
on this topic. However, it is extensively described and I. Sperano 2017. SIG Proceedings Paper in word Format. In
commented in a large body of literature, mostly written by Proceedings of ACM SIGDOC conference, Halifax, Nova Scotia
information architecture (IA), content strategy and UX CANADA, August 2017 (SIGDOC 2017), 10 pages.
professionals. Hence, this is what led to further examine, in this DOI: 10.1145/3121113.3121227
research, a corpus of 200 publications (books, web pages, blog
articles, journal articles) addressing content audit. This study
attempts to take stock and establish a realistic picture of the 1 INTRODUCTION
current knowledge about the method, particularly concerning Evaluation of digital contents and information structures
content audit definition components and content audit types, as consists of assessing their quality and their ease of use. It has
well as to suggest possible means to further develop this several objectives: identifying opportunities to improve a
evaluation method. website, validating an existing structure, maximizing the profits
of an organization, facilitating the search for information, etc.
[1]. It may also support decisions such as choosing between two
CCS CONCEPTS proposals or justifying the value of an investment (website
• Human-centered computing → Human computer redesign, changes in the editorial process, etc.). Similarly,
interaction (HCI) → HCI design and evaluation methods evaluation is a valuable valid tool when it comes to allocating
→ Heuristic evaluations budgets to a project or justifying the creation of a new position
within an organization, for instance [1]. To do so, different
methods of evaluation are used, like user tests, heuristic
evaluation, card sorting, etc. Content audit is one of them.
responsible for establishing and maintaining a digital 2.1 Setting the Corpus
information ecosystem. It does so by establishing a common
The first step of this methodological approach is to set up a
terminology between members of the work team [9].
corpus of publications, which will constitute the disciplinary
knowledge base to code and analyze.
Despite the growing popularity of the method in the last
years, very little research seems to have been conducted on this
topic. Opening the way for research on content audit requires Selecting Publications
first to take stock of knowledge and practices related to this Because content audit has rarely been a subject of research, it
method. This is what led us to inventory and formalize current seemed necessary to embrace the whole discourse about this
practices, and to establish the fundamental knowledge base method, requesting to select a large number of publications.
related to content audit. It constitutes our entry point towards
the study of this method. This paper aims to present an in-depth Books were the first document type to be included in the
examination of content audit as a digital ecosystem assessment corpus. Knowledge about content audit has been developed not
method. only through more traditional media like books, but also by less
formal modes of diffusion such as blogs and other web
One of the major objectives of this paper is to define content publications. This is what led to extend the field of investigation
audit. It is meant to take a further look at the method, at its to web documents of varied types. In contrast, journal articles
experts, and particularly at its definitions and at its different and conference proceedings addressing content audit was added
types2. to include the research knowledge and perspectives in the
panorama. Thus, the identification of documents extends to
three types of publications:
2 METHODOLOGY • Book or chapter
• Web document (blog articles, web page, etc.)
Every discipline has its collection of knowledge included • Journal article or conference proceeding
within its practice. This knowledge is certainly reflected in
designed artefacts and products, but also in books, and articles In order to identify books to integrate in the corpus, a
addressing these topics [10]. Although content audit has not keyword search was conducted on popular online bookstores, as
really been looked at from a research perspective until now, it is well as in library catalogues. A search for keywords such as
extensively described and commented on in a large body of content audit, content inventory, and content matrix, both in
literature. This constitutes a potentially fertile ground to engage French3 and in English was conducted. The selection of web
deep reflection and a rigorous discussion about the method. documents posed a certain challenge much less present with
books: credibility of authors. As it is commonly known, almost
Moreover, such literature seems to occupy a prominent anyone can publish content on the web. It was feared that a
position in the IA, content strategy and UX community, and to search of web publications from major search engines would
exert a significant influence on the professional practice. Indeed, lead to publications of too variable and uncertain quality. A
Burford [11] studied IA professionals. In her study, she observed different means of selection was chosen. It was decided to use
IA professionals adopt several strategies to acquire knowledge in references cited in the book corpus as a base for the web
their field. Reading dedicated literature is an important one. documents search. Thus, the web document selection began by
identifying those cited in the book corpus. Then, by following an
For these reasons, the data was collected by an extensive iterative approach, newly added web publications were explored
literature search about content audit, resulting in a corpus of for other relevant documents until saturation (i.e. until no new
related publications that was then analyzed. This methodological document was identified). Five iterations were conducted before
strategy appeared as a relevant first step towards the reaching a saturation of the corpus. Finally, journal articles and
formalization of current knowledge about the content audit conference proceedings were identified using article database.
method. Thus, this kind of methodological strategy may enable
the depiction of a representative and realistic picture of the body
Corpus’s Composition
of knowledge published about the content audit method.
This process led to select a total of 200 publications. The
corpus is composed as follows:
2These results are part of a larger study conducted about content audit where other
• 117 books or chapters
dimensions of the method have been analyzed (audit activities, criteria, auditor’s
expertise, etc.).
3 The native language of the researcher is French.
2
Content Audit for the Assessment of Digital Information Space:
SIGDOC, August 2017, Halifax, Nova Scotia, CANADA
Definitions and Typology
Publication’s Designation
3.1 Overview of the Corpus
Every publication has been appointed a precise designation
following these nomenclature rules: A Relatively New Method
• First 3 letters of the author’s last name (e.g. ALL). The publication year of the documents was first identified
• Last 2 digits of the year of publication (e.g. 11). (see Figure 1). Considering such variable gives cues regarding
• Type of publication (e.g. W). the emergence of the method.
• O: Book
• C: Chapter
• W: Web Document
• A: Journal article/conference proceedings
For example:
Complete reference. Allen, R. (2011). ROT: The Low-Hanging
Fruit of Content Analysis. Meet Content. Retrieved at
http://meetcontent.com/blog/rot-the-low-hanging-fruit-of-
content-analysis/
Designation: ALL11W
First, these variables about each publication were As shown in the figure, content audit seems to be a relatively
documented: recent method. If some publications address content audit before
• Type of document (e.g. Book) the 2000s, it is not until 2010 that a rise in the number of
publications is observed. The oldest element of the corpus is
• Year of publication (e.g. 2014)
from 1996. This document, ALL96O, is about Director (a CD-
• Title of publication (e.g. The language of content
ROM software) and Lingo programming.
strategy)
• Country of origin (e.g. United States)
An American Point of View
• Authors (e.g. Abel, Scott)
Both the language and the origin of the publications (country
Then, an exploratory data analysis was conducted to get an where the document was published) were looked upon (see
overall picture of content audit. Hence, a coding activity was Figure 2).
3
SIGDOC, August 2017, Halifax, Nova Scotia, CANADA I. Sperano
4
Content Audit for the Assessment of Digital Information Space:
SIGDOC, August 2017, Halifax, Nova Scotia, CANADA
Definitions and Typology
LAN14O A content inventory is a quantitative assessment A Content Audit leverages the inventory to
of all the content on a website–a list of all the provide an assessment of the content and its
pages, images, and other files that make up the quality.
content set as well as data associated with those
files, such as content type and metadata.
WIKW-1 A content inventory is the process and the result
A content audit is a qualitative evaluation of a set of cataloguing the entire contents of a website. An
of content. When you audit content, you assess it allied practice — a content audit — is the process
against a variety of measures depending on your of evaluating that content.
context and goals. WOD09O A content inventory is a tally of everything that
exists on the site and everything you expect to be
added to the site.
YUN02O A content audit is simply a thorough analysis of
all content—text and graphics—on your website
with an eye on what should or should not be
localized.
concept of content evaluation also appears in the content audit Audit types Number of publications,
definitions [LEW12O, DET12W, KAD12O, LAN14O, LEW12O, % of publications
NICW, WIKW-1]. In addition, the use of criteria, of measures (n=200)
to assess content is frequently explicitly stated [LAN14O,] or
Facet
implied [BRO06O, HAL12O].
Quantitative Audit 13 (6.5 %)
It can be noted that some definitions address the area of Qualitative Audit 13 (6.5 %)
investigation of the content audit. Multiple definitions Multidimensional Audit 3 (1.5 %)
restrict the scope of the content audit to websites [BRO06O,
Good Practice Audit 2 (1%)
KAD12O, WOD09O, HAL12O, DET12W, LAN14O, WIKW-1,
YUN02O]. However, a few state that the scope of the audit can Strategy Audit 2 (1%)
be extended beyond the website and consider the contents of ROT Audit 1 (0.5 %)
other information channels [LEW12O, NICW]. (Redundant, Obsolete, Trivial)
Audit CLOUT 1 (0.5%)
Some definitions distinguish content audit and content
Legal Audit 1 (0.5%)
inventory [LAN14O, LEW12O, NICW]. Among those that
distinguish them, there seems to be a tendency to say that Internationalisation Audit 1 (0.5 %)
content inventory is mainly quantitative and content audit Efficacy Audit 1 (0.5 %)
mainly qualitative [LAN14O, NICW]. Visual Audit 1 (0.5%)
With regard to the nature of the content, it can be both Area of Investigation
textual and graphic formats [LAN14O, LYO12O, YUN02O,
Competition Audit 3 (1.5%)
LEW12O]. If most publications indicate that the audit focuses on
the current content, some add that content to add should also Mobile Audit 1 (0.5 %)
be considered in the content audit [WOD09O]. It is interesting to Multichannel Audit 1 (0.5 %)
note that it is sometimes described both as a process and as a Community Audit 1 (0.5%)
result [WIKW-1]. Indeed, the result of a content audit process,
according to [WIKW-1], would also be called a content audit. Content Scope
Full Audit 6 (3%)
Content mapping 4 (2%)
3.3 Content Audit Types First-Level Audit 3 (1.5%)
It was noticed that content audit can be divided in different Key-Content Audit 3 (1.5%)
types. Indeed, 36 publications (18% of publications) of the corpus
Content Sampling Audit 3 (1.5%)
distinguished content audit types. A total of 23 content audit
types were identified (see Table 2). However, while analyzing the Section Audit 1 (0.5%)
results, it was noticed that the different audit types found did not
Number of Criteria
seem to stand on equal footing. A simple inventory of every
audit type may have been useful, but seemed incomplete. It was Brief Audit 2 (1%)
therefore decided to thematically group audit types to facilitate
Moment of Realization
understanding, as well as to gain better insights from these
results. The grouping is organized according to 5 dimensions: Rolling Audit 10 (5%)
facet, area of intervention, content scope, number of criteria and
moment of realization of the method. Dimensions have been
Table 2—Content Audit Types
determined according to the most preeminent characteristics of
the audit types identified.
The determination of a type of audit will vary depending on
the objectives to be achieved through the audit of content, as
WAC13W says:
[y]ou could audit content for all kinds of things,
depending on what you want to learn and be able
to do with the information. [WAC13W]
6
Content Audit for the Assessment of Digital Information Space:
SIGDOC, August 2017, Halifax, Nova Scotia, CANADA
Definitions and Typology
7
SIGDOC, August 2017, Halifax, Nova Scotia, CANADA I. Sperano
You thought we were done with the content design at Laval University may wish to identify all contents
audit? Not just yet. The tool that you used to related to the master of interaction design (MID) program. In this
identify variety and inconsistency can help you case, he would audit the contents of the school of design website,
continue to rein it in after launch. A rolling audit but also the content of its social networks as well as the parent
builds on the initial audit to ensure the and child sites looking specifically for content addressing this
comprehensive, current view of pages or screens subject. The auditor could also review content about MID on
and the content of them remains complete and every Laval University website or even other digital content
accurate. [BLO12O] addressing this subject (e.g. newspaper articles, alumni LinkedIn
page).
ROSW06, on the other hand, suggests undertaking an audit
“continuously”, naming this audit Rolling Audit. Several authors Such a content audit might be interesting for the
thereafter suggested this type of audit. improvement of a particular content. Indeed, a better
understanding of what is being said about the MID would
This is what completes the description of audit types. It is provide a potentially interesting overview of the treatment of
important to note that the categories are not mutually this subject, and then highlight the shortfalls, the inconsistent
exclusive. Indeed, it would be quite possible to undertake a good content, and the possible development tracks for this content.
practice audit (facet) of first-level content (scope of content),
considering mostly mobile users (area of investigation), for
User’s Journey Through the Information Space:
instance.
The Path Content Audit
Fragmented Knowledge Interacting with information will induce the development of
It is interesting to note that only three types of audit are a mental representation of the system by the user [13] which, if
listed in more than 10 publications (in 5% of publications or well done, will facilitate understanding. Considering the
more), namely quantitative audit, qualitative audit and rolling potential user’s routes through the information space is
auditing. therefore crucial to the design of information ecosystems [14].
Indeed, other types of audits are suggested by only few Hence, it seems another audit type could be added, where
authors. content assessment sequence could be taken into account.
This observation shows that the knowledge is scattered Namely, a content audit according to the user’s journey through
through publications. Although each audit type is almost the information space could be an interesting audit type.
exclusive to a single publication, their combination could be seen
as a first step towards a more complete picture regarding Indeed, it could be possible to determine an order of
possible audit types. This could also guide the audit managers assessment of the pages (or content) considering real or possible
and the auditors to make more informed choices. paths of users through the site. One could think of an analysis
sequence based on predetermined routes, which considers
frequent users' tasks, for instance. What could be called a Path
Developing New Audit Types Content Audit (which would assess the content based on
If the consolidation of audit types suggested through the potential user paths through the structure) could be
corpus provides an overview of different audit possibilities, this implemented and added to possible content audit types. This
list may not be exhaustive. In this paper, we suggest two new content audit type could help identify problems related to
audit types: The Specific Subject Content Audit and The Path wayfinding or content granularity, for instance.
Content Audit.
Assessing other Content Channels
Improving a Particular Content: Specific Subject As for the content audit definitions presented above, one can
Content Audit see that content audit types seem to be mainly website focused
(except for the ones under the Area of Investigation category).
It seems relevant to add an audit type related to a particular This low occurrence of social media, as well as other digital
subject or content feature, but which is located in more than one channels seems to show a more traditional approach to digital
location. It could be named Specific Subject Content Audit. For information practices. More recent approaches take a more
example, an auditor working on the website of the school of holistic point of view which includes all the channels of an
9
SIGDOC, August 2017, Halifax, Nova Scotia, CANADA I. Sperano
organization [15]. This is what leads us to suggest that new audit [7] Pernice K. 2004. Content Migration Alone Is Not An Effective Content
Strategy, Alertbox, 2015.
types exploring other information channels could be added. [8] Holzschlag M. 2004. 250 HTML and Web Design Secrets. Indianapolis, IN:
John Wiley & Sons.
[9] Detzi C. 2012. From Content Audit to Design Insight, UX Magazine, 2012.
[10] Hobbs J., Fenn T., and Resmini A. 2010. Maturing a Practice, Journal of
4 CONCLUSION Information Architecture, vol. 2, no 1, p. 37‑54.
[11] Burford S. 2014. A Grounded Theory of the Practice of Web Information
Proper use of discipline-specific methods with a certain Architecture in Large Organizations, Journal of the Association for
degree of formalization contributes to the development of an Information Science and Technology, p. 1‑18.
[12] Leray C. 2008. L’analyse de contenu: de la théorie à la Pratique, la Méthode
expertise [4]. Based on this premise, this paper focused on one Morin-Chartier. PUQ.
particular method used for assessing content quality: content [13] Dambreville S. 2008. Définir la structure de navigation : quelques outils
méthodologiques. In Ergonomie des documents électroniques, A. Tricot and
audit. By highlighting the characteristics of a content audit, this A. Chevalier, Éd. Paris: Presses Universitaires France, p. 159‑179.
study attempts to take stock and establish a picture of the [14] Malik S. 2014. Mapping User Journeys Using Visual Languages, UXmatters.
current knowledge about the method, regarding definitional [15] Resmini A. and Rosati L. 2011. Pervasive Information Architecture:
Designing Cross-Channel User Experiences. Burlington, MA: Morgan
aspects of content audit. Kaufmann.
[16] Resmini A. and Instone K. 2010 Research and Practice in IA, ASIS&T Bulletin.
This research led to produce an unprecedented collection of
knowledge about content audit. It allowed to distinguish certain
trends, both about the method and its experts. It also provided a
factual basis for the method, rather than a series of anecdotal
comments, as is often the case in fields such as UX design,
information architecture and content strategy[10].
REFERENCES
[1] Toub S. 2000. Evaluating Information Architecture, Argus Center for
Information Architecture, 2000.
[2] Jones C. 2010. Clout: The Art and Science of Influential Web Content.
Berkeley, CA: New Riders.
[3] Land P. 2014. Content Audits and Inventories: A Handbook. Laguna Hills,
CA: XML Press.
[4] Martin B. and Hanington B. 2012. Universal Methods of Design: 100 Ways to
Research Complex Problems, Develop Innovative Ideas, and Design Effective
Solutions. Beverly, MA: Rockport.
[5] Halvorson, K. and Rach M. 2012. Content Strategy for the Web, 2e édition.
Berkeley, CA: New Riders Press.
[6] Bloomstein M. 2012. Content Strategy at Work Real-World Stories to
Strengthen Avery Interactive Project. Waltham, MA: Morgan Kaufmann.
10