Tese Doutorado Dusse-FINAL

Universidade Federal da Bahia
Instituto de Matemática e Estatı́stica
Programa de Pós-Graduação em Ciência da Computação
A COMPUTATIONAL REFERENCE MODEL

TO SUPPORT DECISION-MAKING FOR
EMERGENCY MANAGEMENT BASED ON
VISUAL ANALYTICS
Flávio Dusse
TESE DE DOUTORADO
Salvador-BA
15 de Dezembro de 2020
FLÁVIO DUSSE
A COMPUTATIONAL REFERENCE MODEL TO SUPPORT

DECISION-MAKING FOR EMERGENCY MANAGEMENT BASED
ON VISUAL ANALYTICS
Esta Tese de Doutorado foi

apresentada ao Programa de
Pós-Graduação em Ciência da
Computação da Universidade
Federal da Bahia, como requisito
parcial para obtenção do grau de
Doutor em Ciência da Computação.
Orientador: Prof. Dr. Manoel Gomes de Mendonça Neto

Co-orientador: Prof. Dr. Renato Lima Novais
Salvador-BA
15 de Dezembro de 2020
Sistema de Bibliotecas - UFBA
Dusse, Flávio.
A COMPUTATIONAL REFERENCE MODEL TO SUPPORT
DECISION-MAKING FOR EMERGENCY MANAGEMENT BASED ON
VISUAL ANALYTICS / Flávio Dusse – Salvador-BA, 2020.
155p.: il.
Orientador: Prof. Dr. Prof. Dr. Manoel Gomes de Mendonça Neto.

Co-orientador: Prof. Dr. Prof. Dr. Renato Lima Novais.
Tese (Doutorado) – Universidade Federal da Bahia, Instituto de
Matemática e Estatı́stica, 2020.
1. Computational Model. 2. Visual Analytics. 3. Information

Visualization. 4. Emergency Management. I. Mendonça, Manoel Gomes.
II. Novais, Renato Lima. III. Universidade Federal da Bahia. Instituto de
Matemática e Estatı́stica. IV. Tı́tulo.
CDD – XXX.XX
CDU – XXX.XX.XXX
Para minha filha, Maria Eduarda Polonio Ferreira Castro
Dusse, meus pais Roberto Dusse e Izabel Xavier Dusse
e minhas avós in memorian Maria Ferreira Franco e
Hermı́nia Mariotti.
ACKNOWLEDGEMENTS
Agradeço a Duda, por ter vindo a esse mundo me mostrar o que é a vida, o que é amar
e ser amado, o que é amor incondicional que tanto os poetas falam.
A minha famı́lia, amo vocês desde que nasci.
A um amigo que faz uma falta danada, Guilherme (‘veio’ Xoura).
A uma pessoa que mudou vários conceitos da minha vida e me fez ser um ser humano
infinitamente melhor, Wal (Tita).
A minha melhor amiga, que passou a ser segunda depois da Duda, Nakita (Naky).
Aos meus colegas da UFBA, nas duas fases que estive lá de 2000 a 2004 (Rhuddis) e
agora de 2014 a 2020 (Softvis).
Aos meus amigos do Lote de Fela, principalmente Vitor (BA), os demais sintam-se
representado por essa figuraça.
Aos meus amigos da República Casa Véia, principalmente Petrus (Petin), os demais
sintam-se representado por essa outra figuraça.
Aos meus amigos do Labes e do Maiti, principalmente Gabriel (Ceará), os demais
sintam-se representados por mais uma figuraça.
Aos meus amigos(as) das escolas, das faculdades, dos trabalhos, das vizinhanças, das
redes sociais, do futebol, de Salvador, Feira, Recife, Água Preta, João Pessoa, Esperança,
Aracaju, Maceió, São Paulo, São Bernardo, São Carlos, Belo Horizonte, Guarapari,
Canadá, etc. Aqui a lista é muito grande, se citar vou sem querer esquecer de um ou
outro, mas vocês sabem quem são. Vocês são os irmãos e as irmãs que escolhi na Terra.
Aos meus orientadores, Manoel e Renato, pela parceria, experiência e paciência.
A Universidade Federal da Bahia e a CEAPG, por toda prestação e conhecimento
compartilhados. A CAPES, pelo fomento da minha pesquisa.
As professoras Vaninha Vieira, Lais Salvador e aos professores Marcos Borges e José
Carlos Maldonado pela disponibilidade em avaliar a tese e por suas contribuições para
melhoria da versão final.
Aos colegas de PGCOMP: Gau, Thiago Mendes, Paulo, Jorge, Luis Paulo, Jaziel,
Cretchas, Tamires, Katti pela cooperação.
Aos diversos professores, alunos, especialistas (policiais, bombeiros e agentes de
emergência) que tive contato durante essa pesquisa.
Meu muito obrigado a todos.
vii
Regardless of the staggering dimensions of the world about us, the
density of our ignorance, the risks of catastrophes to come, and our
individual weakness within the immense collectivity, the fact remains that
we are absolutely free today if we choose to will our existence in its
finiteness, a finiteness which is open on the infinite. And in fact, any man
who has known real loves, real revolts, real desires, and real will knows
quite well that he has no need of any outside guarantee to be sure of his
goals; their certitude comes from his own drive.
—SIMONE DE BEAUVOIR
RESUMO
Contexto: Gestão de Emergência (GE) refere-se à capacidade de lidar com tarefas

relacionadas a emergência em diferentes fases e iterações. Quem trabalha em uma
emergência geralmente está sob estresse para tomar a decisão certa no momento correto.
Essa pessoa precisa processar uma quantidade muito grande de dados e assimilar as
informações recebidas de forma intuitiva e visual. A sobrecarga de informação, assim
como a falta de certos tipos de informação, são problemas na GE. Visual Analytics
(VA) é potencialmente útil em GE, contudo o seu potencial não está sendo aproveitado.
Objetivo: Este trabalho desenvolveu um modelo de referência computacional para o uso
de VA na fase de preparação da GE. Este modelo incorpora temas atualmente pouco
explorados na área, para melhor apoiar a tomada de decisões em EM. O modelo visa
ajudar os designers de VA a criar interfaces mais eficazes que, por sua vez, ajudarão
os gerentes de emergência a tomar decisões mais rápidas e assertivas no seu dia a
dia. Métodos: Esta pesquisa conduziu um longo estudo in-vivo com vários métodos.
Primeiramente, foi realizado um mapeamento sistemático para conhecer as ferramentas
de visualização de informação disponı́veis e suas aplicações em GE. Para complementar
este esforço, foram estudados documentos oficiais e foram realizados estudos etnográficos,
questionários e grupo focais durante grandes eventos sediados no Brasil nos últimos
anos. Em seguida, foram comparadas ferramentas atuais que visualizam informações
da emergência e foram entrevistados profissionais experientes em GE. Por fim, esses
dados de pesquisa foram cruzados e analisados qualitativamente. Foi possı́vel identificar
as relações entre necessidades visuais e fatores-chave em GE. Com essas descobertas
alinhadas aos conceitos de VA foi desenvolvido um modelo para aprimorar visualizações
na GE. Resultados: O modelo foi avaliado em um estudo exploratório comparando
visualizações de GE projetadas com e sem o suporte do modelo, em um Centro de
Comando e Controle. As visualizações desenvolvidas com o suporte do modelo obtiveram
73,4% das pontuações maiores que as visualizações desenvolvidas sem o modelo, 25%
foram iguais e apenas 1,6% menor. A principal contribuição deste trabalho é o modelo
para conceber e avaliar a eficácia de visualizações em cenários reais de GE. O resultado
da disseminação deste modelo permitirá a evolução da pesquisa sobre a utilização de
VA e melhoria das tarefas em GE. Há também impactos econômicos e sociais claros.
Espera-se que Centros incorporem o modelo em suas rotinas; se as interfaces desenvolvidas
auxiliarem à tomada de decisão rápida e assertiva, a qualidade do serviço prestado à
sociedade deve aumentar. A contribuição deste trabalho a longo termo é, portanto, a
redução das perdas financeiras e, sobretudo, das perdas de vidas em emergências.
Palavras-chave: Modelo Computacional, Visual Analytics, Visualização de Informa-

ção, Gestão de Emergência.
xi
ABSTRACT
Background: Emergency Management (EM) refers to the activity of dealing with

emergency tasks in different phases and iterations. People working in an emergency are
generally under stress to make the right decision at the right time. They have to process
large amounts of data and to assimilate the received information in an intuitive and visual
way. We found that information overload as well as missing information are problems
in EM. Visual Analytics (VA) is potentially useful to analyze and understand the huge
amount of data produced in an emergency. However, we found that the full potential
of VA is not being exploited in EM. Objective: We seek to develop a computational
reference model for using VA in EM preparedness. This model incorporates themes that
are currently under-exploited, to better support decision-making in EM. The model aims
to help visualization designers to create effective VA interfaces that in turn help emergency
managers to make quick and assertive decisions with these interfaces. Methods: We
performed a long-term multi-method in-vivo study. First, we carried out a Systematic
Mapping Study to analyze the available visualization tools and their applications in
EM. To complement this information, we carried out appraisal of official documents,
ethnographic studies, questionnaires and focus groups during large events held in Brazil.
Then, we analyzed actual tools that produces emergency information visualization and
interviewed professionals experienced in EM. We crosschecked and analyzed this data
qualitatively. We identified the relationships between the visual needs and other major
themes of influence for EM. We used our findings aligned with VA concepts to develop our
model for EM visualization. Results: We evaluated our proposal using an exploratory
study in a Brazilian Command and Control Center, comparing EM visualizations designed
with and without the support of the model. The visualizations that were designed with
the support of the model had 73.4% higher scores, 25% equal scores and only 1.6% lower
scores than the ones designed without it. We believe that the main contribution of this
work is to introduce the model to conceive and evaluate the effectiveness of VA in EM
scenarios. The results of the dissemination of this model will foment the research on the
use of VA in EM. We hope that C2 Centers incorporate the use of the proposed model in
their routine; if it helps in timely and assertive decision-making, the quality of the service
provided to society will improve. The ultimate contribution of our work is the potential
reduction of financial and, above all, human losses in emergencies.
Keywords: Computational Model, Visual Analytics, Information Visualization,

Emergency Management.
xiii
CONTENTS
Chapter 1—INTRODUCTION 1
1.1 RESEARCH CONTEXT . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 MOTIVATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 RESEARCH PROBLEM . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 RESEARCH QUESTION, HYPOTHESES AND OBJECTIVES . . . . . 6
1.5 DEVELOPED WORK AND RESEARCH METHODOLOGY . . . . . . 8
1.6 RESEARCH MAIN CONTRIBUTIONS . . . . . . . . . . . . . . . . . . 9
1.7 THESIS OUTLINE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter 2—THEORETICAL BASIS and RELATED WORK 13

2.1 EMERGENCY MANAGEMENT . . . . . . . . . . . . . . . . . . . . . . 13
2.1.1 Incidents, Emergencies and Disasters . . . . . . . . . . . . . . . . 13
2.1.2 Emergency Management Phases . . . . . . . . . . . . . . . . . . . 15
2.1.3 Command and Control Doctrine . . . . . . . . . . . . . . . . . . . 16
2.2 VISUAL ANALYTICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 THE USE OF VISUAL ANALYTICS IN EMERGENCY MANAGEMENT 19
2.4 FINAL REMARKS OF THE CHAPTER . . . . . . . . . . . . . . . . . . 21
Chapter 3—CYCLE I: DEFINITION OF THE RESEARCH SCOPE 23

3.1 ORIGINS OF THE THESIS THEME . . . . . . . . . . . . . . . . . . . . 23
3.1.1 Large Events in Brazil . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1.2 Intelligence and Operation Center, COI . . . . . . . . . . . . . . . 24
3.1.3 Project Rescuer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 DEFINITION OF THE RESEARCH SCOPE . . . . . . . . . . . . . . . 25
Chapter 4—CYCLE II: RESEARCH DATA COLLECTION 29

4.1 FORMAL REVIEW OF THE LITERATURE (SUBCYCLE II.A) . . . . 30
4.1.1 Systematic Mapping Study in a Nutshell . . . . . . . . . . . . . . 30
4.2 GENERAL RESEARCH DATA COLLECTION (SUBCYCLE II.B) . . . 33
4.2.1 C2 Official Documents Appraisal . . . . . . . . . . . . . . . . . . 34
4.2.2 Ethnographic Study and Other Observations . . . . . . . . . . . . 35
4.2.3 Questionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.3 TARGET RESEARCH DATA COLLECTION (SUBCYCLE II.C) . . . . 38
xv
xvi CONTENTS
4.3.1 Comparison of Visualization Tools . . . . . . . . . . . . . . . . . . 39

4.3.2 Survey Protocol (Interviews) . . . . . . . . . . . . . . . . . . . . . 42
Chapter 5—CYCLE III: RESEARCH DATA ANALYSIS 47
5.1 TRIANGULATING FINDINGS OF THE COLLECTED METHODS . . 47

5.2 CODING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.3 FOCUS GROUP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.4 IN DEPTH FINDINGS AND KEY INSIGHTS . . . . . . . . . . . . . . 55
Chapter 6—CYCLE IV: MODEL DEVELOPMENT 57
6.1 COMPUTATIONAL REFERENCE MODEL BASED ON VISUAL

ANALYTICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.2 INSTANTIATING THE MODEL: A WORKED EXAMPLE . . . . . . . 61
6.2.1 Instantiating the CONTEXT theme . . . . . . . . . . . . . . . . 62
6.2.2 Instantiating the HUMAN-BEHAVIOR theme . . . . . . . . . . . 62
6.2.3 Instantiating the PROCESSES theme . . . . . . . . . . . . . . . 65
6.2.4 Instantiating the DATA MANAGEMENT theme . . . . . . . . . 67
6.2.5 Instantiating the INFORMATION VISUALIZATION theme . . . 69
6.2.6 Instantiating the DECISION-MAKING theme . . . . . . . . . . . 71
6.3 SUMMARIZING THE STUDY UP TO HERE . . . . . . . . . . . . . . . 73
Chapter 7—CYCLE V: EVALUATION OF THE MODEL 77
7.1 EXPLORATORY STUDY DESCRIPTION . . . . . . . . . . . . . . . . 78

7.1.1 Subjects in the Exploratory Study . . . . . . . . . . . . . . . . . . 78
7.1.2 Procedures and Materials for the Exploratory Study . . . . . . . . 78
7.2 EVALUATION QUESTIONS (EQ) AND INDICATORS (Ind) . . . . . . 79
7.3 EXPLORATORY STUDY SETTINGS . . . . . . . . . . . . . . . . . . . 80
7.4 EXPLORATORY STUDY PRESENTATIONS . . . . . . . . . . . . . . . 82
7.4.1 First Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.4.1.1 Presentation 1: First control group . . . . . . . . . . . . 83
7.4.1.2 Presentation 2: Experimental group . . . . . . . . . . . 83
7.4.1.3 Presentation 3: Second control group . . . . . . . . . . . 85
7.4.2 Second Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 87
7.4.2.1 First control group: improvement insights . . . . . . . . 88
7.4.2.2 Second control group: improvement insights . . . . . . . 89
7.5 EXPLORATORY STUDY RESULTS . . . . . . . . . . . . . . . . . . . . 89
CONTENTS xvii
Chapter 8—DISCUSSION 95
8.1 COMPLETENESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
8.2 CREATIVITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
8.3 USABILITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
8.4 VIABILITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
8.5 THREATS TO VALIDITY . . . . . . . . . . . . . . . . . . . . . . . . . 98
8.5.1 Threats to Validity of the model development: Cycles II, III and IV 98
8.5.2 Threats to Validity of the model evaluation: Cycle V . . . . . . . 99
Chapter 9—FINAL CONSIDERATIONS 101

9.1 THESIS CONTRIBUTIONS REVISITED . . . . . . . . . . . . . . . . . 102
9.2 SCOPE LIMITATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
9.3 FUTURE WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Bibliography 109
Appendix A—ONLINE QUESTIONNAIRE (CYCLE 2) 117
Appendix B—EXTRACTION OF THE FACETS IN THE TOOLS (CYCLE 2) 121
Appendix C—SURVEY DESIGN (CYCLE 2) 123
Appendix D—CODING CLASSIFICATION (CYCLE 3) 141
Appendix E—DATASETS (CYCLE 5) 147
Appendix F—EVALUATION FORM (CYCLE 5) 151

LIST OF FIGURES
1.1 Example of Visualization X Emergency Management Context . . . . . . 7

1.2 Research Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 Emergency Management Phases (adapted from POSER; DRANSCH (2010)) 15

2.2 Visual Analytics Process (adapted from KEIM et al. (2008)) . . . . . . . 18
2.3 Disciplines to improve the division of labor between human and machine
(adapted from KEIM et al. (2008)) . . . . . . . . . . . . . . . . . . . . . 19
2.4 a) ‘The map view shows aggregated transportations’ (taken from
ANDRIENKO et al. (2007)); b) ‘Spatial pattern of Twitter users
after damages from a strong tornado’ (taken from CHAE et al.
(2014)); c) ‘Geo-message with map and annotation’ (taken from
TOMASZEWSKI; MACEACHREN (2012)); d) ‘Mobility interface’ (taken
from CIRAVEGNA et al. (2018)) . . . . . . . . . . . . . . . . . . . . . . 20
3.1 Integration scheme of the Command and Control Centers in Brazil . . . . 24

3.2 Location on the map of the C2 Centers that we visited . . . . . . . . . . 27
3.3 Images of C2 Centers: a) COI; b) CICC-RJ; c) CEIC; d) DZPPP . . . . 28
4.1 Primary Studies Selection adapted from DUSSE et al. (2016a) . . . . . . 31

4.2 Publications per Year, Venue and Type adapted from DUSSE et al. (2016a) 32
4.3 Publications per: a) Visualization Paradigm; b) Visual Attributes adapted
from DUSSE et al. (2016a) . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.4 Summary of the questionnaires: a) work experience; b) work experience
in C2; c) information quantity in C2 d) information quality in C2; e)
available tools to analyze information for decision-making; f) how visual
tools represent the information . . . . . . . . . . . . . . . . . . . . . . . . 38
4.5 Ushahidi map showing the locations most affected by the 2011 Tsunami
in Japan adapted from HEINZELMAN; WATERS (2010) . . . . . . . . . 40
4.6 Version 1 of Rescuer Tool. Map View adapted from JUNIOR et al. (2017) 40
4.7 Version 2 of Rescuer Tool. Incident Detailed View adapted from JUNIOR
et al. (2017) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.8 Steps of the Survey Protocol adapted from CIOLKOWSKI et al. (2003) . 43
5.1 Coding Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5.2 Clustering codes and categories . . . . . . . . . . . . . . . . . . . . . . . 52
6.1 The model without external components . . . . . . . . . . . . . . . . . . 58
xix
xx LIST OF FIGURES
6.2 Visual Analytics Based Model for Decision-Making in Emergency

Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.3 CONTEXT theme instantiated using Feature Model . . . . . . . . . . . 63
6.4 HUMAN-BEHAVIOR theme instantiated using TBP . . . . . . . . . . . 64
6.5 PROCESSES theme instantiated using ICS . . . . . . . . . . . . . . . . 65
6.6 DAT theme instantiated using an example of Data Science process . . . . 68
6.7 INF theme instantiated using an example of Information Visualization
process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.8 DECISION-MAKING theme instantiated . . . . . . . . . . . . . . . . . . 71
6.9 Summarization of the Study Up to Here . . . . . . . . . . . . . . . . . . 74
7.1 Evaluation Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

7.2 Screenshot of the visualization prototype of the first control group (1/1) . 84
7.3 Screenshot of the visualization prototype of the experimental group for the
full access, normal and landscape orientation viewer (1/1) . . . . . . . . 85
CICOC, normal and landscape orientation viewer (1/1) . . . . . . . . . . 85
full access, color-blind and landscape orientation viewer (1/1) . . . . . . 86
full access, normal and portrait orientation viewer a) 1/2 and b) 2/2 . . . 86
7.7 Screenshot of the visualization prototype of the second control group of
the full access viewer (1/3) . . . . . . . . . . . . . . . . . . . . . . . . . . 87
the no CICOC viewer (1/1) . . . . . . . . . . . . . . . . . . . . . . . . . 88
7.11 First Evaluation: (a) degree of completeness (Ind2.1), (b) degree of
creativity (Ind2.2), (c) degree of usability (Ind2.3) and (d) degree of
viability (Ind2.4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.12 Second Evaluation: (a) degree of completeness (Ind3.1), (b) degree
of creativity (Ind3.2), (c) degree of usability (Ind3.3) and (d) degree of
viability (Ind3.4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
LIST OF TABLES
1.1 Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2 Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.1 Research Question at the Time of the Mapping Study . . . . . . . . . . . 31

4.2 Search String . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.3 Research Question revisited . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.4 Number of Respondents of the Questionnaire . . . . . . . . . . . . . . . . 37
4.5 Facets to Compare the Visualization Tools . . . . . . . . . . . . . . . . . 41
4.6 Preliminary Key Themes . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.1 Knowledge and Findings of Research Data Collection . . . . . . . . . . . 48

5.2 Triangulated research data of the findings in Table 5.1 . . . . . . . . . . 49
5.3 Coding Classification: Categories X Themes . . . . . . . . . . . . . . . . 53
7.1 Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
7.2 Evaluation Questions and Indicators . . . . . . . . . . . . . . . . . . . . 80
7.3 Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7.4 Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7.5 Results Criteria X EVALUATOR X Group . . . . . . . . . . . . . . . . . 92
7.6 Results of the Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
9.1 Practitioners Meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
xxi
ACRONYMS
C2 Command and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
CEIC Command Integration Center of the City of Porto Alegre [Centro Integrado de
Comando de Port Alegre, in Portuguese] . . . . . . . . . . . . . . . . . . . . 26
CEMADEN Brazilian National Center for Early Warning and Monitoring of Natural
Disasters [Centro Nacional de Moniotramento e Alertas de Desastres Naturais, in
Portuguese] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
CICCN-DF National Command and Control Integrating Center of Brazil [Centro

Integrado de Comando e Controle Nacional do Brasil, in Portuguese] . . . . 26
CICCR-BA Regional Command and Control Integrating Center of the State of Bahia
[Centro Integrado de Comando e Controle Regional da Bahia, in Portuguese] 24
CICCR-CE Regional Command and Control Integrating Center of the State of Ceará
[Centro Integrado de Comando e Controle Regional do Ceará, in Portuguese] 26
CICCR-RJ Regional Command and Control Integrating Center of the State of Rio de
Janeiro [Centro Integrado de Comando e Controle Regional do Rio de Janeiro, in
Portuguese] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
COFIC Industrial Promotion Committee of the City of Camaçari [Comitê de Fomento

Industrial de Camaçari, in Portuguese] . . . . . . . . . . . . . . . . . . . . . 26
COI Intelligence and Operation Center [Centro de Operações e Inteligência, in

Portuguese] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
DZPPP Defense Zone of the Paris Police Prefecture . . . . . . . . . . . . . . . . 26
xxiii
xxiv ACRONYMS
EM Emergency Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
FEMA Federal Emergency Management Agency . . . . . . . . . . . . . . . . . . 1
FPC-UFBA Fraunhofer Project Center for Software and Systems Engineering at

UFBA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
GMT Greenwich Mean Time Zone . . . . . . . . . . . . . . . . . . . . . . . . . . 41
GPS Global Positioning System . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
ICS Incident Command System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
ISCRAM International Conference on Information Systems for Crisis Response and

Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
MEUV Military Emergency Unit in Valencia, Spain . . . . . . . . . . . . . . . . 26
NOA Operation Center Assisted of the City of Salvador [Núcleo de Operação Assistida
de Salvador, in Portguese] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
RQ Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
SESGE Extraordinary Secretary of Security for Large Events . . . . . . . . . . . 23
SSP-BA Public Security Secretariat of the State of Bahia [Secretaria de Segurança

Pública da Bahia, in Portuguese] . . . . . . . . . . . . . . . . . . . . . . . . 23
UML Unified Modeling Language . . . . . . . . . . . . . . . . . . . . . . . . . . 73
UTM Universal Transverse Mercator . . . . . . . . . . . . . . . . . . . . . . . . . 41
VA Visual Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
GLOSSARY
Emergency Response Plan is an oral or written plan containing general

objectives reflecting the overall strategy for managing an emergency. It may include the
identification of operational resources and assignments. It may also include attachments
that provide direction and important information for management of the emergency
during one or more operational periods.
Attitude is a disposition or latent tendency to respond in a favorable or unfavorable

way to an object or scenario. A person’s attitude towards the object is the result of his
or hers evaluation of the attributes of that object.
Behavioral beliefs are those according to which people have judgments about the
positive or negative consequences of a behavior. Expectations of results will determine
the attitude of people regarding the realization of personal behavior. A positive or
negative evaluation of performance generates a more or less favorable attitude.
Belief is a state or habit of mind in which trust or confidence is placed in someone,

in something or in some aspect of their environment.
Command is the exercise of authority based upon certain knowledge to attain an

objective.
Context is a set of variables that defines an environment, such as: location, scenario,
stakeholders, resources, status of the resources, and so on.
Control is the process of verifying and correcting activity such that the objective or
goal of command is accomplished.
Crisis is an event that is expected to lead to a dangerous situation, whether it’s

an emergency or a disaster. It’s also defined as a “time of intense difficulty, trouble, or
danger”.
Data (raw data) is a set of values of qualitative or quantitative variables with no

handling. Depending on the handling, data can become information.
Data Management comprises all disciplines related to managing data as a valuable

asset; from planning the data collection to its use, storage or discard, passing by all data
xxv
xxvi GLOSSARY
transformations.
Decision-making is a process that could be seen as a logical sequence of events:

Recognize the existence of a problem → Define the solution alternatives → Decide the
best solution alternative → Validate the decision.
Disaster is a natural or man-made event caused by one or more incidents that

requires a response action to protect life or property that exceeds the response capacity
of the affected community.
Emergency is a natural or man-made event caused by one or more incidents that

requires a response action to protect life or property where the affected community can
solve using the means provided for this purpose.
Emergency Management or EM is the ability to deal with emergency tasks in

all their phases and iterations. EM is the domain of our work.
Emergency manager is a practitioner that make decisions before, during and after
an emergency.
Event (happening) is anything that happens or any kind of problem that needs a
decision.
Event (ceremony) is a ceremony of ritual significance, performed on a special

occasion, especially something important or unusual.
Human behavior is an observable act that is performed by a person. People

construct beliefs about various personal, technological and environmental aspects, which
may facilitate or complicate their attempts to adopt certain behaviors. Attitudes, values,
norms and intentions are the main determinants of human behavior.
Human resource management is responsible for overseeing employee-benefits

design, employee recruitment, training and development, performance appraisal,
organizational climate, personnel motivation and rewarding.
Incident is an occurrence or event, natural or man-made, that requires an emergency

response.
Information is the result of the processing, handling and organization of data.
Information Visualization is the research field that investigates interactive visual

representations of abstract data to reinforce human cognition. The visual representations
may help the person who analyses them to identify important information or patterns
within the dataset that could be difficult to spot otherwise.
GLOSSARY xxvii
Intention is an indication of the promptness of a person to perform a behavior.

Allows estimating the probability of the person to perform certain behavior.
Normative beliefs are ones that people form on the approval or disapproval of their
behavior by individuals or groups important in their lives. The approval or disapproval
of significant groups leads people to realize a norm or social pressure regarding the
behavior, encouraging a more or less favorable intention.
Perceived control beliefs are those that people form about the personal and
environmental factors that can help or hinder one’s attempts to perform the behavior.
They produce an assessment of high or low efficacy with respect to their behavior. It
can generate facilitative or inhibitory behavioral intentions.
Practitioner is a person who is qualified to or registered to practice a particular

occupation. In our text, a EM practitioner is any employee that acts and operates after,
during or before an emergency.
Processes is a series of interrelated tasks and decisions that, together, transform

inputs into a given output.
Research data are data of our research, i.e., the data that we collected and analyzed.
Specialist is a person with great knowledge or skill in a particular field. In our

text, EM specialists or domain specialists are practitioners; specialists that is not a
practitioner we call them EM researcher.
Stakeholder is a person or group that has an interest in or is affected by a company,

business or event. In our text, a EM stakeholders are practitioners, volunteers, victims,
witness.
Visual paradigm is used to create the de facto visual scenes (views) rendered on
the computer screen.
Chapter
1
This Chapter presents the context and the motivation of this work, including our research problem,
objectives and methodology. It also describes the developed work, highlighting its main contributions and
the structure of the thesis.
INTRODUCTION
1.1 RESEARCH CONTEXT

Emergencies have very complex and dynamic contexts where the response process should
suit unforeseen situations without causing social harm or jeopardizing people’s lives
(ESRI, 2009). According to Federal Emergency Management Agency (FEMA) of the
U.S., emergency is any incident(s), human-caused or natural, that requires responsive
action to protect life or property (FEMA, 2014). Examples of emergencies that greatly
stimulated research interest in this area were the attacks of September 11, 2001 (NYT,
2001) and Hurricane Katrina in 2005 (WAUGH; STREIB, 2006), both in the United
States and a sequence of terrorist attacks in European cities (BLOOMBERG, 2017); and
more recently the rupture of mining tailings dams in Brumadinho in Brazil (PORSANI;
JESUS; STANGARI, 2019) and the COVID-19 pandemic (GRASSELLI; PESENTI;
CECCONI, 2020) among others which have reinforced this interest.
Emergency Management (EM) refers to the ability to deal with emergency tasks
in all their phases and iterations (DUSSE et al., 2016a). In general, this is done by
multiple government agencies which do not work together routinely. These agencies
usually use the doctrine of Command and Control (C2) to help in the coordination
of activities (TUROFF et al., 2004). EM is a complex and dynamic domain with an
endless amount of variability (GALTON; WORBOYS, 2011). EM varies because the
context is never the same from one emergency to another. Each management task
requires many different types of information about the incident from several sources
(TUROFF et al., 2004), such as information systems, proprietary databases, resources
in the field, internet of things sensors, social media and other open source media.
Nowadays, C2 Centers are equipped with breakthrough technologies, especially visual
systems, such as the latest generation mobiles, tabletops, video walls and e-caves (ISHIDA
et al., 2012), (ENGELBRECHT; BORGES; VIVACQUA, 2011), (SUTCLIFFE et al.,
1
2 INTRODUCTION
2006). This union of data sources and novel technologies provides C2 Centers with
huge amounts of information of great heterogeneity.
Decision-making during an emergency requires having the right information, provided
in the right way, to the right people, at the right time. While emergency managers
working in an emergency are generally under stress, they have to make effective decisions
with agility. Having reliable and up to date information is essential in order to help in
making decisions (YATES; PAQUETTE, 2011). Emergency managers need to assimilate
the received information in an intuitive way and the information needs to be adaptable
in the context of the current emergency. In addition, if this contextual information is not
presented in effective ways, it can be hard to analyze it and, in the worst cases, it can be
misinterpreted worsening the effects of an incorrect decision (OCHOA et al., 2007).
Literature shows that suitable Information Visualization techniques improve the
understanding of emergency information (CIOLKOWSKI et al., 2003), (WU et al.,
2009), (BOSCH et al., 2013), (THOM et al., 2015). Nevertheless, the development of
visualization tools for EM remains complex because of the emergency context and time
pressure, of the increasing amount of available data, and of the risk of a cognitive overload
instead of the expected cognitive support (PIERKOT; CHRISTOPHE; GIRRES, 2019).
Information overload as well as non-dedicated information are problems in EM (DUSSE
et al., 2016a). Recently, researchers have been using Visual Analytics (VA) to
support understanding in this subject (ANDRIENKO et al., 2007), (TOMASZEWSKI;
MACEACHREN, 2012), (CHAE et al., 2014), (CIRAVEGNA et al., 2018).
Like EM, VA is a multidisciplinary area that combines management sciences,
social sciences and computer sciences. VA is potentially helpful for analyzing and
understanding the huge amount of information produced in an emergency (KEIM
et al., 2008). The benefits brought by VA to the analysis of emergency information
also come along with certain challenges: (1) how to map VA concepts with the
countless variables of an emergency to support the managers in quickly identifying
reliable information to make decisions; (2) whether researchers and practitioners
are using VA techniques and tools effectively to tackle challenge 1; and, (3) how to
enhance VA applications with minimal impact on current EM activities. Briefly, these
challenges point that VA applications need to meet some criteria in the EM domain:
must be complete (challenge 1), creative and usable (challenge 2) and viable (challenge 3).
1.2 MOTIVATION
The number of emergencies around the world has been increasing in recent years. Climate
change, urban growth and the rising number of terrorist attacks are the mostly responsible
for this increase (NORRIS et al., 2008), (KELMAN; GAILLARD; MERCER, 2015).
In 1980, the leading global provider of reinsurance accounted 249 relevant loss only in
natural events, not counting man-made events; this number increased to 848 in 2018
(MUNICHRE, 2018).
EM is quite complex since emergency managers need to examine a wide heterogeneity
of elements in order to make time-critical decisions. Uncertainties generated by
1.2 MOTIVATION 3
subjectivity and imprecision affect the safety and effectiveness of actions, which may
be detrimental to EM goals, i.e. financial losses as well as the loss of lives. In 2018,
natural disasters caused a financial burden of US 160 billion worldwide and led to 10,400
fatalities.(MUNICHRE, 2018).
Besides this, emergency policymakers recommend the use of practices and technologies
to mitigate these losses. They require the use of local-based information (e.g.
demographic density, building materials, city structures); hazard-monitoring systems
(e.g. satellite remote sensing, climate sensors, suspected terrorists) and the use of social
technologies (e.g. social media, crowdsourcing) for emergency risk reduction (TUROFF
et al., 2004). These, in turn, increase the volume and heterogeneity of the available
information, causing problems in the completeness and timely analysis of it. The more
sources of information; the more redundant, ambiguous, conflicting and false information
may appear, which can make it more difficult for the practitioners to grasp what is
happening at the site of the emergency. Using human resources to deal with all this
information may hinder the overall EM. Furthermore, resources are not infinite, but are
costly and have their own particularities.
Data sources (confidential or open source) can increase the capacities of C2
by, for instance, improving the situational awareness (MIDDLETON; MIDDLETON;
MODAFFERI, 2014). However, interpretation and extracting insights from the massive
data are time and resource consuming. Indeed, transforming raw data to usable
information through proper visualizations for decision-making has been the target of
several research papers (AGGARWAL, 2011).
Emergency managers use visualizations to help make sense of all the information
obtained through data sources (CIOLKOWSKI et al., 2003), (WU et al., 2009), (BOSCH
et al., 2013), (THOM et al., 2015). A Systematic Mapping Study (DUSSE et al., 2016a)
shows that most researchers use similar techniques, focusing on maps and icons, to
visualize emergency information at a high level (e.g. affected area, type of emergency).
Those visual paradigms, geo-spatial and iconographic (KEIM; KRIEGEL, 1996), presents
an overview of the emergency. In some cases, emergency managers need interaction
mechanisms, mainly filter, search and zoom, to see the details. However, in most cases
the detailed information of what is happening on emergency location are not present in
the visualization, thus requiring other artifacts to access them, such as other systems,
messaging apps, printed reports, annotations, etc (DUSSE et al., 2016a).
Visual tools may follow the visual information seeking mantra “Overview first, Filter
and Zoom, Details on demand” (SHNEIDERMAN, 1996). But one can argue that during
an emergency, where there is a high demand for information and a short time span for
making decisions, searching for information through common interaction mechanisms
may delay the decision process, which can lead to undesired consequences.
In general, EM visual tools do not take into account the particularities of agencies.
For example, one icon plotted in emergency map tool with the letter ‘H’ which could
represent a H ospital for one agency, while it could represent a H ydrant for another.
One of the objectives of C2 visualization tools is to standardize visual attributes and
terminology because multiple agencies which do not routinely work together make up
the EM C2 (TUROFF et al., 2004). The purpose is to facilitate coordination and the
4 INTRODUCTION
control of actions in an easier way and to measure and evaluate the practitioners and
the processes. Even with training and exhaustive instructions, this standardization can
disrupt the decision process, because practitioners usually feel more comfortable using
legacy systems with their agencies’ own interface and terminology.
A further limitation of these visual tools is that, in general, they use the same
visualizations for any kind of emergency. In addition, they are not effective in managing
two or more emergencies occurring simultaneously (DUSSE et al., 2016a).
No emergency is the same as a previous one and nor the next. However, lessons from
past emergencies can help to manage future ones (TOMASZEWSKI; MACEACHREN,
2012). In general, this “memory” is not stored and most of the information is in
the minds of the practitioners who managed the past emergency (TOMASZEWSKI;
MACEACHREN, 2012). Practitioners may change position, retire, die or simply forget
the past information. Thus, it is important to save all emergency information for future
use regardless of time, place and people. Visualizations may use this historical information
to anticipate current activities by comparing to the past.
Another problem is that the visualization is often not at the beginning of the
decision-making process. Managers can make a wrong decision at any previous step if the
actions are not well orchestrated. For instance, after a decision is made, a manager can
discover that the decision was not the best alternative because the information collected
was poor. It is important that the visualization tools highlight these occurrences.
Moreover, EM visual tools generally do not take into account human behavior and
preferences. For example, some emergency managers prefer to analyze the information
needed on one computer screen, while others prefer to compartmentalize on two or
more screens. As another example, some perceive an ordinal scale varying the size of
visual elements easier than others who prefer varying the color tone. It is common for
decision-makers to base their behavior not only on received information, but also on
environment factors (laws, norms, resources restrictions, etc) and background factors
(individual, organization, society, etc) (AJZEN, 1985). Therefore, it is important that
emergency visualizations represent these factors.
People working in an emergency need to understand and analyze large volumes of
information in seconds. However, it is difficult to visualize information in an effective
manner during the various activities of the EM. Because of this, there is a need to find
out which information is effective for a particular activity, to organize and present it for
these activities.
The motivation of this thesis lies in all the issues discussed in this Section, bad design
may lead to wrong visualizations, or no visualizations at all, of the emergency situation.
Visualizations should not only give quick access to most information happening at the
emergency site but the management of the information should also support insights for
the emergency managers. Some insights are listed below:
• what is the most important information at a given time,
• if this information is reliable and up to date and what aspects can increase the
reliability of this information,
1.3 RESEARCH PROBLEM 5
• where, when, who, why and how was this information managed (searched, collected,
processed, integrated, shared, visualized, etc.),
• if the handling of this information was well orchestrated until the visualization tool
present it,
• if this information is sufficient or not,
• if information of the EM context is well presented in the visualization,
• if information of past emergencies can help at the current situation,
• if the visualization took into account human aspects such as behavior and decision
theories.
All these challenges motivates us to investigate the problems of information overload

and non-dedicate information in EM decision-making. The next Section outlines these
two problems that our thesis addresses.
1.3 RESEARCH PROBLEM

This work addresses two research problems in EM decision-making: information
overload and non-dedicated information. Even with the availability of wide and
heterogeneous information about current emergency(ies), which by itself is a challenge for
decision-making, there are other neglects, limitations and gaps in the area of Information
Visualization applied in EM. In DUSSE et al. (2016a), we can find some issues related to
both problems. Below, we list these issues that this thesis also tackles:
• huge amount of information about current emergency(ies);
• lack of use of internal information, for instance, the particularities of each agency;
• lack of use of information regarding past emergencies;
• no identification of faults in the handling of information;
• neglect of the human behavior element;
• neglect of decision-making theory.
The solution proposed for the problems above focus on Visual Analytics. VA enable
people to detect the expected and discover the unexpected (COOK; THOMAS, 2005).
KEIM et al. (2008) adjusted the Shneiderman’s mantra for VA (SHNEIDERMAN,
1996): “Analyze First - Show the Important - Zoom, Filter and Analyze Further - Details
on Demand”. In the next Section, we propose the use of VA through computational
modeling.
6 INTRODUCTION
1.4 RESEARCH QUESTION, HYPOTHESES AND OBJECTIVES

The Research Question (RQ) that guides this thesis is presented in Table 1.1.
Table 1.1 Research Question
“What themes of Emergency Management are under-exploited

in decision-making and how can Visual Analytics support them?”
To answer the Research Question, we assumed the following research premises (based
on NASCIMENTO (2016)):
(i) Emergency information is highly dynamic, heterogeneous and inconsistent.
(ii) Emergency managers have to deal with an increasing amount of information1
(iii) In Emergency Management, it is impracticable to know what kind of information

emergency managers will have to use or how this information will be visualized or
analyzed in project-time.
(iv) The Emergency Management context defines what kind of information emergency
managers will have to use and how this information will be visualized and analyzed at
run-time.
(v) Visual Analytics has potential to facilitate the understanding of the information
and its relevance at a given time.
(vi) A model that represents the themes of information needs of emergency managers
can help design, evolve, simulate and evaluate visualizations at project-time and/or
run-time.
Based on our RQ and above premises, we define our working hypothesis as shown in
Table 1.2.
The key expression in the hypothesis is ‘Emergency Management context’. The

context makes a particular situation and the circumstances of an entity unique and
understandable (TUROFF et al., 2004). However, context is a difficult concept to define,
model and represent (TOMASZEWSKI; MACEACHREN, 2012). For EM, context
includes endless types of information such as where the emergency is occurring, what
events have transpired, which agencies are responsible for the emergency, who is involved,
what resources are available and so on.
1
including non-dedicated information that we identified in the state-of-the-art and
state-of-the-practice.
1.4 RESEARCH QUESTION, HYPOTHESES AND OBJECTIVES 7
Table 1.2 Hypothesis
“A computational reference model, based on Visual Analytics,

that supports the designing of complete, creative, usable and viable
visualizations driven by the emergency managers
can improve the decision-making process
in a given Emergency Management context.”
See Figure 1.1 for a simple example of use of visualizations in EM contexts. At

the start time (‘t1’) of Emergency ‘E1’, Manager ‘M1’ uses information ‘I1’ within
visualization ‘V1’ (blue arrow); while Manager ‘M2’ uses it within visualization ‘V2’
(orange arrow). Halfway through the Emergency ‘E1’, in time ‘t2’, ‘M1’ begins to use
the same Info ‘I1’ within ‘V2’ (blue arrow) and ‘M2’ changes to ‘V3’(orange arrow).
Emergency ‘E2’ started in ‘t3’, at this point information ‘I1’ is uninteresting for Managers
‘M1’ and ‘M2’; ‘V1’ is suitable for ‘M1’ and ‘M2’ to analyze ‘I2’.
Figure 1.1 Example of Visualization X Emergency Management Context
In our literature search and observations of C2centers , we did not find a

visualization tool similar to the example above. In general, the visualizations used
today are not context-sensitive. In this example, ‘M1’ and ‘M2’ are required to
use the same visualization for any information at any time in any emergency. In fact,
visualization tools may undergo evolution, for example, adding or improving visual
features in the tool. However, until this evolution is released, ‘M1’ and ‘M2’ will use
the same visualization as in ‘E1’, ‘E2’ and other emergencies. We miss an element in
the scheme represented in 1.1. We believe that this element can be an abstract element,
for example, a model to represent the relevant elements of a particular situation and the
interactions between them.
The objective of this thesis is to propose a new computational reference model to
support the designing of VA interfaces for EM. The model will not use VA interfaces
based only on information of the current emergency(ies), but those based on other
8 INTRODUCTION
themes within the EM context. We found that the full potential of VA is not being
applied in the state-of-the-art nor the state-of-the-practice. We expect that extending
research and the application of VA techniques and tools in an effective way will have a
positive effect on the emergency domain. Our approach attempts to make visualizations
less rigid without limiting control, sharing and reusing of the information. The aim is
to standardize the model and not the visualizations themselves, i.e. the model may be
used by everyone but the visualizations will be adapted to a given context. Based on
our hypothesis, we believe that flexible visualizations will better support emergency
managers in making faster and more assertive decisions.
1.5 DEVELOPED WORK AND RESEARCH METHODOLOGY
The Research Question led us to use a multi-method approach that was

underpinned by a long-term exploratory study. The methodology followed
five cycles that led to the final model and its evaluation, as follows: the
scope definition of the study (Cycle I); the research data2 collection (Cycle
II); the research data analysis (Cycle III); the computational reference model
development (Cycle IV); and finally the evaluation of the model (Cycle V).
Figure 1.2 shows the research methodology. We divided our research into five
cycles however, this does not mean that the cycles were sequential, i.e. a cycle started
only with the end of the previous one. At some moments, we carried out activities
within two, or more, cycles in parallel or we went back to do activities from previous
cycles.
After defining the goals, methods and scope of the research in Cycle I, we collected
research data to understand the problem in Cycle II. First we did a Systematic Mapping
Study to understand the area identifying possible gaps and trends. Then, we carried
out appraisal of official documents, ethnographic studies, and questionnaires during large
events held in Brazil in recent years. Lastly, in this cycle of research data collection, we
analyzed and compared current visual tools used in C2 Centers and we interviewed seven3
practitioners experienced in EM. In Cycle III, we crosschecked and analyzed this research
data qualitatively with Coding and Focus Groups. We found that the full potential of VA
is not being applied in the state-of-the-art and state-of-the-practice of the EM domain.
We argue that researchers and practitioners do not pay due attention to key themes that
they should be aware of and incorporate into visualization tools. From these findings,
we developed a computational reference model based on VA (Cycle IV). Finally, we
carried out an exploratory study to evaluate our model (Cycle V). Figure 1.2 summarizes
the research methodology in objectives, methods, contributions and publications of each
cycle. Regarding the publications, the gray one is poster sections, the yellows ones refer
to publication and submission in journals and the green ones are published at conferences.
In this thesis, we detail each work cycle in a corresponding Chapter as described in
2
In order to prevent the reader from confusing data collected in our research and data from the
emergencies, we use the term ’research data’ to refer to the first case
3
Seven = six subjects + one pilot
1.6 RESEARCH MAIN CONTRIBUTIONS 9
Figure 1.2 Research Methodology
Section 1.7.
1.6 RESEARCH MAIN CONTRIBUTIONS
An important contribution of this thesis is our novel approach to handling visual interfaces
driven by a model. ‘To handle’ we mean ‘to know, to understand, to develop, to update,
to simulate and to evaluate’. Our aim is to improve existing visualizations in the
state-of-the-practice and to be a reference in the design of new ones. We expect that
this model can address the issues reported above that cause problems in decision-making
in EM. We built this model based on researching practices and tools from the literature
and from a C2 in-vivo studies.
Our research investigated current tools that support decision-making through the
visualization of the emergency information in depth. Nevertheless, we also investigate
other themes, such as human behavior and decision-making theory that influence the way
we present the visualizations. We believe that extending the research and considering the
10 INTRODUCTION
application of Visual Analytics techniques and tools in a broader way will have a positive
effect on the emergency domain. Consequently, another contribution of this thesis is a
research agenda in the EM area.
The main scientific and technological impacts are to disseminate the state-of-the-art
of VA research and technologies and to evaluate their effectiveness in real scenarios of EM.
The result of this dissemination is the adaptation and evolution of the state-of-the-art
and state-of-the-practice of VA in the EM domain.
There are also clear economic and social impacts for the research. We expect that
government C2 Centers (and maybe C2 Centers of industries as well) will incorporate
the model proposed in this thesis in their routine. If the proposed model helps in timely
and assertive decision-making, the quality of the service provided to society will increase,
(e.g. governments will make better use of public resources).
The ultimate contribution of our work there is the potential reduction in financial
and, above all, human losses in emergencies. The validations, performed with domain
specialists, indicate that our work has a positive impact overall.
1.7 THESIS OUTLINE

After this Introduction Chapter, which describes the motivation, objectives and scope of
the research, this thesis is organized as follows:
• Chapter 2 - THEORETICAL BASIS AND RELATED WORKS: provides a

conceptual background and literature review on the topics of EM and VA, describing
the foundations for this work. Emergency Management is first introduced through a
description of its phases and of Command and Control Doctrine. Following this, we
describe an overview of Visual Analytics showing how the area is multidisciplinary.
The Chapter also cites examples of the literature about the use of Visual Analytics
in Emergency Management (related works).
• Chapter 3 - CYCLE I: DEFINITION OF THE RESEARCH SCOPE: presents the

origin of this thesis and defines the scope of the work, outlining the research cycles
that we used in it. We designed a multi-method approach divided into five cycles:
(I) definition of the research scope, (II) research data collection, (III) research data
analysis, (IV) development of the computational reference model and (V) evaluation
of the model.
• Chapter 4 - CYCLE II: RESEARCH DATA COLLECTION: presents the second

cycle that is the planning and the collection of the research data in several sources.
We carried out a multi-method approach to collect data on visualization applied
to Emergency Management. We divided this cycle in three subcycles: II.A)
literature review, II.B) general research data collection and II.C) target research
data collection. In II.A, we carried out a Systematic Mapping Study. In II.B, we
carried out document appraisal, ethnographies and questionnaires. And in II.C we
did comparison of visual tools and interviews follow a strict Survey Protocol.
1.7 THESIS OUTLINE 11
• Chapter 5 - CYCLE III: RESEARCH DATA ANALYSIS: presents the third cycle
that is the formal qualitative analyse of the research data collected in the previous
cycle. We used the Coding technique and then we evaluated the Coding findings
with Focus Group.
• Chapter 6 - CYCLE IV: MODEL DEVELOPMENT: presents the fourth and main
cycle that is the steps we used to develop our model based on the previous cycles,
research data collection and the analysis of these research data. We highlight that
this is the main cycle because the model is the main contribution of this thesis. We
did not find similar model in the literature and in our in-vivo observations.
• Chapter 7 - CYCLE V: EVALUATION OF THE MODEL presents the fifth and

final cycle that is the evaluation of the model developed in the previous cycle. We
carried out an exploratory study with practitioners of a C2 Center. We compared
EM visualizations designed with and without the support of the model.
• Chapter 8 - DISCUSSION: analyzes the results of the evaluation in practice. It also

discusses limitations and validity threats of the work.
• Chapter 9 - FINAL CONSIDERATIONS: summarizes the final conclusions of this

thesis, as well as outlining the main contributions that were achieved. It also
discusses possible future research in the thesis area.
Chapter
2
This Chapter provides the theoretical background and literature review on the topics, describing the
foundations for this work. Emergency Management is first introduced through a description of its phases
and of Command and Control Doctrine. Following this, we describe an overview of of Visual Analytics
showing the multidisciplinarity of the area. The Chapter also introduces examples of related works about
the use of Visual Analytics in Emergency Management
THEORETICAL BASIS AND RELATED WORK
2.1 EMERGENCY MANAGEMENT

Emergency Management (EM) refers to the ability to deal with emergency tasks in all
their phases and iterations (DUSSE et al., 2016a). In general, this is done by multiple
government agencies which do not work together routinely and have their own Emergency
Response Plan. These agencies usually use the doctrine of C2 to help in the coordination
of activities (TUROFF et al., 2004). C2 is the exercise of authority and direction by a
properly designated commanding office over assigned resources in the accomplishment of
a common goal (BUILDER; BANKES; NORDIN, 1999). In general, this commanding
office coordinates a C2 Center that is responsible for the operational decisions, i.e. plan,
execute, evaluate and improve the response actions that in general is described in the
Emergency Response Plan. In the literature and in the day-to-day, there are uncountable
type of emergencies.
2.1.1 Incidents, Emergencies and Disasters

An emergency is a sudden and usually unforeseen event that calls for immediate measures
to minimize its adverse consequences (DHA, 1992). It can be a natural or man-made
incident that requires a response action to protect life or property. If the response actions
exceed the capacity of the affected location, the emergency becomes a disaster and the
community needs extra resources from superior agencies (FEMA, 2014). In other words,
an incident can become an emergency, which in turn can become a disaster. A crisis is
an event that is expected to lead to a dangerous situation, whether it’s an emergency or
a disaster. It’s also defined as a “time of intense difficulty, trouble, or danger”. The C2
Centers defines the level of response actions to be employed based on the EM complexity.
13
14 THEORETICAL BASIS AND RELATED WORK
The emergency context defines the level of response actions that needs to be employed.
For example, a fire in parked vehicle is an incident; a fire in a building with injured people
is an emergency; and a big fire in an industrial complex with many injured people and
toxic gas leak is a disaster.
Emergencies and disasters have very complex and dynamic contexts in which the
response actions should suit unforeseen situations without causing social harm or
jeopardizing people’s lives (ESRI, 2009). Examples of emergencies that greatly stimulated
research interest in this area were mentioned in the previous Chapter. We highlight the
attacks of September 11, 2001 and Hurricane Katrina in 2005, both in the United States.
More recently, the Hurricanes Harvey, Irma and Maria in 2017 and a sequence of terrorist
attacks in European cities both in 2017 and the COVID-19 pandemic in 2020. In Brazil,
emblematic examples are the fire in the Kiss nightclub (Rio Grande do Sul, 2013) which
culminated in the death of 242 people (GLOBO, 2015) and the rupture of the dam in the
cities of Mariana and Brumadinho which caused the death of almost 300 people dragged
through the mud and the pollution of the ecosystem in the region, especially the Doce
river (ESCOBAR, 2015).
The damage caused in such incidents, emergencies or disasters is hard to measure
and varies according to the scenario. To a certain extent, the consequences can influence
the socioeconomic, political, cultural and mental life of those present in the affected
area and also people outside the emergency area, e.g. families of the victims. The
response actions should be timely and accurate to mitigate these consequences. Sense
making and situational awareness are fundamental for decision-making. Information
retrieval and analysis must be fast and assertive. Identifying the severity of the situation,
injured victims, selecting teams and resources are tasks that require a quick and effective
response. Communication problems should in no way take place. Different agencies such
as the Police, Firefighters, Medical Services, among others follow emergency protocols
respecting their particularities. In addition, non-governmental organization (NGOs),
civilian volunteers and private organizations; such as REDCROSS1 , VISOV2 , COFIC3 ;
can also engage in the emergency activities providing information and resources to
governments agencies. This complex, dynamic and critical domain, with information
overload and different agencies working in the same emergency is a hard challenge.
EM is the broad spectrum of activities and organizations providing effective and
efficient operations, coordination and support applied at all levels of management, using
both governmental and nongovernmental resources to plan, to respond and to recover
from an incident, regardless of cause, size, or complexity (FEMA, 2014). Good practice
in EM is crucial to avoid an incident from becoming an emergency and an emergency
into a disaster, which is even harder to respond and recover from.
1
https://www.redcross.org/
2
http://www.visov.org/
3
http://www.coficpolo.com.br/
2.1 EMERGENCY MANAGEMENT 15
2.1.2 Emergency Management Phases

Emergency Management (EM) is a multidisciplinary area that refers to the ability to
deal with emergency activities in all their phases and iterations, including the mitigation,
preparedness, response and recovery (POSER; DRANSCH, 2010). In general, this is
done by multiple government agencies (e.g. Police, Firefighters, Medical Services, Army),
which do not work together routinely.
In order to manage an emergency, agencies may follow different activities that can
be expressed in the EM phases. Figure 2.1 describes the four traditional phases that
occur before, during and after an emergency. As we said, these phases are mitigation,
preparedness, response and recovery. Each of them has a set of activities that should be
performed by emergency agencies. Further, they are not necessarily sequential, they may
run in parallel.
Figure 2.1 Emergency Management Phases (adapted from POSER; DRANSCH (2010))
Mitigation involves risk appraisal identifying threats and vulnerabilities. This phase
is continuous and focused on reducing or eliminating risks. It also includes activities such
as land use planning and technical prevention.
The preparedness phase corresponds to the moment when EM agencies create and
exercise plans to cope with possible emergencies that may arise at the place where
incidents are possible. It includes activities such as simulation training, monitoring,
capacity building and selecting resources. The development and testing of tools and
processes that can be used during an emergency also occur mainly in the preparation
phase.
Early warning is not considered a phase itself, but in a EM it plays a crucial role in
monitoring possible emergencies. In the eminence of an emergency, early warning systems
can detect and signal disturbances that adversely affect the stability of the physical world.
In general, this gives sufficient time for the response system to prepare resources and
response actions to minimize the negative impact of an emergency.
The response phase is usually the most visible phase of an EM (DUSSE et al., 2016a).
It takes place as soon as an emergency happens and the multiple agencies respond to
control the consequences through commands from the authorities. WAUGH; STREIB
(2006) says that the response to an emergency requires organization and meticulous
planning, however, it can be spontaneous based on tacit knowledge. It includes activities
such as search, rescue, heal, combat, prevent, protect and so on. Actions of humanitarian
aid can also be considered an emergency response, but some researchers prefer to classify
this as recovery activities (DUSSE et al., 2016a).
The recovery phase covers activities related to the reconstruction of the system
affected by the emergency to obtain a new acceptable state of the considered system. The
activities here are associated with assessing the damage caused rehabilitating communities
and injured people or restoring buildings, roads, ecosystems, etc. As the main goal of
humanitarian aid is to rehabilitate injured people, as we said, some people classify it as
a recovery activity.
In an emergency, EM agencies usually use the concepts of Command and Control
(C2) to help in the coordination of the activities (TUROFF et al., 2004).
2.1.3 Command and Control Doctrine
Doctrine, from Latin doctrina, (compare doctor ), can be defined as a set of principles, or
a codification of principles, or a body of teachings and instructions that serve as the basis
for a branch of knowledge or belief system, e.g. religious, political, law, philosophical,
military system, among others (WORDIQ, 2010). C2 doctrine emerged from military
doctrine (BUILDER; BANKES; NORDIN, 1999), it is the expression of how military
forces contribute to campaigns, major operations, battles and engagements (NATO,
2017).
C2 is the exercise of authority and direction by a properly designated commanding
officer over assigned resources in the accomplishment of a common goal (BUILDER;
BANKES; NORDIN, 1999). Although it is usually associated with the military, it can be
used in civilian environments. One of the main characteristics of command and control
doctrines is their degree of secrecy. Because dealing with national security issues is often a
challenge for researchers outside the command and control agencies to carry out in-depth
research in this area.
In general, this commanding officer coordinates a C2 Center which is the physical
location at which the coordination of information and resources to support EM activities
normally takes place. Practitioners in a C2 Center are responsible for the operational
decisions, i.e. planning, executing, evaluating and improving the response actions. A
C2 Center is typically a secure place that operates as the agency’s dispatch, surveillance
monitoring, coordination office and alarm monitoring all in one.
There are two kinds of C2 Centers: local or remote. The main differences are in
the proximity to the emergency site (near or distant), in the life time of the Center
2.2 VISUAL ANALYTICS 17
(temporary or permanent) and in the mobility of the Center (mobile or fixed). In

general, local Centers are in the emergency site, or very close; the Center can be moved
spatially and the Center is disabled when the response phase of the emergency is over.
Barracks, trucks or even a set of desks can configure the installations of a local C2
Center. The practitioners that operate this kind of Center are dedicated to responding to
the local emergency, making decisions and executing actions locally. In contrast, remote
or fixed C2 Centers are installed in buildings and they are permanently activated. In the
case of an emergency, the practitioners pay special attention to the current emergency,
making decisions and executing actions, but they are also aware of the possibility of new
emergencies. In general, in critical emergencies, emergency managers use both kinds of
Centers because local Centers can view the details of the emergency situation taking
operational decisions and remote Centers can obtain an overview of the situation taking
strategic decisions. Besides this, remote center can be a backup for local Center or vice
versa, in case of the collapse of one.
2.2 VISUAL ANALYTICS
We see not with our eyes, but with our brain (CYCLEBACK, 2005). Envisioning
information is the process of building mental models of information characteristics from
a visual representation of it. The research field of Information Visualization investigates
interactive visual representations of abstract data to reinforce human cognition (MAZZA,
2009). The visual representations may help the person that analyses them (the analyst)
to identify important information or patterns within the dataset that could be difficult
to spot otherwise. Researchers discuss that practitioners often rely on a variety of
graphs, charts and maps to make sense of an emergency from the data collected
(THOMAS, 2009). Although insightful, visualizing emergency information requires
careful consideration due to information overload, rumors, scalability, multi-modality and
information manipulation (SCHUTT; O’NEIL, 2013). A multidisciplinary and relatively
recent area of Computer Science has emerged: Visual Analytics (VA).
VA is more than just Information Visualization. It is the science of analytical
reasoning facilitated by highly interactive visual interfaces. Like EM itself, VA is a
multi-disciplinary research area that draws on many disciplines, such as computer science,
management science and social science. It can be seen rather as an integral approach
to decision-making, combining visualization, human factors and data management
(KEIM et al., 2008). Visual analytics combines strengths from information analytics,
geospatial analytics, scientific analytics, statistical analytics, knowledge discovery,
data management, knowledge representation, presentation, production, dissemination,
cognition, perception and interaction. It is a goal-oriented process to gain insight into
heterogeneous, contradictory and incomplete data through the combination of automatic
analysis methods with human background knowledge and intuition (KEIM et al., 2006).
VA enables people to detect the expected and discover the unexpected (COOK;
THOMAS, 2005). Unlike described in the information seeking mantra (“overview first,
zoom/filter, details on demand”), the VA process comprises the application of automatic
analysis methods before and after the interactive visual representation is used. This
is primarily due to the fact that current and especially future datasets are complex on
one hand and too large to be visualized in a straightforward manner on the other hand.
Therefore, KEIM et al. (2006) evolved information seeking mantra: “Analyze First -
Show the Important - Zoom, Filter and Analyze Further - Details on Demand”. This
mantra call for an astute combination of analytical approaches together with advanced
visualization techniques (KEIM et al., 2008).
Figure 2.2 shows summarily the path from data to decision. The process begins with
a decision to produce knowledge for make, in turn, other decisions (dotted arrow). The
second step is to collect data to answer the first decision. There is a tight integration
of visual and information analysis methods with data management for a decision-making
process. This integration is made by visualizations (image), models (hypotheses) and
algorithms to reinforce human perception providing new insight for the decisions. The
VA process must be scalable, recursive and interactive. Scalable because it may be used
in small or big problems. Recursive because the process is continuous and the decisions
must influence a new cycle of knowledge. And interactive because the data analysis may
be semi-automated where humans and machines cooperate using their respective distinct
capabilities for the most effective results (See Figure 2.3).
Figure 2.2 Visual Analytics Process (adapted from KEIM et al. (2008))
According to KEIM et al. (2008), the main difference between traditional visualization
works and VA is that the former does not necessarily deal with analysis tasks nor does
it always also use advanced data analysis algorithms. VA gives higher priority to data
analytics from the start and through all iterations (Figure 2.2). Analyzing the user
interactions on the data, it can be turned into intelligence to tune underlying analytical
processes. Creativity is then needed to understand how perception issues can help bring
more intelligence into the analytical process by learning from users’ behavior and effective
use of the visualization.
2.3 THE USE OF VISUAL ANALYTICS IN EMERGENCY MANAGEMENT 19
Figure 2.3 Disciplines to improve the division of labor between human and machine (adapted
from KEIM et al. (2008))
2.3 THE USE OF VISUAL ANALYTICS IN EMERGENCY MANAGEMENT
New technologies, such as the latest generation mobiles phones, Internet of Things, social
media, are increasing exponentially the volume and variety of the available information in
the world (Big Data), bringing challenges in the completeness and timely analysis of this
information (HASHEM et al., 2015). For EM this reality is more challenging. Recently,
researchers have been using VA to support understanding this phenomenon in EM. It is
potentially helpful to analyze and understand the huge amount of data produced in an
emergency. We identified works in the literature that address these problems in analyzing
EM information by using VA in their approaches. Here, we highlight four relevant works:
ANDRIENKO et al. (2007), TOMASZEWSKI; MACEACHREN (2012), CHAE et al.
(2014) and CIRAVEGNA et al. (2018).
In ANDRIENKO et al. (2007), the authors design a suite of tools to support
transportation planning tasks such as emergency evacuation. The transportation schedule
is a complex construct involving heterogeneous objects with states and positions varying
over time. The tools use genetic algorithms with VA techniques such as interactive visual
interfaces.
In TOMASZEWSKI; MACEACHREN (2012), the authors present visual interfaces to
construct and represent the geographical and historical context in EM and humanitarian
aid. They implemented a tool that supports document foraging and sense-making to
construct the geo-historical context.
In CHAE et al. (2014), the authors analyze social media data (Twitter), with an
interactive visual spatio-temporal analysis and spatial decision support environment to
assist in evacuation planning. They show how to improve investigation by analyzing the
extracted public behavior responses from social media before, during and after natural
emergencies. Their approach focuses on identifying the whereabouts and movements of
people in order to understand mass behavior.
Figure 2.4 a) ‘The map view shows aggregated transportations’ (taken from ANDRIENKO et
al. (2007)); b) ‘Spatial pattern of Twitter users after damages from a strong tornado’ (taken from
CHAE et al. (2014)); c) ‘Geo-message with map and annotation’ (taken from TOMASZEWSKI;
MACEACHREN (2012)); d) ‘Mobility interface’ (taken from CIRAVEGNA et al. (2018))
2.4 FINAL REMARKS OF THE CHAPTER 21
In CIRAVEGNA et al. (2018), the authors developed a VA platform that enables the
identification of mobility patterns from large-scale GPS data traces. This platform allows
decision makers to understand mobility patterns before, during and after emergencies to
devise strategies for evacuations, to facilitate a fast, flexible and effective response, to
gauge the level of recovery and return to normality. We copied figures of these four works
to illustrate visually the tools used by them in Figure 2.4.
Unfortunately, we did not find in these works the global concepts of VA (COOK;
THOMAS, 2005), (KEIM et al., 2006), (KEIM et al., 2008). These works focused on
visualizations in two specific aspects: the data (e.g. archival documents, social media
posts) and the work purposes (e.g. evacuation plans, humanitarian relief, victims and
witness behavior). Moreover, these works visualize information from a single source
(single database, single open source repository of documents, or single Social media).
They fail to use multiple data sources and, consequently, they did not face the challenges
and implications of handling multiple sources of information. On top of all that, the
presented visualizations did not take into account the particularities of the user profile.
The visualizations were standardized, i.e. all users had to access the same visualization.
In the real world, emergency managers need customized information from several data
sources to make decisions. It is clear from the literature that most visualizations applied
in EM are highly context-specific and standardized for all users. The state-of-the-art may
derive general principles of information requirements, but, at this point, the literature
provides a limited basis for doing so. As a result, researchers are driving the development
of visualization tools by analyzing the information needs within a limited scope and in
specific EM contexts. However, the C2 practitioner’s routine requires a multi-context
visualization approach because they deal with emergencies regardless of cause, size,
location, or complexity. Current works lack context-sensitive approaches.
As a result, we conclude that the full potential of VA is not being applied in the
state-of-the-art. We expect that extending research and the application of VA techniques
and tools, in an effective way, will have a positive effect on the emergency domain. A
critical challenge is to know from those who manage emergencies which information they
need and how visualizations can support their decisions. To improve the decision-making
in EM, we need to know how to enhance the visualizations and if there is information
that is not used but is important to be depicted.
The gaps discussed above motivated the development of this thesis. Emergency
managers need contextualized views because their field of expertise is complex and critical.
We work by collecting and analyzing data from literature research and in-vivo studies
using a multi-method approach. This thesis takes steps to fill these gaps by linking VA to
the emergency managers’ decision-making needs. To achieve this, we have to work at a
high level of abstraction of technologies and standards. Therefore, we focus our research
on computational modeling, looking for complete, but usable and viable solutions.
2.4 FINAL REMARKS OF THE CHAPTER
This chapter contextualizes the main concepts that are used in this text – Emergency
Management and Visual Analytics. These concepts form the theoretical basis of this
work. In Section 2.1, we presented the EM concept. It defines a set of activities to

be performed before, during and after an incident. It also introduces the EM phases:
mitigation, preparedness, response and recovery, and the concepts of Command and
Control Doctrine.
Moreover, Section 2.2 presented Visual Analytics and the Computer Science
background in which our work is grounded. VA is a relatively new area in Computer
Science. Its main characteristic is the fact that it is multidisciplinary, which makes it
quite challenging. Since its introduction around 2008, VA has been gaining attention
given its potential to solve many open problems in science.
Section 2.3 referenced works in the literature that address EM problems by
using Visual Analytics (VA) in their approaches (ANDRIENKO et al., 2007),
(TOMASZEWSKI; MACEACHREN, 2012), (CHAE et al., 2014), (CIRAVEGNA et al.,
2018). Analyzing the state-of-the-art, we concluded that current works have generated
excellent results, but in limited scopes of EM. Researchers and practitioners are using
the visualizations for specific activities. However, in reality, emergency managers require
a multi-context visualization approach, because they deal with all kinds of emergencies.
We argue that the visualizations need to be complete, viable and friendly to tackle the
problems we mentioned before.
After this background, in the next Chapter, we define our research scope explaining
where and how the idea of the thesis originated.
Chapter
3
This Chapter presents the origins of this thesis and defines the scope of the work, outlining the research
cycles that we used in it.
CYCLE I: DEFINITION OF THE RESEARCH SCOPE
In recent years, emergencies have generated losses in the economy and society worldwide
(MUNICHRE, 2018). Readiness and mitigation are vital to reduce these losses. Brazil,
in particular, had to prepare itself to receive large events, such as the Soccer World Cup
and the Summer Olympics, in recent years. These large events are known for crowd
formation and are under risk of terrorist attacks. Government and industrial partners
have undertake massive investments in Emergency Management during this period. We
can say that one of the origins of this thesis is related to those initiatives.
3.1 ORIGINS OF THE THESIS THEME

This work originates from three sources: i) the large events that have occurred in recent
years in Brazil, ii) a partnership that we have with the Public Security Secretariat of
the State of Bahia [Secretaria de Segurança Pública da Bahia, in Portuguese] (SSP-BA)
and iii) the Brazil-European Union Rescuer Project (VILLELA et al., 2014). Below, we
explain how these happened.
3.1.1 Large Events in Brazil

In 2011, the Federal Government of Brazil created the Extraordinary Secretary of Security
for Large Events (SESGE), which is a branch of the Brazilian Ministry of Justice. The
main purpose of SESGE is to coordinate the activities of the agencies involved in public
security and civil defense during large events that take place in Brazil. The two main
motivators for the creation of SESGE were the 2014 Soccer World Cup and 2016 Summer
Olympics Games. In fact, those two events have brought about several others in their
wake, e.g. FIFA Confederations Cup (2013), Final Draw of the World Cup (2013),
Olympic Torch Relay (2016), Paralympics (2016), CONMEBOL America Cup (2019),
to mention the most important ones.
23
24 CYCLE I: DEFINITION OF THE RESEARCH SCOPE
In 2013, SESGE set up C2 Centers in the venue cities of the 2014 Soccer World
Cup. They named twelve of those C2 Centers as “Regional Command and Control
Integrating Centers” (CICCR in Portuguese). For example, the C2 Center in the State of
Bahia, was named The Regional Command and Control Integrating Center of the State
of Bahia [Centro Integrado de Comando e Controle Regional da Bahia, in Portuguese]
(CICCR-BA). The CICCR are formed by one “Local Command and Control Integrating
Centers” (CICCL in Portuguese) and two or three “Mobile Command and Control
Integrating Centers” (CICCM in Portuguese). They also established two “National
Command and Control Integrating Centers” (CICCN in Portuguese), one in Brası́lia, the
capital of Brazil, and an alternative center in Rio de Janeiro (CICCNA), the main venue
for the 2014 World Cup and the 2016 Summer Olympics Games. All of these Centers are
integrated according to the Figure 3.1. These Centers bring together multiple agencies to
work in coordinating and sharing resources (human, equipment, software, network, etc)
for Emergency Management during those large events.
Figure 3.1 Integration scheme of the Command and Control Centers in Brazil
ts.
3.1.2 Intelligence and Operation Center, COI

Located in the city of Salvador in the state of Bahia in Brazil, The Intelligence
and Operation Center [Centro de Operações e Inteligência, in Portuguese] (COI) is a
Command and Control (C2) Center created in 2016 in the wake to the World Cup and
the Olympics. It took the place of the CICCR-BA. Due to our partnership with SSP-BA,
we had access to COI, which is a branch of the Secretariat.
Although COI has its own resources, the Center works in collaboration with several
institutions such as other CICCRs, the Federal Police, Military Forces and even with
foreign police services, e.g. Interpol. Among COI’s activities, the following activities
play a crucial role in supporting EM in state of Bahia:
• risk assessment that can support prevention and response actions (mitigation);
3.2 DEFINITION OF THE RESEARCH SCOPE 25
• development, implementation and operation of early warning systems for natural

or man-made emergencies;
• development of emergency response plans and simulations and trainings to evaluate

the effectiveness of these plans (preparedness);
• emergency decisions and actions, like search and rescue (response); and
• support in post-emergency reconstruction (recovery).
All these activities produce a huge amount of information. Without effective tools
and techniques, it is almost impossible to analyze this information to produce knowledge
effectively. Investments in processes and technologies are really necessary for effective
management of a C2 center like COI.
3.1.3 Project Rescuer

From 2014 to 2016, we collaborated as a coworker with the Fraunhofer Project Center
for Software and Systems Engineering at UFBA (FPC-UFBA) and other eight Brazilian
and European institutions in the scope of the RESCUER Project1 , a project funded by
the European Union and the Brazilian Research Council.
The RESCUER project involved developing a smart and interoperable computer
platform for using crowdsourcing information mashed up with open data to support
EM. In the scope of the project, we were interested in visual interfaces for sense and
decision-making in EM. So, we decided to understand how Information Visualization
was being applied in EM. We did this from literature reviews and visiting C2 Centers
in Brazil and Europe. It was then that we defined the “first version” of our Research
Question: “How do Information Visualization tools support Emergency Management?”.
We will see later in this text that we evolved this Research Question to “What themes
of Emergency Management are under-exploited in decision-making and how can Visual
Analytics support them?”
3.2 DEFINITION OF THE RESEARCH SCOPE

In order to better define the scope of our work, we decided to carry out a formal review of
the literature and to run an in-vivo study involving C2 Centers. So, we explored the EM
activities in depth at COI and then visited others Brazilian and European C2 Centers.
To visit other C2 Centers was very helpful to map similarities and inconsistencies among
them. Below, we list of all the C2 centers we have visited during our research.
• Intelligence and Operation Center [Centro de Operações e Inteligência, in

Portuguese] (COI),
1
http://www.rescuer-project.org/
• Regional Command and Control Integrating Center of the State of Rio de

Janeiro [Centro Integrado de Comando e Controle Regional do Rio de Janeiro,
in Portuguese] (CICCR-RJ),
• Command Integration Center of the City of Porto Alegre [Centro Integrado de

Comando de Port Alegre, in Portuguese] (CEIC),
• Defense Zone of the Paris Police Prefecture (DZPPP),
• Military Emergency Unit in Valencia, Spain (MEUV),
• National Command and Control Integrating Center of Brazil [Centro Integrado de

Comando e Controle Nacional do Brasil, in Portuguese] (CICCN-DF),
• Regional Command and Control Integrating Center of the State of Ceará [Centro
Integrado de Comando e Controle Regional do Ceará, in Portuguese] (CICCR-CE),
• Operation Center Assisted of the City of Salvador [Núcleo de Operação Assistida

de Salvador, in Portguese] (NOA),
• Brazilian National Center for Early Warning and Monitoring of Natural Disasters
[Centro Nacional de Moniotramento e Alertas de Desastres Naturais, in Portuguese]
(CEMADEN) and
• Industrial Promotion Committee of the City of Camaçari [Comitê de Fomento

Industrial de Camaçari, in Portuguese] (COFIC)
Figure 3.2 show the localization of these Centers. The red marker means three Centers
in the same state. Figure 3.3 shows photos taken during visits to four of those C2 Centers.
Aligned with this effort, we collected research data over 30 months starting in the
second semester of 2014 and finishing in the first semester of 2017. Then, we carried out
a deep research data analysis with a qualitative technique called Coding, that lasted six
months.
The Research Question led us to propose a computational reference model based
on Visual Analytics (VA) to tackle the problems we are researching. We used a
multi-method approach that was underpinned by a long-term study. The methods
were cyclically and incrementally carried out. To be more didactic, we divided
our research in five cycles: the definition of the scope of the research (Cycle I);
the research data collection (Cycle II); the research data analysis (Cycle III); the
development of the model (Cycle IV); and finally the evaluation of the model (Cycle V).
Revisiting Figure 1.2, it shows the research methodology that was divided into cycles,
but, as we mentioned, this does not mean that the cycles are sequential and linear.
At any moment, we went back and forth to activities of different cycles.
One can see that Figure 1.2 has five columns and seven rows. The rows represent the
cycles, with Cycle 2 divided in three sybcycles that are shown in subrows (II.A, II.B and
II.C). The columns are represented as follow:
Figure 3.2 Location on the map of the C2 Centers that we visited
• The first column is the cycles id and the ‘timeline’ in which they were performed.
As we mentioned, the cycles are sometimes iterative, and if necessary, they go back
to previous cycles.
• The second column is the objective of the cycles. For example, the objective of the
Cycle I is the ‘Definition of the research scope’.
• The third column is the methods used in the cycles. For example, the methods of
the Cycle III are ‘Research Data Triangulation’, ‘Coding’ and ‘Focus Group’.
• The fourth column is the contributions of the cycles. For example, the contribution
of Cycle V is ‘Model accepted by practitioners (or rejected)’.
• The fifth column is the dissemination of the results of the cycles, our publications.
For example, the publication of the Cycle IV is DUSSE; NOVAIS; MENDONÇA
(2020).

This Chapter described Cycle I of this thesis: definition of the scope. In Section 3.1, we
explained the main three origins: i) the large events that have occurred in recent years
Figure 3.3 Images of C2 Centers: a) COI; b) CICC-RJ; c) CEIC; d) DZPPP
in Brazil, ii) a partnership that we have with COI and iii) the Rescuer Project.
Moreover, Section 3.2 mentioned our visits to C2 Centers around the world (8 in
Brazil and 2 in Europe) and, finally, we revisited our Research Question to define the
scope of the work.
In the next Chapter, we explore Cycle II: the research data collection.
Chapter
4
This Chapter presents the second (of five) Cycle of this thesis: data collection. We describe Cycle II, its
methods, the tasks we accomplished during it and the contributions of the cycle to the thesis.
CYCLE II: RESEARCH DATA COLLECTION
As we defined the scope of the research in Cycle I, we used a multi-method approach

to collect data on visualization applied to Emergency Management (EM). We divide the
second cycle of our work into three subcycles (see Figure 1.2). First, in Subcycle II.A ,
we carried out a formal review of the literature to collect data about the state-of-the-art
on the use of information visualization (and data analytics) for EM, which resulted in
a Systematic Mapping Study (DUSSE et al., 2016a). The mapping identified gaps and
trends in the data visualization that are applied in the EM domain. One of these trends is
the use of Visual Analytics (VA). We then analyzed works that address VA in emergencies
which, according to their authors, generated excellent results (ANDRIENKO et al., 2007),
(TOMASZEWSKI; MACEACHREN, 2012), (CHAE et al., 2014), (CIRAVEGNA et al.,
2018). This analysis indicated that the state of the art was not using the full potential
of VA in EM, as discussed in Chapter 2.
Then, in Subcycle II.B, we carried out an in-vivo study to collect general research data
about the state-of-the-practice in Command and Control (C2) Centers. We explored the
EM activities in COI in depth and visited Brazilian and European C2 Centers, as listed
in Chapter 3. Visiting C2 Centers was very helpful to map similarities and inconsistencies
among them. Practitioners in C2 environments work under strict time constraints:
following a rigid schedule in a 24/7/365(6) scheme. Therefore, we took advantage of
the time spent on the visits the best way we could. We made observations through
ethnography, we ran online questionnaires and, when possible and allowed, accessed
official C2 documents to appraise them.
Concomitantly with Subcycle II.B, we also collected target research data comparing
visualization tools available in the C2 Centers that we visited and where we interviewed
experienced emergency managers using a survey protocol. These two methods composed
the Subcycle II.C of our research.
29
30 CYCLE II: RESEARCH DATA COLLECTION
4.1 FORMAL REVIEW OF THE LITERATURE (SUBCYCLE II.A)
The purpose of this research data collection was to provide research data to understand
how the state-of-the-art on Information Visualization applied in EM is structured. We
wanted to know common applications in existing studies published in the literature.
We also wanted to identify research gaps and possible trends in the area. Thus, we
performed literature reviews to understand the challenges and find out what researchers
are publishing in the area. The focus was on knowing the problems in the area and
finding academic solutions proposed to tackle them.
This research data collection involved a Systematic Mapping Study, following the
rules of (KITCHENHAM; CHARTERS, 2007) and (PETERSEN et al., 2008). The
mapping study was our core literature work, but it was not the only one. In fact, during
this cycle and the others, we returned to the literature to investigate the topic deeper
and remain updated on the state-of-the-art.
4.1.1 Systematic Mapping Study in a Nutshell

As implied by the term, a Systematic Mapping methodically outlines the published
literature on an area using a well-defined classification scheme. The Systematic Mapping
method is used to provide a structure for the published research and its results. The
results of a mapping highlights gaps and trends in a given research area (PETERSEN et
al., 2008).
We decided to carry out a Systematic Mapping Study to understand the use of
Information Visualization tools and their application in the EM area. We planned
to identify the most common visualization techniques applied to EM, as well as the
environments and phases in which they were applied, following a strict protocol to search
and extract research data (DUSSE et al., 2016a).
To begin, we searched if a similar mapping study had already been published in the
literature. If this search had returned one or more similar studies we could analyze the
protocol used to replicate or expand the original study. However, we did not find such a
study. Thus, to the best of our knowledge, no other study had systematically investigated
how Information Visualization is being applied to EM systems. Therefore, we designed
our research protocol from scratch.
We then defined the Research Question of the mapping, its search string, inclusion and
exclusion criteria, digital libraries and classification scheme (classes and facets). We also
stipulated the period covered by the review. We searched the primary studies from Jan
01, 2001 to Dec 31, 2013. We decided to begin with the year of 2001 because September
11, 2001 (9/11) is a milestone in the EM Domain (SCHMEMANN, 2001). When we
submitted the results to a well-regarded journal (DUSSE et al., 2016a), the reviewers
asked us to extend the period with works published in 2014 and 2015. So we searched
the primary studies again, this time around, from Jan 01, 2001 to May 31, 2015. We
used the Research Question1 defined in Table 4.1 and search string defined in Table 4.2.
1
When we carried out the Systematic Mapping Study, our old Research Question was the same as
4.1 FORMAL REVIEW OF THE LITERATURE (SUBCYCLE II.A) 31
Table 4.1 Research Question at the Time of the Mapping Study
“How do Information Visualization tools support

Emergency Management Systems?”
Table 4.2 Search String
(“Emergency Management” OR “Disaster Management”

OR “Crisis Management”)
AND (Visualization OR Visual OR Visualisation)
The returned papers were randomly divided among three researchers in such a way
that each paper was read twice. If there was a disagreement in the extraction, the third
researcher gave her/his opinion to reach a consensus.
The first search found 5083 papers, from which 132 remained after passing through a
set of filters with exclusion criteria. After the analysis of the first 132 primary studies, we
used the backward snowballing technique to search in the references of each of them to
find more studies. This led us to 49 more papers. We applied the same technique again
to these 49 studies and selected 15 more papers. We repeated it again and did not find
any new primary studies, which in the end gave us a total of 196 primary studies to
analyze. Figure 4.1 details this search selection.
Figure 4.1 Primary Studies Selection adapted from DUSSE et al. (2016a)
According to our findings, the number of published studies that apply visualizations
to EM is growing, showing the relevance and level of interest in the area (see Figure 4.2).
From the 196 primary studies, 175 (89%) are full papers and 21 (11%) are short papers.
We considered short papers those with up to five pages. Regarding the venue type, 116
(59%) were published at Conferences, 74 (38%) in Journals and 6 (3%) at Workshops.
Considering the distribution per year, we observed that the number of papers grew in
the Research Question of the mapping. As we mentioned in the previous Chapter, due the mapping,
we changed our Research Question to investigate Visual Analytics, over Information Visualization,
understanding that it would be a more promising area for the EM domain
2007 and 2008. In 2014, it reached a peak with almost twice as many compared to 2013.
When we did the search (May 31, 2015), we found 13 primary studies in 2015. We did
not check the total number of studies in 2015. Our expectation is that it continues to
grow.
Figure 4.2 Publications per Year, Venue and Type adapted from DUSSE et al. (2016a)
Figure 4.3 shows that most researchers use similar visual paradigms and attributes
focusing on colors, spatial position, shapes and icons plotted on maps (Geo-spatial 2D
and 3D) to visualize emergency information in a big picture, e.g. affected area and
type of emergency. Practitioners usually have to filter and search the available overview
information to detail it on demand (SHNEIDERMAN, 1996) for the situation at hand.
Figure 4.3 Publications per: a) Visualization Paradigm; b) Visual Attributes adapted from
DUSSE et al. (2016a)
The results show that researchers apply most tools in the response phase of an
emergency. This demonstrates the need that emergency managers have for updated
information about the situation in the location of the incident to support decision-making.
Most works cover a specific type of emergency (e.g. earthquake, hurricane, flood, fire)
4.2 GENERAL RESEARCH DATA COLLECTION (SUBCYCLE II.B) 33
but we also found works that address emergencies in general. We classified 94% of the
works as outdoor scenarios and only 11% as indoor. There are very few works in both
scenarios (5%).
We also identified gaps and trends in the area. One of these trends is the use of
Visual Analytics. We analyzed works which address this research area in emergencies
which, according to the authors, generated excellent results. However, we consider these
studies to be still incipient. They partially use the concepts of VA described in KEIM
et al. (2008) and the visualizations address a specific task (e.g. humanitarian aid and
evacuation plans). Other findings in the mapping study include: the massive use of map
visualizations and icons plotted on it for EM visualization; the use of sensors, such as
Global Positioning System (GPS) and weather sensors, among others, to collect data;
the use of social media to collect data too and send messages (alerts) to the crowd
involved in the emergency; the extensive use of interaction mechanisms over visualization
tools and the growth in the use of mobile devices for EM.
4.2 GENERAL RESEARCH DATA COLLECTION (SUBCYCLE II.B)
Right after the Systematic Mapping, we changed the direction of our research. We evolved
the research area from Information Visualization to Visual Analytics. As we mentioned
in the previous Section, we changed our Research Question to the one defined in Table
4.3. Thus, we began to collect research data to understand the problems that we were
tackling to propose solutions later.
Table 4.3 Research Question revisited
“What themes of Emergency Management are under-exploited

in decision-making and how can Visual Analytics support them?”
We performed a general research data collection through an in-vivo study to know

the key themes and the challenges in the EM domain. The focus was on diagnosing
the open problems in the area. The purpose of this research data collection was to
understand broadly the overall dynamics of processes, comprising the main themes of
how emergency managers analyze information for decision-making.
We had access to practitioners and to materials from Brazilian and European C2
Centers, but our main data source was COI. This research data collection involved
official document appraisal, ethnographic studies and questionnaires following the rules
of (CRANG; COOK, 2007), (FETTERMAN, 2010) and (ROBSON, 2011). The main
objective of these methods of collecting research data was to confirm if our new Research
Question was relevant and feasible.
4.2.1 C2 Official Documents Appraisal

Official Documents are documents, of any kind and in any form, produced and received
by the Executive, Legislative and Judiciary Powers from any National, Provincial or
Local governmental sphere (DUSSE et al., 2016a). Thus, C2 Official Documents,
as the name suggests, are official documents about C2 produced and received by
practitioners of government agencies. The document topics are diverse, e.g. Strategic
Planning, Emergency Response Plans, agency policies, norms of conduct, manuals, among
others. The format of the documents are also diverse, e.g. books, reports, texts, sites,
spreadsheets, presentations, diagrams, videos and so on. Most of them are not published
and the practitioners access them via a private network. EM is a critical area, it is
used in public security and national defense and, for that reason, most C2 documents
are confidential and we do not have permission to publish them2 . However, there
are important official documents that are public. A good example are the documents
associated with Strategic Security Plan of SESGE for Brazil Large Events (SESGE,
2014). Those documents describe SESGE objectives, the duties already provided by
public security and the role of the CICCRs (see Section 3.1).
COI and other C2 Centers granted us access to official documents used during their
implantation and their operation. We started appraising documents of our interest in
2014 and we continue accessing and assessing them to this date. Official documents,
public and confidential, enriched our work and helped us to find evidences about the
doctrine of C2 and critical procedures adopted by C2 Centers, such as:
• C2 management systems, e.g. Incident Command System (ICS) (FEMA, 2014);
• Profile and roles, e.g. command hierarchy;
• Tools, e.g. vehicles, weapons, equipments, softwares, visualizations;
• Inherent Business Process:
– Human Resources, e.g. recruitment, hiring, termination, health treatment;

– Administrative, e.g. facilities, logistics (medical services, food services,
supplies), work schedules;
– Finance e.g. sourcing, procurement, costing;
– Information and Communication Technologies, e.g. infrastructure
(data network, voice network), software systems (operational softwares,
administrative software, commercial Off-The-Shelf software, equipment
(computers, cameras, radios, smartphones);
– Strategic e.g. strategic decisions, main objectives;
2
In Brazil, there are laws that protect confidential information (12.527/2011 Law on Access to
Information [LAI, in the original Portuguese acronym] and personal information (General Law on
Personal Data Protection [13.709/2018 LGPD, in the original Portuguese acronym]).
– Tactic, e.g. tactic decisions, specific objectives, design of the Emergency

Response Plan;
– Operational, e.g. operational decisions, agency operating roles and
limits, execution of the Emergency Response Plans and standard operating
procedures;
– and so on;
These documents also helped us to know the infrastructure and Information

Visualization tools used in the C2 Centers that we had accessed. Thus, the appraisal
of official documents allowed us to establish the theoretical basis needed to enrich our
understanding in the EM domain, focusing on the use of visualizations. It also helped in
the preparation and execution of the other scientific methods in this work (Systematic
Mapping Study, ethnographic study, questionnaires, interviews, formal analysis, Focus
Group, and evaluations). Moreover, we used the research data collected from the
documents to compare and triangulate with other research data collected using other
methods, providing feedback to C2 policymakers when we detected inconsistencies.
4.2.2 Ethnographic Study and Other Observations

To complement our in-vivo study, we carried out two ethnographic studies during the
2014 World Cup and 2016 Summer Olympics in Brazil. Ethnographic research methods
have played a substantial role in sociological research during the last half-century and
researchers are now using them in many disciplines, especially those that involve social
and human factors (CRANG; COOK, 2007). Ethnography is about immersing into the
culture of a group for a period in order to participate in social relations, seeking to
understand actions, how people act and make sense of their environment.
In particular, we adapted our ethnography studies with a limited degree of interaction
between the subjects studied and us. We think that an external observer (and stranger),
in their environment could inhibit some actions or on the other hand could lay emphasis
on actions without need due to our observation. This was particularly helpful in enabling
us to gather unbiased information about staff profiles, processes and technologies used
for EM in COI. It also provided us with real, rich and detailed information.
The CICCR-BA3 prepared itself for possible emergencies that could happen during
the 2014 FIFA World Cup in Brazil. The managers called this operation ‘World Cup
Operation’. We did our first ethnographic study in this scenario with a partnership
with the Fraunhofer Project Center for Software and Systems Engineering (FPC-UFBA)
in the scope of the Rescuer Project (VILLELA et al., 2014). The focus of this study
was crowdsourcing, more specifically, we wanted to investigate C2 theory and practices
to understand how crowdsourcing can improve EM decision-making. We identified the
main agencies working in this operation, their main activities and the main resources
that they used for analyzing information and communicating with each other. We did
3
In 2014, COI was still called CICCR-BA
not witness any impactful emergency, so we did not learn much about emergency response
and recovery. However, we could observe the Center activities to mitigate (risk analysis)
and to prepare (planning and monitoring). Concerning Information Visualization, we
noticed that despite having tools with visualization resources such as emergency maps,
they were scarcely exploited for information analysis and decision-making.
Two years later, COI4 prepared itself for possible emergencies that might happen
during the 2016 Olympics Summer Games. The managers called this operation ‘Olympics
Operation’. We did a second ethnographic study in this new scenario. Unlike the first
one, the focus was C2 visualization tools and therefore more linked to the objectives of
this thesis. As we wanted to understand how visualization could support decision-making,
we started from the collection of emergency data available, going through all the steps
of the transformation of the raw data into information, including how to visualize
the information produced and, finally, how to make decisions based on the visualized
information. We found and confirmed the same evidence as the previous ethnographic
study. However, this time, we focused on how the practitioners visualize the information.
Again, we did not witness any impactful emergencies, so we could not observe effective
emergency response. We confirmed what we observed two years earlier in the ‘World Cup
Operation’. The emergency managers have visual tools but they were underused in the
information analysis for decision-making.
On top of these two studies, we also observed the day-to-day activities of COI, not
necessarily large operations, such as simulated exercises. On two opportunities, the
emergency managers of the state of Bahia simulated terrorist attacks during a football
match (GLOBO, 2016) in the Fonte Nova Arena5 , the football stadium where the World
Cup and Olympics football matches took place in Bahia. Another exercise was carried out
on the Subway system (CORREIO, 2016) simulating bombs, kidnappings and shootings
in an area of mass transportation. Many agencies participated in these exercises, training
their operational forces for possible emergencies.
As said before, COI is installed in the city of Salvador. Every year, in February, the
city holds one of the biggest street parties in the world (GUARDIAN, 2010). That
is when, during eight consecutive days, 2 million people participate in the Carnival
celebrations and parades. The COI managers call this operation ‘Carnival Y Operation’,
where Y is the year. We observed the operations in the Carnivals from 2015 to 2020.
This operation is considered the largest movement of military troops and equipment
in peacetime, so the relevance and the complexity of this operation are enormous. As
well as the other studies mentioned in this Chapter, our observations could identify the
main agencies, the communication between them, the tools and some standard operating
procedures, mainly administrative ones (preparedness). No complex response was used
because the incidents that occur in the ‘Carnival Operation’ have a controllable impact,
e.g. thefts, brawls, drunkenness and minor bodily injury. We saw that visual tools are
used to share information between agencies for sense-making but there is a lack of use
for decision-making.
4
In July 2016, COI was inaugurated incorporating the functions of CICCR-BA
5
http://www.itaipavaarenafontenova.com.br/
We found that the agencies in COI use a common tool in real and simulated
operations. This tool is Risk Manager and it was developed by SESGE for Brazilian
C2 Centers of headquarter cities of the 2014 World Cup. In our observations, we noted
that Risk Manager is underused. Almost all agencies use their own tools to collect and
analyze data that is most relevant for their agency. This impacts information integration.
We found that emergency managers hardly ever used the information visualized by these
tools to make their decisions. We also found that the a small subset of the information
analyzed in these tools is compiled with other information gathered with tacit knowledge
of the practitioners6 in a printed report. If needed, the authorities make strategic
decisions based on these reports. In all of these observed scenarios, we consider that it is
possible to go further in the use of visual tools and VA has the potential to help in this
area.
4.2.3 Questionnaires
During the 2016 Olympics and Paralympics Games, we ran an online questionnaire and 49
officials from various agencies from different C2 Centers participated (Table 4.4). There
were multiple choice and open questions. The questionnaire is available in Appendix A
(in Portuguese).
Table 4.4 Number of Respondents of the Questionnaire

COI CICCR-RJ CICCN-DF CEIC TOTAL
Olympics 37 1 5 1 44
Paralympics 4 1 0 0 5
TOTAL 41 2 5 1 49
The objective of the questionnaire was to investigate if the emergency practitioners

believe that our Research Question is relevant and feasible. We divided the questionnaire
into three parts: (1) characterization of the practitioners, to know their competences
and experience; (2) identification of the problems, to confirm if the problem that we are
addressing is relevant; (3) suggestions for solutions, to listen to the insights from the
practitioners pointing out solutions or workarounds to the problems. We had relevant
feedback from the questionnaires that is showed in Figure 4.4.
First, we found that most of the participants are experienced, 72% had more than
10 years of work experience and 67% had more than 10 years of C2 experience (Figure
6a and 6b). Second, Figure 6c and 6d shows that most agreed that the quantity of
information is huge (63%) and the quality is not good (55% answered ‘Poor’ or ‘Fair’).
Other relevant findings were: 57%, ‘Never’ + ‘Seldom’, said that there is lack of tools
and 65%, ‘Poor’ + ‘Fair’, disagree that the existing tools provide effective information
(Figure 6e and 6f). None of the 49 practitioners answered ‘Always’ for the question in
6
Tacit information is the type of information that is added to a report that was not collected using
a rigorous process, e.g. data from informal communications or tacit knowledge (experience) from the
emergency manager
Figure 4.4 Summary of the questionnaires: a) work experience; b) work experience in C2; c)
information quantity in C2 d) information quality in C2; e) available tools to analyze information
for decision-making; f) how visual tools represent the information
Figure 6e nor ‘Excellent’ for the question in 6f. The last item in the questionnaire was
an open question for the practitioners to comment on the survey. About 10 practitioners
said that visualizations should improve their knowledge of the emergency operation
supporting their activities. The compilation of the questionnaire shows that there is
much to contribute in the area.
4.3 TARGET RESEARCH DATA COLLECTION (SUBCYCLE II.C)
The purpose of this research data collection was to understand the overall dynamics of
EM processes in depth, comprising the main themes of how managers analyze emergency
4.3 TARGET RESEARCH DATA COLLECTION (SUBCYCLE II.C) 39
information for decision-making and primarily if we can improve it by using visualization

techniques.
Two methods were used. First, we carried out a comparison of visualization tools
available in the C2 Centers that we accessed. Second, we conducted an empirical survey
(WOHLIN et al., 2012) based on semi-structured interviews (SEAMAN, 1999), following
a strict protocol to collect research data. In this research data collection, we focused
more on which themes should be important for a computational model solution for the
problems that we found up to that point in the research (information overload and
non-dedicated information). The main objective of these methods of collecting research
data is to know which themes are important for our computational reference model .
4.3.1 Comparison of Visualization Tools
We planned to compare visualization tools developed for EM that we had access to. We
aimed to identify if the tools implement VA to a certain extent and how they use it. Our
aim was not to assess the tool itself, but the visualizations that it provided. We did not
compare non-functional requirements, such as performance or safety, either. We focused
our comparison on parameters of EM and Information Visualization.
In total, we compared seven visualization tools from four C2 Centers (COI,
CICCR-RJ, CEIC and DZPPP), the Ushahidi Platform (HEINZELMAN; WATERS,
2010) and two versions of the academic research project Rescuer tool (JUNIOR et al.,
2017). In general, C2 practitioners use proprietary tools purchased from software vendors.
In a few cases, we found tools developed by an internal team. We did not find C2 Centers
using any open source or academic tool.
As we have mentioned in Subsection 4.2.2, the name of the tool used in the COI is
Risk Manager and a vendor was contracted via a bidding process carried out by SESGE
to develop this tool. For reasons of confidentiality, we do not have authorization to
detail nor to publish screenshots of the tools of the C2 Centers. Ushahidi is an open
source platform that has been used in emergencies such as the Haiti earthquake. Rescuer
tools are deliverables from an international research project supported by the European
Union and the Brazilian research agencies (see Section 3.1). Just to illustrate, Figures 4.5
and 4.6 and 4.7 show print screens of visual examples of Ushahidi and Rescuer project.
The selection of these tools proved to be very convenient. We could compare C2 tools in
production against an open source tool and against a tool developed in a research project.
To compare the tools, we used the same facets used in the systematic mapping of
DUSSE et al. (2016a). Table 4.5 shows the facets. We divide them in two classes:
Information Visualization and EM. In the former, we identify the visual paradigm, visual
attributes, interaction mechanisms, context-sensitiveness and user focus; and in the latter
we identify data sources, phases, types, environments and platforms. The complete
extraction is available in Appendix B.
All the tools provide a combination of maps and icons as visualizations. This visual
paradigm presents an overview of the emergency. The details of what is happening at
the emergency location are not present in the visualizations. Emergency managers need
Figure 4.5 Ushahidi map showing the locations most affected by the 2011 Tsunami in Japan
adapted from HEINZELMAN; WATERS (2010)
Figure 4.6 Version 1 of Rescuer Tool. Map View adapted from JUNIOR et al. (2017)
interaction mechanisms (zoom, filter, query, sort, aggregate, expand, among others) to
see the details. All the tools make use of the same paradigm, maps, icons, shapes, colors
and so on to all respective users. In general, the visualizations of each tool are the same
for all types of emergencies.
Figure 4.7 Version 2 of Rescuer Tool. Incident Detailed View adapted from JUNIOR et al.
(2017)
Table 4.5 Facets to Compare the Visualization Tools

Classes Facets
visual paradigm,
visual attributes,
Information
interaction mechanisms,
Visualization
context-sensitive,
user-centered.
data sources,
phases,
Emergency
types,
Management
environments,
platforms.
A relevant feature of these tools is that users cannot configure the visual attributes as
they wish. They are standardized, for example, the user cannot rename objects (icons),
change objects colors and sizes, etc. The user cannot adjust her/his visual workspace to
facilitate the way she/he works.
The visualizations do not take into account the particularities of agencies. For
example, one icon plotted with the letter ‘H ’ on the emergency map which could represent
a H ospital for one agency, while it could represent a H ydrant for another. Other examples
are temperature, time zone, geographical coordinates, among others. An agency could
use degrees Celsius to measure temperature while another agency could use Fahrenheit,
similarly one could use the local time zone while another Greenwich Mean Time Zone
(GMT) and even more one could use sexagesimal geographical coordinates (degrees,
minutes and seconds) while another can use Universal Transverse Mercator (UTM)
coordinates. These examples are just a few of what we found during the comparison
of the tools.
In addition, we found that all the tools are used in the response phase, outdoor
environments and they use web platforms; except for Ushahidi which also has a mobile
version.
Another relevant finding is that the visualization tools do not consider the behavior
of the users. In fact, the tools are passive, i.e. the tools do not acquire knowledge from
information received such as past users’ interactions or information from past emergencies.
We identified that particularities of the context and the user preferences are themes
that were not taken into account in the development of these visual tools. From the
examination of these tools, we think that it is possible to go further in the use of VA in
visual tools. We found that the state-of-the-practice neglect some relevant requirements,
e.g. being context-sensitive and user-centered.
As we said in Chapter 2, VA is more than Information Visualization, it is a
multi-disciplinary science that attempts to bring more intelligence into the analytical
process by learning from users’ behavior, by the effective use of the visualization and by
the perception of the context in which visualization is being used (KEIM et al., 2008).
So we expect that research and development in VA in the EM context can improve
current tools tackling the issues mentioned above. We believe that the evolution of the
tools making them more complete and friendly can improve the data analysis by the
emergency managers and, consequently, can improve decision-making in EM.
4.3.2 Survey Protocol (Interviews)

During the subcycle II.C, we used a second method to collect targeted research
data. We conducted semi-structured interviews with six emergency managers (domain
practitioners) from different places and countries: five Brazilian and one French, from four
different C2 Centers, three practitioners from COI, one practitioner from CICCR-RJ, one
practitioner from CEIC and the last one from DZPPP. Since the interviewees had strict
time constraints, the interviews took no more than 40 minutes on average.
We formalized the interviews in a Survey Protocol which defines the method for
research data collection based on the rules in CIOLKOWSKI et al. (2003). The survey
approach consisted of six steps (see Figure 4.8):
• Step 1. Survey Definition:
We determined the goal of the survey based on the Research Question. This is to
investigate what the themes are and how they influence EM decision-making and how
we can treat those themes using VA.
• Step 2. Survey Design:

We designed the survey defining interviewees, terms, materials, tasks, criteria,
attributes, among others. We chose the methods to collect and analyze the research
data.
Figure 4.8 Steps of the Survey Protocol adapted from CIOLKOWSKI et al. (2003)
We decided to conduct semi-structured interviews with at least three subjects

(practitioners). Semi-structured interviews are open-ended, more dynamic and allows
improvisation on the part of the interviewer and scope for a fuller exploration of the
studied objects (RUNESON; HÖST, 2009). We decided to use only practitioners as
subjects because they are familiar with the challenges of EM and face them every
day in practice. We made two terms: term of consent and term of authorization of
data publication. In the former, the subject consents to her/his participation with full
knowledge of the nature of the procedures, with free will and without any coercion.
The latter authorizes the authors to publish the collected research data as they see fit,
preserving the confidentiality of individual data such as the name and workplace of the
interviewee.
Based on the Research Question and the research data collected, we defined three
preliminary key themes to guide the research data collection and analysis. Table 4.6
describes these themes.
Table 4.6 Preliminary Key Themes

Id Key Themes Description
This key theme refers to a set of values of
variables collected from a source, with no
1 Raw Data
handling, with which one can obtain
information to support key theme 2.
Information This key theme defines the area that we are
2
Visualization investigating to support theme 3.
This key theme defines the main process,
3 Decision-making that our research aims to support
in the EM area.
We formulated the questions and decided to do a pilot interview (CIOLKOWSKI et

al., 2003). The pilot was important to make sure that the Protocol was strict and reliable.
The pilot allowed us to revise the Survey Design, if we detected failures or omissions in
the Protocol. We selected and invited the three subjects (S0 , S1 and S2 ). S0 was our first
interviewee and we used this interview as the pilot. S1 and S2 were our first two subjects.
After the interviews, we asked the subjects to recommend other potential subjects. The
Survey Design, which includes the preparation of the interview with the questions, the
terms and other materials, is available in Appendix C (in English and Portuguese).
We decided to use the Coding technique to analyze the research data. We chose this
technique because it has been effectively used in literature to analyze qualitative research
datasets, such as transcriptions of interviews (SALDAÑA, 2015). This technique allowed
us to classify the information in codes, categories and major themes.
Lastly, we designed a Focus Group session with practitioners to validate our analyses
(SOBREPEREZ, 2008). The Focus Group involved two researchers, one as moderator
and the other as observer and a group of practitioners, some were interviewed and others
were not.
• Step 3. Survey Implementation:
We provided the material necessary for the interview (subject invitation, room, audio
recorder, pen, paper, etc.). In this step, we also did the pilot study to refine the Protocol,
however, we did not use the research data of the pilot in our analysis. We used it for the
sole purpose of evaluating the effectiveness of the Survey Protocol. The interviews were
conducted over the years 2016 and 2017. We scheduled time and place with the subjects
a few days in advance. In general, we interviewed them in their workplaces and in closed
rooms so that we were not interrupted.
• Step 4. Survey Execution:
As mentioned, we carried out six interviews (seven with the pilot) with practitioners,
from four different C2 Centers and two different countries. We recorded all the interviews.
In the Opening Step of the interview, as already mentioned, the subjects were asked
to sign two terms: the consent form and authorization form. All subjects signed the
consent form but two out of the six signed the authorization form with reservations.
They said that the authors could use what was spoken but without making any reference
to the name of the interviewee nor where she/he worked, as well as the observations that
we had made, e.g. building plant, position of the rooms, staff. Otherwise, we would have
needed authorization from their superior to use such research data. In some visits, we
had to sign terms of confidentiality of the C2 Centers that we were visiting, accepting
not to disclose sensitive information from the C2 Center. For this reason, we did not
connect any name, role or C2 Center with the subjects or research data provided by a
specific subject. For this reason, in this text we will not reference the subject(s), we will
refer to them simply as S0 , S1 , up to S6 .
• Step 5. Survey Analysis:
Between the interviews and the analysis, we transcribed the interviews verbatim
by listening and re-listening to the recording. All the interviews were carried out in
Portuguese, apart from one in English. Concerning this specific interview, after the
transcription, we translated it to Portuguese. The transcripts of the six interviews
produced 116 pages of text.
We discarded one interview (S6 ) because the subject’s answers were too superficial
and, despite our attempts, he was not able to make the link between the preliminary key
themes. We found that this subject did not have adequate knowledge to meet our goal
defined in the Survey Definition. We assumed that discarding this interview would not
affect the results.
After the transcription of all the interviews, we sent emails to the subjects, attaching
the transcribed text of their interview. In general, in an interview, people do not have
much time to think about their answer. The purpose of the emails was to give them the
opportunity to review their answers in their own time. As the interviewees work with
strict time constraints, few returned our emails. Those who replied to our emails did not
make relevant comments.
We also interacted with the subjects, days after the interviews, mainly with the ones
that worked in COI. Whenever doubts arose regarding what some subject meant at some
point in the interview, we looked for the subject and tried to clarify the doubts. These
informal conversations with the practitioners took no more than 2 minutes on average and
they were repeated countless times. These informal conversations do not constitute the
same scientific evidence as the Survey Protocol, because we did not plan for, nor record,
them. However, the research data collected in these conversations were important for our
study, specially to mitigate some threats to validity, as discussed later in Chapter 8.
In our preliminary analysis, we found that according to the answers of the subjects,
managing an emergency is very complex. Many themes can influence the whole process,
from the planning and data collection to the decision-making and the action execution.
In general, we found that some themes were cited frequently by the subjects: the type
of the emergency and its context, lack of resources, C2 management systems, political
issues, the profile of the emergency manager and the particularities of the agency of
this manager. We also found that the behaviour of the managers impacts directly on
the process of decision-making. The subjects said that factors; such as beliefs, values,
attitude, organizational culture, subjective norms, team confidence and autonomy; and
sentiments; such as pride, vanity, motivation, sadness, anger and so on; can change the
behavior of the decision maker. About the visual tools themselves, they outlined that
tools are not prioritized as they should be. They added that they need investment and
research in this area.
We conclude that in a multidisciplinary domain, such as EM, it is very hard to
prioritize the themes that influence decision-making and it largely depends on the context.
You cannot neglect any theme, otherwise the response can be disastrous.
Next, we analyzed the research data in depth using a qualitative analysis method.
We identified specific segments of text to code, categorize and synthesize into major
themes. As mentioned in Step 2, in order to analyze the research data collected we

decided to use the Coding technique that we shall explain in the next Chapter.
• Step 6. Survey Packaging:
The objective of the packaging step is to disseminate the survey so that external
parties can understand the survey scope and the results. We published a paper about
the Systematic Mapping Study (DUSSE et al., 2016a) in a Qualis A1 Journal. We
wrote papers about the initial steps of our work and submitted it to the International
Conference of International Conference on Information Systems for Crisis Response
and Management (ISCRAM), Qualis A3, in three consecutive years: in 2018 (DUSSE;
NOVAIS; MENDONÇA, 2018), in 2019 (DUSSE; NOVAIS; MENDONÇA, 2019)
and, finishing the trilogy, in 2020 (DUSSE; NOVAIS; MENDONÇA, 2020). We
submitted another paper to a relevant journal about the results of our work and we
are waiting for the feedback. We also consider this thesis content as part of the packaging.

This Chapter described the Cycle II of this thesis: data collection. For didactic reasons,
we divided this Cycle into three subcycles and we explained each subcycle in one Section
of this Chapter. In Section 4.1, in a nutshell, we introduced the formal review of the
literature that we carried out: a Systematic Mapping Study (Subcycle II.A).
Section 4.2 presented the second subcycle, which we called the general research
data collection (Subcycle II.B). In it, we used techniques such as documents appraisal,
ethnographies and questionnaires to collect the research data. These techniques helped
us to understand the problem and collect more in-depth data to propose solutions.
Section 4.3 explained the third subcycle, which we called target research data
collection (Subcycle II.C). Here we compared the current visual tools in the market
(proprietary and open source) and academic tools. We also did a Survey Protocol with
a strict design. The Survey Protocol consisted of interviews with practitioners in the
domain. Its design describes all the steps of the survey, with pilot study and so on, to
make it more easily replicable by others.
Next Chapter explores Cycle III: the research data analysis.
Chapter
5
This Chapter presents the third Cycle of this thesis: data analysis. It describes methods, tasks
accomplished and contributions of the cycle to the thesis.
CYCLE III: RESEARCH DATA ANALYSIS
In order to analyze the large amount of research data that we collected, first we
crosschecked and triangulated them. We found many similar findings among the
collection methods that were complementary. For example, we identified an issue in
the interview and we were able to focus on this issue using other methods. Other
times, we obtained research data from literature, observations or questionnaires and
had the opportunity to go deeper while interviewing. Apart from this triangulation
analysis, we are aware that to produce scientifically acceptable findings to answer our
Research Question, a higher level of rigor in the analysis of the data collected is required.
Thus, we applied a formal qualitative technique on the research data collected during
the interviews called Coding (SALDAÑA, 2015). We used the Coding technique to
understand the main themes covered and incorporate these into visualization tools. The
Coding allowed us to perform a thematic synthesis, identifying patterns in excerpts of
text, clustering them into categories, which in turn we clustered into themes.
5.1 TRIANGULATING FINDINGS OF THE COLLECTED METHODS

One of the main challenges of this research was to find proper methods to analyze the
research data, first because of the wide amount of research data that we collected, and
second because the analysis approach must be aligned with our Research Question and
research hypothesis. Therefore, we were very careful in our analysis. The use of methods
and techniques based on the qualitative research approach greatly enriched the content
of this thesis. The fact that the author works with practitioners facilitated contacting
them and helped in executing the data collection activities. The methods applied, such as
documents appraisal, semi-structured interviews, transcriptions, translations, coding and
others, were different in nature and were not automated. This decreased the possibility
of errors and added qualitative value to the results.
47
48 CYCLE III: RESEARCH DATA ANALYSIS
Table 5.1 Knowledge and Findings of Research Data Collection

SUBCYCLE METHOD PRELIMINARY RESEARCH FINDINGS
A1. Number of published studies that apply visualizations
II.A) to EM is growing;
FORMAL Systematic A2. Most research uses similar techniques such as icons
REVIEW Mapping and colorful shapes plotted in maps;
OF THE Study A3. The studies prevail in Response phase;
LITERATURE A4. The studies prevail a specific emergency type;
A5. The studies prevail Outdoor scenarios;
A6. VA is a trend in the area but its full potential is not
being applied in the state-of-the-art.
B7. Evidences about the doctrine of C2;
Official
B8. Know critical procedures adopted by C2 Centers;
Documents
B9. Know the infrastructure and Information Visualization
tools of the C2 Centers.
B10. The quantity of information to analyze is huge;
II.B) B11. The quality of information to analyze is not good;
Questionnaires
COLLECT B12. There is lack of visual tools;
GENERAL B13. The practitioners disagree that the existing tools
RESEARCH provide effective information.
DATA B14. Know the operation of a C2, focusing in the
use of visual tools in decision-making;
B15. Lack of integrated visualizations tools;
B16. The agencies use their own visualization tools for their
Ethnographic
specific activities;
Studies
B17. The common (official) tool is underused;
B18. The tools are used mainly in the Response phase;
B19. The tools are not effective for Indoor scenarios;
B20. The common tool is not context-sensitive nor
user-centered;
B21. Lack of use of visualizations in decision-making.
C22. All tools use similar techniques such as icons and
colorful shapes plotted in maps;
C23. Do not take account the particularities of the agencies;
C24. Do not take account the profile of the practitioners;
II.C)
Visualization C25. Do not permit multi data sources;
COLLECT
Tools C26. The tools are used in Response phase;
TARGET
Comparison C27. The paradigms of visualization of each tool are the
RESEARCH
same for all types of emergencies;
DATA
C28. The tools are used in Outdoor scenarios;
C29. The platforms of the tools are practically 100% web.
C30. Know the use of visualization tools in
Survey
decision-making, what themes can impact it;
Protocol
C31. The visual tools are not prioritized as they should be;
(Interviews)
C32. Decision-makers do not use the visual tools, they use
reports;
C33. Relevant themes, e.g. context and human behaviour,
are being neglected.
5.2 CODING 49
Considering all studies of data collection, we summarize our 33 main research

knowledge and findings in Table 5.1. We highlight and identify each of them with a
letter and a number. The letter is the reference of the subcycle: A = Subcycle II.A, B =
Subcycle II.B and C = Subcycle II.C. The number is sequenced through the subcycles.
By compiling the findings presented in Table 5.1, it is clear that some methods come
to similar outcomes. For example: ‘A2. Most research uses similar techniques such as
icons and colorful shapes plotted in maps;’ is similar with ‘C22. All tools use similar
techniques such as icons and colorful shapes plotted in maps;’. Table 5.2 shows examples
of this similarity.
Table 5.2 Triangulated research data of the findings in Table 5.1

Id Similar finding
A2, C22 Icons and colorful shapes plotted in maps
A3, B18, C26 Response phase
A5, B19, C28 Outdoor scenarios
A6, B13, B17, B21, C32 Lack of use of visualizations in decision-making
B16, C23 Particularities of the agencies
B20, C23, C24, C25 Not context-sensitive nor user-centred
Among all the methods to collect research data reported in this thesis, listening to
the practitioners was the most relevant one. The strict protocol used in the collection
and the relevance of the research data collected demonstrate this. It is important as it
casts light on the respondents’ experiences. Their narrated experiences are very hard to
gather from documents or ethnographic studies. Although very relevant, data collected
through interviews generally have some characteristics which reflect the interviewee’s:
personality, subjectivity and often biased opinions. It is necessary to use techniques to
mitigate and encode the most relevant data in an interview script. The next Section
addresses these issues.
5.2 CODING
Coding is an analytical technique used to extract quantitative data from qualitative
datasets, such as transcriptions of interviews, participant observation field notes,
journals, documents, photographs, video, websites, e-mail correspondence and so on. A
code is most often a word or short phrase that symbolically assigns a summative, salient,
essence-capturing and/or evocative attribute for a portion of language-based or visual
data (SALDAÑA, 2015).
As we mentioned in the previous Chapter, we carried out semi-structured interviews,
more of an informal interview, almost a conversation, that in turn enable us to talk
about various topics without missing important details of the research data collection.
However, this interview technique can also provide irrelevant information for the research.
Therefore, we define in our Survey Design a code inclusion criterion.
We based our code inclusion criterion on the key themes defined in the Survey Protocol
which are: raw data, Information Visualization and Decision-making (see Table 4.6).
We recorded the audio of the interviews to listen and re-listen to them. If the subject
mentioned, directly or indirectly, one of the key themes in the interview, then we extracted
the exact excerpt from their speech and, after summarizing it, we described it with one
or more codes.
In order to facilitate this scheme, we performed steps that we believe were productive
in the Coding process (see Figure 5.1).
Figure 5.1 Coding Steps
• Step 1: We extracted excerpts from the transcribed interviews verbatim and stored
them in a first spreadsheet. If the interview was carried out in other language, we
translated it into Portuguese (step 0);
• Step 2: We mapped each row of the spreadsheet with a code id;
• Step 3: We copied this content in a second spreadsheet and summarized each line,
without losing the spoken information by eliminating connection phrases, repeated
or meaningless words;
• Step 4: We changed some words using synonyms and jargons of the area in order
to standardize the excerpts;
• Step 5: We rephrased negative phrases into affirmative ones without losing the
original meaning (e.g.“the room is not light” we rephrased to “the room is dark”);
• Step 6: We selected keywords from each row and summarized it again in a third
spreadsheet;
• Step 7: Based on the keywords, we defined our codes in a fourth spreadsheet. A

set of keywords can derive one or more codes.
Thus, we simplified multi-line excerpts into a small set of keywords. This greatly
helped our Coding process. Using these keywords, it was easier to code each row of the
third spreadsheet. In some cases, we defined more than one code per row (step 7). In
5.2 CODING 51
these cases, we split the row according to the number of codes. In each new row, we
mapped a unique code with one, some or all keywords, depending on the case.
After the extraction, we searched for patterns in coded data to categorize them. We
used constant comparison, scoring, scaling, filtering and clustering of codes in the analysis
of each dataset collected from the interviews. Using the approach described in BOYATZIS
(1998), of building a qualitative description of the frequency of terms and forming clusters
of them, we identified relationships between the codes summarizing them into categories.
That is, when we found a reference to a particular category during this analysis, we could
either create a new category or refine ones that already existed. Thus, we improved the
Coding classification with only relevant categories.
After the categorization of all codes, we applied an exclusion criterion, that we also
defined in our Survey Design: we eliminated categories that only had codes from the
same subject. When this happened, we tried to reallocate the codes in other categories
that had codes from a distinct subject. However, in some cases it was not possible, so we
excluded the category and consequently the codes. Thus, we only maintained a category
if at least two codes from different subjects belonged to this category. We believe that
this eliminates individual and biased opinions held by the subjects. The relationship
between codes and categories is ‘2..N’ and ‘1..N’, i.e., one (1) category contains two (2)
or more (N) codes and one (1) code is contained in one (1) or more (N) categories.
Then, in order to discern and label the Coding categories, according to the needs of the
investigation, we clustered the categories into major themes based on their relationships.
Themes reduce large amounts of codes into a smaller number of analytic units. The
thematic analysis process consisted of identifying, analyzing and reporting patterns
(themes) within the Coding category. As well as the code and category relationship,
the relationship between categories and themes is ‘2..N’ and ‘1..N’.
This sequence of tasks, i.e. the code inclusion, categorizing the codes, code exclusion
and clustering the categories into themes, is our code process that comprised the overall
hierarchical Coding classification. This process was not trivial, because a code may belong
to more than one category, which in turn may belong to more than one theme. In total,
we built about 300 codes. We did refinement iterations and at the end of the Coding
process, we reduced this to 180 codes, of which 156 are unique (i.e. the same code could
be in more than one category). We classified the 180 codes into 43 categories (35 unique),
from which we synthesized 6 major themes. The clustering is shown in Figure 5.2
We defined a hierarchical Coding classification for the relevant themes that might exert
influence on the Emergency Management (EM) decision-making and how visualizations
can address part or all of these themes. Table 5.3 shows part of this hierarchy (without
the codes). The whole hierarchy is available in Appendix D. In some cases, a code can
have the same name of a category that in turn can have the same name of a theme, e.g.
“Processes” category (id 08) and “PROCESSES” theme. This duality of name happens
in all themes except for “CONTEXT”. The ids of categories “Data Management”,
“Information Visualization” and “Decision-Making” begin with the character ‘P’ to
indicate that they are examples of processes. In Table 5.3, we highlighted the categories
that belong to more than one theme. Bold categories are those that are repeated in
2 different themes, e.g. “Technology X Employees”. Bold and Italic categories are
Figure 5.2 Clustering codes and categories
those that are repeated in 3 different themes, e.g. “Information Visualization”.

Bold, italic and underlined categories are those that are repeated in 4 different themes,
e.g. “Decision-Making”. No category is in 5 or 6 themes.
Lastly, we scheduled a Focus Group session, based on the guideline proposed by
SOBREPEREZ (2008). The objective of the Focus Group is to validate our analysis
with practitioners, mitigating some threats to validity, e.g. biases of the authors of the
research.
5.3 FOCUS GROUP

We looked for EM practitioners to evaluate our data analysis. We decided to carry
out a Focus Group to validate our findings doing a double-check with practitioners, e.g.
categories and themes identified by the Coding.
We executed the Focus Groups session following similar guidelines used in the
interviews. We audio recorded the entire session. In the opening of the session, the
practitioners were asked to sign the same two terms of the Survey Protocol: the consent
form and authorization form. We asked the specialists to turn off their cell phones
to focus on the discussion without interruptions. After the session, we transcribed it
verbatim (in Portuguese) by listening and re-listening to the recording. We also send the
session transcription to the practitioners to review their positions in their own time. We
decided to send the session transcription to all the subjects of the interviews (see Section
4.3). We were interested in their opinions, however, we had no email feedback. And we
also interacted with the Focus Group practitioners, days after the session, to clarify some
issues.
The Focus Group was semi-structured and guided by six open-ended questions. Like
the interviews, the discussions were not restricted by the questions, they were open-ended,
dynamic and we used improvisations on the part of the interviewer within our scope of
interest. We present the questions below.
• Question 1 (Q1): Are there any of the major themes proposed that you do not
5.3 FOCUS GROUP 53
Table 5.3 Coding Classification: Categories X Themes

id CATEGORIES THEMES
01 Lack of Resources
02 Roles of the Agencies
03 Technology X Employees
04 Geography
CONTEXT
05 Communication
06 Risk
07 Resilience
08 Processes
09 Lack of Processes
10 Unknown Processes
11 C2 Management System (ICS)
12 Problems on Changes
13 Function Deviation
PROCESSES
14 Bureaucracy
15 Politics
P1 Data Management
P2 Information Visualization
P3 Decision-Making
16 Information Security
17 Information Needs
18 Raw Data Searching
19 Raw Data Collecting
20 Raw Data Processing
21 Information Amount
22 Information Heterogeneity DATA
23 Information Sharing MANAGEMENT
24 Information Diffusion
25 Information Integration
26 Information Analysis
27 Knowledge
P3 Decision-Making
28 Check Processes
29 Visual paradigms
INFORMATION
26 Information Analysis
VISUALIZATION
27 Knowledge
P3 Decision-Making
30 Inexperience HUMAN
31 Practitioner Competence BEHAVIOR
P3 Decision-Making
DECISION-
32 Human Behavior
MAKING
consider relevant? Please, explain your answer.
• Question 2 (Q2): Are there any of the categories proposed that you do not consider
relevant? Please, explain your answer.
• Question 3 (Q3): Are there any relevant themes that in your opinion are missing?
Which one? Please, explain your answer.
• Question 4 (Q4): Are there any relevant categories that in your opinion are missing?
Which one? Please, explain your answer.
• Question 5 (Q5): Are there any categories in a wrong theme? Please, explain your
answer.
• Question 6 (Q6): Do you have any other comment, recommendation or suggestion

for improving our classification?
The Focus Group meeting was held on February 8th, 2018 at COI, during a large
event called Carnival of Salvador. We set up the meeting with two researchers and three
practitioners. One researcher played the role of moderator and the other one observer
taking notes during the session. One practitioner participated in the interviews (S1 ),
another did the pilot interview (S0 ) and the third practitioner did not participate, until
that moment, in the survey (S7 ).
We previously arranged with S0 and S1 to be silent at the beginning, so S7 was the
first to assess the result of the Coding.1
After a brief explanation of our study goals and what we had done until that moment,
we asked S7 if the categories and the themes are representative (Q1, Q2, Q3, Q4 and Q5).
S7 agreed. Then subjects S0 and S1 also validated our preliminary results positively.
They gave compliments and queried some missing categories but, after a brief discussion,
everyone accepted that the 35 existing ones represented the categories they thought were
missing (e.g. they missed a ‘Budget Constraints’ category, however, they agreed that the
“Lack of Resources” category covered this).
Regarding Q6, “other comments”, subject S0 add two remarks:
“Brazil chooses the military practices of Command and Control. Perhaps, in the last
decade, due to immaturity, it was the right model, but today it is more than proven that
this model is saturated ”.
“Militarism is too rigid. The problems, nowadays, are dynamic. We need flexible
actions, flexible tools, each case has its particularities, which cake recipes do not solve.
So I think that your proposal is complete and has the flexibility we need to operate in the
various (emergency) cases”.
1
It was the first contact of S7 with our study, so we decided to do this to mitigate bias, i.e. avoiding
that she/he just confirmed the opinions of S0 and S1
5.4 IN DEPTH FINDINGS AND KEY INSIGHTS 55
And subject S1 concluded:
“The current tools do not support effective actions to reduce response time or
operational errors. I think this is the reason why there are so many negative opinions on
this topic. I agree with him” (Practitioner S0 ). “I think you are researching themes that
we do not give due attention”.
5.4 IN DEPTH FINDINGS AND KEY INSIGHTS
As we said in Chapter 4, in the description of the Survey Design, one interview was
used as a pilot study, another was discarded, and six interviews were coded, producing
116 pages of text (about 60.000 words). On average, we spent four hours on preliminary
Coding for each transcript (first spreadsheet in Figure 5.1). The other tasks, which
include filling in the other spreadsheets, the identification of codes, categories and themes,
demanded 3 months in total. The transcription of the Focus Group sessions was also time
consuming. The session lasted about 70 minutes and its transcription produced 31 pages
of text (about 15,000 words). All this material produced valuable insights on the use of
VA in EM management.
Our main finding of the data analysis is the Coding classification. We build a
hierarchical classification with these three items: codes, categories and themes. The
main idea of the Coding is to synthesize the raw data (into codes) and to find patterns
(categories and in turn themes) to try to answer the Research Question. With the Coding,
we could identify what aspects are relevant to make a model based on the knowledge of
those who are familiar with the challenges of EM and face them every day in practice:
the EM practitioners. Thus, we built 180 codes (156 unique), 43 categories (35 unique)
and 6 major themes (Table 5.3).
In summary, based on our Research Question, we defined three key themes which are:
raw data, Information Visualization and Decision-making (see Table 4.6). We refined
these three key themes into six Coding themes (CONTEXT, PROCESSES, HUMAN
BEHAVIOR, DATA MANAGEMENT, INFORMATION VISUALIZATION
and DECISION-MAKING). Our data analysis identified the relevance of other
themes, such as context, processes and human behavior which are fundamental to
decision-making. We concluded that researchers and practitioners do not pay due
attention to major themes that they should be aware of nor incorporate these into
visualization tools. Therefore, we found out that tackling information overload is
necessary but not sufficient to contribute effectively in this research area. We need
to tackle another problem that we call non-dedicated information (e.g. information
about the emergency context, processes and human behavior). Likewise, we believe that
Information Visualization is necessary but insufficient to tackle both problems.
During our research, we also identified gaps and trends in the area. One of
these trends is the use of Visual Analytics (VA). We found works in literature,
which addressed VA in emergency contexts that, according to the authors, generated
excellent results (ANDRIENKO; ANDRIENKO; BARTLING, 2008), (TOMASZEWSKI;
MACEACHREN, 2012), (CHAE et al., 2014) and (CIRAVEGNA et al., 2018). VA

enables people to detect the expected and discover the unexpected (COOK; THOMAS,
2005). KEIM et al. (2008) adjusted Shneiderman’s mantra (SHNEIDERMAN, 1996)
from: “Overview first, Zoom, Filter and Details on demand” to: “Analyze first, Show
the Important, Zoom, Filter and Analyze further, Details on demand”. This mantra
is calling for the astute combination of analytical approaches together with advanced
visualization techniques (KEIM et al., 2008). VA concepts led us to research this area in
our work, therefore our solution addresses the two problems above: information overload
and non-dedicated information.
While useful, the use of VA concepts in EM may add activities to the management
processes that may not be easy or fast to adopt. The benefits brought by VA to analyze
emergency information also come along with certain challenges. These include: (1)
how to map VA concepts with the countless variables of an emergency to support the
managers in quickly identifying relevant and reliable information to make decisions; (2)
whether researchers and practitioners are using VA techniques and tools effectively to
tackle challenge 1; and, (3) how to enhance VA applications with minimal impact on
current EM activities. We believe that a computational reference Model based on VA
concepts can be used to tackle these challenges. In turn, our challenge is how to develop
and apply such a model.

This Chapter described Cycle III of this thesis: data analysis. Section 5.1 summarizes all
the findings of the Cycle II. It data triangulates the findings of each collection method used
in our research on the use of VA in EM. The data collection methods were complementary.
For example, we identified an issue in the interview and we were able to focus on this issue
using other methods. Other times, we obtained research data from literature, observations
or questionnaires and had the opportunity to go deeper while interviewing.
In Section 5.2, we presented the Coding technique that we carried out: its steps and
the classification based on a thematic synthesis that we performed clustering excerpts of
the interviews into codes, then into categories and then into themes.
Moreover, Section 5.3 presented the Focus Group that we conducted with domain
practitioners to validate the categories and themes of the Coding classification.
Finally, in Section 5.4, we introduced the in depth findings, covering Coding
themes (CONTEXT, PROCESSES, HUMAN BEHAVIOR, DATA MANAGEMENT,
INFORMATION VISUALIZATION and DECISION-MAKING) and main insights of
our work. Those themes aligned with our Research Question, findings, and key insights
were then used to develop a model based on VA to help the emergency managers in
making decisions, as described in the next Chapter, Cycle IV: the development of the
model.
Chapter
6
This Chapter presents the fourth Cycle of this thesis: model development. It describes Cycle IV, the
tasks accomplished during model development and the contributions of the cycle to the thesis.
CYCLE IV: MODEL DEVELOPMENT
In previous cycles, we found that practitioners and researchers are neglecting important
themes that need to be represented in visualizations tools for decision-making (e.g.
information about emergency context, processes and human behavior). In this Cycle
IV, we report on the results of the findings of the previous Cycles II and III, specifically
detailing the development of the computational reference model. We shall explain how
we derived the analyzed data into a model based on Visual Analytics (VA).
According to BEUREN (1998) a model is an abstraction of reality. Models represent
the relevant components of a situation and the interactions between them. Modeling is an
important tool to simulate a reality through objects, flows or words because it summarizes
the most relevant effects and relationships of a particular situation (MACKENZIE et al.,
2006).
A reference model is an abstract framework for understanding significant relationships
among the components of an environment and for the development of consistent
specifications supporting that environment. A reference model provides a broad view
of, or templates for, the overall setup and its domain implementation. A reference
model is based on a small number of unifying concepts, axioms and relationships within
a particular problem domain and may be used as a basis of reference to stakeholders
(experienced or beginners). It describes entities or domains, their relationships, and the
concerns of stakeholders who need the problem to be solved. A reference model is intended
to promote understanding of a class of problems, not to provide specific solutions for those
problems, being technology agnostic. Therefore, reference models are not directly tied to
any standards, technologies or other concrete implementation details, but they seek to
provide a common semantics that can be used unambiguously across and between different
implementations (MACKENZIE et al., 2006).
A computational model is a mathematical model in Computer Science that requires
extensive computational resources to study the behavior of a complex system by
computer simulation (MELNIK, 2015). The next Section we propose a computational
57
58 CYCLE IV: MODEL DEVELOPMENT
reference model for the domain we are studying.
6.1 COMPUTATIONAL REFERENCE MODEL BASED ON VISUAL

ANALYTICS
Regarding the modeling concepts above and the findings of our research so far, we
propose a computational reference model based on Visual Analytics (VA) to support
decision-making in Emergency Management (EM). This Section will discuss the model
construction and the components of model itself. We used a bottom-up strategy to
develop our model. First, we set up the arrangement of the themes based on our analysis.
Then, we represented the themes and their relationships with geometric shapes, spatial
position (in a 2D plane) and directional edges. We colored and changed the form of the
edge lines because they have different semantics, as we shall explain below.
Let us first focus on the emergency, without considering any external components.
According to our thematic synthesis, given an Emergency x to be managed, there are
the six components to be considered. The first theme is CONTEXT, as the name
suggests it represents the context of the EM. The next theme is HUMAN-BEHAVIOR.
In our model this theme will be the basis of the theme PROCESSES, which
represents the EM process executed by humans. In our model we highlighted
three processes: DATA MANAGEMENT, INFORMATION VISUALIZATION and
DECISION-MAKING. DATA MANAGEMENT encompasses data gathering, storage and
use. INFORMATION VISUALIZATION encompasses the visual interpretation of data
to produce useful information and insights for DECISION-MAKING. Note that there are
several other processes in EM such as planning, administrative, operational, and so on.
However, they are not within the scope our model.
Figure 6.1 The model without external components
Figure 6.1 depicts the relationships of those concepts. The outer square represents
6.1 COMPUTATIONAL REFERENCE MODEL BASED ON VISUAL ANALYTICS 59
a given Emergency x to be managed. Within it, there are the six components we
discussed before. From bottom to top, the first theme is the CONTEXT, represented
by a gray cloud. It is affected by and affects the EM Processes. The next theme
is HUMAN-BEHAVIOR, which is represented by an orange trapezium. As mention
before, this theme is the basis of the theme PROCESSES, which are executed by
human beings to manage the emergency at hand. PROCESSES are represented by an
yellow ellipse and surrounds the other three themes. Within PROCESSES, the model
highlights the processes DATa management, represented by a blue rectangle; INFormation
visualization, represented by a pink rectangle; and, DECISION-MAKING, represented
by a green rectangle. Note that, according to our model, the DATA MANAGEMENT,
INFORMATION VISUALIZATION and DECISION-MAKING themes are a subset of
the theme PROCESSES.
According to the model, the CONTEXT theme always interacts with PROCESSES
theme (red and green arrows). In the real word, the context defines which process tasks
will be executed and how they will flow. The red and green arrows represent these
interactions between the EM context with EM processes. The red arrow is the inputs of
the context that influence the way the processes will be done. In addition, the green arrow
denotes the outputs (results) of the processes that change the context. As an example,
these interactions could be modeled using Business Process Model and Notation (BPMN),
using the gateway element for instance (OMG, 2011).
The shape of HUMAN-BEHAVIOR theme is a trapezoid to draw on a metaphor
of a pyramid base. Our intention is to show that the HUMAN-BEHAVIOR theme
supports the way the decision makers behave, i.e. the way they think and reason to
act, or to react. We highlighted this interaction with a bidirectional blue arrow to and
from DECISION-MAKING. As an example, even implicitly, we consider that the context
influences the human behavior, e.g. decisions maker behavior may change in widespread
panic.
The spatial positions of the three rectangles in the model are intentional and carefully
placed. Similar to a Venn Diagram, DAT, INF and DECISION-MAKING themes
have intersections and disconnections among them. The shaded area represents the
intersection of these three themes. They have intersections in two by two themes and
disjoint areas. These spatial positions mean that the decision-making process uses tasks
of Data Management and Information Visualization processes, but do not always do so.
This modeling implies that Information Visualization is necessary for decision-making,
but it is not sufficient. Other processes may support decision-making, but they are not
within the scope of our work. Moreover, Information Visualization tasks may be used by
other processes, besides decision-making, which are not represented by our model. They
are not within the scope of our work either. Our work focuses on increasing the shaded
area, using visualizations as a means to improve decision-making in EM. This brings us
to the next issue of our model, Visual Analytics and learning from previous experiences.
VISUAL ANALYTICS is a binding element in our model. As discussed in Chapter
2, VA combines strengths from data management, information analytics, knowledge
representation, and knowledge discovery. It, therefore, runs through multiple emergencies
touching issues like, standardization, context awareness, adaption, customization.
Figure 6.2 Visual Analytics Based Model for Decision-Making in Emergency Management
Likewise the C2 DOCTRINE is another binding element, as it establishes how authority

and direction is implemented in C2 Centers so that different operational forces contribute
to the EM efforts.
Figure 6.2 includes those two elements, VISUAL ANALYTICS and C2 DOCTRINE,
into our model. The VISUAL ANALYTICS is a “stable” binding element that interacts
with all six themes in each emergency, for the reasons given above. VA is the basis of
our model. VISUAL ANALYTICS passes through CONTEXT, HUMAN-BEHAVIOR,
PROCESSES, DECISION-MAKING, DATA MANAGEMENT and INFORMATION
VISUALIZATION themes. The purple dashed arrow represents these interactions. As we
mentioned in the Chapter 2, VA is a multidisciplinary area that combines several sciences.
The visualizations follow the VA concepts and shall be adapted depending on aspects
such as: management sciences (context, processes management), social sciences (human
behavior theory and decision-making theory) and computer sciences (Data Management
and Information Visualization). The concepts and techniques of VA shall provide the
guidelines to develop the EM visualization interfaces.
The other important external element is the C2 DOCTRINE. We believe that
decisions made with the support of our model have the potential to improve the C2
Doctrine. We consider that our model can encourage new research in the EM area and in
turn help to evolve the C2 Doctrine. Emergency managers may evaluate a set of decisions
made in a set of n emergencies and may propose changes and improvements in the C2
6.2 INSTANTIATING THE MODEL: A WORKED EXAMPLE 61
Doctrine. The brown dotted line represents this interaction among DECISION-MAKING
theme with C2 DOCTRINE external component.
Our model is not limited to the elements explained above. It provides an overview
of an entire system of Information Visualization not directly tied to any standard or
technology. It indicates that to tackle the problem of making quick and assertive
decisions based on relevant and up-to-date information in the EM domain, the system
needs information on the six themes we have identified. Established models, processes,
tools, systems, theories may emerge based on standards or proprietary technologies to
instantiate these themes. The instantiations of these established objects depend on the
familiarity of the emergency managers with them. For example, the CONTEXT theme
can be instantiated with a Context Model (VIEIRA; TEDESCO; SALGADO, 2011), with
a Feature Model (KANG et al., 1990) or with Ontologies (SIMAS et al., 2017), and so
on. As far as we are concerned, the instantiation of established models, processes, tools,
systems, theories from our model is an open research area and worthy of future work
in various areas of computer science and other sciences as well. Below we presents an
example of instantiation of our model.
6.2 INSTANTIATING THE MODEL: A WORKED EXAMPLE

This Section presents an example of instantiating our reference model. This instantiation
was not formally evaluated with practitioners, but we consider it important to illustrate
the usefulness of our reference model.
Our start point are the themes presented in Figure 6.1. The six themes are
instantiated with six well-known models and processes of the state-of-the-art and of the
state-of-the-practice, as follows:
• the CONTEXT theme is instantiated with the Feature Model (KANG et al., 1990);
• the HUMAN-BEHAVIOR theme is instantiated with the Theory of Behavioral

Prediction (TBP) (FISHBEIN; AJZEN, 2011);
• the PROCESS theme is instantiated with the Incident Command System (ICS)
(FEMA, 2014);
• the DATA-MANAGEMENT theme (DAT ) is instantiated with an adaptation of

the Data Science process of SCHUTT; O’NEIL (2013);
• the INFORMATION VISUALIZATION theme (INF ) is instantiated with an

adaptation of the Information Visualization process of WARE (2012); and
• the DECISION-MAKING theme is instantiated with an adaptation of the

Decision-making process of GLAUTIER; UNDERDOWN (1991).
Once again, we highlight that these themes could be instantiated with other
models, processes, tools, systems, or theories. For instance, we could have instantiated
the CONTEXT theme with the Context Model (VIEIRA; TEDESCO; SALGADO,
2011) or with Ontologies (GRUBER, 1993); or we could have instantiated the

DECISION-MAKING theme with the Analytic Hierarchy Process, AHP (SAATY, 1980)
and so on. The main goal of this Section is to illustrate how our model can be instantiated.
6.2.1 Instantiating the CONTEXT theme

Figure 6.3 shows an example of CONTEXT theme instantiated using a Feature Model.
Techniques used for variability modeling in software product line engineering can be
helpful for modeling commonalities and variabilities in complex and dynamic contexts
(CLEMENTS; NORTHROP, 2002). Regarding the EM context, the benefits of using
variability modeling techniques are related to a comprehensive model to consolidate the
variation points of the emergency, an efficient way to derive parameterized context that
represents a given emergency and the ability to perform consistency evaluation of the
derived context.
We could model the variability of the EM domain through feature modeling (KANG et
al., 1990). As already mentioned, emergencies have characteristics in common; however,
there are particularities related to the current context. Consequently, we mapped the
variation points to enable the configuration of the EM context.
The features have been collected from the methods used in our research in Cycle
III. Eight features (parental features) are essential to define the basic configuration of
Context theme, Datasource, Stakeholder, Agency, Type, Phase, Scenario, Status and
SystemManagement (see Figure 6.3). The relationship between Datasource, Stakeholder
and Agency (parent feature) and their subfeatures (child features) is ‘OR’ (more than
one subfeature can be selected when configuring). For the others, the relationship is
‘ALTERNATIVE ’ (only one subfeature can be selected).
Of course, there are other features that could be used to define an EM context
(TOMASZEWSKI; MACEACHREN, 2012). However, these eight features can represent
the EM context regarding the objectives of our model.
6.2.2 Instantiating the HUMAN-BEHAVIOR theme

Figure 6.4 shows an example of the HUMAN-BEHAVIOR theme instantiated using the
Theory of Behavioral Prediction.
In social psychology, the theories developed by Fishbein & Ajzen could be used
to instantiated our model. The first, called the Theory of Reasoned Action (TRA)
(FISHBEIN; AJZEN, 1975) presents a model to represent the way people behave.
According to TRA, people construct beliefs about various personal, technological and
environmental aspects, which may facilitate or complicate their attempts to adopt
certain behaviors. The TRA suggests that attitudes, norms and intentions are the main
determinants of human behavior.
TRA has been extended to predict and explain human behavior in a specific context.
This model has been renamed as Theory of Planned Behavior (TPB) (AJZEN, 1985).
As in TRA, the central entity in TPB is the individual’s intention to perform a given
behavior and another element called Perceived Behavioral Control (perception of people
about the ease/difficulty of performing the behavior) that affect people’s intentions. TPB
Figure 6.3 CONTEXT theme instantiated using Feature Model
also discusses the beliefs relevant to the behavior, which are the antecedents of attitudes
and perceptions. TPB beliefs can be of three types: behavioral, normative and control.
The two theories were consolidated and called the Theory of Behavioral Prediction
(TBP) (FISHBEIN; AJZEN, 2011), where “human social behavior follows reasonably
Figure 6.4 HUMAN-BEHAVIOR theme instantiated using TBP
and often spontaneously from the beliefs people possess about the behavior under
consideration”. Moreover, there are additional elements such as Background Factors
(emotions, past behavior, culture, knowledge) and Actual Control (skills, habits and
environmental factors) that influence human behavior.
Figure 6.4 instantiates TBP using our model. The dotted blue arrows in Figure 6.4
indicate that, although a given Background Factor may in fact influence Behavioral,
Normative, or Control Beliefs, there is no necessary connection between Background
Factors and beliefs, as well as, there is no necessary connection between Actual Control
and Perceived Behavioral Control elements.
TBP fits in our research context within EM decision-making, because it allows us to
represent the way beliefs, norms and attitudes are formed and their relationship with
behavior and practice. The red arrows shows the inputs from the other themes into
human-behavior, and the green arrows show the outputs from human-behavior into the
other themes.
In Figure 6.4, the CONTEXT theme could interact with the HUMAN-BEHAVIOR
theme by the Environment Factors element within Actual Control element that influences
the Perceived Behavioral Control (dark blue box) and the flow of Intention and Behavior
elements (dashed arrow). The CONTEXT theme could interact also with the Background
Factors element that influences the Behavioral, Normative and Control Beliefs (dotted
arrow).
Moreover, the HUMAN-BEHAVIOR theme could interact with almost all themes
(except CONTEXT theme). The Behavior element could influence all the processes of
our model as detailed in the following Subsections.
6.2.3 Instantiating the PROCESSES theme

Figure 6.5 shows an example of the PROCESSES theme instantiated using the Incident
Command System (ICS).
Figure 6.5 PROCESSES theme instantiated using ICS
We use the Incident Command System (ICS) chart (FEMA, 2014) to represent the
PROCESSES theme in our model for two reasons: 1) the key principle of ICS is its
flexibility; and, 2) Brazilian C2 Centers adopt ICS as a standard system to manage
emergencies.
Regarding the first reason, ICS consists of a standard management hierarchy
and procedures for managing incidents and emergencies of any cause, type or size
(FEMA, 2014). ICS organization may be expanded easily from a very small size, e.g.
routine operations, to a larger organization capable of managing catastrophic disasters.
The second reason emerged from the analysis of official documents, observations and
interviews (Cycle II) of our research.
The ICS concept was created in 1968 at a meeting of Fire Chiefs in the U.S.
(FIRESCOPE, 2014). The program was built primarily resembling the management
hierarchy of the U.S. Navy and it was mainly for fire fighting (SEMS, 2002). During the
1970s, ICS was fully developed during massive wildfire suppression efforts in southwest of
U.S. that followed a series of catastrophic wildfires. Property damage ran into the millions
and many people died or were injured. Studies determined that response problems are
often related to communication and management deficiencies rather than lack of resources
or failure of tactics (FEMA, 2014). It was identified that there was a lack of structure
for the operations command, as well as a difficulty in establishing common objectives
and Emergency Response Plans. ICS guidelines have proven to be excellent tools in
optimizing EM regardless of the nature of the incidents, emergencies or disaster.
With the successful use of ICS for EM, the U.S. government established the National
Incident Management System (NIMS) and the ICS as official tools for the management
of emergencies and disasters in the U.S.
In Brazil, groups of firefighters were sent to U.S. in the 1990s and 2000s to learn the
basics of ICS and to apply ICS in local emergencies. However, ICS was not popular in
Brazilian EM. It was not used as a national policy until the large events held in recent
years (2014 FIFA World Cup and 2016 World Olympics). SESGE promoted training in
ICS with Public Security professionals for these events. The events left a legacy in the
structure of EM in Brazil. Several Brazilian states have begun applying ICS, following
the model developed from the North American ICS precisely. This model is adopted by
COI in the state of Bahia.
Figure 6.5 shows how we adapted the ICS chart to represent our PROCESSES
theme. The composition of ICS defines roles and responsibilities to execute a wide
range of EM processes. The bigger rectangle represents the Command Staff element,
including the Incident Commander, Safety, Public Information and Liaison elements.
The Incident Commander element represent the command of the incident response and
is the final decision-making authority. It can be a single person commander, a unified
command or an area command (depending on the number of individuals sharing the
authority and the number of agencies or jurisdictions involved). The rest of the Command
Staff element reports directly to the Incident Commander element. Safety element
monitors safety conditions and develops measures for assuring the safety of all assigned
personnel. Public Information element serves as the conduit for information to and from
internal and external stakeholders, including the media or other organizations seeking
information directly from the incident. Liaison element serves as the primary contact for
supporting agencies attending an incident. There are rectangles under the Command Staff
element. They represent the organizational level. They have functional responsibility
for primary processes of EM. The Operations element directs all actions to meet the
incident objectives. The Finance/Admin element tracks the incident related costs,
requisitions, administrating procurement contracts and human resource management
(HR). The Planning element collects, analyzes and provides incident information, such
as situation and resource status information, for use in developing Emergency Response
Plans. The Logistics element provides all incident support needs, such as material,
medicines, foods, and vehicles.
Figure 6.5 indicates that there are interactions with the CONTEXT theme. In
fact, the EM context starts the EM processes and the process flow according to the
context conditions. Consider the Feature Model in Figure 6.3 as an example, if the
subsubfeature Fire of the subfeature Man-made of the feature Type is activated, the
agency Fireman (subfeature of the feature Agency) will certainly start processes of fire
response. In other words, the PROCESSES theme queries the CONTEXT theme that
return the configuration of the emergency at that moment. In the Figure 6.5, this kind of
relationship is represented with double green and double red arrows, i.e., all the elements
of CONTEXT bind with all the elements of PROCESSES.
Let us now consider, the HUMAN-BEHAVIOR theme (Figure 6.4) in the
PROCESSES theme. Figure 6.5 shows that there also are interactions between these
two themes. The Behavior element can influence all elements of all processes (double red
arrow). HUMAN-BEHAVIOR can also impact in HR activities in the Finance/Admin
element (dashed red arrow).
6.2.4 Instantiating the DATA MANAGEMENT theme

Figure 6.6 shows an example of the DATA MANAGEMENT theme (DAT ) instantiated
the Data Science process of SCHUTT; O’NEIL (2013).
Data management is a process in our model. Therefore, the DATA MANAGEMENT
theme is part of the PROCESSES theme, i.e. the rectangle DAT is within the ellipse
PROCESSES in Figure 6.1.
In Figure 6.6, the Emergency, represented by an exclamation mark within a red
triangle, produces huge amounts of raw data, which is collected in the Raw Data Collected
element. The Data Processing and the Clean Dataset elements process the data and clean
the processed dataset for preliminary analysis. The Exploratory Data Analysis element
focuses on the quality of the dataset. If the data is not good due to data collection
or data processing issues, the EM practitioners may have to go back to collect more
data, or spending more time cleaning the dataset. This analysis can be manual or
automated. If it is automated, this element (Exploratory Data Analysis) merges with
the next element (Models & Algorithms). However, the Models & Algorithms element
are used for more than preliminary analysis. We can use algorithms and models from
prediction and estimation for example. The usage depends on the type of problem
the practitioners are trying to solve, i.e. the reason they are collecting the data (e.g.
decision-making, sense-making, evaluate some action, among others). We consider that
the element Models & Algorithms (dark blue box) is the most important element on this
model, because it is responsible for aggregating value to the raw data, producing, in
turn, information (Information Production) for decision-making. It is also incorporates
value to the Visual Analytics theme1 . There are two other flows after the Information
1
It is not within the scope of this thesis to investigate algorithms. We believe that it is an excellent
future work opportunity to test our model with machine learning algorithms, such as Decision Trees,
K-NN, Linear Regression, Naive Bayes, Genetic Algorithms, Neural Networks among others.
Figure 6.6 DAT theme instantiated using an example of Data Science process
Production element. The information produced is transformed to visual elements and it

feeds back to the emergency, and the process may repeat. The Visualization Information
element is equal to the INF theme that we will detail in the next Subsection.
Considering the theme interactions, remember that the EM processes flow according
to the EM context conditions. Therefore, the CONTEXT theme inputs all the elements
of DAT theme and vice-versa, because data management is a process in our model.
Moreover, Figures 6.4 and 6.6 indicates that there could be interactions between
HUMAN-BEHAVIOR and DAT themes. The Behavior element could influence all
elements of data management processes especially in activities of the Exploratory Data
Analysis element. There is an input relationship between the HUMAN-BEHAVIOR
theme and Exploratory Data Analysis element and an output relationship between
Human-Behavior theme and Information Visualization element. The analysis process
involves cognition and subjectivity, thus human aspects are crucial there. Figure 6.4
shows that there is a relationship with DAT theme with the Information element (internal
box) that is within the Background factors element of HUMAN-BEHAVIOR theme.
The relationship between DAT and PROCESSES themes is trivial. As data
management is a process, it is part of PROCESSES theme, so we use the aggregation

symbol from UML (UML, 2018) to represent it.
6.2.5 Instantiating the INFORMATION VISUALIZATION theme

Figure 6.7 shows an example of the INFORMATION VISUALIZATION theme (INF )
instantiated using an adaptation of the Information Visualization process of WARE
(2012).
Figure 6.7 INF theme instantiated using an example of Information Visualization process
Much like Data Management, Information Visualization is a process in our model.

Therefore, the INF theme is part of the PROCESSES theme, i.e. the rectangle INF is
within the ellipse PROCESSES in Figure 6.1.
The research field of Information Visualization investigates interactive visual
representations of abstract data to reinforce human cognition (MAZZA, 2009). The
visual representations may help the person who analyses them (the analyst) to identify
important information or patterns within the dataset that could be difficult to spot
otherwise.
WARE (2012) states that the process of Information Visualization includes four basic
stages that have a number of feedback loops. These stages are:
1. The collection and storage of data;
2. The transformation of data into something that can be understood;
3. The hardware displays and the graphics algorithms that produce an image on the
screen;
4. The human perceptual and cognitive system.
Within the whole process, there are three feedback loops: data collection, data
exploration and view manipulation (see Figure 6.7). The data collection loops
comprehends the desire of the analyst to collect more data in order to follow a particular
lead found during his/her analysis. These new data may produce changes in the whole
visualization process. This loop also includes the physical environment, from where the
data is collected, and the social environment, which influences the data provided and how
the analysts interpret the visualizations.
View manipulation encompasses operations that change the visualization providing
new perspectives. For example, the analysts may choose to zoom in or out of he current
view, or seek more information about a specific element in the visualization.
The view manipulation could be described by the Visual Analytics Mantra (KEIM
et al., 2008): “Analyze First - Show the Important - Zoom, Filter and Analyze Further
- Details on Demand”. Visualizing the raw data is unfeasible and rarely reveals any
insight, so the raw data is first analyzed and then displayed. The analyst has an overview
of the dataset represented by the visualization. In this stage, he/she seeks information
of his/her interest using mechanisms of interactions such as zoom, filter, sort, pan, and
others. When a point of interest is found, the analysts could drill down on it and seek
more detail.
Regarding the relationship with the other themes, the interactions between INF theme
and the others themes are similar with the DAT theme. As already said, the EM processes
flow according to the EM context conditions. Therefore, the CONTEXT theme inputs all
the elements of INF theme and vice-versa, because information management is a process
in our model, as well as data management.
Moreover, Figures 6.4 and 6.7 shows that there can be interactions between
HUMAN-BEHAVIOR and INF themes. The element Behavior could influence all
elements of the Information Visualization process, especially in activities of the Visual
and Cognitive Processing - Information Analysis element. We can also define an output
relationship between the HUMAN-BEHAVIOR theme and the Visual and Cognitive
Processing - Information Analysis element. As in the DAT theme, the analysis process
involves cognition and subjectivity, thus human aspects are crucial in this process.
Figure 6.4 also shows that there is a relationship between the INF theme and the
Information element (internal box) that is within the Background factors element of
the HUMAN-BEHAVIOR theme.
The relationship between INF and PROCESSES themes is trivial. As Information
Visualization is a process, it is part of PROCESSES theme, so we use the aggregation
symbol from UML (UML, 2018) to represent it.
Figure 6.6 shows that the Information Visualization element (medium blue box) has
a portion within the Data Management theme, but has another slice that is outside this
theme. We use a brown arrow to indicate that this aggregation is partial, i.e. INF theme
is partially part of the DAT theme.
Figure 6.7 delineates a rectangle with dashed edges with 3 elements: Raw Data, Data
transformations and Graphics engine Visual mappings. Those are components of the
DAT theme. There is a double black arrow that represent ‘equals’, i.e. this arrow binds
the dashed rectangle with DAT theme. Figure 6.6 uses the same double black arrow
binding the Information Visualization element in the DAT theme with the INF theme.
6.2.6 Instantiating the DECISION-MAKING theme
Lastly, Figure 6.8 shows an example of the DECISION-MAKING theme instantiated with
an adaptation of the Decision-making process of GLAUTIER; UNDERDOWN (1991).
Figure 6.8 DECISION-MAKING theme instantiated
Much like Data Management and Information Visualization, the decision-making

is a process within our model. Therefore, the DECISION-MAKING theme is part of
the PROCESSES theme, i.e. the rectangle DECISION-MAKING is within the ellipse
PROCESSES in Figure 6.1.
GLAUTIER; UNDERDOWN (1991) explain that the decision-making process could
be seen as a logical sequence of events and analyzed in steps, as follows:
1. Recognize the existence of a problem or the need to make a decision;
2. Define all the solution alternatives to the problem based on decision policies;
3. Collect all information relevant to the solution alternatives;
4. Measure and analyze the merits of the solution alternatives;
5. Decide the best solution alternative, selecting the best classified;
6. Validate the decision by means of feedback information.
Our model attempts to follow this logical sequence. The Event element represents
any kind of problem that needs a decision2 . In the EM domain, the number of events
are huge, some common examples are: to identify the severity of the situation, to
select teams and resources, to prioritize which task will be carried out first. The
Decision Policies element depends on the nature of the decision-making situation and
the organizational culture of the decision maker. The integration of pre-established
policies and current information (Information element) presuppose the existence of viable
alternatives (Solution Alternatives element). The Information element is colored with a
darker blue because in our model it is the key factor to make assertive decisions. An
effective decision implies on the selection of one alternative and the abandonment of
others. Lastly, the decision address the origin event and can update the decision policies
for new further decisions.
It is complex to characterize the decision policies of emergency managers. One of
the factors that hampers this task is that the decision maker can have more than one
objective to maximize. Moreover, the objectives may be conflicting, e.g. quick response
versus maximum safety. In addition, another difficulty lies in the degree of uncertainty
about the future in most decision-making situations. However, the greatest difficulty in
EM decision-making is that the domain is time-critical. In EM, quick decisions can save
lives. Therefore, in EM, it is important to know and understand relevant information as
quickly as possible.
Regarding the relationships with the previous themes. The interactions between the
DECISION-MAKING theme with the CONTEXT and PROCESSES themes are very
similar to those of the DAT and INF themes. As we said for those two processes,
the EM processes flow according to the EM context conditions. Therefore, much like
Data Management and Information Visualization, the CONTEXT theme inputs all the
elements of the DECISION-MAKING theme, and vice-versa, because decision-making is
a process in our model.
2
Some authors use event as a synonym for incident. In our model, they have different meanings (See
our Glossary)
6.3 SUMMARIZING THE STUDY UP TO HERE 73
Figures 6.4 and 6.8 shows that there can be interactions between the
HUMAN-BEHAVIOR and DECISION-MAKING themes. The Behavior element could
influence all elements of the decision-making process, especially in activities of the
Measurement & Analysis element. We define an output relationship between the
HUMAN-BEHAVIOR theme and the Measurement & Analysis element. As in DAT and
INF themes, the analysis process involves cognition and subjectivity, thus the human
aspects are crucial there. Figure 6.4 shows that there can be a relationship between the
DECISION-MAKING theme and the Information element (internal box) that is within
the Background factors element of HUMAN-BEHAVIOR theme.
The relationship between DECISION-MAKING and PROCESSES themes is trivial.
As decision-making is a process, it is part of PROCESSES theme, so we used the
aggregation symbol from Unified Modeling Language (UML) (UML, 2018) to represent
it.
The relationships between DECISION-MAKING and DAT and INF themes are
similar. We defined that Data Management and Information Visualization are
subprocesses of decision-making3 . So, we use the brown arrow “Partial Aggregation”
in Figure 6.8.
Figures 6.6 and 6.7 also use the brown arrows to highlight other important
relationships with DECISION-MAKING. It gives inputs to all elements of the DAT
theme and receives outputs of the Information Visualization element. For the INF theme,
DECISION-MAKING theme gives input to all its elements and receive outputs of the
Visual and Cognitive Processing - Information Analysis element.
This last instantiation concludes our worked example of instantiation of our reference
model.
6.3 SUMMARIZING THE STUDY UP TO HERE

Our multi-method study is summarized in Figure 6.9. First we defined the scope of the
thesis (Chapter 3 - Cycle I). We started collecting research data from literature, official
documents, ethnographies, questionnaires and visual tools (open sources, academics and
proprietaries). Then, we collected research data with the interviews focusing on three key
themes which are: raw data, Information Visualization and Decision-making (Chapter 4
- Cycle II).
We selected the excerpts of the interviews where the subject mentioned one of the
key themes and carried out the Coding technique. We categorized the codes looking
for patterns (e.g. blue code β and red code γ are in the same category). We excluded
the categories that only had codes from the same subject (e.g. yellow code α). We
clustered the categories in major themes which are CONTEXT, HUMAN-BEHAVIOR,
PROCESSES, DATA MANAGEMENT (DAT ), INFORMATION VISUALIZATION
(INF ) and DECISION-MAKING. We carried out a Focus Group with practitioners to
validate our findings (Chapter 5 - Cycle III). From these, we developed our computational
3
They are not only subprocesses of decision-making process, but also are subprocesses of other EM
processes that are not represented in our model.
Figure 6.9 Summarization of the Study Up to Here
reference VA based model and produced an example of its instantiation (Chapter 6 -

Cycle IV). Next step is to evaluate the model with practitioners to test if it is useful
and effective in making decisions in EM scenarios, i.e. accept or reject our hypothesis:
“A computational reference model, based on Visual Analytics, that supports the designing
of complete, creative, usable and viable visualizations driven by the emergency managers
can improve the decision-making process in a given Emergency Management context”
(Chapter 7 - Cycle V).
This Chapter described Cycle IV of this thesis: model development. Section 6.1 explained
the elements of the model and their relationships. The main elements of the model are
the Coding themes that we found necessary for visual needs in making EM decisions.
Section 6.2 provided a full example of instantiation of our reference model. This
instantiation was not formally evaluated by practitioners, but we consider it important
to illustrate the usefulness of our reference model.
Lastly, Section 6.3 summarized the work done up to this moment.
Now that the model is developed, we have to evaluate it based on the challenges that
we found during the work: (1) how to map VA concepts with the countless variables
of an emergency, to support the managers in quickly identifying reliable information
to make decisions (i.e. the model supports the design of complete VA applications);
(2) whether researchers and practitioners are using VA techniques and tools effectively
to tackle challenge 1 (i.e. the model supports the design of creative and usable VA
applications); and, (3) how to enhance VA applications with minimal impact on current
EM activities (i.e. the model supports the design of viable VA applications). The next
Chapter explores Cycle V: the model evaluation.
Chapter
7
This Chapter presents the fifth (and final) Cycle of this thesis: model evaluation. It describes Cycle V,
its methods, the tasks we accomplished during it and the contributions of the cycle to the thesis.
CYCLE V: EVALUATION OF THE MODEL
We carried out an in-vivo exploratory study involving a reputable C2 Center and its
practitioners with the objective of evaluating the effectiveness and usefulness of the
proposed model. We used quality criteria to compare Emergency Management (EM)
visualizations designed with the support of the model against visualizations designed
without the support of the model.
We defined the criteria based on the main challenges that we expected to meet
when designing visualizations for EM. They are: (1) how to map VA concepts with
the countless variables of an emergency to support the managers in quickly identifying
reliable information to make decisions; (2) whether researchers and practitioners are using
VA techniques and tools effectively to tackle challenge 1; and, (3) how to enhance VA
applications with minimal impact on current EM activities. These challenges meant that
VA applications must be complete (challenge 1), creative and usable (challenge 2) and
viable (challenge 3). The evaluation is done from two perspectives:
1. that of the researchers, who want to evaluate the proposed model; and
2. that of the emergency managers, who want to evaluate the support provided by the
model, through the derived visualizations, in the making of decisions.
In the latter case, we could only consider manager with specific roles in the EM
process (emergency preparedness managers). In the future, we plan to consider
other roles, for example, on-field emergency responders. We conceived the exploratory
study to be easily replicable. This Chapter summarizes the main results of this study.
77
78 CYCLE V: EVALUATION OF THE MODEL
7.1 EXPLORATORY STUDY DESCRIPTION
This exploratory study was carried out in the state of Bahia, Brazil. Bahia currently
has an estimated population of around 15 million people and covers more than 567,000
km2 . The State is representative of large-sized states in Brazil, as well as states in other
countries around the world. All the subjects who participated in the exploratory study
are part of the staff of the Intelligence and Operation Center of the Public Security
Secretary of the State of Bahia (COI, in Portuguese).1
7.1.1 Subjects in the Exploratory Study
We selected two kinds of subjects: software developers who design EM visualizations and
emergency managers who evaluate these visualizations. From now on, to differentiate the
selected subjects, we will describe the first ones as designers and the others evaluators.
The designers are experienced software developers selected from one major Brazilian
organization playing an official role in COI. The daily tasks of the designers include
developing and maintaining information systems to support EM activities. Besides being
software development professionals, they are familiar with current tools for EM and
therefore we can say that they have knowledge about the EM domain.
Much like the designers, the evaluators are also experienced practitioners. They are
mainly involved in risk analysis at COI. Their duties are related to whole emergency
preparedness. One evaluator is a human resources professional specializing in research
and development (E1 ); another is a firefighter (E2 ); the third is a civil police officer
(E3 ), and the last one is a military police officer (E4 ). The selected evaluators are
experts in different aspects of an emergency, ranging from a critical infrastructure crisis
to environmental pollution and from preparedness of crowd events to pandemic control.
We characterized the participants using the form presented in Appendix F. Most of
the subjects in the study have more than 10 years of experience in their field.
7.1.2 Procedures and Materials for the Exploratory Study
The study consisted in asking the designers to construct a series of simple information
visualization prototypes with or without our model, and asking the evaluators to appraise
the built prototypes. We focused the study by giving guidelines to drive the visualizations
design and by asking the designers to build the visualization prototypes using three
datasets. The datasets consist of past data of emergency incidents from the state of
Bahia (CICOC, CIDATA and DD). Appendix E (in Portuguese) provides more details
about the datasets.
Then, we asked the evaluators to score the visualization prototypes after the
presentations of the designers, the evaluators filled in forms assessing four quality criteria
of the visualizations: completeness, creativity, usability and viability. The criteria
are described in Table 7.1. We relied on our Research Question (see 7.2) and on NICOLA;
MELCHIORI; VILLANI (2019) to select those criteria.
1
Chapter 3 provides a description of COI and its main duties.
7.2 EVALUATION QUESTIONS (EQ) AND INDICATORS (IND) 79
Table 7.1 Criteria

Criterion Description
Completeness How close to a functional and complete version is the visualization
prototype in mapping the countless variables of an emergency in quickly
identifying reliable information to make decisions.
Creativity How the visualization prototype provides novelty and innovation and
adds new concepts to the EM domain and the problem considered to
make effective decisions.
Usability How easy it is for the end-user to interact with the visualization
prototype, as well as to understand the information provided to make
effectively decisions.
Viability How the visualization prototype already has access to all the resources
necessary for its operation with minimal impact on current EM activities
and how sustainable is its evolution for upgrades.
As we shall see in the next Sections, the designers were free to build the prototypes
using whatever they wanted. They could use computing, pen and paper or other mock-up
materials. The prototypes could be presented as a sketch, as printed material, mock-up
or as a computer program simulation. They could use any tool and any programming
language to develop it.
We did two evaluations of the resulting prototypes. In the next Section, we introduce
the evaluation questions used in the exploratory study and then we detail its setting.
7.2 EVALUATION QUESTIONS (EQ) AND INDICATORS (IND)
To achieve the aim of the exploratory study, we revisited our Research Question (see 1.1)
to define the following three evaluation questions (EQ#):
• EQ1: Is the problem of making quick and assertive decisions in Emergency

Management through visualizations relevant?
• EQ2: Is the proposed model useful for visualization developers to design high quality
visualizations for decision-making in Emergency Management?
• EQ3: Is the proposed model effective for emergency managers in supporting

decision-making?
Then we used indicators (Ind#) for each of them to measure the level of their
achievement. The questions and related indicators are presented in Table 7.2.
Table 7.2 Evaluation Questions and Indicators

Indicator Description
EQ1 Is the problem of making quick and assertive decisions in
Emergency Management through visualizations relevant?
Ind1.1 Percentage of subjects (designers + evaluators) who perceived this as
relevant.
EQ2 Is the proposed model useful for visualization developers
to design high quality visualizations for decision-making in
Emergency Management?
Ind2.1 Difference in the average score of the criterion completeness between
the experimental group(s) user and control group(s).
Ind2.2 Difference in the average score of the criterion creativity between the
experimental group(s) user and control group(s).
Ind2.3 Difference in the average score of the criterion usability between the
Ind2.4 Difference in the average score of the criterion viability between the
Ind2.5 Percentage of designers who considered the model useful.
Ind2.6 Percentage of designers who thought that the Visual Analytics concepts
embedded in the model can be useful in the short term?
EQ3 Is the proposed model effective for emergency managers in
supporting decision-making?
Ind3.1 Difference in the average score of the criterion completeness between
the first and the second evaluation of the control group(s).
Ind3.2 Difference in the average score of the criterion creativity between the
first and the second evaluation of the control group(s).
Ind3.3 Difference in the average score of the criterion usability between the
Ind3.4 Difference in the average score of the criterion viability between the
Ind3.5 Percentage of evaluators who considered the model effective.
Ind3.6 Percentage of evaluators who thought that the model can improve C2
Doctrines in the long term?
Ind3.7 Percentage of subjects (designers + evaluators) who considered the
model clear and friendly.
Ind3.8 Percentage of subjects (designers + evaluators) who thought that the
application of the model causes minimal impact on current EM activities.
7.3 EXPLORATORY STUDY SETTINGS
We drew up the evaluation instrument with three groups of designers: one experimental
group using the proposed model and two control groups without the model support. So
the treatment of the study is the model. The groups were randomly made up of one
software developer who aimed to design a visualization prototype following six guidelines
(Table 7.3). We provided the guidelines mainly to control the study and mitigate some
7.3 EXPLORATORY STUDY SETTINGS 81
Table 7.3 Guidelines

Guideline Description
#1 In 24 hours, design a visualization prototype using the 3 available
datasets. This visualization must be helpful for emergency managers
in making decisions.
#2 The designers can use computing or pen and paper. Use any tool, any
programming language you want. We will not evaluate the code nor the
front-end; we will evaluate the insights addressed to guideline #1.
#3 Use the concepts of Information Visualization.
#4 Use creativity. No insight will be disapproved.
#5 No communication among the subjects of the groups.
#6 Use English for label texts (optional)
Table 7.4 Scores

Score Description
0 star Bad, a child could do the same or even better.
1 star Poor, did not meet my expectations, needs significant improvements.
2 stars Regular, did not meet my expectations, but it has potential.
3 stars Good, it met my expectations, needs little improvement.
4 stars Very Good, it met my expectations, practically ready to be used in real
cases.
5 stars Excellent, exceeded my expectations, ready to be used in real cases.
threats to validity (see Chapter 8). We highlighted that all groups had 25 minutes of
instruction to get to know the datasets and we asked them to present their prototype one
day after this training.
The experimental group was composed of one developer with 10 years’ experience. We
requested him to design a visualization prototype supported by the model following
all the guidelines and to present it to the board of evaluators. We introduced the model
to him for 30 minutes and gave him a printed version of the model. We asked him not to
share the model (see guideline #5 in Table 7.3). The first control group was composed
of one developer with 13 years’ experience and the second control group was composed
of one developer with 6 years’ experience. For these two groups we requested them to
design a visualization prototype based on the EM tools they knew following all the
guidelines and then to present their prototypes to the same board of evaluators.
We selected four evaluators with at least 15 years’ experience and we asked them
to evaluate the quality of the visualizations without whether they were built with the
support of the model. We did not share the fact that the model was the treatment of the
study with the evaluators. In particular, they gave a score from 0 to 5 for each criterion
and we calculated the average score which is the sum of the scores given by each evaluator
divided by the number of evaluators. The scores are described in Table 7.4.
The presentations were individual and remote with the Microsoft Skype Tool. One
designer did not see the presentation of another. The order of the presentations
was randomly defined. The first presentation was the first control group, the second
presentation was the experimental group and the last one was the other control group.
After the presentations and the respective evaluation, we did a second evaluation
only with the control groups. As we did with the designer of the experimental group, we
introduced the proposed model to the designers of the control groups and asked them
to imagine and conceive changes and improvements on their visualization prototypes.
In order for them not to plagiarize the ideas of the other, each designer explained the
improvements without the presence of the other. Based on the supposed changes we asked
the evaluators to review their scores, imagining the visualization with the changes. Once
they had performed the tasks, we asked all the subjects to answer a form addressing
the exploratory study. The first part of the form is to qualify the evaluator with his
experience. The second part has 7 closed questions, in which 3 questions are for all
subjects and 2 specific questions for designers and 2 specific questions for evaluators.
Each question has two divisions: one close-ended subquestion for the subject to answer
yes, no or don’t know and one open-ended subquestion for the subject to justify the
answer. The form is available in Appendix F (in Portuguese). The tasks depicted in
Figure 7.1 are the flow of our evaluation. In the next Section, we introduce the designers’
presentations.
Figure 7.1 Evaluation Flow
7.4 EXPLORATORY STUDY PRESENTATIONS
In this Section, we will summarize the designers’ presentations. We scheduled the

presentations on March 18, 2020. We planned to do face-to-face presentations, but due
to the Covid-19 pandemic we had to cancel them and do them remotely on June 02.
Again, our focus is not the data nor the tools used by the designers. Our motivation
is to gain general insights (see guideline #2 in Table 7.3). The designers had up to 10
minutes to present their visualization prototype. Although we guided them to use the
7.4 EXPLORATORY STUDY PRESENTATIONS 83
English language on the label texts of the visualization (see guideline #6 in Table 1),
all the designers used their mother language (Portuguese). We decided not to ask the
designers to change the labels nor did we change them in order to keep their prototypes
free from tampering.
After each presentation we opened the floor to the board of evaluators to ask
questions and to make comments. Then we asked the evaluators to score the presented
prototype.
7.4.1 First Evaluation

As we said, we did two evaluations. The first one is comparing the visualization
prototype(s) designed with the support of the model against the visualization prototypes
designed without it. The second one is comparing the insights for improvements that
designers would have made if they had known the model. Below we summarize the
designers’ presentations to the board of evaluators.
7.4.1.1 Presentation 1: First control group The first control group was
composed of one software developer with 13 years’ experience. He presented for 6 minutes.
Figure 7.2 shows a screenshot of his prototype. He used a web page to represent his
visualization and he used C# as the programming language. We shall see that all the
designers used the same technology in their prototypes, which is not surprising since they
work with it routinely. From the datasets, he basically used bar charts. He just extracted
datetime information.
He described his prototype as a simple visualization for emergency managers to make
decisions. He decided to divide the data into a matrix where the rows are the dataset
(the 1st row is CICOC, the 2nd is CIDATA and the 3rd is DD) and the columns are the
datetime division (months, weeks and time ranges respectively). The evaluators agreed
on its simplicity and did not make any relevant comments.
7.4.1.2 Presentation 2: Experimental group The experimental group was

composed of one software developer with 10 years’ experience. He presented for 10
minutes; we had to interrupt his presentation because of the defined maximum time.
Figures 7.3 to 7.6 show screenshots of his prototype. He also used a web page to represent
his visualization and he used C# as the programming language. He used the dashboard
approach with a mix of maps, line chart, bar chart, doughnut chart and pie chart. From
the datasets, he extracted geolocation information, date information (years of the events)
and information about the type of the event.
He described his prototype as a dynamic visualization for emergency managers to
make decisions. The dynamics of the visualization depend on the characteristics of
the viewer. He said that he based it on three themes of the model to categorize the
viewers. The themes that he based it on were PROCESSES, HUMAN-BEHAVIOR and
Figure 7.2 Screenshot of the visualization prototype of the first control group (1/1)
CONTEXT.
Based on the PROCESSES theme, the designer said that the visualization updates
automatically according to the viewer’s access permission to the datasets. One can see
these changes in Figures 7.3 and 7.4. In Figure 7.3, the viewer has full access permission
to the three datasets. In Figure 7.4, the viewer has access only to the dataset of CICOC;
so the line of DD is not visible in the line chart and the pie chart of CIDATA is disabled.
Based on the HUMAN-BEHAVIOR theme, the designer said that the visualization
updates automatically according to the visual deficiencies of the viewer. Charts can differ
in colors or forms. In Figure 7.5, the CICOC line is darker than DD line; and in the other
charts, the bars and the slices differ in color. This setting is used for normal viewers,
ones with no visual deficiencies. However, for color-blind viewers, he used patterns to fill
lines, bars and slices of the charts. In Figure 7.5, the CICOC line is dashed in the line
chart; and in the other charts, the bars and the slices are filled with patterns (squares,
circles, diamonds, etc).
Based on the CONTEXT theme, the designer said that the visualization updates
automatically according to the device that the viewer is using. The spatial position of
the charts differs according to the orientation of the screen. In Figure 7.5, and also
Figures 7.3 and 7.4, the viewer is using a landscape orientation (in general desktops or
notebooks use this position). The charts are positioned side by side. However, in Figure
7.6a and 7.6b, the charts are positioned one below the other2 . He said that this spatial
position is automatically changed if the viewer is using a smartphone or tablet in portrait
orientation.
The evaluators were very impressed with the presentation and praised the designer.
One relevant comment made was that the visual deficiencies of the viewer is an issue
of context and not an issue of behavior. Everybody agreed with that, including the
designer. Another evaluator added that the dataset compartmentalization is also an
issue of context, but everybody agreed that it can be an issue of processes too. The
2
Figure 7.6b is the continuation of Figure 7.6a
evaluators scored and obviously as he already had the support of the model he did not
need to do the second evaluation.
Figure 7.3 Screenshot of the visualization prototype of the experimental group for the full
access, normal and landscape orientation viewer (1/1)
Figure 7.4 Screenshot of the visualization prototype of the experimental group for the CICOC,
normal and landscape orientation viewer (1/1)
7.4.1.3 Presentation 3: Second control group Finally, the second control group
was composed of one software developer with 6 years’ experience. He presented for 8
minutes. Figures 7.7 to 7.10 show screenshots of his prototype3 . Like the previous
designers, he used a web page to represent his visualization and he used C# as the
programming language. He used the dashboard approach with a mix of filters, grids,
maps, bar charts and 3D pie charts. From the datasets, he extracted geolocation
information, data of users who had registered events and datetime information.
3
Figures 7.7, 7.8 and 7.9 are continuous, i.e. Figure 7.9 is the continuation of Figure 7.8 that, in turn,
is the continuation of Figure 7.7
access, color-blind and landscape orientation viewer (1/1)
access, normal and portrait orientation viewer a) 1/2 and b) 2/2
He described his prototype as a visualization for emergency managers to make

decisions. Similar to the first prototype, he divided the visualization into a matrix where
the rows are the information (users, months, weeks, time ranges and shifts) and the
columns are datasets, except for the first line that had a filter, a grid and a map. He
said that the viewer could filter the visualizations using keywords and/or a period of
time (start date and end date) and/or by datasets. In Figure 7.10, a filter by datasets is
applied; the CICOC dataset is disabled and one can see that the grid of CICOC is not
visible, as well as the other CICOC charts in the first column.
Despite this prototype having more information and some interaction, the evaluators
noticed a similarity between this and the first prototype. The designer said that he
respected the guideline #5 and did not communicate with other designers (see Table
7.3). Everybody concluded that, as they work routinely together in COI and use the
same technologies, this was just a coincidence4 .
Figure 7.7 Screenshot of the visualization prototype of the second control group of the full
access viewer (1/3)
access viewer (2/3)
7.4.2 Second Evaluation

As we mentioned in the previous Section, in the case of the control groups we did a second
evaluation. We introduced the proposed model to the designers for about 30 minutes and
4
If plagiarism or communication between designers was proven, we would have to eliminate the two
prototypes in the study.
access viewer (3/3)
Figure 7.10 Screenshot of the visualization prototype of the second control group of the no
CICOC viewer (1/1)
asked them to conceive changes in the visualization prototypes they presented based on
the model. Again, we opened the floor to the board of evaluators and finally asked them
to review their scores.
7.4.2.1 First control group: improvement insights The designer alleged that
he did not care about context, human-behavior and other processes. He said that if he
cared, mainly about the context, he would apply more filters showing only the relevant
data for specific contexts. He did not give examples of specific contexts. He added that
he would build more charts and the charts would be integrated, if a filter is applied to a
chart, the data of the others should also be filtered.
The evaluators asked him about accessibility and used the example of color-blind
7.5 EXPLORATORY STUDY RESULTS 89
viewers. The designer answered that accessibility would be a good improvement to be

done. So we asked the evaluators to review their scores based on the improvements that
the designer had suggested; if the scores increased, we asked why.
Evaluator E1 stated:
“I gave low scores because his prototype is too simple. Any improvement in it, the
score would increase.”.
And E3 added:
“After you introduced the model to him, he realized that many functionalities could
be inserted to help us in making decisions with his prototype, showing that your model is
very useful for the developers”.
7.4.2.2 Second control group: improvement insights The designer said he

would simulate cases according to the six themes of the model. He did not explain
how he would do this. He added that he would mine the unstructured data mapping
geolocation data with datetime data.
The evaluators did not make relevant comments. As we did in the previous evaluation.
We asked the evaluators to review their scores based on the improvements that the
designer talked about; if the scores increased, we asked why.
Evaluator E1 stated:
“Like the other prototype, your model has proven to be a guideline to themes that the
visualizations must have.”.
And Evaluator E2 added:
“The improvements he said he would do would make his prototype similar to the
other one that used the model from the beginning. I would like to have seen his prototype
based on your model ”.
The improvements of the designers as well as the comments of the evaluators showed
us that the model is useful and effective. Next Section details the results of the evaluation.
7.5 EXPLORATORY STUDY RESULTS

The results of the two evaluations are shown in Figures 7.11 and 7.12 and in Table 7.5.
In Figure 7.11, the height of the bars shows the sum of the scores given by each
evaluator for each visualization. The colors associated with the bars identify the groups
in the order of presentation (blue is the first control group, orange is the experimental
group and gray is the second control group). One can observe that in total 16 ‘triple’
bars (4 bars to each 4 evaluators). E1 gave 3 higher scores to the experimental group
Figure 7.11 First Evaluation: (a) degree of completeness (Ind2.1), (b) degree of
creativity (Ind2.2), (c) degree of usability (Ind2.3) and (d) degree of viability (Ind2.4)
and 1 lower (second control group in viability criterion). E2 gave 2 higher scores to the
experimental group and 2 equals (second control group in usability and viability criteria).
E3 gave 3 higher scores to the experimental group and 1 equal (both control groups in
viability criterion). E4 gave 3 higher scores to the experimental group and 1 equal (both
control groups in viability criterion).
Figure 7.12 shows the data of the second evaluation. The colors associated with the
bars identify the groups (blue is the first control group and gray is the second control
group) and the darkness associated with the bars identifies the evaluations (light is the
first evaluation and dark is the second evaluation). One can observe that in total 32 pair
bars (8 pair bars to each 4 evaluators). E1 gave 7 higher scores to the second evaluation
of the control groups and 1 equal (first control group in viability criterion). E2 gave 6
higher scores to the second evaluation of the control groups and 2 equal (both control
group in viability criterion). E3 gave 6 higher scores to the second evaluation of the
control groups and 2 equal (both control groups in viability criterion). and E4 gave 4
higher scores to the second evaluation of the control groups and 4 equal (second control
group in completeness criterion, second control group in creativity criterion, both control
groups in viability criterion).
Table 7.5 shows all results in a matrix. The rows are the score of the evaluators and
the columns are the quality criteria. In each cell, there are two enumerations separated
with hyphens. The top enumeration is the score of the first evaluation of each group in
the order of presentation; so the initial number is the score of the first control group, the
middle number is the score of the experimental group and the final number is the score of
the second control group. We highlighted the background colors of some scores: ‘yellow’
means that the experimental group scored equal to both control groups; ‘pink’ means
7.5 EXPLORATORY STUDY RESULTS 91
Figure 7.12 Second Evaluation: (a) degree of completeness (Ind3.1), (b) degree of
creativity (Ind3.2), (c) degree of usability (Ind3.3) and (d) degree of viability (Ind3.4)
that the experimental group scored equal to one control group and ‘red’ means that the
experimental group scored lower than one control group. Comparing each control group
scores against the experimental group: 24 of 32 scores were higher for the experimental
group which is 75%, 7 scores were equal and only 1 was lower.
The lower enumeration is the score of the second evaluation of the control groups;
so the initial number is the score of the first control group, the ‘X’ represents the
experimental group which did not have the second evaluation and the final number is
the score of the second control group. The bold numbers represent the increased scores.
Comparing the control groups’ scores of the first evaluation with no support of the model
against the second evaluation: 23 of 32 scores increased which is 72%, 9 scores were equal
and none was lower. Finally, in the last row the top enumeration is the average score
of the first evaluation, to measure Ind2.1 to Ind2.4; and the lower enumeration is the
average score of the second evaluation, to measure Ind3.1 to Ind3.4. Adding the results
of both evaluations: with the support of the model, 47 of 64 scores in total were higher,
which is 73.4%, 16 scores were equal, which is 25% and only 1 was lower, which is 1.6%.
The results from the study are summarized in Table 7.6 which provides percentage values
for each indicator described in Table 7.2.
The result for EQ1 (Ind1.1) was obtained through a form filled in by the subjects
(designers and evaluators). All subjects answered ‘yes’ and this confirms that the problem
we are addressing in this work is relevant.
Table 7.5 Results Criteria X EVALUATOR X Group

Completeness Creativity Usability Viability
Evaluator Ind2.1 Ind2.2 Ind2.3 Ind2.4
Ind3.1 Ind3.2 Ind3.3 Ind3.4
1 – 4 – 3 2 – 4 – 3 2 – 4 – 3 4 – 3 – 3
E1
2 –X– 4 3 –X– 4 4 –X– 4 4 –X– 4
1 – 5 – 3 3 – 5 – 4 3 – 4 – 4 4 – 5 – 5
E2
3 –X– 5 4 –X– 5 4 –X– 5 4 –X– 5
2 – 5 – 3 2 – 4 – 3 3 – 5 – 4 5 – 5 – 5
E3
3 –X– 4 3 –X– 4 4 –X– 5 5 –X– 5
2 – 5 – 4 2 – 5 – 4 3 – 5 – 4 5 – 5 – 5
E4
3 –X– 4 3 –X– 4 4 –X– 5 5 –X– 5
1,5 – 4,75 – 3,25 2,25 – 4,5 – 3,5 2,75 – 4,5 – 3,75 4,5 – 4,5 – 4,5
Average
2,75 - X - 4,25 3,25 - X - 4,25 4 - X - 4,75 4,5 - X - 4,75
Table 7.6 Results of the Indicators

Evaluation
Indicator Results
Question
EQ1 Ind1.1 100%
Ind2.1 4.75 2.375 2.375 100%
Ind2.2 4.5 2.875 1.625 57%
Ind2.3 4.5 3.25 1.25 38%
EQ2
Ind2.4 4.5 4.5 0 0%
Ind2.5 100%
Ind2.6 100%
Ind3.1 3.5 2.375 1.125 47%
Ind3.2 3.75 2.875 0.875 30%
Ind3.3 4.25 3.25 1.0 31%
EQ3
Ind3.4 4.625 4.5 0.125 3%
Ind3.5 100%
Ind3.6 75%
Ind3.7 86%
Ind3.8 71%
The values related to EQ2 (Ind2.1 to Ind2.4) were obtained from the scores on the
form filled in by the evaluators after the first evaluation. They scored the quality of
the produced visualizations according to the criteria of: completeness, creativity,
usability and viability. Figure 7.11 shows these scores. Specifically, about Ind2.1 to
Ind2.4, the positive difference achieved for all criteria (except viability criterion whose
difference is null) demonstrates the usefulness of our modeling approach, an encouraging
result. Ind2.1 to Ind2.4 are described as an enumeration in the column of the ‘Results’.
The first subcolumn is the average score of the experimental group. The second
subcolumn is the average score of the control groups. The third subcolumn is the value
of the indicators Ind2.1 to Ind2.4, i.e., the difference between the average scores of
the experimental group and the control groups. Finally, the percentage increase is in the
fourth subcolumn. For instance, for the criterion completeness the results show us that
the scores with the support of the model (with it being used to design the visualization
prototype) were 100% higher on average than without it. The next Chapter discusses
these results in greater depth.
Other results for EQ2 (Ind2.5 and Ind2.6) were obtained from the same form applied
to the designers. All answered ‘yes’ and this confirms the usefulness of the proposed
model in conceiving high quality EM visualizations for decision-making.
The values of EQ3 (Ind3.1 to Ind3.4) were also obtained from the scores on the form
filled in by the evaluators after the second evaluation with the same criteria. Figure
7.12 shows these scores. Ind3.1 to Ind3.4 are shown similar to Ind2.1 to Ind2.4 in Table
7.6. The first subcolumn is the average score of the second evaluation of the control
groups. The second subcolumn is the average score of the first evaluation of the control
groups. The third subcolumn is the value of the indicators Ind3.1 to Ind3.4, i.e., the
difference between the average scores of the first and second evaluation of the control
groups. Finally, the percentage increase is in the fourth subcolumn. For instance, for the
criterion creativity the results show us that the scores with the support of the model (in
this case with it being used to improve the visualization prototype with insights) were
30% higher on average than without it.
Other results for EQ3 (Ind3.5 and Ind3.6) were obtained from the same form filled in
by the evaluators. All answered ‘yes’ to Ind3.5 and 75% to Ind3.6: one Evaluator (25%)
answered ‘no’ and he explained that the C2 doctrines in Brazil are imported, mainly from
the USA, and adapted to the regional context. He added that Brazilian C2 Centers can
adapt the model in their activities but he disagreed that it could be incorporated in
countries which are creators of C2 doctrines. The fact that he said that the model can
be adapted for any context for us is a positive comment even though he answered ‘no’.
And finally other results for EQ3 (Ind3.7 and Ind3.8) were obtained from the same
form filled in by the subjects (designers and evaluators). For Ind3.7, 86% of the subjects
answered ‘yes’ and one evaluator (14%) answered ‘don’t know’. He justified that the
model friendliness depends on the target public of the organization. He said that even
though he had seen the presentation of the model there were still doubts. For Ind3.8,
71% of the subjects answered ‘yes’ and two evaluators (29%) answered ‘don’t know’.
Their answers were very similar, they said that it is very complex for a 24/7 environment
to make changes. They added that change management has to be well planned to avoid
unwanted impacts. For us, the answers were positive so they confirm the effectiveness of
the proposed model in supporting decision-making in EM.
This Chapter described Cycle V of this thesis: model evaluation. In Section 7.1, we
described the evaluation. We decided to carry out an exploratory study with two kinds
of subjects: designers and evaluators. Both kinds of subjects are domain practitioners
of a representative C2 Center: designers are software developers and evaluators are
emergency managers at this Center. We also described the quality criteria that we used
in the evaluation in this Section (completeness, creativity, usability and viability).
The main concept of the study is: some designers used the model to design a visualization
prototype and other designers did not use it, then the evaluators scored the visualizations
based on the four criteria.
Moreover, Section 7.2 showed the evaluation questions and its indicators and Section
7.3 presented the exploratory study settings (guidelines, scores, the group division and
the evaluation flow).
In Section 7.4, we introduced the presentations of the groups and the qualitative
discussion of the subjects. Finally, in Section 7.5, we presented the results.
The next Chapter discusses the results and the threats to the validity of the evaluation
of our model.
Chapter
8
This Chapter analyzes the results of the evaluation in practice. We discuss each of the four criteria that
we used to evaluate our Model: completeness, creativity, usability and viability. It also discusses validity
threats to the work.
DISCUSSION
We provide a discussion of the exploratory study results, based also on direct observation
of the subjects’ behavior during the study and on subsequent discussions we had with
them. The problem of making decisions in Emergency Management (EM) through
visualizations is considered relevant (see EQ1) as it allows emergency managers to
think out-of-the-box about unexpected situations. Subjects reported they were not
aware of similar models to support both the Visual Analytics design activity and the
decision-making in emergency scenarios, which are considered complex, multidisciplinary
and hard to predict.
Therefore, we have used empirical techniques based on the qualitative research
approach to greatly enrich this work. The partnership between practitioners and us
facilitated contact with them and helped in the execution of the study. We carried
out document appraisals, semi-structured interviews, Focus Groups, transcriptions,
translations, coding, categorizations, evaluations and many other tasks in this work which
were not automated. This gave added qualitative value to the results and reduced the
possibility of errors.
Regarding the overall Survey Protocol tasks: the transcriptions of the interviews were
time consuming. We spent on average nine hours transcribing every 1 hour of audio
recorded. We had to repeat some audio excerpts many times pausing, going back, going
forward and so on. We spent on average one hour per page translating from English to
Portuguese. Our Coding tasks required that all the interviewee transcriptions were in
the same language. We decided to translate the interviews from English to Portuguese,
and not the opposite, for two reasons. First, to save time because six interviews (seven
with the pilot) were in Portuguese and only one in English. Second, we did not have
authorization to publish all the data. If we had had authorization to publish, this
would have justified the effort of doing several translations into English instead one to
Portuguese for obvious reasons as English is a universal language in academia.
95
96 DISCUSSION
A fact that should be highlighted is that, none of the practitioners that participated
in our research, from Cycle I, knew the term Visual Analytics, which was to be expected
because it is a new term and infrequent in the industry. The first papers about Visual
Analytics are no more than fifteen years old. Due to this, in our research we could not go
deeper in Visual Analytics concepts and techniques. However, in our exploratory study
to evaluate the model we used visualizations and we proposed using our model in Visual
Analytics applications in future works (see next Chapter). As we saw in the previous
Chapter, the scores given by the emergency managers for the visualizations that were
designed with the support of the model had 73.4% higher scores, 25% equal and only
1.6% lower for the 4 mentioned criteria against the visualizations that were designed
without it.
The usefulness of the model (see EQ2) was confirmed by the three developers.
Furthermore, we observed some initial skepticism that disappeared after a few minutes
of understanding the model. Overall, it should be noted that the quality criteria of the
visualizations generated with the support of the model were superior to the visualizations
without this support (except viability which were equal in first evaluation). This
confirms that our model is engaging.
Concerning the effectiveness of the model (see EQ3), the results of the first evaluation
and mainly the second evaluation confirmed that the model is effective in supporting
decision-making in EM. We discuss our conclusions concerning each quality criterion
below.
8.1 COMPLETENESS
The experimental group obtained the best results in the completeness criterion (Ind2.1).
The difference between the average score of this group and the average score of the control
groups is greater than for the other criteria. The average score of the experimental group
is 4.75 and the average score of the control groups is 2.375, so the difference is 2.375, i.e.,
100% higher. All the evaluators scored the experimental group higher. In particular, the
greatest difference among all scores in this criterion was given by evaluator E2 (he gave
a score of ‘1’ to the first control group and ‘5’ to the experimental group). In the second
evaluation (Ind3.1), all evaluators increased their scores, except for one evaluator (E4 )
who gave a score of ‘4’ and repeated ‘4’ for the second control group.
These results show that the proposed model is tackling our Research Question making
the visualizations more complete. Remembering our Research Question “What themes
of Emergency Management are under-exploited in decision-making and how can Visual
Analytics support them?”. We are able to conclude that our model is complete in
addressing the themes that we found that are under explored. In addition, we mentioned
three big challenges in applying Visual Analytics in Emergency Management. The first
challenge “(1) how to map VA concepts with the countless variables of an emergency
to support the managers in quickly identifying reliable information to make decisions” is
related to the completeness of the visualizations. Again, our results for the completeness
criterion show that our model can tackle this challenge.
8.2 CREATIVITY 97
8.2 CREATIVITY
The experimental group obtained the second best results in the creativity criterion
(Ind2.2). The average score of the experimental group is 4.5 and the average score of
the control groups is 2.875, so the difference is 1.625, i.e., 57% higher. Similarly to
completeness, all the evaluators gave higher scores to the experimental group. In the
second evaluation (Ind3.2), all evaluators increased their scores, except for one evaluator
(E4 ), again, who gave a score ‘4’ and repeated ‘4’ for the second control group.
We concluded that our model allows the creativity of the designers’ visualizations
to emerge. We believe that creativity is a catalyst to tackle information overload and
non-dedicated information which are the problems we are facing. Creativity can help to
bring more intelligence into the analytical process by “learning” from users’ behavior and
effective use of the visualization (KEIM et al., 2008).
8.3 USABILITY
For the usability criterion (Ind2.3), the average score of the experimental group is 4.5 and
the average score of the control groups is 3.25, so the difference is 1.25, i.e., 38% higher.
All evaluators gave higher scores for the experimental group, except for one evaluator
(E2 ) who gave the same ‘4’ score to the second control group. In the second evaluation
(Ind3.3), all the evaluators increased their scores.
Similarly to the creativity criterion, we do believe that usability is also a catalyst
to tackle the problems we raised at the beginning of this thesis. For us, raising the
usability of the visualizations will allow the emergency managers to detect the expected
and discover the unexpected (KEIM et al., 2008). Our results in the creativity and
usability criteria show that our model can tackle the second challenge “(2) whether
researchers and practitioners are using VA techniques and tools effectively”.
8.4 VIABILITY
A premise reported by several emergency managers is that there is no use having a

technique or a tool which, though effective and solves a couple of problems, generates
other problems. New technologies in this area have to be viable. This premise is expressed
by our third challenge “(3) how to enhance VA applications with minimal impact on
current EM activities”. For the criterion viability (Ind2.4), the average score of the
experimental group is 4.5 and the average score of the control groups is 4.5, so the
difference is 0. For all the criteria, all the evaluators gave higher (24 times), or equal (7
times) scores for the experimental group, with the exception of evaluator (E1 ) who gave
a lower score for the viability criterion. That was the only case in 32 score comparisons.
In the second evaluation (Ind3.4), 5 of 8 scores repeated the maximum score ‘5’ given in
the first evaluation. The other 3 scores: 2 kept a ‘4’ and 1 increased from ‘3’ to ‘4’, i.e.,
one evaluator, (E1 ), increased the score of the second control group.
The viability criterion clearly differed from the other three criteria. Our first reaction
was that the evaluators misunderstood our definition of viability (see definitions in Table
7.1). Our guess was that they presumed that simpler approaches were more viable than
98 DISCUSSION
more complete approaches. We discussed this during the presentations. Viability means
that “the visualization prototype already has access to all the resources necessary for
its workings”, but the evaluators confirmed their scores and they had the final word.
Interesting future work would be to replicate this study with other evaluators, making
the definition of viability clearer from the beginning and seeing if the averages of the
groups’ scores remain the same.
8.5 THREATS TO VALIDITY
A fundamental concept to reduce biased evaluations is mitigate Threats to Validity. This

Section discusses how valid the results of our study are (WOHLIN et al., 2012). There
are four types of threats to validity: conclusion, constructions, internal and external.
Threats to construct validity deal with limitations in the study set up. For example,
construct validity could be limited by the general problem of lack of availability of
experienced emergency managers because they are usually involved in front-line activities.
Conclusion validity focuses on issues affecting the ability to draw the correct
conclusion regarding relationships between the treatment and the results of a study. For
example, the conclusion validity could be limited by the behavior of the evaluators who
may score the visualizations with the model higher, simply because they believe that this
will help our work.
Internal validity threats are concerned with the randomness of the results, with the
performances of the hardware/software used and the selection of the subjects. For
example, the internal validity could be limited by the interaction between the designers.
Finally, external validity threats are concerned with the generalizability of the results.
External validity could be limited by geographical context dependence. For instance, the
approach could be valid only if scenarios address zones with similar characteristics of
COI.
The threats to validity of our study and how they were mitigated are described below.
We identify the threats to the validity of our work in two parts. The first are the threats
to validity of the model development and the second is the validity of the evaluation of
the developed model.
8.5.1 Threats to Validity of the model development: Cycles II, III and IV
Emergency Management is complex and dynamic therefore research data collection in this
area is also complex. Concerning threats to construct validity, a poor survey definition
based on a poor theoretical basis could introduce significant threats to the validity of
our results. The data collected could be incomplete or inconsistent. We mitigated this
threat by collecting data from different research sources and using different approaches;
namely literature, documents, software, observations, questionnaires and interviews with
practitioners. We also mitigate this type of threat when we run a pilot interview to
evaluate our Survey Protocol.
Regarding conclusion validity, we compared, crosschecked and triangulated all the
8.5 THREATS TO VALIDITY 99
collected data. Concerning the results of the research execution, the collected data were
coded based on our own judgment. Due to resource and time issues only one researcher
carried out the Coding analysis, while the ideal procedure would be to have at least two
researchers carrying this out independently, and in case of disagreement call on a third
opinion to reach a consensus. We might have classified themes incorrectly or in a biased
manner. In order to mitigate this threat, we interacted repeatedly with the subjects
by email, informal conversations and carried out Focus Groups to validate our work.
Moreover, the subjects could also be biased. Hence, we only included a Coding category
if two or more subjects mentioned it to eliminate any individual and biased opinion of a
unique subject. Concerning the results of the model validation, due to logistical issues,
all the practitioners of the Focus Groups came from the same institution (COI). We shall
see in the next Chapter that we intend to carry out this study at other C2 Centers. We
invited specialists from other C2 Centers and we planed to hold a web-conference, but
we gave up on this idea due to the complexity involved at that moment of pandemics.
We mitigated this threat by sending an email with the session transcription to other EM
specialists in the world to share with them and to get feedback.
As threats to internal validity, the researchers responsible for data collection and
classification came from the same organizations. This may have led to bias in the
interpretation of the concepts analyzed. We could not mitigate this threat.
As threats to external validity, we believe that the model is representative to EM
worldwide. In other words, we propose that any emergency manager, in any city around
the world, can use our model to support decision-making. We attempted to mitigate this
threat by collecting data from more than one C2 Center. In addition, the qualifications
and work experience of the practitioners minimized the impact of this threat.
8.5.2 Threats to Validity of the model evaluation: Cycle V
As well as the threats to construct validity of the model development, a poor survey
definition based on a poor theoretical basis could impose significant threats to the
validity of our model evaluation. In Cycle V, we could not do a pilot study to mitigate
this. However, we defined strict guidelines based on other studies in the literature, e.g.
(NICOLA; MELCHIORI; VILLANI, 2019), and we trust that the subjects did their best
in their evaluation tasks.
Regarding threats to conclusion validity, we observed considerable interest in the
proposed model as it addresses a real problem. We involved subjects from an important
public organization working in the field of EM (COI) and closely followed the evaluation
protocol. However the small number of subjects used in the study remains a threat to
our conclusions. As we said, we intend to replicate this study with other subjects in other
C2 Centers to confirm our conclusions.
With regard to threats to internal validity, we involved three experienced software
developers from one C2 organization (COI) to design the visualizations prototypes, all
knowledgeable in EM. We could have used graduate or postgraduate students to design
the visualizations or even used software industry professionals. However, we found that
100 DISCUSSION
students and industry professionals are not involved in EM in the same way as the software
developers of COI. The allocation of designers in the treatments (with or without the
support of the model) is also a threat. To mitigate this, we evenly divided the subjects
among the treatments in order to make the allocation as fair as possible based on the
experience in concepts and techniques detected in the exploratory study. This threat can
be increased also by the subjectivity of the evaluators. Hence, to improve objectivity, we
involved four different emergency preparedness managers from COI with more than
15 years’ experience to evaluate the visualizations. At that moment, the four evaluators
worked for the same organization, however, they were trained in different agencies, with
different organizational cultures. As a consequence, they have different profiles, duties
and skills. This heterogeneity among the evaluators contributes to mitigating this threat.
Moreover, like the designers, all the evaluators have a deep understanding of EM.
As threats to external validity, we considered the state of Bahia and COI
representatives of different geographical contexts. Bahia is a large state of Brazil and
COI is a well-known C2 Center. The datasets that we used in the exploratory study are
real incidents that have happened in recent years in Bahia. The tacit knowledge and
experience of the subjects also helps to mitigate this threat.

This Chapter discussed the results of our work. Section 8.1 to Section 8.4 discussed
the four evaluation criterion chosen to evaluate our model: (completeness, creativity,
usability and viability).
Section 8.5 addressed the threats to the validity of this work. We divided the analysis
in two. The discussion of the threats to the validity of the model development and the
discussion of the threats to validity of the evaluation of the model.
The next Chapter provides the final remarks of this thesis and pointing out research
directions for future work.
Chapter
9
This Chapter summarizes the final considerations of this thesis, outlines the main contributions that were
achieved, and discusses the work limitations and possible future research in the area.
FINAL CONSIDERATIONS
This research carried out a long-term multi-method in-vivo study to understand the
dynamics of Emergency Management (EM) activities in depth. We investigated the
major themes which managers need to visualize, for complete and fast analysis for
decision-making in EM. This work resulted in a novel Visual Analytics based reference
model for EM. The study was divided into five big cycles: I) the definition of the research
scope; II) the research data collection; III) the analysis of the researched data; IV) the
development of the model itself; and V) its evaluation.
Our research transformed the collected data into themes of visual needs for EM
decision-making. It was possible to identify the relationships between these visual needs
and the major themes that influence EM. We found that practitioners and researchers
are neglecting important themes that need to be represented in visualizations tools
for decision-making (e.g. information about emergency context, processes and human
behavior). In our research, we identified that the concepts of Visual Analytics (VA) can
tackle these themes.
We believe that computational reference models can be used to design visualizations
for decision-making in Em. This supports the emergency managers in constructing
emergency plans, in sharing information among practitioners and as an intermediate
step towards a computable format for simulations (training). Designing emergency
visualizations is not an easy task. Designers should take into account a set of themes that
needs to be represented in the visualization for emergency managers to make assertive
decisions quickly.
Our work is novel because the model provides the major themes needed for
visualizations that we identified during this long-term research. We were unable to find
a similar approach used in the literature with the themes that we integrated.
Our work is also grounded in the real needs of emergency managers. We carried out
studies with practitioners to collect and analyze research data and to evaluate the model
in real world environments. We liaised with many EM professionals. With whom, we had
101
102 FINAL CONSIDERATIONS
Table 9.1 Practitioners Meetings

Number of
Cycle Method Meetings C2 Centers
Practitioners
Presential Hundreds
CICCR-BA (2014)
Ethnographies (minimum (participant
COI (2016)
II contact) observation)
a
Questionnaires Remote 49 Several
a
COI
7
CEI
Interviews Presential (1 pilot and
CICCR-RJ
6 interviews)
DZPPP
Focus
III Presential 3 COI
Groups
7
Exploratory
V Remote (3 designers and COI
Study
4 evaluators)
Centers above and
CICCN-DF
Presential and Hundreds CICCR-CE
All Informal
Remote (observation) MEUV
COFIC
CEMADEN
several formal and informal meetings, in person and remotely. We detail these meetings
in Table 9.1.
Our exploratory study showed the validity of this novel model for designing EM
visualizations to improve preparedness and promptness in managing such situations.
The study involved 7 subjects: 3 software developers (designers) and 4 emergency
managers (evaluators) from a reputable Command and Control Center, the Intelligence
and Operation Center of the State of Bahia, in Brazil (COI). They gave positive feedback
on the proposed approach and the model application. We evaluated visualizations derived
by the model using four quality criteria: completeness, creativity, usability and viability.
Comparing the scores given by emergency managers, the visualizations designed with the
support of the model received 73.4% higher scores, 25% equal scores and only 1.6% lower
scores for the 4 mentioned criteria. In particular, they recognized the usefulness and
the effectiveness of the model for decision-making through the Information Visualization
process.
9.1 THESIS CONTRIBUTIONS REVISITED

Revisiting our Research Question “What themes of Emergency Management are
under-exploited in decision-making and how can Visual Analytics support them? ” and
9.1 THESIS CONTRIBUTIONS REVISITED 103
our Hypothesis “A computational reference model, based on Visual Analytics, that

supports the designing of complete, creative, usable and viable visualizations driven by
the emergency managers can improve the decision-making process in a given Emergency
Management context” we accepted our hypothesis with our results.
We believe that the area of this thesis is fresh and relevant both to academics
and to practitioners. We are investigating the decision-making process in the
state-of-the-practice and we found a lack of the use of visualizations in this process.
We found that the visualizations are not effective because the decision makers do not use
them, on the other hand, the decision makers do not use the visualizations because they
think that the visualizations are not effective. Given this vicious circle, we developed a
computational reference model to tackle the problems specified in this text. For us the
main contribution of our thesis is the computational reference model itself. However, we
have made other contributions.
An important contribution of this thesis is the novel approach we developed to be a
reference for handling visual interfaces driven by our model. By ‘to handle’ we
mean ‘to know, to understand, to design, to update, to simulate and to evaluate’. Our
aim is to improve existing visualizations in the state-of-the-practice and to be a reference
in the development of new ones. We expect that this model can address the issues
reported above that cause problems in decision-making in EM. We built the model based
on research practices and tools from the literature and from a C2 exploratory in-vivo
study.
We carried out a formal qualitative analysis of our research data using the
Coding technique and refined the results with a Focus Group. The result is a
complete classification that can be used by other researchers and practitioners as a
taxonomy of relevant themes needed in EM visualizations. The themes we
found are: CONTEXT, HUMAN-BEHAVIOR, PROCESS, DATA-MANAGEMENT,
INFORMATION VISUALIZATION and DECISION-MAKING. This taxonomy is
summarized with themes and categories in Table 5.3 and is completely listed in Appendix
D with themes, categories and codes.
As far as we concerned, our model is multiuse. Besides its use as a
reference to handle visualizations, we can instantiate accepted models, processes, tools,
systems, theories in our model (metamodel). Section 6.2 provides an example of
instantiating the model using Feature Model (KANG et al., 1990) in the CONTEXT
theme; Theory of Behavioral Prediction (TBP) (FISHBEIN; AJZEN, 2011) in the
HUMAN-BEHAVIOR theme; Incident Command System (ICS) (FEMA, 2014) in
the PROCESS theme; an adaptation of Data Science process of SCHUTT; O’NEIL
(2013) in the DATA-MANAGEMENT theme (DAT ); an adaptation of the Information
Visualization process WARE (2012) in the INFORMATION VISUALIZATION theme
(INF ); and an adaptation of Decision-making process GLAUTIER; UNDERDOWN
(1991) in the DECISION-MAKING theme.
Our research investigates current tools that support decision-making through the
visualization of the emergency information in depth. Nevertheless, we also investigate
other themes more broadly, such as human behavior and decision-making theory that
influence the way we present the visualizations. We believe that extending the research
and the application of Visual Analytics techniques and tools in an effective way will have
a positive effect on the emergency domain. Consequently, another contribution of this
thesis is raising a research agenda in the EM area.
Dissemination of knowledge always involves contributions to the research area
and expands scientific knowledge updating academic literature. The dissemination of our
work is shown in the column ‘Publications’ of Figure 1.2. One of our first contributions
was a Systematic Mapping Study. We published this study in 2016, in Elsevier’s journal
Expert Systems with Applications (Qualis A1) (DUSSE et al., 2016a). The mapping of
this research area indicated several relevant findings. The main finding is a growing trend
of works applying VA in EM despite not using its full potential. This finding motivated
us to research VA in EM.
The results of the Systematic Mapping Study helped us to redefine the research
scope. We revisited Cycle I, we evolved our Research Question and we published other
works. We published a poster at the International Congress of Mass Disasters (CIDEM),
held in Salvador, BR (DUSSE; NOVAIS; MENDONÇA, 2016b) and we presented our
work at the Doctoral Symposium of the 14th International Conference on Information
Systems for Crisis Response and Management (ISCRAM), held in Albi, FR (DUSSE;
NOVAIS; MENDONÇA, 2017). ISCRAM is the main conference of this research area.
We used various scientific methods to collect our research data. These methods are
shown in rows II.B and II.C in figure 1.2 (official documents appraisal, ethnographic
studies, questionnaires, comparison of visualization tools, interviews). We had a core
paper accepted at the 15th ISCRAM, held in Rochester, NY, U.S. in May 2018 (DUSSE;
NOVAIS; MENDONÇA, 2018). The title of the paper is ‘Investigating the Use of
Visual Analytics to Support Decision-Making in Crisis Management: A Multi-Method
Approach’.
We carried out methods to analyze the research data. These methods are shown
in row III in figure 1.2 (Research Data Triangulation, Coding and Focus Group’). We
had another core paper accepted at the 16th ISCRAM, held in Valencia, ESP on May
2019 (DUSSE; NOVAIS; MENDONÇA, 2019). The title of the paper is ‘Understanding
the Main Themes Towards a Visual Analytics Based Model for Crisis Management
Decision-Making’.
With the results of the above papers, we developed our computational reference model.
Recently, we have had a third core paper accepted at the 17th ISCRAM, to be held in
Blacksburg, VA, U.S. on May 2021 (DUSSE; NOVAIS; MENDONÇA, 2020). The title of
the paper is ‘A Visual Analytics Based Model for Crisis Management Decision-Making’.
Spontaneously, without premeditation, we put together a trilogy with these 3 papers. We
searched for researchers who published three related works in sequential years in the same
venue and we did not find anyone. One publication would already be a contribution to the
research area; publications in the format of trilogy we believe to be a greater contribution.
The main scientific and technological impacts of this work are to disseminate the
state-of-the-art of VA research and technologies through our computational reference
model and to evaluate their effectiveness in real scenarios of EM activities. As we
have said, the results of this dissemination help spread a new research agenda and
the improvement of EM tasks. We believe that the decision makers will improve their
9.2 SCOPE LIMITATION 105
decisions with the support of our model.

There are also clear economic and social impacts for the research. We hope that
Command and Control Centers of governments and industries will incorporate the
computational reference model proposed in this thesis in their routine. If the model helps
decision-making to be faster and more assertive, the quality of the service provided to
society will improve, (e.g. governments will make better use of public resources).
We introduced the model in COI and the high-level secretariat authorized us to continue
our research at the Center.
The final and main contribution of our work is its potential to reduce financial
and, above all, human losses in emergencies.
9.2 SCOPE LIMITATION
One of the main challenges in this area is the multidisciplinary nature of research in
the role of EM and VA (VARDA et al., 2009). It combines management sciences,
social sciences, engineering sciences and computer sciences. While these areas are
complementary, they often address the same problems in different ways. This means that
effective approaches towards understanding and proposing solutions to the underlying
challenges must be multidisciplinary and broad. Our work would benefit from inputs from
all those areas, unfortunately we did not have the time or the workforce to investigate all
theme that could influence our work.
Another limitation of our work is the number of subjects involved in the evaluation
cycle of our thesis. Practitioners in C2 environments work under strict time constraints,
it is too expensive for managers to give up their time to outside activities. Using small
number of subjects is more susceptible to bias. There is no statistical rigor for inferences
that we made to accept our hypothesis with a high confidence level. The scope of the
evaluation was also limited, in fact, we delved deeper into the evaluation of the theme
INFORMATION VISUALIZATION and more superficially into the other themes of the
model.
The norms of a university postgraduate program also limit a thesis. This thesis is
part of a Computer Science program. For this reason, although we state that some
themes are important for visualizations in EM, our research does not delve into all
the themes, for example, human behavior and decision-making theory. Our thesis is
in Applied Computing and Visualization which are sub-areas of Computer Science.
The author of this thesis works in a Brazilian C2 Center and is interested in
continuing this work in areas not explored in the thesis because of time limits. We would
like to apply the model as a production tool, first in the C2 Center where the author
works and then in other partner Centers. The next Section discusses possible further
research that may spin off our work.
9.3 FUTURE WORK

Throughout the thesis, we have raised some limitations that were not fully explored
because of organizational issues, time constraints or were beyond our research scope.
We consider that overcoming these issues could improve our results. In this Section, we
revisit these and propose other future research.
• Application of the model with emergency response managers: Most of our

COI subjects are not experienced in EM activities in the field, as they are mostly
involved in simulation, training and ex-post analysis, i.e., in the preparedness
of an emergency. It would be helpful to tests the use of our model among
on-field emergency responders, where visualizations can be related to the (many)
scenarios known and shared inside the whole organization. Its application could
also be tested with managers working on the mitigation and recovery phases of
emergencies. Therefore, we consider a promising future work to apply the model in
other emergency phases, highlighting which components of the model will be most
stressed or will need updates.
• Application of the model at other C2 centers: Another work that we have

in mind is to evaluate our model with subjects from other organizations in other
states of Brazil and even from organizations in other countries. The aim is to
confirm whether the results of COI are similar to other organizations.
• Going deeper into VA techniques and tools: VA is a relatively new area and
we found that practitioners do not use VA tools. To develop VA tools in a PhD
work is costly in time and budget. A future work that needs to be done is to enact
our model within VA tools.
• Going deeper into themes of the model that we have explored

superficially: Despite their relevance, human behavior theory and decision-making
theory were under-explored in our thesis. There is an area of Computer Science
that we believe we could do further investigation to explore the human behavior
theory, that is Human Computer Interaction (HCI) (PREECE et al., 1994). As
well as, for decision-making theory, we could use the Analytic Hierarchy Process
(AHP) that is a structured technique for complex decisions, based on mathematics
and psychology (SAATY, 1980). Applying the state-of-the-art of HCI and AHP in
these themes will gather relevant results for EM decision-making. We believe that
our model can be the first step of a larger work incorporating Knowledge Sharing
with specialists from the various disciplines (themes) that we had studied and, in
the future, these steps can develop an Em Ecosystems.
• Evaluating other uses of the model: As we said in the previous Section,

our model is multiuse. A future work is to evaluate the instantiation of the
model (metamodel) presented in Section 6.2 with qualitative analysis methods.
Other relevant future work is to evaluate the instancing of our model with models,
processes, tools, systems, theories other than those instantiated in Section 6.2.
9.3 FUTURE WORK 107
• Application of the model in real EM scenarios (and other domains):

Finally, another future work is to test our model in real emergency case studies
to guide future research directions. We believe that using our model to simulate
what practitioners call a tabletop exercise is also a relevant future work. This
simulation can answer several what-if questions. We also plan to further validate
the model in other domains. Despite the fact that the model built here was based on
visualizations for decision-making in Emergency Management, it could be validated
and applied in other areas, such as Software Engineering, Education, Economics,
Health.
Besides the future work listed above, there are certainly other directions for further
research that can be performed based on the results of this thesis. They should improve
our model and facilitate its adoption in C2 Centers.
BIBLIOGRAPHY
AGGARWAL, C. C. An introduction to social network data analytics. Social network

data analytics, Springer, p. 1–15, 2011.
AJZEN, I. From intentions to actions: A theory of planned behavior. In: Action control.
[S.l.]: Springer, 1985. p. 11–39.
ANDRIENKO, G.; ANDRIENKO, N.; BARTLING, U. Visual analytics approach to
user-controlled evacuation scheduling. Information Visualization, SAGE Publications
Sage UK: London, England, v. 7, n. 1, p. 89–103, 2008.
ANDRIENKO, G.; ANDRIENKO, N.; JANKOWSKI, P.; KEIM, D.; KRAAK, M.-J.;
MACEACHREN, A.; WROBEL, S. Geovisual analytics for spatial decision support:
Setting the research agenda. International Journal of Geographical Information Science,
Taylor & Francis, v. 21, n. 8, p. 839–857, 2007.
BEUREN, I. M. Gerenciamento da informação: um recurso estratégico no processo de
gestão empresarial. [S.l.: s.n.], 1998.
BLOOMBERG. Here Are the Major Terror Attacks in Europe, From Paris to
Oslo. 2017. Online in: hhttps://www.bloomberg.com/news/articles/2017-06-19/
here-are-the-major-terror-attacks-in-europe-from-paris-to-osloi, Available in:
31/01/2018.
BOSCH, H.; THOM, D.; HEIMERL, F.; PÜTTMANN, E.; KOCH, S.; KRÜGER, R.;
WÖRNER, M.; ERTL, T. Scatterblogs2: Real-time monitoring of microblog messages
through user-guided filtering. IEEE Transactions on Visualization and Computer
Graphics, IEEE, v. 19, n. 12, p. 2022–2031, 2013.
BOYATZIS, R. E. Transforming qualitative information: Thematic analysis and code
development. [S.l.]: sage, 1998.
BUILDER, C. H.; BANKES, S. C.; NORDIN, R. Command concepts: A theory derived
from the practice of command and control. [S.l.], 1999.
CHAE, J.; THOM, D.; JANG, Y.; KIM, S.; ERTL, T.; EBERT, D. S. Public behavior
response analysis in disaster events utilizing visual analytics of microblog data. Computers
& Graphics, Elsevier, v. 38, p. 51–60, 2014.
CIOLKOWSKI, M.; LAITENBERGER, O.; VEGAS, S.; BIFFL, S. Practical experiences
in the design and conduct of surveys in empirical software engineering. Empirical methods
and studies in software engineering, Springer, p. 104–128, 2003.
109
110 BIBLIOGRAPHY
CIRAVEGNA, F.; GAO, J.; INGRAM, C.; IRESON, N.; LANFRANCHI, V.;
SIMANJUNTAK, H. Mapping mobility to support crisis management. In: ISCRAM.
[S.l.: s.n.], 2018. p. 305–316.
CLEMENTS, P.; NORTHROP, L. Software product lines. [S.l.]: Addison-Wesley,, 2002.
COOK, K. A.; THOMAS, J. J. Illuminating the path: The research and development
agenda for visual analytics. IEEE Computer Society, Los Alamitos, CA, United States
(US)., 2005.
CORREIO. Subway station receives safety simulation for the Olympics.

2016. Online in: hhttp://www2.correio24horas.com.br/detalhe/salvador/noticia/
estacao-de-metro-de-piraja-recebe-simulado-de-seguranca-para-as-olimpiadas/?cHash=
0f95bf4dadae144007969be0ab500fb3i, Available in: 31/01/2018.
CRANG, M.; COOK, I. Doing ethnographies. [S.l.]: Sage, 2007.
CYCLEBACK, D. R. Eye/brain physiology and human perception of external reality.

Cycleback. com, 2005.
DHA, U. Internationally agreed glossary of basic terms related to disaster management.

UN DHA (United Nations Department of Humanitarian Affairs), Geneva, 1992.
DUSSE, F.; JÚNIOR, P. S.; ALVES, A. T.; NOVAIS, R.; VIEIRA, V.; MENDONÇA,
M. Information visualization for emergency management: A systematic mapping study.
Expert Systems with Applications, Elsevier, v. 45, p. 424–437, 2016a.
DUSSE, F.; NOVAIS, R.; MENDONÇA, M. Visual Analytics in Emergency Management.

Salvador, Brazil: [s.n.], 2016b. Poster Section in International Congress of Mass Disasters,
II CIDEM.
DUSSE, F.; NOVAIS, R.; MENDONÇA, M. Visual Analytics in Crisis Management.

Albi, France: [s.n.], 2017. Doctoral Symposium in International Conference on
Information Systems for Crisis Response and Management, 14º ISCRAM.
DUSSE, F.; NOVAIS, R.; MENDONÇA, M. Investigating the use of visual analytics
to support decision-making in crisis management: A multi-method approach. In:
International Conference on Information Systems for Crisis Response and Management,
15º ISCRAM. Rochester, NY, U.S.: [s.n.], 2018.
DUSSE, F.; NOVAIS, R.; MENDONÇA, M. Understanding the main themes towards
a visual analytics based model for crisis management decision-making. In: International
Conference on Information Systems for Crisis Response and Management, 16º ISCRAM.
Valencia, ES: [s.n.], 2019.
DUSSE, F.; NOVAIS, R.; MENDONÇA, M. A visual analytics based model for crisis
management decision-making. In: International Conference on Information Systems for
Crisis Response and Management, 17º ISCRAM. Blacksburg, Virginia, U.S.: [s.n.], 2020.
BIBLIOGRAPHY 111
ENGELBRECHT, A.; BORGES, M. R.; VIVACQUA, A. S. Digital tabletops for

situational awareness in emergency situations. In: IEEE. Computer Supported Cooperative
Work in Design (CSCWD), 2011 15th International Conference on. [S.l.], 2011. p.
669–676.
ESCOBAR, H. Mud tsunami wreaks ecological havoc in brazil. Science, American
Association for the Advancement of Science, v. 350, n. 6265, p. 1138–1139, 2015.
ESRI. GIS for Comprehensive Emergency Management. 2009. Online in: hhttp://
esribulgaria.com/wp-content/uploads/2013/07/gis-for-emergency-management-II.pdfi,
Available in: 31/01/2018.
FEMA. Federal Emergency Management. 2014. Online in: hhttp://www.fema.gov/i,
Available in: 31/01/2018.
FETTERMAN, D. M. Ethnography: Step-by-step. [S.l.]: Sage, 2010.
FIRESCOPE. FIrefighting REsources of Southern California Organized for Potential
Emergencies. 2014. Online in: hhttps://www.firescope.org/i, Available in: 31/01/2018.
FISHBEIN, M.; AJZEN, I. Belief, attitude, intention and behavior: An introduction to
theory and research. [S.l.: s.n.], 1975.
FISHBEIN, M.; AJZEN, I. Predicting and changing behavior: The reasoned action
approach. [S.l.]: Taylor & Francis, 2011.
GALTON, A.; WORBOYS, M. An ontology of information for emergency management.
In: 8th International Conference on Information Systems for Crisis Response and
Management (ISCRAM 2011), Lisbon. [S.l.: s.n.], 2011.
GLAUTIER, M.; UNDERDOWN, B. Accounting. Theory and Practice, Pitman, London,
UK, 1991.
GLOBO. Errors that contributed to kiss tragedy. 2015. Online
in: hhttp://g1.globo.com/rs/rio-grande-do-sul/noticia/2015/01/
dois-anos-depois-veja-24-erros-que-contribuiram-para-tragedia-na-kiss.htmli, Available
in: 31/01/2018.
GLOBO. Olympic Games stage, Fonte Nova receives chemical attack
simulation. 2016. Online in: hhttp://g1.globo.com/bahia/noticia/2016/07/
simulacao-de-ataque-quimico-em-jogo-de-futebol-e-feita-em-salvador.htmli, Available
in: 31/01/2018.
GRASSELLI, G.; PESENTI, A.; CECCONI, M. Critical care utilization for the covid-19
outbreak in lombardy, italy: early experience and forecast during an emergency response.
Jama, American Medical Association, v. 323, n. 16, p. 1545–1546, 2020.
GRUBER, T. R. A translation approach to portable ontology specifications. Knowledge
acquisition, Elsevier, v. 5, n. 2, p. 199–220, 1993.
112 BIBLIOGRAPHY
GUARDIAN, T. Salvador, Brazil’s real party capital. 2010. Online in: hhttps://www.
theguardian.com/travel/2010/jun/19/salvador-brazil-partyi, Available in: 31/01/2018.
HASHEM, I. A. T.; YAQOOB, I.; ANUAR, N. B.; MOKHTAR, S.; GANI, A.; KHAN,
S. U. The rise of “big data” on cloud computing: Review and open research issues.
Information Systems, Elsevier, v. 47, p. 98–115, 2015.
HEINZELMAN, J.; WATERS, C. Crowdsourcing crisis information in disaster-affected

Haiti. [S.l.]: US Institute of Peace Washington, DC, 2010.
ISHIDA, T.; SAKURABA, A.; UCHIDA, N.; HASHIMOTO, K.; SHIBATA, Y. A

unified large scale disaster information presentation system using ultra gis based tiled
display environment. In: IEEE. Network-Based Information Systems (NBiS), 2012 15th
International Conference on. [S.l.], 2012. p. 550–555.
JUNIOR, P. S.; RAIMUNDO, P. O.; NOVAIS, R.; VIEIRA, V.; MENDONçA, M.

Supporting decision making during emergencies through information visualization of
crowdsourcing emergency data. In: Proceedings of the 19th International Conference on
Enterprise Information Systems. [S.l.: s.n.], 2017.
KANG, K. C.; COHEN, S. G.; HESS, J. A.; NOVAK, W. E.; PETERSON, A. S.

Feature-oriented domain analysis (FODA) feasibility study. [S.l.], 1990.
KEIM, D.; ANDRIENKO, G.; FEKETE, J.-D.; GORG, C.; KOHLHAMMER, J.;
MELANÇON, G. Visual analytics: Definition, process, and challenges. Lecture notes
in computer science, Springer, v. 4950, p. 154–176, 2008.
KEIM, D. A.; KRIEGEL, H.-P. Visualization techniques for mining large databases: A
comparison. IEEE Transactions on knowledge and data engineering, IEEE, v. 8, n. 6, p.
923–938, 1996.
KEIM, D. A.; MANSMANN, F.; SCHNEIDEWIND, J.; ZIEGLER, H. Challenges

in visual data analysis. In: IEEE. Information Visualization, 2006. IV 2006. Tenth
International Conference on. [S.l.], 2006. p. 9–16.
KELMAN, I.; GAILLARD, J.; MERCER, J. Climate change’s role in disaster risk
reduction’s future: Beyond vulnerability and resilience. International Journal of Disaster
Risk Science, Springer, v. 6, n. 1, p. 21–27, 2015.
KITCHENHAM, B.; CHARTERS, S. Guidelines for performing Systematic Literature

Reviews in Software Engineering. Technical Report EBSE 2007-001, Keele University
and Durham University Joint Report, 2007.
MACKENZIE, C. M.; LASKEY, K.; MCCABE, F.; BROWN, P. F.; METZ, R.;
HAMILTON, B. A. Reference model for service oriented architecture 1.0. OASIS
standard, v. 12, p. 18, 2006.
BIBLIOGRAPHY 113
MAZZA, R. Introduction to information visualization. [S.l.]: Springer Science & Business

Media, 2009.
MELNIK, R. Mathematical and computational modeling: with applications in natural and

social sciences, engineering, and the arts. [S.l.]: John Wiley & Sons, 2015.
MIDDLETON, S. E.; MIDDLETON, L.; MODAFFERI, S. Real-time crisis mapping of

natural disasters using social media. IEEE Intelligent Systems, IEEE, v. 29, n. 2, p. 9–17,
2014.
MUNICHRE. Munichre 2018. 2018. Online in: hhttps://www.munichre.

com/topics-online/en/climate-change-and-natural-disasters/natural-disasters/
the-natural-disasters-of-2018-in-figures.htmli, Available in: 31/01/2020.
NASCIMENTO, B. S. Seleção e Apresentação de Informações Contextuais em Ambientes

Complexos. Tese (PhD Thesis) — PPGI/UFRJ, Rio de Janeiro, RJ, 2016.
NATO. NATO Terminology. 2017. Online in: hhttps://nso.nato.int/nso/zzlinks/

terminology public non-classified%20nato%20glossaries.htmli, Available in:
31/01/2018.
NICOLA, A. D.; MELCHIORI, M.; VILLANI, M. L. Creative design of emergency

management scenarios driven by semantics: An application to smart cities. Information
Systems, Elsevier, v. 81, p. 21–48, 2019.
NORRIS, F. H.; STEVENS, S. P.; PFEFFERBAUM, B.; WYCHE, K. F.;

PFEFFERBAUM, R. L. Community resilience as a metaphor, theory, set of capacities,
and strategy for disaster readiness. American journal of community psychology, Wiley
Online Library, v. 41, n. 1-2, p. 127–150, 2008.
NYT. Hijacked Jets Destroy Twin Towers and Hit Pentagon. 2001. Online in: hhttp:
//www.nytimes.com/learning/general/onthisday/big/0911.html#articlei, Available in:
31/01/2018.
OCHOA, S. F.; NEYEM, A.; PINO, J. A.; BORGES, M. R. Supporting group decision
making and coordination in urban disasters relief. Journal of Decision Systems, Taylor
& Francis Group, v. 16, n. 2, p. 143–172, 2007.
OMG. Business process model and notation specification version 2.0. 2011. Online in:
hhttps://www.omg.org/spec/bpmn/2.0/i, Available in: 31/01/2018.
PETERSEN, K.; FELDT, R.; MUJTABA, S.; MATTSSON, M. Systematic mapping

studies in software engineering. In: EASE. [S.l.: s.n.], 2008. v. 8, p. 68–77.
PIERKOT, C.; CHRISTOPHE, S.; GIRRES, J.-F. Exploring multiplexing tools for
co-visualization in crisis units. In: . [S.l.: s.n.], 2019. p. 403–420.
114 BIBLIOGRAPHY
PORSANI, J. L.; JESUS, F. A. N. d.; STANGARI, M. C. Gpr survey on an iron

mining area after the collapse of the tailings dam i at the córrego do feijão mine in
brumadinho-mg, brazil. Remote Sensing, Multidisciplinary Digital Publishing Institute,
v. 11, n. 7, p. 860, 2019.
POSER, K.; DRANSCH, D. Volunteered geographic information for disaster management

with application to rapid flood damage estimation. Geomatica, v. 64, n. 1, p. 89–98, 2010.
PREECE, J.; ROGERS, Y.; SHARP, H.; BENYON, D.; HOLLAND, S.; CAREY, T.
Human-computer interaction. [S.l.]: Addison-Wesley Longman Ltd., 1994.
ROBSON, C. Real world research: A resource for users of social research methods in
applied settings 3rd edition. [S.l.]: West Sussex: John Wiley & Sons, 2011.
RUNESON, P.; HÖST, M. Guidelines for conducting and reporting case study research in
software engineering. Empirical software engineering, Springer, v. 14, n. 2, p. 131, 2009.
SAATY, T. L. The analytic hierarchy process (ahp). The Journal of the Operational
Research Society, v. 41, n. 11, p. 1073–1076, 1980.
SALDAÑA, J. The coding manual for qualitative researchers. [S.l.]: Sage, 2015.
SCHMEMANN, S. Hijacked Jets Destroy Twin Towers and Hit

Pentagon. 2001. Online in: hhttps://www.pilotopolicial.com.br/
carnaval-2015-30-anos-de-axe-e-190-anos-da-policia-militar-da-bahia/i, Available
in: 31/01/2018.
SCHUTT, R.; O’NEIL, C. Doing data science: Straight talk from the frontline. [S.l.]: ”
O’Reilly Media, Inc.”, 2013.
SEAMAN, C. B. Qualitative methods in empirical studies of software engineering. IEEE

Transactions on software engineering, IEEE, v. 25, n. 4, p. 557–572, 1999.
SEMS. Standardized Emergency Management System (SEMS). Introductory Course of

Instruction. 2002. Online in: hhttp://www.scc-ares-races.org/sems.htmi, Available in:
28/03/2018.
SESGE. Ministry of Justice of Brazil. Extraordinary Secretary of Security for Major

Events (SESGE). (2014) Strategic Security Planning for the FIFA World Cup Brazil
2014 (in Portuguese). 2014. Online in: hhttp://www.copa2014.gov.br/pt-br/noticia/
divulgado-o-plano-estrategico-de-seguranca-paracopa-do-mundo-de-2014i, Available in:
31/01/2018.
SHNEIDERMAN, B. The eyes have it: A task by data type taxonomy for information
visualizations. In: IEEE. Visual Languages, 1996. Proceedings., IEEE Symposium on.
[S.l.], 1996. p. 336–343.
BIBLIOGRAPHY 115
SIMAS, F.; BARROS, R.; SALVADOR, L.; WEBER, M.; AMORIM, S. A data exchange
tool based on ontology for emergency response systems. In: SPRINGER. Research
Conference on Metadata and Semantics Research. [S.l.], 2017. p. 74–79.
SOBREPEREZ, P. Using plenary focus groups in information systems research: more
than a collection of interviews. Electronic Journal of Business Research Methods,
Academic Conferences International, v. 6, n. 2, p. 181–188, 2008.
SUTCLIFFE, A.; GAULT, B.; FERNANDO, T.; TAN, K. Investigating interaction
in cave virtual environments. ACM Transactions on Computer-Human Interaction
(TOCHI), ACM, v. 13, n. 2, p. 235–267, 2006.
THOM, D.; KRÜGER, R.; ERTL, T.; BECHSTEDT, U.; PLATZ, A.; ZISGEN, J.;
VOLLAND, B. Can twitter really save your life? a case study of visual social media
analytics for situation awareness. In: IEEE. Visualization Symposium (PacificVis), 2015
IEEE Pacific. [S.l.], 2015. p. 183–190.
THOMAS, J. Visual analytics techniques that enable knowledge discovery: detect the
expected and discover the unexpected. Director, National Visualization and Analytics
Center, http://www. hiit. fi/vakd09/vakd09thomas-slides. pdf, 2009.
TOMASZEWSKI, B.; MACEACHREN, A. M. Geovisual analytics to support crisis
management: Information foraging for geo-historical context. Information Visualization,
Sage Publications Sage UK: London, England, v. 11, n. 4, p. 339–359, 2012.
TUROFF, M.; CHUMER, M.; WALLE, B. Van de; YAO, X. The design of a dynamic
emergency response management information system (dermis). JITTA: Journal of
Information Technology Theory and Application, Association for Information Systems,
v. 5, n. 4, p. 1, 2004.
UML. OMG - UML. 2018. Online in: hhttp://www.uml.org/i, Available in: 17/03/2018.
VARDA, D. M.; FORGETTE, R.; BANKS, D.; CONTRACTOR, N. Social network
methodology in the study of disasters: Issues and insights prompted by post-katrina
research. Population Research and Policy Review, Springer, v. 28, n. 1, p. 11–29, 2009.
VIEIRA, V.; TEDESCO, P.; SALGADO, A. C. Designing context-sensitive systems: An
integrated approach. Expert Systems with Applications, Elsevier, v. 38, n. 2, p. 1119–1138,
2011.
VILLELA, K.; BREINER, K.; NASS, C.; MENDONÇA, M.; VIEIRA, V. A smart and
reliable crowdsourcing solution for emergency and crisis management. Interdisciplinary
Information Management Talks, Podebrady, Czech Republic, p. 213–220, 2014.
WARE, C. Information visualization: perception for design. [S.l.]: Elsevier, 2012.
WAUGH, W. L.; STREIB, G. Collaboration and leadership for effective emergency
management. Public administration review, Wiley Online Library, v. 66, n. s1, p. 131–140,
2006.
116 BIBLIOGRAPHY
WOHLIN, C.; RUNESON, P.; HÖST, M.; OHLSSON, M. C.; REGNELL, B.; WESSLÉN,
A. Experimentation in software engineering. [S.l.]: Springer Science & Business Media,
2012.
WORDIQ. Doctrine - Definition. 2010. Online in: hhttps://web.archive.org/

web/20100510202232/http://www.wordiq.com/definition/Doctrinei, Available in:
17/03/2018.
WU, A.; ZHANG, X.; CONVERTINO, G.; CARROLL, J. M. Civil: support

geo-collaboration with information visualization. In: ACM. Proceedings of the ACM 2009
international conference on Supporting group work. [S.l.], 2009. p. 273–276.
YATES, D.; PAQUETTE, S. Emergency knowledge management and social media

technologies: A case study of the 2010 haitian earthquake. International journal of
information management, Elsevier, v. 31, n. 1, p. 6–13, 2011.
Appendix
A
This Appendix presents the online questionnaire (in Portuguese) that we ran during the 2016 Olympics
and Paralympics Games. We send it to hundreds of practitioners in Brazil and 49 officials from various
agencies from different C2 Centers participated. There were multiple choice and open questions.
ONLINE QUESTIONNAIRE (CYCLE 2)
117
QUESTIONÁRIO FORMULÁRIO
I) OPCIONAL
1º) Função no órgão?
2º) Ano de entrada?
2015-2016 2010-2014 2005-2009 2000-2004 antes de 2000
3º) Função no CICCR?
4º) Ano que trabalhou a 1ª vez no CICCR?
2013 2014 2015 2016 antes de 2013
5º) Quantas vezes você já trabalhou no CICCR?
1ª vez 2 vezes 3 - 5 vezes 6 - 10 vezes mais de 10
6º) Já teve experiência com outros Centros de Comando e Controle, gerenciando

emergências por meio de cooperação entre multiagências antes do CICCR?
II) OBRIGATÓRIO
1º) Na sua opinião, qual a maior dificuldade de desempenhar seus trabalhos no

CICCR?
2º) Na sua opinião, você considera um desafio entender as informações, suas

causas/efeitos para tomada de decisões? Caso sim, justifique; caso não, pule para a
última (7ª) questão.
3º) As ferramentas que você utiliza permite entender as informações, suas

causas/efeitos para tomada de decisões?
( ) Não sabe ( ) Nunca ( ) Pouco ( ) Muito ( ) Sempre
4º) Além dos sistemas de informação do CIICR (Risk Manager, SICOP) e das Mídias
Sociais (Facebook, WhatsApp), você utiliza mais algum? Proprietário ou gratuito? É
possível dizer quais?
5º) Sugestões de melhorais das ferramentas para entender as informações, suas

causas/efeitos para tomada de decisões:
6º) Com relação as informações?
Qualidade da informação
( ) Não sabe ( ) Péssima ( ) Ruim ( ) Boa ( ) Ótima
Quantidade de informação
( ) Não sabe ( ) Muita ( ) Ideal ( ) Pouca
Coleta dessas informações
Visualização dessas informações
Análise dessas informações
Difusão dessas informações
Com relação as tomadas de decisões
7º) Comentários:
Appendix
B
This Appendix presents the complete extraction of the facets in the 7 (seven) tools that he have accessed
for comparison.
EXTRACTION OF THE FACETS IN THE TOOLS

(CYCLE 2)
121
Appendix
C
This Appendix presents the Survey Design (in Portuguese and English), which includes the preparation
of the interview with the questions, the terms and other materials.
SURVEY DESIGN (CYCLE 2)
123
SURVEY DESIGN
In this Appendix, we detail the Survey Design, which includes the preparation of the interviews
with the questions, the terms and other materials.
SURVEY DEFINITION
 Premises
1. One of the researcher works at the Operations and Intelligence Center of
Bahia. He is aware of the challenges and experiences problems in this field on
a daily basis.
2. Systematic mapping was carried out, which provided input to conduct this
survey. Of the various results obtained from the mapping, two are highlighted:
a. A lack of standardized models to visualize information;
b. The use of visual analytics tools and techniques in some projects.
3. Two ethnographic studies were carried out. One during the FIFA World Cup
and the second during the Olympic Games. The aim of these studies was to
prepare the requirements for a crowdsourcing solution in the first case, and an
information visualization solution in the second. Some points were identified,
such as:
a. The lack of standardization in the action plans of agencies which work
in an integrated manner;
b. The lack of standardization of terms used by agencies;
c. Visual objects dealt with inconsistently in the agencies’ tools or
reports;
d. [...]
4. A questionnaire was prepared with 49 specialists in the field, who worked in

Command and Control Centers in the States of Bahia, Rio de Janeiro and the
Federal District during the 2016 Olympic and Paralympic Games in the city of
Rio de Janeiro. Two important results were attained from this questionnaire,
from the specialists’ responses:
a. An enormous amount of data is collected in crises and emergency
situations;
b. The lack of tools to deal with and understand this data.
● Objectives
GENERAL OBJECTIVE OF THE SURVEY

To confirm the premises defined in activities preceding this survey and prepare the
requirements to construct a model, in order to tackle the problems.
SPECIFIC OBJECTIVES OF THE SURVEY

- To understand the challenges and their causes from the specialists’ point of view.
- To investigate what solutions the specialists suggest, to mitigate these challenges.
- To evaluate the hypothesis that one of the main challenges is to carefully analyze the
information produced in an emergency, to subsidize quicker and more assertive
decision-making.
- To verify if the way that the data is dealt with; the way that the data is visualized and if
human factors influence decision-making in emergency situations.
- To implicitly propose visual analytics concepts to mitigate these challenges, evaluating
if the specialists agree.
● Review of literature
As mentioned in item 2 of the Premises, literature was reviewed using systematic
mapping, which mapped projects using information visualization concepts in
emergency management in any way. Of the work reviewed, no projects which carried
out a survey specifically in this area were found.
SURVEY DESIGN
1. Identify the target population and the survey sample

 X specialists were identified and agreed to be interviewed. The first interview
was used as a pilot. The date and location for the interviews were scheduled.
The following information was gathered on the specialist:
a. Name;
b. Original agency;
c. Length of service;
d. Current agency;
e. Length of service at the current institution;
f. Number of times that s/he has worked at Command Centers, and
g. Duties;
 And, lastly, the history of any links between the interviewer and the specialists
was analyzed.
 Representativeness [...]
 The snowballing technique was used. At the end of each interview, the
specialist was asked if s/he could suggest other specialists who could answer
the survey questions.
2. Conceptual model of the survey subjects and variables

 EM data collected;
 EM data analyzed;
 EM information produced;
 M information consumed;
 Data and information entry and exit;
 Decisions taken;
 Influencing human factors.
3. Data collection approach

○ Interview
4. Questions (Appendix A)
5. Data analysis approach

 Descriptive statistics and aggregation of data for the report.
 Compare divergent and similar responses; relate responses with the institution
and position; pinpoint associations, gaps and trends and analyze scoring consistencies.
 Coding.
 [...]
6. Issue of validity
 Internal validity;
 External validity;
 [...]
SURVEY IMPLEMENTATION
● Prepare material for the pilot and interviews.

● The material is based on audio equipment which recorded the interviews and pen and
paper for making notes.
● A pilot was held [...]
○ We used the first version of questions, which was then adapted accordingly
[...]
○ We restructured the process, with emphasis on survey packing [...]
SURVEY EXECUTION
7 INTERVIEW STEPS (Wanderley, 1998)
1. PREPARATION
 Concentrate on the previous phases: survey definition and survey design

 Complete the following form with each interviewee:
SPECIALIST:
Name: Agency Position

X Y Z
SPECIALIST X’S SPECIFIC OBJECTIVES

- Gather desirable objectives according to the expectations which you wish to obtain
from specialist X.
2. OPENING
● Thank the participant for giving their time

● Introduce yourself
● Ice-breaker:
○ Create a relaxed environment
○ Let the specialist talk about their daily activities, responsibilities and
pressure, etc...
○ At this time, allow the specialist to stray away from the objective of
the interview, talking about other everyday topics.
○ Avoid negative topics.
○ Avoid straying away from the objective for too long.
● Evaluate the specialist’s behavior. This evaluation will guide how the interview
is conducted.
3. INVESTIGATION
● Explain the aim of the project and interview

○ Advise the interviewee that you are studying for a PhD
○ Mention the project’s area of research (visual analytics)
○ Talk about what has been done; what is being done and what will be
carried out in the doctorate.
○ Advise of the impacts envisaged for this work (scientific, technological,
economic, social and environmental).
○ Mention the interview objectives.
○ Focus on the objectives of the interview from now on. Whenever the
specialist strays from the interview topic, thank them for the
information and immediately return to the initial objectives.
● Ask questions.
● Discover factors, reasons and the specialist’s needs.
● Look for common interests.
4. PRESENTATION
● Relate the interviewer’s expectations with those of the specialist, showing

solutions and benefits.
5. CLARIFICATION
● Clarify any questionable points with the interviewer.

● Summarize and share the information recorded with the specialist.
● From the summary, ask if any of the information was misunderstood or partly
understood.
● Ask if the specialist has anything to add.
6. FINAL ACTION
● Thank the specialist for their time.

● Produce a document with the information recorded by the interviewer and
send it to the specialist by email, asking for confirmation of receipt.
7. CONTROL / EVALUATION
● Evaluate if the objectives envisaged from the interview were achieved.

● Compare the original expectation with what was actually achieved at the
interview.
SURVEY ANALYSIS
SURVEY PACKAGING
QUESTIONNAIRE AND INTERVIEW
The questions below are asked in the INVESTIGATION phase. However, depending on how the
interview is conducted, some questions could be asked or repeated in other phases.
PRESENT THE PHASES OF AN EMERGENCY
1- Generic question: In your opinion, what is the greatest difficulty in performing

emergency management activities at CICCR?
2- Speaking of technology: develop

If not mentioned: Do you think that technology is important? What are the advantages
and disadvantages?
3- Speaking of information: develop

If not mentioned: And how about the information? Do you see a problem with the
quality and quantity of the information produced?
4- Speaking of communication failures: develop

If not mentioned: And how about failure to exchange or not exchanging this
information?
5- Speaking of data sources: develop

If not mentioned: And how about the various data sources? Operational strength,
population directly (190, Report, Occurrence), population indirectly (open sources and
social networks) and sensors?
When dealing with data, what data is required to effectively perform your CICCR
activities?
6- Speaking of tools: develop

If not mentioned: Besides CIICR (Risk Manager and SICOP) information systems and
social media (Facebook and Whats App), do you use any others? Can you state which
ones?
Do the tools which you use enable you to understand the information and its
causes/effects for decision-making?
Suggestion for tool improvements in order to understand the information and its
causes/effects for decision-making:
7- Speaking of Visualization: develop

If not mentioned: Do you think that visual tools would help with this problem?
Of the data mentioned in the previous question, what data should be visualized and
manipulated using tools?
Could you make suggestions of visualization and manipulation?
8- Speaking of a visual analytics concept: develop

If not mentioned: With human aspects in mind, do you think that visual tools together
with semi-automatic data analysis would facilitate decision-making?
PRESENT THE KNOWLEDGE PRODUCTION CYCLE

AVALIAÇÃO
DIFUSÃO PLANEJAMENTO
& &
INTEGRAÇÃO DIREÇÃO
MISSÃO
ANÁLISE BUSCA
& &
PRODUÇÃO COLETA
PROCESSAMENTO
&
EXPLORAÇÃO
& FEEDBACK
EVALUATION AND FEEDBACK
DISSEMINATION AND INTEGRATION
ANALYSIS AND PRODUCTION
PROCESSING AND INVESTIGATION
SEARCH AND COLLECT
PLANNING AND MANAGEMENT
MISSION
EVALUATION AND FEEDBACK
I will ask some multiple-choice questions. I would like you to justify your choice for each
answer.
Quality of information
( ) Don’t know ( ) Terrible ( ) Bad ( ) Good ( ) Excellent
Amount of information. (Two questions: to know if the user understands that working with a
lot of information is good or bad or understands that working with very little information is
good or bad).
( ) Don’t know ( ) A lot ( ) Ideal ( ) Very little
COLLECTION
Collecting this information
PROCESSING AND INVESTIGATION

Processing this information
Storing this information

Sharing this information

ANALYSIS AND PRODUCTION

Visualizing this information
Analyzing this information

DISSEMINATION AND INTEGRATION

Disseminating this information
PLANNING AND MANAGEMENT
With regards to cycle planning
In relation to decision-making
Do you believe that human factors interfere with visualizing and analyzing this
information? If yes, please justify.
Do you have any further comments to make?:
Could you suggest someone to be interviewed?

QUESTIONNAIRE
I) OPTIONAL
1) Position at the institution?
2) Year started?
2015-2016 2010-2014 2005-2009 2000-2004 before 2000
3) Position at CICCR?
4) Year that you worked at CICCR for the first time?
2013 2014 2015 2016 before 2013
5) How many times have you worked at CICCR?
Once Twice 3 - 5 times 6 - 10 times more than 10 times
6) Do you have experience with other Command and Control Centers, managing
emergencies through multiagency cooperation before CICCR?
II) MANDATORY
1) In your opinion, what is the greatest difficulty in doing your job at CICCR?
2
2) In your opinion, do you consider that understanding the information and its
causes/effects for decision-making is a challenge? If yes, please justify; if no, please
move on to the last (7) question.
3) Do the tools you use enable you to understand the information and its
causes/effect for decision-making?
( ) Don’t know ( ) Never ( ) A little ( ) A lot ( ) Always
4) Besides CIICR (Risk Manager and SICOP) information systems and social media
(Facebook and Whats App), do you use any others? Proprietary or free? Please state
which ones?
5) Suggestion for tool improvements in order to understand the information and its
causes/effects for decision-making:
6) With regards to the information?
Information quality
7) Comments:
TERMO DE CONSENTIMENTO
Nome:_______________________________________________________________________________
As informações contidas neste formulário visam firmar acordo por escrito, mediante o qual o
especialista de domínio consente sua participação na pesquisa, com pleno conhecimento da natureza dos
procedimentos a que se submeterá para ser participante do estudo e com capacidade de livre arbítrio e
sem qualquer coação. Esta participação é voluntária e o sujeito deste experimento tem a liberdade de
retirar seu consentimento a qualquer momento e deixar de participar do estudo, sem qualquer prejuízo ao
atendimento a que está sendo ou será submetido.
I - TÍTULO DO TRABALHO EXPERIMENTAL:
Coleta de Dados para desenvolver uma abordagem original que aplique visualização de
informação em Gestão de Emergência.
II – TEMA
Visualização de informação aplicada em Gestão de Emergência.
III – OBJETIVO DO ESTUDO
Coletar dados em que interfere na aplicação de visualização .de informação em Gestão de

Emergência.
IV – INSTITUIÇÃO RESPONSÁVEL
V – PESQUISADORES RESPONSÁVEIS
 Manoel Mendonça, UFBA
 Renato Lima Novais, IFBA
 Flávio Dusse, UFBA
VI – CONSENTIMENTO PÓS-INFORMAÇÃO
Eu,_________________________________________________________________, certifico que, tendo

lido as informações acima e suficientemente esclarecido de todos os itens, estou plenamente de acordo
com a realização da entrevista. Assim, eu autorizo a execução do trabalho de pesquisa exposto acima.
______________ , ______ de _____________________ de ______.
Assinatura do Participante: _________________________________________________

TERMO DE AUTORIZAÇÃO
Nome:_______________________________________________________________________________
As informações contidas neste formulário visam firmar acordo por escrito, mediante o qual o
especialista de domínio autoriza que os dados narrados durante a entrevista sejam publicados em
workshops, conferências, periódicos, relatórios técnicos e/ou teses de dissertação sem que seus dados
pessoais sejam expostos.
I - TÍTULO DO TRABALHO EXPERIMENTAL:
Coleta de Dados para desenvolver uma abordagem original que aplique visualização de
informação em Gestão de Emergência.
II – TEMA
Visualização de informação aplicada em Gestão de Emergência.
III – OBJETIVO DO ESTUDO
Coletar dados em que interfere na aplicação de visualização .de informação em Gestão de

Emergência.
IV – INSTITUIÇÃO RESPONSÁVEL
V – PESQUISADORES RESPONSÁVEIS
 Manoel Mendonça, UFBA
 Renato Lima Novais, IFBA
 Flávio Dusse, UFBA
VI – CONSENTIMENTO PÓS-INFORMAÇÃO
Eu,_________________________________________________________________, certifico que, tendo

lido as informações acima e suficientemente esclarecido de todos os itens, estou plenamente de acordo
com a publicação dos dados narrados sem que meus dados pessoais sejam expostos. Assim, eu
autorizo a execução do trabalho de pesquisa exposto acima.
______________ , ______ de _____________________ de ______.
Assinatura do Participante: _________________________________________________

Appendix
D
This Appendix presents the whole hierarchy of the coding classification: codes, categories and themes.
We believe that researchers and practitioners can use this classification as a taxonomy of relevant themes
needed in EM visualizations.
CODING CLASSIFICATION (CYCLE 3)
141
id_cod CODE id_cat CATEGORY THEME
A008 lack equipments
A012 lack software tools
B014 workload
B015 workload
B017 public expenditure
1 Lack of Resources
B065 obsolete technology
B067 obsolete technology
C018 lack budget
C020 integration
E124 workload
A001 multiagencies
A002 multiagencies 2 Roles of the Agencies
E120 crisis type
B094 ia
E138
B095
resistance to technology
topografy
C
4 Geography
E139 geografy O
B096 evolution of communication
5 Communication
E140 evolution of communication N
B116 false scenarios
C117 risk analysis 6 Risk T
E146
C114
risk analysis
resilience
E
7 Resilience
E144 resilience X
C100 3 center levels
C156 3 factors T
C102 flexible structure
C103 doctrine
C104 3 decision levels
C105 integration
C106 integration
C107 data flow
8 Processes
C108 data flow
C109 data flow
C110 3 cooperation levels
C111 profiteers
C112 citizenship
D113 data redundancy
E143 simulation
E131 storage
C019 documentation
C021 pdca
9 Lack of Process
E158 documentation
F173 tacit knowledge
A039 whole awareness
10 Unknown Process
D127 whole awareness
D118 doctrine C2 Management System
11
E119 doctrine (ICS)
B097 GUI
B098 useless complexity 12 Problems on Changes
E141 doctrine
B099 police to everything Deviation from agreed
13
D142 Deviation from agreed role position
C115 bureaucracy
14 Bureaucracy P
E145 international agreements
B165 politics
15 Politics
R
C166 politics
A167 data management O
P1 Data Management
E168
A071
data management
IA
C
B072 false scenarios E
B073 job redundancy
B074 evolution S
B075 screen clutter
C076 color scheme S
C077
C078
color scheme
reports
P2 Information Visualization E
C079 dashboard S
C080 dashboard
C081 smatphone
D152 attributes
E133 individualized
E153 temporal
A089 data consistency
B090 today less tacit
C091 meet managers quickly
C092 doctrine
P3 Decision-Making
C093 tabletop
D137 doctrine
D154 prompt answer
E155 transformation data into information
A029 context
A030 role x tools
A031 situational awareness
A032 ia
A064 storage
B033 emergency x planned operation
B034 radio
16 Information Needs
B035 street names
C036 situational awareness
E126 everything of everybody
A086 environment
B087 pdcs
B088 systemic vision
E136 planning
A028 information security
17 Information Security
E125 terrorism
A159 raw data collection
B160 raw data collection
D
18 Data Search
E161 raw data collection A
F178 search request
A049 raw data collection T
B050 data quality
B052 intelligence A
B053 multifunctional
B054 data flow starting point 19 Data Collection
B055 data quality
B056 raw data collection
C059 activity matrix M
E129 raw data collection
A009 lack software tools A
E122 workload 20 Data Amount
F175 bigdata
N
A010
E123
lack software tools
workload 21 Data Heterogeneity
A
F176 bigdata G
A060 processing
B061 processing E
B066 data mining
C062 processing
22 Data Processing M
C063
E130
doubts
processing
E
F174 transformation data into information
23
Transforming Data Into N
G177 transformation data into information Information
A068 integration
24 Information Sharing
T
E132 data across borders
A162 integration
25 Information Integration
C070 environment sharing
A083 diffusion
B084 diffusion
26 Information Diffusion
C085 data samples
E135 two-hand diffusion
See this same category in Theme: PROCESSES P2 Information Visualization

A082 manually
E134 analysis 27 Information Analysis
D157 analysis
A163 knowledge
28 Knowledge
B164 knowledge production
See this same category in Theme: PROCESSES P3 Decision-Making

A171 check
29 Check Process I
B172 PDCA
A169 visual paradigm N
30 Visual Paradigms
D170 visual paradigm F
See this same category in Theme: DATA MANAGEMENT 27 Information Analysis O
See this same category in Theme: DATA MANAGEMENT 28 Knowledge
V
I
See this same category in Theme: PROCESSES P3 Decision-Making S
See this same category in Theme: CONTEXT 3 Technology X Employees
B
A003 integration
A004 information flow E
H
B005 workload H
C006 militar defense x civil defense 31 Inexperience U
C007 adapt doctrine A
D147 follow doctrine
M
V
E121 terrorism A
B149 cognition I
D150 depth 32 Practitioner Competence N
E151 amplitude
O
See this same category in Theme: PROCESSES P3 Decision-Making R
See this same category in Theme: PROCESSES P2 Information Visualization D
A040 leadership type E M
A041 cognition
A042 waste C A
B043
B044
data value
cognition
I K
B045 repetition 33 Humam Behavior S I
B046 doctrine
B048 create doctrine
I N
D128 human unpredictability O G
E148 emotions
F011 perception N
-
Appendix
E
This Appendix presents the description of the datasets used in the evaluation of the model (in Portuguese).
The datasets consist of past data of emergency incidents from the state of Bahia (CICOC, CIDATA and
DD).
DATASETS (CYCLE 5)
147
CONJUNTO DE DADOS
I) CICOC:
Descrição: Centro Integrado de COmando e Controle, aplicação web criada para acompanhar
24/7 os principais incidentes principalmente Crimes Violentos Letais e Intencionais (CVLI),
mas também Crimes Violentos Contra o Patrimônio (CVP).
Tecnologia: VB.net e Sql Server.
Qtde de registros disponibilizados: 21.317.
Dados disponibilizados:
Campo Tipo Descrição

cicoc_codigo Int Código único, chave primária
cicoc_texto Text Texto do incidente
cicoc_usuario Varchar Usuário que cadastrou
cicoc_data-cadastro Date Data do cadastro
cicoc_plantao-anterior Boolean Se foi do Plantão Anterior
cicoc_ativo Boolean Se incidente está ativo ou inativo (excluído)
cicoc_fds Int Plantão do incidente
cicoc_tipo Varchar Tipo do incidente
II) CIDATA
Descrição: Centro de Inteligência e Tomada de Decisão Tática, aplicação web criada para
acompanhar nos horários não comercial (das 19h às 7h do dia seguinte e nos finais de
semana e feriados) os principais incidentes.
Tecnologia: VB.net e Sql Server.

cidata_codigo Int Código único, chave primária
cidata_texto Text Texto do incidente
cidata_usuario Varchar Usuário que cadastrou
cidata_data-cadastro Date Data do cadastro
cidata_aditado Boolean Se foi aditado
cidata_ativo Boolean Se incidente está ativo ou inativo (excluído)
cidata_numeracao-oficial Int Numeração do incidente
cidata_data-fato Date Data do incidente
III) DD
Descrição: Disque Denúncia, aplicação desktop criada para acompanhar 24/7 os incidentes
relatados pela população de forma anônima.
Tecnologia: FoxPro e Sql Server.

dd_codigo Int Código único, chave primária
dd_texto Text Texto do incidente
dd_usuario Varchar Usuário que cadastrou
dd_data-cadastro Date Data do cadastro
dd_versao Int Aditamentos do incidente
dd_numeracao-oficial Int Numeração do incidente
dd_tipo Varchar Tipo do incidente
Appendix
F
This Appendix presents the evaluation form of the exploratory study (in Portuguese). The first part of
the form is to characterize the evaluator with his experience and skills. The second part has 8 closed
questions, in which 3 questions are for all subjects and 2 specific questions for designers and 3 specific
questions for evaluators. Each question has two division: one close-ended subquestion for the subject to
answer yes, no or don’t know and one open-ended subquestion for the subject to justify the answer.
EVALUATION FORM (CYCLE 5)
151
FORMULÁRIO DE AVALIAÇÃO
I) TODOS - QUALIFICAÇÃO
a) Função de formação?
b) Ano de entrada?
c) Função no COI?
d) Ano de entrada no COI?
e.1) For designers: Você tem experiência em desenvolver ferramentais visuais para
análise de dados? Quanto tempo?
e.2) For evaluators: Você usa ferramentais visuais para análise de dados? Quanto
tempo?
f) Já ouviu falar em Visual Analytics?
___________________________________________________________________________
II) QUESTÕES DE AVALIAÇÃO
EQ1: Ind1.1) Tornar rápida e assertiva a tomada de decisões na gestão de

emergência através visualização da informação é relevante?
( ) Não sabe ( ) Sim ( ) Não
Justifique:
EQ3: Ind3.7) O modelo proposto é fácil de se usar?

Justifique:
EQ3: Ind3.8) Aplicar o modelo proposto hoje causará mínimo impacto nas
atividades atuais?
Justifique:
Comentários sobre o estudo; críticas, elogios, sugestões:
___________________________________________________________________________
 SOMENTE PARA DESIGNERS
EQ2: Ind2.5) O modelo proposto é útil para desenvolvedores projetar

visualizações de alta qualidade para tomada de decisão em Gestão de Emergência?
Justifique:
EQ2: Ind2.6) Os conceitos de Visual Analytics embutidos no modelo podem ser

úteis a curto prazo?
Justifique:
_____________________________________________________________
 SOMENTE PARA AVALIADORES
EQ2: Ind2.1 a Ind2.4) A partir da apresentação quantas estrelas você pontua para
os critérios:
Critério Descrição
Completude Quão próximo de uma versão totalmente funcional e conceitualmente completa
está o protótipo de visualização.
Criatividade Como o protótipo de visualização traz novidade e/ou inovação e agrega novos
conceitos ao domínio do problema considerado.
Usabilidade Quão fácil é para o usuário final interagir com o protótipo de visualização, bem
como entender as informações fornecidas.
Viabilidade Como o protótipo de visualização já tem acesso a todos os recursos necessários
ao seu funcionamento e quão sustentável é sua evolução para upgrades.
Pontuação Descrição
0 estrela Ruim, uma criança poderia fazer o mesmo ou até melhor.
1 estrela Pobre, não atendeu minhas expectativas, precisa de várias melhorias.
2 estrelas Regular, não atendeu minhas expectativas, mas tem potencial.
3 estrelas Bom, atendeu às minhas expectativas, precisa de poucas melhorias.
4 estrelas Muito bom, atendeu às minhas expectativas, praticamente pronto para ser
usado em casos reais.
5 estrelas Excelente, superou minhas expectativas, pronto para ser usado em casos reais.
EQ3: Ind3.1 a Ind3.4) A partir da apresentação e com base nas melhorias sugeridas
para uma versão baseada no modelo, quantas estrelas você pontua para os mesmos
critérios acima:
EQ3: Ind3.5) O modelo proposto é efetivo para gerentes de emergências tomarem

decisões a partir de visualizações de alta qualidade projetadas com base no
modelo?
Justifique:
EQ3: Ind3.6) O modelo proposto pode aprimorar doutrinas de Comando e

controle a longo prazo?
Justifique:

Tese Doutorado Dusse-FINAL

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Tese Doutorado Dusse-FINAL

Enviado por

Direitos autorais:

Formatos disponíveis

Universidade Federal da Bahia

Instituto de Matemática e Estatı́stica

Programa de Pós-Graduação em Ciência da Computação

A COMPUTATIONAL REFERENCE MODEL

A COMPUTATIONAL REFERENCE MODEL TO SUPPORT

Esta Tese de Doutorado foi

Orientador: Prof. Dr. Manoel Gomes de Mendonça Neto

Orientador: Prof. Dr. Prof. Dr. Manoel Gomes de Mendonça Neto.

1. Computational Model. 2. Visual Analytics. 3. Information

Contexto: Gestão de Emergência (GE) refere-se à capacidade de lidar com tarefas

Palavras-chave: Modelo Computacional, Visual Analytics, Visualização de Informa-

Background: Emergency Management (EM) refers to the activity of dealing with

Keywords: Computational Model, Visual Analytics, Information Visualization,

Chapter 2—THEORETICAL BASIS and RELATED WORK 13

Chapter 3—CYCLE I: DEFINITION OF THE RESEARCH SCOPE 23

Chapter 4—CYCLE II: RESEARCH DATA COLLECTION 29

4.3.1 Comparison of Visualization Tools . . . . . . . . . . . . . . . . . . 39

Chapter 5—CYCLE III: RESEARCH DATA ANALYSIS 47

5.1 TRIANGULATING FINDINGS OF THE COLLECTED METHODS . . 47

Chapter 6—CYCLE IV: MODEL DEVELOPMENT 57

6.1 COMPUTATIONAL REFERENCE MODEL BASED ON VISUAL

Chapter 7—CYCLE V: EVALUATION OF THE MODEL 77

7.1 EXPLORATORY STUDY DESCRIPTION . . . . . . . . . . . . . . . . 78

Chapter 9—FINAL CONSIDERATIONS 101

Appendix A—ONLINE QUESTIONNAIRE (CYCLE 2) 117

Appendix B—EXTRACTION OF THE FACETS IN THE TOOLS (CYCLE 2) 121

Appendix C—SURVEY DESIGN (CYCLE 2) 123

Appendix D—CODING CLASSIFICATION (CYCLE 3) 141

Appendix E—DATASETS (CYCLE 5) 147

Appendix F—EVALUATION FORM (CYCLE 5) 151

1.1 Example of Visualization X Emergency Management Context . . . . . . 7

2.1 Emergency Management Phases (adapted from POSER; DRANSCH (2010)) 15

3.1 Integration scheme of the Command and Control Centers in Brazil . . . . 24

4.1 Primary Studies Selection adapted from DUSSE et al. (2016a) . . . . . . 31

5.1 Coding Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

6.1 The model without external components . . . . . . . . . . . . . . . . . . 58

6.2 Visual Analytics Based Model for Decision-Making in Emergency

7.1 Evaluation Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

1.1 Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4.1 Research Question at the Time of the Mapping Study . . . . . . . . . . . 31

5.1 Knowledge and Findings of Research Data Collection . . . . . . . . . . . 48

9.1 Practitioners Meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

C2 Command and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

CICCN-DF National Command and Control Integrating Center of Brazil [Centro

COFIC Industrial Promotion Committee of the City of Camaçari [Comitê de Fomento

COI Intelligence and Operation Center [Centro de Operações e Inteligência, in

DZPPP Defense Zone of the Paris Police Prefecture . . . . . . . . . . . . . . . . 26

FEMA Federal Emergency Management Agency . . . . . . . . . . . . . . . . . . 1

FPC-UFBA Fraunhofer Project Center for Software and Systems Engineering at

GMT Greenwich Mean Time Zone . . . . . . . . . . . . . . . . . . . . . . . . . . 41

GPS Global Positioning System . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

ICS Incident Command System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

ISCRAM International Conference on Information Systems for Crisis Response and

MEUV Military Emergency Unit in Valencia, Spain . . . . . . . . . . . . . . . . 26

SESGE Extraordinary Secretary of Security for Large Events . . . . . . . . . . . 23

SSP-BA Public Security Secretariat of the State of Bahia [Secretaria de Segurança

UML Unified Modeling Language . . . . . . . . . . . . . . . . . . . . . . . . . . 73

UTM Universal Transverse Mercator . . . . . . . . . . . . . . . . . . . . . . . . . 41

Emergency Response Plan is an oral or written plan containing general

Attitude is a disposition or latent tendency to respond in a favorable or unfavorable

Belief is a state or habit of mind in which trust or confidence is placed in someone,

Command is the exercise of authority based upon certain knowledge to attain an

Crisis is an event that is expected to lead to a dangerous situation, whether it’s